[Pvfs2-users] Problems with basic distribution (with xattrs)

[Phil Carns]

Hi Tony,

There are actually a little more text on this in trunk now.  I recently 
went through a similar learning experience about some of the 
distributions :)

Kyle added some comments here:

http://www.pvfs.org/fisheye/browse/PVFS/src/io/description/dist-twod-stripe.c?r=1.6#l7

And I filled in some draft distribution information here, which talks a 
little about incast:

http://www.pvfs.org/fisheye/browse/PVFS/doc/pvfs2-tuning.tex?r=1.2#l267

As for the 2 groups problem: that is actually not really a valid setting 
for a single server case.  For the 2d dist to be beneficial the 
num_groups needs to be no larger than the number of servers, and ideally 
evenly divisible into the number of servers.  I don't know if you are on 
the pvfs2-developers list, but there is an ongoing discussion thread 
about the issue you brought up of the default 2d param values not 
working in the single server case:

http://www.beowulf-underground.org/pipermail/pvfs2-developers/2008-July/004092.html

The short story is just use more servers or else reduce the value of 
num_groups for the time being if you want to experiment with it.

-Phil


Tony Kew wrote:
> Dear Phil,
> 
> Its looking good now on a single node test - I will expand to a 16 node 
> test next...
> The twod_stripe distribution is still seeing the errors about 2 groups I 
> sent in my
> last email, but the file is created anyhow.
> 
> In that regard, does someone have a good explanation of the 2D stripe & how
> it works, i.e. what the "num_groups"  and "group_strip_factor" do to 
> change the
> stripe behavior?  The only comment I could find was that the 
> distribution is
> designed to combat incast, but I can't find a reference to what "incast" is
> either!
> 
> Many, Many thanks,
> Tony