[Pvfs2-users] Problems with basic distribution (with xattrs)

[Tony Kew]

Dear Phil,

Thanks for the info, I am on the dev list, so I've seen the twod_dist 
debate.

They seem to have gone for the "fix on the fly" approach, where I would have
preferred the setfattr to fail with a useful error message, or have the 
server
reset the client settings to ones "in bounds" so you could tell from the 
client
that the settings are getting overridden.  I would also like to avoid 
filling the
server with error messages - as happens at the moment if there is a problem
with num_groups > number_of_servers


Thanks Again,

Tony

Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
 in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203

CoE Office: (716) 881-8930           Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
      Cell: (716) 560-0910          Home: (716) 874-2126

"I love deadlines, I love the whooshing noise they make as they go by."
                                                          Douglas Adams



Phil Carns wrote:
> Hi Tony,
>
> There are actually a little more text on this in trunk now.  I 
> recently went through a similar learning experience about some of the 
> distributions :)
>
> Kyle added some comments here:
>
> http://www.pvfs.org/fisheye/browse/PVFS/src/io/description/dist-twod-stripe.c?r=1.6#l7 
>
>
> And I filled in some draft distribution information here, which talks 
> a little about incast:
>
> http://www.pvfs.org/fisheye/browse/PVFS/doc/pvfs2-tuning.tex?r=1.2#l267
>
> As for the 2 groups problem: that is actually not really a valid 
> setting for a single server case.  For the 2d dist to be beneficial 
> the num_groups needs to be no larger than the number of servers, and 
> ideally evenly divisible into the number of servers.  I don't know if 
> you are on the pvfs2-developers list, but there is an ongoing 
> discussion thread about the issue you brought up of the default 2d 
> param values not working in the single server case:
>
> http://www.beowulf-underground.org/pipermail/pvfs2-developers/2008-July/004092.html 
>
>
> The short story is just use more servers or else reduce the value of 
> num_groups for the time being if you want to experiment with it.
>
> -Phil
>
>
> Tony Kew wrote:
>> Dear Phil,
>>
>> Its looking good now on a single node test - I will expand to a 16 
>> node test next...
>> The twod_stripe distribution is still seeing the errors about 2 
>> groups I sent in my
>> last email, but the file is created anyhow.
>>
>> In that regard, does someone have a good explanation of the 2D stripe 
>> & how
>> it works, i.e. what the "num_groups"  and "group_strip_factor" do to 
>> change the
>> stripe behavior?  The only comment I could find was that the 
>> distribution is
>> designed to combat incast, but I can't find a reference to what 
>> "incast" is
>> either!
>>
>> Many, Many thanks,
>> Tony