[Pvfs2-users] Problems with basic distribution (with xattrs)

[Tony Kew]

Dear Phil,

On a fist pass it looks like the patch fixes the basic_dist, but breaks
Murali's  varstrip_dist xattrs patch.  I'll try some more tests just to
make sure I didn't fat finger something ...

I'm also beginning to suspect that there is problem with xattrs
settings for twod_stripe.  Testing on a single node (which I must
say makes no sense for a 2D stipe) & setting the strip to use one
group (since there is only one node) the server logfile shows errors
regarding using 2 groups with one server, so the xattr setting might
not be getting honored (or you cannot use this distribution without
at least two servers)

e.g.

    $ mkdir twod_stripe
    $ setfattr -n "user.pvfs2.dist_name" -v "twod_stripe" twod_stripe
    $ setfattr -n "user.pvfs2.dist_params" \
    >  -v "strip_size:2048;num_groups:1;group_strip_factor:512" twod_stripe
    $ cd twod_stripe/
    $ dd if=/dev/urandom of=testfile bs=16k count=16
    16+0 records in
    16+0 records out
    262144 bytes (262 kB) copied, 0.162147 seconds, 1.6 MB/s
    $ ls -l
    total 260
    -rw-r--r-- 1 tonykew ccrstaff 262144 Jun 30 15:52 testfile
    $

The fie is created (and is the correct size), but the server logfile
shows many errors:

    [E 06/30 15:52] physical_to_logical_offset: Invalid 
num_groups/server_ct options: gr:2 server:1
    [E 06/30 15:52] next_mapped_offset: Invalid num_groups/server_ct 
options: gr:2 server:1
    [E 06/30 15:52] logical_to_physical_offset: Invalid 
num_groups/server_ct options: gr:2 server:1
    [E 06/30 15:52] next_mapped_offset: Invalid num_groups/server_ct 
options: gr:2 server:1
    [E 06/30 15:52] logical_to_physical_offset: Invalid 
num_groups/server_ct options: gr:2 server:1
    etc...

If I read this correctly, the number of groups "gr:2" is still 2 even 
though I set it
to 1 with the fsetattr command



Many Thanks for your help,
Tony

Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
 in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203

CoE Office: (716) 881-8930           Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
      Cell: (716) 560-0910          Home: (716) 874-2126

"I love deadlines, I love the whooshing noise they make as they go by."
                                                          Douglas Adams



Phil Carns wrote:
> Hi Tony,
>
> Thanks for the bug report.  I have not looked at the varstrip issue 
> yet, but the attached patch (also available in cvs head) should at 
> least fix the basic_dist problem.
>
> The problem was that basic_dist only works for files with 1 datafile, 
> but the logic that lets it override the number of datafiles was 
> happening before the directory hints were processed.
>
> Someone correct me if I'm wrong, but I reorganized the code a little 
> chose the number of datafiles in this priority (after making sure the 
> final distribution for the file is set first, via hints or otherwise):
>
> 1) count specified in attr's passed into sys_create
>    - example: -n option to pvfs2-cp
> 2) directory hints
>    - example: "user.pvfs2.num_dfiles" extended attribute
> 3) mount options
>    - example: num_dfiles option in /etc/pvfs2tab
> 4) system default
>    - example: DefaultNumDfiles in server conf, or number of servers
>
> The distribution itself gets a chance to override the value at the end 
> if (as in the case of basic_dist) it doesn't support an arbitrary 
> number of datafiles.
>
> I'll look into the varstrip issue soon.
>
> -Phil
>
> Tony Kew wrote:
>> Dear PVFS users list,
>>
>> Mounting a pvfs 7.2.1, creating directories and setting the distribution
>> type with setfattr for each directory, only simple_stipe and twod_stripe
>> would allow me to write a file in the given directory.
>>
>> Adding the varstrip_dist xattrs patch from Murali Vilayannur attached to
>> this message thread:
>>
>>    http://osdir.com/ml/file-systems.pvfs2.users/2007-01/msg00050.html
>>
>> fixes the variable strip distibution, but the basic distribution still
>> doesn't work for me.
>>
>> The patch adds
>>
>>    #include "pvfs2-dist-basic.h
>>
>> to src/client/sysint/sys-create.sm so I wonder if no one has tried 
>> this...???
>> Perhaps there is a little more code needed to make basic_dist work with
>> xargs?
>>
>> The patch above is against 2.6, but works (and is needed for 
>> varstrip_dist
>> with xargs) with 7.2.1  & the latest cvs (best I can tell) - perhaps 
>> this patch
>> could be added to both?
>>
>> Let me know if there is useful debug info I can provide...
>>
>>
>> Thanks Much
>> Tony
>>
>> Tony Kew
>> SAN Administrator
>> The Center for Computational Research
>> New York State Center of Excellence
>> in Bioinformatics & Life Sciences
>> 701 Ellicott Street, Buffalo, NY 14203
>>
>> CoE Office: (716) 881-8930           Fax: (716) 849-6656
>> CSE Office: (716) 645-3797 x2174
>>      Cell: (716) 560-0910          Home: (716) 874-2126
>>
>> "I love deadlines, I love the whooshing noise they make as they go by."
>>                                                          Douglas Adams
>>
>> _______________________________________________
>> Pvfs2-users mailing list
>> Pvfs2-users at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users