[Pvfs2-developers] [PATCH] modify twod-stripe's default params and add some documentation

[Phil Carns]

There are a couple of options for dealing with the case where num_groups 
 > num_dfiles if we want to keep the current defaults:

1) Transparently reduce the num_groups within the distribution 
calculation.  From the user's perspective the params wouldn't change; 
the distribution would just automatically fall back to acting like 
simple stripe.  We would just need to add this to the top of a few of 
the dist functions:

if(num_groups > server_ct)
{
    num_groups = server_ct;
}

2) Return an explicit error at create time.  The varstrip distribution 
has this behavior now; it actually prevents you from creating a file if 
the dfile count isn't plausible for the distribution parameters.  The 
way to do this is to implement the get_num_dfiles() function in 
twod_stripe so that it can perform a safety check and return an error at 
create time if the user tries to use too high of a num_groups setting. 
On the command line you would get a generic errno (EINVAL or whatever) 
but we could put a descriptive message in the logs.

Option 1) seems reasonable to me; it is a small code change, we get to 
keep the existing desirable default parameters, and the single server 
case just transparently falls back to simple stripe.

-Phil

Rob Ross wrote:
> I'm glad we agree that we should make the distribution work with default 
> parameters, and I agree there is more than one way to accomplish that.
> 
> If you can find a way to have it default to something more sensible, 
> great. If not, I'm happy to have it behave like simple stripe if the 
> user doesn't do any tuning.
> 
> -- Rob
> 
> On Jul 4, 2008, at 2:21 PM, Sam Lang <slang at mcs.anl.gov> wrote:
> 
>>
>> Having it work with default parameters doesn't necessarily mean making 
>> those default parameters exactly like simple stripe.  Our best all 
>> around solution is to fix this bug with a simple check in the 2d code 
>> and set the default parameters to something reasonable for a 2d 
>> distribution in the general case.  We looked at this last year and 
>> decided that num_groups=2 and striping factor = 256 were reasonable, 
>> maybe they're not, but it doesn't make sense to make them identical to 
>> the simple stripe dist.
>> -sam
>>
>> On Jul 1, 2008, at 8:08 PM, Rob Ross wrote:
>>
>>> I agree that it's best to have it work for everyone with default 
>>> parameters.
>>>
>>> -- Rob
>>>
>>> On Jul 1, 2008, at 4:15 PM, "Kyle Schochenmaier" <kschoche at gmail.com> 
>>> wrote:
>>>
>>>> We should use simple stripe in that case, which is why i have it
>>>> defaulted to it now, someone was playing with twod-dist today and
>>>> noticed that it defaults to having 2 groups, which causes errors when
>>>> you run it on only a single server.   I would much rather have the
>>>> defaults allow the distribution to work on *any* setup, rather than
>>>> just a n>1 servers setup.
>>>>
>>>> Its not particularly useful, but I belive it is the right thing to do,
>>>> maybe I'm wrong?
>>>> (the original default settings are worthless for my applications so I
>>>> have to change them anyhow)
>>>>
>>>> Kyle
>>>>
>>>> On Tue, Jul 1, 2008 at 4:04 PM, Sam Lang <slang at mcs.anl.gov> wrote:
>>>>>
>>>>> Kyle,
>>>>>
>>>>> How is this patch useful for you guys?  With a single server (and 
>>>>> only one
>>>>> stripe), it doesn't matter what the factor and num_groups are, does 
>>>>> it?  It
>>>>> still behaves the same as simple-stripe in that case.  With 
>>>>> multiple servers
>>>>> setting the factor and num_groups to 1 does make the 2d 
>>>>> distribution act
>>>>> like simple-stripe, but why not just use simple stripe in that case?
>>>>>
>>>>> -sam
>>>>>
>>>>>
>>>>> On Jul 1, 2008, at 3:19 PM, Kyle Schochenmaier wrote:
>>>>>
>>>>>> Patch is against cvs head, and modifies the default num_groups to 
>>>>>> be 1
>>>>>> and the group_strip_factor = 1  so twod-stripe's defaults still work
>>>>>> when you use it on a single-server, it will now by default behave the
>>>>>> same as simple-stripe until the user modifies the num_groups and
>>>>>> strip_factor xattrs.
>>>>>>
>>>>>> Also added a brief description of the definitions for the dist_params
>>>>>> in twod-stripe.
>>>>>>
>>>>>> Tested fully functional on my servers.
>>>>>>
>>>>>> Kyle
>>>>>>
>>>>>>
>>>>>> Index: src/io/description/dist-twod-stripe.c
>>>>>> ===================================================================
>>>>>> RCS file: /anoncvs/pvfs2/src/io/description/dist-twod-stripe.c,v
>>>>>> retrieving revision 1.5
>>>>>> diff -r1.5 dist-twod-stripe.c
>>>>>> 6a7,15
>>>>>>>
>>>>>>> /* twod-stripe will take all of the servers in the filesystem and
>>>>>>> * partition them into num_groups groups.  Data will then be 
>>>>>>> striped to
>>>>>>> * each group before we move onto the next group.  The 
>>>>>>> strip_factor will
>>>>>>> * determine how many chunks of strip_size are written to each server
>>>>>>> * in each group before we transition to the next group.
>>>>>>> * The striping on the group level is done round-robin in the same
>>>>>>> * fashion as simple-stripe
>>>>>>> */
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Index: include/pvfs2-dist-twod-stripe.h
>>>>>> ===================================================================
>>>>>> RCS file: /anoncvs/pvfs2/include/pvfs2-dist-twod-stripe.h,v
>>>>>> retrieving revision 1.4
>>>>>> diff -r1.4 pvfs2-dist-twod-stripe.h
>>>>>> 17c17
>>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 2
>>>>>> ---
>>>>>>>
>>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 1
>>>>>>
>>>>>> 19c19
>>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 256
>>>>>> ---
>>>>>>>
>>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 1
>>>>>>
>>>>>>
>>>>>> -
>>>>>> Kyle Schochenmaier
>>>>>> <twod-dist.patch>_______________________________________________
>>>>>> Pvfs2-developers mailing list
>>>>>> Pvfs2-developers at beowulf-underground.org
>>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> Kyle Schochenmaier
>>>> _______________________________________________
>>>> Pvfs2-developers mailing list
>>>> Pvfs2-developers at beowulf-underground.org
>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers