[Pvfs2-developers] [PATCH] modify twod-stripe's default params and add some documentation

[Rob Ross]

I'm glad we agree that we should make the distribution work with  
default parameters, and I agree there is more than one way to  
accomplish that.

If you can find a way to have it default to something more sensible,  
great. If not, I'm happy to have it behave like simple stripe if the  
user doesn't do any tuning.

-- Rob

On Jul 4, 2008, at 2:21 PM, Sam Lang <slang at mcs.anl.gov> wrote:

>
> Having it work with default parameters doesn't necessarily mean  
> making those default parameters exactly like simple stripe.  Our  
> best all around solution is to fix this bug with a simple check in  
> the 2d code and set the default parameters to something reasonable  
> for a 2d distribution in the general case.  We looked at this last  
> year and decided that num_groups=2 and striping factor = 256 were  
> reasonable, maybe they're not, but it doesn't make sense to make  
> them identical to the simple stripe dist.
> -sam
>
> On Jul 1, 2008, at 8:08 PM, Rob Ross wrote:
>
>> I agree that it's best to have it work for everyone with default  
>> parameters.
>>
>> -- Rob
>>
>> On Jul 1, 2008, at 4:15 PM, "Kyle Schochenmaier"  
>> <kschoche at gmail.com> wrote:
>>
>>> We should use simple stripe in that case, which is why i have it
>>> defaulted to it now, someone was playing with twod-dist today and
>>> noticed that it defaults to having 2 groups, which causes errors  
>>> when
>>> you run it on only a single server.   I would much rather have the
>>> defaults allow the distribution to work on *any* setup, rather than
>>> just a n>1 servers setup.
>>>
>>> Its not particularly useful, but I belive it is the right thing to  
>>> do,
>>> maybe I'm wrong?
>>> (the original default settings are worthless for my applications  
>>> so I
>>> have to change them anyhow)
>>>
>>> Kyle
>>>
>>> On Tue, Jul 1, 2008 at 4:04 PM, Sam Lang <slang at mcs.anl.gov> wrote:
>>>>
>>>> Kyle,
>>>>
>>>> How is this patch useful for you guys?  With a single server (and  
>>>> only one
>>>> stripe), it doesn't matter what the factor and num_groups are,  
>>>> does it?  It
>>>> still behaves the same as simple-stripe in that case.  With  
>>>> multiple servers
>>>> setting the factor and num_groups to 1 does make the 2d  
>>>> distribution act
>>>> like simple-stripe, but why not just use simple stripe in that  
>>>> case?
>>>>
>>>> -sam
>>>>
>>>>
>>>> On Jul 1, 2008, at 3:19 PM, Kyle Schochenmaier wrote:
>>>>
>>>>> Patch is against cvs head, and modifies the default num_groups  
>>>>> to be 1
>>>>> and the group_strip_factor = 1  so twod-stripe's defaults still  
>>>>> work
>>>>> when you use it on a single-server, it will now by default  
>>>>> behave the
>>>>> same as simple-stripe until the user modifies the num_groups and
>>>>> strip_factor xattrs.
>>>>>
>>>>> Also added a brief description of the definitions for the  
>>>>> dist_params
>>>>> in twod-stripe.
>>>>>
>>>>> Tested fully functional on my servers.
>>>>>
>>>>> Kyle
>>>>>
>>>>>
>>>>> Index: src/io/description/dist-twod-stripe.c
>>>>> === 
>>>>> ================================================================
>>>>> RCS file: /anoncvs/pvfs2/src/io/description/dist-twod-stripe.c,v
>>>>> retrieving revision 1.5
>>>>> diff -r1.5 dist-twod-stripe.c
>>>>> 6a7,15
>>>>>>
>>>>>> /* twod-stripe will take all of the servers in the filesystem and
>>>>>> * partition them into num_groups groups.  Data will then be  
>>>>>> striped to
>>>>>> * each group before we move onto the next group.  The  
>>>>>> strip_factor will
>>>>>> * determine how many chunks of strip_size are written to each  
>>>>>> server
>>>>>> * in each group before we transition to the next group.
>>>>>> * The striping on the group level is done round-robin in the same
>>>>>> * fashion as simple-stripe
>>>>>> */
>>>>>>
>>>>>
>>>>>
>>>>> Index: include/pvfs2-dist-twod-stripe.h
>>>>> === 
>>>>> ================================================================
>>>>> RCS file: /anoncvs/pvfs2/include/pvfs2-dist-twod-stripe.h,v
>>>>> retrieving revision 1.4
>>>>> diff -r1.4 pvfs2-dist-twod-stripe.h
>>>>> 17c17
>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 2
>>>>> ---
>>>>>>
>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 1
>>>>>
>>>>> 19c19
>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 256
>>>>> ---
>>>>>>
>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 1
>>>>>
>>>>>
>>>>> -
>>>>> Kyle Schochenmaier
>>>>> <twod-dist.patch>_______________________________________________
>>>>> Pvfs2-developers mailing list
>>>>> Pvfs2-developers at beowulf-underground.org
>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>>>
>>>>
>>>
>>>
>>>
>>> -- 
>>> Kyle Schochenmaier
>>> _______________________________________________
>>> Pvfs2-developers mailing list
>>> Pvfs2-developers at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>