[Pvfs2-developers] [PATCH] modify twod-stripe's default params
and add some documentation
Sam Lang
slang at mcs.anl.gov
Mon Jul 7 11:47:20 EDT 2008
On Jul 7, 2008, at 10:24 AM, Phil Carns wrote:
> There are a couple of options for dealing with the case where
> num_groups > num_dfiles if we want to keep the current defaults:
>
> 1) Transparently reduce the num_groups within the distribution
> calculation. From the user's perspective the params wouldn't
> change; the distribution would just automatically fall back to
> acting like simple stripe. We would just need to add this to the
> top of a few of the dist functions:
>
> if(num_groups > server_ct)
> {
> num_groups = server_ct;
> }
>
> 2) Return an explicit error at create time. The varstrip
> distribution has this behavior now; it actually prevents you from
> creating a file if the dfile count isn't plausible for the
> distribution parameters. The way to do this is to implement the
> get_num_dfiles() function in twod_stripe so that it can perform a
> safety check and return an error at create time if the user tries to
> use too high of a num_groups setting. On the command line you would
> get a generic errno (EINVAL or whatever) but we could put a
> descriptive message in the logs.
>
> Option 1) seems reasonable to me; it is a small code change, we get
> to keep the existing desirable default parameters, and the single
> server case just transparently falls back to simple stripe.
There's a third option, and what I've been proposing -- instead of
changing the actual parameter, adjust the value during the
logical_to_physical calculation.
-sam
>
>
> -Phil
>
> Rob Ross wrote:
>> I'm glad we agree that we should make the distribution work with
>> default parameters, and I agree there is more than one way to
>> accomplish that.
>> If you can find a way to have it default to something more
>> sensible, great. If not, I'm happy to have it behave like simple
>> stripe if the user doesn't do any tuning.
>> -- Rob
>> On Jul 4, 2008, at 2:21 PM, Sam Lang <slang at mcs.anl.gov> wrote:
>>>
>>> Having it work with default parameters doesn't necessarily mean
>>> making those default parameters exactly like simple stripe. Our
>>> best all around solution is to fix this bug with a simple check in
>>> the 2d code and set the default parameters to something reasonable
>>> for a 2d distribution in the general case. We looked at this last
>>> year and decided that num_groups=2 and striping factor = 256 were
>>> reasonable, maybe they're not, but it doesn't make sense to make
>>> them identical to the simple stripe dist.
>>> -sam
>>>
>>> On Jul 1, 2008, at 8:08 PM, Rob Ross wrote:
>>>
>>>> I agree that it's best to have it work for everyone with default
>>>> parameters.
>>>>
>>>> -- Rob
>>>>
>>>> On Jul 1, 2008, at 4:15 PM, "Kyle Schochenmaier" <kschoche at gmail.com
>>>> > wrote:
>>>>
>>>>> We should use simple stripe in that case, which is why i have it
>>>>> defaulted to it now, someone was playing with twod-dist today and
>>>>> noticed that it defaults to having 2 groups, which causes errors
>>>>> when
>>>>> you run it on only a single server. I would much rather have the
>>>>> defaults allow the distribution to work on *any* setup, rather
>>>>> than
>>>>> just a n>1 servers setup.
>>>>>
>>>>> Its not particularly useful, but I belive it is the right thing
>>>>> to do,
>>>>> maybe I'm wrong?
>>>>> (the original default settings are worthless for my applications
>>>>> so I
>>>>> have to change them anyhow)
>>>>>
>>>>> Kyle
>>>>>
>>>>> On Tue, Jul 1, 2008 at 4:04 PM, Sam Lang <slang at mcs.anl.gov>
>>>>> wrote:
>>>>>>
>>>>>> Kyle,
>>>>>>
>>>>>> How is this patch useful for you guys? With a single server
>>>>>> (and only one
>>>>>> stripe), it doesn't matter what the factor and num_groups are,
>>>>>> does it? It
>>>>>> still behaves the same as simple-stripe in that case. With
>>>>>> multiple servers
>>>>>> setting the factor and num_groups to 1 does make the 2d
>>>>>> distribution act
>>>>>> like simple-stripe, but why not just use simple stripe in that
>>>>>> case?
>>>>>>
>>>>>> -sam
>>>>>>
>>>>>>
>>>>>> On Jul 1, 2008, at 3:19 PM, Kyle Schochenmaier wrote:
>>>>>>
>>>>>>> Patch is against cvs head, and modifies the default num_groups
>>>>>>> to be 1
>>>>>>> and the group_strip_factor = 1 so twod-stripe's defaults
>>>>>>> still work
>>>>>>> when you use it on a single-server, it will now by default
>>>>>>> behave the
>>>>>>> same as simple-stripe until the user modifies the num_groups and
>>>>>>> strip_factor xattrs.
>>>>>>>
>>>>>>> Also added a brief description of the definitions for the
>>>>>>> dist_params
>>>>>>> in twod-stripe.
>>>>>>>
>>>>>>> Tested fully functional on my servers.
>>>>>>>
>>>>>>> Kyle
>>>>>>>
>>>>>>>
>>>>>>> Index: src/io/description/dist-twod-stripe.c
>>>>>>> =
>>>>>>> =
>>>>>>> =
>>>>>>> ================================================================
>>>>>>> RCS file: /anoncvs/pvfs2/src/io/description/dist-twod-stripe.c,v
>>>>>>> retrieving revision 1.5
>>>>>>> diff -r1.5 dist-twod-stripe.c
>>>>>>> 6a7,15
>>>>>>>>
>>>>>>>> /* twod-stripe will take all of the servers in the filesystem
>>>>>>>> and
>>>>>>>> * partition them into num_groups groups. Data will then be
>>>>>>>> striped to
>>>>>>>> * each group before we move onto the next group. The
>>>>>>>> strip_factor will
>>>>>>>> * determine how many chunks of strip_size are written to each
>>>>>>>> server
>>>>>>>> * in each group before we transition to the next group.
>>>>>>>> * The striping on the group level is done round-robin in the
>>>>>>>> same
>>>>>>>> * fashion as simple-stripe
>>>>>>>> */
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Index: include/pvfs2-dist-twod-stripe.h
>>>>>>> =
>>>>>>> =
>>>>>>> =
>>>>>>> ================================================================
>>>>>>> RCS file: /anoncvs/pvfs2/include/pvfs2-dist-twod-stripe.h,v
>>>>>>> retrieving revision 1.4
>>>>>>> diff -r1.4 pvfs2-dist-twod-stripe.h
>>>>>>> 17c17
>>>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 2
>>>>>>> ---
>>>>>>>>
>>>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_GROUPS 1
>>>>>>>
>>>>>>> 19c19
>>>>>>> < #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 256
>>>>>>> ---
>>>>>>>>
>>>>>>>> #define PVFS_DIST_TWOD_STRIPE_DEFAULT_FACTOR 1
>>>>>>>
>>>>>>>
>>>>>>> -
>>>>>>> Kyle Schochenmaier
>>>>>>> <twod-dist.patch>_______________________________________________
>>>>>>> Pvfs2-developers mailing list
>>>>>>> Pvfs2-developers at beowulf-underground.org
>>>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Kyle Schochenmaier
>>>>> _______________________________________________
>>>>> Pvfs2-developers mailing list
>>>>> Pvfs2-developers at beowulf-underground.org
>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>>
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
More information about the Pvfs2-developers
mailing list