[Pvfs2-developers] PVFS-hint

[Sam Lang]

On Oct 6, 2006, at 1:48 PM, Julian Martin Kunkel wrote:

> Hi,
>> The server-settable-dist would be implemented to store the indices
>> for the IO servers in the params field of the PVFS_sys_dist
>> structure.  I used server indices, because the PINT_dist_* interfaces
>> allow for that (through the PINT_request_file_data struct), but its a
>> bit ugly and probably confusing to the user.  We could change that
>> though, and use something like server aliases or hostnames, passing
>> them through to the distribution parameters instead of indices.  This
>> would require a change to the distribution method calls, and the
> I see what you mean, but that is certainly more efford and not  
> useful for the
> majority of the users. I think new functionality like the setting  
> of the
> hosts can be tested easily with the hint mechanisms and once they  
> are used
> more frequently during calls the interfaces can adapt it...
> If the parameters are stored in the dist parameters like you  
> suggest that will
> be a problem for the migration and changed dataservers.

Really?  It seems like knowing what the actual distribution is before  
you migrate allows you to make informed decisions about how to  
migrate.  I guess this is mostly an implementation details argument,  
so it doesn't matter too much.  You guys are the ones that are going  
to be playing with this so I don't mind leaving it up to you.

> Also it will not
> allow to set the servers for all distributions...

Yeah I can't imagine wanting to ever do that.  It would mean passing  
in a distribution different from the default simple-stripe, as well  
as a hint saying you want a specific set of servers in the same  
call.  Seems sort of yucky to me.  I'd rather have all the  
information about the distribution in the distribution.  You're even  
able to use the distribution field in the directory hints structure  
to specify per-directory IO server lists.  Not that you would ever  
want to do that either...

>
>> Right, I don't like the idea of 'overriding' a distribution's
>> behavior with a hint.  Conceptually it seems like the distribution
>> should take care of this, and if it doesn't, we should fix it so it
>> does.
> I agree completely with that one :)
>> Also, the server list hint gets a little ugly across sysint calls,
>> since the list doesn't actually get stored anywhere for that file
>> (does it?).  With the distribution, we only have to specify it once
>> at creation time, instead of passing it along to all future IO calls
>> on that file.
> It is not necessary to store it explicitly, even with the hint it  
> has to be
> specified only during the creation like you propose. Then it  
> chooses the
> datafile servers depending on the names, create dfiles only on  
> those and sets
> the datafile_array for the metafile later according to that list.  
> Thus the
> information is stored outside of the distribution. This is very  
> useful for my
> migration and also works on all distributions.
>
> To come back to the hint struct, I would prefer this one:
> typedef struct
> {
>         char * type; /* null terminated */
>         char * hint; /* null terminated */
> } PVFS_hint;
>
> I do not see the usefulness of letting the user specify a non null  
> terminated
> hint, users/researchers propably will use something like that to  
> cast an
> arbitrary structure into the hint field, which is problematic for
> compatibilty with other architectures.

I'm assuming that only well-known hints (the ones you've defined in  
your internal enum) get encoded and sent to the server.  Do we want  
to require all hints to be ascii?

-sam

>
> Thanks,
> Julian
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>