[Pvfs2-developers] Distribution by hostname

[Sam Lang]

On Oct 7, 2006, at 2:09 PM, Rob Ross wrote:

> I agree completely with Pete. I think we might consider just  
> adjusting the input parameters on the client side to allow for  
> inclusion of the list of aliases. There is no reason for these to  
> be stored as part of the distribution information on the server, as  
> once the objects are created we already know where they are.

I agree storing them in a file's distribution is redundant.  The  
distribution stored on a directory in the extended attribute is the  
only reason I can think of for storing the list in the distro.

In terms of interfaces, the distribution parameter we pass into  
create is really a sort of hint:  We don't require it (defaulting to  
simple stripe if its not specified), its opaque to the caller  
(specified by a key string: "varstrip" and arbitrary parameters), and  
it functions as a 'hint' in the way we seem to have a need for them,  
changing the distribution of that file.

It might be possible to design a hints interface that both allows us  
to express a distribution as one of possibly many generic hints being  
passed into create, without losing the convenience of just passing in  
a distribution, or a list of servers.

-sam

>
> We have what, 3 distributions that will need to be adjusted to this  
> new scheme? That shouldn't be too bad, and they overall cover a  
> very wide range of options.
>
> Regards,
>
> Rob
>
> Pete Wyckoff wrote:
>> slang at mcs.anl.gov wrote on Fri, 06 Oct 2006 16:33 -0500:
>>> On Oct 6, 2006, at 1:48 PM, Julian Martin Kunkel wrote:
>>>> Also it will not
>>>> allow to set the servers for all distributions...
>>> Yeah I can't imagine wanting to ever do that.  It would mean  
>>> passing  in a distribution different from the default simple- 
>>> stripe, as well  as a hint saying you want a specific set of  
>>> servers in the same  call.  Seems sort of yucky to me.  I'd  
>>> rather have all the  information about the distribution in the  
>>> distribution.  You're even  able to use the distribution field in  
>>> the directory hints structure  to specify per-directory IO server  
>>> lists.  Not that you would ever  want to do that either...
>> I agree with Sam that this is yucky.  I'm hijacking this thread.
>> Let's forget about hints for a moment and decide how we want to
>> extend the concept of distributions, as seen by users, in such a way
>> that they can specify particular IO servers by name.  If this is an
>> interface people want, we should design it properly, not just
>> implement it with hints because we (might) have them.
>> Some issues, please suggest approaches and other issues.  (I'm using
>> "name" here to mean host alias.)
>> 1.  What kind of control do users want?
>>     - all data on one server by name?
>>     - arbirtrary control of stripe sizes and host names?
>> 2.  New distribution name, or extension to existing ones?
>>     - dist-varstrip has a lot of flexibility, but no hostnames
>>     - maybe a new "dist-single-host-by-name" is all that is desired
>> 3.  Store hostnames in on-disk distribution?
>>     - guessing no for the single-stripe distro, but perhaps somebody
>>     can really think of a use case for this?
>> 4.  User API
>>     - through PVFS_dist_create
>>     - (please not through both PVFS_dist_create + some hint)
>>     - via environment variable too?
>> If our design happens to end up as something that would be
>> implemented well by hints, then we can think about using them.  For
>> now, let's just get the design correct.
>> We can come back and argue the merits of a generic hint interface in
>> a different thread.
>> 		-- Pete
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>