[Pvfs2-developers] (no subject)

[slang]

This was all sounding awefully familiar to me...see:

http://www.beowulf-underground.org/pipermail/pvfs2-developers/2007- 
March/003269.html

So I guess this is turning into a monthly discussion. :-)  It sounds  
like people prefer that scheme over the indices array idea I proposed  
at the beginning of this thread.  I agree that wasn't terribly well  
thought-out.

The issue I have with the current implementation (and I think the  
proposed solution in that thread linked above) is that the initial  
server is still chosen randomly in the PINT_cached_config_get_next_ 
[io|meta] call.  Maybe there's a good reason for this?  It seems like  
that code should be part of the distribution.  Pete had a good use- 
case for this:  debugging IO or metadata can be hard to pin-down if  
the first server is always chosen randomly.  We discussed this  
yesterday a bit, and came up with yet another param (optionally  
exists over all distros?), that would specify an ordering to how the  
servers are chosen:

none: 1...N
random-start: choose the first randomly and the rest in order
random: chose all randomly

One could imagine interesting algorithms here that do things like  
randomize within subsets of the range.

Anyway, the distribution callbacks would need to be augmented to  
support this.  Somehow the distribution would have to specify the  
indices, or order the server aliases differently.  So here's a  
proposed solution:

The distribution struct has a new get_ordering callout that takes a  
list of server aliases, and returns a separate list based on the  
algorithm specified (default would be random-start), or if the server  
map were specified directly as part of the distribution, just return  
that.  It seems like the num_dfiles param could also be used to  
figure out the result server list.

int (*get_ordering) (int incount, char *inaliases, int *outcount,  
char *outaliases);

Seem reasonable?  Does this fit in well with everyone's requirements?

-sam




On Apr 27, 2007, at 1:03 PM, Rob Ross wrote:

> I like the idea of specifying aliases rather than handle ranges,  
> etc., if that is feasible. I also agree that we want this interface  
> to allow this specification at create time, but then we want to use  
> our existing scheme for long-term metadata storage (e.g. just keep  
> the handles).
>
> Regards,
>
> Rob
>
> Julian Martin Kunkel wrote:
>> Hi,
>> In general like the proposed solution better than the current, but  
>> an extension which allows the distribution to see the aliases of  
>> the servers would be neat as well. This certainly would solve the  
>> issue of placing datafiles.
>>> Julian, how would this stuff fit in with your migration/hints stuff?
>> I use the actual server aliases as targets to ensure proper  
>> placing of the datafiles in an environment with miscellaneous sets  
>> of dataservers.
>> In my opinion a decoupled system for file creation and the  
>> distribution parameters seems important to allow later relocation  
>> of datafiles to another data servers without conflicting with the  
>> distribution parameters. Therefore, the location of the datafiles  
>> should be transparent for the distribution and never be set in the  
>> distribution parameters itself. Another reason not to store this  
>> info in the distribution parameters is that once the target  
>> dataservers are determined this information simply becomes  
>> obsolete. Julian
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>