[Pvfs2-developers] Distribution by hostname

[Walter B. Ligon III]

Hmm, that's interesting.
Of course, the issue of interface and internal rep are different.  We 
can allow the interface to pass generic hints but pass the distribution 
related parameters through the SM as before.  For that matter, the 
proposed server array really shouldn't be part of that distribution data 
- distributions don't actually have anything to do with what servers the 
data goes to.  As non-intuitive as that sounds its the way we do things 
currently.  The number and selection of servers is irrelevant to the 
distribution - it only needs to know how many servers there are and 
which one it is currently processing.

But this issue of the client not having access to server identities is 
even more troubling.

As an asside - what is the application for this anyway?  I've never 
encountered any such application, so I'm not clear why we think it is so 
important.

Walt

Sam Lang wrote:
> 
> On Oct 9, 2006, at 12:05 PM, Julian Martin Kunkel wrote:
> 
>> Hi,
>>
>>> So I think we're mostly trying to work out what our API should really
>>> be, whether we should extend the distro functionality vs. going  totally
>>> to hints, and if we go to hints what that API should look like,  right?
>>
>> probably the distribution needs a function which initializes/sets  the 
>> internal
>> parameter depending on the string. Maybe another one which fancy  
>> writes the
>> parameters to a char buffer.
>>
>>> I'm down with that.  For the record, my concern over strings isn't in
>>> our processing them - its the users having to muck with them -
>>> especially when passing numeric data.  I just worry about lots of
>>> mallocing/string copying blech when all you really want to do is  set 
>>> the
>>> stripe size to the variable x.
>>
>> I think extra mallocing is not really necessary only once to  generate 
>> the
>> string which can be generated with snprintf  the string could look
>> like  
>> "ioservercount=2;ioservers=node1,node4;dist=simplestripe;dist_stripesi 
>> ze=65536"
> 
> 
> I'd rather just leave the PINT_dist_set_param function the way it is  
> and have some other methods in the PINT_dist_methods structure as well:
> 
> get_num_dfiles
> get_server_array
> 
> This way the create sm (and the rest of the pvfs client code) is able  
> to keep treating the distribution hint as an opaque object.
> 
> We don't actually expose the server list (from the config file) to  the 
> client at all, and I think we have good reasons for doing this,  
> especially when we talk about server migration and failover where  
> servers may possibly come and go over time (dynamic configs).  The  
> client is required to 'guess' the appropriate server list via some  
> out-of-band mechanism.
> 
> If we want to specify the list of servers through the system  
> interfaces, we'll probably have to provide some more mgmt functions.   
> The client doesn't have direct access to the list of servers, since  we 
> don't export any config functions.  There's a  
> PVFS_mgmt_get_server_array call, but it returns an array of  
> PVFS_BMI_addr_t types.  That addr_t is an opaque reference (uint64_t)  
> to the server.  So we would need to provide an external wrapper  around 
> BMI_addr_rev_lookup.  Alternatively, we can just write a  management 
> interface that give us a list of aliases.
> 
> -sam
> 
>> whereas options without the prefix "dist_" may be processed by code  
>> common to
>> all distributions.
>>
>> I do not see the benefit of putting the responsibility of server  
>> mapping to
>> the distribution right now (maybe for raid ?). I like outsourced/ common
>> processing which is done in the create sm right now.
>>
>>> I personally don't think the environment variable approach is all  that
>>> useful for PVFS. I believe this because there are only a *very*  limited
>>> number of cases where it would apply cleanly (e.g. pvfs2-cp) that  there
>>> isn't another mechanism for doing the same thing (e.g. MPI-IO  hints). I
>>> think that MPI-IO hints should have some mechanism along those lines,
>>> and I wouldn't mind discussing that separately, but that's a  different
>>> topic.
>>
>> Ok I think its true that it might not be that useful for the common  
>> case and
>> that MPI hints should be used.
>> I found it useful for testing and experimenting with PVFS and I  think 
>> its no
>> big deal if environment processing will not be added to CVS. I will  
>> add the
>> functionality fitting to the new hint interface in my branch.
>>
>> I don't think that I actually might be useful for implementing the  
>> new hint
>> mechanism because I think you have to revise the code anyway (and  it 
>> won't be
>> so much work I guess, either), but that is no big deal.
>>
>> Julian
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>
> 
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers

-- 
Dr. Walter B. Ligon III
Associate Professor
ECE Department
Clemson University