[Pvfs2-developers] Re: unifying config files

[Kyle Schochenmaier]

Sam Lang wrote:
>
> On Jun 20, 2007, at 4:52 PM, Murali Vilayannur wrote:
>
>> Sam,
>>>
>>> That's true.  The multiple servers per node stuff was added to
>>> genconfig just for debugging.  I can't think of any good uses for
>>> that in practice.  Are there?  If not, we might be able to get away
>>> with just using a -p <port> argument in cases where multiple servers/
>>> host is desired.
>>
>> If a single node has multiple cpus and multiple disks, it may not be 
>> a bad idea
>> to run multiple server processes on it..
>
> Hopefully we can saturate the processors as much as possible with 
> threads in the server process, and multiple disks can be software 
> raided.  I hear what you're saying though.  Esp. if there are multiple 
> NICs on the box.  Maybe with an 80-core processor we can just drive 80 
> disks and 80 NICs all in one box!  No bus bottlenecks there for sure.  
> ;-)
>
I'm late jumping into this convo, but I've seen situations where a 
dual-cpu node running 2 pvfs2 processes, each with its own hardware raid 
array can outperform the single pvfs2-server process which uses a sw 
raid-0 of the two hardware arrays.

In certain -very controlled- circumstances, it can be advantageous.

my 2c,

KyleI guess the alias is in some ways a succint abbreviation of what host,
>> port and interface
>> a server process is designed to run on, no?
>
> Sure but how does a server that doesn't have the config yet lookup the 
> alias?  I was thinking that  server startup would look something like:
>
> pvfs2-server <master endpoint>
>
> If I'm the master, I listen for others.  If I'm not, I report to the 
> master.
>
>>
>>>
>>> Maybe its not a big deal -- and we make people change their scripts
>>> and config files.  That's certainly easier/cleaner.
>>
>> Indeed :)
>
> Ok. :-)  If there are no other objections I'll commit it.  Sorry for 
> taking so long on that.
>
>>
>>> This is an interesting idea, and allows us to leverage infrastructure
>>> that's already on most systems.
>>>
>>> It seems just as easy though to give each server a master endpoint to
>>> send his host,port,etc. config info there.  That avoids a tcp/ip and
>>> portmapper dependency doesn't it?  There's a startup issue, but
>>> servers could retry until they succeed.
>>
>> yep; definitely. The only problem with this approach is that we don't
>> have a standardized port number on which to query on..
>> Hold on..
>> Sigh..actually I now realize why I hadnt ventured on this track 
>> originally.
>> We could have multiple server processes on the same physical node
>> and we cannot do the standard master endpoint port technique or the 
>> portmapper
>> technique since portmapper cannot deal with multiple ephemeral ports
>> for a single
>> RPC program number..
>> We could do the following trick for the multiple server case on same 
>> node..
>> Have a range of program numbers that are typically unallocated on a
>> regular Linux
>> distro and keep retrying until some program number <-> port
>> registration succeeds.
>> The admin tool will scan for port numbers on a set of machines for the
>> program number
>> ranges and push the config file information and alias information..
>> I think it is all getting a little handwavy without any prototype..
>> What do you guys think?
>
> I guess I'm not following you with the program number.  Couldn't the 
> master just create a new config setup each time he gets another server 
> asking to join?  Servers could be removed from the list as well.  Of 
> course, this gets a little messy with already existing files being 
> distributed over servers that are no longer in the group, but we could 
> check each server list on a getattr that all the servers are there, 
> and report some error otherwise.  This might make migration a bit 
> easier, and would allow some server nodes to fall over (even if only 
> temporary) without taking down the whole file system.
>
> -sam
>
>>
>>>
>>> It also alleviates the need to have an admin tool that queries all
>>> the portmappers on all the hosts for all the server ports, and pushes
>>> that info back to all the servers.
>>
>> This is not that big a deal, is it? I am trying to understand how we
>> can make it zero-conf
>> without telling the system something :)
>> thanks,
>> Murali
>>
>>> -sam
>>>
>>> > thanks,
>>> > Murali
>>> >
>>> >>
>>> >> -sam
>>> >>
>>> >> > thanks,
>>> >> > Murali
>>> >> >
>>> >> > On 6/19/07, Sam Lang <slang at mcs.anl.gov> wrote:
>>> >> >>
>>> >> >> Hi Murali,
>>> >> >>
>>> >> >> Just never got around to commit it.  My only complaint is that it
>>> >> >> doesn't allow the server to optionally take a server config
>>> >> file (for
>>> >> >> legacy scripts, etc.).  I think that might be hard to implement
>>> >> >> though...
>>> >> >>
>>> >> >> -sam
>>> >> >>
>>> >> >> On Jun 20, 2007, at 1:19 AM, Murali Vilayannur wrote:
>>> >> >>
>>> >> >> > Hey Sam,
>>> >> >> > Any interest in merging this patch to unify config files while
>>> >> >> > starting up pvfs2 servers?
>>> >> >> > I have reworked it against HEAD..
>>> >> >> > Relevant thread..
>>> >> >> > http://www.beowulf-underground.org/pipermail/pvfs2-developers/
>>> >> 2007-
>>> >> >> > February/003209.html
>>> >> >> > thanks,
>>> >> >> > Murali
>>> >> >> > <unify-config-files.patch>
>>> >> >>
>>> >> >>
>>> >> >
>>> >>
>>> >>
>>> >
>>>
>>>
>>
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>