[Pvfs2-developers] Re: unifying config files

[Sam Lang]

On Jun 20, 2007, at 4:52 PM, Murali Vilayannur wrote:

> Sam,
>>
>> That's true.  The multiple servers per node stuff was added to
>> genconfig just for debugging.  I can't think of any good uses for
>> that in practice.  Are there?  If not, we might be able to get away
>> with just using a -p <port> argument in cases where multiple servers/
>> host is desired.
>
> If a single node has multiple cpus and multiple disks, it may not  
> be a bad idea
> to run multiple server processes on it..

Hopefully we can saturate the processors as much as possible with  
threads in the server process, and multiple disks can be software  
raided.  I hear what you're saying though.  Esp. if there are  
multiple NICs on the box.  Maybe with an 80-core processor we can  
just drive 80 disks and 80 NICs all in one box!  No bus bottlenecks  
there for sure.  ;-)

> I guess the alias is in some ways a succint abbreviation of what host,
> port and interface
> a server process is designed to run on, no?

Sure but how does a server that doesn't have the config yet lookup  
the alias?  I was thinking that  server startup would look something  
like:

pvfs2-server <master endpoint>

If I'm the master, I listen for others.  If I'm not, I report to the  
master.

>
>>
>> Maybe its not a big deal -- and we make people change their scripts
>> and config files.  That's certainly easier/cleaner.
>
> Indeed :)

Ok. :-)  If there are no other objections I'll commit it.  Sorry for  
taking so long on that.

>
>> This is an interesting idea, and allows us to leverage infrastructure
>> that's already on most systems.
>>
>> It seems just as easy though to give each server a master endpoint to
>> send his host,port,etc. config info there.  That avoids a tcp/ip and
>> portmapper dependency doesn't it?  There's a startup issue, but
>> servers could retry until they succeed.
>
> yep; definitely. The only problem with this approach is that we don't
> have a standardized port number on which to query on..
> Hold on..
> Sigh..actually I now realize why I hadnt ventured on this track  
> originally.
> We could have multiple server processes on the same physical node
> and we cannot do the standard master endpoint port technique or the  
> portmapper
> technique since portmapper cannot deal with multiple ephemeral ports
> for a single
> RPC program number..
> We could do the following trick for the multiple server case on  
> same node..
> Have a range of program numbers that are typically unallocated on a
> regular Linux
> distro and keep retrying until some program number <-> port
> registration succeeds.
> The admin tool will scan for port numbers on a set of machines for the
> program number
> ranges and push the config file information and alias information..
> I think it is all getting a little handwavy without any prototype..
> What do you guys think?

I guess I'm not following you with the program number.  Couldn't the  
master just create a new config setup each time he gets another  
server asking to join?  Servers could be removed from the list as  
well.  Of course, this gets a little messy with already existing  
files being distributed over servers that are no longer in the group,  
but we could check each server list on a getattr that all the servers  
are there, and report some error otherwise.  This might make  
migration a bit easier, and would allow some server nodes to fall  
over (even if only temporary) without taking down the whole file system.

-sam

>
>>
>> It also alleviates the need to have an admin tool that queries all
>> the portmappers on all the hosts for all the server ports, and pushes
>> that info back to all the servers.
>
> This is not that big a deal, is it? I am trying to understand how we
> can make it zero-conf
> without telling the system something :)
> thanks,
> Murali
>
>> -sam
>>
>> > thanks,
>> > Murali
>> >
>> >>
>> >> -sam
>> >>
>> >> > thanks,
>> >> > Murali
>> >> >
>> >> > On 6/19/07, Sam Lang <slang at mcs.anl.gov> wrote:
>> >> >>
>> >> >> Hi Murali,
>> >> >>
>> >> >> Just never got around to commit it.  My only complaint is  
>> that it
>> >> >> doesn't allow the server to optionally take a server config
>> >> file (for
>> >> >> legacy scripts, etc.).  I think that might be hard to implement
>> >> >> though...
>> >> >>
>> >> >> -sam
>> >> >>
>> >> >> On Jun 20, 2007, at 1:19 AM, Murali Vilayannur wrote:
>> >> >>
>> >> >> > Hey Sam,
>> >> >> > Any interest in merging this patch to unify config files  
>> while
>> >> >> > starting up pvfs2 servers?
>> >> >> > I have reworked it against HEAD..
>> >> >> > Relevant thread..
>> >> >> > http://www.beowulf-underground.org/pipermail/pvfs2- 
>> developers/
>> >> 2007-
>> >> >> > February/003209.html
>> >> >> > thanks,
>> >> >> > Murali
>> >> >> > <unify-config-files.patch>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >>
>> >
>>
>>
>