[Pvfs2-users] Multiple Metadata servers on pNFS/PVFS2

[Phil Carns]

You'll probably need to refer to this page (and the wiki link there) for 
the latest pNFS information:

http://www.citi.umich.edu/projects/asci/pnfs/linux/

thanks,
-Phil

Nirmal Thacker wrote:
> Hi Phil,
> 
> Yes, I recently learned that having multiple pNFS MDS servers are not
> possible.
> 
> Has there been some work [either some benchmarks/new design] in this
> direction?
> 
> Thanks
> Nirmal
> Phil Carns wrote:
>> Hi Nirmal,
>>
>> Are you trying to have multiple pNFS MDS servers, or multiple meta
>> servers in the PVFS volume?
>>
>> I don't believe that the former is supported (but I was hoping someone
>> else would chime in- I have not used the pNFS/PVFS implementation
>> first hand).
>>
>> The latter should work fine without any particular changes to the
>> installation steps except for the PVFS configuration files.
>>
>> In either case I am pretty sure that the bind option to mount is not
>> going to do what you are looking for.
>>
>> -Phil
>>
>>
>> Nirmal Thacker wrote:
>>> Hello,
>>>
>>> I have a question regarding CITI's implementation of pNFS/PVFS2
>>>
>>> Alternatively you could point me to a mailing lists for users of
>>> pNFS/PVFS2 by the CITI group. I cannot seem to find a forum/group who
>>> discuss issues of pNFS/PVFS2 [if it is not this one]
>>>
>>> My problem is as follows:
>>>
>>> I want to use more than a single Metadata server [MDS] in CITI's
>>> pNFS/PVFS2. I use the instructions described here:
>>>
>>> http://www.citi.umich.edu/projects/asci/pnfs/docs/pnfspvfs2.html
>>>
>>> I understand these are not of the current build, but I am using an older
>>> build of pNFS/PVFS2.
>>>
>>> Now if there are two or more MDS nodes, then how will clients mount the
>>> MDS nodes on a single mount point? I am familiar with the mount --bind
>>> argument
>>> [http://www.redhatmagazine.com/2007/03/19/how-do-i-mount-an-nfsv4-filesystem-in-two-locations-on-a-single-client/].
>>>
>>> Is this what we must do?
>>>
>>> This situation arises on two occasions
>>> 1. When the MDS mounts a PVFS2 filesystem. Multiple MDS then mount the
>>> MDS PVFS2 filesystems at one point.
>>>
>>> To quote from the instructions :
>>>
>>> // Load kernel interface
>>> // Run on MDS
>>> insmod /usr/local/bin/pvfs2-server/pvfs2.ko
>>> /usr/local/bin/pvfs2-server/sbin/pvfs2-client -p
>>> /usr/lo:cal/bin/pvfs2-server/sbin/pvfs2-client-core
>>> mount -t pvfs2 tcp://"mds_server_hostname":3334/pvfs2-fs /mnt/pvfs2
>>>
>>> 2. When the clients mount an NFS4 filesystem [essentially a PVFS2
>>> filesystem exported by the MDS]
>>>
>>> To quote:
>>>
>>> Step 7: Mount PVFS2 file system using pNFS
>>>
>>>     * Copy /etc/pvfs2tab file to every client
>>>     * Example:
>>>
>>>       // On client
>>>       insmod /usr/local/bin/pvfs2-layout/pvfs2-pnfs.ko
>>>       /usr/local/bin/pvfs2-layout/sbin/pvfs2-client -p
>>> /usr/local/bin/pvfs2-layout/sbin/pvfs2-client-core
>>>       mount -t nfs4 "mds_server_name":/ /mnt/nfs4/
>>>
>>>
>>> In both cases, multiple MDS cause the mounts to overlap. Although I have
>>> not tried this out, I suspect this is would result in an incorrect
>>> configuration.
>>>
>>> Have you come across this issue when you worked with pNFS/PVFS2 before?
>>>
>>> Thanks for your time.
>>>
>>> Nirmal Thacker
>>>
>>> _______________________________________________
>>> Pvfs2-users mailing list
>>> Pvfs2-users at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>