[Pvfs2-users] pvfs configuration questions

[Phil Carns]

rcbord at wm.edu wrote:
> Hi all,
>   We are wanting to configure the compute nodes on our cluster as a 
> local/scratch directory for Batch jobs.  Each compute node as 30+ Gbytes
>  available.  We need to be able support an ocean modeling code that
> generates 130+Gbyte outputs files.  We have a single login node
> with 21 compute nodes mix of amd single and dual core nodes.  We have an
> infiniband switch and a fast ethernet switch.  There are 3 snap servers 
> with 1-4 TByte of disk.  Right now the we have NFS mounted home 
> directories from the snap servers.
> 
> So how do we integrate pvfs?  Do we need a seperate io server or can we
> use the snap servers? or the login node?  Can we do this so it is 
> transparent to the users?

Hi,

I don't know much about snap servers, but I would assume that you 
probably cannot install third party software on them.

You could put PVFS servers processes on your compute nodes.  That may be 
Ok if you just need it to be scratch space (or at least for evaluation 
purposes), but it will only give you a 630 GB volume and you may 
possibly run short on bandwidth with the apps and servers running on the 
same nodes.  Otherwise you would need to come up with some extra 
servers/storage to run as PVFS servers.

Regardless of where you run the PVFS services it should be transparent 
to your users.  You can mount PVFS on all of your clients and access it 
just like an NFS volume, or if you are using MPI-IO you can build MPI to 
include ROMIO/PVFS support.

-Phil