[Pvfs2-users] PVFS2 vs. lustre for HPC

[Michael Sternberg]

I am designing a relatively small HPC cluster for use in materials  
modeling and experimental data analysis.  It will be used in a  
research setting, so while a few target applications are known, the  
eventual set will likely extend much further.  The file system use  
will be primarily for traditional "home directory" I/O, not so much  
MPI I/O, though the latter may see a larger presence in the  
application arena.

For the server side, I need failover and extensibility, which is  
where NFS falls short (in addition to its lame concurrency).  Of  
course, I'm looking at lustre and PVFS2.  In the past, it seemed (to  
me) that PVFS(1) wasn't quite suitable to step in for NFS, but PVFS2  
seems to do that just fine now, and it does MPI I/O well by design.   
PVFS2 also seems to have a somewhat more open development process  
than lustre, is easier to configure, and perhaps therefore does not  
need the rather expensive support contracts.  The closest to an  
honest assessment seems to be a recent thread on this mailing list,

	http://www.beowulf-underground.org/pipermail/pvfs2-users/2006- 
December/001664.html

... and one "nearby":

	http://www.beowulf.org/archive/2006-December/017005.html

Are the opinions expressed therein the current consensus?  Are there  
more resources available to address this question (it's got to be a  
FAQ)?  Google gets one hit for lustre on pvfs.org, and that's for the  
SC05 Agenda ;-)


On Tue Dec 12 09:01:52 PST 2006 Robert Latham robl at mcs.anl.gov wrote:
> What Lustre has done a much better job of than we have
> is documenting the HA process.  This is one of our (PVFS) areas of
> focus in the near-term.
>
> We may not have documented the process in enough detail, but one can
> definitely set up PVFS servers with links to shared storage and make
> use of things like IP takeover to deliver resiliancy in the face of
> disk failure, and have had this ability for several years now (PVFS
> users can check out 'pvfs2-ha.pdf' in our source for a starting
> point).

That's dated  June, 2004  -- any updates?  BTW, how difficult would  
it be to put that pdf directly on the web? ;-)


Regards, Michael