[Pvfs2-users] pvfs2 as 15T filesystem
murali.vilayannur at gmail.com
Wed Oct 3 13:18:27 EDT 2007
> I am in the process of ordering 5 3T 1U machines. I was originally
> going to try to use gfs in this cluster to create a large contiguous
> filesystem for my users. The gfs irc gents told me that their product
> was not designed for this, at least not at my scale. I need it to scale
> out to 120T possibly. I'm thinking of using pvfs2 as my filesystem.
Remember that pvfs2 does not have a native on-disk format and relies on
other on-disk file systems. Therefore your scaling will be limited by the choice
of on-disk FS that you choose on each of the 3T nodes.
ext3's max volume size ranges from 2TB to 32 TB (block sizes 1KB to 8KB)
Remember that block sizes are directly related to the max.
architectural limit of PAGE size
and the higher block sizes don't work on IA32.
XFS scales upto 8 EB and hence won't have any problems.
ext4 will also probably scale to this capacity *if* it stabilizes.
> looked at gfs, pvfs2, and lustre, but I like pvfs2 the most because I
> can actually compile and run it pretty simply. Unfortunately, there
> isn't really system level fallbacks, so if one of my nodes goes, so too
> does my fs, and of course I cant store a db on it, which might be nice.
> What do people out there suggest? Is there something else better for my
> purposes? I'm not storing homes on this fs, though that might be nice,
> mostly I'll be storing images and text files. Thanks for any comments!
It seems like PVFS2 should do just fine for storing images and text
files. Keep in mind though that there are
no redundancy options at the PVFS layers and if any of the disks go
there is a good chance some/all of the FS will be left inaccessible.
Node failures is not that big a problem. It is disk failures that will
leave you in a bad state.
Cannot comment on gfs or lustre since I havent' used them.
Hope this helps,
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
More information about the Pvfs2-users