[PVFS2-users] Redundant data storage

Rob Ross rross at mcs.anl.gov
Thu Sep 2 16:51:32 EDT 2004


Hi Stuart,

There just isn't a good way, at the moment, to do what you're trying to do 
using PVFS2 (or from a practical perspective any other widely available 
cluster or parallel file system, to my knowledge).

Rob

On Thu, 2 Sep 2004, Stuart Anderson wrote:

> Nuts. Anyone know of any way to get there from here? There must be
> a way to mirror across 2 parallel cluster filesystems?
> 
> What about SCSI Target Emulation to get linux boxes to show up
> on the network as block devices? Presumably PVFS could then be
> run in the more traditional fault tolerant mode where the Linux
> software RAID is put underneath PVFS?
> 
> 
> According to Rob Ross:
> > Hi Stuart,
> > 
> > You can't do this.  Linux software RAID works with block devices, not file 
> > systems.  PVFS2 does not create a large block device, it creates a large 
> > file system.  So you can't build a big Linux software RAID on top of 
> > PVFS2 (at least not in an obvious way).
> > 
> > Rob
> > 
> > On Thu, 2 Sep 2004, Stuart Anderson wrote:
> > 
> > > What about splitting my cluster in half and creating 2 100-node PVFS
> > > filesystems that each node mounts as a software raid-1 device.
> > > 
> > > I lose half my potential storage, but will PVFS play nicely with
> > > Linux software RAID so that if any one node crashes (disk and/or CPU)
> > > no other node else will notice? Including the re-syncing of the
> > > crashed PVFS filesystem after the node/disk are replaced and it is rebuilt?
> > > 
> > > According to Rob Ross:
> > > > Hi Stuart,
> > > > 
> > > > There are plans for mirroring between servers in PVFS2, but that is 
> > > > currently a research project and not something that will be deployable in 
> > > > the near future.  There are also plans for a more coarse-grain ECC-based 
> > > > system, but that's probably even further out.
> > > > 
> > > > Both of these are actually hard problems, which is why you see so many
> > > > groups relying on some shared storage and failover techniques.  I don't
> > > > know of anyone providing this capability in a production-quality system
> > > > that could be deployed on a cluster of that size, but I'm not quite as up 
> > > > on all the commercial products as I should be.
> > > > 
> > > > If you do find something, I'd appreciate a quick email pointing me to such 
> > > > a solution :)!
> > > > 
> > > > Regards,
> > > > 
> > > > Rob
> > > > 
> > > > On Wed, 25 Aug 2004, Stuart Anderson wrote:
> > > > 
> > > > > Are there any plans for PVFS2 to support redundant storage of data at
> > > > > the PVFS layer?
> > > > > 
> > > > > I have in mind a cluster of 200 Linux boxes with cheap internal IDE drives
> > > > > that I would like to turn into a large fast shared filesystem. I understand
> > > > > that if I put multiple drives in each box, or attach external shared
> > > > > storage and use IP fail-over software I can obtain redundant paths to the data.
> > > > > However, will PVFS2 provide any HA for single storage devices visible only
> > > > > to single nodes?
> > > > > 
> > > > > To borrow the RAID nomenclature, if we call a simple PVFS file system RAIN-0
> > > > > will PVFS2 allow RAIN-1 or possibly even RAIN-5?
> > > > > 
> > > > > If not, can anyone point me in the right direction for an alternative
> > > > > cluster filesystem solution?
> > > > > 
> > > > > Thanks.
> > > > > 
> > > > > -- 
> > > > > Stuart Anderson  sba at srl.caltech.edu  http://www.srl.caltech.edu/personnel/sba
> > > > 
> > > 
> > > -- 
> > > Stuart Anderson  sba at srl.caltech.edu  http://www.srl.caltech.edu/personnel/sba
> > > 
> > > 
> > 
> 
> -- 
> Stuart Anderson  sba at srl.caltech.edu  http://www.srl.caltech.edu/personnel/sba
> 
> 


More information about the PVFS2-users mailing list