[Pvfs2-users] Question about redundancy

[Erich Weiler]

Hi All-

I've looked through the list a bit (and read the documentation on pvfs2) 
but I'm a bit unclear on the status of redundancy in this file system. 
Just as an example, if I have:

server1 (metadata server, client and I/O server)
server2 (client and I/O server)
server3 (client and I/O server)
server4 (client and I/O server)

and suddenly server3 literally explodes, like, there's no way I can get 
anything on the disk back, is my entire file system screwed?  Or just 
some files?  For example, if I have a 4MB file, is that single file 
spread out over the 4 nodes or is it striped in a linear fashion all on 
nodes, i.e. does node 1 fill up first then go to node 2, etc.?  I'm 
assuming it's not exactly set up as a RAID5 type redundant system, such 
that if I lose a node and then replace it, it will "rebuild" the file 
system data on that node?

I need to be clear on this before putting a lot of time into it, but it 
sounds like this might be a good solution for our firm, as we have a 200 
node cluster each with one 500GB disk, 400GB of which can be leveraged 
to a massive parallel file system (400GB x 200 nodes = one big ~80TB 
distributed file system).  But that assumes that there is no redundancy, 
other wise that 80TB would be more like 50-60TB max or something because 
there would be some redundancy in there... ?

Thanks for any clarification!

-Erich