[Pvfs2-users] Question about redundancy

[Murali Vilayannur]

Erich,

> and suddenly server3 literally explodes, like, there's no way I can get
> anything on the disk back, is my entire file system screwed?  Or just
> some files?

You will lose,
 - Any file whose stripe is stored on that node is screwed.
 - You won't lose the entire file system unless the node that crashed
was the MD server and you configured the system with exactly 1 MD
server.
- Access to any files whose metadata was stored on the node that crashed.

You can still access parts of the file provided the server that went
down did not store the metadata required to access the file.

>  For example, if I have a 4MB file, is that single file
> spread out over the 4 nodes or is it striped in a linear fashion all on
> nodes, i.e. does node 1 fill up first then go to node 2, etc.?

It is striped in a linear fashion across all the nodes in a
round-robin fashion (by default). There are other distributions for a
given file.
Spose file stripe is 4 MB, you will have 0-4MB on node 0, 4-8 MB on
node1, and so on. The starting node number is randomized on file
creation so that we don't fill up node 0's disk space unfairly and
cause needless hotspots.

>I'm
> assuming it's not exactly set up as a RAID5 type redundant system, such
> that if I lose a node and then replace it, it will "rebuild" the file
> system data on that node?

Nope; there is no RAID-5. It is a classic RAID-0 style distribution.

>
> I need to be clear on this before putting a lot of time into it, but it
> sounds like this might be a good solution for our firm, as we have a 200
> node cluster each with one 500GB disk, 400GB of which can be leveraged
> to a massive parallel file system (400GB x 200 nodes = one big ~80TB
> distributed file system).  But that assumes that there is no redundancy,
> other wise that 80TB would be more like 50-60TB max or something because
> there would be some redundancy in there... ?

There is no in-built redundancy yet. Unless you setup something like drbd.
Hope you find this information useful.
Do keep us posted.
thanks,
Murali
>
> Thanks for any clarification!
>
> -Erich
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>