[PVFS-developers] TODO

[Jeffrey B Layton]

"Walter B. Ligon III" wrote:

> >
> > Rob,
> >
> >    Just thought I would chime in :)    I'm not a hacker in any sense
> > (except maybe aircraft design/analysis codes ....), but what I care
> > about is that if I lose a node while not preforming I/O, I would like
> > to be able to recover the data that was on the node onto the
> > remaining I/O nodes. Now, being lazy it would be nice that PVFS
> > detected this and then "recovered" the data onto the remaining
> > I/O nodes without my intervention (perhaps some sort of heartbeat
> > mechanism - extremely low bandwidth, CPU cycles). Now, if I
> > lose a node, I have to take down the I/O daemon, change the
> > /pvfs/.iodtab and then restart all the daemons (not hard, just
> > somewhat time consuming).
> >    I guess this scenario goes along with my image of RAID5
> > for PVFS. This is just part of my wish-list. The intermediate
> > steps look good and very easy to implement (from what I've
> > read).
> >    Thanks for the hard work!
> >
> > Jeff
>
> Jeff, I think all of what you are asking for here is pretty much
> independent of the RAID level being used to provide the redundancy.
> All of this would work fine with a mirroring scheme - in fact the
> file system wouldn't have to do anything at all if a node fails,
> the other copy would be out there so it would just keep working.
> Ideally another node would be select as a mirror and the mirror
> copy rebuilt in the background.

Walt,


I agree. I like RAID5 because of the nice way to rebuild
the missing files (transparent file rebuilds). Yes, it would
involve locking and the performance hit would be large.
I guess the performance hit kind of negates the idea of PVFS
as a high-speed filesystem.


>
>
> The difference between RAID 5 and RAID 2 (mirroring) is the storage
> efficiency.  With RAID 2 50% of your physical storage is used for
> redundancy. With RAID 4, it is quite a bit less, depending on how
> it is configured, but a typical setup might have 20% or less used
> for redundancy.  Of course, the physical space used for the redundancy,
> the better (because you have more usable space) BUT, in this case
> there are ALSO performance implications.
>
> If you look at the original design of RAID 5, it was NOT intended to
> be a high performance solution.  Many people assume that because it
> involves striping that it automatically gives better performance, and
> in some cases it does, but typically RAID 2 does better performance-wise.

Agreed. I like RAID 5 for redundancy (the ability to lose a drive
and not any data). Hardware RAID controllers and massive disk
RAM caches have helped regained some of the lost performance,
but of course the cost goes up.

>
>
> So, we're all about getting you what we want, and I think in the long
> run we'd even like to have RAID 5 as an option - but RAID 2 is probably
> a better target for the short term.

I vote for cheap, fast, and the ability to tolerate a drive loss. So, this
points to mirroring and non-RAID hardware controllers. I'm all for
it. I guess my purpose behind my comments is that I want to take
PVFS from the level now where I tell my users, "It's fast, it's huge,
it's not backed-up, and if we lose a drive or a node you're data's
gone," to something like, "It's fast, it's huge, it's not backed-up,
but it has some redundancy so we can lose X drives or nodes before
we lose data." So far my users are very excited with the high-speed
and the huge amount of usable space. I just want to give them some
more comfort (also so they don't start copying HUGE files into their
/home directories, scream for more quota, and then force me to back
up everything - I hate switching tapes).

Thanks for the comments and the hard work.

Jeff


>
>
> Walt
>
> --
> Dr. Walter B. Ligon III
> Associate Professor
> ECE Department
> Clemson University
>
> _______________________________________________
> PVFS-developers mailing list
> PVFS-developers@www.beowulf-underground.org
> http://beowulf-underground.org/mailman/listinfo/pvfs-developers