[Pvfs2-developers] Active Storage?

[Rob Ross]

Hi David,

We have discussed with the group at PNNL the idea of integrating some of 
their active storage work into PVFS in the future, but that work hasn't 
started quite yet.

The idea of using FUSE as a way to hook into the I/O path is pretty 
interesting and might be a good way to prototype.

pvfs2-viewdist can be used to look at how files are distributed. Data 
for files is stored in a single UNIX file on each server in a 
subdirectory under the storage space directory. Generally you can choose 
how many servers a file is spread across and what strip size is used 
through configuration parameters and extended attributes.

Regards,

Rob

David Brown wrote:
> I know there's been lots of research into investigating the
> possibilities of putting some form of processing on archives with
> massive amounts of disks (active-storage). I was wondering if anyone
> has approached the pvfs developers with ideas of integrating an
> active-storage like system to the servers to allow them to perform
> preprocessing on the data before it gets sent over the network?
> 
> One could implement something very quick using fuse and have the
> servers work on top of the fuse file system. However, that would
> depend on how the raw data of the files are stored on each of the
> storage servers. Could someone give me a place to start looking as to
> where the raw data of the files are located? algorithms used for
> distributing files among the storage servers? block sizes available?
> 
> Any information would be helpful.
> 
> Thanks,
> - David Brown
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>