[Pvfs2-developers] Separate data and metadata storage paths -

[Sam Lang]

Yes with metadata and data in the same server process, the same  
storage space (and databases) are used for storing metadata objects  
and data objects.  The advantages of using the same databases in the  
same process are from coalescing and grouping of operations together.   
That only happens when both metadata and I/O workloads occur together  
on the file system.

Splitting the metadata and data really just boils down to putting the  
bstreams in a separate path, specified probably in the config file.   
What are your plans there?

-sam

On May 29, 2009, at 11:06 AM, Nicholas Mills wrote:

> Sumit -
>
> You're correct. Running two servers on the same node (one for data,  
> one for metadata) was the first thing we tried. In the end we  
> decided it would be better just to go ahead and add this  
> functionality directly to Trove. I'm not sure how significant the  
> performance hit would be from running two servers.
>
> --Nick
>
> On Fri, May 29, 2009 at 9:52 AM, Sumit Narayan <sumnaray at gmail.com>  
> wrote:
> This is pretty interesting.
>
> I am just curious, can't this be done by running two servers on the
> same node and treating one as metadata server and other as I/O server
> - each having their own data space. It might add just a little extra
> load on the node. I am guessing PVFS takes some advantage of putting
> metadata and data on same node. Can someone fill me in as to what they
> are.
>
> Thanks,
> Sumit.
>
>
>
> On Thu, May 28, 2009 at 5:17 PM, Sam Lang <slang at mcs.anl.gov> wrote:
> >
> > Hi David,
> >
> > I don't see any problems with putting that in future releases once  
> its
> > ready.  It would be great if you could update the developers list  
> with your
> > planned changes, and possibly send along patches as you have them,  
> allowing
> > everyone to provide input, etc.
> >
> > -sam
> >
> > On May 28, 2009, at 5:10 PM, David Bonnie wrote:
> >
> >> Everyone -
> >>
> >> Nick and I are working at LANL this summer and one of our projects
> >> involves modifying PVFS2 to use separate storage paths for  
> metadata versus
> >> file data on a single PVFS2 server.  The goal is to be able to  
> place the
> >> metadata database files on faster drives (SSDs, RAM drives, etc)  
> to speed up
> >> metadata operations while still being able to service data  
> requests from a
> >> standard HDD.  Does anyone forsee a problem with this being  
> pushed into the
> >> main distribution once it is finished?
> >>
> >> Thanks,
> >>
> >> - Dave
> >> _______________________________________________
> >> Pvfs2-developers mailing list
> >> Pvfs2-developers at beowulf-underground.org
> >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2- 
> developers
> >
> > _______________________________________________
> > Pvfs2-developers mailing list
> > Pvfs2-developers at beowulf-underground.org
> > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
> >
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20090529/c648d6c4/attachment.htm