[Pvfs2-developers] PVFS2 Performance Problem -

[Sam Lang]

On Jul 1, 2009, at 5:05 PM, David Bonnie wrote:

> Rob -
>
> Performance is down across all PVFS2 installations.  The benchmark  
> simply creates files of a random size (between 1 and 25 MB) in a  
> single folder on the mounted PVFS2 partition, 16 KB at a time.  It's  
> not anywhere near ideal, but it's the workload I'm working with.

How many files?
-sam

>
> Prior to this problem we were getting ~22 MB/s write throughput and  
> we're down to about 2.5 MB/s for no apparent reason.  Reads are down  
> from about 55 MB/s to 30 MB/s.  No hardware has changed and as far  
> as I can tell no hardware has died either.
>
> - Dave
>
>
> On Wed, Jul 1, 2009 at 4:00 PM, Rob Ross <rross at mcs.anl.gov> wrote:
> Do you mean that 2.8.0 is fast and 2.8.1 is slow? Can you describe  
> the benchmark and how you are doing your measurements?
>
> Rob
>
>
> On Jul 1, 2009, at 4:43 PM, David Bonnie wrote:
>
> Hello all -
>
> I'm having trouble figuring out a problem with performance  
> depredation on a simple 10 node cluster.  Prior runs on the cluster  
> (before this problem manifested itself) resulted in bandwidth and  
> IOPS about 10 times higher on a small file creation workload.  Each  
> node is running as a metadata server and a data server.
>
> The problem is persistent between versions and installations of  
> PVFS2 2.8.0 and 2.8.1.  Rebooting all of the nodes didn't improve  
> anything.  The network connections (simple GigE) showed no errors or  
> dropped packets.  Using different physical disks (both SAS and FC)  
> didn't improve things.  The kernel logs didn't show anything out of  
> place nor did the pvfs2 server or client logs.  It seems like a  
> network issue but I can't seem to find anything wrong with any of  
> the connections.
>
> Has anyone seen this kind of problem before?  I seem to remember  
> something on the list before about performance suddenly dropping but  
> I can't find the message now (of course).  Any insight would be  
> appreciated!
>
> Thanks,
>
> - Dave
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20090701/bcf5e0b8/attachment.htm