[Pvfs2-users] pvfs performance over IB

[Rob Ross]

Perhaps you could try a server built with --without-bmi-tcp, just to get 
a feel for the impact of the TCP polling. That would help us understand 
how close you're getting to what you expect from the rest of the system.

Regards,

Rob

Salmon, Rene wrote:
> Hi,
> 
> 
> Searched through the list archives and it found some posts that show IB
> performance around 400MB/s.  Also found the post that states in order to
> get best performance over IB to configure with "--without-bmi-tcp".  I
> need tpc as well as IB so I this is not an option for me.  
> 
> This is a simple setup. Single pvfs server with a single pvfs client.
> DDR IB connect the two as well as 1Gb ethernet. On the server there is
> an xfs filesystem that can write at about 500MB/sec that is what the
> pvfs server will use.
> 
> 
> I would expect that my bottle neck would be the xfs filesystem at
> 500MB/sec.  When I run pvfs-cp on the client I only get around 260
> MB/sec bandwidth:
> 
> hpcxe005(salmr0)37:/usr/local/opt/pvfs-2.6.3/bin/pvfs2-cp
> -t /test/pvfs2/testfile1 /test/pvfs2/testfile5
> Wrote 655360000 bytes in 2.355496 seconds. 265.336905 MB/seconds
> 
> Any ideas on how to get this to perform a bit better?
> 
> client mount command:
> ---------------------
> mount -t pvfs2 ib://hpcxe007:3335/pvfs2-fs /test/pvfs2
> 
> 
> server pvfs2-fs.conf:
> ----------------------
> <Aliases>
>         Alias hpcxe007 ib://hpcxe007:3335,tcp://hpcxe007:3334
> </Aliases>
> 
> 
> pvfs2-server.conf-hpcxe007:
> ---------------------------
> 
> StorageSpace /pvfs2-storage-space
> HostID "ib://hpcxe007:3335,tcp://hpcxe007:3334"
> 
> 
> Thanks
> Rene
> 
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>