[Pvfs2-users] PVFS2 over Infiniband

[Robert Latham]

On Fri, Sep 08, 2006 at 02:39:28PM +0900, Á¶Çý¿µ wrote:
> I have a problem to do the PVFS2(ver.1.5.1) configuration with OpenIB(OFED-
> 1.0).
> 
> After I installed the OpenIB and pvfs2, I made pvfs2tab
> file(ib://11.0.0.4:3335/pvfs2-fs /mnt/pvfs2 pvfs2 default,noauto 0 0) and
> run the mount_pvfs2.sh.

> The pvfs2 file system was mounted, but the speed was very slow(just about
> 3~20MB/seconds). 
> 
> Is there any problem in my configuration?

It could be a problem with how you measured performance.  Did you use
cp(1)?  Try measuring performance with the pvfs2-cp utility: it uses a
larger block size and should give you a better initial value for
single-node performance.  
 
> So I have questions. PVFS2 say it support IB. It means IP over IB or RDMA
> over IB?? It is possible both? 

I'll have to defer to Pete on this one. I think it supports real RDMA
over IB.   If you want to run IP over IB, then I think you would just
use tcp. 

> In addition I want to know how to configure pvfs2 with IB.

I think you've done everything correctly.  Tell us how you measured
the slow performance and we can either fix it or suggest other
approaches.

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B