[Pvfs2-users] Re: PVFS2 over IB RDMA

[Troy Benjegerdes]

Try changing your client line from:

ib://dn2-ib:3335/pvfs2 /mnt/pvfs2 pvfs2 defaults,noauto 0 0

to

ib://dn2-ib:3335/pvfs2-fs /mnt/pvfs2 pvfs2 defaults,noauto 0 0


Will you be doing a demo with this at supercomputing? Maybe we can try 
out something on the show floor later in the week..


Chen, Helen Y wrote:
> I am having trouble installing pvfs2 2.8.0 to use IB RDMA under centos  5.3 with 2.6.18-128, and hope that someone on the mailing list can offer some help.  
>
> 1. The build:  I ran ./configure --with-openib=/usr --without-bmi-tcp --with-kernel=/usr/src/kernels/2.6.18-128.el5-x86_64.
>
> 2. I  configured the meta and I/O server on a single node (dn2).  And here is the content of my  /etc/pvfs2-fs.conf:
>
> <Defaults>
>         UnexpectedRequests 50
>         EventLogging none
>         EnableTracing no.
>         LogStamp datetime
>         BMIModules bmi_ib
>         FlowModules flowproto_multiqueue
>         PerfUpdateInterval 1000
>         ServerJobBMITimeoutSecs 30
>         ServerJobFlowTimeoutSecs 30
>         ClientJobBMITimeoutSecs 300
>         ClientJobFlowTimeoutSecs 300
>         ClientRetryLimit 5
>         ClientRetryDelayMilliSecs 2000
>         PrecreateBatchSize 512
>         PrecreateLowThreshold 256
>
>         StorageSpace /pvfs2-storage-space
>         LogFile /tmp/pvfs2-server.log
> </Defaults>
>
> <Aliases>
>         Alias dn2-ib ib://dn2-ib:3335
> </Aliases>
>
> <Filesystem>
>         Name pvfs2-fs
>         ID 695113894
>         RootHandle 1048576
>         FileStuffing yes
>         <MetaHandleRanges>
>                 Range dn2-ib 3-4611686018427387904
>         </MetaHandleRanges>
>         <DataHandleRanges>
>                 Range dn2-ib 4611686018427387905-9223372036854775806
>         </DataHandleRanges>
>         <StorageHints>
>                 TroveSyncMeta yes
>                 TroveSyncData no
>                 TroveMethod alt-aio
>         </StorageHints>
> </Filesystem>
>
> 3. I built an xfs filesystem on /dev/md0 and mounted it on /pvfs2.
>
> 4. I then started the server with "/usr/local/sbin/pvfs2-server /etc/pvfs2-fs.conf -f " for the 1st time and without "-f" subsequent times.
>
> 4. On a second node (dn3, the pvfs client) I created /etc/pvfs2tab with the following line:
>
> ib://dn2-ib:3335/pvfs2 /mnt/pvfs2 pvfs2 defaults,noauto 0 0
>
> 5. Pvfs2-ping -m /mnt/pvfs2 failed.  Unfortunately I did'nt capture the erroe message in time before the nodes were grabbed by my colleague. I remembered vaguely that it complained about the exported pvfs2 filesystem not initialized. 
>
> 6. I verified that the IB RDMA network worked using rping.
>
> Please let me know if I have done something wrong and whether you need more info to troubleshoot.  Your help will be appreciated.
>
> hycsw at sandia.gov
>
>
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>