[Pvfs2-users] Re: PVFS2 over IB RDMA

[Kyle Schochenmaier]

Helen -

Having the error message would be very helpful here so we know what issue
were looking at.
On first glance it woould appear that possibly you did not start the client
process for mounting on the client machine.  I think pvfs2-ping does look at
a mount point at some level.
I would try pvfs2-ls next time also to see if it has nothing to do with the
client process/mount.
The other thing I would check is to make sure the server processes are
actually running and haven't failed silently.  If they have failed the
server logs will indicate why.

Once you have that all setup, if you still have issues let's try to get some
logs or error messages if at all possible.

Regards,

Kyle Schochenmaier

On Oct 28, 2009 3:33 PM, "Chen, Helen Y" <hycsw at sandia.gov> wrote:

I am having trouble installing pvfs2 2.8.0 to use IB RDMA under centos  5.3
with 2.6.18-128, and hope that someone on the mailing list can offer some
help.

1. The build:  I ran ./configure --with-openib=/usr --without-bmi-tcp
--with-kernel=/usr/src/kernels/2.6.18-128.el5-x86_64.

2. I  configured the meta and I/O server on a single node (dn2).  And here
is the content of my  /etc/pvfs2-fs.conf:

<Defaults>
       UnexpectedRequests 50
       EventLogging none
       EnableTracing no.
       LogStamp datetime
       BMIModules bmi_ib
       FlowModules flowproto_multiqueue
       PerfUpdateInterval 1000
       ServerJobBMITimeoutSecs 30
       ServerJobFlowTimeoutSecs 30
       ClientJobBMITimeoutSecs 300
       ClientJobFlowTimeoutSecs 300
       ClientRetryLimit 5
       ClientRetryDelayMilliSecs 2000
       PrecreateBatchSize 512
       PrecreateLowThreshold 256

       StorageSpace /pvfs2-storage-space
       LogFile /tmp/pvfs2-server.log
</Defaults>

<Aliases>
       Alias dn2-ib ib://dn2-ib:3335
</Aliases>

<Filesystem>
       Name pvfs2-fs
       ID 695113894
       RootHandle 1048576
       FileStuffing yes
       <MetaHandleRanges>
               Range dn2-ib 3-4611686018427387904
       </MetaHandleRanges>
       <DataHandleRanges>
               Range dn2-ib 4611686018427387905-9223372036854775806
       </DataHandleRanges>
       <StorageHints>
               TroveSyncMeta yes
               TroveSyncData no
               TroveMethod alt-aio
       </StorageHints>
</Filesystem>

3. I built an xfs filesystem on /dev/md0 and mounted it on /pvfs2.

4. I then started the server with "/usr/local/sbin/pvfs2-server
/etc/pvfs2-fs.conf -f " for the 1st time and without "-f" subsequent times.

4. On a second node (dn3, the pvfs client) I created /etc/pvfs2tab with the
following line:

ib://dn2-ib:3335/pvfs2 /mnt/pvfs2 pvfs2 defaults,noauto 0 0

5. Pvfs2-ping -m /mnt/pvfs2 failed.  Unfortunately I did'nt capture the
erroe message in time before the nodes were grabbed by my colleague. I
remembered vaguely that it complained about the exported pvfs2 filesystem
not initialized.

6. I verified that the IB RDMA network worked using rping.

Please let me know if I have done something wrong and whether you need more
info to troubleshoot.  Your help will be appreciated.

hycsw at sandia.gov



_______________________________________________
Pvfs2-users mailing list
Pvfs2-users at beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20091028/368512d6/attachment.htm