[PVFS-users] u2p failure to open destination file

[David J Glaze]

Hello All,

I'm having some problems with an installation of PVFS on a group
of Gentoo Linux machines (that were all RedHat 7.3 machines about
a month ago).  Everything worked very well under RedHat with PVFS 
versions up to 1.6.0.  I'm using 1.6.2 now.

My first problem is with 'u2p', which I was using for some crude
performance tests.  Observe:

-----------------------------------------------------------------
# ls -l /scratch/athlon/test
total 102400
-rw-r--r--    1 root     root     104857600 Jan 25 17:45 100MB.1

# u2p /scratch/athlon/test/100MB.1 /scratch/athlon/test/100MB.2
pvfs_open: unable to open destfile /scratch/athlon/test/100MB.2

# cp /scratch/athlon/test/100MB.1 /scratch/athlon/test/100MB.2

# ls -l /scratch/athlon/test
total 204800
-rw-r--r--    1 root     root     104857600 Jan 25 17:45 100MB.1
-rw-r--r--    1 root     root     104857600 Jan 25 18:05 100MB.2

#
-----------------------------------------------------------------

This PVFS filesystem is mounted at /scratch/athlon, and the 
above commands were executed on the mgr node.  The 'u2p' command
works fine if the destination is not in the PVFS filesystem.  

I'm executing the above commands as root, but regular users have 
the same problem.  There are no messages in the system logs upon 
the above failure.  I did an 'strace' of the above u2p command,
but I didn't see anything obviously wrong.  For the morbidly 
curious, I have the (long) strace output here:
http://ristretto.ecn.purdue.edu/~glaze/tmp/u2p.strace

This is a fairly generic installation with 16 iod nodes and a 
separate mgr node (named 'ristretto.ecn.purdue.edu' in the strace
output), all connected through a private 100Mbps network with a 
gigabit uplink to the mgr node, which then connects to the other
cluster subnets and the rest of the world.  I can provide any 
of the various config files upon request.

All machines are using a plain-vanilla unpatched 2.4.24 kernel and
glibc 2.3.2.

Am I doing something braindead here?  Are there any nice tools
or techniques I'm unaware of to help diagnose this problem?  (Yes,
I know I can dig through the source and attack the problem with a 
debugger, but I don't have much time for that at the moment.  I was
hoping someone had seen this before.  :-)

On another note...  My second problem is that the PVFS read/write
speed seems to be about an order of magnitude slower under Gentoo 
than under RedHat.  I'm still trying to diagnose this one, so I'll
save it for another post when I know more.  (I'm actually hoping 
that the above u2p problem is somehow related to the performance 
issue.)

Thanks!

Dave