[Pvfs2-users] pvfs2-client-core crashes with unrecognized/unimplemented vfs operation of type ff000000

[Sam Lang]

On Mar 26, 2009, at 3:43 PM, Rene Salmon wrote:

> HI,
>
> We are getting some strange behavior out of pvfs-2.8.1 clients running
> on some sles 10 sp 1 nodes.
>
> The pvfs2 clients can mount the pvfs2 file system with no problems we
> then start an MPI job that runs on a small number of nodes.  The  
> problem
> happens when we try to kill the mpi job.  As soon as we send the kill
> signal to the mpi job several of our pvfs2 client nodes have their
> pvfs2-client-core deamon die with this message:
>
> hpcp6671:~ # ps -ef |grep pvfs
> root     25767     1  0 12:21 ?
> 00:00:00 /bphpc5/vol0/salmr0/opt/pvfs-2.8.1/x86_64/sles10sp1/sbin/ 
> pvfs2-client -p /bphpc5/vol0/salmr0/opt/pvfs-2.8.1/x86_64/sles10sp1/ 
> sbin/pvfs2-client-core
> root     16117 25767  0 15:02 ?        00:00:00 [pvfs2-client-co]
>
>
>
> hpcp6671:~ # cat /tmp/pvfs2-client.log
> [E 12:21:35.567169] PVFS Client Daemon Started.  Version 2.8.1
> [D 12:21:35.567434] [INFO]: Mapping pointer 0x2acdf7aa3000 for I/O.
> [D 12:21:35.579256] [INFO]: Mapping pointer 0x2acdf8ea5000 for I/O.
> [E 15:02:54.988860] PVFS2 client: signal 11, faulty address is 0x41d5,
> from 0x408d81
> [E 15:02:54.989282] [bt] pvfs2-client-core [0x408d81]
> [E 15:02:54.989294] [bt] pvfs2-client-core [0x408d81]
> [E 15:02:54.989302] [bt] pvfs2-client-core(main+0xbc3) [0x40a173]
> [E 15:02:54.989309] [bt] /lib64/libc.so.6(__libc_start_main+0xf4)
> [0x2acdf788b154]
> [E 15:02:54.989315] [bt] pvfs2-client-core [0x403519]
> [E 15:02:54.991351] Child process with pid 25768 was killed by an
> uncaught signal 6

Hi Rene,

This is a segfault in the client process.  The daemon is restarting  
itself, which may be what the error below is from.  I'll have to  
figure out what that 0x408d81 pointer maps to.  Might not be all that  
useful though.  Would you be willing to recompile with debugging  
enabled (rerun configure with CFLAGS=-g, and then rebuild)?  That  
would at least give us line numbers to look at.

>
> [E 15:02:54.993980] PVFS Client Daemon Started.  Version 2.8.1
> [D 15:02:54.994242] [INFO]: Mapping pointer 0x2b94619a2000 for I/O.
> [D 15:02:55.008318] [INFO]: Mapping pointer 0x2b9462da4000 for I/O.
> [E 15:02:55.312456] Got an unrecognized/unimplemented vfs operation of
> type ff000000.
> [E 15:02:55.312497] Post of op: PVFS_VFS_OP_INVALID failed!

I would try to fix the above before worrying about this one.  It could  
be just fallout from the first failure.

-sam
>
>
>
> Any ideas?
>
> thanks
> Rene
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users