[Pvfs2-users] I/O server won't start

[Eric J. Walter]

Hi,

I have a problem starting up an I/O node.  It is one of 3 servers that
we run v2.8.1 on
over Inifiniband.  It is not used for metadata.   After a finding a file
which
had '?--?--?' like permissions, I decided to restart the pvfs servers
and remount all
of the clients.  Now, one of the three I/O nodes can't start it's
pvfs2-server.
The other two start correctly.

Here is the server log from the problem server:

[D 02/08 19:40] PVFS2 Server version 2.8.1 starting.
[E 02/08 19:40] dbpf_dspace_iterate_handles_op_svc: Invalid argument
[E 02/08 19:40] Error adding handle range
1537228672809129303-3074457345618258602,6148914691236517203-7686143364045646502
to filesystem pvfs2-fs
[E 02/08 19:40] Error: Could not initialize server interfaces; aborting.
[E 02/08 19:40] Error: Could not initialize server; aborting.

I am also using db4-4.2.52-7.1 of the DB software.  Reading through the
previous
mailing lists discussions, I found that running db_recover on the .db
files (after backing them up) could be helpful.  The only .db file which
has any problems with verify is
dataspace_attributes.db on the problem I/O node.  Here is what it reports:

>># db_verify -o dataspace_attributes.db
db_verify: Page 865: item 57 of unrecognizable type
db_verify: Page 865: gap between items at offset 1376
db_verify: Page 865: item order check unsafe: skipping
db_verify: DB->verify: dataspace_attributes.db: DB_VERIFY_BAD: Database
verification failed

So I tried db_recover -v in the same directory and in the directory
above (I am not sure where to run it) and all I get is:

db_recover: Finding last valid log LSN: file: 1 offset 28

and a small binary file named "log.0000000001".

This step seems to do nothing, i.e. the db_verify report doesn't change
after this.

I have also tried db_dump -r followed by db_load and this also does not
change the
db_verify output.

Is there anything else I can do except wipe the filesystem and rebuild?

Thanks for any help I can get.

Eric J. Walter
Department of Physics
College of William and Mary