[Pvfs2-users] problem with pvfs2 over diskless cluster

[Wenbin]

Thank you, Murali, for your timely response. While I tried your patch, and 
paid attention to the output of the make process, I found several places 
complained about cannot find 'db.h', including src/io/trove/trove-dbpf/
as well as src/apps/admin/. I fixed that, without applying your patch, 
since you said it is wrong, I have following results:

[root at phoenix pvfs2]pvfs2-ping -m /mnt/pvfs2/	still works cleanly.
[root at phoenix pvfs2]pvfs2-ls /mnt/pvfs2/	produces following in the 
server log:

[E 10/12 03:00] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:00] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
[E 10/12 03:00] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:00] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument

[root at phoenix pvfs2]/usr/local/bin/pvfs2-cp 
~wenbin/AMRs/pvfs/mpi-io-test.c /mnt/pvfs2/
PVFS_sys_create: File exists
Could not open /mnt/pvfs2/

server log shows:
[E 10/12 03:04] TROVE:DBPF:Berkeley DB: keyval_db->get: DB_BUFFER_SMALL: 
User memory too small for return value
[E 10/12 03:04] TROVE:DBPF:Berkeley DB: keyval_db->put: DB_KEYEXIST: 
Key/data pair already exists

However, file successfully copied to /mnt/pvfs2/
[root at phoenix pvfs2]# /usr/local/bin/pvfs2-ls /mnt/pvfs2/ 
vimrc
backup.sh
lost+found
mpi-io-test.c

Now server log says:
[E 10/12 03:07] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:07] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
[E 10/12 03:07] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:07] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument

[root at phoenix pvfs2]# /usr/local/bin/pvfs2-ls -l /mnt/pvfs2/
Failed to get attributes on handle 1048574,1906098859
Getattr failure: No such file or directory
Failed to get attributes on handle 1048573,1906098859
Getattr failure: No such file or directory
drwxrwxrwx    1 root     root            4096 2006-10-11 12:36 lost+found
Failed to get attributes on handle 1048574,1906098859
Getattr failure: No such file or directory

The server log says:
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
allocation flag on DBT data
[E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument

Good thing is that the infinite loop is gone. Is now the situation better? 
Where could the problem be?

Thanks,

wenbin.

On Thu, 12 Oct 2006, Murali Vilayannur wrote:

> Hi Wenbin,
>> Hello, I tried to install pvfs2, version 1.5.1 on a diskless cluster. The 
>> head node is to be both the metadata server as well as the io server. The 
>> clients are nfs mounted to the head node. The underlying database is the 
>> Berkeley 4.5.20. The check of "pvfs2-ping -m /mnt/pvfs2/" is allright. 
>> However, when I tried to do "pvfs2-cp" or "pvfs2-ls", the commands hang. 
>> The /tmp/pvfs2-server.log shows following, only 2 lines cut out of many:
>> 
>> [E 10/11 14:45] TROVE:DBPF:Berkeley DB: keyval_db->get: DB_BUFFER_SMALL: 
>> User memory too small for return value
>> [E 10/11 14:45] TROVE:DBPF:Berkeley DB: keyval_db->get: DB_BUFFER_SMALL: 
>> User memory too small for return value
>> ....
>> 
>> The database is configured with:
>> ../dist/configure --enable-pthread_api --disable-libtool-lock 
>> --enable-debug
>> 
>> the pvfs2-1.5.1 is configured with:
>> ./configure -with-db=/usr/local/BerkeleyDB.4.5
> I dont know how/what is the right way to fix this problem.
> Sam will probably find the right fixes for this. It does look like a bdb 4.5 
> specific issue.
>
> Attached patch somehow  fixes it for me although it seems (and probably is) 
> obviously incorrect. Also, it does look like the server goes off in an 
> infinite loop once it hits this error, but that is a separate issue I think..
> thanks,
> Murali
>
>