[Pvfs2-users] problem with pvfs2 over diskless cluster

Rob Ross rross at mcs.anl.gov
Thu Oct 12 08:55:55 EDT 2006


Hi Wenbin,

Thanks for the detailed report. How did you end up fixing the "cannot 
find db.h" compile errors?

This "DB_THREAD mandates memory allocation flag on DBT data" seems to 
indicate that the semantics have changed in Berkeley DB, and they're 
requiring a certain way of returning data that wasn't required before 
(probably for thread safety)? That's irritating.

The "User memory too small for return value" looks like a bug.

The "Key/data pair already exists" might be normal.

Rob

Wenbin wrote:
> Thank you, Murali, for your timely response. While I tried your patch, 
> and paid attention to the output of the make process, I found several 
> places complained about cannot find 'db.h', including 
> src/io/trove/trove-dbpf/
> as well as src/apps/admin/. I fixed that, without applying your patch, 
> since you said it is wrong, I have following results:
> 
> [root at phoenix pvfs2]pvfs2-ping -m /mnt/pvfs2/    still works cleanly.
> [root at phoenix pvfs2]pvfs2-ls /mnt/pvfs2/    produces following in the 
> server log:
> 
> [E 10/12 03:00] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:00] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> [E 10/12 03:00] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:00] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> 
> [root at phoenix pvfs2]/usr/local/bin/pvfs2-cp 
> ~wenbin/AMRs/pvfs/mpi-io-test.c /mnt/pvfs2/
> PVFS_sys_create: File exists
> Could not open /mnt/pvfs2/
> 
> server log shows:
> [E 10/12 03:04] TROVE:DBPF:Berkeley DB: keyval_db->get: DB_BUFFER_SMALL: 
> User memory too small for return value
> [E 10/12 03:04] TROVE:DBPF:Berkeley DB: keyval_db->put: DB_KEYEXIST: 
> Key/data pair already exists
> 
> However, file successfully copied to /mnt/pvfs2/
> [root at phoenix pvfs2]# /usr/local/bin/pvfs2-ls /mnt/pvfs2/ vimrc
> backup.sh
> lost+found
> mpi-io-test.c
> 
> Now server log says:
> [E 10/12 03:07] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:07] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> [E 10/12 03:07] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:07] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> 
> [root at phoenix pvfs2]# /usr/local/bin/pvfs2-ls -l /mnt/pvfs2/
> Failed to get attributes on handle 1048574,1906098859
> Getattr failure: No such file or directory
> Failed to get attributes on handle 1048573,1906098859
> Getattr failure: No such file or directory
> drwxrwxrwx    1 root     root            4096 2006-10-11 12:36 lost+found
> Failed to get attributes on handle 1048574,1906098859
> Getattr failure: No such file or directory
> 
> The server log says:
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory 
> allocation flag on DBT data
> [E 10/12 03:14] TROVE:DBPF:Berkeley DB: keyval_db->get: Invalid argument
> 
> Good thing is that the infinite loop is gone. Is now the situation 
> better? Where could the problem be?
> 
> Thanks,
> 
> wenbin.
> 
> On Thu, 12 Oct 2006, Murali Vilayannur wrote:
> 
>> Hi Wenbin,
>>> Hello, I tried to install pvfs2, version 1.5.1 on a diskless cluster. 
>>> The head node is to be both the metadata server as well as the io 
>>> server. The clients are nfs mounted to the head node. The underlying 
>>> database is the Berkeley 4.5.20. The check of "pvfs2-ping -m 
>>> /mnt/pvfs2/" is allright. However, when I tried to do "pvfs2-cp" or 
>>> "pvfs2-ls", the commands hang. The /tmp/pvfs2-server.log shows 
>>> following, only 2 lines cut out of many:
>>>
>>> [E 10/11 14:45] TROVE:DBPF:Berkeley DB: keyval_db->get: 
>>> DB_BUFFER_SMALL: User memory too small for return value
>>> [E 10/11 14:45] TROVE:DBPF:Berkeley DB: keyval_db->get: 
>>> DB_BUFFER_SMALL: User memory too small for return value
>>> ....
>>>
>>> The database is configured with:
>>> ../dist/configure --enable-pthread_api --disable-libtool-lock 
>>> --enable-debug
>>>
>>> the pvfs2-1.5.1 is configured with:
>>> ./configure -with-db=/usr/local/BerkeleyDB.4.5
>> I dont know how/what is the right way to fix this problem.
>> Sam will probably find the right fixes for this. It does look like a 
>> bdb 4.5 specific issue.
>>
>> Attached patch somehow  fixes it for me although it seems (and 
>> probably is) obviously incorrect. Also, it does look like the server 
>> goes off in an infinite loop once it hits this error, but that is a 
>> separate issue I think..
>> thanks,
>> Murali
>>
>>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
> 


More information about the Pvfs2-users mailing list