[Pvfs2-users] error is setting up storage space for PVFS2 1.5.1

[Sam Lang]

On Jul 13, 2006, at 9:49 AM, Phil Carns wrote:

> I haven't followed this thread that closely, but if the problem is  
> shared memory support in the db library, then you could try setting  
> this config file option:
>
> <StorageHints>
> ...
> 	DBCacheType mmap
> ...
> </StorageHints>
>
>
> That will make berkeley db use mmap for the cache rather than  
> shared memory.  If the error code is well defined then it should be  
> possible to insert some code to have it fall back to mmap if the  
> shmem options fail,

Good idea Phil, I forgot that we added that option.  Unfortunately, I  
think RobL added the backtrack and retry with mmap code to 1.5.0.  I  
think the only way to fix this for db's configured in this way is to  
add another failure check for EINVAL and try to open the db env with  
DB_PRIVATE instead.  I'm not sure what the repercussions of that will  
be, probably a performance hit at the least. I looked through the  
berkeley db source for this error and found this:

         /*
          * Currently we support one kind of mutex that is intra- 
process only,
          * POSIX 1003.1 pthreads, because a variety of systems don't  
support
          * the full pthreads API, and our only alternative is test- 
and-set.
          */
#ifdef HAVE_MUTEX_THREAD_ONLY
         if (!LF_ISSET(DB_PRIVATE)) {
                 __db_err(dbenv,
     "Berkeley DB library configured to support only DB_PRIVATE  
environments");
                 return (EINVAL);
         }
#endif

I guess even with the mmap option, the db env uses MAP_SHARED, so it  
can't use that either.  Its a little surprising to me that a recent  
linux distro would release berkeley db binaries configured this way.   
Perhaps centos db packages need a bit of upgrading.

-sam


> or maybe just print a descriptive error message suggesting that the  
> user try that option.
>
> I happen to have a centos 4 test box here, but I haven't seen this  
> problem before.  Here are the kernel and db versions that it is using:
>
> $ uname -a
> Linux centos 2.6.9-34.0.1.EL #1 Wed May 24 07:40:56 CDT 2006 i686  
> i686 i386 GNU/Linux
>
> $ rpm -qa |grep db4
> db4-devel-4.2.52-7.1
> db4-4.2.52-7.1
> db4-utils-4.2.52-7.1
>
> -Phil
>
> Sam Lang wrote:
>> Hi Greg,
>> The path of least resistance for you is to probably build your  
>> own  db4 and point pvfs2 at that with the --with-db to configure.   
>> The db  you have installed seems to have been configured with some  
>> options  that disable using shared memory for the cache.
>> In the long term we should probably look at handling the case  
>> where  the db isn't configured to work with shared memory.
>> -sam
>> On Jul 11, 2006, at 7:04 PM, Greg Bruno wrote:
>>> On 7/11/06, Robert Latham <robl at mcs.anl.gov> wrote:
>>>
>>>> On Tue, Jul 11, 2006 at 09:13:12AM -0700, Greg Bruno wrote:
>>>> > ./configure --prefix=/opt/pvfs2 --with-kernel=/usr/src/linux-2.6
>>>> > make
>>>> > make install
>>>>
>>>> Can you tell us what kernel you are running?  It sounds like  
>>>> you  might
>>>> be running a stock kernel.org kernel?
>>>
>>>
>>> the node has pre-compiled kernel downloaded from a centos mirror:
>>>
>>> # uname -a
>>> Linux compute-0-0.local 2.6.9-34.0.1.ELsmp #1 SMP Wed May 24  
>>> 08:14:29
>>> CDT 2006 i686 i686 i386 GNU/Linux
>>>
>>>
>>> # rpm -qi kernel-smp
>>> Name        : kernel-smp                   Relocations: (not   
>>> relocatable)
>>> Version     : 2.6.9                             Vendor: CentOS
>>> Release     : 34.0.1.EL                     Build Date: Wed 24 May
>>> 2006 05:22:43 AM PDT
>>> Install Date: Mon 10 Jul 2006 09:31:34 AM PDT      Build Host:   
>>> build-i386
>>> Group       : System Environment/Kernel     Source RPM:
>>> kernel-2.6.9-34.0.1.EL.src.rpm
>>> Size        : 28928141                         License: GPLv2
>>> Signature   : DSA/SHA1, Wed 24 May 2006 02:17:52 PM PDT, Key ID   
>>> a53d0bab443e1821
>>> Packager    : Johnny Hughes <johnny at centos.org>
>>> Summary     : The Linux kernel compiled for SMP machines.
>>> Description :
>>> This package includes a SMP version of the Linux kernel. It is
>>> required only on machines with two or more CPUs as well as  
>>> machines  with
>>> hyperthreading technology.
>>>
>>> Install the kernel-smp package if your machine uses two or more  
>>> CPUs.
>>>
>>>
>>> - gb
>>> _______________________________________________
>>> Pvfs2-users mailing list
>>> Pvfs2-users at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>
>> _______________________________________________
>> Pvfs2-users mailing list
>> Pvfs2-users at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>