[Pvfs2-users] Unable to create shared system memory region: No space left on device

[Matthieu Dorier]

Hello,

Louis-Claude being working in my lab I just answered him in person, but I
thought the answer could be interesting for other users and more
particularly for the developers:

It seems that the script provided to stop the server simply calls the
"kill" command, but pvfs creates some shared memory objects using shmget
and these objects are not deleted. A call to ipcrm solves the problem, but
it could be good to have the script properly cleaning these shared memory
objects too.

Matthieu

2012/2/27 Louis-Claude Canon <louis-claude.canon at inria.fr>

> Hello,
>
> I am facing a "no space left" problem when configuring two servers with
> OrangeFS 2.8.5. In my experiment, I configure two OrangeFS servers, perform
> some transfers, clean the configuration and repeat the process
> indefinitely. After around 1 thousand iterations with the same two servers
> (same nodes and same configuration options), I have the following situation
> when launching the first server:
>
> $ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr/tmp/pvfs2.conf -f
> [S 02/27/2012 14:34:15] PVFS2 Server on node
> paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting...
> [D 02/27/2012 14:34:15] PVFS2 Server: storage space created. Exiting.
> $ /usr/sbin/pvfs2-server -a paradent-19.rennes.grid5000.fr /tmp/pvfs2.conf
> [S 02/27/2012 14:34:17] PVFS2 Server on node
> paradent-19.rennes.grid5000.fr version 2.8.5-orangefs starting...
> $ cd /tmp
> $ cat pvfs2-server.log
> [D 02/27/2012 14:34:17] PVFS2 Server version 2.8.5-orangefs starting.
> [E 02/27/2012 14:34:17] (null): shmget: key: 2043294211: unable to create
> shared system memory region: No space left on device
> [E 02/27/2012 14:34:17] dbpf_getdb_env(//tmp/pvfs2-**storage/53405cc2/):
> No space left on device
> [E 02/27/2012 14:34:17] Error initializing trove for filesystem pvfs2-fs
> [E 02/27/2012 14:34:17] Error: Could not initialize server subsystems
> [E 02/27/2012 14:34:17] Error: Could not initialize server; aborting.
> [E 02/27/2012 14:34:17] Trove collection not defined.
> $ df -h .
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sda5             279G  192M  265G   1% /tmp
> $ cat pvfs2.conf
> <Defaults>
>        UnexpectedRequests 50
>        EventLogging none
>        EnableTracing no
>        LogStamp datetime
>        BMIModules bmi_tcp
>        FlowModules flowproto_multiqueue
>        PerfUpdateInterval 1000
>        ServerJobBMITimeoutSecs 30
>        ServerJobFlowTimeoutSecs 30
>        ClientJobBMITimeoutSecs 300
>        ClientJobFlowTimeoutSecs 300
>        ClientRetryLimit 5
>        ClientRetryDelayMilliSecs 2000
>        PrecreateBatchSize 0,32,512,32,32,32,0
>        PrecreateLowThreshold 0,16,256,16,16,16,0
>
>        DataStorageSpace /tmp/pvfs2-storage
>        MetadataStorageSpace /tmp/pvfs2-storage
>
>        LogFile /tmp/pvfs2-server.log
> </Defaults>
>
> <Aliases>
>        Alias paradent-19.rennes.grid5000.fr tcp://paradent-19.rennes.**
> grid5000.fr:3334 <http://paradent-19.rennes.grid5000.fr:3334>
>        Alias paradent-2.rennes.grid5000.fr tcp://paradent-2.rennes.**
> grid5000.fr:3334 <http://paradent-2.rennes.grid5000.fr:3334>
> </Aliases>
>
> <Filesystem>
>        Name pvfs2-fs
>        ID 1396726978
>        RootHandle 1048576
>        FileStuffing yes
> <MetaHandleRanges>
>                Range paradent-19.rennes.grid5000.fr 3-2305843009213693953
>                Range paradent-2.rennes.grid5000.fr 2305843009213693954-**
> 4611686018427387904
> </MetaHandleRanges>
> <DataHandleRanges>
>                Range paradent-19.rennes.grid5000.fr 4611686018427387905-**
> 6917529027641081855
>                Range paradent-2.rennes.grid5000.fr 6917529027641081856-**
> 9223372036854775806
> </DataHandleRanges>
> <StorageHints>
>                TroveSyncMeta yes
>                TroveSyncData no
>                TroveMethod alt-aio
> </StorageHints>
> </Filesystem>
>
> This is not a blocking problem for me, but I would prefer to understand
> what is going on. Any ideas?
>
> Regards,
>
> Louis
> ______________________________**_________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-**underground.org<Pvfs2-users at beowulf-underground.org>
> http://www.beowulf-**underground.org/mailman/**listinfo/pvfs2-users<http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users>
>



-- 
Matthieu Dorier
ENS Cachan, Brittany (Computer Science dpt.)
IRISA Rennes, Office C113
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20120227/39aa066b/attachment.htm