[Pvfs2-users] PVFS2 miscellaneous issues.

[Murali Vilayannur]

Hi Vikrant,

> 1. pvfs2-TCP:
>     This installation seems to be generally robust. But one issues has
> been that on the  metadata-server the load average is always at a
> minimum of two.

I suspect your system/distro may have some broken threading libraries..
Is the server linked against the nptl or the old pthread libs?
(ldd src/server/pvfs2-server)?

> By this I mean that even with no process showing up as
> using CPU in "top" the load average using uptime is two. This I started
> to notice after the pvfs2 installation, so Im beginning to suspect it
> might be due to it but I cant verify it right now. And this observation
> is when there is no pvfs2 file system activity.

>     - Another problem is related to using unix "cp ". if i try to copy a
> file onto or from pvfs2 it throws the following message, but the file
> seems to be ok. This might be just some configuration issue. This
> problem is also observed for pvfs2-IB setup!
>
>     $ cp /tmp/file /mnt/pvfs2/file
>     cp: setting attribute `system.pvfs2.dh' for `file': Invalid argument
>     cp: setting attribute `system.pvfs2.md' for `file': Invalid argument

This is also a distro (bug?) with the coreutils package.
http://www.mail-archive.com/pvfs2-users@beowulf-underground.org/msg00366.html
Long story short, try using
cp --attributes=- /tmp/file /mnt/pvfs2/testfile
and see if that fixes the error?
Alternatively, you could upgrade to HEAD where we prevent system level
attributes from showing up in listxattr's output..

>
> 2. pvfs2-IB: ( using topspin IB)
>     This installation has been quite problematic to the point that it
> required restarting the pvfs2 server and client a couple of times. There
> are a host of issues. Interestingly load average is not an issue on the
> pvfs2-IB installation.
>     - pvfs2-client-core uses up 100% CPU. I have observed this once and
> other people using the file system have also observer the same. Seems to
> be IB specific only.
>     - metadata operations like "ls" at times dont return and the
> terminal has to be killed.

Is the above also IB specific?

>     - files and directories get created that cannot be deleted

This one too?

>     - I/O performance also keeps on fluctuating a lot. ( I see on pvfs2
> FAQs solution to similar problems, one using "noreservation mode on ext3
> volumes" and the other by removing pvfs2 from cron jobs, but I havent
> been able to look into it in detail)

With tcp as well?
>
> Any help in resolving these issues would be much appreciated!
Thanks,
Murali