[Pvfs2-users] Problems with intense parallel i/o via kernel VFS

[Becky Ligon]

Andrew:

When you configured OrangeFS, did you enable the readahead cache (
--enable-mmap-racache )?

Please send me your config.log from your OrangeFS build, just to verify.

The message in the pvfs2-client.log indicates that the unknown type is a
call to PVFS2_VFS_OP_MMAP_RA_FLUSH.

Becky

On Tue, May 15, 2012 at 12:20 AM, Andrew Savchenko <bircoph at gmail.com>wrote:

> Hello, Becky!
>
> On Mon, 14 May 2012 15:24:51 -0400 Becky Ligon wrote:
> > Can you run the parallel make using strace, so I can see which system
> calls
> > are made?  That will help narrow down where the problem exists.
>
> I ran
> strace -ff -o ~/pvfs2/strace make -j10 > ~/pvfs2/make.log 2>&1
>
> Actually it died on several cp processes run in parallel. And dead
> argument is a source file, not a destination.
>
> pvfs2-client outputs at the same time the folliwing to the log:
> [E 07:17:17.368239] Completed upcall of unknown type ff00000d!
> multiple times.
>
> > Please clarify:
> > AFTER the make process has completed, you can access the files using
> > pvfs2-cp but not through the kernel module?  If this is true, can you
> send
> > me the output of an "ls -al" and "pvfs2-ls -al"?
>
> Yes, this is true. With an exception after "make process has failed" :)
>
> ls -al shows:
> ?????????? ? ?       ?         ?            ? site_que_attr_def.ht
> pvfs2-ls -al shows:
> -rw-rw-r--    1 bircoph  users           4621 2010-12-07 01:44
> site_que_attr_def.ht
>
> I can copy this file using pvfs2-cp, but:
> 1) Copying works if I copy from pvfs2 to a local file system,
> destination file is normal and readable.
> 2) Copying fails if I copy this file from original directory to some
> other directory on the same pvfs2 filesystem. More correctly copying
> itself is normal, but it produces the same unreadable inaccessible
> file.
>
> All logs, strace output per process, full ls,pvfs2-ls outputs and
> pvfs2-client log file are in an attached tarball, try1 directory.
>
> And I have forgot to mention in the original e-mail that I use an
> experimental --enable-threaded-kmod-helper. I tried to disable this
> feature, but result is similar. Filesystem configuration is the same
> as in previous e-mail.
>
> I remounted filesystem, removed build tree, unpacked and configured
> it anew, remounted filesystem again, restarted pvfs2-client and reloaded
> a kernel module, then give it another try.
>
> This time I was able to move further: now offending files are in
> src/lib/Libattr, there are two of them and they were generated during a
> build, and pvfs2-client log says nothing, pvfs2-cp behaviour is the same:
> works for local destination and copies unreadable file when destination
> is on pvfs2. All logs are in try2 directory of the attached tarball.
>
> Best regards,
> Andrew Savchenko
>



-- 
Becky Ligon
OrangeFS Support and Development
Omnibond Systems
Anderson, South Carolina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20120515/cea4a97b/attachment.htm