[Pvfs2-users] Problems with intense parallel i/o via kernel VFS
ligon at omnibond.com
Thu May 24 14:33:04 EDT 2012
Just an update: I have created a VM with linux 3.3.6. Still working on
the OrangeFs installation.
I thought I'd try the latest linux to see if I still get the problem.
On Thu, May 17, 2012 at 5:32 PM, Becky Ligon <ligon at omnibond.com> wrote:
> Thanks so much for running the tests. Let me take a look at the debug
> output and then let you know if more debugging needs to be turned on. In
> addition, I need to stand up a VM with your version of the kernel and
> Emmanuel's changes to see if the code path has changed.
> I'll get back in touch with you when I have something to report.
> Thanks again,
> On Thu, May 17, 2012 at 4:01 PM, Andrew Savchenko <bircoph at gmail.com>wrote:
>> I found that pvfs2fuse client works without this problems, I was able
>> to run make -j10 successfully, so problem is definitely somewhere in
>> the kernel module.
>> For tests I changed my OFS configuration to be close to yours (see
>> attached new.config.log), but please note there is one significant
>> difference: I use a patch for 3.2 kernels by Emmanuel Florac:
>> otherwise I wouldn't be able to use kernel module at all.
>> Problem indeed doesn't depend on server configuration: I tried with
>> 15 data+meta and with 3 data+meta serves. Also this is not a compiler
>> issue, I tried a build with disabled CFLAGS and LDFLAGS, problem was
>> not affected.
>> On Thu, 17 May 2012 12:20:50 -0400 Becky Ligon wrote:
>> > So, if you remount the filesystem, you don't get "?" when you "ls"?
>> > If this is the case, then I suspect that OrangeFS isn't handling an
>> > condition correctly, resulting in incorrect inode information. When you
>> > run your tests, if you will turn on OrangeFS kernel debugging on the
>> > client, I may see where the problem is happening.
>> > To turn on the appropriate kernel debug flags while the system is
>> > echo "file" > /proc/sys/pvfs2/kernel-debug
>> > The kernel debug messages will appear in your /var/log/messages file (or
>> > wherever your system messages go on your system).
>> > I would only need to see the messages from a test that failed.
>> I enabled file debug and ran two tests, results are attached. Aside
>> from kernel log I added make.log (to see failed files) and strace for
>> processes accessing failed files.
>> From test1 it is clearly seen that kernel.log has no messages about
>> failed file itself and ends just before corruption. Each kernel.log
>> ends with debug mask change message to be sure buffers were synced.
>> Maybe I should enable another debug level?
>> Best regards,
>> Andrew Savchenko
> Becky Ligon
> OrangeFS Support and Development
> Omnibond Systems
> Anderson, South Carolina
OrangeFS Support and Development
Anderson, South Carolina
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pvfs2-users