[PVFS-users] Poor performance

[Jon Combe]

At work I have to use a Linux system that uses PVFS for most of it's
storage but the performance I get from standard Unix commands such as
wc, sort or diff is dreadful. Typically such commands will use less
than 5% CPU time, with pvfsd using a similar amount and the box
otherwise idle.

As an example I did "wc -l" on a file, on the PVFS file system that
took 8 minutes 26 seconds. I then copied the same file to a disk
mounted locally using the ext2 file system (/tmp) and ran the same
command. It ran in 4 seconds, a massive increase in speed than on
PVFS.

It has been suggested by support staff that this is because PVFS is
better at reading in large blocks and commands such as those I've
listed typically request small blocks of data. They suggested solution
was to pipe through dd using a large block size. So I tried this
instead

dd if=myfile bs=16777216 | wc -l

That ran in 2 minutes 33 seconds, so quite a big increase in
performance but still very very slow compared with the performance I
get on the native local filesystem.

However I've been advised by support staff that I'll likely never get
good performance from standard Unix commands running on files stored
on the PVFS disks.

I assume this may well be the case but should I really expect to see
commands take such a massive amount longer when running on PVFS? Can
anyone suggest any configuration options that might help speed this
up?

I believe the data is being accessed via Kernel mode since I see pvfsd
using CPU time when running the commands.

Thanks.
Jon.