[PVFS-users] Poor performance

[Jon Combe]

On 6/6/05, Nathan Poznick <kraken at wang-fu.org> wrote:
> > As an example I did "wc -l" on a file, on the PVFS file system that
> > took 8 minutes 26 seconds. I then copied the same file to a disk
> > mounted locally using the ext2 file system (/tmp) and ran the same
> > command. It ran in 4 seconds, a massive increase in speed than on
> > PVFS.
>
> The 4 second number isn't realistic however.  Since you'd just copied
> the file to local disk it was most likely fully or at least mostly
> contained in the page cache.  Thus, a 4 second number in that situation
> would be more like a scan through memory than across disk.

I re-ran the command this morning, it took 11 seconds to run, so it is
still a lot quicker than on PVFS.

> > So I tried this
> > instead
> >
> > dd if=myfile bs=16777216 | wc -l
> >
> > That ran in 2 minutes 33 seconds, so quite a big increase in
> > performance but still very very slow compared with the performance I
> > get on the native local filesystem.
>
> How big is this file?  Are there other things happening on the machine,
> network-wise at the time?

The file is 1549403622 bytes. As far as I can see (via top) nothing
else is happening on the box.

>
> > However I've been advised by support staff that I'll likely never get
> > good performance from standard Unix commands running on files stored
> > on the PVFS disks.
>
> Most of the standard tools use quite small blocksizes, when compared to
> what is optimal for PVFS.  wc for instance uses a 16k blocksize.  cp
> uses the contents of the struct stat field st_blksize on the destination
> filesystem (which for an ext2/3 filesystem is 4k).

Well thats fair enough, but are you saying the sort of performance
difference I'm seeing above is normal for PVFS?

Jon.