[PVFS-users] large files and stability

[Janis Bardins]

Hey there ...

I'm playing with a system of 8 equal nodes serving as compute- and
io-nodes and an additional manager node. First I just want to test the
read performance by reading 4GB non-striped (using -n 1) files on the
node where they are locally resided. I did not set up the system so I
don't know where the encounterd problems are to be tracked down.

versions used: pvfs-1.5.3-1, glibc-2.2.4-29, kernel-2.4.9-31

I don't know if LFS is enabled:
u2p does not work with files larger than 2GB but if I first create a
pvfs-file of 1GB and then successively append 3 GB ls shows the right
size and copying with dd works ... at least sometimes ...

Quite often the copying (to /dev/null locally on the node) seems to hang
or at least is really slow. ls on the file fails ...

[bzcbardi@compute-0-1 bzcbardi]$ ls /mnt/pvfs/ddjs/compute-0-1/d02 
ls: /mnt/pvfs/ddjs/compute-0-1/d02: No such file or directory
[bzcbardi@compute-0-1 bzcbardi]$ ls /mnt/pvfs/ddjs/compute-0-1/
ls: /mnt/pvfs/ddjs/compute-0-1/d02: No such file or directory
[bzcbardi@compute-0-1 bzcbardi]$ ls /mnt/pvfs/ddjs/
ls: /mnt/pvfs/ddjs/d00: Broken pipe
compute-0-0  compute-0-2  compute-0-4  compute-0-6  d02  d06  d10  d14  
compute-0-1  compute-0-3  compute-0-5  compute-0-7  d04  d08  d12
[bzcbardi@compute-0-1 bzcbardi]$ls /mnt/pvfs/ddjs/compute-0-1/d02
/mnt/pvfs/ddjs/compute-0-1/d02

... and then operates again.

With smaller files everything works, but I got execution time differing
about 50 % (1GB file takes 25-55 secs). Is that just an effect of
caching or do I have to expect pvfs vary so much?

Thanks for any advise.

Janis