[Pvfs2-users] An I/O test

[Kyle Schochenmaier]

It would be interesting to know what the numbers are for copying
pvfs2->local_memory  and then back, local_memory->pvfs2 and compare
that to doing pvfs2->pvfs2.
If I were wanting to further look at these results and try to discern
what the slow parts are, some important things to note are cpu
utilization on the servers and clients.

You should also ideally have a difference in cpu utilization when you
do 'cp -r'  vs. the pvfs2-cp  which doesnt go through the kernel
module.

What is the cpu load on the client, I/O, MD servers?
If you find that the MD server is getting hit heavily, you might be
able to startup a second MD server, maybe even on the same box with
multiple-cores.

I'm also assuming you are using very small files.  PVFS2 does not
particularly shine with large numbers of small files in general,
especially when compared to systems that do caching (nfs).
>From what I understand about whats going on in the servers during
these operations, these numbers kind of fall in line with what I would
have expected.

Just some random thoughts, good luck!

Kyle

On Thu, Sep 4, 2008 at 4:13 PM, hawaii <forapart at gmail.com> wrote:
> A pvfs2 I/O test
> (for comments)
> ===================
>
> pvfs version: 2.7.1
> configuration: 1 meta server + 3 data servre. Each server has 2 cpus
> reserved
>
> Tests: within pvfs2; from pvfs2 to local; nfs
> Use pvfs2-cp-r, pvfs2-cp-r.pl, and cp
> (thanks Mark Bartelt for providing pvfs2-cp-r and pvfs2-cp-r.pl)
>
> Source direcotry: WRF
> df -k WRF
>   lots of subdirctories
> 511868  WRF
>
> (1) pvfs2 -> pvfs2
>
> time pvfs2-cp-r WRF tmp
> real    4m16.414s
> user    0m38.385s
> sys     0m31.874s
> I/O 2 MB/s
>
> time pvfs2-cp-r.pl WRF tmp
> Copying files from "WRF" to "tmp/WRF" using pvfs2-cp
> real    4m21.573s
> user    0m38.741s
> sys     0m32.293s
> I/O 1.96 MB/s
>
> time cp -rp WRF tmp
> Lots of 0 files (unreliable)
> real    7m21.473s
> user    0m0.100s
> sys     0m5.792s
> I/O 1.16 MB/s
>
> (2) pvfs2 -> local hard drive
>
> time pvfs2-cp-r.pl /pvfs2-dir/WRF /home/user
> Copying files from "/pvfs2-dir/WRF" to "/home/user/WRF" using pvfs2-cp
> real    1m19.311s
> user    0m19.008s
> sys     0m29.327s
> I/O     6.4 MB/s
>
> time pvfs2-cp-r /pvfs2-dir/WRF /home/user
> real    1m10.739s
> user    0m20.609s
> sys     0m30.021s
> I/O     7.25 MB/s
>
> time cp -rp /pvfs2-dir/WRF /home/user
> Lots of 0 files (unreliable)
> real    1m34.071s
> user    0m0.113s
> sys     0m3.906s
> I/O     5.45 MB/s
>
> (3) cp an nfs file to local disk
> 2057207616 t.dat
> time cp -p /nfs-dir/t.dat /home/usr/.
> real    2m55.894s
> user    0m0.257s
> sys     0m6.946s
> Read 11.7 MB/s
>
> Conclusion from this test:
> (1) pvfs2-cp-r is the fastest and cp -r is the slowest
> (2) it is slower to copy directory inside pvfs2 than to local disk
> (3) I/O pvfs2 is much slower than nfs !!!
> Any comments on the results?
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
>



-- 
Kyle Schochenmaier