[Pvfs2-users] pvfs performance over IB

[hyeyoung cho]

Dear Kyle,
Please refer the following result;

------------------------------------------------------------------------
[root at rose1 tmp]# lsmod | grep pvfs2 
pvfs2                  98884  1 

[root at rose1 tmp]# ps -aux | grep pvfs2
root      3718  0.0  0.0  4984  884 ?        Ss   16:54   0:00
/usr/local/sbin/pvfs2-client
root      3719  0.0  3.9 75048 40356 ?       SLl  16:54   0:00
pvfs2-client-core -a 5 -n 5 -L /tmp/pvfs2-client.log
root      3770  0.0  0.0  1700  468 pts/1    S+   17:00   0:00 grep pvfs2

[root at rose1 tmp]# cat /etc/pvfs2tab 
ib://10.0.0.2:3335/pvfs2-fs /mnt/pvfs2 pvfs2 default,noauto 0 0

[root at rose1 tmp]# mount -t pvfs2 ib://rose2:3335/pvfs2-fs /mnt/pvfs2

[root at rose1 tmp]# ls -l /mnt/pvfs2
total 16
drwxrwxrwt  1 root root 4096 Oct 11 16:58 .
drwxr-xr-x  5 root root 4096 Oct  9 11:42 ..
drwxrwxrwx  1 root root 4096 Oct 11 16:58 lost+found

[root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2 -t
Wrote 1073741824 bytes in 21.799692 seconds. 46.973141 MB/seconds

[root at rose1 tmp]# ls -l /mnt/pvfs2
total 1048596
drwxrwxrwt  1 root root       4096 Oct 11 17:01 .
drwxr-xr-x  5 root root       4096 Oct  9 11:42 ..
drwxrwxrwx  1 root root       4096 Oct 11 16:58 lost+found
-rwxr-xr-x  1 root root 1073741824 Oct 11 17:01 test1G

[root at rose1 tmp]# umount /mnt/pvfs2
[root at rose1 tmp]# ls -l /mnt/pvfs2
total 12
drwxr-xr-x  2 root root 4096 Oct  5 14:47 .
drwxr-xr-x  5 root root 4096 Oct  9 11:42 ..

[root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2 -t
Target file test1G already exists
Could not open /mnt/pvfs2

[root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2/test02 -t
-------> It was stalled!!!!!  So we terminated it with 'CTRL+C'.
[root at rose1 tmp]# cp /tmp/test1G /mnt/pvfs2   

[root at rose1 tmp]# ls -l /mnt/pvfs2 
total 1049616
drwxr-xr-x  2 root root       4096 Oct 11 17:08 .
drwxr-xr-x  5 root root       4096 Oct  9 11:42 ..
-rwxr-xr-x  1 root root 1073741824 Oct 11 17:08 test1G 
------------------------------------------------------------------------
-----Original Message-----
From: Kyle Schochenmaier [mailto:kschoche at gmail.com] 
Sent: Thursday, October 11, 2007 2:42 PM
To: hyeyoung cho
Cc: Kevin Harms; pvfs2-users at beowulf-underground.org
Subject: Re: [Pvfs2-users] pvfs performance over IB

Quick question then:

You have /mnt/pvfs2/* as your pvfs2 filesystem, is this a mounted
filesystem through the vfs?  or is this the location of the filesystem
without the pvfs2-client-core/vfs layer?

In my observations with ib, I've noticed in that the vfs performance
can be quite lacking.

can you try unmounting the vfs and rerunning the test to make sure
you're using libpvfs2 and not the vfs?

~Kyle