[Pvfs2-users] pvfs performance over IB

[Kyle Schochenmaier]

Can you try this same procedure, but do a `pvfs2-rm /mnt/pvfs2/test1G`
after unmounting the filesystem?
It appears that you tried to overwrite the original test file (from
the first tests).

See below:

On 10/11/07, hyeyoung cho <chohy at kisti.re.kr> wrote:
>
> Dear Kyle,
> Please refer the following result;
>
> ------------------------------------------------------------------------
> [root at rose1 tmp]# lsmod | grep pvfs2
> pvfs2                  98884  1
>
> [root at rose1 tmp]# ps -aux | grep pvfs2
> root      3718  0.0  0.0  4984  884 ?        Ss   16:54   0:00
> /usr/local/sbin/pvfs2-client
> root      3719  0.0  3.9 75048 40356 ?       SLl  16:54   0:00
> pvfs2-client-core -a 5 -n 5 -L /tmp/pvfs2-client.log
> root      3770  0.0  0.0  1700  468 pts/1    S+   17:00   0:00 grep pvfs2
>
> [root at rose1 tmp]# cat /etc/pvfs2tab
> ib://10.0.0.2:3335/pvfs2-fs /mnt/pvfs2 pvfs2 default,noauto 0 0
>
> [root at rose1 tmp]# mount -t pvfs2 ib://rose2:3335/pvfs2-fs /mnt/pvfs2
>
> [root at rose1 tmp]# ls -l /mnt/pvfs2
> total 16
> drwxrwxrwt  1 root root 4096 Oct 11 16:58 .
> drwxr-xr-x  5 root root 4096 Oct  9 11:42 ..
> drwxrwxrwx  1 root root 4096 Oct 11 16:58 lost+found
>
> [root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2 -t
> Wrote 1073741824 bytes in 21.799692 seconds. 46.973141 MB/seconds
>
> [root at rose1 tmp]# ls -l /mnt/pvfs2
> total 1048596
> drwxrwxrwt  1 root root       4096 Oct 11 17:01 .
> drwxr-xr-x  5 root root       4096 Oct  9 11:42 ..
> drwxrwxrwx  1 root root       4096 Oct 11 16:58 lost+found
> -rwxr-xr-x  1 root root 1073741824 Oct 11 17:01 test1G

Since you still have the filesystem mounted, you will be able to see
/mnt/pvfs2/test1G
Once you unmount it, /mnt/pvfs2 no longer points at the filesystem, so
when you `ls` on the directory it should show up empty, however, the
file still exists on the filesystem, and pvfs2-cp can and will find
the file at /mnt/pvfs2/test1G.  Try removing it with pvfs2-rm and
rerunning the test here.

>
> [root at rose1 tmp]# umount /mnt/pvfs2
> [root at rose1 tmp]# ls -l /mnt/pvfs2
> total 12
> drwxr-xr-x  2 root root 4096 Oct  5 14:47 .
> drwxr-xr-x  5 root root 4096 Oct  9 11:42 ..
>
> [root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2 -t
> Target file test1G already exists
> Could not open /mnt/pvfs2
>
> [root at rose1 tmp]# /usr/local/bin/pvfs2-cp /tmp/test1G /mnt/pvfs2/test02 -t
> -------> It was stalled!!!!!  So we terminated it with 'CTRL+C'.
> [root at rose1 tmp]# cp /tmp/test1G /mnt/pvfs2
>
> [root at rose1 tmp]# ls -l /mnt/pvfs2
> total 1049616
> drwxr-xr-x  2 root root       4096 Oct 11 17:08 .
> drwxr-xr-x  5 root root       4096 Oct  9 11:42 ..
> -rwxr-xr-x  1 root root 1073741824 Oct 11 17:08 test1G

I'm not sure how you copied this file here and were able to see it
with ls -l, it is possible that you copied this to your root
filesystem and not your pvfs2 filesystem here.  You shouldnt be able
to see this file if the client dameon is not running.
As far as the copy stalling, how long did it stall for?  Sometimes it
is exceptionally slow after a failure, though I'm sure we've worked on
that in the past.

~Kyle


> ------------------------------------------------------------------------
> -----Original Message-----
> From: Kyle Schochenmaier [mailto:kschoche at gmail.com]
> Sent: Thursday, October 11, 2007 2:42 PM
> To: hyeyoung cho
> Cc: Kevin Harms; pvfs2-users at beowulf-underground.org
> Subject: Re: [Pvfs2-users] pvfs performance over IB
>
> Quick question then:
>
> You have /mnt/pvfs2/* as your pvfs2 filesystem, is this a mounted
> filesystem through the vfs?  or is this the location of the filesystem
> without the pvfs2-client-core/vfs layer?
>
> In my observations with ib, I've noticed in that the vfs performance
> can be quite lacking.
>
> can you try unmounting the vfs and rerunning the test to make sure
> you're using libpvfs2 and not the vfs?
>
> ~Kyle
>
>
>


-- 
Kyle Schochenmaier