[Pvfs2-users] PVFS2 Performance

[Nasr Y.M.J.O.]

Thanks Dean. Well, my /etc/exports file has this entry:

/export 10.0.0.0/255.0.0.0(rw,async)

and the pvfs2-fs.conf file has these entries:

<StorageHints>
             TroveSyncMeta  Yes
             TroveSyncData   No
</StorageHints>

So both nfs and pvfs2 are set to async. But the meta is set to sync, it may 
be the reason for this. Hmm... Still not clear! Maybe I need to set Meta to 
'no' even though its not advisable.

Thanks
nasr

>From: Dean Hildebrand <dhildebz at eecs.umich.edu>
>To: "Nasr Y.M.J.O." <nasr974 at hotmail.com>
>CC: vilayann at mcs.anl.gov, pvfs2-users at beowulf-underground.org
>Subject: Re: [Pvfs2-users] PVFS2 Performance
>Date: Wed, 20 Sep 2006 13:03:29 -0400
>
>
>>NFS:
>>-----
>>
>>Access size per process = 67108864 bytes, ntimes = 5
>>Write bandwidth without file sync = 70.134351 Mbytes/sec
>>Write bandwidth including file sync = 71.789752 Mbytes/sec
>>
>>PVFS2
>>--------
>>Access size per process = 67108864 bytes, ntimes = 5
>>Write bandwidth without file sync = 329.618503 Mbytes/sec
>>Write bandwidth including file sync = 121.779536 Mbytes/sec
>>
>>I did the test after tuning TCP using netpipe benchmark. I can see there 
>>is a big gap between write and read with sync in pvfs2 as compared to NFS 
>>(121, 320 compared to 71, 80). Why?
>I'll take a stab at this.  In order for NFS to maintain open-to-close 
>semantics, all data is flushed to stable storage at file close.  Issuing a 
>fsync just before file close does not do anything extra.  So your NFS 
>experiments are always doing the same thing.  I would guess PVFS2 does not 
>sync data to stable storage unless it is specifically requested.
>
>If you want to try removing this behavior from NFS, simply change your 
>/etc/exports file to export data asynchronously.  Once this is done, your 
>server may or may not flush data to stable storage.  Here is the relevant 
>part from the exports man page:
>       async  This  option  allows  the NFS server to violate the NFS 
>protocol
>              and reply to requests before any changes made  by  that  
>request
>              have been committed to stable storage (e.g. disc drive).
>
>              Using  this option usually improves performance, but at the 
>cost
>              that an unclean server restart (i.e. a crash) can cause data  
>to
>              be lost or corrupted.
>
>              In  releases  of nfs-utils upto and including 1.0.0, this 
>option
>              was the default.  In this  and  future  releases,  sync  is  
>the
>              default,  and  async  must  be explicit requested if needed.  
>To
>              help make system adminstrators aware of this change,  
>âexportfsâ
>              will issue a warning if neither sync nor async is specified.
>
>
>
>Dean
>>
>>Many thanks,
>>nasr
>>
>>---------------------------------------------------------------------------------------------------------------------------------------
>>
>>io.c program error
>>
>>
>># /opt/mpich1/intel/bin/mpicc io.c -g -Wall -o io
>>
>>io.c(49): error: identifier "O_LARGEFILE" is undefined
>>  int     amode         = O_RDWR | O_CREAT | O_LARGEFILE;
>>                                                                ^
>>
>>io.c(53): remark #1419: external declaration in primary source file
>>  int parse_args(int argc, char **argv);
>>      ^
>>
>>io.c(54): remark #1419: external declaration in primary source file
>>  double Wtime(void);
>>            ^
>>
>>io.c(57): remark #1419: external declaration in primary source file
>>  extern char *optarg;
>>                    ^
>>
>>io.c(58): remark #1419: external declaration in primary source file
>>  extern int optind, opterr;
>>                ^
>>
>>io.c(58): remark #1419: external declaration in primary source file
>>  extern int optind, opterr;
>>                           ^
>>
>>compilation aborted for io.c (code 2)
>>
>>-------------------------------------------------------------------------------------------------------------------------------------------
>>
>>
>>_________________________________________________________________
>>Find love online with MSN Personals. 
>>http://match.msn.com.my/match/mt.cfm?pg=channel
>>
>>_______________________________________________
>>Pvfs2-users mailing list
>>Pvfs2-users at beowulf-underground.org
>>http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
>--
>Dean Hildebrand
>Ph.D. Candidate
>University of Michigan
>

_________________________________________________________________
Find love online with MSN Personals. 
http://match.msn.com.my/match/mt.cfm?pg=channel