[Pvfs2-users] PVFS2 Performance

[Dean Hildebrand]

> NFS:
> -----
>
> Access size per process = 67108864 bytes, ntimes = 5
> Write bandwidth without file sync = 70.134351 Mbytes/sec
> Write bandwidth including file sync = 71.789752 Mbytes/sec
>
> PVFS2
> --------
> Access size per process = 67108864 bytes, ntimes = 5
> Write bandwidth without file sync = 329.618503 Mbytes/sec
> Write bandwidth including file sync = 121.779536 Mbytes/sec
>
> I did the test after tuning TCP using netpipe benchmark. I can see 
> there is a big gap between write and read with sync in pvfs2 as 
> compared to NFS (121, 320 compared to 71, 80). Why?
I'll take a stab at this.  In order for NFS to maintain open-to-close 
semantics, all data is flushed to stable storage at file close.  Issuing 
a fsync just before file close does not do anything extra.  So your NFS 
experiments are always doing the same thing.  I would guess PVFS2 does 
not sync data to stable storage unless it is specifically requested.

If you want to try removing this behavior from NFS, simply change your 
/etc/exports file to export data asynchronously.  Once this is done, 
your server may or may not flush data to stable storage.  Here is the 
relevant part from the exports man page:
       async  This  option  allows  the NFS server to violate the NFS 
protocol
              and reply to requests before any changes made  by  that  
request
              have been committed to stable storage (e.g. disc drive).

              Using  this option usually improves performance, but at 
the cost
              that an unclean server restart (i.e. a crash) can cause 
data  to
              be lost or corrupted.

              In  releases  of nfs-utils upto and including 1.0.0, this 
option
              was the default.  In this  and  future  releases,  sync  
is  the
              default,  and  async  must  be explicit requested if 
needed.  To
              help make system adminstrators aware of this change,  
âexportfsâ
              will issue a warning if neither sync nor async is specified.



Dean
>
> Many thanks,
> nasr
>
> --------------------------------------------------------------------------------------------------------------------------------------- 
>
> io.c program error
>
>
> # /opt/mpich1/intel/bin/mpicc io.c -g -Wall -o io
>
> io.c(49): error: identifier "O_LARGEFILE" is undefined
>  int     amode         = O_RDWR | O_CREAT | O_LARGEFILE;
>                                                                ^
>
> io.c(53): remark #1419: external declaration in primary source file
>  int parse_args(int argc, char **argv);
>      ^
>
> io.c(54): remark #1419: external declaration in primary source file
>  double Wtime(void);
>            ^
>
> io.c(57): remark #1419: external declaration in primary source file
>  extern char *optarg;
>                    ^
>
> io.c(58): remark #1419: external declaration in primary source file
>  extern int optind, opterr;
>                ^
>
> io.c(58): remark #1419: external declaration in primary source file
>  extern int optind, opterr;
>                           ^
>
> compilation aborted for io.c (code 2)
>
> ------------------------------------------------------------------------------------------------------------------------------------------- 
>
>
> _________________________________________________________________
> Find love online with MSN Personals. 
> http://match.msn.com.my/match/mt.cfm?pg=channel
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

-- 
Dean Hildebrand
Ph.D. Candidate
University of Michigan