[PVFS2-users] syncing in PVFS2

[Dean Hildebrand]

Thanks Neill.  NFS does a fsync so things should work the same.  O_DIRECT
is an open option, but I'm not using it.

I don't think my previous comment about Ext3 was correct as my iozone
tests run with close and sync options (-Raec), so everything should be
sync'd to disk before the client returns.

In my tests, running iozone on the server disk gives 13MB/s writes.  When
storing the PVFS2 file system on the same disk and then accessing from a
remote PVFS2 client I get 44MB/s.  Am I missing something?  I don't
remember if the Berkley DB is involved in data or just metadata so I don't
know if it could affect things here.

Also, as an aside, what is the new default behavior without the
TroveSyncMode option?  I notice that the pvfs2_fysnc function in file.c is
empty, so I guess syncs are independant of user requests.

Thanks,
Dean

On Mon, 11 Oct 2004 neillm at mcs.anl.gov wrote:

> On Fri, Oct 08, 2004 at 06:28:12PM -0400, Dean Hildebrand wrote:
> > Another question, again with PVFS2 0.5.1.  I have the TroveSyncMode set to
> > sync (the default), but I'm noticing that asynchronous disk flushes are
> > occuring.  The PVFS2 client returns before the data is flushed to disk.
> > As is to be expected, /pvfs2-storage-space is sitting in a ext3 parition
> > which does async I/O.  Is this the reason?  When NFS exports a file
>
> Yes, that's the reason.  With that (now outdated) mode enabled, we
> sync after every disk operation by issuing an fsync (or db sync) on
> any object being accessed.  What the underlying file system does with
> that is file system dependent (and I see what you see for a default
> ext3 setup).
>
> > in sync mode, it doesn't return until all data has been flushed to
> > disk, i.e., no disk activity after the client returns.  My setup is 2
> > machines with the server and metadata server on one and the client on the
> > other.
>
> I don't know what NFS does, but it's possible that it issues a sync,
> rather than fsync command.  It's also file system dependent on what
> happens in that case.  Also, NFS appears to have some O_DIRECT support
> -- perhaps that's used somehow in conjunction with their sync mode?  I
> can only answer the PVFS2 related question, but if you find out for
> sure what NFS does, feel free to let us know.
>
> Best regards,
> -Neill.
>