[PVFS2-users] syncing in PVFS2

[neillm at mcs.anl.gov]

On Fri, Oct 08, 2004 at 06:28:12PM -0400, Dean Hildebrand wrote:
> Another question, again with PVFS2 0.5.1.  I have the TroveSyncMode set to
> sync (the default), but I'm noticing that asynchronous disk flushes are
> occuring.  The PVFS2 client returns before the data is flushed to disk.
> As is to be expected, /pvfs2-storage-space is sitting in a ext3 parition
> which does async I/O.  Is this the reason?  When NFS exports a file

Yes, that's the reason.  With that (now outdated) mode enabled, we
sync after every disk operation by issuing an fsync (or db sync) on
any object being accessed.  What the underlying file system does with
that is file system dependent (and I see what you see for a default
ext3 setup).

> in sync mode, it doesn't return until all data has been flushed to
> disk, i.e., no disk activity after the client returns.  My setup is 2
> machines with the server and metadata server on one and the client on the
> other.

I don't know what NFS does, but it's possible that it issues a sync,
rather than fsync command.  It's also file system dependent on what
happens in that case.  Also, NFS appears to have some O_DIRECT support
-- perhaps that's used somehow in conjunction with their sync mode?  I
can only answer the PVFS2 related question, but if you find out for
sure what NFS does, feel free to let us know.

Best regards,
-Neill.