[PVFS2-users] syncing in PVFS2
dhildebz at eecs.umich.edu
Mon Oct 11 13:06:36 EDT 2004
Thanks Neill. NFS does a fsync so things should work the same. O_DIRECT
is an open option, but I'm not using it.
I don't think my previous comment about Ext3 was correct as my iozone
tests run with close and sync options (-Raec), so everything should be
sync'd to disk before the client returns.
In my tests, running iozone on the server disk gives 13MB/s writes. When
storing the PVFS2 file system on the same disk and then accessing from a
remote PVFS2 client I get 44MB/s. Am I missing something? I don't
remember if the Berkley DB is involved in data or just metadata so I don't
know if it could affect things here.
Also, as an aside, what is the new default behavior without the
TroveSyncMode option? I notice that the pvfs2_fysnc function in file.c is
empty, so I guess syncs are independant of user requests.
On Mon, 11 Oct 2004 neillm at mcs.anl.gov wrote:
> On Fri, Oct 08, 2004 at 06:28:12PM -0400, Dean Hildebrand wrote:
> > Another question, again with PVFS2 0.5.1. I have the TroveSyncMode set to
> > sync (the default), but I'm noticing that asynchronous disk flushes are
> > occuring. The PVFS2 client returns before the data is flushed to disk.
> > As is to be expected, /pvfs2-storage-space is sitting in a ext3 parition
> > which does async I/O. Is this the reason? When NFS exports a file
> Yes, that's the reason. With that (now outdated) mode enabled, we
> sync after every disk operation by issuing an fsync (or db sync) on
> any object being accessed. What the underlying file system does with
> that is file system dependent (and I see what you see for a default
> ext3 setup).
> > in sync mode, it doesn't return until all data has been flushed to
> > disk, i.e., no disk activity after the client returns. My setup is 2
> > machines with the server and metadata server on one and the client on the
> > other.
> I don't know what NFS does, but it's possible that it issues a sync,
> rather than fsync command. It's also file system dependent on what
> happens in that case. Also, NFS appears to have some O_DIRECT support
> -- perhaps that's used somehow in conjunction with their sync mode? I
> can only answer the PVFS2 related question, but if you find out for
> sure what NFS does, feel free to let us know.
> Best regards,
More information about the PVFS2-users