[Pvfs2-developers] tuning the 2.6 kernels for write performance
vilayann at mcs.anl.gov
Fri Mar 24 13:42:48 EST 2006
> We looked at this a little bit again recently, but not in the context of
> buffer cache saturation. I don't have numbers handy, but I can share
> some general impressions. It still looks like data=writeback is stil
> probably the fastest mode in general, although it isn't entirely clear
> to me what the filesystem integrity tradeoff is.
It is possible that metadata points to corrupt/stale blocks in this mode
since only metadata is journaled.
> I had high hopes for
> data=journal after reading about experiences other people had (outside
> of the PVFS2 world), but it was a bust. data=journal did better than
> the default options for berkeley db and metadata intensive access, but
> absolutely stunk for I/O throughput.
data=journal would probably make sense for random I/O workloads which
would normally cause the disk arm to move all over the disk. data=journal
makes random I/O workloads look like sequential I/O workloads since all
I/O goes in contiguous chunks to the journal (regardless of which file
or blocks they are being written to). I don't imagine this mode will be
useful for a single app streaming writes to the same file.
> All of these tests were done with default ext3 options.
data=ordered is the default mode, right?
> I am guessing it only wakes up when the timeout passes, but I don't
> really know. I tried cranking down the dirty_background_ratio in
> conjunction with reducing those *centisecs values so it would wake up
> quicker and start writing out, but it still didn't help like the
> dirty_ratio did.
So I looked the code,
It seems to be do the following,
a) to initiate background write out if the number of reclaimable pages
(dirty + unstable (used by NFS to indicate those pages on server that have not
yet the disk)) goes above the lower background_threshold level (which in
turn is calculated using the dirty_ratio and available memory)
or b) if the timer has fired (for periodic writeout using the
So it is indeed weird that those ratios did not work as expected...
More information about the Pvfs2-developers