[Pvfs2-developers] concurrent metadata and pvfs2-lsplus improvements

[Phil Carns]

There are a couple of performance bug fixes in trunk and 2.7 branch now 
that I thought might be of interest to the list.

The first went in about two weeks ago and fixed a condition variable bug 
inside of the trove testcontext() function.  There was a race condition 
where it was possible for operations to complete without waking up the 
caller, therefore causing the function to sleep longer than it should have.

I don't know how hard it was to trigger in general, but I was able to 
see it on SMP servers with default sync coalescing and a large number of 
concurrent metadata modifications.  I was testing it with an artificial 
benchmark that emulated 10,000 concurrent creates on a server without 
network traffic, and fixing this bug improved the overall performance of 
this scenario by a factor of 16.  Hopefully some of this will translate 
into real world improvement if you have a big concurrent metadata workload.

The second bug fix just went in this week.  The problem was basically 
some leftover (and very slow) deprecated code in the trove list_attr() 
function used by readdirplus().  It showed up in the pvfs2-lsplus 
command line utility, which was supposed to be a much faster version of 
ls (particularly with -l), but actually ran slower than pvfs2-ls for 
many cases.

At any rate, this list_attr() problem is fixed now as well.  The result 
is that it sped up the pvfs2-lsplus by at least a factor of 4 for the 
cases that I was looking at with thousands of files on a single server. 
  pvfs2-lsplus is definitely a big improvement now if you need a fast 
way to list files.

-Phil