[Pvfs2-developers] Listing performance patch

[Sam Lang]

Hi Bart,

Thanks for the patch.  For users with that many files in a directory,  
using pvfs2-ls is probably a good alternative.

The kernel does readdir requests 32 entries at a time, so increasing  
MAX_NUM_DIRENTS won't help for ls.  Long listings requires getting the  
size of files, which in PVFS is fairly expensive.

Unfortunately, we haven't kept up with the readdirplus implementation,  
some bugs have probably crept in since Murali added that tool.  If you  
were motivated to look at where the servers were crashing, we'd  
certainly be interested in helping with the debugging there.

Thanks again,
-sam


On Sep 3, 2008, at 1:46 PM, Bart Taylor wrote:

> Hey guys,
>
> Attached is a small patch that increases ls and pvfs2-ls  
> performance. It increases
> some #defines from 32/64 to 512.
>
> I did not see any change for small directories, but listing speeds  
> were increased
> up to 26% for large directories (100,000+ files). Long listing  
> performance with
> ls and pvfs2-ls seem to be generally unchanged. The file system used  
> is a 5 node
> SAN setup with HA running 271.
>
> Also attached are some specifics of the performance we saw.
>
> I did not include an increase for pvfs2-lsplus. Increasing its  
> MAX_NUM_DIRENTS
> value resulted in crashing all of the file system servers a couple  
> of times.
>
> Bart.
> <DIRENT-Results.xls><dirent- 
> increase.patch>_______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers