[Pvfs2-developers] readdir count and token position

[Phil Carns]

Robert Latham wrote:
> On Wed, Oct 15, 2008 at 10:42:23AM -0400, Phil Carns wrote:
>> FYI, I reduced the limit (in the kernel only, not pvfs2-ls) down to 96  
>> for the time being in cvs trunk.  This avoids the token rewind problem  
>> triggered by bonnie++, but hopefully still keeps some of the ls speed  
>> improvement.
> 
> are there workloads for which the token rewind problem would still
> show up?

Yes, its possible, even with the original 32 entries per request.  The 
problem is that the vfs is trying to fill up a buffer of a particular 
size (usually 4KB), but our protocol operates in terms of the number of 
entries without regard for total size.  Capping it at 96 means that the 
rewind problem will reappear if the average length of names in the 
directory goes over (4096/96 = 42) characters, minus a few characters 
for whatever other dirent struct fields take up space in the buffer.

Bumping this down just makes this case less likely.

I don't know how many applications besides bonnie++ have to worry about 
this case.  "rm -rf", for example, doesn't have this problem at all even 
with 512 directory entries.  I haven't looked at the code, but it must 
have better logic for deleting all of the entries in a directory.

-Phil