[Pvfs2-developers] nfs reexport problem in trunk

[Phil Carns]

I think the nfs reexporting is working again now in trunk, but I had to 
make another minor dcache change.

The new pvfs d_delete() function (needed to get the simul stat cases 
right) was always returning 1, which means that pvfs was always 
encouraging the vfs to uncache our dentries (and therefore inodes) when 
it had a safe opportunity.

The kernel nfs daemon however, drops references to inodes sometimes and 
expects to be able to retrieve them later just using the inode number. 
This doesn't work in PVFS if we no longer have the inode cached, because 
an inode number by itself isn't enough to reconstruct the PVFS handle 
(the inode is only 32 bits; see iget5 etc.).  The normal solution that 
many file systems use is to override the default export ops so that you 
can tell knfsd to use a better key than the inode number to reacquire 
inodes when it needs them.  Long story, but I believe this solution 
would be somewhat hard to implement in PVFS.

I instead adjusted the d_delete() function so that it only returns 1 
when the inode is bad.  I tweaked the revalidate inode function to make 
certain it marks the inode bad on failure.  This should keep the simul 
test case working but keep the vfs from uncaching trustworthy PVFS 
dentries/inodes that knfsd might want to use again.  They entries will 
still be revalidated when the time comes.

This isn't ideal, but as far as I can tell it gets us back to the same 
nfs re-exporting behavior we had in 2.7.0.

-Phil

Robert Latham wrote:
> On Fri, Jan 04, 2008 at 12:16:52PM -0500, Phil Carns wrote:
>> I think something has changed since 2.7.0 that causes some trouble for 
>> NFS reexporting.  If I NFS mount PVFS2 from the 2.7.0 release, things 
>> look fine as far as I can tell.  If I NFS mount PVFS2 from a current 
>> trunk build, I get this behavior:
> 
> Hey Phil
> 
> The nightly builds for both HEAD and the 2.7 branch have been in a bad
> state for a bit.  System interface and MPI-IO tests pass, but VFS
> tests (specifically, dbench) hang. I'm surprised to see iget4/iget5
> complaints, but I suspect once the nightlies pass then NFS exporting
> will work better too.
> 
>> I'll probably poke around at it a little bit, but I wanted to go ahead 
>> and record it on the mailing list and see if anyone has suggestions.
> 
> There aren't *too* many vfs-related changesets in CVS since the last
> "known good" revision.  If you're looking for an angle of attack on
> this one, maybe see if any of those changesets look suspect to you.
> 
> ==rob
>