[PVFS2-developers] last email

Rob Ross rross at mcs.anl.gov
Thu Mar 11 09:58:25 EST 2004


On Thu, 11 Mar 2004 neillm at mcs.anl.gov wrote:

> On Wed, Mar 10, 2004 at 08:23:54PM -0600, Rob Ross wrote:
> > I realize that was a PVFS1 bug, but it sounded like an interesting case, 
> > and Neill was already looking at it, so I thought I would let everyone see 
> > what he figured out.
> 
> Do we have access to his code (or know what it's doing in more
> detail)?  Multi-threaded apps/execution in the general case is not a
> problem for pvfs2, as I regularly compile and execute iozone on pvfs2
> volumes and test runs using something like:
> 
> iozone -t 4 [ use processes ]
> and 
> iozone -T 4 [ use pthreads ]
> 
> [ where '4' is tested most often, but other variations of are tested
> occasionally, i.e. '8' ]
> 
> And mostly unrelated to that: since we only have mmap read support it
> doesn't matter how many people mmap it and access it from the page
> cache, no?  Though we can invalidate the page cache as often as we
> like at the cost of performance, until I see a case where we need to
> do that, I'd rather not (... it *really* kills our performance even in
> the simple execution case).

The problem that we were seeing originally in PVFS1 had to do with 
binaries being rewritten.  What we would see was, because we weren't 
clearing out the page cache appropriately, old pages of binaries would sit 
around, and weird things would happen.

That function that the PVFS1 user mentions cleared all pages out 
associated with a file at mmap time, so we knew there wouldn't be any more 
around.  Clearly that wasn't as robust a solution as we would like.

We should be able to reproduce the behavior by overwriting executables
between runs on one client w/out going through the VFS on that client (so
that the client has no way of knowing locally that the file has changed).

This is related in some ways to the UNIX "I can still access things I 
removed if I have references to them" semantics, but it isn't exactly the 
same issue.

I would like for this to work if it is technically possible given our 
system, but this is definitely *not* a high priority.  Erring on the side 
of correctness rather than performance is absolutely the right thing to 
do.  I would much rather just tell people "executing off PVFS2 is very 
slow" than have to explain the problem in detail.

Rob



More information about the PVFS2-developers mailing list