[Pvfs2-developers] Use of pvfs_isys_io and pvfs_sys_wait

[Phil Carns]

Could you break down what the app is doing at a little bit higher level 
in this time frame?  (ie, how many writes is it posting, how many reads 
is it posting, which are concurrent, when it calls wait for each).

 From what I can tell, it looks like there are 30 total isys_io's 
posted; the first 15 are writes (triggered by pvfs2_aio_flush) and the 
last 15 are reads (triggered by pvfs2_aio_fill).  It doesn't look like 
there are any waits in between the two, though, if I am reading it right.

Are you calling wait() (or some other variant) between the writes and 
the reads?  The pvfs2 system interface doesn't order any of the 
operations, so it might just be that some of your reads happen to be 
hitting the server before your writes have put the data there.

This is different from the standard posix aio; I think their api 
automatically orders every I/O operation at least at a file descriptor 
level.  The PVFS system interface doesn't do anything like that to 
prevent I/O operations from getting out of order once they are posted.

-Phil

Troy Benjegerdes wrote:
> I have an IO shim for a fortran application that uses pvfs_isys_io and 
> pvfs_sys_wait directly, and when using the IB BMI module, I issue a 
> whole bunch of writes, but it looks like the last one doesn't actually 
> make it all the way to disk. I get a failure where I go back to read the 
> last 32M block of the file, and the total_completed value is 10mb 
> instead of the 32MB it's supposed to be.
> 
> I've got debugging output (PVFS2_DEBUGMASK=all) for both a single 
> server, and a single client at:
> 
> http://scl.ameslab.gov/~troy/pvfs/corruption/pvfs2-server-da13.log
> http://scl.ameslab.gov/~troy/pvfs/corruption/qtest.err
> 
> Where should I be looking for problems?
> 
> 
> When the file already exists, I don't get this error, but I seem to get 
> data that was already in the file, instead of what I just wrote.
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers