[PVFS-users] terrible performance ... help!

[Kaveh Moallemi]

Hello,

Thank you both Nathan and Robert.  I'm starting to get a picture of
whats going on ....

> Thus spake Robert Latham:
> > fprintf() (and the other <stdio.h> routines) make use of caches and
> > buffers.  pvfs_write() does no caching and is more analagous to
> > unix write().  Each call to pvfs_write means you will go over the
> > network to the io servers.
> >
> > It's quite likely that calls to fprintf will end up getting buffered
> > and serviced in one network round trip.

I didn't know that.


> > I don't know how familiar you are with MPI-IO, but you might want to
> > look into the MPI_File_* routines, particularly the support for
> > collective and noncontigous IO.

I'm new to MPI programming ... so I haven't looked into MPI-IO yet.


> Alternately, before doing any fprintf statements do:
> 
> setvbuf(filehandle, NULL, _IONBF, 0);

Yes, I'm getting very poor results by using unbuffered IO ... even
worst than the pvfs version of my program.  Earlier in my original
code, I was using a 256KB buffer for fully buffered IO by calling:

setvbuf(f3, NULL, _IOFBF, 256*1024);

I didn't notice much difference, so I commented it out.


I guess I'll modify the pvfs version so that all writes are
accumulated into a large buffer first, then written using
pvfs_write().


Thank you all for your help.


../kaveh