[Pvfs2-users] Re: MPICH/ROMIO/PVFS2

[Nasr Y.M.J.O]

Hello Murali,

The chemistry programs Im using are

(1) NWChem (http://www.emsl.pnl.gov/docs/nwchem/nwchem.html) which is
a free program.
(2) GAMESS-UK (http://www.cfs.dl.ac.uk/) which is free for academics
in the UK but the source code can be obtained freely under "Joint
Software Development Licence".
(3) Gaussian (http://www.gaussian.com/), commercial, I just use the
serial version.

Basicly, programs 1 and 2 are using the same tools for
parallelization. For IO part, both use chemio. Im not really an expert
on this but I can refer you to the following website:

http://www-fp.mcs.anl.gov/chemio/

I can see now that chemio is originated from the same place where
mpich, romio and pvfs2 are created :-).

Thanks a lot,
nasr

On 9/20/06, Murali Vilayannur <vilayann at mcs.anl.gov> wrote:
> Hi Nasr,
>
> > Well, for the previous trials, the configuration step goes to
> > completion with no errors but the log file contains the errors you
> > saw. So I decided to proceed with the build and I could build mpich
> > 1.2.7p1 with intel compilers even though there are many warnings and I
> > hope ignoring them is safe.
>
> Oh I think it should be safe to ignore them if the build proceeds without
> any errors.
> Configure does a lot of checks and some of them are bound to fail. So
> unless we have reason to believe that they don't work, lets leave them
> alone.
>
> > To do a simple test, I compiled the benchmark "netpipe" using mpicc
> > and I could run it without any errors in a pvfs2 mount directory so
> > for now it looks fine.
>
> Hmm.. I suspect netpipe does not do any actual I/O to the pvfs2 file
> system. I think it would be good to run something like the attached
> program
> % mpicc io.c -g -Wall -o io
> % mpiexec -machinefile </path/to/machines/file> -n <number of processors>
> ./io -f </path/to/PVFS2/mount/point/file> -u -c -i 10
>
> and hopefully you should be able to see that the program finishes and
> prints out results on how fast the operation went.
>
>
> > I have some questions about pvfs2.
> >
> > 1- After running the netpipe benchmark (as a user, not a root), when I
> > tried umount pvfs2 (as a root) I get "umount ..pvfs2 device is busy".
> > I waited for sometime and retried but the error still showing.
>
> If you cd'ed into the pvfs2 directory before running the netpipe
> benchmark, it is not possible to umount the directory without changing to
> a non-pvfs2 mounted directory.
> Basically, you cannot unmount if any process has a reference to any file
> on the pvfs2 volume.
> You could do
>
> % /usr/sbin/lsof /pvfs2/mount/point
> to get the names of all processes that have a reference to the
> mounted pvfs2 volume.
> After killing all those processes, you should be able to umount the
> volume.
> If you cannot unmount the volume still, then it is a bug and we will
> investigate that and fix it.
>
> > 2- Can I run executables from pvfs2 mount directory? My applications
> > are chemistry packages. I used to do the compiling in an NFS exported
> > directory, and I used my home directory as the working and scratch
> > directory but the executables remain in the /export directory. What is
> > the best approach under PVFS2?
>
> Yes, you can run executables off a pvfs2 mounted volume!
> You could keep both your executables and scratch data files on PVFS2,
> although I suspect it might be better (from a performance viewpoint) if
> you choose to keep executables on your home directory.
> Since we do not do any client-side caching (yet!), performance of small
> I/O's is not very good and hence running executables may not perform well
> with pvfs2.
> If you decide to use pvfs2 for executables, you could enable a
> client-side optimization by rerunning configure
>
> ./configure --enable-mmap-racache ..<usual arguments>
>
> This should improve performance quite a bit (i.e. running executables will
> benefit from this optimization)
> Hopefully, before the next release we should have a few more optimizations
> that will help workloads such as these.
>
>
> > Many thanks for your patience for a non-conputer-scientist like me
> > :-). I appreciate your help.
>
> No problem! Thanks for all the detailed reports!
> Please feel free to let us know if you hit any issues (performance,
> stability etc).
>
> Also, is it possible for you to describe what kind of access patterns we
> can expect from your chemistry applications? A rough overview is
> sufficient. Alternatively, are these apps available someplace?
>
> thanks,
> Murali
>
> >
> > nasr
> >
> >
> >
> > On 9/19/06, Murali Vilayannur <vilayann at mcs.anl.gov> wrote:
> > > Hi Nasr,
> > > Something with your compiler/build environment must be broken..:(
> > > Does configure itself choke or does the subsequent make choke for you?
> > > If so, what was the exact error that you saw..?
> > > It increasingly looks like this may be an MPICH build problem or a build
> > > problem on your platform due to a buggy compiler etc..
> > > thanks,
> > > Murali
> > >
> > > On Tue, 19 Sep 2006, Nasr Omar wrote:
> > >
> > > > Hello Murali,
> > > >
> > > > Attached is the configuration file for mpich2 1.0.4p1. The errors are
> > > > there just like in the case of mpich 1.2.7p1 I was trying to compile
> > > > earlier.
> > > >
> > > > Thanks in advance,
> > > > nasr
> > > >
> > > >
> > > > # cp /root/Desktop/cluster/mpich2-1.04p1/mpich2-1.0.4p1.tar.gz /usr/local/src
> > > > # cd /usr/local/src
> > > > # tar -xzf mpich2-1.0.4p1.tar.gz
> > > > # mkdir -p /opt/mpich2/gnu
> > > > # mkdir -p /tmp/mpich2/gnu
> > > > # export RSHCOMMAND=ssh
> > > > # export CFLAGS="-I/opt/pvfs2/include"
> > > > # export LDFLAGS="-L/opt/pvfs2/lib"
> > > > # export LIBS="-L/opt/pvfs2/lib -lpvfs2"
> > > > # cd /tmp/mpich2/gnu
> > > > # /usr/local/src/mpich2-1.0.4p1/configure --prefix=/opt/mpich2/gnu
> > > > --with-device=ch3:sock --enable-sharedlibs=gcc --enable-romio
> > > > --with-file-system=pvfs2+nfs+ufs --enable-f77 --enable-cxx
> > > >
> > > >
> > >
> >
> >
>
>