[Pvfs2-users] install process failure ... incomplete directory setup

[Murali Vilayannur]

Hi Khalil,

> I understand that benchmark programs should support MPI-I/O to closely
> measure PVFS2. I have found the following tools specific to IO throughput
> measurement in parallel environments:
> - b_eff_io
> - IOR
>
Yep. b_eff_io is a synthetic workload. IOR on the other hand is
probably a realistic workload.

> I understand that  IO performance using posix pthreads-enabled programs have
> not been tested yet on PVFS2?

pthreaded I/O programs should work. They may not work well..:)

> I tried to compile the program (mpicc -o a.out iox.c) , but I get this
> error:
> iox.c:37: error: 'O_LARGEFILE' undeclared here (not in a function)
> (I have openmpi installed but not mpich2)
> I had to use the following command: mpicc -D_GNU_SOURCE -o a.out iox.c

hmm.. this is a bug. we should fix this in the module.mk.in file..
this flag would/should have been added automatically to the build process..

> please see appendix [PS1-iox.c] for results of mpirun on a single machine
> I have gone through the code carefully yet, but I would highly appreciate a
> brief primer on the iox.c functioning. Please.

iox.c allows you to measure I/O bandwidth when issuing I/O to/from a
single file.
It is an mpi program where several processes open the same file,
performa a barrier, do read/writes to the file for a specified number
of iterations, perform another barrier, optionally sync the contents
of the file to disk and close the file.
The program allows the user to specify the number of non-contiguous
memory buffers and the number of non-contiguous file regions
(offsets/size pairs).
Based on a command line option, I/O can be done using
a) repeated invocations of read/write() system calls over all the
memory and file regions
b) repeated invocations of readv/writev() system calls over all the file regions
c) a single invocation of readx/writex() system call to cover all the
memory and file regions..

NOTE: readx/writex requires a modified Linux kernel (patches for that
are part of the patches subdirectory) and won't work on a vanilla
kernel.
I/O bandwidth is then  measured as the time taken by the weakest link
(i.e. the time taken by the slowest process to finish the I/O to/from
the file)

> Since I am new to this topic (Parallel File Systems), I am not sure exactly
> what I might do for research. My goal in general is to do some performance
> analysis of IO throughtput for PVFS2 mainly, and optionally compare it with
> Apache's Hadoop and other PFS.. Ideally I would want to run a set of
> operation (e.g.: cp, ls, rm) on a large set of data distributed on a 4-PCs
> cluster, and would measure IO throughtput of each read/write operations set.
> This is my very basic theoretical approach to the problem, any suggestions
> are very welcome.
cp, ls and rm are single threaded programs and don't really need a
parallel file system.
In fact they may not even show up favorably in terms of results for a
parallel fs.
>
> > (...) So if you need some
> > sorting capabilities, you could write one yourself or modify the ones
> > mentioned here to use MPI and run on Linux.
> > http://research.microsoft.com/barc/SortBenchmark/
>
> Thanks. I shall work then on porting such sorting functions to linux.
> Results will be posted when available.
cool.
> I see. Then, I delay lustre installation for  after running the first tests
> with pvfs2. In any case, I can still install lustre on a single machine.
> Results will be posted when available.
ok;
thanks,
hope this helps,
Murali
>
>
> Thank you very much for your patience,
>
> K. Honsali
>
> ---------------------------------------------------------------------------------------------------------------------------
> [PS1-iox.c ]
>
> >>    mpirun -np 2 iox.out -m 0 -s 1
>  # Using read/write mode
>  # nr_procs = 2, nr_iter = 1, blk_sz = 65536, stream_ct = 1
>  # total_size = 131072
>  # Write:  min_t = 0.001239, max_t = 0.001436, mean_t = 0.001338, var_t =
> 0.000000
>  # Read:  min_t = 0.000159, max_t = 0.000163, mean_t = 0.000161, var_t =
> 0.000000
>  Write bandwidth = 91.2761 Mbytes/sec
>  Read bandwidth = 804.913 Mbytes/sec
>
> >>   mpirun -np 20 iox.out -m 0 -s 1
> # Using read/write mode
> # nr_procs = 20, nr_iter = 1, blk_sz = 65536, stream_ct = 1
> # total_size = 1310720
> # Write:  min_t = 0.000764, max_t = 0.009601, mean_t = 0.003343, var_t =
> 0.000006
> # Read:  min_t = 0.000157, max_t = 0.000295, mean_t = 0.000193, var_t =
> 0.000000
> Write bandwidth = 136.517 Mbytes/sec
> Read bandwidth = 4444.27 Mbytes/sec
>
>  >>   mpirun -np 200 iox.out -m 0 -s 1
> # Using read/write mode
> # nr_procs = 200, nr_iter = 1, blk_sz = 65536, stream_ct = 1
> # total_size = 13107200
> # Write:  min_t = 1.341870, max_t = 14.611615, mean_t = 6.855819, var_t =
> 4.484518
> # Read:  min_t = 0.000152, max_t = 7.557576, mean_t = 0.725230, var_t =
> 2.805418
> Write bandwidth = 0.89704 Mbytes/sec
> Read bandwidth = 1.73431 Mbytes/sec
>
>