[PVFS2-developers] Multiple trove implementations

[Julian Martin Kunkel]

Hi,
I'm currently working on my bachelor-thesis. The topic of the work is 
performance analysis of pvfs2, especially the layers between the application 
and trove. Therefore I implement a new trove implementation, which will throw 
away I/O data and only takes care of the metadata as far as the metadata is 
necessary for the correct operation of further pvfs2 operations. So I want to 
find bottlenecks and ways to improve pvfs2. I already have implemented a 
bunch of functions and got first results. The new module is called tas.

But first I want to discuss the current way of switching between different 
trove implementations (trove modules). 
Currently there is a dummy implementation of the function 
map_coll_id_to_method(int coll_id) in trove-mgmt.c which always returns 0.

I come to think that it would be handy to select the trove module for every 
filesystem in fs.conf like "TroveModule DBPF" and to select a different 
TroveModule for collection management (the trove storage management module 
might 
be selected by the user as well). This module stores the information about the 
selected module for every filesystem/collection. Therefore a new trove 
management function tas_get_collection_method_id can be used.
The selected module may be cached in memory to avoid access to the underlaying 
trove module. I think a new initialization and finalize functions should be 
provided for the trove storage management module.
What do you think about this approach ?

I'm currently creating a lot performance tests with mpi-io, when they are 
finished I will create some for some applications witch use directly the pvfs2 
system interface and compare them. For metadata I will compare the dbpf, 
dbpf+nosync, dbpf on top of tmpfs and dbpf on tmpfs with nosync for metadata. 
The configuration for the cluster are 5 Xeon 2Ghz PCs with a 1GBit ethernet 
network.
Enclosed you will find some pdfs which show performance data for small 
transfered data (10MB) and metadata opens per second from one client (which 
is the metadata server). 

Notes for the metadata graph:
For a small number of files (see Meta-create...-part) it seems that a better 	
caching might improve metadata throughput a lot. I wonder at the dropping 
performance if the file count totally written by test/mpi-md-test.c increases.  
Maybe a problem with the romio implementation ? If I start the program more 
often to create the same amount of files the performance depredation seems 
not to happen ! I will dig in if the gathering of performance data is 
finished.

Notes for the IO graph:
For small file-sizes and used buffers It seems that the DBPF implementation is 
rather good. For better analysis I will do more benchmarks, but with larger 
IO PVFS2 crashes sometimes :( 

I have even a lot more graphs if you are interested I will send them also...
Maybe you have some suggestions...

Have a nice day
Julian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Meta-create-1Meta-5Data-1Clients-all.pdf
Type: application/pdf
Size: 5867 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/Meta-create-1Meta-5Data-1Clients-all-0001.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Meta-create-1Meta-5Data-1Clients-part.pdf
Type: application/pdf
Size: 5491 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/Meta-create-1Meta-5Data-1Clients-part-0001.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IO-smallblock-10MB-1Meta-5Data-10Clients-all-Read+Write.pdf
Type: application/pdf
Size: 5778 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/IO-smallblock-10MB-1Meta-5Data-10Clients-all-ReadWrite-0001.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IO-smallblock-10MB-1Meta-5Data-10Clients-part-Read+Write.pdf
Type: application/pdf
Size: 5387 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/IO-smallblock-10MB-1Meta-5Data-10Clients-part-ReadWrite-0001.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IO-smallblock-10MB-1Meta-5Data-5Clients-all-Read+Write.pdf
Type: application/pdf
Size: 5733 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/IO-smallblock-10MB-1Meta-5Data-5Clients-all-ReadWrite-0001.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IO-10240x1KB-1Meta-5Data-XClients-Read+Write.pdf
Type: application/pdf
Size: 6449 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20051009/a8c00bcc/IO-10240x1KB-1Meta-5Data-XClients-ReadWrite-0001.pdf