[Pvfs2-users] pvfs2 performance with btio

[Dries Kimpe]

Florin Isaila schreef:
> I have a question about the PVFS2 write performance.
> 
> We did some measurements with BTIO over PVFS2 on lonestar at TACC
> (http://www.tacc.utexas.edu/services/userguides/lonestar/)
> 
> and we get pretty bad write results with classes B and C:
> 
> http://www.arcos.inf.uc3m.es/~florin/btio.htm
> 
> We used 16 I/O servers, the default configuration parameters and upto
> 100 processes. We realized that all I/O servers were used also as
> metadata servers, but BTIO uses just one file.
> 
> The times are in seconds, contain only I/O time (no compute time) and
> are aggregated per each BTIO run (BTIO performs several writes).
> 
> TroveSyncMeta was set to yes (by default). Could this cause the I/O to
> be serialized? It looks as if there were a serialization.
> 
> Or could the fact that all nodes were also launched as metadata
> managers affect the performance?
> 
I usually take only 1 metadata server; I've never tested configurations
where metaservercount == servercount.

However, I did see jumps in write performance before; In my case,
it was caused by the network itself. I had some older IB drivers that
were loosing packets under high load, which in turn would cause delays until some timeout
leading to really bad performance.

I assume pvfs2 is configured to use native IB?

Do you see any timeouts/warnings in the kernel log?

Does the same problem appear using different write patterns?
(for example continuous writes?) The attached program makes each CPU write into a different
file... If you have any network congestion problems, the same pattern should show up here.

   Dries
-------------- next part --------------
A non-text attachment was scrubbed...
Name: contwrite.c
Type: text/x-csrc
Size: 2100 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20070717/7a4164dc/contwrite.bin