[Pvfs2-users] Performance of varstrip_dist configures as a stripe verses simple_stripe

[Tony Kew]

Dear Phil,

Phil Carns wrote:
> Tony Kew wrote:
> [...]
>> The test dir was set up with 16K strips on each node for a 1024K stripe.
>> iozone set up to write in 1024K blocks.  The job took about an hour and
>> five minutes to run...
>>
>> Initial write 668,321.05 KB/sec
>>       Rewrite 677,999.64 KB/sec
>>          Read 300,937.30 KB/sec
>>       Re-read 313,756.56 KB/sec
>
> You realize that you can create this same setup with the default 
> simple stripe distribution too if you use a 16K strip size, right?
>
> [...]

Yes, I just wanted a test I could then compare with the results of similar
distributions using the simple_stripe and twod_stripe distributions.
In my tests so far varstrip_dist set up to be the same as a simple_stripe
is slower, markedly slower when the filesystem is on the same nodes
as iozone is running on (i.e. an "on the fly" filesystem built across
the nodes in a PBS job)  - I don't know if this is what you might reasonably
expect

from 
http://www.ccr.buffalo.edu/display/WEB/PVFS+version+2+scratch+filesystem+on+U2+-+Test+Results

16 client nodes to 16 (different) server nodes using simple_strip 
distribution:

Initial write
	
321,058.83 KB/sec
Rewrite
	
212,626.53 KB/sec
Read
	
161,611.95 KB/sec
Re-read
	
154,799.58 KB/sec



16 client nodes to 16 (different) server nodes using varstrip_dist 
distribution
configured to create the same stripe as above:

Initial write
	
298,962.00 KB/sec
Rewrite
	
197,318.79 KB/sec
Read
	
203,210.17 KB/sec
Re-read
	
186,866.10 KB/sec



For an "on the fly" filesystem i.e. 16 clients accessing a PVFSv2 
filesystem on
the same nodes using simple_strip distribution:

Initial write
	
219,306.19 KB/sec
Rewrite
	
130,799.13 KB/sec
Read
	
183,249.66 KB/sec
Re-read
	
191,565.02 KB/sec


For an "on the fly" filesystem i.e. 16 clients accessing a PVFSv2 
filesystem on
the same nodes using varstrip_dist distribution configured to create the 
same
stripe as above:

Initial write
	
69,381.52 KB/sec
Rewrite
	
69,611.84 KB/sec
Read
	
64,910.00 KB/sec
Re-read
	
65,248.22 KB/sec



It may be the case (given the 64 node 1024K stripe results) that on-the-fly
filesystems need larger strips (a larger stripe) to get similar 
performance from
a varstrip_dist distribution


Thanks for your help,

Tony

Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
 in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203

CoE Office: (716) 881-8930           Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
      Cell: (716) 560-0910          Home: (716) 874-2126

"I love deadlines, I love the whooshing noise they make as they go by."
                                                          Douglas Adams


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20081027/a40cd4c5/attachment.htm