[Pvfs2-developers] Suggested testing for bmi_mx?

[Scott Atchley]

Hi Murali,

Ok, I will check them out.

In the meantime, I have written a test similar to IMB PingPong that  
uses BMI directly. It should work with TCP, GM, MX, and IB. Below are  
some various results for some old Xeons with Myrinet-2000 cards (250  
MB/s link rate).

The latency is one-way and throughput is bi-directional.

I will also write a version that tests unexpected messages up to  
unexpected max size.

Scott

bmi_mx results:

Length		Lat (us)	BW (MB/s)
1		7.97		0.13
64		8.85		7.23
256		11.99		21.35
512		13.76		37.20
1024		17.76		57.67
4096		32.49		126.07
8192		54.38		150.65
32768		158.41		206.85
1048576		4583.91		228.75

With the registration cache:
1048576		4305.72		243.53


For comparison, these are mx_pingpong (raw MX) results for the same  
message sizes:

    Length   Latency(us)    Bandwidth(MB/s)
         1       3.466          0.288
        64       4.587         13.954
       256       7.141         35.852
       512       9.089         56.329
      1024      12.937         79.153
      4096      26.696        153.431
      8192      46.815        174.987
     32768     154.087        212.659
   1048576    4271.931        245.457

BMI and/or bmi_mx adds about 4.5 us additional latency. It can get  
close to line rate with or without the MX registration cache.

Scott