[Pvfs2-developers] pvfs2-cp performance with single client and single server

[Scott Atchley]

On Dec 29, 2006, at 2:01 PM, Pete Wyckoff wrote:

> atchley at myri.com wrote on Fri, 29 Dec 2006 13:35 -0500:
>> Do these require the kernel module? I have not tried using that yet.
>
> Nope, but you need to figure out MPI.

MPI, we have many in house to play with. ;-)

I asked about the module because of the "-f pvfs2:/pvfs-ib/x1" in:

mpiexec -n 1 2402/perf -n 10 -s 800m -c 100m -f pvfs2:/pvfs-ib/x1

I assumed it meant use the file stored on a PVFS mount. I'll look  
into this app.

>>> BMI_OPTIMISTIC_BUFFER_REG.
>>
>> What does this do? Internally, MX can cache some registrations (the
>> API does not expose it). It is in my best interest to try to reuse
>> buffers.
>
> Just chunks up the memory registrations.  If you don't have an MX API
> knob for it, you'll never be able to use it.  Very handy for IB to
> amortize registration overhead though.
>
> Let us know if you manage to figure out why you're not getting good
> MX performance.  Perhaps these other tests can shed some light.  You
> might look at detailed server or client logs with usec timestamps to
> figure out where the time is going.  And oprofile may be useful if
> you think the time is going to the MX libs or driver, or pthread
> overhead.
>
> 		-- Pete

That is next on my list. Patrick suggested prof since both sides are  
in user-space. I have oprofile compiled but I do not have a valid  
vmlinux (it would require a new kernel compile and install).

Scott