[Pvfs2-developers] BMI send implemtations

[Rob Ross]

Hi Scott,

You might want to ping your colleagues at ORNL and see how many distinct 
peers they expect on upcoming XT* systems. Could be more than 64K, if 
they don't do I/O forwarding (which we at ANL all hope for...).

Rob

Scott Atchley wrote:

> MX provides 64 bits of match info for sends/recvs. I plan to partition 
> the bits as follows:
> 
> bits    comment
> 0-3    msg_type (conn_req, conn_ack, expected, unexpected, ...?)
> 4-7    reserved for credits (if used)
> 8-15     reserved
> 15-31    peer id (16 bits => 65K peers)
> 32-63    BMI tag
> 
> The conn_req and con_ack message types allow me to do a simple handshake 
> to establish MX state (hostname, NIC index, endpoint index) and agree on 
> a version number. I'm reserving bits for credits should flow control be 
> necessary and another 8 bits for future use if needed. I am assuming 
> that 65K peers will be enough for awhile. If you think that PVFS will be 
> deployed on systems with more than 65K peers, I can pull from the 
> reserved bits to increase this id. Lastly, I will pass the BMI tag, 
> which is 32 bits.