[Pvfs2-developers] BMI send implemtations

[Pete Wyckoff]

atchley at myri.com wrote on Tue, 22 Aug 2006 16:18 -0400:
> MX provides 64 bits of match info for sends/recvs. I plan to  
> partition the bits as follows:
> 
> bits	comment
> 0-3	msg_type (conn_req, conn_ack, expected, unexpected, ...?)
> 4-7	reserved for credits (if used)
> 8-15 	reserved
> 15-31	peer id (16 bits => 65K peers)
> 32-63	BMI tag
>
> The conn_req and con_ack message types allow me to do a simple  
> handshake to establish MX state (hostname, NIC index, endpoint index)  
> and agree on a version number. I'm reserving bits for credits should  
> flow control be necessary and another 8 bits for future use if  
> needed. I am assuming that 65K peers will be enough for awhile. If  
> you think that PVFS will be deployed on systems with more than 65K  
> peers, I can pull from the reserved bits to increase this id. Lastly,  
> I will pass the BMI tag, which is 32 bits.
> 
> I am assuming that a call to BMI_post_send() will have a matching  
> call to BMI_post_recv() on the remote peer and that both will share  
> the same BMI tag. If not, then please let me know.

Sounds like you have a good handle on things.  Tags do have to
match, and you do in-order matching on identical peer+tag.  So your
network better not re-order or you need to add a sequence number.

		-- Pete