[Pvfs2-developers] BMI send implemtations

[Scott Atchley]

On Aug 22, 2006, at 11:22 AM, Pete Wyckoff wrote:

> atchley at myri.com wrote on Tue, 22 Aug 2006 09:52 -0400:
>> What is the purpose of the Request To Send (RTS) and Clear To Send
>> (CTS) messages in the IB and other BMI implementations? Is it to
>> check if resources are available on the receiver (i.e. to see if a
>> matching receive has been posted) or to explicitly tell the receiver
>> to post a receive?
>
> The former.  I'm sure Patrick would be happy to quote from his rant
> on how much IB sucks in imposing this netowrk semantic.  :)

He is always happy to rant. :-)

What should a BMI method do if the receive is not posted? Wait?  
Cancel with failure?

>> From my discussions with Sam and Murali, they mentioned that in any
>> exchange the first message is always unexpected. These messages are
>> guaranteed to below a preset size (determined by the BMI method). Any
>> expected message should then find a matching receive on the peer. If
>> so, why use the RTS and CTS messages?
>
> BMI does not guarantee that receives be preposted.  It also does not
> look into message contents to divine that, for instance, an
> unexpected message implies a following expected message.  Even if we
> did integrate BMI into the server protocol, there are situations
> where you wouldn't know the response size in advance.
>
> 		-- Pete

I am still trying to have a clear understanding of how a send varies  
from an unexpected send. The receiver will have some number of pre- 
posted, "generic" receives (i.e. can receive from any peer) to catch  
unexpected sends. If the send is _not_ unexpected (i.e. expected),  
then it implies that the receiver will post a receive for the  
expected send (either pre-posted or slightly late due to clock  
drift), no? If not, what am I missing?

Is the purpose of the RTS/CTS messages then to stall the sending  
until the receiver has posted the receive? If so, would the receiver  
ever send a CTS to indicate that a match is not forthcoming? Or does  
the receiver only send a CTS when the receive is posted? In the  
latter case, the sender may time out waiting for a CTS and thus  
cancel the send?

Thanks,

Scott