[Pvfs2-developers] BMI initial contact between new peers

[Sam Lang]

Scott Atchley wrote:
> Hi Sam,
> 
>> Hi Scott,
>>
>>> Since clients initiate all first contact with the servers, what is the
>>> standard procedure? Send an unexpected message only or post a receive
>>> and then send an unexpected message and wait for an expected send from
>>> the server?
>>
>> The latter.
>>
>>> I am trying to determine whether a send or receive is posted first for a
>>> new peer.
>>
>> The msgpairarray state machine in src/common/misc/msgpairarray.sm is the
>> engine for most of the request/response style unexpected messages from
>> client to server.  The msgpairarray_post function first calls
>> job_bmi_recv which posts the receive (calls BMI_post_recv), and then
>> calls job_bmi_send_list, which basically translates to calling
>> BMI_post_sendunexpected_list.
> 
> Then I need to have code in the recv function to create a new peer's 
> state if it is the first contact. In Lustre, this only happens in the 
> send function.

Could that be done in addr_lookup?  The PVFS_BMI_addr_t that we pass to 
each of the BMI functions is returned by that call, and we always call 
it before posting any receives or sends.

> 
>>> Also, can more than one BMI method be called at anyone time (e.g.
>>> concurrent send and/or recvs)?
>>
>> Yes.  All the BMI calls are non-blocking, and we often post many
>> receives and sends and let the test call drive completion of them.
>>
>> -sam
> 
> I should have have asked it a different way. Can multiple calls to the 
> send function happen at the same time (i.e. is BMI threaded and make 
> simultaneous calls to the send function or recv function or any 
> combination of functions)? I am wondering about the potential for race 
> conditions and when I need to protect against them.
> 

The BMI calls are meant to be thread-safe, although right now in the 
code I think all BMI calls are made from within the same thread.  That's 
certainly true on the client at least.  On the server, the flow code 
might end up calling a bmi function from the trove thread and another 
from the server thread.  I think the short answer is yes. :-)

-sam

> Thanks,
> 
> Scott
>