[Pvfs2-developers] Hang at server shutdown

[Sam Lang]

Hi Scott,

That's not what I suspected but its still helpful.  It looks like the  
hang is in trove_handle_mgmt_finalize().  Probably trying to lock the  
trove_handle_mutex, but I'm not sure.   I don't see any other cases  
where that mutex would already be locked somewhere else.  Can you run  
the server in gdb, Ctrl-C when it hangs, and send the backtrace?

Thanks,

-sam

On Jan 18, 2007, at 1:30 PM, Scott Atchley wrote:

> On Jan 18, 2007, at 2:13 PM, Sam Lang wrote:
>
>> From the output it looks like something in trove isn't getting  
>> cleaned up properly.  The attached patch should give us more debug  
>> info as to what's causing the hang.
>>
>> -sam
>
> [E 01/18 14:26]
> PVFS2 server got signal 2 (server_status_flag: 262143)
> [D 01/18 14:26] *** server shutdown in progress ***
> [D 01/18 14:26] [+] halting state machine processor   [   ...   ]
> [D 01/18 14:26] [-]         state machine processor   [ stopped ]
> [D 01/18 14:26] [+] halting cached config interface   [   ...   ]
> [D 01/18 14:26] [-]         cached config interface   [ stopped ]
> [D 01/18 14:26] [+] halting event profiling interface [   ...   ]
> [D 01/18 14:26] [-]         event profiling interface [ stopped ]
> [D 01/18 14:26] [+] halting request scheduler         [   ...   ]
> [D 01/18 14:26] [-]         request scheduler         [ stopped ]
> [D 01/18 14:26] [+] halting job interface             [   ...   ]
> [D 01/18 14:26] [-]         job interface             [ stopped ]
> [D 01/18 14:26] [+] halting job time mgr interface    [   ...   ]
> [D 01/18 14:26] [-]         job time mgr interface    [ stopped ]
> [D 01/18 14:26] [+] halting flow interface            [   ...   ]
> [D 01/18 14:26] [SYNC_COALESCE]: dbpf_sync_context_destroy for  
> context 0 called
> [D 01/18 14:26] [-]         flow interface            [ stopped ]
> [D 01/18 14:26] [+] halting bmi interface             [   ...   ]
> [D 01/18 14:26] bmi_mx: entering BMI_mx_finalize.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_finalize.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop mx://fog34:0:0.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: entering BMI_mx_set_info.
> [D 01/18 14:26] bmi_mx: BMI_mx_set_info drop NULL.
> [D 01/18 14:26] bmi_mx: leaving BMI_mx_set_info.
> [D 01/18 14:26] [-]         bmi interface             [ stopped ]
> [D 01/18 14:26] [+] halting storage interface         [   ...   ]
> [D 01/18 14:26] dbpf_thread_function ending
> [D 01/18 14:26] dbpf_thread_finalize: finalized
> [D 01/18 14:26] dbpf_attr_cache_finalized
> [D 01/18 14:26] syncing collection attrs db [ ... ]
> [D 01/18 14:26]         collection attrs db [closed]
> [D 01/18 14:26] syncing dspace db [ ... ]
> [D 01/18 14:26]         dspace db [closed]
> [D 01/18 14:26] syncing keyval db [ ... ]
> [D 01/18 14:26]         keyval db [closed]
> [D 01/18 14:26] dbpf keyval pcache finalized
> [D 01/18 14:26] syncing storage attrs db [ ... ]
> [D 01/18 14:26]         storage attrs db [closed]
> [D 01/18 14:26] syncing collection db [ ... ]
> [D 01/18 14:26]         collection db [closed]
>