[PVFS2-CVS] commit by pw in pvfs2/src/io/job: thread-mgr.c

[Phil Carns]

Hey Pete,

Neill and I ran into some performance problems that we tracked down to 
this optimization.  For the default TCP/IP case, it is causing the 
client and server to both burn more cpu than they need to during I/O 
transfers.  On my laptop for example, (testing with both client and 
server on same host), it makes the transfer rate for small I/O buffers 
so bad that it takes a few minutes to import a 1 M file using 4k buffers.

I'm going to back it out for the moment, but I think I see why you would 
want this for the IB case- IB really doesn't give you that many options 
for controlling poll vs. sleep behavior.

It would be nice to come up with another option, but maybe for the time 
being we could just push this logic down to the bmi_ib module?  For 
example, the bmi_ib testcontext call could see if had serviced any 
operations in the last N milliseconds, and if so it could elect to 
ignore the idle_time_ms argument and not sleep?  I _think_ this would 
get similar behavior to the thread-mgr change, but would limit it to the 
  ib module.

-Phil

CVS commit program wrote:
> Update of /projects/cvsroot/pvfs2/src/io/job
> In directory parlweb:/tmp/cvs-serv6387/src/io/job
> 
> Modified Files:
> 	thread-mgr.c 
> Log Message:
> auto-tune test_timeout to poll frantically when something appears to be going on, else back off to default timeout
> 
> 
> Index: thread-mgr.c
> ===================================================================
> RCS file: /projects/cvsroot/pvfs2/src/io/job/thread-mgr.c,v
> diff -u -p -u -r1.18 -r1.19
> --- thread-mgr.c	5 Mar 2004 17:54:29 -0000	1.18
> +++ thread-mgr.c	17 May 2004 21:04:11 -0000	1.19
> @@ -7,6 +7,7 @@
>  #include <pthread.h>
>  #include <stdlib.h>
>  #include <assert.h>
> +#include <sys/time.h>
>  
>  #include "pvfs2-types.h"
>  #include "thread-mgr.h"
> @@ -15,7 +16,7 @@
>  #include "bmi.h"
>  #include "trove.h"
>  
> -#define THREAD_MGR_TEST_COUNT 5
> +#define THREAD_MGR_TEST_COUNT 20
>  #define THREAD_MGR_TEST_TIMEOUT 10
>  static int thread_mgr_test_timeout = THREAD_MGR_TEST_TIMEOUT;
>  
> @@ -142,11 +143,12 @@ static void *trove_thread_function(void 
>  static void *bmi_thread_function(void *ptr)
>  {
>      int ret = -1;
> -    int quick_flag = 0;
>      int incount, outcount;
>      int i=0;
>      int test_timeout = thread_mgr_test_timeout;
>      struct PINT_thread_mgr_bmi_callback *tmp_callback;
> +    static struct timeval last_action;
> +    struct timeval now;
>  
>  #ifdef __PVFS2_JOB_THREADED__
>      while (bmi_thread_running)
> @@ -182,8 +184,8 @@ static void *bmi_thread_function(void *p
>  	     * operations as we can handle to indicate that we should cycle
>  	     * quickly 
>  	     */
> -	    if(outcount == THREAD_MGR_TEST_COUNT)
> -		quick_flag = 1;
> +	    if (outcount > 0)
> +		gettimeofday(&last_action, 0);
>  	}
>  	else
>  	{
> @@ -191,15 +193,17 @@ static void *bmi_thread_function(void *p
>  	}
>  
>  	/* decide how long we are willing to wait on the main test call */
> -	if(quick_flag)
> -	{
> -	    quick_flag = 0;
> -	    test_timeout = 0;
> +	gettimeofday(&now, 0);
> +	now.tv_sec -= last_action.tv_sec;
> +	now.tv_usec -= last_action.tv_usec;
> +	if (now.tv_usec < 0) {
> +	    --now.tv_sec;
> +	    /* now.tv_usec += 1000000; */
>  	}
> -	else
> -	{
> +	if (now.tv_sec > 0)  /* 1 sec decay */
>  	    test_timeout = thread_mgr_test_timeout;
> -	}
> +	else
> +	    test_timeout = 0;  /* go immediate */
>  
>  	/* indicate that a test is in progress */
>  	gen_mutex_lock(&bmi_test_mutex);
> @@ -240,6 +244,8 @@ static void *bmi_thread_function(void *p
>  	    tmp_callback->fn(tmp_callback->data, stat_bmi_actual_size_array[i],
>  		stat_bmi_error_code_array[i]);
>  	}
> +	if (bmi_test_count > 0)
> +	    gettimeofday(&last_action, 0);
>      }
>  
>      return (NULL);
> 
> _______________________________________________
> PVFS2-CVS mailing list
> PVFS2-CVS at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-cvs
>