[Pvfs2-developers] handle ledger

[Sam Lang]

On Jan 28, 2008, at 6:43 PM, Pete Wyckoff wrote:

> slang at mcs.anl.gov wrote on Mon, 28 Jan 2008 16:38 -0600:
>> Attached patch disables the handle ledger.  For those not familiar,  
>> the
>> handle ledger is an in-memory structure that maintains allocated  
>> handles
>> for a given server.  I'm disabling it because reading the entire  
>> database
>> each time the server loads is extremely expensive for large  
>> filesystems.
>> Instead of choosing a handle from the ledger, the patch picks one  
>> randomly.
>> This means we have to deal with collisions now, but because of our  
>> large
>> handle space, they only occur every 100 billion times or so.
>>
>> I didn't blow away the handle allocation code entirely...I just  
>> disabled
>> the calls that we had been using to invoke the handle ledger, and  
>> added
>> some functionality that picks a random handle from a given range.   
>> In the
>> dspace code, I modified the create function to continue up to 32  
>> times if a
>> collision with an already existing handle occurs.
>
> Great change.  Never liked that myself either.  Some comments.
>
>> diff -u -a -p -r1.152 dbpf-dspace.c
>> --- src/io/trove/trove-dbpf/dbpf-dspace.c	8 Nov 2007 21:48:22 -0000	 
>> 1.152
>> +++ src/io/trove/trove-dbpf/dbpf-dspace.c	28 Jan 2008 21:55:49 -0000
> [..]
>> +    } while(ret != DB_NOTFOUND && ++attempts >  
>> MAX_HANDLE_ALLOC_ATTEMPTS);
>
> Uh, maybe <.

Are you arguing for increasing the max number of attempts, or just  
retrying forever?

>
>
> [..]
>> diff -u -a -p -r1.46 trove-handle-mgmt.c
>> --- src/io/trove/trove-handle-mgmt/trove-handle-mgmt.c	15 Aug 2007  
>> 18:43:09 -0000	1.46
>> +++ src/io/trove/trove-handle-mgmt/trove-handle-mgmt.c	28 Jan 2008  
>> 21:55:49 -0000
> [..]
>> +#ifdef TROVE_HANDLE_LEDGER_ENABLED
>
> Just kill it for good.  Too many ifdefs as it is.

Ok.

>
>
>> +    rfd = open("/dev/urandom", O_RDONLY, 0);
>> +    if(rfd < 0)
>> +    {
>> +        return -PVFS_EINVAL;
>> +    }
>
> Painted ourselves into a linux-specific corner here.  Maybe have the
> usual time() etc. srand option here too if open fails.
>
>> +    random_r(&trove_handle_random_data, &r1);
>> +    i = r1 % extent_array->extent_count;
>
> May want a feature test for this.  Not sure if POSIX has gotten
> itself into all the OSes on which people may run servers.

Right, I was concerned with making sure I got a good seed here.  It  
needs to generate both a very large random sequence from the seed, as  
well as not pick the same seed over and over on server startup.  Using  
initstate_r with an array size of 256 makes the values returned by  
random_r much more random, and passing the current time ensures that  
the seed will be different on each server startup.

If we use the more primitive forms of getting a random number, its  
just more likely to get repeated values for handles. Is that  
acceptable?  Does it become the user's problem his random handle  
values aren't so random?


>
>
>> +    if(sizeof(r1) == 4)
>> +    {
>> +        random_r(&trove_handle_random_data, &r2);
>> +        handle |= ((PVFS_handle)r2) << 32;
>> +    }
>
> You declared r1 as int32_t.  Of course the size will be 4.

Yeah, not sure what's going on there...
-sam

>
>
> 		-- Pete
>