[Pvfs2-developers] bstream bucket spreading

Sam Lang slang at mcs.anl.gov
Fri Jan 18 12:51:57 EST 2008


On Jan 17, 2008, at 3:26 PM, Pete Wyckoff wrote:

> I just noticed this bit in trove-dbpf/dbpf.h.  The code carefully
> builds ((<collid> << 24) | <handleid>) then does a modulo against
> a power of 2, effectively stripping off all but the low bits of
> the handleid.
>
> The collid is not used in the hash at all.  Just in case anybody
> thought this was important.  I've ever only used one collection per
> server.
>

Hi Pete,

KEYVAL_GET_BUCKET is left over from the days when we did a keyval DB  
per file.  With just one keyval DB now, we don't use that macro --  
there's no where in the code it gets called -- so you could safely  
remove it if you want.

As for BSTREAM_GET_BUCKET, that's funny that the coll-id isn't used in  
the hash, but it turns out not to matter.  The buckets end up as  
directories that allow us to keep the bstream directories smaller, but  
here too, all those directories are within a directory for the entire  
collection.  Go ahead and pull out the coll_id bits of the hash if you  
want.

-sam

> 		-- Pete
>
> /*
>  for more efficient host filesystem accesses, we have a simple
>  *_MAX_NUM_BUCKETS define that can be thought of more or less as
>  buckets for storing bstreams based on a simple hash of the unique
>  handle/coll_id combination.
>
>  in practice, this means we spread all bstreams into
>  *_MAX_NUM_BUCKETS directories instead of keeping all bstream entries
>  in a flat bstream directory on the host filesystem.
> */
> #define DBPF_KEYVAL_MAX_NUM_BUCKETS   32
> #define DBPF_BSTREAM_MAX_NUM_BUCKETS  64
>
> #define DBPF_KEYVAL_GET_BUCKET(__handle,  
> __id)                           \
> (((__id << ((sizeof(__id) - 1) * 8)) | __handle)  
> %                       \
>   DBPF_KEYVAL_MAX_NUM_BUCKETS)
>
> #define DBPF_BSTREAM_GET_BUCKET(__handle,  
> __id)                          \
> (((__id << ((sizeof(__id) - 1) * 8)) | __handle)  
> %                       \
>   DBPF_BSTREAM_MAX_NUM_BUCKETS)
>
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>



More information about the Pvfs2-developers mailing list