[Pvfs2-developers] patch: PVFS2 group permission checking for groups with large number of members

[Phil Carns]

Thanks David- this is committed to CVS trunk now.

-Phil

David Metheny wrote:
> Round 2... 
> 
> It turns out that getting the groups a UID is part of is more costly,
> especially on systems like LDAP. The LDAP system will have to scan all the
> groups to determine that info, where as the call to get the members of a
> group don't require the scan. Using NSCD, or something similar, will negate
> the cost, but not all environments are configured or have that option.
> 
> Attached is a patch that increases the buffer size for the group permission
> checking to around 1MB. That seems excessive, but 1024 bytes failed at
> around 200 members of a group, and since the system can't determine the
> correct size via sysconf, I just picked something that would take a long
> time to break. 
> 
> -----Original Message-----
> From: Phil Carns [mailto:pcarns at gmail.com] On Behalf Of Phil Carns
> Sent: Tuesday, March 24, 2009 9:40 AM
> To: david.metheny at gmail.com
> Cc: pvfs2-developers at beowulf-underground.org
> Subject: Re: [Pvfs2-developers] patch: PVFS2 group permission checking for
> groups with large number of members
> 
> Thanks David!  This is a big improvement over the way the code worked 
> before.
> 
> The patch is in cvs trunk now with some minor formatting updates.
> 
> -Phil
> 
> David Metheny wrote:
>> The attached patch corrects a bug where group permission checking was 
>> failing for a group with 221 members. The patch changes the logic for 
>> group permission checking up a bit. Rather than get the user list from 
>> the group, this patch gets the groups that the user is a member of. I'm 
>> taking the assumption that groups probably have a large number of 
>> members, but users usually have less number of groups. This also seemed 
>> to help out a bit on the required memory allocations.
>>
>>  
>>
>>  
>>
>> ------------------------------------------------------------------------
>>
>> *From:* pvfs2-developers-bounces at beowulf-underground.org 
>> [mailto:pvfs2-developers-bounces at beowulf-underground.org] *On Behalf Of 
>> *David Metheny
>> *Sent:* Thursday, March 12, 2009 11:32 AM
>> *To:* pvfs2-developers at beowulf-underground.org
>> *Subject:* [Pvfs2-developers] PVFS2 group permission checking failing 
>> forgroups with large number of members
>>
>>  
>>
>> I've run into an issue where permissions are failing during group 
>> permission checking (PINT_check_group).
>>
>>  
>>
>> The example case is where userA is a member of groupA.  groupA has 221 
>> member users.
>>
>>  
>>
>> drwxrwx---  1 userB groupA       4096 Mar 11 15:35 subdir
>>
>>  
>>
>> The PINT_check_mode uses the sysconf call to determine the size of the 
>> buffer to use for lookups on groups and users. In this case, the 
>> users/groups are stored in LDAP. The call here is 
>> check_group_gr_buffer_size = sysconf(_SC_GETGR_R_SIZE_MAX). As a 
>> default, if we can't make the sysconf call, it is being set to 1024 
>> bytes. For RHEL4 systems in our environments, the sysconf is also 
>> returning 1024.
>>
>>  
>>
>> In the above situation, 1024 isn't a large enough buffer to allocate all 
>> of the member usernames associated with the group, and the getgrgid_r 
>> function call returns an error.
>>
>>  
>>
>> As easy enough change would be to increase the buffer size used. Another 
>> idea would be to change the call up a bit to get the groups that the 
>> user is part of, and compare those groups against the group for the 
>> file. Are there any thoughts on which approach to implement? I'm not 
>> sure which would be faster in general, but in the environments I'm 
>> seeing, users are in less groups, than groups have members. If we change 
>> the logic on group checking, I'm assuming we would also want to 
>> implement a larger buffer size for both the group and user lookups.
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>  
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers