[Pvfs2-developers] ACL/getxattr overhead in ls

[David Metheny]

> Hi,
> Can you pop a
> #ifdef HAVE_LINUX_POSIX_ACL_XATTR_H
> #include <linux/posix_acl_xattr.h>
> #endif
> near the top of xattr.c?
> Does that fix the error?
> Thanks,
> Murali

Yep, that fixed the compile errors. 

On RHEL3, the HAVE_GENERIC_GETXATTR is #undef'd out... On RHEL4, it's set to
1. 

I'm currently still stuck with insmod errors on the clemson CVS head, so I
added this xattr patch to a previous snapshot we had. However, this seems to
have had an affect on RHEL3... The only getxattr call now is:
getxattr("/mnt/datagrid1", "system.posix_acl_access", (nil), 0) = -1
EOPNOTSUPP (Operation not supported)

Also, on RHEL4, the getxattr call doesn't result in a network call. 

> 
> On Thu, 4 May 2006, David Metheny wrote:
> 
> > Hmmm, I'm missing something.... I see these defined in 
> pvfs2-kernel.h, 
> > and xattr.c includes this.
> >
> >   CC [M]  /tmp/pvfs2/src/kernel/linux-2.4/xattr.o
> > xattr.c: In function `pvfs2_setxattr':
> > xattr.c:77: `POSIX_ACL_XATTR_DEFAULT' undeclared (first use in this
> > function)
> > xattr.c:77: (Each undeclared identifier is reported only once
> > xattr.c:77: for each function it appears in.)
> > xattr.c:78: `POSIX_ACL_XATTR_ACCESS' undeclared (first use in this 
> > function)
> > xattr.c: In function `pvfs2_getxattr':
> > xattr.c:102: `POSIX_ACL_XATTR_DEFAULT' undeclared (first use in this
> > function)
> > xattr.c:103: `POSIX_ACL_XATTR_ACCESS' undeclared (first use in this
> > function)
> > make: *** [xattr.o] Error 1
> >
> > > -----Original Message-----
> > > From: Murali Vilayannur [mailto:vilayann at mcs.anl.gov]
> > > Sent: Thursday, May 04, 2006 11:07 AM
> > > To: David Metheny
> > > Cc: 'Phil Carns'; 'PVFS2-developers'
> > > Subject: RE: [Pvfs2-developers] ACL/getxattr overhead in ls
> > >
> > > Hi David,
> > > Interesting. Must have slipped through some of the 
> configure kernel 
> > > feature checks or something.
> > >
> > > > --- This is a strace on RHEL3
> > > > getxattr("/mnt/datagrid1", "system.posix_acl_access",
> > > (nil), 0) = -1
> > > > ENODATA (No data available) getxattr("/mnt/datagrid1", 
> > > > "system.posix_acl_default", (nil), 0) = -1 ENODATA (No data
> > > available)
> > >
> > > On this system, what does HAVE_GENERIC_GETXATTR have?
> > > Presumably it is set to 0. Here, I think we are issuing network 
> > > calls since pvfs2_getxattr() directly calls
> > > pvfs2_inode_getxattr() without any checks for whether acl's are 
> > > supported...
> > > Does this patch help prevent the network calls?
> > > There are just way too many ways to get into the xattr 
> routines. I 
> > > am not sure if/why we even need to support xattr's if 
> > > HAVE_GENERIC_GETXATTR is set to 0. :(
> > >
> > >
> > > > --- This is a strace on RHEL4
> > > > getxattr("/mnt/datagrid1", "system.posix_acl_access", 
> 0x0, 0) = -1 
> > > > EOPNOTSUPP (Operation not supported)
> > >
> > > and here, I'd imagine HAVE_GENERIC_GETXATTR being set to 1.
> > > In this case, I dont think we are issuing any network 
> getxattr calls..
> > > Can you confirm that?
> > > Thanks,
> > > Murali
> > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: pvfs2-developers-bounces at beowulf-underground.org
> > > > > [mailto:pvfs2-developers-bounces at beowulf-underground.org]
> > > On Behalf
> > > > > Of Murali Vilayannur
> > > > > Sent: Thursday, May 04, 2006 10:31 AM
> > > > > To: Phil Carns
> > > > > Cc: PVFS2-developers
> > > > > Subject: Re: [Pvfs2-developers] ACL/getxattr overhead in ls
> > > > >
> > > > > Hi Phil,
> > > > >
> > > > > > We have noticed on RedHat RHEL3 and RHEL4 that when you
> > > do a long
> > > > > > listing of a directory (on any file system, not just pvfs2)
> > > > > that the
> > > > > > ls triggers a getxattr of "system.posix_acl_access" and 
> > > > > > "system.posix_acl_default" for every single entry that
> > > is listed.
> > > > > > This happens regardless of whether ACLs are enabled for the
> > > > > file system or not.
> > > > > >
> > > > > > Digging in the coreutils/ls source, it turns out that
> > > ls (with -l
> > > > > > options at least) is checking to see if any acl's are
> > > set on each
> > > > > > entry so that it can display them differently, maybe with a
> > > > > "+" character.
> > > > > >
> > > > > > On RHEL3 at least (2.4 kernel) these extra getxattrs
> > > are causing 2
> > > > > > serialized getxattr requests that are probably
> > > > > significantly slowing
> > > > > > down the listing, which is a shame because the 2.4
> > > kernel module
> > > > > > doesn't even support ACLs so there is never any point
> > > in checking.
> > > > > >
> > > > > > Does it seem reasonable to add a test in the kernel module
> > > > > so that if
> > > > > > a) acl's are not enabled (which can be determined by
> > > checking the
> > > > > > super block acl flag), and b) the getxattr name matches
> > > > > "system.posix_acl"
> > > > > > then we just immediately return an appropriate error code
> > > > > rather than
> > > > > > servicing the getxattr?
> > > > >
> > > > > I don't understand how this situation even happens..
> > > > > There is already such a check in acl.c in the function
> > > > > pvfs2_xattr_get_acl() to see if we support acl's or 
> not before 
> > > > > calling the underlying getxattr.
> > > > > Can you find out if the strace of an ls -l causes the
> > > > > getxattr() to return -EOPNOTSUPP instead of -ENODATA?
> > > > >
> > > > > Thanks,
> > > > > Murali
> > > > >
> > > > > >
> > > > > > The stock coreutils package doesn't currently have this
> > > problem,
> > > > > > because their acl check is broken for Linux.  RedHat
> > > > > supplies a patch
> > > > > > with their rpm that adds acl checks that work on linux
> > > (using the
> > > > > > acl_extended_file() function rather than the 
> unsupported acl() 
> > > > > > function).  I imagine there are other distributions
> > > that have done
> > > > > > likewise, and probably it will be "fixed" in the official
> > > > > gnu version
> > > > > > eventually also.
> > > > > >
> > > > > > -Phil
> > > > > >
> > > > > > _______________________________________________
> > > > > > Pvfs2-developers mailing list
> > > > > > Pvfs2-developers at beowulf-underground.org
> > > > > >
> > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-develope
> > > > > > rs
> > > > > >
> > > > > >
> > > > > _______________________________________________
> > > > > Pvfs2-developers mailing list
> > > > > Pvfs2-developers at beowulf-underground.org
> > > > >
> > > 
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
> > > > >
> > > >
> > > >
> > >
> >
> >
>