[Pvfs2-developers] Review: readcaching code

[Murali Vilayannur]

Sam,
The problem is not in the system call (fsetxattr) but the arguments to it..
user.pvfs2.meta_hint is the key and val is actually a uint64 which is
a bitwise OR
of PVFS_IMMUTABLE_FL, other pvfs flags.
modify_val() in pvfs2-xattr.c will give an example of this usage.
Sorry, it is a little convoluted ..:(
but I couldn't/didn't want to do more string parsing on server side.
Feel free to change that if you think it is needlessly convoluted.
thanks,
Murali

PS: let me know how the caching patches work out :)
I havent had too much time to play with it since Feb though.
Hope it works :)


On 8/17/07, Sam Lang <slang at mcs.anl.gov> wrote:
>
> Hi Murali,
>
> I wrote a little program to test the performance of the read-caching
> immutable file stuff.  With the  attached program, I get a EINVAL
> error on the read of the file after the immutable attribute has been
> set (using fsetxattr).  Also, ls -la gives me really strange results
> for the files that I've set that immutable attribute on.  In the
> below listing, tmpfile1 and tmpfile10 didn't have the immutable
> attribute set.  It looks like the problem is with the fsetxattr
> system call.  The setfattr util does the same thing.  When I set the
> xattr with pvfs2-xattr though, I don't see the corruption in listing
> the file.  I'll try to investigate what fsetxattr is doing, but are
> you aware of any problems with using the system call?
>
> -sam
>
> root at bb22:/tmp/pvfsmnt# ls -la
> total 10260
> drwxrwxrwt 1 slang mpi      4096 2007-08-17 16:35 .
> drwxrwxrwt 5 root  root     4096 2007-08-17 15:47 ..
> drwxrwxrwx 1 slang mpi      4096 2007-08-17 15:47 lost+found
> -rw-r--r-- 1 root  root        0 2007-08-17 16:24 tmpfile1
> -rw-r--r-- 1 root  root 10485760 2007-08-17 16:34 tmpfile10
> ?--------- ? ?     ?           ?                ? tmpfile11
> ?--------- ? ?     ?           ?                ? tmpfile2
> ?--------- ? ?     ?           ?                ? tmpfile3
> ?--------- ? ?     ?           ?                ? tmpfile4
> ?--------- ? ?     ?           ?                ? tmpfile5
> ?--------- ? ?     ?           ?                ? tmpfile6
> ?--------- ? ?     ?           ?                ? tmpfile7
> ?--------- ? ?     ?           ?                ? tmpfile9
> root at bb22:/tmp/pvfsmnt#
>
>
>
> On Feb 20, 2007, at 1:06 AM, Murali Vilayannur wrote:
>
> > Hi all,
> > Finally, I got some time to whip up the read-caching patches for
> > non-mutable files into a semblance of shape and stability.
> > With this patch, I am able to get I/Os to a file (marked immutable)
> > serviced from the page-cache. One can tag a file as immutable by
> > running,
> > ./src/apps/admin/pvfs2-xattr -s -k user.pvfs2.meta_hint -v
> > "+immutable" /path/to/pvfs2-file
> > To verify if a file is indeed tagged immutable,
> > ./src/apps/admin/pvfs2-xattr -t -k user.pvfs2.meta_hint /path/to/
> > pvfs2-file
> > (or)
> > ./src/apps/admin/pvfs2-stat /path/to/pvfs2/file
> >
> > I have also added some preliminary statistics exported via
> > /proc/sys/pvfs2/stats/
> > that can be used as a placeholder for more interesting statistics
> > later on.
> > Currently, it only shows # of reads, writes, hits in thepage-cache
> > and misses.
> >
> > For some reason now, cache hits do not happen across a file close.
> > Within a file open-close session, all reads get serviced from the
> > cache though. Very weird.
> > My hunch is that file pages are somehow getting removed from the radix
> > tree of the address space due to some page-ref counting issues. I will
> > dig into this later this week.
> >
> > In any case, this code should not cause any regression of older code
> > paths (hopefully!) and should not impose any performance penalties for
> > workloads making use of the page-cache because of the way we aggregate
> > cache miss I/Os to the server.
> > It was really nice to be able to make use of the iox() infrastructure
> > that was already in place to service non-contigous file and memory
> > I/O.
> > More details of the implementation is described in the thread below.
> > http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006-
> > November/002847.html
> > Hopefully, I have addressed most of Pete's comments.
> > More comments and testing welcome!
> > thanks,
> > Murali
> > <read-cache-5.patch>
> > _______________________________________________
> > Pvfs2-developers mailing list
> > Pvfs2-developers at beowulf-underground.org
> > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>
>
>
>