[Pvfs2-developers] Review: readcaching code

[Sam Lang]

Hi Murali,

I wrote a little program to test the performance of the read-caching  
immutable file stuff.  With the  attached program, I get a EINVAL  
error on the read of the file after the immutable attribute has been  
set (using fsetxattr).  Also, ls -la gives me really strange results  
for the files that I've set that immutable attribute on.  In the  
below listing, tmpfile1 and tmpfile10 didn't have the immutable  
attribute set.  It looks like the problem is with the fsetxattr  
system call.  The setfattr util does the same thing.  When I set the  
xattr with pvfs2-xattr though, I don't see the corruption in listing  
the file.  I'll try to investigate what fsetxattr is doing, but are  
you aware of any problems with using the system call?

-sam

root at bb22:/tmp/pvfsmnt# ls -la
total 10260
drwxrwxrwt 1 slang mpi      4096 2007-08-17 16:35 .
drwxrwxrwt 5 root  root     4096 2007-08-17 15:47 ..
drwxrwxrwx 1 slang mpi      4096 2007-08-17 15:47 lost+found
-rw-r--r-- 1 root  root        0 2007-08-17 16:24 tmpfile1
-rw-r--r-- 1 root  root 10485760 2007-08-17 16:34 tmpfile10
?--------- ? ?     ?           ?                ? tmpfile11
?--------- ? ?     ?           ?                ? tmpfile2
?--------- ? ?     ?           ?                ? tmpfile3
?--------- ? ?     ?           ?                ? tmpfile4
?--------- ? ?     ?           ?                ? tmpfile5
?--------- ? ?     ?           ?                ? tmpfile6
?--------- ? ?     ?           ?                ? tmpfile7
?--------- ? ?     ?           ?                ? tmpfile9
root at bb22:/tmp/pvfsmnt#

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-rcaching.c
Type: application/octet-stream
Size: 3148 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20070817/2e50969a/test-rcaching.obj
-------------- next part --------------

On Feb 20, 2007, at 1:06 AM, Murali Vilayannur wrote:

> Hi all,
> Finally, I got some time to whip up the read-caching patches for
> non-mutable files into a semblance of shape and stability.
> With this patch, I am able to get I/Os to a file (marked immutable)
> serviced from the page-cache. One can tag a file as immutable by
> running,
> ./src/apps/admin/pvfs2-xattr -s -k user.pvfs2.meta_hint -v
> "+immutable" /path/to/pvfs2-file
> To verify if a file is indeed tagged immutable,
> ./src/apps/admin/pvfs2-xattr -t -k user.pvfs2.meta_hint /path/to/ 
> pvfs2-file
> (or)
> ./src/apps/admin/pvfs2-stat /path/to/pvfs2/file
>
> I have also added some preliminary statistics exported via
> /proc/sys/pvfs2/stats/
> that can be used as a placeholder for more interesting statistics  
> later on.
> Currently, it only shows # of reads, writes, hits in thepage-cache  
> and misses.
>
> For some reason now, cache hits do not happen across a file close.
> Within a file open-close session, all reads get serviced from the
> cache though. Very weird.
> My hunch is that file pages are somehow getting removed from the radix
> tree of the address space due to some page-ref counting issues. I will
> dig into this later this week.
>
> In any case, this code should not cause any regression of older code
> paths (hopefully!) and should not impose any performance penalties for
> workloads making use of the page-cache because of the way we aggregate
> cache miss I/Os to the server.
> It was really nice to be able to make use of the iox() infrastructure
> that was already in place to service non-contigous file and memory
> I/O.
> More details of the implementation is described in the thread below.
> http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006- 
> November/002847.html
> Hopefully, I have addressed most of Pete's comments.
> More comments and testing welcome!
> thanks,
> Murali
> <read-cache-5.patch>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers