[Pvfs2-developers] Review: readcaching code

[Sam Lang]

Hi Murali,

Ah ok.  That makes sense.  A bad value for an extended attribute  
shouldn't make the common attributes get garbled though.  Is there a  
range check we can do either on the server when the xattr is set or  
in the kernel module when the hint is checked?

Also, should we have checks in set-eattr.sm and del-eattr.sm that  
prevent the immutable keyval from being unset?  Once a user sets it  
they shouldn't be able to unset it without deleting the file.

-sam

On Aug 17, 2007, at 10:09 PM, Murali Vilayannur wrote:

> Sam,
> The problem is not in the system call (fsetxattr) but the arguments  
> to it..
> user.pvfs2.meta_hint is the key and val is actually a uint64 which is
> a bitwise OR
> of PVFS_IMMUTABLE_FL, other pvfs flags.
> modify_val() in pvfs2-xattr.c will give an example of this usage.
> Sorry, it is a little convoluted ..:(
> but I couldn't/didn't want to do more string parsing on server side.
> Feel free to change that if you think it is needlessly convoluted.
> thanks,
> Murali
>
> PS: let me know how the caching patches work out :)
> I havent had too much time to play with it since Feb though.
> Hope it works :)
>
>
> On 8/17/07, Sam Lang <slang at mcs.anl.gov> wrote:
>>
>> Hi Murali,
>>
>> I wrote a little program to test the performance of the read-caching
>> immutable file stuff.  With the  attached program, I get a EINVAL
>> error on the read of the file after the immutable attribute has been
>> set (using fsetxattr).  Also, ls -la gives me really strange results
>> for the files that I've set that immutable attribute on.  In the
>> below listing, tmpfile1 and tmpfile10 didn't have the immutable
>> attribute set.  It looks like the problem is with the fsetxattr
>> system call.  The setfattr util does the same thing.  When I set the
>> xattr with pvfs2-xattr though, I don't see the corruption in listing
>> the file.  I'll try to investigate what fsetxattr is doing, but are
>> you aware of any problems with using the system call?
>>
>> -sam
>>
>> root at bb22:/tmp/pvfsmnt# ls -la
>> total 10260
>> drwxrwxrwt 1 slang mpi      4096 2007-08-17 16:35 .
>> drwxrwxrwt 5 root  root     4096 2007-08-17 15:47 ..
>> drwxrwxrwx 1 slang mpi      4096 2007-08-17 15:47 lost+found
>> -rw-r--r-- 1 root  root        0 2007-08-17 16:24 tmpfile1
>> -rw-r--r-- 1 root  root 10485760 2007-08-17 16:34 tmpfile10
>> ?--------- ? ?     ?           ?                ? tmpfile11
>> ?--------- ? ?     ?           ?                ? tmpfile2
>> ?--------- ? ?     ?           ?                ? tmpfile3
>> ?--------- ? ?     ?           ?                ? tmpfile4
>> ?--------- ? ?     ?           ?                ? tmpfile5
>> ?--------- ? ?     ?           ?                ? tmpfile6
>> ?--------- ? ?     ?           ?                ? tmpfile7
>> ?--------- ? ?     ?           ?                ? tmpfile9
>> root at bb22:/tmp/pvfsmnt#
>>
>>
>>
>> On Feb 20, 2007, at 1:06 AM, Murali Vilayannur wrote:
>>
>>> Hi all,
>>> Finally, I got some time to whip up the read-caching patches for
>>> non-mutable files into a semblance of shape and stability.
>>> With this patch, I am able to get I/Os to a file (marked immutable)
>>> serviced from the page-cache. One can tag a file as immutable by
>>> running,
>>> ./src/apps/admin/pvfs2-xattr -s -k user.pvfs2.meta_hint -v
>>> "+immutable" /path/to/pvfs2-file
>>> To verify if a file is indeed tagged immutable,
>>> ./src/apps/admin/pvfs2-xattr -t -k user.pvfs2.meta_hint /path/to/
>>> pvfs2-file
>>> (or)
>>> ./src/apps/admin/pvfs2-stat /path/to/pvfs2/file
>>>
>>> I have also added some preliminary statistics exported via
>>> /proc/sys/pvfs2/stats/
>>> that can be used as a placeholder for more interesting statistics
>>> later on.
>>> Currently, it only shows # of reads, writes, hits in thepage-cache
>>> and misses.
>>>
>>> For some reason now, cache hits do not happen across a file close.
>>> Within a file open-close session, all reads get serviced from the
>>> cache though. Very weird.
>>> My hunch is that file pages are somehow getting removed from the  
>>> radix
>>> tree of the address space due to some page-ref counting issues. I  
>>> will
>>> dig into this later this week.
>>>
>>> In any case, this code should not cause any regression of older code
>>> paths (hopefully!) and should not impose any performance  
>>> penalties for
>>> workloads making use of the page-cache because of the way we  
>>> aggregate
>>> cache miss I/Os to the server.
>>> It was really nice to be able to make use of the iox()  
>>> infrastructure
>>> that was already in place to service non-contigous file and memory
>>> I/O.
>>> More details of the implementation is described in the thread below.
>>> http://www.beowulf-underground.org/pipermail/pvfs2-developers/2006-
>>> November/002847.html
>>> Hopefully, I have addressed most of Pete's comments.
>>> More comments and testing welcome!
>>> thanks,
>>> Murali
>>> <read-cache-5.patch>
>>> _______________________________________________
>>> Pvfs2-developers mailing list
>>> Pvfs2-developers at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>
>>
>>
>>
>