[Pvfs2-developers] Review: readcaching code
Sam Lang
slang at mcs.anl.gov
Sun Aug 19 21:49:50 EDT 2007
On Aug 19, 2007, at 6:31 PM, Murali Vilayannur wrote:
> Sam,
> immutable files are by definition to prevent users from deleting
> (including admins who may make a mistake)
> Once immutable file bit is set, you cannot delete the file or do
> anything to it even as root!
> It is a file-level lockdown.
>
> The only way out of the lockdown is to be able to unset the bit.
> You should of course be able to unset the immutable file bit
> property as root.
> That should be the only thing we should allow...
> Hence the check should not be there in set-eattr.sm and del-eattr.sm
This of course doesn't map well to distributed file systems. How are
you going to deal with cached files once the immutable attribute is
removed?
I've attached a patch that adds the ioctls to enable setting the
immutable with the chattr command.
-sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ioc-flags.patch
Type: application/octet-stream
Size: 1700 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-developers/attachments/20070819/222b922b/ioc-flags-0001.obj
-------------- next part --------------
> thanks,
> Murali
>
>> Maybe I'm missing something but I think deleting the file should be
>> allowed, in fact it should be the only way to remove the immutable
>> attribute.
>>
>> -sam
>>
>>> thanks,
>>> Murali
>>>>
>>>> -sam
>>>>
>>>> On Aug 17, 2007, at 10:09 PM, Murali Vilayannur wrote:
>>>>
>>>>> Sam,
>>>>> The problem is not in the system call (fsetxattr) but the
>>>>> arguments
>>>>> to it..
>>>>> user.pvfs2.meta_hint is the key and val is actually a uint64
>>>>> which is
>>>>> a bitwise OR
>>>>> of PVFS_IMMUTABLE_FL, other pvfs flags.
>>>>> modify_val() in pvfs2-xattr.c will give an example of this usage.
>>>>> Sorry, it is a little convoluted ..:(
>>>>> but I couldn't/didn't want to do more string parsing on server
>>>>> side.
>>>>> Feel free to change that if you think it is needlessly convoluted.
>>>>> thanks,
>>>>> Murali
>>>>>
>>>>> PS: let me know how the caching patches work out :)
>>>>> I havent had too much time to play with it since Feb though.
>>>>> Hope it works :)
>>>>>
>>>>>
>>>>> On 8/17/07, Sam Lang <slang at mcs.anl.gov> wrote:
>>>>>>
>>>>>> Hi Murali,
>>>>>>
>>>>>> I wrote a little program to test the performance of the read-
>>>>>> caching
>>>>>> immutable file stuff. With the attached program, I get a EINVAL
>>>>>> error on the read of the file after the immutable attribute has
>>>>>> been
>>>>>> set (using fsetxattr). Also, ls -la gives me really strange
>>>>>> results
>>>>>> for the files that I've set that immutable attribute on. In the
>>>>>> below listing, tmpfile1 and tmpfile10 didn't have the immutable
>>>>>> attribute set. It looks like the problem is with the fsetxattr
>>>>>> system call. The setfattr util does the same thing. When I set
>>>>>> the
>>>>>> xattr with pvfs2-xattr though, I don't see the corruption in
>>>>>> listing
>>>>>> the file. I'll try to investigate what fsetxattr is doing,
>>>>>> but are
>>>>>> you aware of any problems with using the system call?
>>>>>>
>>>>>> -sam
>>>>>>
>>>>>> root at bb22:/tmp/pvfsmnt# ls -la
>>>>>> total 10260
>>>>>> drwxrwxrwt 1 slang mpi 4096 2007-08-17 16:35 .
>>>>>> drwxrwxrwt 5 root root 4096 2007-08-17 15:47 ..
>>>>>> drwxrwxrwx 1 slang mpi 4096 2007-08-17 15:47 lost+found
>>>>>> -rw-r--r-- 1 root root 0 2007-08-17 16:24 tmpfile1
>>>>>> -rw-r--r-- 1 root root 10485760 2007-08-17 16:34 tmpfile10
>>>>>> ?--------- ? ? ? ? ? tmpfile11
>>>>>> ?--------- ? ? ? ? ? tmpfile2
>>>>>> ?--------- ? ? ? ? ? tmpfile3
>>>>>> ?--------- ? ? ? ? ? tmpfile4
>>>>>> ?--------- ? ? ? ? ? tmpfile5
>>>>>> ?--------- ? ? ? ? ? tmpfile6
>>>>>> ?--------- ? ? ? ? ? tmpfile7
>>>>>> ?--------- ? ? ? ? ? tmpfile9
>>>>>> root at bb22:/tmp/pvfsmnt#
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Feb 20, 2007, at 1:06 AM, Murali Vilayannur wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> Finally, I got some time to whip up the read-caching patches for
>>>>>>> non-mutable files into a semblance of shape and stability.
>>>>>>> With this patch, I am able to get I/Os to a file (marked
>>>>>>> immutable)
>>>>>>> serviced from the page-cache. One can tag a file as immutable by
>>>>>>> running,
>>>>>>> ./src/apps/admin/pvfs2-xattr -s -k user.pvfs2.meta_hint -v
>>>>>>> "+immutable" /path/to/pvfs2-file
>>>>>>> To verify if a file is indeed tagged immutable,
>>>>>>> ./src/apps/admin/pvfs2-xattr -t -k user.pvfs2.meta_hint /path/
>>>>>>> to/
>>>>>>> pvfs2-file
>>>>>>> (or)
>>>>>>> ./src/apps/admin/pvfs2-stat /path/to/pvfs2/file
>>>>>>>
>>>>>>> I have also added some preliminary statistics exported via
>>>>>>> /proc/sys/pvfs2/stats/
>>>>>>> that can be used as a placeholder for more interesting
>>>>>>> statistics
>>>>>>> later on.
>>>>>>> Currently, it only shows # of reads, writes, hits in thepage-
>>>>>>> cache
>>>>>>> and misses.
>>>>>>>
>>>>>>> For some reason now, cache hits do not happen across a file
>>>>>>> close.
>>>>>>> Within a file open-close session, all reads get serviced from
>>>>>>> the
>>>>>>> cache though. Very weird.
>>>>>>> My hunch is that file pages are somehow getting removed from the
>>>>>>> radix
>>>>>>> tree of the address space due to some page-ref counting
>>>>>>> issues. I
>>>>>>> will
>>>>>>> dig into this later this week.
>>>>>>>
>>>>>>> In any case, this code should not cause any regression of older
>>>>>>> code
>>>>>>> paths (hopefully!) and should not impose any performance
>>>>>>> penalties for
>>>>>>> workloads making use of the page-cache because of the way we
>>>>>>> aggregate
>>>>>>> cache miss I/Os to the server.
>>>>>>> It was really nice to be able to make use of the iox()
>>>>>>> infrastructure
>>>>>>> that was already in place to service non-contigous file and
>>>>>>> memory
>>>>>>> I/O.
>>>>>>> More details of the implementation is described in the thread
>>>>>>> below.
>>>>>>> http://www.beowulf-underground.org/pipermail/pvfs2-developers/
>>>>>>> 2006-
>>>>>>> November/002847.html
>>>>>>> Hopefully, I have addressed most of Pete's comments.
>>>>>>> More comments and testing welcome!
>>>>>>> thanks,
>>>>>>> Murali
>>>>>>> <read-cache-5.patch>
>>>>>>> _______________________________________________
>>>>>>> Pvfs2-developers mailing list
>>>>>>> Pvfs2-developers at beowulf-underground.org
>>>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-
>>>>>>> developers
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
More information about the Pvfs2-developers
mailing list