[Pvfs2-users] Problems with xattr settings with long arguments

Tony Kew tonykew at ccr.buffalo.edu
Fri Oct 17 15:24:22 EDT 2008


Dear Phil,

I verified that this is not an issue with the underlying filesystem:

c23n12$ mount | grep \ /scratch\
/dev/sda3 on /scratch type ext2 (rw)
c23n12$ sudo umount /scratch
c23n12$ sudo mount -o rw,user_xattr /dev/sda3 /scratch
c23n12$ mount | grep \ /scratch\
/dev/sda3 on /scratch type ext2 (rw,user_xattr)
c23n12$ cd /scratch
c23n12$ mkdir test
c23n12$ setfattr -n "user.pvfs2.dist_name" -v "varstrip_dist" test
c23n12$ getfattr test/
# file: test
user.pvfs2.dist_name

c23n12$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K" 
test
c23n12$ getfattr -n user.pvfs2.dist_params test# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v "strips:0:16K" test
c23n12$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K" 
test
c23n12$ getfattr -n user.pvfs2.dist_params test# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K" 
test
c23n12$ getfattr -n user.pvfs2.dist_params test# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v "strips:0:16K" 
testc23n12$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K;38:16K;39:16K;40:16K;41:16K;42:16K;43:16K;44:16K;45:16K;46:16K;47:16K;48:16K;49:16K;50:16K;51:16K;52:16K;53:16K;54:16K;55:16K;56:16K;57:16K;58:16K;59:16K;60:16K;61:16K;62:16K;63:16K" 
test
c23n12$ getfattr -n user.pvfs2.dist_params test# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K;38:16K;39:16K;40:16K;41:16K;42:16K;43:16K;44:16K;45:16K;46:16K;47:16K;48:16K;49:16K;50:16K;51:16K;52:16K;53:16K;54:16K;55:16K;56:16K;57:16K;58:16K;59:16K;60:16K;61:16K;62:16K;63:16K"

c23n12$ setfattr -n "user.pvfs2.dist_params" -v "strips:0:16K" test
c23n12$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K"

c23n12$

I tried changing PVFS_MAX_XATTR_VALUELEN in include/pvfs2-types.h from
256 to 512 (only, rather than 16K) - still has problems , setting the 
"user.pvfs2.dist_params"
param with a attribute value >256 characters works the first time you 
try it, but no value
( irrespective of how ling it is) can be set on the directory 
subsequently...


$ mount | grep /var/tmp/PVFSv2/mnt
tcp://ramones:3334/pvfs2-fs-kew on /var/tmp/PVFSv2/mnt type pvfs2 (rw)
$ cd /var/tmp/PVFSv2/mnt
$ mkdir test
$ setfattr -n "user.pvfs2.dist_name" -v "varstrip_dist" test
$ getfattr test
# file: test
user.pvfs2.dist_name

$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K" 
test
$ getfattr test
# file: test
user.pvfs2.dist_name
user.pvfs2.dist_params

$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K"

$ setfattr -n "user.pvfs2.dist_params" -v "strips:0:16K" test
$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K"

$ setfattr -n "user.pvfs2.dist_params" -v 
"strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K" 
test
$ getfattr -n user.pvfs2.dist_params test
# file: test
user.pvfs2.dist_params="strips:0:16K;1:16K;2:16K;3:16K;4:16K;5:16K;6:16K;7:16K;8:16K;9:16K;10:16K;11:16K;12:16K;13:16K;14:16K;15:16K;16:16K;17:16K;18:16K;19:16K;20:16K;21:16K;22:16K;23:16K;24:16K;25:16K;26:16K;27:16K;28:16K;29:16K;30:16K;31:16K;32:16K;33:16K;34:16K;35:16K;36:16K;37:16K"

$ setfattr -n "user.pvfs2.dist_params" -v "strips:0:16K" testsetfattr: 
test: Invalid argument
$

Thanks,
Tony

Tony Kew
SAN Administrator
The Center for Computational Research
New York State Center of Excellence
 in Bioinformatics & Life Sciences
701 Ellicott Street, Buffalo, NY 14203

CoE Office: (716) 881-8930           Fax: (716) 849-6656
CSE Office: (716) 645-3797 x2174
      Cell: (716) 560-0910          Home: (716) 874-2126

"I love deadlines, I love the whooshing noise they make as they go by."
                                                          Douglas Adams



Phil Carns wrote:
> Tony Kew wrote:
>> Dear Phil,
>>
>> I can't test the setfattr command on a local filesystem as far as I 
>> can tell:
>>
>> $ cd /var/tmp
>> $ mkdir test
>> $ setfattr -n "user.pvfs2.dist_name" -v "varstrip_dist" test
>> setfattr: test: Operation not supported
>> $
>>
>> ...or is there a test I can do on an ext3 filesystem dir?
>>
>> I changed the value in include/pvfs2-types.h to 16K (should be plenty
>> big enough I guess...)  & recompiled - now setfattr works the first 
>> time,
>> but fails subsequently..
>> Note: testing on a single RHEL5 node, if that makes a difference 
>
> <cut>
>
>> It is not easy for me to test this code on 64 nodes.
>> I don't know if we can go further with this without me doing so?
>
> Well, it looks like there are still problems just setting the 
> attributes, so there isn't really any need to try testing it on 64 
> nodes until that much is fixed.
>
> Regarding how to test on ext3, you will need to follow Emmanuel 
> Florac's suggestion about the mount option to use for ext3.  I just 
> want to make sure there isn't a system limit on the attribute size 
> before trying to bump it up in PVFS.
>
> 16K may be too large for the request messaging system.  If you 
> experiment with that it might be safer to pick 8K or smaller.
>
> -Phil


More information about the Pvfs2-users mailing list