[Pvfs2-users] goofy permission denied error

[Murali Vilayannur]

Hi,
Is this with CVS HEAD or with pvfs2-1.5.1? what platform and distro?
Can you enable kmod debugging, rerun cp and send the relevant parts of the
logs?
sudo sh -c "echo 1 > /proc/sys/pvfs2/debug" (for pvfs2-1.5.1)
sudo sh -c "echo 32767 > /proc/sys/pvfs2/debug" (for cvs head)
thanks!
Murali

On Mon, 25 Sep 2006, Robert Olson wrote:

> This is weird. I appear to getting permission denied on every other
> file to be copied from a local filesystem to the PVFS2 filesystem.
> This ring a bell with anyone?
>
> Thanks,
> --bob
>
> olson at bio-ppc-1:/scratch$ cp -r simtab.0917 /disks/pvfs2/olson
> cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims.
> 10': Permission denied
> cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims.
> 101': Permission denied
> cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims.
> 103': Permission denied
> cp: cannot create regular file `/disks/pvfs2/olson/simtab.0917/sims.
> 105': Permission denied
>
>
> olson at bio-ppc-1:/scratch$ ls -l /disks/pvfs2/olson/simtab.0917/
> total 88044
> -rw-r--r--  1 olson seed-users 37994316 Sep 25 17:26 sims.1
> -rw-r--r--  1 olson seed-users    30049 Sep 25 17:26 sims.100
> -rw-r--r--  1 olson seed-users     7226 Sep 25 17:26 sims.102
> -rw-r--r--  1 olson seed-users  1915643 Sep 25 17:26 sims.104
> -rw-r--r--  1 olson seed-users 50200576 Sep 25 17:26 sims.106
>
> olson at bio-ppc-1:/scratch$ for i in simtab.0917/*; do echo $i; /vol/
> pvfs2/bin/pvfs2-cp -t $i /disks/pvfs2/olson/$i; done
> simtab.0917/sims.1
> Wrote 37994316 bytes in 0.437222 seconds. 82.873696 MB/seconds
> simtab.0917/sims.10
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.10
> Could not open /disks/pvfs2/olson/simtab.0917/sims.10
> simtab.0917/sims.100
> Wrote 30049 bytes in 0.001754 seconds. 16.339859 MB/seconds
> simtab.0917/sims.101
> Wrote 18867 bytes in 0.035888 seconds. 0.501365 MB/seconds
> simtab.0917/sims.102
> Wrote 7226 bytes in 0.001195 seconds. 5.766959 MB/seconds
> simtab.0917/sims.103
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.103
> Could not open /disks/pvfs2/olson/simtab.0917/sims.103
> simtab.0917/sims.104
> Wrote 1915643 bytes in 0.022985 seconds. 79.482314 MB/seconds
> simtab.0917/sims.105
> Wrote 113037406 bytes in 3.774026 seconds. 28.563891 MB/seconds
> simtab.0917/sims.106
> Wrote 177660236 bytes in 5.191931 seconds. 32.633331 MB/seconds
> simtab.0917/sims.107
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.107
> Could not open /disks/pvfs2/olson/simtab.0917/sims.107
> simtab.0917/sims.108
> Wrote 152382253 bytes in 5.079284 seconds. 28.610931 MB/seconds
> simtab.0917/sims.109
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.109
> Could not open /disks/pvfs2/olson/simtab.0917/sims.109
> simtab.0917/sims.11
> Wrote 55995114 bytes in 1.893243 seconds. 28.206152 MB/seconds
> simtab.0917/sims.110
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.110
> Could not open /disks/pvfs2/olson/simtab.0917/sims.110
> simtab.0917/sims.111
> Wrote 85578997 bytes in 2.864490 seconds. 28.491805 MB/seconds
> simtab.0917/sims.112
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.112
> Could not open /disks/pvfs2/olson/simtab.0917/sims.112
> simtab.0917/sims.113
> Wrote 159349826 bytes in 5.469951 seconds. 27.782303 MB/seconds
> simtab.0917/sims.114
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.114
> Could not open /disks/pvfs2/olson/simtab.0917/sims.114
> simtab.0917/sims.115
> Wrote 385509 bytes in 0.043475 seconds. 8.456602 MB/seconds
> simtab.0917/sims.116
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.116
> Could not open /disks/pvfs2/olson/simtab.0917/sims.116
> simtab.0917/sims.117
> Wrote 125580543 bytes in 4.246360 seconds. 28.203670 MB/seconds
> simtab.0917/sims.118
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.118
> Could not open /disks/pvfs2/olson/simtab.0917/sims.118
> simtab.0917/sims.119
> Wrote 74459114 bytes in 2.527315 seconds. 28.096912 MB/seconds
> simtab.0917/sims.12
> open: Permission denied
> could not open /disks/pvfs2/olson/simtab.0917/sims.12
> Could not open /disks/pvfs2/olson/simtab.0917/sims.12
> [etc]
>
>
> pvfs-statfs is happy:
>
> olson at bio-ppc-1:~$ /vol/pvfs2/bin/pvfs2-statfs -m /mnt/pvfs2
>
> aggregate statistics:
> ---------------------------------------
>
>          fs_id: 1239905522
>          total number of servers (meta and I/O): 4
>          handles available (meta and I/O):       5365714538
>          handles total (meta and I/O):           5368709115
>          bytes available:                        1218774892544
>          bytes total:                            1495224942592
>
> NOTE: The aggregate total and available statistics are calculated based
> on an algorithm that assumes data will be distributed evenly; thus
> the free space is equal to the smallest I/O server capacity
> multiplied by the number of I/O servers.  If this number seems
> unusually small, then check the individual server statistics below
> to look for problematic servers.
>
> meta server statistics:
> ---------------------------------------
>
> server: tcp://bio-ppc-head-4:3334
>          RAM bytes total  : 4107878400
>          RAM bytes free   : 13307904
>          uptime (seconds) : 4045972
>          load averages    : 4704 9088 5664
>          handles available: 2145227856
>          handles total    : 2147483646
>          bytes available  : 310944141312
>          bytes total      : 373806235648
>          mode: serving both metadata and I/O data
>
>
> I/O server statistics:
> ---------------------------------------
>
> server: tcp://bio-ppc-head-4:3334
>          RAM bytes total  : 4107878400
>          RAM bytes free   : 13307904
>          uptime (seconds) : 4045972
>          load averages    : 4704 9088 5664
>          handles available: 2145227856
>          handles total    : 2147483646
>          bytes available  : 310944141312
>          bytes total      : 373806235648
>          mode: serving both metadata and I/O data
>
> server: tcp://bio-ppc-head-1:3334
>          RAM bytes total  : 4107878400
>          RAM bytes free   : 2062213120
>          uptime (seconds) : 832980
>          load averages    : 30624 12352 3872
>          handles available: 1073741272
>          handles total    : 1073741823
>          bytes available  : 307329638400
>          bytes total      : 373806235648
>          mode: serving only I/O data
>
> server: tcp://bio-ppc-head-2:3334
>          RAM bytes total  : 4107878400
>          RAM bytes free   : 12697600
>          uptime (seconds) : 4045977
>          load averages    : 10592 6720 1792
>          handles available: 1073372712
>          handles total    : 1073741823
>          bytes available  : 304693723136
>          bytes total      : 373806235648
>          mode: serving only I/O data
>
> server: tcp://bio-ppc-head-3:3334
>          RAM bytes total  : 3578347520
>          RAM bytes free   : 12324864
>          uptime (seconds) : 4046045
>          load averages    : 13056 9376 3136
>          handles available: 1073372698
>          handles total    : 1073741823
>          bytes available  : 309065900032
>          bytes total      : 373806235648
>          mode: serving only I/O data
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
>