[Pvfs2-users] PVFS2 Installation
Jeff Pummill
jpummil at uark.edu
Tue Apr 18 07:42:39 EDT 2006
And things continue to improve!
[jpummil at prospero bin]$ ./pvfs2-ping -m /mnt/pvfs2
(1) Parsing tab file...
(2) Initializing system interface...
(3) Initializing each file system found in tab file: /etc/pvfs2tab...
/mnt/pvfs2: Ok
(4) Searching for /mnt/pvfs2 in pvfstab...
PVFS2 servers: tcp://comp-pvfs-0-24.local:3334
Storage name: pvfs2-fs
Local mount point: /mnt/pvfs2
meta servers:
tcp://pvfs2-meta-server-0-0:3334
data servers:
tcp://comp-pvfs-0-24:3334
tcp://comp-pvfs-0-25:3334
tcp://comp-pvfs-0-26:3334
tcp://comp-pvfs-0-27:3334
tcp://comp-pvfs-0-28:3334
tcp://comp-pvfs-0-29:3334
(5) Verifying that all servers are responding...
meta servers:
tcp://pvfs2-meta-server-0-0:3334 Ok
data servers:
tcp://comp-pvfs-0-24:3334 Ok
tcp://comp-pvfs-0-25:3334 Ok
tcp://comp-pvfs-0-26:3334 Ok
tcp://comp-pvfs-0-27:3334 Ok
tcp://comp-pvfs-0-28:3334 Ok
tcp://comp-pvfs-0-29:3334 Ok
(6) Verifying that fsid 1594492348 is acceptable to all servers...
Ok; all servers understand fs_id 1594492348
(7) Verifying that root handle is owned by one server...
Root handle: 1048576
Warning: non PVFS2 error code (16):
PVFS_mgmt_setparam_all: Invalid argument
Failure: failed to check root handle.
[jpummil at prospero bin]$ ./pvfs2-statfs -m /mnt/pvfs2
aggregate statistics:
---------------------------------------
fs_id: 1594492348
total number of servers (meta and I/O): 7
handles available (meta and I/O): 3681400536
handles total (meta and I/O): 3681400536
bytes available: 25643876352
bytes total: 35269730304
NOTE: The aggregate total and available statistics are calculated based
on an algorithm that assumes data will be distributed evenly; thus
the free space is equal to the smallest I/O server capacity
multiplied by the number of I/O servers. If this number seems
unusually small, then check the individual server statistics below
to look for problematic servers.
meta server statistics:
---------------------------------------
I/O server statistics:
---------------------------------------
server: tcp://comp-pvfs-0-24:3334
RAM bytes total : 1059573760
RAM bytes free : 312512512
uptime (seconds) : 509398
load averages : 66624 66560 65536
handles available: 613566756
handles total : 613566756
bytes available : 4274118656
bytes total : 5878288384
mode: serving only I/O data
server: tcp://comp-pvfs-0-25:3334
RAM bytes total : 1059573760
RAM bytes free : 317526016
uptime (seconds) : 509296
load averages : 65536 66016 65536
handles available: 613566756
handles total : 613566756
bytes available : 4274139136
bytes total : 5878288384
mode: serving only I/O data
server: tcp://comp-pvfs-0-26:3334
RAM bytes total : 1059573760
RAM bytes free : 345608192
uptime (seconds) : 509243
load averages : 67680 67008 65536
handles available: 613566756
handles total : 613566756
bytes available : 4274171904
bytes total : 5878288384
mode: serving only I/O data
server: tcp://comp-pvfs-0-27:3334
RAM bytes total : 1059573760
RAM bytes free : 315330560
uptime (seconds) : 509187
load averages : 69920 70304 66432
handles available: 613566756
handles total : 613566756
bytes available : 4274118656
bytes total : 5878288384
mode: serving only I/O data
server: tcp://comp-pvfs-0-28:3334
RAM bytes total : 1059573760
RAM bytes free : 321171456
uptime (seconds) : 509141
load averages : 68128 69760 66144
handles available: 613566756
handles total : 613566756
bytes available : 4274126848
bytes total : 5878288384
mode: serving only I/O data
server: tcp://comp-pvfs-0-29:3334
RAM bytes total : 1059573760
RAM bytes free : 296800256
uptime (seconds) : 509101
load averages : 65536 66624 65536
handles available: 613566756
handles total : 613566756
bytes available : 4273979392
bytes total : 5878288384
mode: serving only I/O data
Next, I tried to do a pvfs2-cp. This seemed to have issues....
[root at prospero bin]# ./pvfs2-cp
/home/jpummil/libgfortran-4.0.2-8.fc4.i386.rpm
/mnt/pvfs2/libgfortran-4.0.2-8.fc4.i386.rpm
[11:41:50.473719] Object Type mismatch error: Not a directory
[11:41:50.473971] PINT_sm_common_parent_getattr_failed: Not a directory
PVFS_sys_create: Not a directory
Could not open /mnt/pvfs2/libgfortran-4.0.2-8.fc4.i386.rpm
[jpummil at prospero bin]$ ps -ef | grep pvfs
root 6815 1 0 Apr12 ? 00:00:00 /usr/sbin/automount
--timeout=60 --ghost /mnt/pvfs2 file /etc/auto.pvfs2
root 6869 1 0 Apr12 ? 00:00:00
/opt/pvfs2/sbin/pvfs2-client -p /opt/pvfs2/sbin/pvfs2-client-core
root 6870 6869 0 Apr12 ? 00:00:01 pvfs2-client-core -a 5
jpummil 18952 15737 0 11:39 pts/4 00:00:00 grep pvfs
Jeff F. Pummill
Robert Latham wrote:
> On Tue, Apr 18, 2006 at 05:55:43AM -0500, Jeff Pummill wrote:
>
>> I have re-run pvfs2-genconfig using the /pvfs2-data directory found on
>> the I/O and meta nodes as the storage target.
>>
>> Oddly, the I/O and meta nodes appear to be running the pvfs2 client-core
>> rather than the server.
>>
>
> Thanks for including your steps. You've got the right idea, but there
> are a few things you are missing before 'pvfs2-ping' will work:
>
> You've got your config files now (/etc/pvfs2-fs.conf and
> seven files like /etc/pvfs2-server.conf-comp-pvfs-0-XX ), so that's a
> good start. When i say 'comp-pvfs-0-XX' i mean "the name of an I/O
> or metadata server"
>
> You'll have to somehow get those files from prospero to the other
> nodes.
> http://www.pvfs.org/pvfs2/pvfs2-quickstart.html#sec:server-config
> describes one approach using 'scp', but it's probably easiest to just
> "scp /etc/pvfs2-* comp-pvfs-0-XX:/etc/" for each of your servers.
>
> Then on each node, you need to create the storage space with the '-f'
> option to pvfs2-server:
>
> pvfs2-server -f pvfs2-fs.conf pvfs2-server.conf-comp-pvfs-0-XX
>
> After you create the storage space, then you can start the servers.
> Again, do this on each node (note, there's no -f option):
>
> pvfs2-server pvfs2-fs.conf pvfs2-server.conf-comp-pvfs-0-XX
>
> Once all the servers are running (one on each of your machines) you'll
> be able to run pvfs2-ping. Once pvfs2-ping reports everything is OK,
> then you can move on to the kerrnel interface.
>
> ==rob
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20060418/9bfcb4d7/attachment.htm
More information about the Pvfs2-users
mailing list