[Pvfs2-users] Re: Multiple data stores on one machine

[Murali Vilayannur]

Hi Steve,

> Yes indeed I have two separate storage files, an error in my email, sorry. I
> should have pasted both parts rather than duplicate and edit from memory.

Cool!
>
>
> Do the ranges have to be in some proportions to the disk size used ?
No. The size of the handle ranges are sorta equivalent to the # of
on-disk inodes supported by most disk=based file systems such as ext3.
They don't have any correlation to the amount of disk size etc..
thanks,
Murali

>
>
>
> steve
>
>
>
> -------Original Message-------
>
>
>
> From: Murali Vilayannur
>
> Date: 12/01/2007 17:55:08
>
> To: Steve
>
> Cc: pvfs2-users at beowulf-underground.org; Greg Kurtzer
>
> Subject: Re: Multiple data stores on one machine
>
>
>
> Hi Steve,
>
>
>
> Looks good. See inline comments below...
>
>
>
> > <Aliases>
>
> > Alias caos1 TCP://caos:3334
>
> > Alias caos2 TCP://caos:3335
>
> > </Aliases>
>
> > <Filesystem>
>
> > Name pvfs2-fs
>
> > ID 699094865
>
> > RootHandle 1048576
>
> > <MetaHandleRanges>
>
> > Range caos1 4-214748365>
>
> > </MetaHandleRanges>
>
> >
>
> > <DataHandleRanges>
>
> >
>
> > Range caos1 2147483651-4294967297
>
> >
>
> > Range caos2 4294967298-6442450944
>
> >
>
> > </DataHandleRanges>
>
> >
>
> > <StorageHints>
>
> >
>
> > TroveSyncMeta yes
>
> >
>
> > TroveSyncData no
>
> >
>
> > </StorageHints>
>
> >
>
> > </Filesystem>
>
> >
>
> >
>
> >
>
> > /etc/pvfs2-server.conf-caos1
>
> >
>
> > StorageSpace /pvfs2-storage-space
>
>
>
> (^) See the comment on why it is a bad idea to share storage spaces below...
>
>
>
>
> >
>
> > HostID "TCP://caos:3334"
>
> >
>
> > LogFile /tmp/pvfs2-server.log
>
> >
>
> >
>
> >
>
> > /etc/pvfs2-server.conf-caos2
>
> >
>
> > StorageSpace /pvfs2-storage-space
>
>
>
> ^^^^^^^^
>
> Please don't use the same storage space for 2 different pvfs2-server
>
> Processes on the same node! They may end up trampling over each other.
>
> I thought the whole point of what you were trying to do was to add
>
> Some new storage space/drives to the pvfs2 volume!! :)
>
> >
>
> > Pvfs2-server /etc/pvfs2-fs.conf /etc/pvfs2-server.conf-caos1 -f
>
> >
>
> > Pvfs2-server /etc/pvfs2-fs.conf /etc/pvfs2-server.conf-caos2 -f
>
> >
>
> >
>
> >
>
> > Then started 2 servers
>
> >
>
> >
>
> >
>
> > Pvfs2-server /etc/pvfs2-fs.conf /etc/pvfs2-server.conf-caos1
>
> >
>
> > Pvfs2-server /etc/pvfs2-fs.conf /etc/pvfs2-server.conf-caos2
>
> >
>
> >
>
> >
>
> > Is that correct that I need to run 2 server daemons ?
>
>
>
> Yes! Running both the daemons is correct..
>
>
>
> > The mount command then seems a little confusing
>
> >
>
> > Mount -t pvfs2 TCP://caos:3334/pvfs2-fs /mnt/pvfs2
>
> >
>
> > As you only reference one port.
>
>
>
> This is probably documented somewhere, but it is possible to use any
>
> Of the servers for mounting!!
>
> All you need to pass is a valid bmi URL for a valid config server, so
>
> In this case
>
> Mount -t pvfs2 TCP://caos:3335/pvfs2-fs /mnt/pvfs2
>
> Would also have worked fine.
>
>
>
> > In this instance I started with genconfig editing the files to get a
>
> > starting point. Can you explain the numbers and how a would choose the
> range
>
> > to add another files store ? Is it a simple as calculating the difference
>
> > from one range and adding that to last one used ?
>
>
>
> Yes it is! Just make sure that no handle ranges overlap with each other.
>
> For fairness, it would be best if all servers see roughly the same #
>
> Of handle ranges
>
> So that the handle allocator can fairly disperse the handles to all the
> servers.
>
> Hope this explains all your questions,
>
> Thanks,
>
> Murali
>
>
>
> >
>
> >
>
> >
>
> > Steve
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
>
>