[Pvfs2-developers] Rebalancing after adding additional nodes

[Ti Leggett]

For all interested, it appears that my rebalance script works great.  
It took a while, but in the end everything ended up being rebalanced  
and my free space is now distributed across all nodes evenly (went  
from a reported 200G free to 1.1T free). It'd probably be worth it to  
add a flag to disable all the checksumming to speed up the process on  
scratch filesystems.

On Aug 30, 2006, at Wed,Aug 30, 10:42 AM, Ti Leggett wrote:

> It does, but I did it so long ago, I forgot to do it. So this is  
> really my bad.
>
> On Aug 30, 2006, at Wed,Aug 30, 10:29 AM, Rob Ross wrote:
>
>> pvfs2-migrate-collection really ought to print out some message  
>> that indicates how to remove the old stuff once you're sure  
>> everything is ok.
>>
>> thanks Ti!
>>
>> rob
>>
>> Ti Leggett wrote:
>>> The culprit was finally found. Prior to adding the 4 new nodes I  
>>> upgraded to pvfs2-1.5.1 from 1.4.2. This required me to run pvfs2- 
>>> migrate-collection which made a backup of the pvfs2 data by  
>>> creating hard links in another directory. So when I removed the  
>>> original file the bstream files from the backup remained and  
>>> since the reference of the inode wasn't 0 the backup data  
>>> remained. Runnging pvfs2-migrate-collection --all --clean on all  
>>> the original nodes freed up lots of space. So now I have the  
>>> rebalance running again.
>>> On Aug 28, 2006, at Mon,Aug 28, 1:20 PM, Julian Martin Kunkel wrote:
>>>> Hi,
>>>> Maybe the old datafiles are not deleted properly ? Maybe we  
>>>> could try the
>>>> following to figure out if that is the case, choose a file on  
>>>> the old setting
>>>> (4 servers) (or create a new one lets say 100MByte) printing out  
>>>> the real file
>>>> names in the storage space with pvfs2-viewdist, in addition do a  
>>>> statfs. Then
>>>> balance this file on the new setting with 8 servers. Then look  
>>>> at the statfs
>>>> again. Also check the old  bytestream files in the dataspace  
>>>> again. Are they
>>>> still present ?
>>>> I think it might also be interesting to run the normal UNIX df  
>>>> command before
>>>> and after the migration on the server which should delete the  
>>>> file to verify
>>>> these results.
>>>>
>>>> Julian
>>>> _______________________________________________
>>>> Pvfs2-developers mailing list
>>>> Pvfs2-developers at beowulf-underground.org
>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2- 
>>>> developers
>>>>
>>> _______________________________________________
>>> Pvfs2-developers mailing list
>>> Pvfs2-developers at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>>
>