[Pvfs2-developers] Rebalancing after adding additional nodes

[Sam Lang]

Hi Ti,

Can you or did you try restarting the servers (after shutting down  
the clients)?

-sam

On Aug 26, 2006, at 1:04 PM, Ti Leggett wrote:

> Right. Here's the deal, when I started this whole process each one  
> of those nodes that is now full had like 28G free. So by copying  
> the file to a another location, removing the old file, and then  
> moving the file back in place did not result in a net increase of 0  
> like I think it should. I know which files are the largest and I  
> could probably move those off of the PVFS2 volume to local disk  
> temporarily, but if by trying to rebalance these files I'll end up  
> losing space, then I'm doing something wrong or something is going  
> wrong in PVFS2. Either way, I need to find out why this process is  
> using space instead of resulting of a net usage of 0 like it should.
>
> On Aug 26, 2006, at Sat,Aug 26, 12:06 PM, Robert Latham wrote:
>
>> On Sat, Aug 26, 2006 at 11:14:45AM -0500, Ti Leggett wrote:
>>> Well I was hoping I was doing something wrong :)
>>>
>>> Here's the statfs output.
>>
>>
>>> server: tcp://nightcrawler.ci.uchicago.edu:3334
>>> 	handles available: 858976089
>>> 	handles total    : 858993459
>>> 	bytes available  : 5894144
>>> 	bytes total      : 246131359744
>>> 	mode: serving only I/O data
>>
>> This is not so good: only 6 megs of free disk space.
>>
>>> server: tcp://rogue.ci.uchicago.edu:3334
>>> 	handles available: 858976089
>>> 	handles total    : 858993459
>>> 	bytes available  : 11202560
>>> 	bytes total      : 246131359744
>>> 	mode: serving only I/O data
>>
>> 11 megs here isn't so hot either
>>
>>> server: tcp://wolverine.ci.uchicago.edu:3334
>>> 	handles available: 858976089
>>> 	handles total    : 858993459
>>> 	bytes available  : 9928704
>>> 	bytes total      : 246131359744
>>> 	mode: serving only I/O data
>>
>> space is a little tight here, too
>>
>>
>> So we need to focus on those three first.  nightcrawler is the reason
>> you only have 5*8=40 MB free, but the other two aren't helping much
>> either:  Are those three the original servers?  Do they have a log
>> file or anything else consuming space in the pvfs2 storage-space
>> that's not part of PVFS2?
>>
>> Do we have a good way to reverse-map data files on disk to pvfs2
>> files?  If so, you could find what the largest datafiles on
>> nightcrawler, rouge, and wolverine were, and then free up some space
>> by moving them somehwere temporarily.
>>
>> I think we can figure this out.  Just might take a few rounds.
>>
>> ==rob
>>
>> -- 
>> Rob Latham
>> Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
>> Argonne National Labs, IL USA                B29D F333 664A 4280 315B
>>
>
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
>