[Pvfs2-users] Confusion in size

[Phil Carns]

Hi Abhinav,

To summarize, in total it sounds like you have PVFS servers running on 
14 nodes.  Most of those nodes only have 60 GB hard drives, but one of 
them has 640 GB of storage.

PVFS doesn't have any mechanism to account for unbalanced storage on 
each of the servers.  If any one of your servers runs out of space then 
the PVFS volume is pretty much out of space as well.  So when it 
calculates the total space on the file system (for df) it uses this 
algorithm:

<capacity of smallest server> * <number of servers>

... which gets you to the 850 GB value that df is reporting.

As far as the 52% being allocated already, this probably means that on 
at least some of your servers, the server's storage directory is on the 
same partition as your root file system (or some other data).  That 
means that things external to PVFS are counting against the capacity on 
some servers.  There is nothing wrong with that necessarily.  Most 
likely one or more of your machines has used up half of its 60 GB 
already and PVFS therefore only has maybe 30 GB * 14 available out of 
the 60 GB * 14 total.

If you want more detail about how much space is available on each 
individual server, you can use the pvfs2-statfs command.

There isn't any way to set a limit on each server from the PVFS 
perspective.  You could do it at the node level by using a separate 
partition for the PVFS data, though.

-Phil

Abhinav Chawade wrote:
> I have a 14 node cluster with 1 storage server 1 head node and 12 
> computational node running centos 5.2
> I deployed pvfs2 on all of them and head and storage node are metadata 
> servers. Each computational node has 60GB hard drive and the storage 
> server has 640 GB storage.
> When i mount the file system and ping it, all servers are running. I try 
> to see how much free space is left using df command. df output shows all 
> in all 850 GB storage out of which 52% is allocated already. There isn't 
> a single file in pvfs2 mounted partition. How can i check size of pvfs2 
> space i created? Is there a way to set limit on storage on each server?
> 
> Abhinav
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users