[PVFS-users] Re: Quick Question

[Robert Ross]

Jeff,

Good question!  This should be in the FAQ.

First, let's talk about the number of open FDs under linux.  It is
actually a dynamically settable value, but it defaults to 1024 under 2.2.
If you look at the output of "limit" (under tcsh) or "ulimit -a" (under
bash), you'll see the 1024 value (or whatever is set on your machine).

In the PVFS code we use the value from /usr/include/linux/fs.h, which is
1048576.  There is some really poor allocation going on WRT this value
also, I now notice :(.  Something that should be fixed.

So in theory you could have TONS of open files.  However, you can't
really, because we use up FD when we open connections to I/O servers all
over the place.  So really the practical limit of open files for a single
client under PVFS is N/(I+2), where "I" is the number of I/O nodes in your
system and "N" is the number of open files for Linux.  The constant value
comes from the connection to the manager (1 more FD) and an FD used as a
placeholder.

So, for example in a system with 8 I/O nodes and default resource limits,
I'd expect you could open as many as 100 PVFS files simultaneously.  With
48 I/O nodes, you're down to 10 PVFS files.

Obviously there is room for improvement in our resource utilization :).

Rob

(I'm CC'ing this to the pvfs-users list for posterity.)

On Thu, 26 Oct 2000, Jeffrey B Layton wrote:

>    Is there a limit to the number of open files an application can
> have when using the PVFS library (pvfs_open()  )? I know under Linux
> the usual number is 256.