[Pvfs2-users] I'm in the dark, need someone to shed some light

[belcampo]

Rob Ross wrote:
> Hi Henk,
> 
> Please be sure to CC pvfs2-users on future emails.
Sorry stupid I didin't do that.
> 
> Without any additional information, my guess is that every application 
> you're using in this workflow performs very small I/Os. These operations 
> are passed into the kernel, back out to pvfs2-client, across the network 
> and received by the PVFS server, who then performs I/O on your 
> application's behalf. If operations are particularly small, this can be 
> a lot of overhead.
Top tells me that server-side 99.6% idle client-side 95% idle, how could 
I determine what is causing the abnormal delays.
Starting to play a dvd takes about 12 secs. After a few seconds it 
starts stuttering.
By nfs of 1 of the servers it takes about 1 sec to start and never stutters.
> 
> Other networked file systems can hide some of this latency by caching 
> data (either coherently or not) on the client. PVFS does not do this, so 
> each little operation goes across the wire. 
Can this be investigated with some networktool and if yes, how ?
> There's really no advantage 
> to using a parallel file system for the workload you have described, 
But should the disadvantage be in this order of magnitude ?
> unless you're planning on having a lot of systems doing this process in 
> parallel and want a single place to store the output.
> 
> What sort of network do you have in this system? What sort of nodes are 
> you using for the PVFS servers?
All AMD 4000+ systems with 1Gb networkcards and 320GB disk in each or them.
Copying from to clients to these 3 servers is > 100MB/sec pretty close 
to what Gb ethernet can do.

Regards


Henk Schoneveld