[Pvfs2-users] Fwd: Okay some test data and question

[David Brown]

> Running the clients and servers on the same machine might actually
> hurt your performance a bit.  Since PVFS doesn't have a native
> quadrics method, maybe you save a lot of overhead skipping the
> tcp-over-quadrics stuff.

Yeah but I wasn't running over the quadrics I was using the
gig-ethernet however the data should be using the loopback then going
right to disk... One of these days we'll get an infiniband box we can
test on ;)

> It might be easier to see patterns if you held servers constant and
> increased the number of clients, or held clients constant while
> varying numbers of servers.  To visualize both you'd end up with a
> 3-d plot...

That'd be interesting but where would the data go for the odd clients
as you get more clients than servers? and how would you control that
to make the graph make sense?

> I'm looking at 255-test.csv.  That's 256 nodes (acting as servers and
> clients), each client running dd to write 10 GB to a single server?

Yeah each client dd'd a 10Gb file to itself through pvfs.

> I don't know why that workload would take about a minute for up to 64
> clients, speed up for 65-141 clients, and then go back to being slower
> for the rest of the runs, except for a cluster of fast runs at 173-183
> clients.
>
> Since you've got things set up so each client talks to a single server
> locally, we shouldn't be seeing network contention or switch
> wierdness.  Since you have a single client talking to a single server,
> the access pattern from each client should look pretty regular to
> pvfs2-server.   The tight bimodal distribution of results suggests...
> I don't know... fortunate placement of files on the storage device for
> some runs? http://www.coker.com.au/bonnie++/zcav

Well we ran on top of the xfs filesystem which normally gets about
300Mb/s to its raid5 set, and if it was fortunate placement of the
files then wouldn't it be more random, instead of bimodal?

Thanks,
- David Brown