=?gb2312?B?u9i4tA==?=:Re: [Pvfs2-users] Samba over PVFS2 performance
jy at huawei.com
Wed Oct 11 11:32:48 EDT 2006
Thanks for your help these days!
I do some improve of Samba write over PVFS2.
And 2 IO nodes,1 client node can provide 160MB/s write performance from Samba clients now.
But the Samba read performance is not good,just 80MB/s. :(
1.PVFS2 client seems not use the read cache,Can I use it?
2.Can I use the "readahead" in PVFS2?
3.What shall I do,modify any PVFS2 source code?
Please give me some advice,thanks so much.wait for your letter.
Thank you Murali.
> Hi Jiang,
> > HI Murali,
> > I test the write performance just on pvfs2 client with big
> blocks,the> performance seems good.
> > and I test samba with COPY form windows clients to local disk(JBOD)
> > without pvfs2,the local file system is XFS,and the write
> performance is
> > good too.about 100MB/s.
> > But when I add the pvfs2 file system,the write performance is
> bad,just> 20MB/s.
> Hmm.. This definitely seems to indicate that Samba over PVFS2 uses
> verysmall block sizes..
> > 1.you said " Due to the absence of any client-side caches,
> multiple round
> > trips ensues and performance drops.",shall I use the client-sid
> caches?How> can I use it ?
> Nope. We haven't implemented this yet.
> > 2.you said "Can you check if the smbd server uses pvfs2 writev
> function> instead of write? (perhaps by strace'ing the server?)",
> > How can I check it ?
> If the samba server is a UNIX userspace process, could you start
> up the
> samba server like so
> strace -f -o /tmp/log.txt -e file /path/to/smbd <arguments>
> AFter the server is started, remount samba on all clients and redo
> Check the contents of the file /tmp/log.txt and see if there are
> calls to
> writev or write()?If there are calls to write(), check the sizes
> of the
> system call arguments..
> Hopefully that will point us to the problems and possibly solutions.
> > Thank for help me very much!
> > Best regards
> > ----------------------------------------------
> > ----- Original Message -----
> > From: "Murali Vilayannur" <vilayann at mcs.anl.gov>
> > Cc: <pvfs2-users at beowulf-underground.org>
> > Sent: Monday, September 25, 2006 11:43 PM
> > Subject: Re: [Pvfs2-users] Samba over PVFS2 performance
> >> Hi Philips,
> >> > I met a Samba over PVFS2 prolem.So would anybody help me
> please?>> > I use the pvfs2 file system and export the files
> with Samba.The
> >> hardware are two JBOD with
> >> > 146G 10k FC disk, 4 pvfs2 IO servers,8 pvfs2 clients and some
> >> windows clients.Every IO server
> >> > manages 6+1 RAID5.the stripe size is 64k.
> >> > My problem is that How can we tuning the read/write
> performance of
> >> this system ?
> >> > this architecture,I test the performance with 2G
> files.and find
> >> the read performance is fast,but the
> >> > write performance is bad.Would you tell me what should
> be care
> >> about Samba over PVFS2?
> >> Can you verify whether PVFS2 performance (no Samba export, just
> native>> PVFS2) is bad on your setup first? What
> >> benchmark did you use to measure the performance on your system?
> >> Just wanted to make sure that it is a pvfs2 performance problem
> and not
> >> a
> >> samba tuning issue..
> >> I suspect what might happen with writes is that the VFS/smbd
> writes out
> >> single pages and hence PVFS2 sees only 4K/8K bytes at a time.
> >> Due to the absence of any client-side caches, multiple round trips
> >> ensues
> >> and performance drops.
> >> Can you check if the smbd server uses pvfs2 writev function
> instead of
> >> write? (perhaps by strace'ing the server?)
> >> thanks,
> >> Murali
More information about the Pvfs2-users