=?gb2312?B?u9i4tA==?=:Re: [Pvfs2-users] Samba over PVFS2 performance

[jiangyi 60012480]

HI Murali
Thanks for your help these days!
I do some improve of Samba write over PVFS2.
And 2 IO nodes,1 client node can provide 160MB/s write performance from Samba clients now.

But the Samba read performance is not good,just 80MB/s.  :(
1.PVFS2 client seems not use the read cache,Can I use it?
2.Can I use the "readahead" in PVFS2?
3.What shall I do,modify any PVFS2 source code?

Please give me some advice,thanks so much.wait for your letter.
Thank you Murali.


Best regards
--------------------------------------------------


> Hi Jiang,
> 
> > HI Murali,
> > I test the write performance just on pvfs2 client with big 
> blocks,the> performance seems good.
> > and I test samba with COPY form windows clients to local disk(JBOD)
> > without pvfs2,the local file system is XFS,and the write 
> performance is
> > good too.about 100MB/s.
> > But when I add the pvfs2 file system,the write performance is 
> bad,just> 20MB/s.
> 
> Hmm.. This definitely seems to indicate that Samba over PVFS2 uses 
> verysmall block sizes..
> 
> > 1.you said " Due to the absence of any client-side caches, 
> multiple round
> > trips ensues and performance drops.",shall I use the client-sid 
> caches?How> can I use it ?
> 
> Nope. We haven't implemented this yet.
> 
> > 2.you said "Can you check if the smbd server uses pvfs2 writev 
> function> instead of write? (perhaps by strace'ing the server?)",
> >    How can I check it ?
> 
> If the samba server is a UNIX userspace process, could you start 
> up the
> samba server like so
> strace -f -o /tmp/log.txt -e file /path/to/smbd <arguments>
> AFter the server is started, remount samba on all clients and redo 
> yourtests.
> 
> Check the contents of the file /tmp/log.txt and see if there are 
> calls to
> writev or write()?If there are calls to write(), check the sizes 
> of the
> system call arguments..
> Hopefully that will point us to the problems and possibly solutions.
> THanks,
> Murali
> 
> > Thank for help me very much!
> >
> > Best regards
> > ----------------------------------------------
> >
> > ----- Original Message -----
> > From: "Murali Vilayannur" <vilayann at mcs.anl.gov>
> > Cc: <pvfs2-users at beowulf-underground.org>
> > Sent: Monday, September 25, 2006 11:43 PM
> > Subject: Re: [Pvfs2-users] Samba over PVFS2 performance
> >
> >
> >> Hi Philips,
> >>
> >> >     I met a Samba over PVFS2 prolem.So would anybody help me 
> please?>> >     I use the pvfs2 file system and export the files 
> with Samba.The
> >> hardware are two JBOD with
> >> >     146G 10k FC disk, 4 pvfs2 IO servers,8 pvfs2 clients and some
> >> windows clients.Every IO server
> >> >     manages 6+1 RAID5.the stripe size is 64k.
> >> >     My problem is that How can we tuning the read/write 
> performance of
> >> this system ?
> >> >     this architecture,I test the performance with 2G 
> files.and find
> >> the read performance is fast,but the
> >> >      write performance is bad.Would you tell me what should 
> be care
> >> about Samba over PVFS2?
> >>
> >> Can you verify whether PVFS2 performance (no Samba export, just 
> native>> PVFS2) is bad on your setup first? What
> >> benchmark did you use to measure the performance on your system?
> >> Just wanted to make sure that it is a pvfs2 performance problem 
> and not
> >> a
> >> samba tuning issue..
> >>
> >> I suspect what might happen with writes is that the VFS/smbd 
> writes  out
> >> single pages and hence PVFS2 sees only 4K/8K bytes at a time.
> >> Due to the absence of any client-side caches, multiple round trips
> >> ensues
> >> and performance drops.
> >> Can you check if the smbd server uses pvfs2 writev function 
> instead of
> >> write? (perhaps by strace'ing the server?)
> >> thanks,
> >> Murali
> >>
> >
> >
> 
> 
>