[PVFS-developers] Re: [PVFS-users] Recompile pvfs module for SuSE 2.4.19-NUMA

[Rob Ross]

Oh, I misunderstood what you were saying before.  I thought that the "few 
MB" was your file size, not your access size.

How many I/O servers do you have in the system?  How much memory do you 
have in your client?

These four /proc values are the default and maximum socket buffer sizes, 
if I understand things correctly:
  /proc/sys/net/core/rmem_default
  /proc/sys/net/core/rmem_max
  /proc/sys/net/core/wmem_default
  /proc/sys/net/core/wmem_max

Also, you might want to adjust the following in your iod.conf file (see 
man pages for details): socket_buf, access_size.

About where does the dropoff start to occur?

Regards,

Rob

On Mon, 8 Mar 2004, Claude Pignol wrote:

> Thanks Rob,
> 
> Another fact:
> I found that the read works very well with 64K I/O: the read speed is 
> better than the write speed.
> The read perf start degrading when I increase the I/O size
> 
> I agree that there is a starting cost but there is the read ahead mechanism
> that speed up the disk access.
> I am testing with file of min 1GB
> 
> I have tested with dynamic buffering (the default) and the static buffering.
> Same problem.
> How do you increase tcp buffer size?
> net.ipv4.tcp_rmem
> net.ipv4.tcp_wmem
> net.ipv4.tcp_mem
> 
> 
> Claude
> 
> 
> Rob Ross wrote:
> 
> >Hi Claude,
> >
> >Sorry we didn't get back to you sooner.  I'm glad that the kernel update 
> >fixed the problem.
> >
> >What block size (bs=XXX) are you using in your tests?
> >
> >Note that when reading no I/O can start until data is read off disk, while 
> >in the write case data can start moving right away.  So you may just be 
> >seeing startup costs.
> >
> >You could look at increasing TCP buffer sizes on your system as a first 
> >step.
> >
> >Regards,
> >
> >Rob
> >
> >On Mon, 8 Mar 2004, Claude Pignol wrote:
> >
> >  
> >
> >>Greetings,
> >>
> >>An upgrade to 2.4.21 fixes the problem.
> >>Compile and start OK.
> >>I have noticed a performance problem in reading from PVFS.
> >>With big I/O (few MB) reading is around 1/3 of the performance of writing.
> >>Pvfs deamons with default parameters
> >>Reading/Writing from on node to pvfs using dd.
> >>I have verified the disk performance of all the 10 I/O nodes
> >>I have also verified the network perf to all the nodes.
> >>What is the best strategy/tools to address this kind of problem?
> >>Thanks
> >>
> >>
> >>Claude Pignol wrote:
> >>
> >>    
> >>
> >>>Greetings,
> >>>
> >>>I try to do a benchmark of pvfs with the SuSE 2.4.19-NUMA kernel
> >>>to compare with the SuSE 2.4.19-SMP kernel.
> >>>No problem to compile and load the pvfs.o module with the SMP kernel
> >>>
> >>>With the NUMA kernel I get 3 undefined symbols when I try to load the 
> >>>module
> >>>pvfs.o: unresolved symbol __pollwait
> >>>pvfs.o: unresolved symbol mem_map
> >>>pvfs.o: unresolved symbol iget4
> >>>
> >>>The kernel source is installed.
> >>>Any idea?
> >>>Thanks in advance
> >>>Claude
> >>>
> >>>
> >>>_______________________________________________
> >>>PVFS-users mailing list
> >>>PVFS-users at www.beowulf-underground.org
> >>>http://www.beowulf-underground.org/mailman/listinfo/pvfs-users
> >>>
> >>>      
> >>>
> >>_______________________________________________
> >>PVFS-developers mailing list
> >>PVFS-developers at www.beowulf-underground.org
> >>http://www.beowulf-underground.org/mailman/listinfo/pvfs-developers
> >>
> >>
> >>    
> >>
> >
> >  
> >
> 
> -- 
> 
> 
>