[Pvfs2-cvs] commit by slang in pvfs2/doc: pvfs2-tuning.tex
module.mk.in pvfs2-faq.tex tuning.tex
CVS commit program
cvs at parl.clemson.edu
Thu Oct 19 18:16:36 EDT 2006
Update of /projects/cvsroot/pvfs2/doc
In directory parlweb1:/tmp/cvs-serv5758/doc
Modified Files:
Tag: WALT3
module.mk.in pvfs2-faq.tex
Added Files:
Tag: WALT3
pvfs2-tuning.tex
Removed Files:
Tag: WALT3
tuning.tex
Log Message:
reverse merge of HEAD to WALT3 branch.
--- /dev/null 2004-06-24 14:04:38.000000000 -0400
+++ pvfs2-tuning.tex 2006-10-19 18:16:36.000000000 -0400
@@ -0,0 +1,154 @@
+\documentclass[11pt,letterpaper]{article}
+\usepackage{html}
+\usepackage{charter}
+\pagestyle{empty}
+
+\topmargin 0.0in
+\textwidth 6.5in
+\textheight 9.0in
+\columnsep 0.25in
+\oddsidemargin 0.0in
+\evensidemargin 0.0in
+\headsep 0.0in
+\headheight 0.0in
+
+\title{PVFS Tuning}
+\author{ PVFS Development Team }
+
+\begin{document}
+\maketitle
+
+\tableofcontents
+\thispagestyle{empty}
+
+\section{Introduction}
+
+The default settings for PVFS (those provided and in the source code
+and added to the config files by \texttt{pvfs2-genconfig})
+provide good performance on most systems and for a wide variety of workloads.
+This document describes system level and PVFS specific parameters
+that can be tuned to improve
+performance on specific hardware, or for specific workloads and usage
+scenarios.
+
+In general performance tuning should begin with the available hardware
+and system software, to maximize the bandwidth of the network and
+transfer rates of the storage hardware. From there, PVFS server
+parameters can be tuned to improve performance of the
+entire system, especially
+if specific usage scenarios are targetted. Finally, file system
+extended attributes and hints can be tweaked by different users to
+improve individual performance within a system with varying workloads.
+
+Some (especially system level) parameters can be tuned to provide better
+performance without sacrificing another property of the system.
+Tuning some parameters though, may have a direct effect on the
+performance of other usage scenarios, or some other property of the
+system (such as durability).
+Our discussion of performance tuning will include the tradeoffs
+that must be made during the tuning process, but the final decisions are best
+made by the administrators to determine the optimal
+setup that meets the needs of their users.
+
+\section{Cluster Partitioning}
+
+For users that have one use case, and a generic cluster, what's the best
+partition of compute/IO nodes? Is this section needed?
+
+\section{Storage}
+
+\subsection{Server Configuration}
+
+How many IO servers? \footnote{The FAQ already answers this to some
+degree}
+How many MD servers?
+Should IO and MD servers be shared?
+
+\subsection{Local File System}
+
+\begin{itemize}
+\item ext3
+\item xfs
+\end{itemize}
+
+\subsection{Disk Synchronization}
+
+The easiest way to see an improvement in performance is to set the
+\texttt{TroveSyncMeta} and \texttt{TroveSyncData} attributes to ``no''
+in the \texttt{<StorageHints>} section. If those attributes are set to
+``no'' then Trove will read and write data from a cache and not the
+underlying file. Performance will increase greatly, but if the server
+dies at some point, you could lose data. At this point in PVFS2
+development, server crashes are rare outside of hardware failures.
+PVFS2 developers should probably leave these settings to ``yes''. If
+PVFS2 hosts the only copy of your data, leave these settings to ``yes''.
+Otherwise, give ``no'' a shot.
+
+Sync or not, metadata, data
+ coalescing
+
+ distributed metadata
+
+\subsection{Metadata}
+\subsubsection{Coalescing}
+
+\subsection{Data}
+
+\section{Networks}
+
+\subsection{Network Independent}
+
+\begin{enumerate}
+\item Unexpected message size
+\item Flow Parameters
+\end{enumerate}
+
+\begin{itemize}
+\item buffer size
+\item count
+\end{itemize}
+
+\subsection{TCP}
+
+\subsubsection{Kernel Parameters}
+\subsubsection{Socket Buffer Sizes}
+\subsubsection{Listening Backlog (?)}
+
+\subsection{Infiniband}
+\subsection{Myrinet Express}
+
+\section{VFS Layer}
+
+\subsection{Maximum I/O Size}
+
+\subsection{Workload Specifics}
+
+\subsection{Extended Attributes}
+
+\subsubsection{Directory Hints}
+
+\begin{itemize}
+\item Number of Datafiles
+\item Stripe Size
+\item Distribution
+\end{itemize}
+
+\section{Workloads}
+
+\subsection{Small files}
+\subsection{Large Files}
+\subsection{Concurrent IO}
+
+\section{Benchmarking}
+
+\begin{itemize}
+\item mpi-io-test
+\item mpi-md-test
+\end{itemize}
+
+\section{References}
+
+\end{document}
+
+
+% vim: tw=72
Index: module.mk.in
===================================================================
RCS file: /projects/cvsroot/pvfs2/doc/module.mk.in,v
diff -p -u -r1.5 -r1.5.36.1
--- module.mk.in 29 Jun 2004 20:18:45 -0000 1.5
+++ module.mk.in 19 Oct 2006 22:16:36 -0000 1.5.36.1
@@ -4,7 +4,8 @@ DOCSRC +=\
$(DIR)/pvfs2-guide.tex \
$(DIR)/pvfs2-faq.tex \
$(DIR)/pvfs2-ha.tex \
- $(DIR)/pvfs2-status.tex
+ $(DIR)/pvfs2-status.tex \
+ $(DIR)/pvfs2-tuning.tex
# some extra dependency information
doc/pvfs2-guide.pdf: doc/terms.tex doc/intro.tex doc/basics.tex doc/code-tree.tex doc/io-api-terms.tex
Index: pvfs2-faq.tex
===================================================================
RCS file: /projects/cvsroot/pvfs2/doc/pvfs2-faq.tex,v
diff -p -u -r1.47.2.1 -r1.47.2.2
--- pvfs2-faq.tex 18 Sep 2006 15:04:56 -0000 1.47.2.1
+++ pvfs2-faq.tex 19 Oct 2006 22:16:36 -0000 1.47.2.2
@@ -644,6 +644,23 @@ kernels. However, on older 2.4 kernels t
the page-cache and thus there is a real possibility of the data being served stale.
Therefore users of the {\tt sendfile} system call are warned to be wary of this detail.
+\subsection{Trove-dbpf metadata format version mismatch!}
+
+In PVFS2-1.5.0 or newer the format of the metadata storage has change from
+previous versions (1.4.0 or earlier). This affects users that have created
+file systems with the earlier versions of pvfs2, and wish to upgrade to the
+most recent version. We've provided a migration tool that must be run
+(a one-time only procedure) to convert the file system from the old format
+to the new one. The migration tool can be used as follows:
+\begin{verbatim}
+$PVFS2_INSTALL/bin/pvfs2-migrate-collection --all fs.conf server.conf-<hostname>
+\end{verbatim}
+This command finds all the pvfs2 storage collections specified in the
+configuration files and migrates them to the new format. Instead of
+using {\tt --all}, the option {\tt --fs} can be used to specify the name of
+the storage collection that needs to be migrated (usually there's only
+one storage collection, with the default name of 'pvfs2-fs').
+
%
% PERFORMANCE
%
--- tuning.tex 2006-10-19 18:16:36.000000000 -0400
+++ /dev/null 2004-06-24 14:04:38.000000000 -0400
@@ -1,20 +0,0 @@
-\section{Tuning PVFS2}
-\label{sec:tuning}
-
-The default settings -- those provided by \texttt{pvfs2-genconfig} --
-will give correct behavior and decent performance. You can use more
-aggressive settings that will yield better performance, but you should
-understand the consequences of altering the defaults.
-
-\subsection{Sync Mode}
-
-The easiest way to see an improvement in performance is to set the
-\texttt{TroveSyncMeta} and \texttt{TroveSyncData} attributes to ``no''
-in the \texttt{<StorageHints>} section. If those attributes are set to
-``no'' then Trove will read and write data from a cache and not the
-underlying file. Performance will increase greatly, but if the server
-dies at some point, you could lose data. At this point in PVFS2
-development, server crashes are rare outside of hardware failures.
-PVFS2 developers should probably leave these settings to ``yes''. If
-PVFS2 hosts the only copy of your data, leave these settings to ``yes''.
-Otherwise, give ``no'' a shot.
More information about the Pvfs2-cvs
mailing list