1 NetPIPE Network Protocol Independent Performance Evaluator, Release 2.4
2 Copyright 1997, 1998, 1999 Iowa State University Research Foundation, Inc.
3
4 $Id: README,v 1.1 2003/03/28 18:14:44 robbiew Exp $
5
6 This program is free software; you can redistribute it and/or modify
7 it under the terms of the GNU General Public License as published by
8 the Free Software Foundation. You should have received a copy of the
9 GNU General Public License along with this program; if not, write to the
10 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
11
12 The URL for this document:
13
14 ftp://ftp.scl.ameslab.gov/pub/netpipe/README
15
16 Getting NetPIPE
17 ---------------
18
19 The NetPIPE implementation in C can be found at:
20
21 ftp://ftp.scl.ameslab.gov/pub/netpipe/netpipe-2.4.tar.gz
22
23 The source code for NetPIPE 2.4 is provided as a gzipped tar archive,
24 which can be uncompressed with "gunzip netpipe-2.4.tar.gz" (or "gzip
25 -d netpipe-2.4.tar.gz"), and then extracted from the uncompressed
26 archive with the command "tar xvf netpipe-2.4.tar". If you do not
27 have the gzip program, it can be obtained as:
28
29 ftp://prep.ai.mit.edu/pub/gnu/gzip-1.2.4.tar
30
31 Similarly, the NetPIPE implementation in Java can be found at:
32
33 ftp://ftp.scl.ameslab.gov/pub/netpipe-Java-1.0.tar.gz
34
35 The instructions that follow apply to the C implementation of
36 NetPIPE.
37
38 What is NetPIPE?
39 ----------------
40
41 NetPIPE is a protocol independent performance tool that encapsulates
42 the best of ttcp and netperf and visually represents the network
43 performance under a variety of conditions. By taking the end-to-end
44 application view of a network, NetPIPE clearly shows the overhead
45 associated with different protocol layers. Netpipe answers such
46 questions as: how soon will a given data block of size k arrive at its
47 destination? Which network and protocol will transmit size k blocks
48 the fastest? What is a given network's effective maximum throughput
49 and saturation level? Does there exist a block size k for which the
50 throughput is maximized? How much communication overhead is due to the
51 network communication protocol layer(s)? How quickly will a small (< 1
52 kbyte) control message arrive, and which network and protocol are best
53 for this purpose?
54
55 For a paper fully describing NetPIPE and sample investigation of
56 network performance issues using NetPIPE, see
57 http://www.scl.ameslab.gov/netpipe/paper/full.html.
58
59 Building NetPIPE
60 ----------------
61
62 NetPIPE is provided with protocol-specific shims for TCP (using the
63 Berkeley sockets interface), MPI, and PVM. If you do not have MPI or
64 PVM, don't worry; TCP is the typical shim used. It should be easy to
65 write new interfaces for other protocols based on the examples shown
66 by the TCP, MPI and PVM interfaces.
67
68 NetPIPE requires an ANSI C compiler.
69
70 Review the provided Makefile and change any necessary settings, such
71 as the CFLAGS compiler flags, required extra libraries, and MPI or PVM
72 library & include file pathnames if you have these communication
73 libraries. If you want to turn on getrusage calls to get CPU time
74 required for communication, add "-DHAVE_GETRUSAGE" to the CFLAGS line
75 in the Makefile.
76
77 Compile NetPIPE with the desired communication interface by using the
78 command "make TCP", "make MPI", or "make PVM" as appropriate,
79 corresponding to the executable files NPtcp, NPmpi, or NPpvm
80 respectively.
81
82 Consult the appropriate section below for details on running NetPIPE
83 over TCP, MPI, or PVM, and the following section on interpreting the
84 results.
85
86 Running NPtcp
87 -------------
88
89 For TCP, run a NetPIPE receiver on one computer by issuing the command
90 "NPtcp -r". Run a NetPIPE sender on another computer by issuing the
91 command "NPtcp -t -h <receiver's address> -o <output file> -P" and any
92 other options as appropriate (each option affects only the process on
93 which it is specified -- options are not negotiated between the
94 transmitter and the receiver):
95
96 -A: specify buffers alignment e.g. "-A 4096"
97
98 -a: asynchronous receive (a.k.a. preposted receive)
99 This option currently has no effect on TCP
100
101 -b: specify send and receive TCP buffer sizes e.g. "-b 32768"
102
103 -h: specify hostname of receiver e.g. "-h mumblehost"
104
105 -i: specify increment step size e.g. "-i 64"
106 Default is exponential increment calculated at runtime
107
108 -l: lower bound (start value for block size) e.g. "-l 1"
109
110 -O: specify buffer offset e.g. "-O 127"
111
112 -o: specify output filename e.g. "-o output.txt"
113
114 -P: print real-time results on stdout
115
116 -p: specify port e.g. "-p 5150"
117
118 -s: stream option (default mode is "ping pong")
119 If this option is used, it must be specified on both
120 the sending and receiving processes
121
122 -u: upper bound (stop value for block size) e.g. "-u 1048576"
123
124 Running NPmpi
125 -------------
126
127 For MPI, how you run NPmpi may depend on the MPI implementation you
128 are using. Assuming you are using the "p4" device (for a cluster of
129 individual systems interconnected using TCP/IP) in the Argonne MPICH
130 implementation, you could run NPmpi one of two ways.
131
132 If your system's default machine file begins with the two
133 names of the systems you want to test, use "mpirun -np 2
134 NPmpi", followed by any of the NetPIPE options listed below.
135
136 Otherwise, create a file that contains the host names of the
137 two systems you want to include in the test, one host name on
138 each line (assume the file is named "machines.p4"). Then, use
139 the command "mpirun -machinefile machines.p4 -np 2 NPmpi",
140 followed by any of the NetPIPE options listed below.
141
142 To find out how to run NPmpi using any other implementation of MPI,
143 please consult the implementation's documentation.
144
145 The NetPIPE options for MPI are:
146
147 -A: specify buffers alignment e.g. "-A 4096"
148
149 -a: asynchronous receive (a.k.a. preposted receive)
150 May not have any effect, depending on your MPI
151 implementation
152
153 -i: specify increment step size e.g. "-i 64"
154 Default is exponential increment calculated at runtime
155
156 -l: lower bound (start value for block size) e.g. "-l 1"
157
158 -O: specify buffer offset e.g. "-O 127"
159
160 -o: specify output filename e.g. "-o output.txt"
161
162 -P: print real-time results on stdout
163
164 -s: stream option (default mode is "ping pong")
165 If this option is used, it must be specified on both
166 the sending and receiving processes
167
168 -u: upper bound (stop value for block size) e.g. "-u 1048576"
169
170 Running NPpvm
171 -------------
172
173 First, start PVM with the command "pvm" on one machine and a second
174 machine with the PVM command "add <othermachine>", where
175 <othermachine> is the name of the other computer to include in the
176 test. Exit the PVM command line interface. Start the receiver
177 process on one of the machines with the command "NPpvm -r". Finally,
178 start the transmitter process on the other machine with the command
179 "NPpvm -t -o <output file> -P" and any other options as appropriate
180 (each option affects only the process on which it is specified --
181 options are not negotiated between the transmitter and the receiver):
182
183 -A: specify buffers alignment e.g. "-A 4096"
184
185 -a: asynchronous receive (a.k.a. preposted receive)
186 This option has no effect on PVM
187
188 -i: specify increment step size e.g. "-i 64"
189 Default is exponential increment calculated at runtime
190
191 -l: lower bound (start value for block size) e.g. "-l 1"
192
193 -O: specify buffer offset e.g. "-O 127"
194
195 -o: specify output filename e.g. "-o output.txt"
196
197 -P: print real-time results on stdout
198
199 -s: stream option (default mode is "ping pong")
200 If this option is used, it must be specified on both
201 the sending and receiving processes
202
203 -u: upper bound (stop value for block size) e.g. "-u 1048576"
204
205
206 Interpreting the Results
207 ------------------------
208
209 NetPIPE's output file contains five columns: time to transfer the block,
210 bits per second, bits in block, bytes in block, and variance. These
211 columns may be graphed to represent and compare the network's
212 performance. For example, the "network signature" graph can be
213 created by graphing the time column versus the bits per second column
214 (see the NetPIPE report at the URL above for the details why this
215 graph is important and how to interpret it). The more traditional
216 "throughput versus block size" graph can be created by
217 graphing the bytes column versus the bits per second column.
218
219 See http://www.scl.ameslab.gov/Projects/ClusterCookbook/nprun.html for
220 a detailed tutorial on running NetPIPE and graphing the results.
221
222 Help
223 ----
224
225 NetPIPE is currently maintained by Guy Helmer. Email
226 "ghelmer (a] scl.ameslab.gov" or call 515-294-9469 for help or
227 suggestions.
228
229 Changes
230 -------
231
232 version 2.4 (12/16/99)
233 * Add getrusage calls to get CPU time used by communication if
234 HAVE_GETRUSAGE is defined (be aware that no studies have been
235 conducted to test the accuracy of results across different systems)
236 * Use "unsigned int" instead of "unsigned long" to communicate 32-bit
237 integers in TCP.c (this solves interoperability problems between
238 Compaq/DEC Alphas and most other systems)
239 * Add dummy "echo" commands after TCP, MPI, and PVM targets in the
240 Makefile. Some implementations of make(1) (such as those found
241 in Linux distributions) interpret the targets with no following
242 statements as a rule to do something silly like
243 "cc -O -o TCP TCP.c" after the dependency is satisfied.
244
245 version 2.3 (9/24/98)
246 * Add PVM interface contributed by Clark E. Dorman <dorman (a] s3i.com>
247
248 * Revamp README file with instructions for NPmpi and NPpvm, and
249 clarify some instructions for NPtcp
250
251 version 2.2 (8/21/98):
252 * Carefully check all return values from write(2) and read(2)
253 system calls in TCP.c. Handle short reads properly. Make the Sync()
254 function transmit and receive a useful string which can be
255 checked for validity.
256
257 * Correct the overloading of SendTime() and RecvTime() functions
258 by breaking out SendRepeat() and RecvRepeat() as separate
259 functions.
260
261 * Handle systems whose accept(2) system call does not carry socket
262 options over from the listening socket. In particular, set the
263 TCP_NODELAY flag and socket buffers on an accepted socket.
264