xref: /external/netperf/Release_Notes

These are the Release Notes for Revision 2.7.0 of netperf:

*) Add bits/s (-f b) and Bytes/s (-f B) as selectable throughput
   units. This may make life easier for folks doing post-processing of
   things like interim results.

*) Miscelaneous fixes

*) Much of the now-seeming kruft for things peripheral to netperf's
   core mission - have been removed though the code itself remains in
   the repository.  So, things like looking-up egress interface,
   driver/slot information etc have been disabled in a manner
   requiring more than just a ./configure to bring-back.

   It wasn't clear those features were being used.  This is a test of
   that hypothesis.

These are the Release Notes for Revision 2.6.0 of netperf:

*) Initial pass at support for --enable-intervals (WANT_INTERVALS) for
   Windows, courtesy of Jonathan Cook.

*) When in demo mode (./configure --enable-demo and a global -D
   <interval> option netperf will make sure it emits "one last
   interval result" when the test is terminated.  This should assist
   when post-processing results through the likes of rrdtool when
   there is a slow-down in the performance just at the end that would
   have stretched the interval to beyond the test termination.

*) A fix to have the AF_UNIX tests realize that the value for "take
   the system default" socket buffer size became -1 years ago.  Bug
   found by Eric Dumazet.

*) Include a patch from Dave Taht to enable symbolic manipulation of
   IP_TOS values.

*) Include a patch from Sachar Raindel to enable the omni tests to get
   ENOBUFS under Linux when the socket buffer is larger than the tx
   queue of the egress interface.  This will help preclude netperf's
   reporting a larger than link-rate send-side figure.

*) Fix a problem with late checking of the return from select() in
   src/netserver.c. Reported by Waqar Sheikh.

*) A new global -Z option has been added to netperf and netserver.
   This takes as an argument a passphrase.  In the case of netserver
   it will expect a control message with the passphrase as the first
   thing it receives on the control connection.  If netserver does not
   receive a control message with the passphrase it will close the
   control connection and move-on.  If the netserver receives a
   control message with a passprhase when it is not lookign for one,
   it will be ignored. There is at present a 20 second timeout on the
   attempted receipt of the request message. In the case of netperf,
   the passphrase will be the first thing sent on the control
   connection.  There is no response to a passphrase control message.

*) Demo mode output format will now track the omni output format. So,
   if the omni ouput format is CSV then the interim results will be
   emitted in csv. Likewise for keyval.  If the mode is human (default
   and test-specific -O) then the output remains unchanged.  Keyval
   output includes the count of interval, with a mind towards being
   able to source it in shells and whatnot.  Subject to change without
   notice.

*) A patch to correctly handle IPv6 addresses in the control messages,
   courtesy of Bjoern Zeeb.

*) The global -F option can now be used specify a local and/or remote
   fill file.

*) It is now possible to set/get the TCP congestion control algorithm
   being used by either end of the test connection when using the omni
   code.  The output selectors are LOCAL_CONG_CONTROL and
   REMOTE_CONG_CONTROL and setting is via the test-specific -K option.

*) Stop leaking file descriptors when looking-up probable egress
   interface names and I/O slot numbers.

*) The global -Y option can be used to set IP_TOS on those platforms
   which support it.  Since this is specific to IP (v4 or v6) it may
   move to a test-specific otion in the future.  It is presently
   global for foolish consistency with the -y option to set
   SO_PRIORITY.

*) The global -y option can be used to set SO_PRIORITY on those
   platforms which support it.  Based on patches from Amir Vidai.

*) The control message size has been increased from 256 bytes to 512
   bytes. THIS WILL BREAK COMPATABILITY WITH PREVIOUS VERSIONS OF
   NETPERF. However, we need more room on the pinhead on which the
   angels dance.

*) Make the "sum" field of the histogram structure a 64 bit int to
   avoid having it wrap-around on tests where the sum of all the
   measured latencies was larger than 31 bits. This was causing
   statistics like stddev to go negative in some cases.

*) If the time delta between two events is negative, do not bother
   doing any math with it in the histogram/statistics code, just
   increment the ridiculous count and move-on.

*) Fixed a bug which caused local transport retransmissions to be
   reported as -1 even though the getsockopt() call was
   successful. (Linux).  Later included remote transport
   retransmissions.

*) The src/nettest_omni.c and re-written src/netserver.c code are now
   known to have compiled under Windows 7 x64 with the Microsoft
   WDK. There remains a timing issue with confidence intervals which
   is yet to be addressed, and may have been there for ages. Netserver
   has been run as a non-spawning (-f) server, netperf has been run,
   both have run "classic" and "omni" tests.

These are the Release Notes for Revision 2.5.0 of netperf:

*) Add a new -N option to netserver which will suppress all creation
   of debug files and so debugging output.  While this would put a
   serious crimp in debugging a problem in netserver, it will enable
   folks using small embedded systems to avoid soaking-up their /tmp
   filesystem with clutter.

*) A refactoring and partial re-write of the src/netserver.c code to
   untangle years of accumulated spaghetti code.  Included is the
   ability to not daemonize netserver when launched from the
   command-line (-D) and also to not fork/spawn child processes upon
   the acceptance of a control connection (-f).  Combined, the two
   options will cause netserver to remain in the forground and not
   spawn children - in effect netserver will handle only one test at a
   time.

*) As it has been two years since the defect in Solaris getaddrinfo()
   was submitted, it should be the case that a fix is available, so it
   should no longer be necessary to "hide" the "Hey your platform's
   getaddrinfo() call is buggy" warning.  Consequently, it is no
   longer being suppressed.

*) A new global command line option - -S - has been added to enable
   setting of SO_KEEPALIVE on the data socket.  This will affect the
   netperf side of the "classic" netperf tests, and will also affect
   the netserver side of an "omni" or migrated classic test as only
   the control message for the omni tests has the requisite flags
   field to communicate the desire to set SO_KEEPALIVE.

   Ostensibly, this may help when netperf is (ab)used in functional
   testing situations and netservers end-up orphaned and out in the
   cold because their corresponding netperfs went away and the
   notification was lost amid the roar of traffic over-saturating the
   interconnect(s).

   The default is to behave as before - SO_KEEPALIVE not set.

*) Base on the frequency at which the author has used the
   functionality, the default for --enable-burst is now "yes."  To
   disable support for burst mode one must now include a
   --enable-burst=no when performing the ./configure prior to
   compiling the bits.

*) The output of the -D global command line option (./configure
   --enable-demo) has been enhanced to include seconds and
   milliseconds since the epoch as returned by a gettimeofday() call
   with a null pointer for the timezone.  This is in support of being
   able to easily shove interim results into an rrdtool Round-Robin
   Database (RRD).

*) The "omni" tests will be compiled-in by default, and WANT_MIGRATION
   is the default.  One must ./configure with --enable-omni=no to
   disable this.

*) When ./configured with --enable-intervals and intervals are
   actually used, the round-trip latency reported by an omni (or
   migrated classic) request/response test should better reflect
   reality rather than the length of the pacing interval.  It and the
   MEAN_LATENCY from the histogram and -j output will still differ
   slightly and probably always will.

*) The histogram code has been enhanced to track more than one latency
   at a time and so --enable-histogram and --enable-burst are now
   compatible - for the omni tests or migrated "classic" tests only
   however.  This change was inspired/instigated by Jim Gettys and his
   work on overly-large queues of buffers

*) WANT_MIGRATION is enabled when one specifies --enable-omni on the
   configure command line.

*) Massage and encorporate a patch from Google that enables
   randomization of the IP addresses used in a test.  An optional mask
   length in the standard '/' notation can be added to the end of the
   IP/name in the test-specific -H or -L options of an Omni test.

*) Massage and include a DEBUG_LOG_FILE patch for Android from
   Josselin Costanzi

*) Add intial attempt to report Slot ID on HP-UX 11.31.

*) Add global -s option to cause omni tests to pause between
   setting-up the test and actually starting it.  pause is in seconds.
   Poor man's way to (attempt to) avoid issues when starting many,
   Many, MANY concurrent netperf tests. Based on patches from Google.

*) Additional timing statistics will be kept by the omni tests when
   the global "-j" option is specified.  The additional statistics are
   min, max, mean, stddev and the 50th, 90th and 99th percentiles on
   the timings measured by histograms. Based on patches from Google.

*) Add a workaround to get Linux to report TX queue drops in a
   UDP_STREAM test when the socket buffer size is larger than the
   TX queue.  Provided by Andrew Gallatin.

*) Fix the configure script to know it does not have to look for an 
   SCTP library on FreeBSD 8.X

*) The BSD and "omni" tests now have a test-specific -R option which
   is a boolean controlling whether or not SO_DONTROUTE will be set on
   the data socket.  By default, any unidirectional UDP test will have
   SO_DONTROUTE set unless a -R 1 option is given.  All other tests
   (including UDP request/response tests) will not have SO_DONTROUTE
   set unless a -R 0 option is given.  This is put into place to make
   it take longer for blithering idiots to shoot themselves in the
   foot by running tests on setups they shouldn't.

*) At least the beginnings of support for RDS, based on a circa 2007
   patch against 2.4.2 by Vladimir Sokolovsky. Rather than create the
   "RDS_STREAM" test of his patch, the intention this time around is
   to enable RDS for the "omni" tests by using an omni test-specific
   -T rds specifyer for the "transport" to use.

*) Missing fprintf format statements provided by Bruno Cornec

*) Numerous cleanups from Jose Pedro Oliveira

*) Fixes to allow netperf -H ::1 to work without having to add -6 or
    an AF_INET6 -L option

These are the Release Notes for Revision 2.4.5 of netperf:

Things changed in this release:

*) Fixes for Linux procstat-based CPU utilization on newer kernels
   from Andrew Gallatin.

*) Fix for a TCP_RR hang from Michael Shuldman

*) Compilation cleanups for MingW cnd MSDOS (djgpp) ourtesy of Gisle
   Vanem.

*) Changes to enable compilation and building of netperf for
   VMware. Kudos to the person who did the first port, I will be happy
   to name that person when told it is OK :)

*) Fixes from Adam Bidema for launching netserver children when the
   path to netserver.exe is very long.  

*) For the first time, netperf2 has a dependency, albeit optional, on
   another non-base-os bit of code - libsmbios under Linux.  It will
   attept to detect this at compile time and use it to report the
   system model name in an omni test.  If libsmbios is there we will
   try to use it, otherwise we will not.  If the associated include
   file is also there (eg the -dev package in apt-get-speak), we will
   use it to get the prototype for SMBIOSGetSystemName, otherwise we
   make a guess as to the prototype for SMBIOSGetSystemName(), which
   is the only call we make to libsmbios.

*) Fixes for BSD CPU utilization to deal with different BSD variants
   using different types.  Courtesy of Simon Burge <simonb (a] NetBSD.org>

*) The "omni" suite has been added on an experimental basis.  If it
   works-out then many of the tests in src/nettest_bsd.c,
   src/nettest_sdp.c, and src/nettest_sctp.c will be "migrated" to use
   the "omni infrastructure" (aka two routines to measure them
   all...).  Apart from reduced socket code, the omni suite has
   user-configurable output in either "human readable," CSV or
   keyword=value format.  By default, a VERY large quantity of data is
   output when asking for csv format (test-specific -o option) or
   keyword format (test-specific -k option).  The omni suite is not
   yet documented (there are some as-yet undiagnosed problems with
   doc/netperf.texi in emacs texinfo mode and updating nodes and links
   and such - any help there would be appreciated) but there is a
   small text file in doc/ describing the names (most) of the
   available output's.  For the most up-to-date list consult
   src/nettest_omni.c and the enum netperf_output_name. Or, you can
   pass-in a "filename" of '?' to either of the -O, -o or -k options
   and netperf will emit a list of the known available outputs.

*) Coming along for the ride are some new platform specific files to
   determine the probable egress interface for each end of a test, as
   well as driver information for that interface.  There is also
   reporting of "uname" like information for both local and remote
   system, and eventually perhaps something about the vendor's model
   name for the systems as well as the processor types.  The end goal
   is to make it easy to get most if not all what one would want in a
   database of netperf results.

*) The UDP_RR test now understands the global -f option to change
   output units.  It also understands the -B option to tag
   results. Courtesy of Alexander Duyck.

*) A fix has been added for hanging UDP_RR tests under
   Windows. Courtesy of Alexander Duyck.

*) Use vfork() on those platforms without fork(), courtesy of Matt
   Waddel

*) Track the bouncing interfaces that are linux processor affinity

*) Fixes for Solaris sendfilev usage.

*) A TCP_MSS test has been added which will report the MSS for a data
   connection setup as if the test were a TCP_STREAM test.  While the
   remote (netserver) is tricked into thinking it is to accept a
   TCP_STREAM test, no actual data will flow over the connection.
   This means that if the MSS is one which might change over the life
   of the connection, it will not be reflected in the test output.
   Should this prove to be a problem a single send() can be arranged
   along with the return of the shutdown();recv() handshake.

   The idea is that this might be useful for netperf scripts wanting
   to parameterize things based on the MSS - for example the
   packet_byte_script.

*) The width of the confidence interval can be specified in fractions
   of a percent for the confidence of a clean, close, comfortable
   calculation. :)

*) Honor the global -B option in a TCP_SENDFILE test.

*) Correct the sense of Send/Recv in the banner of a TCP_MAERTS test.

These are the Release Notes for Revision 2.4.4 of netperf:

Things changed in this release:

*) The LOC_CPU and REM_CPU tests will report their respective beliefs
   as to the number of CPUs present when the verbosity is set to more
   than one.  This can be used when trying to diagnose issues with CPU
   utilization.

*) A kind soul who wishes to remain anonymous provided a patch to
   enable use of sendfile() on OSX.

*) Fix a misplaced \n in a format string of send_tcp_maerts, courtesy
   of Alexander Duyck.

*) There is an experimental global -r option which will allow one to
   include CPU utilization measurements, but make the decision about
   hitting confidence based on the result only.  The test banner will
   reflects this when -r is used.

*) It is no longer necessary to specify a file with the global -F
   option when running a _SENDFILE test.  Netperf will create a
   temporary file and populate it with random data and use that.  If
   running aggregate tests it is strongly suggested one use a -F
   option. Otherwise, the overhead spent creating and populating the
   temporary file will be included in the CPU utilization calculation.

*) The configure script recognizes Solaris 11 and selects the correct
   CPU utilization mechanism - or rather it selects the same mechanism
   as is used in Solaris 10.  Fix courtesy of Andrew Gallatin.

*) Convert a number of struct sockaddr_in's to struct
   sockaddr_storage's and add requisite casts to deal with some abort
   problems on Windows and perhaps other platforms as well. Kudos to
   Alexander Duyck.

*) One can now pass a value of 'x' to the global -f option to specify
   the units as transactions per second.  This is the default for any
   request/response test, which is determined by there being a "double
   `r'" in the name - eg "RR," "rr," "Rr," or "rR."  At present only
   the TCP_RR test actually looks for this to be set.

*) One can request bits/bytes per second as the primary output of a
   TCP_RR test by setting the global -f option to [kmgKMG] as with any
   of the "STREAM" tests.  This converts the primary throughput metric
   to a bitrate (byterate) following the verbosity rules for a STREAM
   test.  Service demand remains usec/Transaction regardless of the
   setting of the global -f option.

   A verbosity level of 2 or more will cause the TCP_RR test to report
   calculated average RTT latency, transaction rate, and inbound and
   outbound transfer rates regardless of the primary units selected
   with the global -f paramter.  If the primary output is transactions
   per second, the reported inbound and outbound transfer rates will
   be 10^6 bits per second, otherwise, they honor the setting of the
   global -f option.

   All of this is EXPERIMENTAL and subject to change without prior
   notice in future versions of netperf.

*) Replace "break" with "break 2" in acinclude.m4 for a socklen macro

*) The default for the requested socket buffer size is changed from 0
   to -1 to enable passing a value of 0 under Windows, which tells that
   stack one wishes to enable copy-avoidance.

*) Call fflush() on each interim result displayed in demo mode to make
   things happier for folks redirecting same to a file.  From Dan
   Yost.

*) In theory each distinct netserver child will have a debug log with
   its pid appended to the name, somewhat like what appears to happen
   under Windows.

*) A new global, command-line option to netperf and netserver has been
   added. The -V option will cause netperf/netserver to display its
   version and exit.

*) Setting -I without setting -i will now implicitly set the iteration
   minimum and maximums as if a -i 10,3 were set.  Also, some further
   sanity checking on the bounds for each is made.

*) Fixed a typo in the manual (found by Emir Halepovic) so the
   description for the -s and -S options properly specifies they
   affect the data connection.

These are the Release Notes for Revision 2.4.3 of netperf:

Things changed in this release:

*) The UDP_STREAM test includes --enable-demo support, courtesy of
   patches from Scott Weitzenkamp.

*) The nettest_dns.* files have been removed from the release and the
   repository.  Those wishing to perform DNS server tests should
   migrate to netperf4 which has better support for DNS test.

*) Fixes for compiling under Windows with Mingw/gcc courtesy of Gisle
   Vanem.

*) A new global option - -N - has been added. When specified, this
   option will tell netperf to not bother to try to establish a
   control connection with a remote netserver.  Instead, netperf will
   only attempt to make a data connection to the remote system.  By
   default, this will be to the "discard" service for a "STREAM" or
   "SENDFILE" test, the "echo" service for a "RR" test and the
   "chargen" service for a "MAERTS" test.  Any "remote" settings are
   changed to reflect their being unused in the test, and a "no
   control" tag is added to the test banner when -N is specified.

   This still needs to be propagated to other test files - at least
   for those for which it may make sense.

*) The tests in nettest_bsd.c have been altered to not actually take
   timestamps and deltas in --enable-histogram unless the verbosity
   level has been set to actually display a histogram.  This reduces
   the overhead measurably, even on systems with "fast" time calls,
   which _may_ mean that a future release of netperf may have
   histogram support enabled by default.

   This still needs to be propagated to other test files.  Patches
   from the community would be most welcome :)

*) Eliminate a bogus fprintf from the signal catching routine which
   was being executed when both intervals and demo mode were active at
   the same time.

*) The nettest_ipv6.* files are no longer included in the source
   tar/zip file. IPv6 functionality has been subsumed into the
   nettest_bsd.* files for some time now. 

*) Use a higher resolution "time" source for HISTOGRAM support under
   Windows, courtesy of Spencer Frink. Prior to this it had no better
   than 10ms granularity which could lead to some rather strange
   looking results :)

*) A bug fix reporting recv_size rather than send_size in TCP_MAERTS
   when CPU utilization was requested.

*) A bug fix for buffer filling from a file to properly advance the
   buffer pointer when the file is smaller than the send buffer.

*) Enable certain UDP tests which previously used unconnected sockets 
   to use connected sockets.  Courtesy of Shilpi Agarwal.

*) The OSX CPU utilization code actually gets put into the tarball in
   a make dist now :)

*) The check to make sure that getaddrinfo returned ai_protocol and/or
   ai_socktype's matching that which we requested is done for all socket
   and/or protocol types and a warning is emitted if it returns any which
   do not match.

*) The linux CPU affinity code has been made capable of binding to
   CPU's >=32 on a 32-bit compilation and >=64 on a 64-bit
   compilation.

*) More complete closing/redirecting of stdin/stdout/stderr/where in
   netserver to make it easier to launch netserver at the far-end of a
   remote shell.  Courtesy of Hans Blom.

*) Sendfile changes for Solaris courtesy of Andrew Gallatin.

*) "spec" file support to generate RPMs courtesy of Martin Brown

These are the Release Notes for Revision 2.4.2 of netperf:

Things changed in this release:

*) Fixes for floating point format differences, courtesy of George
   Davis.

*) Additions for CPU util support on MacOS X, courtesy of Anonymous.

*) Processor affinity is now supported on AIX 5.3 (perhaps earlier)
   via the bindprocessor system call.

*) Fixes for test lockups with TCP_CRR and TCP_CC under Windows
   courtesy of Dikon Reed.

*) Fixes to netcpu_looper.c to get it to actually compile :)

*) Have netcpu_looper use the bind_to_specific_processor() call
   provided by netlib since that knows about more platforms than the
   code in netcpu_looper did. The looper CPU binding will use a
   mapping to handle cases where the CPU id's on the system may not be
   a contiguous space starting from zero.  At present, the code that
   setups the mapping only knows about retrieving actual CPU ids under
   HP-UX.

*) The netcpu_sysctl method becomes calibration-free, courtesy of 
   Andrew Gallatin

These are the Release Notes for Revision 2.4.1 of netperf:

Things changed in this release:

*) There is now a -B global command-line argument that will append its
   parameter as a string to the end of result lines when test banners
   have been suppressed.  this is to make it easier to distinguish one
   result from another when aggregate restults are being run in
   parallel, without having to resort to having the individual results
   shell redirected to a file. This has been done for some of the
   tests in nettest_bsd.c, but not all of them, nor for the tests in
   the other nettest_mumble.c files.

*) There is now an --enable-spin configure option that will enable
   intervals if not already enabled and will have the sender sit and
   spin in a tight loop until time for the next interval rather than
   wait for an interval timer to expire.  This means it should be
   possible to have a much finer granularity on the interval, at the
   expense of an EXTREME increase in CPU utilization.  (To the extent
   I'm considering disabling measurement of local CPU utilization when
   that mode is enabled, and bursts have been requested - your
   feedback on that topic would be most appreciated) 

   If only --enable-intervals is used with configure, the old set the
   interval timer and wait method is still used.

   If --enable-spin is configured, the test banner will include "spin
   intervals" rather than the "intervals" from a plain
   --enable-intervals.  The sit and spin will either use
   gettimeofday(), or gethrtime() if gethrtime() is available.

   This has been implemented in the tests of nettest_bsd.c but none of
   the others.  Volunteers would be most welcome.  I would entertain
   the notion of making the implementation a series of inline
   functions in netlib. This holds true for the demo mode - why will
   become clear when you look at nettest_bsd.c.  While things are
   considerably cleaner than they were before, with reuse within
   nettest_bsd.c, there is no resuse with the rest of the
   nettest_mumble.c files.

*) the -w option for the interval time now takes three optional
   suffixes. if the suffix is 'm' (eg 10m) it will assume the user has
   specified time in units of milliseconds.  if the suffix is 'u' it
   will assume microseconds, and if 's' seconds. no suffix remains
   milliseconds for backwards compatability with previous netperf
   versions.

*) It should be possible to successfully compile with
   --enable-intervals.

These are the Release Notes for Revision 2.4.1 of netperf:

Things changed in this release:

*) netcpu_pstatnew.c has been altered to workaround a bug in the
   interrupt cycle accounting in HP-UX 11.23 that is not expected to
   be resolved until a later release.  basically, some interrupt time
   is not counted, which means the sum of idle, user, kernel and
   interrupt is less than the cycles per second multiplied by the
   elapsed time.  the workaround preserves the "no calibration
   required" nature of the pstatnew CPU utilization mechanism.  you
   can see more in netcpu_pstatnew.c and/or in debug output.

*) in netlib.c recv_response has been renamed
   recv_response_timed(addl_time) which is now used in
   calibrate_remote_cpu in place of the "sleep(40);recv_response()"
   sequence.  This then allows the REM_CPU test to complete in less
   than 40 seconds when the remote's CPU utilization mechanism does
   not require calibration.  The value of "addl_time" is added to the
   tc_sec field of the select() timeout.  A "new" recv_response has
   been added that simply calls recv_response_timed(0) - this is to
   minimize the number of changes needed elsewhere in the code.

*) hopefully, this release fixes problems people have been having with
   the configure script failing when picking a type for socklen_t.
   now, instead of generating an error, it emits a warning and simply
   tries socklen_t

*) the configure script no longer looks for the size of an in_port_t

*) netlib.c now has code to perform processor binding for Tru64, but
   the configure script may or may not detect it correctly. This means
   that one may have to edit the config.h file by hand to get the
   functionality.

*) it is known that netperf will compile under Windows XP and 2003
   using the DDK it is possible that netperf 2.4.1 will compile on a
   Windows system under VC++/Visual Studio.  It might even work!-) See
   the README.window file for additional details.

Things _NOT_ changed in this release:

*) The automagic determination of the number and type of parameters to
   sched_setaffinity under Linux remains brittle at best.

These are the Release Notes for Revision 2.4.0 of netperf:

Things changed in this release:

*) Netperf has been converted to use a configure script.  Yes boys and
   girls, after 12 years of distributing netperf with just a makefile
   I have finally bitten the bullet and cast my fate to autoconf,
   automake, etc.  To get the most basic netperf built all you should
   need to do is:

   cd to the netperf directory
   ./configure
   make
   and perhaps
   make install

   (Note, I've not done much with make install - I'm hemming and
   hawing over what the default installation location should be)

   Please keep in mind that this is the first time I've tried to use
   autoconf et al. I am sure there are things that should be done
   differently and would welcome any and all constructive criticisms.

   I suspect there are several places where I've not fully
   demonstrated being of the autoconf body - particulary as pertains
   to include files being in "#if mumble #endif" blocks.  Fixes would
   be most welcome.

*) Speaking of becomming one with various GNU tools, work on a new
   netperf manual has begun, with the source being a texinfo document
   that is converted to "all" the other formats.  This resides in doc/
   .

*) The platform-specific parts of CPU utilization measurement have
   been broken-out into separate .c files and selected at configure
   time a la the pcap_mumble files of tcpdump.  This makes
   src/netlib.c _much_ easier to read and the addition of new CPU
   utilization mechanisms much easier.

*) New HP-UX 11.23 and Solaris 10 CPU utilization measurement
   mechanisms (called pstatnew and kstat10 respectively) need no
   calibration step.  Both have variations on microstate accounting.
   HP-UX 11.23 still identifies the method in the headers as 'P' for
   pstat.  The kstat10 method is identified as 'M' for Microstate.

   Scripts which make calibration runs with LOC_CPU and REM_CPU may
   continue to do so, they will just run forty to eighty seconds
   faster on platforms with the calibration-free CPU util mechanisms.

*) Automatic detection of CPU utilization mechanism for HP-UX, Linux,
   AIX, *BSD and Solaris.  If you do not like what the configure
   script selects, you can use --enable-cpuutil=<foo> .

*) The "times" (aka 'T') CPU utilization mechanism has been removed.
   It was never very accurate at all, only showing CPU time charged to
   the process, and with interrupts and other network processing it is
   rarely chaged to a or the correct process.  It and other methods
   may remain in the format_cpu_method() routine of src/netlib.c for
   historical purposes only.

*) CAVEAT - the "kstat" mechanism is KNOWN TO BE BOGUS for Solaris.
   It does not include time spent processing interrupts, and
   networking benchmarks will generate at least a few of those...
   This affects _ALL_ versions of Solaris with kstat.

   So, do NOT trust any CPU util figures where netperf says the method
   was 'K' for kstat - unless perhaps it reports 100% CPU util.

   Solaris 10 takes a step in the right direction adding microstate
   accounting similar to what netperf uses on HP-UX 11.23.  HOWEVER,
   Solaris 10's accounting for user/kernel/idle is done in _parallel_
   with interrupt, which means they overlap. Doubleplusungood. Netperf
   attempts to compensate for that with some handwaving
   (src/netcpu_kstat10.c)

*) Initial support for SCTP has been added with the SCTP_STREAM and
   SCTP_RR tests.  These tests use the libsctp mechanisms for
   increased portability.  It has been explained that libsctp should
   not impart all that much overhead and it does make things rather
   simpler.

*) Netperf now uses getaddrinfo() to resolve hostnames and IP
   addresses. A replacement getaddrinfo() is provided for those
   platforms where the configure script cannot tell that getaddrinfo
   is present.   

   There are cases where a host's getaddrinfo call may return results
   that ignore the hints for protocol.  Netperf catches these and
   reports a warning so you can pester your OS source for fixes.

   Solaris getaddrinfo() seems to return results with SCTP procotol
   cleared.

   Mac OS X getaddrinfo botches when the service/port is specified as
   "0" so one must specify a port number on the netperf command line.

   AIX 5.something getaddrinfo has a different but similar problem
   with "0" as a port/service name as well.

   Linux 2.6 and HP-UX 11i getaddrinfo seem to be fine - at least as
   far as netperf goes :)

*) A "Demo Mode" has been added to the main BSD Sockets/TCP/UDP tests:
   TCP_STREAM, TCP_MAERTS, TCP_SENDFILE, TCP_RR, TCP_CC, TCP_CRR and
   UDP_RR. It has not been added to UDP_STREAM.  This mode is enabled
   with --enable-demo when configuring netperf, which activates a
   global "-D" option.  By default, -D will cause interim results
   (throughput or transactions/s only, not CPU util) from the
   netperf's perspective to be emitted no sooner than once per second.
   An optional parameter can specify another interval in units
   (floating point) of seconds:

      -D 1.5

   will make the reporting interval at least 1.5 seconds.

   This mode makes no use of explicit interval timers since that can
   be so, well fun on different platforms.  Instead, an initial guess
   of how many units of work must be done to consume the desired
   reporting interval is made, and that guess is refined throughout
   the entire test.  If something happens to dramatically slow-down
   the test, the reproting interval may become must larger for a few
   intervals.  When things speed-up it is detected very quickly.  As
   with the --enable-historgram support, if gethrtime() is available
   on the platform, it will be used in lieu of gettimeofday().  In any
   case, the number of calls to gettimeofday()/gethrtime() is much,
   Much, MUCH smaller than for --enable-histogram so while there may
   be a measurable effect on the results, it should be rather small.

*) The global -H option has been enhanced to take an optional address
   family specification for the control connection:

   -H <remote>,<family>

   Unlike other comma-separated options, where specifying only one
   thing will set both, here specifying only one thing will be
   ass-u-me-d to be the <remote> and will leave <family> defaulted
   (AF_UNSPEC).   Family can be specified as "4" or "inet" for
   AF_INET, "6" or "inet6" for AF_INET6.

*) A new global -L option has been added to specify the local name/IP
   and/or address family for the control connection:

   -L <local>,<family>

   Unlike other comma-separated options, where specifying only one
   thing will set both, here specifying only one thing will be
   ass-u-me-d to be the <local> and will leave <family> defaulted
   (AF_UNSPEC).  Family can be specified as "4" or "inet" for
   AF_INET, "6" or "inet6" for AF_INET6.

*) Test-specific -H and -L options are present for the TCP, UDP and
   SCTP tests, which are now (intended to be) IP protocol version
   agnostic.

*) Global -4 and -6 options will set the both the local and remote
   address family to either AF_INET or AF_INET6 respectively.

*) Test-specific -4 and -6 options have been added for TCP, UDP and
   SCTP tests.

*) Since the basic TCP UDP and SCTP tests are no longer IPv4-only, the
   nettest_ipv6.[ch] files are only included in the source
   distribution for historical interest.

*) The main test banners for the TCP, UDP and SCTP tests have been
   enhanced to give both local and remote addressing information for
   the data connection.

*) Compilation under Windows is likely FUBAR at this point.  I _hope_
   to start trying to do builds under the DDK soon, but am not sure
   when I'll be able to start.  Any and all assistance you can give
   there would be most welcome.

*) Various and sundry fixes.  TCP_RR should no longer go into an
   infinite loop when you abort netperf.  I'm sure there are others.

*) Unix domain socket tests are compiled-in with --enable-unix=yes at
   configure time.

*) DLPI tests are compiled-in with --enable-dlpi=yes at configure
   time.

*) XTI tests are compiled-in with --enable-xti=yes at configure time. 

Things not changed in this release:

*) Seems like everything has changed :)

These are the Release Notes for Revision 2.3pl2 of netperf:

Things changed in this release

*) One can bind netperf or netserver to specific CPUs with the -T
   option. This is a generalization of some HP-UX and netserver specific
   work from 2.3pl1.

*) Extend the kludge to workaround the Linux setsockopt/getsockopt
   bizzarreness to the socket buffer sizes for the remote side in
   addition to the local side.

*) Fix the lack of initialization of times_up in recv_tcp_maerts()
   that caused confidence intervals to fail miserably.

*) Other misc fixes - than you to all of you who sent them.

These are the Release Notes for revision 2.3pl1 of netperf:

Things changed in this release

*) The bind() call in create_data_socket() in the file nettest_bsd.c
   is no longer conditional on the user's specifying an IP address or
   port number to which the data socket should be bound.  This fixes
   the "connection refused" errors in the UDP tests.

*) Some experimental code to allow one to specify a CPU to which the
   remote netserver should be bound.  This is intended to allow one to
   get greater certainty (as in confidence intervals) on SMP
   systems. At present the functionality is HP-UX specific.
   Submittals of changes for a more general approach are welcomed.

These are the Release Notes for revision 2.3 of netperf:

Things changed in this release

*) The user can now specify local and/or remote port numbers for the
   data connection using the -P test-specific option.  This is to
   support those folks who want to run netperf through those evil,
   end-to-end-breaking things known as firewalls... :)  This changes
   the format of some of the control messages, hence the bump in the
   update number in the VUF. While it may be possible to mix 2.3 and
   pre-2.3 netperf and netserver, it is not supported.

*) The user can now specify local and/or remote IP addresses for the
   data connection using the -I test-specific option.  This is to
   support those folks who want to run netperf through those evil,
   end-to-end-breaking things known as firewalls... :) This changes
   the format of some of the control messages, hence the bump in the
   update number in the VUF. While it may be possible to mix 2.3 and
   pre-2.3 netperf and netserver, it is not supported.

*) Set DL_mumble  message priorities in the DLPI tests

*) Fix error return check for getaddrinfo()

*) Those systems with gethrtime() can define -DHAVE_GETHRTIME to use
   gethrtime() instead of gettimeofday() and reduce the measurement
   overhead when enabling the -DHISTOGRAM functionality.

*) The default for -DHISTOGRAM compilation now adds a UNIT_USEC and
   TEN_USEC row and renames TENTH_MSEC to HUNDRED_USEC.  If you want
   the old behaviour add -DOLD_HISTOGRAM to CFLAGS.

*) Add missing '!' in the recv_udp*_stream so we recognize the end of 
   a timed test correctly.

*) Replace "||" with "&&" to fix an infinite loop in
   recv_tcp_conn_rr() most likely introduced in 2.2pl5. 

*) Code has been added to kludge around the bug in Linux getsockopt()
   where it almost always returns twice the value for which one
   asks unlike virtually every other stack on the face of the
   planet. This was doing some unpleasant things to tests in which
   confidence intervals were requested. 

Things not changed in this release

*) Lots :)

These are the Release Notes for revision 2.2pl5 of netperf:

Things changed in this release

*) Improved (perhaps even usable :) support for Windows, including
compilation and run on Win64.

*) Fixes for MacOS X and FreeBSD

Things not changed in this release

*) Specifying the port number(s) for the data connection

These are the Release Notes for Revision 2.2pl4 of netperf:

Things changed in this release

*) USE_SYSCTL available on suitable FreeBSD releases to measure CPU
   utilization without having to resort to -DUSE_LOOPER.

*) Include Solaris 9 with the Linux sendfile path under -DHAVE_SENDFILE

This still outstanding in this release

*) Knowing why signals are not interrupting socket calls under
   OpenVMS.  A quick try to use threads for timing a la Win32 worked,
   but also cut performance in half.  Any and all assistance in this
   area would be most welcome.

These are the Release Notes for revisoin 2.2pl3 of netperf:

Things changed in this release

*) I started practicing what I preach and will set SO_REUSEADDR before
   netserver tries to bind to its well-known port.

*) Initial port to OpenVMS.  This includes support for the OVMS
   Auxilliary server (inetd replacement).  See README.ovms for more
   details on what is involved in compiling and running netperf under
   OpenVMS.

*) Testname comparisons are now case insensitive.  This is a side
   effect of OpenVMS downshifting commandlines to lowercase.  I made
   the change and decided it was OK to keep it that way, even though
   for OpenVMS one _has_ to set the right defines to disable that
   downshifting or the command-line options will not work. For example
   "-H" will become "-h" which isn't quite the same thing...

*) Misc fixes for nettest_ipv6.c.

*) Support for sendfile() under Linux

Thins I would like to have changed but did not know how or didn't have
time:

*) Allow netserver to run as a standalone daemon under OpenVMS
*) Allow netserver to run as a standalone daemon under Windows
*) Rediscover an inetd-like facility for Windows
*) Figure-out how to get low-overhead, accurate, per-CPU utilization
   figures under OpenVMS
*) Get the UDP_RR and UDP_STREAM tests to work under OpenVMS, and get
   the TCP_RR test to work based on time rather than transaction
   count.  There is some bug (possibly in OpenVMS?) where the SIGALRM
   fires, but a socket call will not return an EINTR.

Things that changed prior to this release:

*) Addition of the TCP_MAERTS test - this is a TCP_STREAM test where
   the data flows from the netserver to the netperf rather than from
   the netperf to the netserver.  This can be useful in those
   situations where netperf (netserver) is installed on a remote
   system, but the tester has no shell access and wishes to get
   performance data for the path from netserver to netperf.

These are the Release Notes for the 2.2 revision of netperf:

Things changed in this release

*) Various and sundry bugs fixed (in theory) for platforms such as
   FreeBSD and Linux. If I left-out your bug fix, it was purely
   accidental - my mind has a very small cache, and sometimes I will
   "lose" email in the shuffle.

*) Initial support for sendfile() on HP-UX. This test will use the
   sendfile() call instead of send() to send data to the
   remote. Netperf "lies" to netserver and calls it a TCP_STREAM test
   since what netserver needs to do is exactly the same. A future
   patch may change that and simply have netserver call the same
   routine for both test types. Kudos to Charles Harris for the
   initial prototype.

*) The Fore ATM API and HiPPI tests have been dropped from the
   distribution. 

Things I would have liked to have changed, but did not have time for:

*) Conversion of the source and makefile to use the GNU configure/autoconf 
   utility to make it easier for folks to build by not having to edit
   makefiles... You will notice that I have started to switch from
   "DO_MUMBLE" to "HAVE_MUMBLE"

as always - happy benchmarking,

rick jones <raj (a] cup.hp.com>

---------------------------------------------------------------------

These are the Release Notes for the 2.1pl3 revision of netperf:

*) An OBOB (Off By One Bug) in netlib.c that was causing a core dump
   on Irix should be fixed.

*) Irix systems should now be able to determine the number of CPU's
   present automagically (code from outside, not tested yet because I
   have no MP Irix systems at my disposal)

*) An alpha version of a TCP_CC test has been added - this is a
   TCP_CRR test with out the "RR."

*) The -Ae has been removed from the default makefile. If someone has
   a nice way to automagically generate the correct makefile for
   different platforms I would like to learn how.

happy benchmarking,

rick jones <raj (a] cup.hp.com>

----------------------------------------------------------------------

These are the Release Notes for the 2.1 revision of netperf:

Things Changed in this release:

*) The XTI (Version 2 of the spec) tests are now documented in the
   manual. 

*) The TCP_CRR (Connect Request/Response) test is now documented in
   the manual, including a description of how it mimics the behaviour
   of http (the protocol underlying the WWW).

*) Support for for Windows NT 3.51 OS in the BSD Sockets tests (ok, so
   they are really Winsock in that case :). Other test suites may be
   ported as required/desired/appropriate. 

*) Tests for TCP and UDP, using the IPv6 extensions to BSD sockets are
   included in this release. They are included by adding -DUSE_IPv6 to
   the makefile and recompiling.

*) Support for a "long long" datatype should only be required for
   -DUSE_PSTAT compilation which is an HP-UX only thing. The
   *unbundled* HP compilers from at least "HP92453-01 A.09.61 HP C
   Compiler" and later should have the required support. The bundled
   compiler may not. GCC should work - check the archives listed in
   the comp.sys.hp.hpux FAQ for copies. The FAQ is archived on
   rtfm.mit.edu under the path pub/usenet/comp.sys.hp.hpux.

*) A "proper" fix for double data type alignment has been included.

*) A new script is included with this release which can be used to
   measure aggregate TCP_RR performance (multiple, concurrent
   instances of the TCP_RR test). A related use of this script would
   be measuring MP scaling. A single-byte TCP_RR test is good for this
   purpose for two reasons:

      1) it excercises the control/protocol paths heavily without
         using much in the way of data copies which may be easier to
         scale.
      2) most systems can easily saturate cards with bandwidth, but
         not so easily with request/response

   Of course, feedback on this is most welcome.

*) When measuring CPU utilization, the units for service demand have
   been changed from milliseconds (designated ms) of CPU per unit (KB
   or Transaction) to microseconds (desginated us).

*) For accurate reporting of service demand, netperf needs to know the
   number of CPU's present on a system. On some systems (HP-UX), this
   is automatic. For others (All), it is necessary to add a global "-n
   <numcpu>" option to both netperf and netserver.

   !! IF THIS IS LEFT-OUT CPU UTILIZATION AND SERVICE DEMAND FOR !!
	     !! MULTI-PROCESSOR SYSTEMS WILL BE WRONG. !!

   If you know of ways to programatically determine the number of
   active CPUs on a system, please let the author Rick Jones
   <raj (a] cup.hp.com> know.

*) other things I've probably forgotten :)

Things Not Changed in this release:

*) The ancillary test suites are essentially unchanged - DLPI,
   HiPPI/LLA, Unix Domain, and Fore ATM API. Unless there is much
   interest expressed in these tests, 2.1 may be the last release in
   which they are included. The order of retirement would likely be
   Unix Domain, HiPPI/LLA, Fore ATM API, and then DLPI.

Miscelaneous Comments:

*) The -DUSE_LOOPER CPU utilization _seems_ to be nice and low-impact
   on HP-UX, Digital Unix, and IRIX. It does not yet seem to be
   low-impact on Solaris (I need an example of priocntl usage), AIX
   (setpri only works if you are root), and NT (not sure of the
   reason). Help with those problems would be most appreciated.