Home | History | Annotate | Download | only in perf 
      1 =============================================================================
      2 Notes about performance benchmarks
      3 =============================================================================
      4 For each benchmark, here is a brief description and notes about its
      5 strengths and weaknesses.
      6 
      7 -----------------------------------------------------------------------------
      8 Artificial stress tests
      9 -----------------------------------------------------------------------------
     10 bigcode1, bigcode2:
     11 - Description: Executes a lot of (nonsensical) code.
     12 - Strengths:   Demonstrates the cost of translation which is a large part
     13                of runtime, particularly on larger programs.
     14 - Weaknesses:  Highly artificial.
     15 
     16 heap:
     17 - Description: Does a lot of heap allocation and deallocation, and has a lot
     18                of heap blocks live while doing so.
     19 - Strengths:   Stress test for an important sub-system; bug #105039 showed
     20                that inefficiencies in heap allocation can make a big
     21                difference to programs that allocate a lot.
     22 - Weaknesses:  Highly artificial -- allocation pattern is not real, and only
     23                a few different size allocations are used.
     24 
     25 sarp:
     26 - Description: Does a lot of stack allocation and deallocation.
     27 - Strengths:   Tests for a specific performance bug that existed in 3.1.0 and
     28                all earlier versions.
     29 - Weaknesses:  Highly artificial.
     30 
     31 -----------------------------------------------------------------------------
     32 Real programs
     33 -----------------------------------------------------------------------------
     34 bz2:
     35 - Description: Burrows-Wheeler compression and decompression.
     36 - Strengths:   A real, widely used program, very similar to the 256.bzip2
     37                SPEC2000 benchmark.  Not dominated by any code, the hottest
     38                55 blocks account for only 90% of execution.  Has lots of
     39                short blocks and stresses the memory system hard.
     40 - Weaknesses:  None, really, it's a good benchmark.
     41 
     42 fbench:
     43 - Description: Does some ray-tracing.
     44 - Strengths:   Moderately realistic program.
     45 - Weaknesses:  Dominated by sin and cos, which are not widely used, and are
     46                hardware-supported on x86 but not on other platforms such as
     47                PPC.
     48 
     49 ffbench: 
     50 - Description: Does a Fast Fourier Transform (FFT).
     51 - Strengths:   Tests common FP ops (mostly adding and multiplying array
     52                elements), FFT is a very important operation.
     53 - Weaknesses:  Dominated by the inner loop, which is quite long and flatters
     54                Valgrind due to the small dispatcher overhead.
     55 
     56 tinycc:
     57 - Description: A very small and fast C compiler.  A munged version of
     58                Fabrice Bellard's TinyCC compiling itself multiple times.
     59 - Strengths:   A real program, lots of code (top 100 blocks only account for
     60                47% of execution), involves large irregular data structures
     61                (presumably, since it's a compiler).  Does lots of
     62                malloc/free calls and so changes that make a big improvement
     63                to perf/heap typically cause a small improvement.
     64 - Weaknesses   None, really, it's a good benchmark.
     65 
     66