Home | History | Annotate | Download | only in docs 
      1 <!-- Copyright 2015 The Chromium Authors. All rights reserved.
      2      Use of this source code is governed by a BSD-style license that can be
      3      found in the LICENSE file.
      4 -->
      5 
      6 # Telemetry: Run Benchmarks Locally
      7 
      8 ## Set Up
      9 
     10 If you don't have a Chromium checkout, download the
     11 [latest Telemetry archive](https://storage.googleapis.com/chromium-telemetry/snapshots/telemetry.zip).
     12 Unzip the archive. If you're running on Mac OS X, you're all set! For
     13 Windows, Linux, Android, or Chrome OS, read on.
     14 
     15 #### Windows
     16 
     17 Some benchmarks require you to have
     18 [pywin32](http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/).
     19 Be sure to install a version that matches the version and bitness of the Python
     20 you have installed.
     21 
     22 #### Linux
     23 
     24 Telemetry on Linux tries to scan for attached Android devices with
     25 [adb](https://developer.android.com/tools/help/adb.html).
     26 The included adb binary is 32-bit. On 64-bit machines, you need to install the
     27 libstdc++6:i386 package.
     28 
     29 #### Android
     30 
     31 Running on Android is supported with a Linux or Mac OS X host. Windows is not
     32 yet supported. There are also a few additional steps to set up:
     33 
     34   1. Telemetry requires [adb](http://developer.android.com/tools/help/adb.html).
     35      If you're running from the zip archive, adb is already included. But if
     36      you're running with a Chromium checkout, ensure your .gclient file contains
     37      target\_os = ['android'], then resync your code.
     38   2. If running from an OS X host, you need to run ADB as root. First, you need
     39      to install a "userdebug" build of Android on your device. Then run adb
     40      root. Sometimes you may also need to run adb remount.
     41   3. Enable [debugging over USB](http://developer.android.com/tools/device.html)
     42      on your device.
     43   4. You can get the name of your device with `adb devices` and use it with
     44      Telemetry via --device=<device\_name>.
     45 
     46 #### Chrome OS
     47 
     48 See [Running Telemetry on Chrome OS](http://www.chromium.org/developers/telemetry/running-telemetry-on-chrome-os).
     49 
     50 ## Benchmark Commands
     51 
     52 Telemetry benchmarks can be run with run\_benchmark.
     53 
     54 In the Telemetry zip archive, this is located at `telemetry/run_benchmark`.
     55 
     56 In the Chromium source tree, this is located at `src/tools/perf/run_benchmark`.
     57 
     58 #### Running a benchmark
     59 
     60 List the available benchmarks with `telemetry/run_benchmark list`.
     61 
     62 Here's an example for running a particular benchmark:
     63 
     64 `telemetry/run_benchmark --browser=canary smoothness.top_25_smooth`
     65 
     66 #### Running on another browser
     67 
     68 To list available browsers, use:
     69 
     70 `telemetry/run_benchmark --browser=list`
     71 
     72 For ease of use, you can use default system browsers on desktop:
     73 
     74 `telemetry/run_benchmark --browser=system`
     75 
     76 and on Android:
     77 
     78 `telemetry/run_benchmark --browser=android-system-chrome`
     79 
     80 If you're running telemetry from within a Chromium checkout, the release and
     81 debug browsers are what's built in out/Release and out/Debug, respectively.
     82 
     83 To run a specific browser executable:
     84 
     85 `telemetry/run_benchmark --browser=exact --browser-executable=/path/to/binary`
     86 
     87 To run on a Chromebook:
     88 
     89 `telemetry/run_benchmark --browser=cros-chrome --remote=[ip_address]`
     90 
     91 #### Options
     92 
     93 To see all options, run:
     94 
     95 `telemetry/run_benchmark run --help`
     96 
     97 Use --pageset-repeat to run the test repeatedly. For example:
     98 
     99 `telemetry/run_benchmark smoothness.top_25 --pageset-repeat=30`
    100 
    101 If you want to re-generate HTML results and add label, you can do this locally
    102 by using the parameters `--reset-results --results-label="foo"`
    103 
    104 `telemetry/run_benchmark smoothness.top_25 --reset-results
    105 --results-label="foo"`
    106 
    107 ####Comparing Two Runs
    108 
    109 `telemetry/run_benchmark some_test --browser-executable=path/to/version/1
    110 --reset-results --results-label="Version 1"`
    111 
    112 `telemetry/run_benchmark some_test --browser-executable=path/to/version/2
    113 --results-label="Version 2"`
    114 
    115 The results will be written to in the `results.html` file in the same location
    116 of the `run_benchmark` script.
    117