Home | History | Annotate | Download | only in docs
      1 # test\_droid: Quick Primer
      2 
      3 ## References
      4 
      5 [Autotest Best Practices](best-practices.md)
      6 
      7 [test\_that Basic Usage](test-that.md)
      8 
      9 ## Objective
     10 This document contains instructions for Brillo/Android developers interested in
     11 running automated integration tests at their desk.
     12 
     13 ## Usage
     14 The autotest repository, the `test_droid` tool, and all tests are checked out
     15 in both AOSP and internal Android trees at external/autotest.  You need a copy
     16 of the autotest source code and an `adb` binary in your PATH to run tests.
     17 
     18 ### Running tests against a single local device under test
     19 Once you have a local copy of the autotest source, you can easily run tests
     20 against a DUT connected directly to your workstation via a USB cable. Please
     21 note your first time running `test_droid` it will download and install a number
     22 of required packages locally into your autotest checkout.
     23 
     24 Run site\_utils/test\_droid.py from your autotest checkout to launch a test
     25 against a given DUT:
     26 
     27 ```
     28  $ ./site_utils/test_droid.py <Test Name>
     29 ```
     30 
     31 For example, to run the brillo\_WhitelistedGtests test:
     32 
     33 ```
     34  $ ./site_utils/test_droid.py brillo_WhitelistedGtests
     35 ```
     36 
     37 `test_droid` can run multiple tests at once:
     38 
     39 ```
     40  $ ./site_utils/test_droid.py brillo_WhitelistedGtests brillo_KernelVersionTest
     41 ```
     42 
     43 As well as test suites:
     44 
     45 ```
     46  $ ./site_utils/test_droid.py suite:brillo-bvt
     47 ```
     48 
     49 #### Selecting a specific device
     50 If you have more than one device connected, you'll have to specify its serial
     51 number.  First, look it up:
     52 
     53 ```
     54  $ adb devices
     55 * daemon started successfully *
     56 List of devices attached
     57 7d52318 device
     58 ```
     59 
     60 Then use it when running:
     61 
     62 ```
     63  $ ./site_utils/test_droid.py -s 7d52318 brillo_WhitelistedGtests
     64 ```
     65 
     66 ### Running tests that require multiple devices under test
     67 Autotest now supports the concept of testbeds, which are multiple devices being
     68 controlled by a single test. `test_droid` supports running these tests
     69 by specifying a comma separated list of serials as the test device:
     70 
     71 ```
     72  $ adb devices
     73 List of devices attached
     74 emulator-5554 device
     75 7d52318 device
     76 
     77  $ ./site_utils/test_droid.py -s emulator-5554,7d52318 testbed_DummyTest
     78 ```
     79 
     80 ### Running tests against a remote device under test
     81 `test_droid` can run tests against devices connected to a remote server.  This
     82 requires passwordless SSH access from the workstation to the remote server.
     83 If no username is specified, `test_droid` will try the root and adb users.
     84 If using the adb user, make sure it has passwordless sudo
     85 rights to run the adb and fastboot commands. You can specify a
     86 different user in the remote host name (the same passwordless requirement
     87 applies).
     88 
     89 The easiest way to set this up is to use the
     90 [Chrome OS testing keys](https://www.chromium.org/chromium-os/testing/autotest-developer-faq/ssh-test-keys-setup).
     91 Add to your SSH config an entry that looks like the following:
     92 
     93 ```
     94 HostName <Remote Server IP or Hostname>
     95   Port 9222
     96   User root
     97   CheckHostIP no
     98   StrictHostKeyChecking no
     99   IdentityFile ~/.ssh/testing_rsa
    100   Protocol 2
    101 ```
    102 
    103 To run the test:
    104 
    105 ```
    106  $ ./site_utils/test_droid.py \
    107        -r <Remote Server IP or Hostname> \
    108        <Test Name>
    109 
    110  $ ./site_utils/test_droid.py \
    111        -r <User>@<Remote Server IP or Hostname> \
    112        <Test Name>
    113 
    114  $ ./site_utils/test_droid.py -r 100.96.48.119 suite:brillo-bvt
    115 ```
    116 
    117 ### Advanced: Uploading Commits for Review
    118 Currently Autotest in AOSP is read-only, so you cannot use repo upload to
    119 upload code changes. If you do edit or add a new test, make a commit and upload
    120 it to https://chromium-review.googlesource.com.
    121 
    122 Due to the way Android does its automerging, the AOSP mirror of Autotest cannot
    123 be uploaded directly back to the Chromium upstream because the upstream does not
    124 contain the merge commits.  It will also reject the automerge committer ID.
    125 To work around this, if you need to upload a commit to Autotest, you first have
    126 to fetch and check out the upstream branch before making your changes:
    127 
    128 ```
    129  $ git remote add -t master cros \
    130      https://chromium.googlesource.com/chromiumos/third_party/autotest
    131  $ git fetch cros
    132  $ git checkout cros/master
    133  $ git checkout -b <local branch name>
    134  $ git branch --set-upstream-to=cros/master
    135 ```
    136 
    137 Be sure to run pylint on every file you touch:
    138 
    139 ```
    140  $ ./utils/run_pylint.py <file name>
    141 ```
    142 
    143 Run autotest unittests (which usually requires external packages):
    144 
    145 ```
    146  $ utils/build_externals.py  # install any missing os packages with apt-get
    147  $ utils/unittest_suite.py
    148 ```
    149 
    150 Then upload your commit for review:
    151 
    152 ```
    153  $ git push cros <local branch name>:refs/for/master
    154 ```
    155 
    156 ## Limitations
    157 
    158 Testing on Brillo/Android (and `test_droid` by extension) is currently limited
    159 to server-side tests, which run on an autotest server and control a
    160 Brillo/Android DUT (device under test) via remote command execution.  In the
    161 context of `test_droid`, the test logic is running on your development machine,
    162 and controlling devices with commands executed via `adb`.
    163 
    164 `test_droid` does not support the autoupdate end-to-end test. For instructions
    165 on how to run this test please refer to the Running Brillo/Android Autoupdate
    166 End-to-End Test doc.
    167