Home | History | Annotate | Download | only in doc 
      1 # VTS User Manual
      2 
      3 Linux is officially supported for building and running VTS. Building on Windows is not supported, but it is possible to [run VTS on Windows](#run_vts) with Python, Java, and ADB installed.
      4 
      5 The following instructions assume Linux environment.
      6 
      7 ## 1. Setup
      8 
      9 ### 1.1. Host setup
     10 
     11 Please follow:
     12 
     13 * [Setup Manual](setup/index.md)
     14 
     15 (Optional)
     16 During VTS test runs, required Python packages are downloaded from the [Python Package Index](https://pypi.python.org/simple/). There is an option to instead install these packages from a local directory during test runtime by predownloading the packages. First run the lunch command, then set an environment variable VTS_PYPI_PATH as a new local directory to host the Python packages. Then run the download-pypi-packages.sh script:
     17 
     18 * `$ cd ${branch}`
     19 
     20 * `$ . test/vts/script/download-pypi-packages.sh`
     21 
     22 ### 1.2. Checkout master git repository
     23 
     24 [Download Android Source Code](https://source.android.com/source/downloading.html)
     25 
     26 `$ export branch=master`
     27 
     28 `$ mkdir ${branch}`
     29 
     30 `$ cd ${branch}`
     31 
     32 `$ repo init -b ${branch} -u persistent https://android.googlesource.com/platform/manifest`
     33 
     34 `$ repo sync -j 8`
     35 
     36 ### 1.3. Build an Android image
     37 
     38 `$ cd ${branch}`
     39 
     40 `$ . build/make/envsetup.sh`
     41 
     42 `$ lunch aosp_arm64-userdebug  # or <your device>-userdebug`
     43 
     44 The below is an optional step:
     45 
     46 `$ make -j 8`
     47 
     48 If this fails, please do:
     49 
     50 `$ repo sync -j 8`
     51 
     52 `$ make -j 8`
     53 
     54 Such can happen because tip of tree (ToT) may not always be buildable.
     55 
     56 ### 1.4. Build a VTS package
     57 
     58 `$ cd ${branch}`
     59 
     60 `$ make vts -j8`
     61 
     62 Or use the exact command:
     63 
     64 `$ make -j8 vts showcommands dist TARGET_PRODUCT=aosp_arm64 WITH_DEXPREOPT=false TARGET_BUILD_VARIANT=userdebug`
     65 
     66 ### 1.5. Connect to an Android device
     67 
     68 Let's connect an Android device and a host computer using a USB cable.
     69 
     70 * On an Android device, Setting -> About Phone -> Click repeatedly 'Build number' until developer mode is enabled.
     71 * On an Android device, Setting -> Developer options -> Turn on 'USB debugging'
     72 * On a host, run `adb devices` from a command line shell.
     73 * On a Android device, confirm that the host is trusted.
     74 * On a host, type `adb shell` and if that works, we're ready.
     75 
     76 ## 2. Run VTS Tests
     77 
     78 ### <a name="run_vts" /> 2.1. Run a VTS test plan
     79 
     80 For Linux users,
     81 
     82 `$ vts-tradefed`
     83 
     84 `> run vts`
     85 
     86 For Windows users, please build on Linux. Then copy the following zip file to Windows and extract it.
     87 
     88 `out/host/linux-x86/vts/android-vts.zip`
     89 
     90 Launch the batch file in the extracted folder.
     91 
     92 `$ android-vts\tools\vts-tradefed_win.bat`
     93 
     94 `> run vts`
     95 
     96 Example stdout:
     97 
     98 ```
     99 
    100 
    101 08-16 09:36:03 I/ResultReporter: Saved logs for device_logcat in .../out/host/linux-x86/vts/android-vts/logs/2016.08.16_09.17.13/device_logcat_7912321856562095748.zip
    102 08-16 09:36:03 I/ResultReporter: Saved logs for host_log in .../out/host/linux-x86/vts/android-vts/logs/2016.08.16_09.17.13/host_log_2775945280523850018.zip
    103 08-16 09:36:04 I/ResultReporter: Invocation finished in 18m 50s. PASSED: 18, FAILED: 0, NOT EXECUTED: 2, MODULES: 8 of 10
    104 08-16 09:36:04 I/ResultReporter: Test Result: .../out/host/linux-x86/vts/android-vts/results/2016.08.16_09.17.13/test_result.xml
    105 08-16 09:36:04 I/ResultReporter: Full Result: .../out/host/linux-x86/vts/android-vts/results/2016.08.16_09.17.13.zip
    106 ```
    107 
    108 ### 2.2. Test report for APFE (Android Partner Front-End)
    109 
    110 The uploadable report xml file can be found at
    111 
    112 `out/host/linux-x86/vts/android-vts/results/`
    113 
    114 After Android O release, you will be able to upload that xml file to [AFPE](https://partner.android.com)
    115 and obtain a certificate.
    116 
    117 ### 2.3. Check the test logs
    118 
    119 `$ vi out/host/linux-x86/vts/android-vts/logs/`
    120 
    121 Then select a directory which captures the time stamp of your test run (e.g., 2016.08.16_09.17.13).
    122 
    123 Then select `host_log_<timestamp>.zip` and host_log.txt in that zip file for host log.
    124 
    125 Then select `device_logcat_<timestamp>.zip` and device_logcat.txt in that zip file for device log.
    126 
    127 ## 3. Run Options for Advanced Users
    128 
    129 ### 3.1. List of VTS Plans
    130 
    131 Documented at [here](../tools/vts-tradefed/res/config/plans.md).
    132 
    133 ### 3.2. Run kernel test cases
    134 
    135 LTP (Linux Test Project) is part of vts-kernel.
    136 [This doc](developer_testing/kernel/ltp.md) shows how to run each LTP test case.
    137 
    138 ## 4. Debugging
    139 
    140 ### 4.1. Run VTS tests directly for debugging
    141 
    142 [Run directly from command line](testcase_develop_manual/run_vts_directly.md)
    143