1 <html devsite> 2 <head> 3 <title>Trade Federation Overview</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 <p>Trade Federation (tradefed or TF for short) is a continuous test framework designed for running tests 27 on Android devices. It's a Java application which runs on a host computer, and communicates to one or 28 more Android devices using ddmlib (the library behind DDMS) over adb.</p> 29 30 <p>We've listed some of TF's main features below, along with a couple sample usecases. That said, 31 if you want to jump right in and get started, you can head straight for the 32 <a href="/devices/tech/test_infra/tradefed/fundamentals/index.html">Start Here</a> page.</p> 33 34 <h2 id="features">Features</h2> 35 <ul> 36 <li>modular, flexible, scalable design</li> 37 <li>has built in support for running many different types of Android tests: 38 <a href="http://developer.android.com/tools/testing/testing_android.html#Instrumentation">instrumentation</a>, 39 <a href="http://developer.android.com/tools/testing/testing_ui.html">uiautomator</a>, 40 native/gtest, host-based JUnit, etc</li> 41 <li>provides reliability and recovery mechanisms on top of adb</li> 42 <li>supports scheduling and running tests on multiple devices in parallel</li> 43 </ul> 44 45 <h2 id="usecases">Sample Trade Federation Usecases</h2> 46 <p>Trade Federation's modularity makes it straightforward to slot into environments with existing 47 build, test, and reporting infrastructures. We list below a few demonstrative 48 usecases where tradefed could enable efficient, scalable test practices.</p> 49 50 <p>First, it is useful to consider the landscape of potential usecases in terms of the question 51 "which parts are modifiable, and what parts are static?" For instance, a Device OEM can modify the 52 framework, the system, and the hardware, but has little or no influence over existing applications. 53 An application developer, on the other hand, can modify the app, but has little control over most 54 aspects of the system or the framework.</p> 55 56 <p>As a result, an entity in each usecase will have different testing goals, and will have different 57 options in the case of a set of test failures. Despite these differences, Trade Federation can 58 help make each of their test processes efficient, flexible, and scalable.</p> 59 60 <h3 id="oem">Device OEM</h3> 61 <p>A Device OEM builds hardware, and will often tweak the Android system and frameworks to run well 62 on that hardware. The OEM might strive to accomplish those goals while retaining stability 63 and performance at the hardware and system levels, and making sure the framework changes don't break 64 compatibility with existing applications.</p> 65 66 <p>The OEM could implement a device flashing module that will execute during the Target Setup stage 67 of the <a href="/devices/tech/test_infra/tradefed/fundamentals/lifecycle.html">lifecycle</a>. That 68 module would have complete control over the device during its execution period, which would allow 69 it to potentially force the device into the bootloader, flash, and then force the device to reboot 70 back into userspace mode. Combined with a module to tie into a continuous build system, this would 71 allow the OEM to run tests on their device as they make changes to the system-level firmware and 72 Java-level frameworks.</p> 73 74 <p>Once the device is fully booted, the OEM would be able to leverage existing JUnit-based tests, 75 or write new ones, to verify the functionality of interest. Finally, they could write one or more 76 result reporting modules to tie into existing test-result repositories, or to report results 77 directly (for instance, 78 <a href="/reference/com/android/tradefed/result/EmailResultReporter.html">by email</a>).</p> 79 80 <h3 id="developer">App Developer</h3> 81 <p>An Application Developer builds an app which needs to run well across a variety of platform 82 versions and a variety of devices. If an issue comes up on a particular platform version and/or 83 device, the only remedy is to add a workaround and move on. For larger developers, the test 84 process might be incorporated into a continuous build sequence. For smaller developers, it 85 might be kicked off periodically or by hand.</p> 86 87 <p>Most app developers would use the apk test installation modules that already exist in TF. 88 There's a version that <a href="/reference/com/android/tradefed/targetprep/InstallApkSetup.html" 89 >installs from the local filesystem</a>, as well as a version that can 90 <a href="/reference/com/android/tradefed/targetprep/InstallBuildEnvApkSetup.html">install 91 apks downloaded from a build service</a>. It is important to note that the latter version would 92 continue to work properly with arbitrarily many TF instances running on the same host machine.</p> 93 94 <p>Because of TF's proficiency at dealing with multiple devices, it would be straightforward to 95 classify each test result by the type of device that was used for that test. Thus, TF could 96 potentially generate a 2-dimensional (or multi-dimensional) compatibility matrix for every 97 build of the application.</p> 98 99 <h3 id="testing">Testing Service</h3> 100 <p>A Test Service might, for instance, allow app developers to submit apps and run tests on devices 101 instrumented with power-measurement tools to determine power usage for the app. This differs from 102 the prior two usecases in that the service builder does not control the devices or the applications 103 that are being run.</p> 104 105 <p>Because Trade Federation can run any Java class that implements the simple 106 <a href="/reference/com/android/tradefed/testtype/IRemoteTest.html"><code>IRemoteTest</code></a> interface, it's 107 trivial to write drivers that can coordinate some external piece of hardware with the test case 108 that's being run on the device. The driver itself can spawn Threads, send requests to other 109 servers, or do anything else that it might need. Moreover, the simplicity and versatility of the 110 result reporting interface, 111 <a href="/reference/com/android/tradefed/result/ITestInvocationListener.html" 112 ><code>ITestInvocationListener</code></a>, means that it is likewise straightforward to 113 represent arbitrary test results (including, for instance, numerical power metrics) into the 114 standard result reporting pipeline.</p> 115 116 </body> 117 </html> 118
