1 page.title=Trade Federation Overview 2 @jd:body 3 4 <!-- 5 Copyright 2013 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <p>Trade Federation (tradefed or TF for short) is a continuous test framework designed for running tests 20 on Android devices. It's a Java application which runs on a host computer, and communicates to one or 21 more Android devices using ddmlib (the library behind DDMS) over adb.</p> 22 23 <p>We've listed some of TF's main features below, along with a couple sample usecases. That said, 24 if you want to jump right in and get started, you can head straight for the 25 <a href="/devices/tech/test_infra/tradefed/fundamentals/index.html">Start Here</a> page.</p> 26 27 <h2>Features</h2> 28 <ul> 29 <li>modular, flexible, scalable design</li> 30 <li>has built in support for running many different types of Android tests: 31 <a href="http://developer.android.com/tools/testing/testing_android.html#Instrumentation">instrumentation</a>, 32 <a href="http://developer.android.com/tools/testing/testing_ui.html">uiautomator</a>, 33 native/gtest, host-based JUnit, etc</li> 34 <li>provides reliability and recovery mechanisms on top of adb</li> 35 <li>supports scheduling and running tests on multiple devices in parallel</li> 36 </ul> 37 38 <h2>Sample Trade Federation Usecases</h2> 39 <p>Trade Federation's modularity makes it straightforward to slot into environments with existing 40 build, test, and reporting infrastructures. We list below a few demonstrative 41 usecases where tradefed could enable efficient, scalable test practices.</p> 42 43 <p>First, it is useful to consider the landscape of potential usecases in terms of the question 44 "which parts are modifiable, and what parts are static?" For instance, a Device OEM can modify the 45 framework, the system, and the hardware, but has little or no influence over existing applications. 46 An application developer, on the other hand, can modify the app, but has little control over most 47 aspects of the system or the framework.</p> 48 49 <p>As a result, an entity in each usecase will have different testing goals, and will have different 50 options in the case of a set of test failures. Despite these differences, Trade Federation can 51 help make each of their test processes efficient, flexible, and scalable.</p> 52 53 <h3>Device OEM</h3> 54 <p>A Device OEM builds hardware, and will often tweak the Android system and frameworks to run well 55 on that hardware. The OEM might strive to accomplish those goals while retaining stability 56 and performance at the hardware and system levels, and making sure the framework changes don't break 57 compatibility with existing applications.</p> 58 59 <p>The OEM could implement a device flashing module that will execute during the Target Setup stage 60 of the <a href="/devices/tech/test_infra/tradefed/fundamentals/lifecycle.html">lifecycle</a>. That 61 module would have complete control over the device during its execution period, which would allow 62 it to potentially force the device into the bootloader, flash, and then force the device to reboot 63 back into userspace mode. Combined with a module to tie into a continuous build system, this would 64 allow the OEM to run tests on their device as they make changes to the system-level firmware and 65 Java-level frameworks.</p> 66 67 <p>Once the device is fully booted, the OEM would be able to leverage existing JUnit-based tests, 68 or write new ones, to verify the functionality of interest. Finally, they could write one or more 69 result reporting modules to tie into existing test-result repositories, or to report results 70 directly (for instance, 71 <a href="/reference/com/android/tradefed/result/EmailResultReporter.html">by email</a>).</p> 72 73 <h3>App Developer</h3> 74 <p>An Application Developer builds an app which needs to run well across a variety of platform 75 versions and a variety of devices. If an issue comes up on a particular platform version and/or 76 device, the only remedy is to add a workaround and move on. For larger developers, the test 77 process might be incorporated into a continuous build sequence. For smaller developers, it 78 might be kicked off periodically or by hand.</p> 79 80 <p>Most app developers would use the apk test installation modules that already exist in TF. 81 There's a version that <a href="/reference/com/android/tradefed/targetprep/InstallApkSetup.html" 82 >installs from the local filesystem</a>, as well as a version that can 83 <a href="/reference/com/android/tradefed/targetprep/InstallBuildEnvApkSetup.html">install 84 apks downloaded from a build service</a>. It is important to note that the latter version would 85 continue to work properly with arbitrarily many TF instances running on the same host machine.</p> 86 87 <p>Because of TF's proficiency at dealing with multiple devices, it would be straightforward to 88 classify each test result by the type of device that was used for that test. Thus, TF could 89 potentially generate a 2-dimensional (or multi-dimensional) compatibility matrix for every 90 build of the application.</p> 91 92 <h3>Testing Service</h3> 93 <p>A Test Service might, for instance, allow app developers to submit apps and run tests on devices 94 instrumented with power-measurement tools to determine power usage for the app. This differs from 95 the prior two usecases in that the service builder does not control the devices or the applications 96 that are being run</p> 97 98 <p>Because Trade Federation can run any Java class that implements the simple 99 <a href="/reference/com/android/tradefed/testtype/IRemoteTest.html">IRemoteTest</a> interface, it's 100 trivial to write drivers that can coordinate some external piece of hardware with the test case 101 that's being run on the device. The driver itself can spawn Threads, send requests to other 102 servers, or do anything else that it might need. Moreover, the simplicity and versatility of the 103 result reporting interface, 104 <a href="/reference/com/android/tradefed/result/ITestInvocationListener.html" 105 >ITestInvocationListener</a>, means that it is likewise straightforward to 106 represent arbitrary test results (including, for instance, numerical power metrics) into the 107 standard result reporting pipeline.</p> 108
