1 page.title=CTS Development 2 @jd:body 3 4 <!-- 5 Copyright 2015 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 <div id="qv-wrapper"> 20 <div id="qv"> 21 <h2>In this document</h2> 22 <ol id="auto-toc"> 23 </ol> 24 </div> 25 </div> 26 27 <h2 id="initializing-your-repo-client">Initializing your Repo client</h2> 28 <p>Follow the <a href="{@docRoot}source/downloading.html">instructions</a> 29 to get and build the Android source code but specify a particular CTS branch 30 name, for example<code>-b android-5.0_r2</code>, when issuing the <code>repo 31 init</code> command. This assures your CTS changes will be included in the 32 next CTS release and beyond.</p> 33 34 <h2 id="building-and-running-cts">Building and running CTS</h2> 35 36 <p>Execute the following commands to build CTS and start the interactive 37 CTS console:</p> 38 <p class="note"><strong>Note:</strong> You may supply one of these other values 39 for <code>TARGET_PRODUCT</code> to build for different architectures: 40 <code>aosp_x86_64</code> or <code>aosp_mips</code></p> 41 <pre><code>cd <em>/path/to/android/root</em> 42 make cts -j32 TARGET_PRODUCT=aosp_arm64 43 cts-tradefed 44 </code></pre> 45 46 <p>At the cts-tf console, enter e.g.:</p> 47 <pre><code>run cts --plan CTS 48 </code></pre> 49 50 <h2 id="writing-cts-tests">Writing CTS tests</h2> 51 52 <p>CTS tests use JUnit and the Android testing APIs. Review the 53 <a href="https://developer.android.com/tools/testing/testing_android.html">Testing and Instrumentation</a> 54 tutorial while perusing the existing tests under the 55 <code>cts/tests</code> directory. You will see that CTS tests mostly follow the same 56 conventions used in other Android tests.</p> 57 <p>Since CTS runs across many production devices, the tests must follow 58 these rules:</p> 59 <ul> 60 <li>Must take into account varying screen sizes, orientations, and keyboard layouts.</li> 61 <li>Only use public API methods. In other words, avoid all classes, methods, and fields that are annotated with the "hide" annotation.</li> 62 <li>Avoid relying upon particular view layouts or depend on the dimensions of assets that may not be on some device.</li> 63 <li>Don't rely upon root privileges.</li> 64 </ul> 65 66 <h3 id="test-naming-and-location">Test naming and location</h3> 67 68 <p>Most CTS test cases target a specific class in the Android API. These tests 69 have Java package names with a <code>cts</code> suffix and class 70 names with the <code>Test</code> suffix. Each test case consists of 71 multiple tests, where each test usually exercises a particular API method of 72 the API class being tested. These tests are arranged in a directory structure 73 where tests are grouped into different categories like "widgets" and "views."</p> 74 75 <p>For example, the CTS test for <code>android.widget.TextView</code> is 76 <code>android.widget.cts.TextViewTest</code> found under the 77 <code>cts/tests/tests/widget/src/android/widget/cts</code> directory with its 78 Java package name as <code>android.widget.cts</code> and its class name as 79 <code>TextViewTest</code>. The <code>TextViewTest</code> class has a test called <code>testSetText</code> 80 that exercises the "setText" method and a test named "testSetSingleLine" that 81 calls the <code>setSingleLine</code> method. Each of those tests have <code>@TestTargetNew</code> 82 annotations indicating what they cover.</p> 83 84 <p>Some CTS tests do not directly correspond to an API class but are placed in 85 the most related package possible. For instance, the CTS test, 86 <code>android.net.cts.ListeningPortsTest</code>, is in the <code>android.net.cts</code>, because it 87 is network related even though there is no <code>android.net.ListeningPorts</code> class. 88 You can also create a new test package if necessary. For example, there is an 89 "android.security" test package for tests related to security. Thus, use your 90 best judgement when adding new tests and refer to other tests as examples.</p> 91 92 <p>Finally, a lot of tests are annotated with @TestTargets and @TestTargetNew. 93 These are no longer necessary so do not annotate new tests with these.</p> 94 <h3 id="new-test-packages">New sample packages</h3> 95 <p>When adding new tests, there may not be an existing directory to place your 96 test. In that case, refer to the example under <code>cts/tests/tests/example</code> and 97 create a new directory. Furthermore, make sure to add your new package's 98 module name from its <code>Android.mk</code> to <code>CTS_COVERAGE_TEST_CASE_LIST</code> in 99 <code>cts/CtsTestCaseList.mk</code>. This Makefile is used by <code>build/core/tasks/cts.mk</code> 100 to glue all the tests together to create the final CTS package.</p> 101 102 <h2 id="Fix-remove-tests">Fix or remove tests</h2> 103 <p>Besides adding new tests there are other ways to contribute to CTS: Fix or 104 remove tests annotated with "BrokenTest" or "KnownFailure."</p> 105 <h2 id="submitting-your-changes">Submitting your changes</h2> 106 <p>Follow the <a href="{@docRoot}source/submit-patches.html">Submitting Patches workflow</a> 107 to contribute changes to CTS. A reviewer 108 will be assigned to your change, and your change should be reviewed shortly!</p> 109 110 <h2 id="release-schedule">Release schedule and branch information</h2> 111 112 <p>CTS releases follow this schedule.</p> 113 114 <p class="note"><strong>Note</strong>: This schedule is tentative and may be 115 updated from time to time as CTS for the given Android version matures.</p> 116 117 <table> 118 <tr> 119 <th>Version</th> 120 <th>Branch</th> 121 <th>Frequency</th> 122 </tr> 123 </thead> 124 <tbody> 125 <tr> 126 <td>5.1</td> 127 <td>lollipop-mr1-cts-dev</td> 128 <td>Monthly</td> 129 </tr> 130 <tr> 131 <td>5.0</td> 132 <td>lollipop-cts-dev</td> 133 <td>Monthly</td> 134 </tr> 135 <tr> 136 <td>4.4</td> 137 <td>kitkat-cts-dev</td> 138 <td>Odd month (Jan, Mar, etc.)</td> 139 </tr> 140 <tr> 141 <td>4.3</td> 142 <td>jb-mr2-cts-dev</td> 143 <td>First month of each quarter</td> 144 </tr> 145 <tr> 146 <td>4.2</td> 147 <td>jb-mr1.1-cts-dev</td> 148 <td>First month of each quarter</td> 149 </tr> 150 </table> 151 152 <h3 id="important-dates">Important Dates during month of the release</h3> 153 154 <ul> 155 <li><strong>End of 1st Week</strong>: Code Freeze. At this point, 156 submissions on the current branch will no longer be accepted and will not be 157 included in the next version of CTS. Once we have chosen a candidate for 158 release, the branch will again be open and accepting new submissions. 159 160 <li><strong>Second or third week</strong>: CTS is published in the Android 161 Open Source Project (AOSP). 162 </ul> 163 164 <h3 id="auto-merge">Auto-merge flow</h3> 165 166 <p>CTS development branches have been setup so that changes submitted to each 167 branch will automatically merge as below:<br> 168 jb-dev-> jb-mr1.1-cts-dev -> jb-mr2-cts-dev -> kitkat-cts-dev -> 169 lollipop-cts-dev -> lollipop-mr1-cts-dev -> <private-development-branch for 170 Android M></p> 171 172 <p>If a changelist (CL) fails to merge correctly, the author of the CL will get 173 an email with instructions on how to resolve the conflict. In most of the 174 cases, the author of the CL can use the instructions to skip the auto-merge of 175 the conflicting CL.</p> 176
