1 page.title=Running the tests 2 @jd:body 3 4 <!-- 5 Copyright 2015 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 20 <div id="qv-wrapper"> 21 <div id="qv"> 22 <h2>In this document</h2> 23 <ol id="auto-toc"> 24 </ol> 25 </div> 26 </div> 27 28 29 <h2 id=linux_and_windows_environments>Linux and Windows environments</h2> 30 31 <p>The following files and directories must be copied to the target.</p> 32 33 <table> 34 <tr> 35 <th>Module</th> 36 <th>Directory</th> 37 <th>Target</th> 38 </tr> 39 40 <tr> 41 <td><p>Execution Server</p></td> 42 <td><code>build/execserver/execserver</code></td> 43 <td><code><dst>/execserver</code></td> 44 </tr> 45 46 <tr> 47 <td><p>EGL Module</p></td> 48 <td><code>build/modules/egl/deqp-egl</code></td> 49 <td><code><dst>/deqp-egl</code></td> 50 </tr> 51 52 <tr> 53 <td><p>GLES2 Module</p></td> 54 <td><code>build/modules/gles2/deqp-gles2data/gles2</code></td> 55 <td> 56 <code> 57 <dst>/deqp-gles2<br/> 58 <dst>/gles2 59 </code> 60 </td> 61 </tr> 62 63 <tr> 64 <td><p>GLES3 Module</p></td> 65 <td><code>build/modules/gles3/deqp-gles3data/gles3</code></td> 66 <td> 67 <code> 68 <dst>/deqp-gles3<br/> 69 <dst>/gles3 70 </code> 71 </td> 72 </tr> 73 74 <tr> 75 <td><p>GLES3.1 Module</p></td> 76 <td><code>build/modules/gles31/deqp-gles31data/gles31</code></td> 77 <td> 78 <code> 79 <dst>/deqp-gles31<br/> 80 <dst>/gles31 81 </code> 82 </td> 83 </tr> 84 </table> 85 86 <p>Execution service and test binaries can be deployed anywhere in the target file system. Test binaries expect to find data directories in the current working directory.</p> 87 88 <p>Start the Test Execution Service on the target device. For more details on 89 starting the service, see <a href="port-tests.html#test_execution_service">Test execution service</a>.</p> 90 91 <h2 id=command_line_arguments>Command line arguments</h2> 92 93 <p>The following table lists command line arguments that affect execution of all test programs. </p> 94 95 <table width="100%"> 96 <col style="width:50%"> 97 <col style="width:50%"> 98 <tr> 99 <th>Argument</th> 100 <th>Description</th> 101 </tr> 102 103 <tr> 104 <td><code> 105 --deqp-case=<casename></code></td> 106 <td><p>Run cases that match a given pattern. Wildcard (*) is supported.</p> 107 </td> 108 </tr> 109 110 <tr> 111 <td><code> 112 --deqp-log-filename=<filename></code></td> 113 <td><p>Write test results to the file whose name you provide. </p> 114 <p>The test execution service will set the filename when starting a test.</p> 115 </td> 116 </tr> 117 118 <tr> 119 <td><code> 120 --deqp-stdin-caselist<br/> 121 --deqp-caselist=<caselist><br/> 122 --deqp-caselist-file=<filename></code></td> 123 <td><p>Read case list from stdin or from a given argument. The test execution service 124 will set the argument according to the execution request received. See the next 125 section for a description of the case list format.</p> 126 </td> 127 </tr> 128 <tr> 129 <td><code> 130 --deqp-test-iteration-count=<count></code></td> 131 <td><p>Override iteration count for tests that support a variable number of 132 iterations. </p> 133 </td> 134 </tr> 135 <tr> 136 <td><code> 137 --deqp-base-seed=<seed></code></td> 138 <td><p>Base seed for the test cases that use randomization.</p> 139 </td> 140 </tr> 141 </table> 142 143 <h3 id=gles2_and_gles3-specific_arguments>GLES2 and GLES3-specific arguments</h3> 144 145 The following table lists the GLES2- and GLES3-specific arguments. 146 147 <table> 148 <table width="100%"> 149 <col style="width:50%"> 150 <col style="width:50%"> 151 <tr> 152 <th>Argument</th> 153 <th>Description</th> 154 </tr> 155 <tr> 156 <td><code> 157 --deqp-gl-context-type=<type></code></td> 158 <td><p>OpenGL context type. Available context types depend on the platform. On 159 platforms supporting EGL, the value <code>egl</code> can be used to select the EGL context.</p> 160 </td> 161 </tr> 162 <tr> 163 <td><code> 164 --deqp-gl-config-id=<id></code></td> 165 <td><p>Run tests for the provided GL configuration ID. Interpretation is 166 platform-dependent. On the EGL platform, this is the EGL configuration ID.</p> 167 </td> 168 </tr> 169 <tr> 170 <td><code> 171 --deqp-gl-config-name=<name></code></td> 172 <td><p>Run tests for a named GL configuration. Interpretation is platform-dependent. 173 For EGL, the format is <code>rgb(a)<bits>d<bits>s<bits></code>. For example, a value of <code>rgb888s8</code> will select the first configuration where the color buffer is RGB888 and the 174 stencil buffer has 8 bits.</p> 175 </td> 176 </tr> 177 <tr> 178 <td><code> 179 --deqp-gl-context-flags=<flags></code></td> 180 <td><p>Creates a context. Specify <code>robust</code> or <code>debug</code>.</p> 181 </td> 182 </tr> 183 <tr> 184 <td><code> 185 --deqp-surface-width=<width><br/> 186 --deqp-surface-height=<height></code></td> 187 <td><p>Try to create a surface with a given size. Support for this is optional.</p> 188 </td> 189 </tr> 190 <tr> 191 <td><code> 192 --deqp-surface-type=<type></code></td> 193 <td><p>Use a given surface type as the main test rendering target. Possible types are <code>window</code>, <code>pixmap</code>, <code>pbuffer</code>, and <code>fbo</code>.</p> 194 </td> 195 </tr> 196 <tr> 197 <td><code> 198 --deqp-screen-rotation=<rotation></code></td> 199 <td><p>Screen orientation in increments of 90 degrees for platforms that support it.</p> 200 </td> 201 </tr> 202 </table> 203 204 <h3 id=test_case_list_format>Test case list format</h3> 205 206 <p>The test case list can be given in two formats. The first option is to list the 207 full name of each test on a separate line in a standard ASCII file. As the test 208 sets grow, the repetitive prefixes can be cumbersome. To avoid repeating the 209 prefixes, use a trie (also known as a prefix tree) syntax shown below.</p> 210 211 <pre> 212 {nodeName{firstChild{},lastChild{}}} 213 </pre> 214 215 <p>For example, please review the following:</p> 216 217 <pre> 218 {dEQP-EGL{config-list,create_context{rgb565_depth_stencil}}} 219 </pre> 220 221 <p>That list would translate into two test cases:</p> 222 223 <pre> 224 dEQP-EGL.config_list 225 dEQP-EGL.create_context.rgb565_depth_stencil 226 </pre> 227 228 <h2 id=android>Android</h2> 229 230 <p>The Android application package contains everything required, including the 231 test execution service, test binaries, and data files. The test activity is a <code>NativeActivity </code>and it uses EGL, which requires Android 3.2 or later.</p> 232 233 <p>The application package can be installed with the following command. The name shown is the name of the APK in the Android CTS package. The name depends on the build:</p> 234 <pre> 235 adb d install r com.drawelements.deqp.apk 236 </pre> 237 238 <p>To launch the test execution service and to setup port forwarding, use the 239 following:</p> 240 <pre> 241 adb d forward tcp:50016 tcp:50016 242 adb d shell am start n com.drawelements.deqp/.execserver.ServiceStarter 243 </pre> 244 245 <p>Debug prints can be enabled by executing the following before starting the 246 tests:</p> 247 <pre> 248 adb d shell setprop log.tag.dEQP DEBUG 249 </pre> 250 251 <h3 id=executing_tests_on_android_without_android_cts>Executing tests on Android without Android CTS</h3> 252 253 <p>If you want to manually start the test execution activity, construct an Android 254 intent that targets <code>android.app.NativeActivity</code>. The activities can be found in the <code>com.drawelements.deqp</code> package. The command line must be supplied as an extra string with key <code>"cmdLine"</code> in the Intent.</p> 255 256 <p>A test log will be written to <code>/sdcard/dEQP-log.qpa</code>. If the test run does not start normally, additional debug information is 257 available in the device log.</p> 258 259 <p>The activity can be launched from the command line using the <code>"am"</code> utility. For example, to run <code>dEQP-GLES2.info</code> tests on a platform supporting <code>NativeActivity,</code> the following command can be used:</p> 260 261 <pre> 262 adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e 263 cmdLine "deqp --deqp-case=dEQP-GLES2.info.* 264 --deqp-log-filename=/sdcard/dEQP-Log.qpa 265 </pre> 266 267 <h3 id=debugging_on_android>Debugging on Android</h3> 268 269 <p>To run the tests under the GDB debugger on Android, first compile and install 270 the debug build by running the following two scripts:</p> 271 272 <pre> 273 python android/scripts/build.py --native-build-type=Debug 274 python android/scripts/install.py 275 </pre> 276 277 <p>After the debug build is installed on the device, to launch the tests under GDB 278 running on the host, run the following command:</p> 279 280 <pre> 281 python android/scripts/debug.py 282 --deqp-commandline="--deqp-log-filename=/sdcard/TestLog.qpa 283 --deqp-case=dEQP-GLES2.functional.*" 284 </pre> 285 286 <p>The deqp command line will depend on test cases to be executed and other 287 required parameters. The script will add a default breakpoint into the 288 beginning of the deqp execution (<code>tcu::App::App</code>).</p> 289 290 <p>The <code>debug.py </code>script accepts multiple command line arguments, e.g. for the following:</p> 291 292 <ul> 293 <li> Setting the breakpoints for debugging 294 <li> gdbserver connection parameters 295 <li> Paths to additional binaries to debug 296 </ul> 297 298 <p>Running <code>debug.py --help</code> will list all command line parameters, with explanations.</p> 299 300 <p>The script copies some default libraries from the target device to get symbol 301 listings. </p> 302 303 <p>If there is a need to step through driver code, more libraries can be added via <code>debug.py</code> command line parameters. This would be applicable, for 304 example, if the GDB needs to know the locations of the binaries with full debug 305 information. The <code>debug.py</code> script writes out a configuration file for the GDB starting from line 132 of 306 the script file. Additional paths to binaries, etc., can be added there, but 307 supplying correct command line parameters should be enough.</p> 308 309 <p><strong>Notes:</strong></p> 310 311 <ul> 312 <li> On Windows, the gdb binary requires <code>libpython2.7.dll</code>. Add 313 <code><path to ndk>/prebuilt/windows/bin</code> to the PATH variable before launching <code>debug.py</code>. 314 <li> Native code debugging does not work on stock Android 4.3. See the Android bug 315 report below for suggested workarounds. The bug has been fixed in Android 4.4; 316 see the following: <a href="https://code.google.com/p/android/issues/detail?id=58373">https://code.google.com/p/android/issues/detail?id=58373</a> 317 </ul> 318
