1 page.title=Running the tests 2 @jd:body 3 4 <!-- 5 Copyright 2015 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18 --> 19 20 <div id="qv-wrapper"> 21 <div id="qv"> 22 <h2>In this document</h2> 23 <ol id="auto-toc"> 24 </ol> 25 </div> 26 </div> 27 <p>This page provides instructions for running deqp tests in Linux and Windows 28 environments, using command line arguments, and working with the Android 29 application package.</p> 30 31 <h2 id=linux_and_windows_environments>Linux and Windows environments</h2> 32 33 <p>Start by copying the following files and directories to the target.</p> 34 35 <table> 36 <tr> 37 <th>Module</th> 38 <th>Directory</th> 39 <th>Target</th> 40 </tr> 41 42 <tr> 43 <td>Execution Server</td> 44 <td><code>build/execserver/execserver</code></td> 45 <td><code><dst>/execserver</code></td> 46 </tr> 47 48 <tr> 49 <td>EGL Module</td> 50 <td><code>build/modules/egl/deqp-egl</code></td> 51 <td><code><dst>/deqp-egl</code></td> 52 </tr> 53 54 <tr> 55 <td rowspan=2 style="vertical-align:middle">GLES2 Module</td> 56 <td><code>build/modules/gles2/deqp-gles2</code></td> 57 <td><code><dst>/deqp-gles2</code></td> 58 </tr> 59 60 61 <tr> 62 <td><code>data/gles2</code></td> 63 <td><code><dst>/gles2</code></td> 64 </tr> 65 66 67 68 <tr> 69 <td rowspan=2 style="vertical-align:middle">GLES3 Module</td> 70 <td><code>build/modules/gles3/deqp-gles3</td> 71 <td><code><dst>/deqp-gles3</code></td> 72 </tr> 73 74 <tr> 75 <td><code>data/gles3</code></td> 76 <td><code><dst>/gles3</code></td> 77 </tr> 78 79 <tr> 80 <td rowspan=2 style="vertical-align:middle">GLES3.1 Module</td> 81 <td><code>build/modules/gles31/deqp-gles31</code></td> 82 <td><code><dst>/deqp-gles31</code></td> 83 </tr> 84 85 <tr> 86 <td><code>data/gles31</code></td> 87 <td><code><dst>/gles31</code></td> 88 </tr> 89 90 91 <tr> 92 <td rowspan=2 style="vertical-align:middle">GLES3.2 Module</td> 93 <td><code>build/modules/gles32/deqp-gles32</code></td> 94 <td><code><dst>/deqp-gles32</code></td> 95 </tr> 96 97 <tr> 98 <td><code>data/gles32</code></td> 99 <td><code><dst>/gles32</code></td> 100 </tr> 101 102 </table> 103 104 <p>You can deploy the execution service and test binaries anywhere in the target 105 file system; however, test binaries expect to find data directories in the 106 current working directory. When ready, start the Test Execution Service on the 107 target device. For details on starting the service, see 108 <a href="{@docRoot}devices/graphics/port-tests.html#test_execution_service">Test 109 execution service</a>.</p> 110 111 <h2 id=command_line_arguments>Command line arguments</h2> 112 113 <p>The following table lists command line arguments that affect execution of all 114 test programs.</p> 115 116 <table width="100%"> 117 <col style="width:50%"> 118 <col style="width:50%"> 119 <tr> 120 <th>Argument</th> 121 <th>Description</th> 122 </tr> 123 124 <tr> 125 <td><code>--deqp-case=<casename></code></td> 126 <td>Run cases that match a given pattern. Wildcard (*) is supported.</td> 127 </tr> 128 129 <tr> 130 <td><code>--deqp-log-filename=<filename></code></td> 131 <td>Write test results to the file whose name you provide. The test execution 132 service will set the filename when starting a test.</td> 133 </tr> 134 135 <tr> 136 <td><code>--deqp-stdin-caselist<br/> 137 --deqp-caselist=<caselist><br/> 138 --deqp-caselist-file=<filename></code></td> 139 <td>Read case list from stdin or from a given argument. The test execution 140 service will set the argument according to the execution request received. See 141 the next section for a description of the case list format.</td> 142 </tr> 143 <tr> 144 <td><code>--deqp-test-iteration-count=<count></code></td> 145 <td>Override iteration count for tests that support a variable number of 146 iterations.</td> 147 </tr> 148 <tr> 149 <td><code>--deqp-base-seed=<seed></code></td> 150 <td>Base seed for the test cases that use randomization.</td> 151 </tr> 152 </table> 153 154 <h3 id=gles2_and_gles3-specific_arguments>GLES2 and GLES3-specific arguments</h3> 155 156 The following table lists the GLES2- and GLES3-specific arguments. 157 158 <table> 159 <table width="100%"> 160 <col style="width:50%"> 161 <col style="width:50%"> 162 <tr> 163 <th>Argument</th> 164 <th>Description</th> 165 </tr> 166 <tr> 167 <td><code>--deqp-gl-context-type=<type></code></td> 168 <td>OpenGL context type. Available context types depend on the platform. On 169 platforms supporting EGL, the value <code>egl</code> can be used to select 170 the EGL context.</td> 171 </tr> 172 <tr> 173 <td><code>--deqp-gl-config-id=<id></code></td> 174 <td>Run tests for the provided GL configuration ID. Interpretation is 175 platform-dependent. On the EGL platform, this is the EGL configuration ID.</td> 176 </tr> 177 <tr> 178 <td><code>--deqp-gl-config-name=<name></code></td> 179 <td><p>Run tests for a named GL configuration. Interpretation is 180 platform-dependent. For EGL, the format is 181 <code>rgb(a)<bits>d<bits>s<bits></code>. For example, a 182 value of <code>rgb888s8</code> will select the first configuration where the 183 color buffer is RGB888 and the stencil buffer has 8 bits.</td> 184 </tr> 185 <tr> 186 <td><code>--deqp-gl-context-flags=<flags></code></td> 187 <td>Creates a context. Specify <code>robust</code> or <code>debug</code>.</td> 188 </tr> 189 <tr> 190 <td><code>--deqp-surface-width=<width><br/> 191 --deqp-surface-height=<height></code></td> 192 <td>Try to create a surface with a given size. Support for this is optional.</td> 193 </tr> 194 <tr> 195 <td><code>--deqp-surface-type=<type></code></td> 196 <td>Use a given surface type as the main test rendering target. Possible 197 types are <code>window</code>, <code>pixmap</code>, <code>pbuffer</code>, 198 and <code>fbo</code>.</td> 199 </tr> 200 <tr> 201 <td><code>--deqp-screen-rotation=<rotation></code></td> 202 <td>Screen orientation in increments of 90 degrees for platforms that 203 support it.</td> 204 </tr> 205 </table> 206 207 <h3 id=test_case_list_format>Test case list format</h3> 208 209 <p>The test case list can be given in two formats. The first option is to list 210 the full name of each test on a separate line in a standard ASCII file. As the 211 test sets grow, the repetitive prefixes can be cumbersome. To avoid repeating 212 the prefixes, use a trie (also known as a prefix tree) syntax shown below.</p> 213 214 <pre> 215 {nodeName{firstChild{},lastChild{}}} 216 </pre> 217 218 <p>For example:</p> 219 220 <pre> 221 {dEQP-EGL{config-list,create_context{rgb565_depth_stencil}}} 222 </pre> 223 224 <p>Translates into the following two test cases:</p> 225 226 <pre> 227 dEQP-EGL.config_list 228 dEQP-EGL.create_context.rgb565_depth_stencil 229 </pre> 230 231 <h2 id=android>Android</h2> 232 233 <p>The Android application package contains all required components, including 234 the test execution service, test binaries, and data files. The test activity is 235 a <code>NativeActivity</code> that uses EGL (requires Android 3.2 or higher).</p> 236 237 <p>The application package can be installed with the following command (name 238 shown is the name of the APK in the Android CTS package; which name depends on 239 the build):</p> 240 <pre>$ adb d install r com.drawelements.deqp.apk</pre> 241 242 <p>To launch the test execution service and to setup port forwarding, use the 243 following:</p> 244 <pre> 245 $ adb d forward tcp:50016 tcp:50016 246 $ adb d shell am start n com.drawelements.deqp/.execserver.ServiceStarter 247 </pre> 248 249 <p>Debug prints can be enabled by executing the following before starting the 250 tests:</p> 251 252 <pre> 253 $ adb d shell setprop log.tag.dEQP DEBUG 254 </pre> 255 256 <h3 id=executing_tests_on_android_without_android_cts>Executing tests on 257 Android without Android CTS</h3> 258 259 <p>To manually start the test execution activity, construct an Android intent 260 that targets <code>android.app.NativeActivity</code>. The activities can be 261 found in the <code>com.drawelements.deqp</code> package. The command line must 262 be supplied as an extra string with key <code>"cmdLine"</code> in the Intent.</p> 263 264 <p>A test log is written to <code>/sdcard/dEQP-log.qpa</code>. If the test run 265 does not start normally, additional debug information is available in the device 266 log.</p> 267 268 <p>You can launch an activity from the command line using the <code>am</code> 269 utility. For example, to run <code>dEQP-GLES2.info</code> tests on a platform 270 supporting <code>NativeActivity,</code> use the following commands.</p> 271 272 <pre>$ adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e \ 273 cmdLine "deqp --deqp-case=dEQP-GLES2.info.* --deqp-log-filename=/sdcard/dEQP-Log.qpa 274 </pre> 275 276 <h3 id=debugging_on_android>Debugging on Android</h3> 277 278 <p>To run the tests under the GDB debugger on Android, first compile and install 279 the debug build by running the following two scripts:</p> 280 281 <pre> 282 $ python android/scripts/build.py --native-build-type=Debug 283 $ python android/scripts/install.py 284 </pre> 285 286 <p>After the debug build is installed on the device, to launch the tests under 287 GDB running on the host, run the following command:</p> 288 289 <pre>$ python android/scripts/debug.py \ 290 --deqp-commandline="--deqp-log-filename=/sdcard/TestLog.qpa --deqp-case=dEQP-GLES2.functional.*" 291 </pre> 292 293 <p>The deqp command line depends on the test cases to be executed and other 294 required parameters. The script adds a default breakpoint at the beginning of 295 the deqp execution (<code>tcu::App::App</code>).</p> 296 297 <p>The <code>debug.py</code> script accepts multiple command line arguments for 298 actions such as setting breakpoints for debugging, gdbserver connection 299 parameters, and paths to additional binaries to debug (use <code>debug.py 300 --help</code> for all arguments and explanations). The script also copies some 301 default libraries from the target device to get symbol listings.</p> 302 303 <p>To step through driver code (such as when the GDB needs to know the locations 304 of the binaries with full debug information), add more libraries via 305 <code>debug.py</code> command line parameters. This script writes out a 306 configuration file for the GDB starting from line 132 of the script file. You 307 can provide additional paths to binaries, etc., but supplying correct command 308 line parameters should be enough.</p> 309 310 <p class="note"><strong>Note:</strong> On Windows, the GDB binary requires 311 <code>libpython2.7.dll</code>. Before launching <code>debug.py</code>, add 312 <code><path-to-ndk>/prebuilt/windows/bin</code> to the PATH variable.</p> 313 314 <p class="note"><strong>Note:</strong> Native code debugging does not work on 315 stock Android 4.3; for workarounds, refer to 316 <a href="https://code.google.com/p/android/issues/detail?id=58373">https://code.google.com/p/android/issues/detail?id=58373</a>. 317 Android 4.4 and higher do not contain this bug.</p> 318
