1 <html devsite> 2 <head> 3 <title>Running the tests</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 <p>This page provides instructions for running deqp tests in Linux and Windows 26 environments, using command line arguments, and working with the Android 27 application package.</p> 28 29 <h2 id=linux_and_windows_environments>Linux and Windows environments</h2> 30 31 <p>Start by copying the following files and directories to the target.</p> 32 33 <table> 34 <tr> 35 <th>Module</th> 36 <th>Directory</th> 37 <th>Target</th> 38 </tr> 39 40 <tr> 41 <td>Execution Server</td> 42 <td><code>build/execserver/execserver</code></td> 43 <td><code><dst>/execserver</code></td> 44 </tr> 45 46 <tr> 47 <td>EGL Module</td> 48 <td><code>build/modules/egl/deqp-egl</code></td> 49 <td><code><dst>/deqp-egl</code></td> 50 </tr> 51 52 <tr> 53 <td rowspan=2 style="vertical-align:middle">GLES2 Module</td> 54 <td><code>build/modules/gles2/deqp-gles2</code></td> 55 <td><code><dst>/deqp-gles2</code></td> 56 </tr> 57 58 59 <tr> 60 <td><code>data/gles2</code></td> 61 <td><code><dst>/gles2</code></td> 62 </tr> 63 64 65 66 <tr> 67 <td rowspan=2 style="vertical-align:middle">GLES3 Module</td> 68 <td><code>build/modules/gles3/deqp-gles3</code></td> 69 <td><code><dst>/deqp-gles3</code></td> 70 </tr> 71 72 <tr> 73 <td><code>data/gles3</code></td> 74 <td><code><dst>/gles3</code></td> 75 </tr> 76 77 <tr> 78 <td rowspan=2 style="vertical-align:middle">GLES3.1 Module</td> 79 <td><code>build/modules/gles31/deqp-gles31</code></td> 80 <td><code><dst>/deqp-gles31</code></td> 81 </tr> 82 83 <tr> 84 <td><code>data/gles31</code></td> 85 <td><code><dst>/gles31</code></td> 86 </tr> 87 88 89 <tr> 90 <td rowspan=2 style="vertical-align:middle">GLES3.2 Module</td> 91 <td><code>build/modules/gles32/deqp-gles32</code></td> 92 <td><code><dst>/deqp-gles32</code></td> 93 </tr> 94 95 <tr> 96 <td><code>data/gles32</code></td> 97 <td><code><dst>/gles32</code></td> 98 </tr> 99 100 </table> 101 102 <p>You can deploy the execution service and test binaries anywhere in the target 103 file system; however, test binaries expect to find data directories in the 104 current working directory. When ready, start the Test Execution Service on the 105 target device. For details on starting the service, see 106 <a href="/devices/graphics/port-tests.html#test_execution_service">Test 107 execution service</a>.</p> 108 109 <h2 id=command_line_arguments>Command line arguments</h2> 110 111 <p>The following table lists command line arguments that affect execution of all 112 test programs.</p> 113 114 <table width="100%"> 115 <col style="width:50%"> 116 <col style="width:50%"> 117 <tr> 118 <th>Argument</th> 119 <th>Description</th> 120 </tr> 121 122 <tr> 123 <td><code>--deqp-case=<casename></code></td> 124 <td>Run cases that match a given pattern. Wildcard (*) is supported.</td> 125 </tr> 126 127 <tr> 128 <td><code>--deqp-log-filename=<filename></code></td> 129 <td>Write test results to the file whose name you provide. The test execution 130 service will set the filename when starting a test.</td> 131 </tr> 132 133 <tr> 134 <td><code>--deqp-stdin-caselist<br/> 135 --deqp-caselist=<caselist><br/> 136 --deqp-caselist-file=<filename></code></td> 137 <td>Read case list from stdin or from a given argument. The test execution 138 service will set the argument according to the execution request received. See 139 the next section for a description of the case list format.</td> 140 </tr> 141 <tr> 142 <td><code>--deqp-test-iteration-count=<count></code></td> 143 <td>Override iteration count for tests that support a variable number of 144 iterations.</td> 145 </tr> 146 <tr> 147 <td><code>--deqp-base-seed=<seed></code></td> 148 <td>Base seed for the test cases that use randomization.</td> 149 </tr> 150 </table> 151 152 <h3 id=gles2_and_gles3-specific_arguments>GLES2 and GLES3-specific arguments</h3> 153 154 The following table lists the GLES2- and GLES3-specific arguments. 155 156 <table> 157 <table width="100%"> 158 <col style="width:50%"> 159 <col style="width:50%"> 160 <tr> 161 <th>Argument</th> 162 <th>Description</th> 163 </tr> 164 <tr> 165 <td><code>--deqp-gl-context-type=<type></code></td> 166 <td>OpenGL context type. Available context types depend on the platform. On 167 platforms supporting EGL, the value <code>egl</code> can be used to select 168 the EGL context.</td> 169 </tr> 170 <tr> 171 <td><code>--deqp-gl-config-id=<id></code></td> 172 <td>Run tests for the provided GL configuration ID. Interpretation is 173 platform-dependent. On the EGL platform, this is the EGL configuration ID.</td> 174 </tr> 175 <tr> 176 <td><code>--deqp-gl-config-name=<name></code></td> 177 <td><p>Run tests for a named GL configuration. Interpretation is 178 platform-dependent. For EGL, the format is 179 <code>rgb(a)<bits>d<bits>s<bits></code>. For example, a 180 value of <code>rgb888s8</code> will select the first configuration where the 181 color buffer is RGB888 and the stencil buffer has 8 bits.</td> 182 </tr> 183 <tr> 184 <td><code>--deqp-gl-context-flags=<flags></code></td> 185 <td>Creates a context. Specify <code>robust</code> or <code>debug</code>.</td> 186 </tr> 187 <tr> 188 <td><code>--deqp-surface-width=<width><br/> 189 --deqp-surface-height=<height></code></td> 190 <td>Try to create a surface with a given size. Support for this is optional.</td> 191 </tr> 192 <tr> 193 <td><code>--deqp-surface-type=<type></code></td> 194 <td>Use a given surface type as the main test rendering target. Possible 195 types are <code>window</code>, <code>pixmap</code>, <code>pbuffer</code>, 196 and <code>fbo</code>.</td> 197 </tr> 198 <tr> 199 <td><code>--deqp-screen-rotation=<rotation></code></td> 200 <td>Screen orientation in increments of 90 degrees for platforms that 201 support it.</td> 202 </tr> 203 </table> 204 205 <h3 id=test_case_list_format>Test case list format</h3> 206 207 <p>The test case list can be given in two formats. The first option is to list 208 the full name of each test on a separate line in a standard ASCII file. As the 209 test sets grow, the repetitive prefixes can be cumbersome. To avoid repeating 210 the prefixes, use a trie (also known as a prefix tree) syntax shown below.</p> 211 212 <pre class="devsite-click-to-copy"> 213 {nodeName{firstChild{},lastChild{}}} 214 </pre> 215 216 <p>For example:</p> 217 218 <pre class="devsite-click-to-copy"> 219 {dEQP-EGL{config-list,create_context{rgb565_depth_stencil}}} 220 </pre> 221 222 <p>Translates into the following two test cases:</p> 223 224 <pre class="devsite-click-to-copy"> 225 dEQP-EGL.config_list 226 dEQP-EGL.create_context.rgb565_depth_stencil 227 </pre> 228 229 <h2 id=android>Android</h2> 230 231 <p>The Android application package contains all required components, including 232 the test execution service, test binaries, and data files. The test activity is 233 a <code>NativeActivity</code> that uses EGL (requires Android 3.2 or higher).</p> 234 235 <p>The application package can be installed with the following command (name 236 shown is the name of the APK in the Android CTS package; which name depends on 237 the build):</p> 238 <pre class="devsite-terminal devsite-click-to-copy"> 239 adb d install r com.drawelements.deqp.apk 240 </pre> 241 242 <p>To launch the test execution service and to setup port forwarding, use the 243 following:</p> 244 <pre class="devsite-click-to-copy"> 245 <code class="devsite-terminal">adb d forward tcp:50016 tcp:50016</code> 246 <code class="devsite-terminal">adb d shell am start n com.drawelements.deqp/.execserver.ServiceStarter</code> 247 </pre> 248 249 <p>Debug prints can be enabled by executing the following before starting the 250 tests:</p> 251 252 <pre class="devsite-terminal devsite-click-to-copy"> 253 adb d shell setprop log.tag.dEQP DEBUG 254 </pre> 255 256 <h3 id=executing_tests_on_android_without_android_cts>Executing tests on 257 Android without Android CTS</h3> 258 259 <p>To manually start the test execution activity, construct an Android intent 260 that targets <code>android.app.NativeActivity</code>. The activities can be 261 found in the <code>com.drawelements.deqp</code> package. The command line must 262 be supplied as an extra string with key <code>"cmdLine"</code> in the Intent.</p> 263 264 <p>A test log is written to <code>/sdcard/dEQP-log.qpa</code>. If the test run 265 does not start normally, additional debug information is available in the device 266 log.</p> 267 268 <p>You can launch an activity from the command line using the <code>am</code> 269 utility. For example, to run <code>dEQP-GLES2.info</code> tests on a platform 270 supporting <code>NativeActivity,</code> use the following commands.</p> 271 272 <pre class="devsite-terminal devsite-click-to-copy"> 273 adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e \ 274 cmdLine "deqp --deqp-case=dEQP-GLES2.info.* --deqp-log-filename=/sdcard/dEQP-Log.qpa 275 </pre> 276 277 <h3 id=debugging_on_android>Debugging on Android</h3> 278 279 <p>To run the tests under the GDB debugger on Android, first compile and install 280 the debug build by running the following two scripts:</p> 281 282 <pre class="devsite-click-to-copy"> 283 <code class="devsite-terminal">python android/scripts/build.py --native-build-type=Debug</code> 284 <code class="devsite-terminal">python android/scripts/install.py</code> 285 </pre> 286 287 <p>After the debug build is installed on the device, to launch the tests under 288 GDB running on the host, run the following command:</p> 289 290 <pre class="devsite-terminal devsite-click-to-copy"> 291 python android/scripts/debug.py \ 292 --deqp-commandline="--deqp-log-filename=/sdcard/TestLog.qpa --deqp-case=dEQP-GLES2.functional.*" 293 </pre> 294 295 <p>The deqp command line depends on the test cases to be executed and other 296 required parameters. The script adds a default breakpoint at the beginning of 297 the deqp execution (<code>tcu::App::App</code>).</p> 298 299 <p>The <code>debug.py</code> script accepts multiple command line arguments for 300 actions such as setting breakpoints for debugging, gdbserver connection 301 parameters, and paths to additional binaries to debug (use <code>debug.py 302 --help</code> for all arguments and explanations). The script also copies some 303 default libraries from the target device to get symbol listings.</p> 304 305 <p>To step through driver code (such as when the GDB needs to know the locations 306 of the binaries with full debug information), add more libraries via 307 <code>debug.py</code> command line parameters. This script writes out a 308 configuration file for the GDB starting from line 132 of the script file. You 309 can provide additional paths to binaries, etc., but supplying correct command 310 line parameters should be enough.</p> 311 312 <p class="note"><strong>Note:</strong> On Windows, the GDB binary requires 313 <code>libpython2.7.dll</code>. Before launching <code>debug.py</code>, add 314 <code><path-to-ndk>/prebuilt/windows/bin</code> to the PATH variable.</p> 315 316 <p class="note"><strong>Note:</strong> Native code debugging does not work on 317 stock Android 4.3; for workarounds, refer to <a href="https://issuetracker.google.com/issues/36976703">this public bug</a>. 318 Android 4.4 and higher do not contain this bug.</p> 319 320 </body> 321 </html> 322
