1 <html devsite> 2 <head> 3 <title>Using CTS Verifier</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 <p>The Android Compatibility Test Suite Verifier (CTS Verifier) is a supplement to 27 the Compatibility Test Suite (CTS). While CTS checks those APIs and functions 28 that can be automated, CTS Verifier provides tests for those APIs and functions 29 that cannot be tested on a stationary device without manual input, like audio 30 quality, touchscreen, accelerometer, camera, etc.</p> 31 <h2 id=test_preparation>Test preparation</h2> 32 <p>The device must have verified Android API compatibility by successfully passing 33 the Compatibility Test Suite.</p> 34 <h3 id=hardware_requirements>Hardware requirements</h3> 35 <ul> 36 <li> A Linux computer with USB 2.0 compatible port 37 <li> A second Android device with a known compatible Bluetooth, Wi-Fi direct, and 38 NFC Host Card Emulation (HCE) implementation 39 </ul> 40 <h3 id=setup>Setup</h3> 41 <ul> 42 <li>Install the <a href="http://developer.android.com/sdk/index.html">Android 43 SDK</a> on the Linux computer <li>Download the appropriate <a 44 href="downloads.html">CTS Verifier.apk</a> for the 45 version of Android under test. 46 <li>Install CTS Verifier.apk to the <em>Device Under Test</em> (DUT). 47 <pre class="devsite-terminal devsite-click-to-copy">adb install -r -g CtsVerifier.apk</pre> 48 <li>Ensure that the device has its system data and time set correctly. 49 </ul> 50 <h2 id=cts_test_procedure>CTS Verifier test procedure</h2> 51 <ol> 52 <li>After the CTS Verifier.apk has been installed, launch the CTS Verifier 53 application: 54 55 <img src="/compatibility/cts/images/cts-verifier-icon.png" alt="CTS Verifier icon in launcher" id="figure1" /> 56 <p class="img-caption"> 57 <strong>Figure 1.</strong> CTS Verifier icon 58 </p> 59 60 <li>Once opened, the CTS Verifier displays a list of all test sets available for 61 manual verification: 62 63 <img src="/compatibility/cts/images/cts-verifier-menu.png" alt="CTS Verifier menu of tests" id="figure2" /> 64 <p class="img-caption"> 65 <strong>Figure 2.</strong> CTS Verifier menu of tests 66 </p> 67 68 <li>Each test contains a set of common elements (in some tests, Pass/Fail is 69 determined automatically): 70 <ul> 71 <li><em>Info</em>a set of instructions to run the test. This will appear as a popup the first 72 time each test is opened or whenever the <strong>Info</strong> button () is pressed. 73 <li><em>Pass</em>If the DUT meets the test requirements per the instructions from Info, press 74 the <strong>Pass</strong> button (). 75 <li><em>Fail</em>If the DUT does not meet the test requirements per the instructions from Info, 76 press the <strong>Fail</strong> button (). 77 </ul> 78 79 <img src="/compatibility/cts/images/video-verifier.png" alt="Streaming video quality verifier" id="figure3" /> 80 <p class="img-caption"> 81 <strong>Figure 3.</strong> Video quality verifier 82 </p> 83 84 </ol> 85 86 <h2 id=specific_test_requirements>Specific test requirements</h2> 87 <h3 id=usb_accessory>USB Accessory</h3> 88 <p>In order to run the USB Accessory test, you need a Linux computer to run the 89 USB desktop machine (host) program.</p> 90 <ol> 91 <li>Connect the DUT to a computer. 92 <li>Execute the cts-usb-accessory program on the computer found in the CTS Verifier 93 package. 94 <pre class="devsite-terminal devsite-click-to-copy">./cts-usb-accessory</pre> 95 <li>A popup message will appear on the DUT. Select <strong>OK</strong> and go into the USB Accessory Test in the CTS Verifier application. 96 <br> 97 <img src="/compatibility/cts/images/screen-lock-test.png" alt="CTS Verifier screen lock test" id="figure4" /> 98 <p class="img-caption"> 99 <strong>Figure 4.</strong> Screen lock test 100 </p> 101 102 <li>Output similar to below will appear on the computers console. 103 </ol> 104 <pre class="devsite-click-to-copy"> 105 CTS USB Accessory Tester 106 Found possible Android device (413c:2106) - attempting to switch to accessory 107 mode... 108 Failed to read protocol version 109 Found Android device in accessory mode (18d1:2d01)... 110 [RECV] Message from Android device #0 111 [SENT] Message from Android accessory #0 112 [RECV] Message from Android device #1 113 [SENT] Message from Android accessory #1 114 [RECV] Message from Android device #2 115 [SENT] Message from Android accessory #2 116 [RECV] Message from Android device #3 117 [SENT] Message from Android accessory #3 118 [RECV] Message from Android device #4 119 [SENT] Message from Android accessory #4 120 [RECV] Message from Android device #5 121 [SENT] Message from Android accessory #5 122 [RECV] Message from Android device #6 123 [SENT] Message from Android accessory #6 124 [RECV] Message from Android device #7 125 [SENT] Message from Android accessory #7 126 [RECV] Message from Android device #8 127 [SENT] Message from Android accessory #8 128 [RECV] Message from Android device #9 129 [SENT] Message from Android accessory #9 130 [RECV] Message from Android device #10 131 [SENT] Message from Android accessory #10 132 </pre> 133 <h3 id=camera_field_of_view_calibration>Camera field of view calibration</h3> 134 <p>This field of view calibration procedure is designed to be a quick way to 135 determine the device field of view with moderate accuracy. 136 <p><strong>Setup</strong> - Print the <a 137 href="/compatibility/calibration-pattern.pdf">calibration-pattern.pdf</a> 138 target file and mount it on a rigid backing. Print on 11 x 17 or A3. Orient 139 the camera device and the printed target as shown in the diagram below:</p> 140 141 <img src="/compatibility/cts/images/camera-printed-target.png" alt="Camera printed target" id="figure5" /> 142 <p class="img-caption"> 143 <strong>Figure 5.</strong> Camera printed target 144 </p> 145 146 <p><strong>Setting the target width</strong> - Measure the distance between the 147 solid lines on the target pattern in centimeters to account for printing 148 inaccuracies (~38 cm).</p> 149 <ol> 150 <li>Start the calibration application. 151 <li>Press the setup button and select Marker distance to enter the distance. 152 <li>Measure and enter the distance to the target pattern (~100 cm). 153 <li>Press the back button to return to the calibration preview. 154 </ol> 155 <p><strong>Calibration process</strong> - Verify that the device and target are 156 placed as shown in the figure and the correct distances have been entered into 157 the setup dialog.The preview will display the image with a vertical line 158 overlaid onto it. This line should align with the center line of the target 159 pattern. The transparent grid can be used with the other vertical lines to 160 ensure that the optical axis is orthogonal to the target.</p> 161 <ol> 162 <li>Select an image resolution to test from the selector at the bottom left. 163 <li>Tap the screen to take a photo and enter the calibration mode (described 164 below). 165 <li>Hit the back button and repeat for all supported image resolutions. 166 </ol> 167 <p><strong>Calibration test (per resolution)</strong> In the calibration mode, the photo will be displayed with two vertical lines 168 overlaid onto the image.These lines should align with the vertical lines on the target pattern within a 169 few pixels. If they do not, then the reported field of view for that mode is 170 inaccurate (assuming the setup is correct).Adjust the slider at the bottom of the screen until the overlay aligns with the 171 target pattern as closely as possible. The displayed field of view will be a 172 close approximation to the correct value when the overlay and the target 173 pattern image are aligned. The reported field of view should be within +/-1 174 degree of the calibration value.</p> 175 <h2 id=exporting_test_reports>Exporting test reports</h2> 176 <ol> 177 <li> 178 After all tests are completed, tap the <strong>Save (disk)</strong> icon. 179 <br> 180 <img src="/compatibility/cts/images/verifier-save-icon.png" alt="CTS Verifier Save icon" id="figure6" /> 181 <p class="img-caption"> 182 <strong>Figure 6.</strong> CTS Verifier Save icon. <em>Note:</em> In 183 Android 7.0 and later, the preview feature is removed: 184 <img src="/compatibility/cts/images/verifier-preview-icon.png" width="24" height="24"> 185 </p> 186 </li> 187 <li> 188 A path to the saved report will be displayed in pop-up (e.g. 189 <code>/sdcard/verifierReports/ctsVerifierReport-date-time.zip</code>). 190 Record the path. 191 <br> 192 <img src="images/path-saved-report.png" alt="CTS Verifier path to saved report " id="figure7" /> 193 <p class="img-caption"> 194 <strong>Figure 7.</strong> CTS Verifier path to saved report 195 </p> 196 </li> 197 <li> 198 Connect the device via USB to a computer with the SDK installed. 199 </li> 200 <li> 201 From the computers SDK installation, run 202 <code>adb pull CTSVerifierReportPath</code> to download the report from the device. 203 <ul> 204 <li> 205 To download all reports run: 206 <pre class="devsite-terminal devsite-click-to-copy">adb pull /sdcard/verifierReports</pre> 207 For Android 6.0 and earlier, run: 208 <pre class="devsite-terminal devsite-click-to-copy">adb pull /mnt/sdcard/ctsVerifierReports/</pre> 209 </li> 210 <li> 211 The name of the reports are time-stamped based on the DUTs system time. 212 </li> 213 <li> 214 To clear results after they have been selected, select 215 <strong>Menu > Clear</strong>. This will clear the Pass/Fail results. 216 </li> 217 </ul> 218 </li> 219 </ol> 220 221 </body> 222 </html> 223
