1 <html devsite> 2 <head> 3 <title>Camera ITS-in-a-Box</title> 4 <meta name="project_path" value="/_project.yaml" /> 5 <meta name="book_path" value="/_book.yaml" /> 6 </head> 7 <body> 8 <!-- 9 Copyright 2017 The Android Open Source Project 10 11 Licensed under the Apache License, Version 2.0 (the "License"); 12 you may not use this file except in compliance with the License. 13 You may obtain a copy of the License at 14 15 http://www.apache.org/licenses/LICENSE-2.0 16 17 Unless required by applicable law or agreed to in writing, software 18 distributed under the License is distributed on an "AS IS" BASIS, 19 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 20 See the License for the specific language governing permissions and 21 limitations under the License. 22 --> 23 24 25 26 <p>Android Camera Image Test Suite (ITS) is part of Android Compatibility Test 27 Suite (CTS) Verifier and includes tests that verify image content. As of CTS 28 7.0_r8, CTS Verifier supports ITS test automation via Camera ITS-in-a-box; 29 support for manual tests continues to ensure coverage for all Android device 30 form factors.</p> 31 32 <p>ITS-in-a-box brings the following benefits:</p> 33 <ul> 34 <li><strong>Automation</strong>. No human intervention is needed during the 35 test.</li> 36 <li><strong>Reduced testing time</strong>. Parallel testing of front/back camera 37 cuts 38 testing cycle time by 50%.</li> 39 <li><strong>Easier troubleshooting</strong>. Consistency of test environment 40 leads to fewer setup errors and increases reproducibility.</li> 41 <li><strong>Efficiency</strong>. Ability to retry for individual Camera/Scene 42 improves test execution efficiency.</li> 43 </ul> 44 45 <h2 id=get-started>Getting started</h2> 46 <p>ITS-in-a-box consists of a plastic box that is laser cut from computer-aided 47 design (CAD) drawings, a chart tablet, and a device under test (DUT). To get 48 started with the Camera ITS-in-a-box:</p> 49 <ol> 50 <li><a href="camera_its_iab_tech_drawings.zip">Download the technical 51 drawings</a>.</li> 52 <li><a href="camera-its-box-assembly.html">Build the box</a>.</li> 53 <li><a href="#configure-tablet">Configure a tablet</a> with Camera ITS 54 software.</li> 55 <li><a href="#run-tests">Run tests</a>.</li> 56 <li><a href="#get-results">Get results</a> from the DUT.</li> 57 </ol> 58 59 <h2 id=configure-tablet>Configuring the tablet</h2> 60 <p>This section provides step-by-step instructions for setting up a Pixel C 61 tablet for use with the CameraITS software.</p> 62 63 <p class="note"><strong>Note:</strong> The CameraITS python scripts 64 automatically set the following options on the tablet for you: 65 <br><em>Settings > Display > Sleep > After 30 minutes of inactivity</em> 66 <br><em>Adaptive brightness > OFF</em> 67 </p> 68 69 <ol> 70 <li>Charge the tablet and power it on. If prompted to set up an account, skip it 71 (CameraITS does not require any account paired with the tablet).</li> 72 <li>Update the tablet to Android 7.0 or later (Android 6.x and earlier versions 73 do not support CameraITS).</li> 74 <li>Enable developer mode by going to <em>Settings > About tablet</em> and 75 tapping <strong>Build number</strong> five times.</li> 76 <li>Return to <em>Settings</em> and select <strong>Developer options</strong>. 77 78 <table> 79 <tr> 80 <th>Enable options</th> 81 <td> 82 <ul> 83 <li>On</li> 84 <li>Stay awake</li> 85 <li>USB debugging (This allows the host to run the tablet in debug mode. When you 86 connect the tablet to the host for the first time, the tablet prompts you to 87 "Allow USB debugging?" If the tablet does not display the debug prompt, 88 disconnect then reconnect tablet.)</li> 89 </ul> 90 </td> 91 </tr> 92 <tr> 93 <th>Disable options</th> 94 <td> 95 <ul> 96 <li>Automatic system updates</li> 97 <li>Verify apps over USB</li> 98 </ul> 99 </td> 100 </tr> 101 </table> 102 </li> 103 <li>Determine DUT and chart IDs by running <code>$ adb devices</code> to list 104 available devices. To determine <code>device_id</code> and 105 <code>chart_id</code>, plug and unplug devices and observe the devices that 106 connect and disconnect.</li> 107 <li>Perform three test runs to suppress hints and user prompts that can obscure 108 charts on the tablet screen. 109 <ol style=list-style-lower-alpha> 110 <li>Position the tablet face-up on a table (do not attach the tablet to the back 111 panel of the box)</li> 112 <li>Run the following command: 113 <pre class="devsite-terminal devsite-click-to-copy"> 114 python tools/run_all_tests.py device=$device_id camera=0 chart=$chart_id scenes=2,3 115 </pre> 116 Scenes 2 and 3 require the tablet to display an image, so the tablet prompts you 117 to "Allow Drive to access photos, media, and files on your device?". Clear this 118 prompt (and prevent future prompts) by pressing <strong>Allow</strong>.</li> 119 <li>Run the command again. The tablet prompts you to "Keep a copy of this file?" 120 and suggests Google Drive. Clear this prompt (and prevent future prompts) by 121 pressing the Drive icon then <strong>Cancel</strong> for upload to drive.</li> 122 <li>Finally, run <code>tools/run_all_tests.py</code> and confirm that scenes 123 change automatically as script cycles through different scenes. While most 124 tests will FAIL (as the camera is not pointed at the chart), you can verify the 125 tablet correctly cycles through the scenes without displaying any prompts or 126 other pop-ups on the screen.</li></ol></li></ol> 127 128 <h2 id=run-tests>Running tests</h2> 129 <p>Before running the ITS-in-a-box, ensure your test setup includes the following 130 hardware and software:</p> 131 <ul> 132 <li>One (1) ITS-in-a-box</li> 133 <li>One (1) Pixel C for displaying Scenes, S/N: 5811000011</li> 134 <li>Two (2) DUTs that use the same build fingerprint and have the CTS Verifier 135 7.0_8+ application installed. Example DUTs: 136 137 <ul> 138 <li>One (1) Pixel NOF26W for the back camera(0) testing, S/N: FA6BM0305016. To 139 install the CTS Verifier app, unzip android-cts-verifier.zip then run 140 <pre class="devsite-terminal devsite-click-to-copy"> 141 adb -s FA6BM0305016 install -r android-cts-verifier/CtsVerifier.apk 142 </pre></li> 143 <li>One (1) Pixel NOF26W for the front camera(1) testing, S/N: FA6BM0305439. To 144 install the CTS Verifier app, unzip android-cts-verifier.zip then run 145 <pre class="devsite-terminal devsite-click-to-copy"> 146 adb -s FA6BM0305439 install -r android-cts-verifier/CtsVerifier.apk 147 </pre> 148 </li> 149 </ul> 150 </li></ul> 151 152 <h3 id=scenes-0-4>Running scenes 0-4</h3> 153 <p>To run scenes 0-4 on the front and back camera in parallel:</p> 154 <pre class="devsite-click-to-copy"> 155 <code class="devsite-terminal">cd android-cts-verifier/CameraITS</code> 156 <code class="devsite-terminal">. build/envsetup.sh</code> 157 <code class="devsite-terminal">python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011</code> 158 </pre> 159 160 <p>Examples:</p> 161 <table> 162 <tr> 163 <td ><img src=/compatibility/cts/images/camera_its_cam0.png align="center"></td> 164 <td align="center"><img src=/compatibility/cts/images/camera_its_cam0.png></td> 165 </tr> 166 <tr> 167 <td align="center"><p class=caption><strong>Figure 1</strong>. Camera 0 S/N: 168 FA6BM0305016</p> 169 </td> 170 <td align="center"><p class=caption><strong>Figure 2</strong>. Camera 1 S/N: 171 FA6BM0305439</p> 172 </td> 173 </tr> 174 </table> 175 176 <h3 id=retry-scenes>Retrying scenes</h3> 177 <p>You can retry scenes for both front and back cameras or a single camera: 178 <ul> 179 <li>To retry scenes on front and back cameras in parallel: 180 <pre class="devsite-terminal devsite-click-to-copy"> 181 python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011 scenes=3,4 182 </pre> 183 </li> 184 <li>To retry scenes on a single camera: 185 <pre class="devsite-terminal devsite-click-to-copy"> 186 python tools/run_all_tests.py device=FA6BM0305016 chart=5811000011 camera=1 scenes=3,4 187 </pre> 188 </li> 189 </ul> 190 191 <h3 id=scenes-0-4>Running scene 5</h3> 192 <p>Scene 5 requires special setup with specific lighting (for details, refer to 193 CameraITS.pdf in the CTS Verifier download). You can run Scene 5 separately 194 (outside of the box) to test two devices in parallel.</p> 195 <ul> 196 <li>To run Scene 5 on front and back cameras on two devices in parallel: 197 <pre class="devsite-terminal devsite-click-to-copy"> 198 python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011 scenes=5 199 </pre> 200 <br><img src=/compatibility/cts/images/camera_its_scene5.png width="50%"><br> 201 <strong>Figure 3</strong>. Camera scene 5.</li> 202 <li>To run Scene 5 for front and back cameras on a single device: 203 <pre class="devsite-click-to-copy"> 204 <code class="devsite-terminal">python tools/run_all_tests.py device=FA6BM0305016 camera=0 scenes=5</code> 205 <code class="devsite-terminal">python tools/run_all_tests.py device=FA6BM0305016 camera=1 scenes=5</code> 206 </pre></li> 207 </ul> 208 209 <h2 id=get-results>Getting results</h2> 210 <p>You can view results during testing and save the completed results as a report. 211 </p> 212 <ul> 213 <li><strong>View progress of running tests</strong>. The command 214 <code>run_parallel_tests</code> prints results only after Camera-Scene tests 215 have finished, so to view results during test execution you must use Android 216 Device Monitor or <code>adb logcat</code> to verify progress and/or view 217 screenshots.<br> 218 <br>Example adb command: 219 <pre class="devsite-terminal devsite-click-to-copy"> 220 adb -s FA6BM0305016 logcat -v time 221 </pre> 222 Example screenshots command: 223 <pre class="devsite-click-to-copy"> 224 <code class="devsite-terminal">adb -s FA6BM0305016 shell screencap -p /sdcard/screencap.png</code> 225 <code class="devsite-terminal">adb -s FA6BM0305016 pull /sdcard/screencap.png</code> 226 <code class="devsite-terminal">display ./screencap.png</code> 227 </pre></li> 228 <li><strong>View results</strong>. To save Camera ITS results as a report: 229 <ol> 230 <li>Press <strong>Pass</strong> and save the report: 231 <br><img src=/compatibility/cts/images/camera_its_results.png width="50%"><br> 232 <strong>Figure 4</strong>. Camera ITS report.</li> 233 <li>Pull reports from the device: 234 <pre class="devsite-terminal devsite-click-to-copy"> 235 adb -s FA6BM0305016 pull /sdcard/verifierReports 236 </pre> 237 </li> 238 <li>Unzip the report file and view the test_result.xml. 239 <br><img src=/compatibility/cts/images/camera_its_reports.png><br> 240 <strong>Figure 5</strong>. Camera ITS reports.<br></li> 241 </ol> 242 </li> 243 </ul> 244 </body> 245 </html> 246
