1 # Power testing 2 *[go/cros-plt-doc][11]* 3 4 ## Description 5 6 Modern mobile computers are sold with an advertised battery life, ranging from a 7 few to tens of hours. Nonetheless, when these devices are used on a daily basis, 8 many users report that their actual battery life doesnt match up with the 9 advertised numbers. For Chrome OS devices, we wanted to try and report battery 10 life that is as close as possible to what an average user experiences. Thus 11 `power_LoadTest` was created to emulate average user behavior and measure the 12 resultant battery life. This test is as an [open source][1] Chrome extension 13 anyone can install and run. 14 15 The `power_LoadTest` runs as a series of one hour long iterations until battery 16 exhaustion. Within each iteration, a load mix known as **60/20/10/10** is run: 17 * **[First 60%]** Browsing: a new website is loaded every minute. The web page 18 loaded is scrolled down one page (600px) every 10 seconds, then scrolled back 19 up at the same rate. 20 * **[Next 20%]** Email: Gmail is loaded in the foreground tab and audio is 21 streamed from a background tab. 22 * **[Next 10%]** Documents: Various Google Docs are loaded. 23 * **[Final 10%]** Video: A full screen 720p YouTube Video is played. 24 25 The parameters of the device under test (DUT) are as follows: 26 * Backlight 27 * Screen: Set to platform default brightness on battery or 40% if default can 28 not be determined via 29 [backlight_tool --get_initial_brightness --lux=150][2]. 30 * Keyboard: Depends on existance of ALS / Hover sensor 31 32 | ALS | hover | keyboard backlight level | 33 |-----|-------|----------------------------------------| 34 | No | No | default | 35 | Yes | No | 40% of default | 36 | No | Yes | System with this config does not exist | 37 | Yes | Yes | 30% of default | 38 39 * Power management: 40 * Dimming, blanking the screen and transitions to standby state are disabled. 41 * Ambient light sensor readings are ignored. 42 * Battery: 43 * Device only powered by battery (no AC power). 44 * Battery charged to 100% prior to initializing test. Test continues in 1 hour 45 iterations until battery passes low threshold (typically set at 3%). Initial 46 & remaining battery charge is recorded. 47 * USB: No external devices connected 48 * Network: Device is associated with a wireless access point via WiFi. 49 * Audio: Built-in speakers at 10% volume, Built-in microphone at 10% volume. 50 51 Throughout the duration of the test, there are 5 sites loaded in background 52 tabs. These sites were chosen to represent typical actions of a user on a daily 53 basis: 54 * Searching 55 * Reading news 56 * Checking on finance 57 * Shopping 58 * Communication 59 60 ## Running 61 62 ### Via cros_sdk & autotest 63 64 If you are interested in running `power_LoadTest` on a Chrome OS system, you 65 will need a Chromium OS test image that can be built by following [Build your 66 own Chromium image][3] instruction with `./build_image --board=${BOARD} test` 67 command. After the test image is built, you can follow the [Installing Chromium 68 OS on your device][4] instruction to install the test image to your DUT. 69 70 Since running `power_LoadTest` requires that the device is disconnected from the 71 wired Ethernet (including USB-Ethernet) as well as from the AC power source, it 72 is trickier to run it compared to [running other autotests][5] if you do not 73 have both the build machine and the DUT under a same private WiFi network 74 connected to the Internet. 75 76 If your build machine and the DUT are in the same WiFi network, you can run 77 `power_LoadTest` by running the following command in scripts directory inside 78 [chroot][6]. (Make sure you have battery fully charged with AC power source and 79 Ethernet disconnected from the DUT before running the test.) 80 ```bash 81 test_that ${DUT_ipaddr} power_LoadTest 82 ``` 83 84 If your build machine is not on the same private WiFi network as the DUT 85 (applicable for most Googlers and any who have the build machine connected to 86 a corporate network), follow the instructions below. 87 88 * Keep the AC power source plugged and the wired Ethernet connected to the same 89 network as your build machine. 90 * Run the same command as above. It is expected to fail as the pure purpose is 91 to copy the test suite over to the DUT from the build machine. 92 ```bash 93 test_that ${DUT_ipaddr} power_LoadTest 94 ``` 95 * Disconnect the wired Ethernet. 96 * On the DUT, enter VT2 by pressing `Ctrl + Alt + F2` (right arrow or refresh 97 key on the Chrome keyboard), and login as `root` (May require password. 98 Default is likely `test0000`. See additional details [here][7] if that doesn't 99 work. 100 ```bash 101 cd /usr/local/autotest* 102 # remove your AC power source right before running the test. 103 bin/autotest tests/power_LoadTest/control* 104 ``` 105 * Re-enter VT1 by pressing `Ctrl + Alt + F1` (left arrow on the Chrome keyboard). 106 * Now you just need to wait until the battery runs out. (Don't worry, the test 107 result is stored on the DUT.) 108 * Reconnect the AC power source, boot up, and enter VT2 as `root`. 109 ```bash 110 cd /usr/local/autotest/results/default/power_LoadTest/results/ 111 ``` 112 113 ### Via extension only 114 115 As mentioned earlier, `power_LoadTest` uses [chrome extension][8] to drive the 116 various workloads. As such it can be run directly on a 'normal mode' machine 117 once the extension is installed. 118 119 To run with this method, 120 * Download the `power_LoadTest` [extension][9] to your device and unpack 121 tarball. 122 * Navigate to `chrome://extensions` and click *load unpacked extension*. 123 * Choose the *extension* directory from tarball. 124 125 You should now have the extension installed and clicking on it will start the 126 test but before you do that read the following caveats & hints to make this run 127 go smoother. 128 129 * Before starting, note the battery state-of-charge (SOC). The extension will 130 only run workload for 1 hour so you'll need to extrapolate total runtime from 131 that. 132 * If device has keyboard backlight be sure its off. See [keyboard backlight][10] 133 documentation for details on manually controlling keyboard backlight. 134 * In order to remove impact of ambient light changes, use brightness keys to set 135 the panel brightness to your preferred brightness first. Note, autotest 136 typically sets brightness to ~80nits. 137 * When test completes be sure to note battery SOC again for calculating battery 138 life. 139 140 Estimate your total battery life by this calculation, 141 142 100 / (battery_soc_start - battery_soc_end) 143 144 For example if you started test at 50% SOC and it ended with 40% SOC your 145 battery life would be, 146 147 100 / (50 - 40) = 10 hours 148 149 ## Interpreting Results 150 151 If you ran via cros_sdk & autotest there will be a keyvals file at 152 `power_LoadTest/results/keyval`. The test will publish *minutes_battery_life* 153 which we use to track platforms battery life. However that only tells part of 154 the story. As with any other real world test, the results have other 155 measurements that should be examined to ensure the battery life estimate is 156 genuine. 157 158 Keyvals of particular interest beyond *minutes_battery_life* to judge quality of 159 test results are: 160 161 * ext_*\_failed_loads 162 * Any non-zero values for these keyvals can be problematic. The keyval itself 163 is a string with counts separated by underscores. Each count is the failures 164 logged by the particular loop. For example 0_1_2 would mean no failures for 165 loop0, 1 for loop1 and 2 for loop2. While failures arent good if the 166 subsequent ext_*\_successful_loads equals the correct quantity then data may 167 be deemed ok. 168 * percent_cpuidle_* 169 * These numbers typically stay roughly the same for a particular platform so 170 be cogniscent that they roughly match previously qualified PLT runs or the 171 differences should be investigated. 172 * percent_cpufreq_* 173 * Similar to cpuidle above this represents the P-state the processor frequency 174 runs in. These too stay roughly the same and should be compared to previously 175 qualified numbers. 176 * loop*\_system_pwr_avg 177 * Each loop should be roughly the same average power. Often the first loop 178 (loop0) consumes more as webpage caching hasnt occurred yet. 179 * percent_usb_suspended_* 180 * Should be ~100% unless run is being done with expected USB device that has 181 been attached externally or has had selective suspend disabled for some 182 functional reason. 183 184 ## Conclusion 185 186 While the initial version of `power_LoadTest` seems to emulate well what users 187 experience every day on Chrome OS devices, this test will be constantly 188 improved. As we learn more about how users use Chrome OS devices and how 189 experienced battery life differs from tested battery life, we will use this 190 data to refine the test, potentially changing the load mix or the parameters 191 of the test. Our goal is to ensure that when you purchase a device, you know - 192 with reasonable certainty - how long that device will last in your daily use. 193 194 [1]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/power_LoadTest/ 195 [2]: https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/tools/backlight_tool.cc#154 196 [3]: http://www.chromium.org/chromium-os/developer-guide#TOC-Build-a-disk-image-for-your-board 197 [4]: http://www.chromium.org/chromium-os/developer-guide#TOC-Installing-Chromium-OS-on-your-Device 198 [5]: http://www.chromium.org/chromium-os/testing/autotest-user-doc#TOC-Running-tests 199 [6]: https://chromium.googlesource.com/chromiumos/docs/+/master/developer_guide.md#enter-the-chroot 200 [7]: https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/debugging-features 201 [8]: https://developer.chrome.com/extensions/getstarted 202 [9]: https://chromium.googlesource.com/chromiumos/third_party/autotest/+archive/master/client/site_tests/power_LoadTest.tar.gz 203 [10]: https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/docs/keyboard_backlight.md 204 [11]: http://go/cros-plt-doc 205
