1 Validing libc Assembler Routines 2 ================================ 3 This document describes how to verify incoming assembler libc routines. 4 5 ## Quick Start 6 * First, benchmark the previous version of the routine. 7 * Update the routine, run the bionic unit tests to verify the routine doesn't 8 have any bugs. See the [Testing](#Testing) section for details about how to 9 verify that the routine is being properly tested. 10 * Rerun the benchmarks using the updated image that uses the code for 11 the new routine. See the [Performance](#Performance) section for details about 12 benchmarking. 13 * Verify that unwind information for new routine looks sane. See the [Unwind Info](#unwind-info) section for details about how to verify this. 14 15 When benchmarking, it's best to verify on the latest Pixel device supported. 16 Make sure that you benchmark both the big and little cores to verify that 17 there is no major difference in performance on each. 18 19 Benchmark 64 bit memcmp: 20 21 /data/benchmarktest64/bionic-benchmarks/bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 22 23 Benchmark 32 bit memcmp: 24 25 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 26 27 Locking to a specific cpu: 28 29 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks --bionic_cpu=2 --bionic_xml=string.xml --benchmark_filter=memcmp 30 31 ## Performance 32 The bionic benchmarks are used to verify the performance of changes to 33 routines. For most routines, there should already be benchmarks available. 34 35 Building 36 -------- 37 The bionic benchmarks are not built by default, they must be built separately 38 and pushed on to the device. The commands below show how to do this. 39 40 mmma -j bionic/benchmarks 41 adb sync data 42 43 Running 44 ------- 45 There are two bionic benchmarks executables: 46 47 /data/benchmarktest64/bionic-benchmarks/bionic-benchmarks 48 49 This is for 64 bit libc routines. 50 51 /data/benchmarktest/bionic-benchmarks/bionic-benchmarks 52 53 This is for 32 bit libc routines. 54 55 Here is an example of how the benchmark should be executed. For this 56 command to work, you need to change directory to one of the above 57 directories. 58 59 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=memcmp 60 61 The last argument is the name of the one function that you want to 62 benchmark. 63 64 Almost all routines are already defined in the **string.xml** file in 65 **bionic/benchmarks/suites**. Look at the examples in that file to see 66 how to add a benchmark for a function that doesn't already exist. 67 68 It can take a long time to run these tests since it attempts to test a 69 large number of sizes and alignments. 70 71 Results 72 ------- 73 Bionic benchmarks is based on the [Google Benchmarks](https://github.com/google/benchmark) 74 library. An example of the output looks like this: 75 76 Run on (8 X 1844 MHz CPU s) 77 CPU Caches: 78 L1 Data 32K (x8) 79 L1 Instruction 32K (x8) 80 L2 Unified 512K (x2) 81 ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead. 82 ------------------------------------------------------------------------------------------- 83 Benchmark Time CPU Iterations 84 ------------------------------------------------------------------------------------------- 85 BM_string_memcmp/1/0/0 6 ns 6 ns 120776418 164.641MB/s 86 BM_string_memcmp/1/1/1 6 ns 6 ns 120856788 164.651MB/s 87 88 The smaller the time, the better the performance. 89 90 Caveats 91 ------- 92 When running the benchmarks, CPU scaling is not normally enabled. This means 93 that if the device does not get up to the maximum cpu frequency, the results 94 can vary wildly. It's possible to lock the cpu to the maximum frequency, but 95 is beyond the scope of this document. However, most of the benchmarks max 96 out the cpu very quickly on Pixel devices, and don't affect the results. 97 98 Another potential issue is that the device can overheat when running the 99 benchmarks. To avoid this, you can run the device in a cool environment, 100 or choose a device that is less likely to overheat. To detect these kind 101 of issues, you can run a subset of the tests again. At the very least, it's 102 always a good idea to rerun the suite a couple of times to verify that 103 there isn't a high variation in the numbers. 104 105 If you want to verify a single benchmark result, you can run a single test 106 using a command like this: 107 108 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=BM_string_memcmp/1/1/0 109 110 Where the argument to the filter argument is the name of the benchmark from 111 the output. Sometimes this filter can still match multiple benchmarks, to 112 guarantee that you only run the single benchmark, you can execute the benchmark 113 like so: 114 115 bionic-benchmarks --bionic_xml=string.xml --benchmark_filter=BM_string_memcmp/1/1/0$ 116 117 NOTE: It is assumed that these commands are executed in adb as the shell user 118 on device. If you are trying to run this using adb directly from a host 119 machine, you might need to escape the special shell characters such as **$**. 120 121 ## Testing 122 123 Run the bionic tests to verify that the new routines are valid. However, 124 you should verify that there is coverage of the new routines. This is 125 especially important if this is the first time a routine is assembler. 126 127 Caveats 128 ------- 129 When verifying an assembler routine that operates on buffer data (such as 130 memcpy/strcpy), it's important to verify these corner cases: 131 132 * Verify the routine does not read past the end of the buffers. Many 133 assembler routines optimize by reading multipe bytes at a time and can 134 read past the end. This kind of bug results in an infrequent and difficult to 135 diagnosis crash. 136 * Verify the routine handles unaligned buffers properly. Usually, a failure 137 can result in an unaligned exception. 138 * Verify the routine handles different sized buffers. 139 140 If there are not sufficient tests for a new routine, there are a set of helper 141 functions that can be used to verify the above corner cases. See the 142 header **bionic/tests/buffer\_tests.h** for these routines and look at 143 **bionic/tests/string\_test.cpp** for examples of how to use it. 144 145 ## Unwind Info 146 It is also important to verify that the unwind information for these 147 routines are properly set up. Here is a quick checklist of what to check: 148 149 * Verify that all labels are of the format .LXXX, where XXX is any valid string 150 for a label. If any other label is used, entries in the symbol table 151 will be generated that include these labels. In that case, you will get 152 an unwind with incorrect function information. 153 * Verify that all places where pop/pushes or instructions that modify the 154 sp in any way have corresponding cfi information. Along with this item, 155 verify that when registers are pushed on the stack that there is cfi 156 information indicating how to get the register. 157 * Verify that only cfi directives are being used. This only matters for 158 arm32, where it's possible to use ARM specific unwind directives. 159 160 This list is not meant to be exhaustive, but a minimal set of items to verify 161 before submitting a new libc assembler routine. There are difficult 162 to verify unwind cases, such as around branches, where unwind information 163 can be drastically different for the target of the branch and for the 164 code after a branch instruction. 165
