1 # Feedback-driven fuzzing # 2 3 Honggfuzz is capable of performing feedback-guided (code coverage driven) fuzzing. It can utilize the following sources of data: 4 * (Linux) Hardware-based counters (instructions, branches) 5 * (Linux) Intel BTS code coverage (kernel >= 4.2) 6 * (Linux) Intel PT code coverage (kernel >= 4.2) 7 * Sanitzer-coverage instrumentation (`-fsanitize-coverage=bb`) 8 * Compile-time instrumentation (`-finstrument-functions` or `-fsanitize-coverage=trace-pc[-guard],indirect-calls,trace-cmp` or both) 9 10 Developers should provide the initial file corpus which will be gradually improved upon. It can even comprise of a single 1-byte initial file, and honggfuzz will try to generate better inputs starting from there. 11 12 --- 13 # Requirements for software-based coverage-guided fuzzing # 14 * `-fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp` - Clang >= 4.0 15 * `-fsanitize-coverage=bb` - Clang >= 3.7 16 * `-finstrument-functions` - GCC or Clang 17 * [older, slower variant] `-fsanitize-coverage=trace-pc,indirect-calls` - Clang >= 3.9 18 19 _Note_: The _-fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp_ set of flags will be automatically added to clang's command-line switches when using [hfuzz-clang](https://github.com/google/honggfuzz/tree/master/hfuzz_cc) binary. 20 21 ``` 22 $ [honggfuzz_dir]/honggfuzz/hfuzz_cc/hfuzz-clang terminal-test.c -o terminal-test 23 ``` 24 25 --- 26 # Requirements for hardware-based counter-based fuzzing # 27 * GNU/Linux OS 28 * Relatively modern Linux kernel (4.2 should suffice) 29 * CPU which is supported by the [perf subsystem](https://perf.wiki.kernel.org/index.php/Main_Page) for hardware-assisted instruction and branch counting 30 31 --- 32 # Requirements for hardware-based coverage-feedback fuzzing (Intel) # 33 * CPU supporting [BTS (Branch Trace Store)](https://software.intel.com/en-us/forums/topic/277868?language=en) for hardware assisted unique pc and edges (branch pairs) counting. Currently it's available only in some newer Intel CPUs (unfortunately no AMD support for now) 34 * CPU supporting [Intel PT (Processor Tracing)](https://software.intel.com/en-us/blogs/2013/09/18/processor-tracing) for hardware assisted unique edge (branch pairs) counting. Currently it's available only in some newer Intel CPUs (since Broadwell architecture) 35 * GNU/Linux OS with a supported CPU; Intel Core 2 for BTS, Intel Broadwell for Intel PT 36 * Intel's [ibipt library](http://packages.ubuntu.com/yakkety/libipt1) for Intel PT 37 * Linux kernel >= v4.2 for perf AUXTRACE 38 39 --- 40 # Fuzzing strategy # 41 The implemented strategy is trying to identify files which add new code coverage (or increased instruction/branch counters). Then those inputs are added (dynamically stored in memory) corpus, and reused during following fuzzing rounds 42 43 There are 2 phases of feedback-driven the fuzzing: 44 * Honggfuzz goes through each file in the initial corpus directory (-f). It adds files which hit new code coverage to the dynamic input corpus (as well as saving them on the disk, using *COVERAGE_DATA.PID.pid.RND.time.rnd* pattern 45 * Honggfuzz choses randomly files from the dynamic input corpus (in-memory), mutates them, and runs a new fuzzing round (round in persistent mode, exec in non-persistent mode). If the newly created file induces new code path (extends code coverage), it gets added to the dynamic input corpus 46 47 # ASAN Code coverage (-C) # 48 In order to make this mode work, one needs to compile the fuzzed tool (_xmllint_ here) with _-fsanitize=address -fsanitize-coverage=bb_ 49 50 ``` 51 $ [honggfuzz_dir]/honggfuzz -C -f IN.corpus/ -- ./xmllint --format --nonet ___FILE___ 52 ============================== STAT ============================== 53 Iterations: 1419 54 Start time: 2016-03-15 16:43:57 (16 seconds elapsed) 55 Input file/dir: 'IN/' 56 Fuzzed cmd: './xmllint --format --nonet ___FILE___' 57 Fuzzing threads: 3 58 Execs per second: 41 (avg: 88) 59 Crashes: 0 (unique: 0, blacklist: 0, verified: 0) 60 Timeouts: 0 61 Number of dynamic files: 251 62 Coverage (max): 63 - total hit #bb: 8634 (coverage 11%) 64 - total #dso: 1 (instrumented only) 65 - discovered #bb: 1 (new from input seed) 66 - crashes: 0 67 ============================== LOGS ============================== 68 [2016-03-15T16:49:00+0100][I][2094] fuzz_sanCovFeedback():463 SanCov Update: file size (Cur): 2141, newBBs:9, counters (Cur,New): 8569/1,1666/1 69 ``` 70 71 # Compile-time instrumentation with clang/gcc (default mode) # 72 73 Here you can use the following: 74 * gcc/clang `-finstrument-functions` (less-precise) 75 * clang's (>= 4.0) `-fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp` 76 (trace-cmp adds additional comparison map to the instrumentation) 77 78 _Note_: The _-fsanitize-coverage=trace-pc-guard,indirect-calls,trace-cmp_ set of flags will be automatically added to clang's command-line switches when using [hfuzz-clang](https://github.com/google/honggfuzz/tree/master/hfuzz_cc) binary. The [hfuzz-clang](https://github.com/google/honggfuzz/tree/master/hfuzz_cc) binary will also link your code with _libhfuzz.a_ 79 80 Two persistent modes can be used here: 81 82 ### LLVM-style LLVMFuzzerTestOneInput ### 83 84 ```c 85 $ cat test.c 86 #include <inttypes.h> 87 #include <testlib.h> // Our API to test 88 89 extern int LLVMFuzzerTestOneInput(uint8_t **buf, size_t *len); 90 91 int LLVMFuzzerTestOneInput(uint8_t *buf, size_t len) { 92 _FuncFromFuzzedLib_(buf, len); 93 return 0; 94 } 95 ``` 96 97 ``` 98 $ [honggfuzz_dir]/honggfuzz/hfuzz_cc/hfuzz-clang -c fuzzedlib.c -o fuzzedlib.o 99 $ [honggfuzz_dir]/honggfuzz/hfuzz_cc/hfuzz-clang test.c fuzzedlib.o -o test 100 $ [honggfuzz_dir]/honggfuzz -P -f INPUT.corpus -- ./test 101 ``` 102 103 `LLVMFuzzerInitialize(int *argc, char **argv)` is supported as well 104 105 ### Fetching input with HF_ITER() ### 106 107 ```c 108 $ cat test.c 109 #include <inttypes.h> 110 #include <testlib.h> // Our API to test 111 112 // Get input from the fuzzer 113 extern void HF_ITER(uint8_t **buf, size_t *len); 114 115 int main(void) { 116 for (;;) { 117 uint8_t *buf; 118 size_t len; 119 HF_ITER(&buf, &len); 120 _FuncFromFuzzedLib_(buf, len); 121 } 122 return 0; 123 } 124 ``` 125 ``` 126 $ [honggfuzz_dir]/honggfuzz/hfuzz_cc/hfuzz-clang -c fuzzedlib.c -o fuzzedlib.o 127 $ [honggfuzz_dir]/honggfuzz/hfuzz_cc/hfuzz-clang test.c fuzzedlib.o -o test 128 $ [honggfuzz_dir]/honggfuzz -P -f INPUT.corpus -- ./test 129 ``` 130 131 Example: 132 ``` 133 $ [honggfuzz_dir]/honggfuzz -P -f IN.server/ -- ./persistent.server.openssl.1.0.2i.asan 134 ------------------------------[ honggfuzz v0.8 ]------------------------------ 135 Iterations : 3,275,169 [3.28M] 136 Run Time : 2 hrs 17 min 16 sec (since: 2016-09-27 07:30:04) 137 Input Dir : 'IN.server/' 138 Fuzzed Cmd : './persistent.server.openssl.1.0.2i.asan' 139 Fuzzing Threads : 2, CPUs: 8, CPU: 759.0% (94.9%/CPU) 140 Speed (Round) : 86/sec (avg: 397) 141 Crashes : 0 (unique: 0, blacklist: 0, verified: 0) 142 Timeouts : 0 [10 sec.] 143 Corpus size : 393 (max file size: 40,000 bytes) 144 Coverage : 145 *** blocks seen: 3,545, comparison map: 204,542 146 -----------------------------------[ LOGS ]----------------------------------- 147 ``` 148 149 PS. You can also use a non-persistent mode here (without the __-P__ flag), in which case you need to read data either from a file passed at command-line (`___FILE___`), or from the standard input (e.g. with `read(0, buf, sizeof(buf))`. The compile-time instrumentation will still work in such case. 150 151 # Hardware-based coverage # 152 ## Unique branch pair (edges) counting (--linux_perf_bts_edge) ## 153 154 This mode will take into consideration pairs (tuples) of jumps, recording unique from-to jump pairs. The data is taken from the Intel BTS CPU registers. 155 156 ``` 157 $ [honggfuzz_dir]/honggfuzz --linux_perf_bts_edge -f IN.corpus/ -- /usr/bin/xmllint -format ___FILE___ 158 ============================== STAT ============================== 159 Iterations: 1 160 Start time: 2016-02-16 18:37:08 (1 seconds elapsed) 161 Input file/dir: 'IN/' 162 Fuzzed cmd: '/usr/bin/xmllint -format ___FILE___' 163 Fuzzing threads: 2 164 Execs per second: 1 (avg: 1) 165 Crashes: 0 (unique: 0, blacklist: 0, verified: 0) 166 Timeouts: 0 167 Number of dynamic files: 251 168 Coverage (max): 169 - BTS unique edges: 2341 170 ============================== LOGS ============================== 171 [2016-02-16T18:37:09+0100][I][14944] fuzz_perfFeedback():420 New: (Size New,Old): 257,257, Perf (Cur,New): 0/0/0/0/0/0,0/0/0/2341/0/0 172 ``` 173 174 ## Unique branch points counting (--linux_perf_ipt_block) ## 175 176 This mode will utilize Interl's PT (Process Trace) subsystem, which should be way faster than BTS (Branch Trace Store), but will currently produce less precise results. 177 178 ``` 179 $ [honggfuzz_dir]/honggfuzz --linux_perf_ipt_block -f IN.corpus/ -- /usr/bin/xmllint -format ___FILE___ 180 ============================== STAT ============================== 181 Iterations: 0 182 Start time: 2016-02-16 18:38:45 (0 seconds elapsed) 183 Input file/dir: 'IN/' 184 Fuzzed cmd: '/usr/bin/xmllint -format ___FILE___' 185 Fuzzing threads: 2 186 Execs per second: 0 (avg: 0) 187 Crashes: 0 (unique: 0, blacklist: 0, verified: 0) 188 Timeouts: 0 189 Number of dynamic files: 251 190 Coverage (max): 191 - PT unique blocks: 243 192 ============================== LOGS ============================== 193 ``` 194 195 ## Instruction counting (--linux_perf_instr) ## 196 197 This mode tries to maximize the number of instructions taken during each process iteration. The counters will be taken from the Linux perf subsystems. Intel, AMD and even other CPU architectures are supported for this mode. 198 199 ``` 200 $ [honggfuzz_dir]/honggfuzz --linux_perf_instr -f IN.corpus -- /usr/bin/xmllint -format ___FILE___ 201 ============================== STAT ============================== 202 Iterations: 2776 203 Start time: 2016-02-16 18:40:51 (3 seconds elapsed) 204 Input file/dir: 'CURRENT_BEST' 205 Fuzzed cmd: '/usr/bin/xmllint -format ___FILE___' 206 Fuzzing threads: 2 207 Execs per second: 922 (avg: 925) 208 Crashes: 0 (unique: 0, blacklist: 0, verified: 0) 209 Timeouts: 0 210 Number of dynamic files: 251 211 Coverage (max): 212 - cpu instructions: 1369752 213 ============================== LOGS ============================== 214 [2016-02-16T18:40:54+0100][I][17406] fuzz_perfFeedback():420 New: (Size New,Old): 2497,2496, Perf (Cur,New): 1369752/0/0/0/0/0,1371747/0/0/0/0/0 215 [2016-02-16T18:40:54+0100][I][17406] fuzz_perfFeedback():420 New: (Size New,Old): 2497,2497, Perf (Cur,New): 1371747/0/0/0/0/0,1372273/0/0/0/0/0 216 [2016-02-16T18:40:54+0100][I][17406] fuzz_perfFeedback():420 New: (Size New,Old): 2497,2497, Perf (Cur,New): 1372273/0/0/0/0/0,1372390/0/0/0/0/0 217 [2016-02-16T18:40:54+0100][I][17406] fuzz_perfFeedback():420 New: (Size New,Old): 2497,2497, Perf (Cur,New): 1372390/0/0/0/0/0,1372793/0/0/0/0/0 218 ``` 219 220 ## Branch counting (--linux_perf_branch) ## 221 222 As above, it will try to maximize the number of branches taken by CPU on behalf of the fuzzed process (here: djpeg.static) while performing each fuzzing iteration. Intel, AMD and even other CPU architectures are supported for this mode. 223 224 ``` 225 $ [honggfuzz_dir]/honggfuzz --linux_perf_branch -f IN/ -F 2500 -- /usr/bin/xmllint -format ___FILE___ 226 ============================== STAT ============================== 227 Iterations: 0 228 Start time: 2016-02-16 18:39:41 (0 seconds elapsed) 229 Input file/dir: 'IN/' 230 Fuzzed cmd: '/usr/bin/xmllint -format ___FILE___' 231 Fuzzing threads: 2 232 Execs per second: 0 (avg: 0) 233 Crashes: 0 (unique: 0, blacklist: 0, verified: 0) 234 Timeouts: 0 235 Number of dynamic files: 251 236 Coverage (max): 237 - cpu branches: 0 238 ============================== LOGS ============================== 239 [2016-02-16T18:39:41+0100][I][16738] fuzz_perfFeedback():420 New: (Size New,Old): 2500,2500, Perf (Cur,New): 0/0/0/0/0/0,0/258259/0/0/0/0 240 [2016-02-16T18:39:41+0100][I][16738] fuzz_perfFeedback():420 New: (Size New,Old): 2500,2500, Perf (Cur,New): 0/258259/0/0/0/0,0/258260/0/0/0/0 241 [2016-02-16T18:39:41+0100][I][16738] fuzz_perfFeedback():420 New: (Size New,Old): 2500,2500, Perf (Cur,New): 0/258260/0/0/0/0,0/258261/0/0/0/0 242 [2016-02-16T18:39:41+0100][I][16738] fuzz_perfFeedback():420 New: (Size New,Old): 2500,2500, Perf (Cur,New): 0/258261/0/0/0/0,0/258263/0/0/0/0 243 ``` 244
