1 ======================================================= 2 libFuzzer a library for coverage-guided fuzz testing. 3 ======================================================= 4 .. contents:: 5 :local: 6 :depth: 1 7 8 Introduction 9 ============ 10 11 LibFuzzer is a library for in-process, coverage-guided, evolutionary fuzzing 12 of other libraries. 13 14 LibFuzzer is similar in concept to American Fuzzy Lop (AFL_), but it performs 15 all of its fuzzing inside a single process. This in-process fuzzing can be more 16 restrictive and fragile, but is potentially much faster as there is no overhead 17 for process start-up. 18 19 The fuzzer is linked with the library under test, and feeds fuzzed inputs to the 20 library via a specific fuzzing entrypoint (aka "target function"); the fuzzer 21 then tracks which areas of the code are reached, and generates mutations on the 22 corpus of input data in order to maximize the code coverage. The code coverage 23 information for libFuzzer is provided by LLVM's SanitizerCoverage_ 24 instrumentation. 25 26 Contact: libfuzzer(#)googlegroups.com 27 28 Versions 29 ======== 30 31 LibFuzzer is under active development so a current (or at least very recent) 32 version of Clang is the only supported variant. 33 34 (If `building Clang from trunk`_ is too time-consuming or difficult, then 35 the Clang binaries that the Chromium developers build are likely to be 36 fairly recent: 37 38 .. code-block:: console 39 40 mkdir TMP_CLANG 41 cd TMP_CLANG 42 git clone https://chromium.googlesource.com/chromium/src/tools/clang 43 cd .. 44 TMP_CLANG/clang/scripts/update.py 45 46 This installs the Clang binary as 47 ``./third_party/llvm-build/Release+Asserts/bin/clang``) 48 49 The libFuzzer code resides in the LLVM repository, and requires a recent Clang 50 compiler to build (and is used to `fuzz various parts of LLVM itself`_). 51 However the fuzzer itself does not (and should not) depend on any part of LLVM 52 infrastructure and can be used for other projects without requiring the rest 53 of LLVM. 54 55 56 57 Getting Started 58 =============== 59 60 .. contents:: 61 :local: 62 :depth: 1 63 64 Building 65 -------- 66 67 The first step for using libFuzzer on a library is to implement a fuzzing 68 target function that accepts a sequence of bytes, like this: 69 70 .. code-block:: c++ 71 72 // fuzz_target.cc 73 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { 74 DoSomethingInterestingWithMyAPI(Data, Size); 75 return 0; // Non-zero return values are reserved for future use. 76 } 77 78 Next, build the libFuzzer library as a static archive, without any sanitizer 79 options. Note that the libFuzzer library contains the ``main()`` function: 80 81 .. code-block:: console 82 83 svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer 84 # Alternative: get libFuzzer from a dedicated git mirror: 85 # git clone https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer 86 clang++ -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer 87 ar ruv libFuzzer.a Fuzzer*.o 88 89 Then build the fuzzing target function and the library under test using 90 the SanitizerCoverage_ option, which instruments the code so that the fuzzer 91 can retrieve code coverage information (to guide the fuzzing). Linking with 92 the libFuzzer code then gives an fuzzer executable. 93 94 You should also enable one or more of the *sanitizers*, which help to expose 95 latent bugs by making incorrect behavior generate errors at runtime: 96 97 - AddressSanitizer_ (ASAN) detects memory access errors. Use `-fsanitize=address`. 98 - UndefinedBehaviorSanitizer_ (UBSAN) detects the use of various features of C/C++ that are explicitly 99 listed as resulting in undefined behavior. Use `-fsanitize=undefined -fno-sanitize-recover=undefined` 100 or any individual UBSAN check, e.g. `-fsanitize=signed-integer-overflow -fno-sanitize-recover=undefined`. 101 You may combine ASAN and UBSAN in one build. 102 - MemorySanitizer_ (MSAN) detects uninitialized reads: code whose behavior relies on memory 103 contents that have not been initialized to a specific value. Use `-fsanitize=memory`. 104 MSAN can not be combined with other sanirizers and should be used as a seprate build. 105 106 Finally, link with ``libFuzzer.a``:: 107 108 clang -fsanitize-coverage=edge -fsanitize=address your_lib.cc fuzz_target.cc libFuzzer.a -o my_fuzzer 109 110 Corpus 111 ------ 112 113 Coverage-guided fuzzers like libFuzzer rely on a corpus of sample inputs for the 114 code under test. This corpus should ideally be seeded with a varied collection 115 of valid and invalid inputs for the code under test; for example, for a graphics 116 library the initial corpus might hold a variety of different small PNG/JPG/GIF 117 files. The fuzzer generates random mutations based around the sample inputs in 118 the current corpus. If a mutation triggers execution of a previously-uncovered 119 path in the code under test, then that mutation is saved to the corpus for 120 future variations. 121 122 LibFuzzer will work without any initial seeds, but will be less 123 efficient if the library under test accepts complex, 124 structured inputs. 125 126 The corpus can also act as a sanity/regression check, to confirm that the 127 fuzzing entrypoint still works and that all of the sample inputs run through 128 the code under test without problems. 129 130 If you have a large corpus (either generated by fuzzing or acquired by other means) 131 you may want to minimize it while still preserving the full coverage. One way to do that 132 is to use the `-merge=1` flag: 133 134 .. code-block:: console 135 136 mkdir NEW_CORPUS_DIR # Store minimized corpus here. 137 ./my_fuzzer -merge=1 NEW_CORPUS_DIR FULL_CORPUS_DIR 138 139 You may use the same flag to add more interesting items to an existing corpus. 140 Only the inputs that trigger new coverage will be added to the first corpus. 141 142 .. code-block:: console 143 144 ./my_fuzzer -merge=1 CURRENT_CORPUS_DIR NEW_POTENTIALLY_INTERESTING_INPUTS_DIR 145 146 147 Running 148 ------- 149 150 To run the fuzzer, first create a Corpus_ directory that holds the 151 initial "seed" sample inputs: 152 153 .. code-block:: console 154 155 mkdir CORPUS_DIR 156 cp /some/input/samples/* CORPUS_DIR 157 158 Then run the fuzzer on the corpus directory: 159 160 .. code-block:: console 161 162 ./my_fuzzer CORPUS_DIR # -max_len=1000 -jobs=20 ... 163 164 As the fuzzer discovers new interesting test cases (i.e. test cases that 165 trigger coverage of new paths through the code under test), those test cases 166 will be added to the corpus directory. 167 168 By default, the fuzzing process will continue indefinitely at least until 169 a bug is found. Any crashes or sanitizer failures will be reported as usual, 170 stopping the fuzzing process, and the particular input that triggered the bug 171 will be written to disk (typically as ``crash-<sha1>``, ``leak-<sha1>``, 172 or ``timeout-<sha1>``). 173 174 175 Parallel Fuzzing 176 ---------------- 177 178 Each libFuzzer process is single-threaded, unless the library under test starts 179 its own threads. However, it is possible to run multiple libFuzzer processes in 180 parallel with a shared corpus directory; this has the advantage that any new 181 inputs found by one fuzzer process will be available to the other fuzzer 182 processes (unless you disable this with the ``-reload=0`` option). 183 184 This is primarily controlled by the ``-jobs=N`` option, which indicates that 185 that `N` fuzzing jobs should be run to completion (i.e. until a bug is found or 186 time/iteration limits are reached). These jobs will be run across a set of 187 worker processes, by default using half of the available CPU cores; the count of 188 worker processes can be overridden by the ``-workers=N`` option. For example, 189 running with ``-jobs=30`` on a 12-core machine would run 6 workers by default, 190 with each worker averaging 5 bugs by completion of the entire process. 191 192 193 Options 194 ======= 195 196 To run the fuzzer, pass zero or more corpus directories as command line 197 arguments. The fuzzer will read test inputs from each of these corpus 198 directories, and any new test inputs that are generated will be written 199 back to the first corpus directory: 200 201 .. code-block:: console 202 203 ./fuzzer [-flag1=val1 [-flag2=val2 ...] ] [dir1 [dir2 ...] ] 204 205 If a list of files (rather than directories) are passed to the fuzzer program, 206 then it will re-run those files as test inputs but will not perform any fuzzing. 207 In this mode the fuzzer binary can be used as a regression test (e.g. on a 208 continuous integration system) to check the target function and saved inputs 209 still work. 210 211 The most important command line options are: 212 213 ``-help`` 214 Print help message. 215 ``-seed`` 216 Random seed. If 0 (the default), the seed is generated. 217 ``-runs`` 218 Number of individual test runs, -1 (the default) to run indefinitely. 219 ``-max_len`` 220 Maximum length of a test input. If 0 (the default), libFuzzer tries to guess 221 a good value based on the corpus (and reports it). 222 ``-timeout`` 223 Timeout in seconds, default 1200. If an input takes longer than this timeout, 224 the process is treated as a failure case. 225 ``-rss_limit_mb`` 226 Memory usage limit in Mb, default 2048. Use 0 to disable the limit. 227 If an input requires more than this amount of RSS memory to execute, 228 the process is treated as a failure case. 229 The limit is checked in a separate thread every second. 230 If running w/o ASAN/MSAN, you may use 'ulimit -v' instead. 231 ``-timeout_exitcode`` 232 Exit code (default 77) to emit when terminating due to timeout, when 233 ``-abort_on_timeout`` is not set. 234 ``-max_total_time`` 235 If positive, indicates the maximum total time in seconds to run the fuzzer. 236 If 0 (the default), run indefinitely. 237 ``-merge`` 238 If set to 1, any corpus inputs from the 2nd, 3rd etc. corpus directories 239 that trigger new code coverage will be merged into the first corpus 240 directory. Defaults to 0. This flag can be used to minimize a corpus. 241 ``-reload`` 242 If set to 1 (the default), the corpus directory is re-read periodically to 243 check for new inputs; this allows detection of new inputs that were discovered 244 by other fuzzing processes. 245 ``-jobs`` 246 Number of fuzzing jobs to run to completion. Default value is 0, which runs a 247 single fuzzing process until completion. If the value is >= 1, then this 248 number of jobs performing fuzzing are run, in a collection of parallel 249 separate worker processes; each such worker process has its 250 ``stdout``/``stderr`` redirected to ``fuzz-<JOB>.log``. 251 ``-workers`` 252 Number of simultaneous worker processes to run the fuzzing jobs to completion 253 in. If 0 (the default), ``min(jobs, NumberOfCpuCores()/2)`` is used. 254 ``-dict`` 255 Provide a dictionary of input keywords; see Dictionaries_. 256 ``-use_counters`` 257 Use `coverage counters`_ to generate approximate counts of how often code 258 blocks are hit; defaults to 1. 259 ``-use_traces`` 260 Use instruction traces (experimental, defaults to 0); see `Data-flow-guided fuzzing`_. 261 ``-only_ascii`` 262 If 1, generate only ASCII (``isprint``+``isspace``) inputs. Defaults to 0. 263 ``-artifact_prefix`` 264 Provide a prefix to use when saving fuzzing artifacts (crash, timeout, or 265 slow inputs) as ``$(artifact_prefix)file``. Defaults to empty. 266 ``-exact_artifact_path`` 267 Ignored if empty (the default). If non-empty, write the single artifact on 268 failure (crash, timeout) as ``$(exact_artifact_path)``. This overrides 269 ``-artifact_prefix`` and will not use checksum in the file name. Do not use 270 the same path for several parallel processes. 271 ``-print_final_stats`` 272 If 1, print statistics at exit. Defaults to 0. 273 ``-detect-leaks`` 274 If 1 (default) and if LeakSanitizer is enabled 275 try to detect memory leaks during fuzzing (i.e. not only at shut down). 276 ``-close_fd_mask`` 277 Indicate output streams to close at startup. Be careful, this will 278 remove diagnostic output from target code (e.g. messages on assert failure). 279 280 - 0 (default): close neither ``stdout`` nor ``stderr`` 281 - 1 : close ``stdout`` 282 - 2 : close ``stderr`` 283 - 3 : close both ``stdout`` and ``stderr``. 284 285 For the full list of flags run the fuzzer binary with ``-help=1``. 286 287 Output 288 ====== 289 290 During operation the fuzzer prints information to ``stderr``, for example:: 291 292 INFO: Seed: 3338750330 293 Loaded 1024/1211 files from corpus/ 294 INFO: -max_len is not provided, using 64 295 #0 READ units: 1211 exec/s: 0 296 #1211 INITED cov: 2575 bits: 8855 indir: 5 units: 830 exec/s: 1211 297 #1422 NEW cov: 2580 bits: 8860 indir: 5 units: 831 exec/s: 1422 L: 21 MS: 1 ShuffleBytes- 298 #1688 NEW cov: 2581 bits: 8865 indir: 5 units: 832 exec/s: 1688 L: 19 MS: 2 EraseByte-CrossOver- 299 #1734 NEW cov: 2583 bits: 8879 indir: 5 units: 833 exec/s: 1734 L: 27 MS: 3 ChangeBit-EraseByte-ShuffleBytes- 300 ... 301 302 The early parts of the output include information about the fuzzer options and 303 configuration, including the current random seed (in the ``Seed:`` line; this 304 can be overridden with the ``-seed=N`` flag). 305 306 Further output lines have the form of an event code and statistics. The 307 possible event codes are: 308 309 ``READ`` 310 The fuzzer has read in all of the provided input samples from the corpus 311 directories. 312 ``INITED`` 313 The fuzzer has completed initialization, which includes running each of 314 the initial input samples through the code under test. 315 ``NEW`` 316 The fuzzer has created a test input that covers new areas of the code 317 under test. This input will be saved to the primary corpus directory. 318 ``pulse`` 319 The fuzzer has generated 2\ :sup:`n` inputs (generated periodically to reassure 320 the user that the fuzzer is still working). 321 ``DONE`` 322 The fuzzer has completed operation because it has reached the specified 323 iteration limit (``-runs``) or time limit (``-max_total_time``). 324 ``MIN<n>`` 325 The fuzzer is minimizing the combination of input corpus directories into 326 a single unified corpus (due to the ``-merge`` command line option). 327 ``RELOAD`` 328 The fuzzer is performing a periodic reload of inputs from the corpus 329 directory; this allows it to discover any inputs discovered by other 330 fuzzer processes (see `Parallel Fuzzing`_). 331 332 Each output line also reports the following statistics (when non-zero): 333 334 ``cov:`` 335 Total number of code blocks or edges covered by the executing the current 336 corpus. 337 ``bits:`` 338 Rough measure of the number of code blocks or edges covered, and how often; 339 only valid if the fuzzer is run with ``-use_counters=1``. 340 ``indir:`` 341 Number of distinct function `caller-callee pairs`_ executed with the 342 current corpus; only valid if the code under test was built with 343 ``-fsanitize-coverage=indirect-calls``. 344 ``units:`` 345 Number of entries in the current input corpus. 346 ``exec/s:`` 347 Number of fuzzer iterations per second. 348 349 For ``NEW`` events, the output line also includes information about the mutation 350 operation that produced the new input: 351 352 ``L:`` 353 Size of the new input in bytes. 354 ``MS: <n> <operations>`` 355 Count and list of the mutation operations used to generate the input. 356 357 358 Examples 359 ======== 360 .. contents:: 361 :local: 362 :depth: 1 363 364 Toy example 365 ----------- 366 367 A simple function that does something interesting if it receives the input 368 "HI!":: 369 370 cat << EOF > test_fuzzer.cc 371 #include <stdint.h> 372 #include <stddef.h> 373 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { 374 if (size > 0 && data[0] == 'H') 375 if (size > 1 && data[1] == 'I') 376 if (size > 2 && data[2] == '!') 377 __builtin_trap(); 378 return 0; 379 } 380 EOF 381 # Build test_fuzzer.cc with asan and link against libFuzzer.a 382 clang++ -fsanitize=address -fsanitize-coverage=edge test_fuzzer.cc libFuzzer.a 383 # Run the fuzzer with no corpus. 384 ./a.out 385 386 You should get an error pretty quickly:: 387 388 #0 READ units: 1 exec/s: 0 389 #1 INITED cov: 3 units: 1 exec/s: 0 390 #2 NEW cov: 5 units: 2 exec/s: 0 L: 64 MS: 0 391 #19237 NEW cov: 9 units: 3 exec/s: 0 L: 64 MS: 0 392 #20595 NEW cov: 10 units: 4 exec/s: 0 L: 1 MS: 4 ChangeASCIIInt-ShuffleBytes-ChangeByte-CrossOver- 393 #34574 NEW cov: 13 units: 5 exec/s: 0 L: 2 MS: 3 ShuffleBytes-CrossOver-ChangeBit- 394 #34807 NEW cov: 15 units: 6 exec/s: 0 L: 3 MS: 1 CrossOver- 395 ==31511== ERROR: libFuzzer: deadly signal 396 ... 397 artifact_prefix='./'; Test unit written to ./crash-b13e8756b13a00cf168300179061fb4b91fefbed 398 399 400 PCRE2 401 ----- 402 403 Here we show how to use libFuzzer on something real, yet simple: pcre2_:: 404 405 COV_FLAGS=" -fsanitize-coverage=edge,indirect-calls,8bit-counters" 406 # Get PCRE2 407 wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre2-10.20.tar.gz 408 tar xf pcre2-10.20.tar.gz 409 # Build PCRE2 with AddressSanitizer and coverage; requires autotools. 410 (cd pcre2-10.20; ./autogen.sh; CC="clang -fsanitize=address $COV_FLAGS" ./configure --prefix=`pwd`/../inst && make -j && make install) 411 # Build the fuzzing target function that does something interesting with PCRE2. 412 cat << EOF > pcre_fuzzer.cc 413 #include <string.h> 414 #include <stdint.h> 415 #include "pcre2posix.h" 416 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { 417 if (size < 1) return 0; 418 char *str = new char[size+1]; 419 memcpy(str, data, size); 420 str[size] = 0; 421 regex_t preg; 422 if (0 == regcomp(&preg, str, 0)) { 423 regexec(&preg, str, 0, 0, 0); 424 regfree(&preg); 425 } 426 delete [] str; 427 return 0; 428 } 429 EOF 430 clang++ -g -fsanitize=address $COV_FLAGS -c -std=c++11 -I inst/include/ pcre_fuzzer.cc 431 # Link. 432 clang++ -g -fsanitize=address -Wl,--whole-archive inst/lib/*.a -Wl,-no-whole-archive libFuzzer.a pcre_fuzzer.o -o pcre_fuzzer 433 434 This will give you a binary of the fuzzer, called ``pcre_fuzzer``. 435 Now, create a directory that will hold the test corpus: 436 437 .. code-block:: console 438 439 mkdir -p CORPUS 440 441 For simple input languages like regular expressions this is all you need. 442 For more complicated/structured inputs, the fuzzer works much more efficiently 443 if you can populate the corpus directory with a variety of valid and invalid 444 inputs for the code under test. 445 Now run the fuzzer with the corpus directory as the only parameter: 446 447 .. code-block:: console 448 449 ./pcre_fuzzer ./CORPUS 450 451 Initially, you will see Output_ like this:: 452 453 INFO: Seed: 2938818941 454 INFO: -max_len is not provided, using 64 455 INFO: A corpus is not provided, starting from an empty corpus 456 #0 READ units: 1 exec/s: 0 457 #1 INITED cov: 3 bits: 3 units: 1 exec/s: 0 458 #2 NEW cov: 176 bits: 176 indir: 3 units: 2 exec/s: 0 L: 64 MS: 0 459 #8 NEW cov: 176 bits: 179 indir: 3 units: 3 exec/s: 0 L: 63 MS: 2 ChangeByte-EraseByte- 460 ... 461 #14004 NEW cov: 1500 bits: 4536 indir: 5 units: 406 exec/s: 0 L: 54 MS: 3 ChangeBit-ChangeBit-CrossOver- 462 463 Now, interrupt the fuzzer and run it again the same way. You will see:: 464 465 INFO: Seed: 3398349082 466 INFO: -max_len is not provided, using 64 467 #0 READ units: 405 exec/s: 0 468 #405 INITED cov: 1499 bits: 4535 indir: 5 units: 286 exec/s: 0 469 #587 NEW cov: 1499 bits: 4540 indir: 5 units: 287 exec/s: 0 L: 52 MS: 2 InsertByte-EraseByte- 470 #667 NEW cov: 1501 bits: 4542 indir: 5 units: 288 exec/s: 0 L: 39 MS: 2 ChangeBit-InsertByte- 471 #672 NEW cov: 1501 bits: 4543 indir: 5 units: 289 exec/s: 0 L: 15 MS: 2 ChangeASCIIInt-ChangeBit- 472 #739 NEW cov: 1501 bits: 4544 indir: 5 units: 290 exec/s: 0 L: 64 MS: 4 ShuffleBytes-ChangeASCIIInt-InsertByte-ChangeBit- 473 ... 474 475 On the second execution the fuzzer has a non-empty input corpus (405 items). As 476 the first step, the fuzzer minimized this corpus (the ``INITED`` line) to 477 produce 286 interesting items, omitting inputs that do not hit any additional 478 code. 479 480 (Aside: although the fuzzer only saves new inputs that hit additional code, this 481 does not mean that the corpus as a whole is kept minimized. For example, if 482 an input hitting A-B-C then an input that hits A-B-C-D are generated, 483 they will both be saved, even though the latter subsumes the former.) 484 485 486 You may run ``N`` independent fuzzer jobs in parallel on ``M`` CPUs: 487 488 .. code-block:: console 489 490 N=100; M=4; ./pcre_fuzzer ./CORPUS -jobs=$N -workers=$M 491 492 By default (``-reload=1``) the fuzzer processes will periodically scan the corpus directory 493 and reload any new tests. This way the test inputs found by one process will be picked up 494 by all others. 495 496 If ``-workers=$M`` is not supplied, ``min($N,NumberOfCpuCore/2)`` will be used. 497 498 Heartbleed 499 ---------- 500 Remember Heartbleed_? 501 As it was recently `shown <https://blog.hboeck.de/archives/868-How-Heartbleed-couldve-been-found.html>`_, 502 fuzzing with AddressSanitizer_ can find Heartbleed. Indeed, here are the step-by-step instructions 503 to find Heartbleed with libFuzzer:: 504 505 wget https://www.openssl.org/source/openssl-1.0.1f.tar.gz 506 tar xf openssl-1.0.1f.tar.gz 507 COV_FLAGS="-fsanitize-coverage=edge,indirect-calls" # -fsanitize-coverage=8bit-counters 508 (cd openssl-1.0.1f/ && ./config && 509 make -j 32 CC="clang -g -fsanitize=address $COV_FLAGS") 510 # Get and build libFuzzer 511 svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer 512 clang -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer 513 # Get examples of key/pem files. 514 git clone https://github.com/hannob/selftls 515 cp selftls/server* . -v 516 cat << EOF > handshake-fuzz.cc 517 #include <openssl/ssl.h> 518 #include <openssl/err.h> 519 #include <assert.h> 520 #include <stdint.h> 521 #include <stddef.h> 522 523 SSL_CTX *sctx; 524 int Init() { 525 SSL_library_init(); 526 SSL_load_error_strings(); 527 ERR_load_BIO_strings(); 528 OpenSSL_add_all_algorithms(); 529 assert (sctx = SSL_CTX_new(TLSv1_method())); 530 assert (SSL_CTX_use_certificate_file(sctx, "server.pem", SSL_FILETYPE_PEM)); 531 assert (SSL_CTX_use_PrivateKey_file(sctx, "server.key", SSL_FILETYPE_PEM)); 532 return 0; 533 } 534 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { 535 static int unused = Init(); 536 SSL *server = SSL_new(sctx); 537 BIO *sinbio = BIO_new(BIO_s_mem()); 538 BIO *soutbio = BIO_new(BIO_s_mem()); 539 SSL_set_bio(server, sinbio, soutbio); 540 SSL_set_accept_state(server); 541 BIO_write(sinbio, Data, Size); 542 SSL_do_handshake(server); 543 SSL_free(server); 544 return 0; 545 } 546 EOF 547 # Build the fuzzer. 548 clang++ -g handshake-fuzz.cc -fsanitize=address \ 549 openssl-1.0.1f/libssl.a openssl-1.0.1f/libcrypto.a Fuzzer*.o 550 # Run 20 independent fuzzer jobs. 551 ./a.out -jobs=20 -workers=20 552 553 Voila:: 554 555 #1048576 pulse cov 3424 bits 0 units 9 exec/s 24385 556 ================================================================= 557 ==17488==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x629000004748 at pc 0x00000048c979 bp 0x7fffe3e864f0 sp 0x7fffe3e85ca8 558 READ of size 60731 at 0x629000004748 thread T0 559 #0 0x48c978 in __asan_memcpy 560 #1 0x4db504 in tls1_process_heartbeat openssl-1.0.1f/ssl/t1_lib.c:2586:3 561 #2 0x580be3 in ssl3_read_bytes openssl-1.0.1f/ssl/s3_pkt.c:1092:4 562 563 Note: a `similar fuzzer <https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md>`_ 564 is now a part of the BoringSSL_ source tree. 565 566 Advanced features 567 ================= 568 .. contents:: 569 :local: 570 :depth: 1 571 572 Dictionaries 573 ------------ 574 LibFuzzer supports user-supplied dictionaries with input language keywords 575 or other interesting byte sequences (e.g. multi-byte magic values). 576 Use ``-dict=DICTIONARY_FILE``. For some input languages using a dictionary 577 may significantly improve the search speed. 578 The dictionary syntax is similar to that used by AFL_ for its ``-x`` option:: 579 580 # Lines starting with '#' and empty lines are ignored. 581 582 # Adds "blah" (w/o quotes) to the dictionary. 583 kw1="blah" 584 # Use \\ for backslash and \" for quotes. 585 kw2="\"ac\\dc\"" 586 # Use \xAB for hex values 587 kw3="\xF7\xF8" 588 # the name of the keyword followed by '=' may be omitted: 589 "foo\x0Abar" 590 591 Data-flow-guided fuzzing 592 ------------------------ 593 594 *EXPERIMENTAL*. 595 With an additional compiler flag ``-fsanitize-coverage=trace-cmp`` (see SanitizerCoverageTraceDataFlow_) 596 and extra run-time flag ``-use_traces=1`` the fuzzer will try to apply *data-flow-guided fuzzing*. 597 That is, the fuzzer will record the inputs to comparison instructions, switch statements, 598 and several libc functions (``memcmp``, ``strcmp``, ``strncmp``, etc). 599 It will later use those recorded inputs during mutations. 600 601 This mode can be combined with DataFlowSanitizer_ to achieve better sensitivity. 602 603 Fuzzer-friendly build mode 604 --------------------------- 605 Sometimes the code under test is not fuzzing-friendly. Examples: 606 607 - The target code uses a PRNG seeded e.g. by system time and 608 thus two consequent invocations may potentially execute different code paths 609 even if the end result will be the same. This will cause a fuzzer to treat 610 two similar inputs as significantly different and it will blow up the test corpus. 611 E.g. libxml uses ``rand()`` inside its hash table. 612 - The target code uses checksums to protect from invalid inputs. 613 E.g. png checks CRC for every chunk. 614 615 In many cases it makes sense to build a special fuzzing-friendly build 616 with certain fuzzing-unfriendly features disabled. We propose to use a common build macro 617 for all such cases for consistency: ``FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION``. 618 619 .. code-block:: c++ 620 621 void MyInitPRNG() { 622 #ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION 623 // In fuzzing mode the behavior of the code should be deterministic. 624 srand(0); 625 #else 626 srand(time(0)); 627 #endif 628 } 629 630 631 632 AFL compatibility 633 ----------------- 634 LibFuzzer can be used together with AFL_ on the same test corpus. 635 Both fuzzers expect the test corpus to reside in a directory, one file per input. 636 You can run both fuzzers on the same corpus, one after another: 637 638 .. code-block:: console 639 640 ./afl-fuzz -i testcase_dir -o findings_dir /path/to/program @@ 641 ./llvm-fuzz testcase_dir findings_dir # Will write new tests to testcase_dir 642 643 Periodically restart both fuzzers so that they can use each other's findings. 644 Currently, there is no simple way to run both fuzzing engines in parallel while sharing the same corpus dir. 645 646 You may also use AFL on your target function ``LLVMFuzzerTestOneInput``: 647 see an example `here <https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/afl/afl_driver.cpp>`__. 648 649 How good is my fuzzer? 650 ---------------------- 651 652 Once you implement your target function ``LLVMFuzzerTestOneInput`` and fuzz it to death, 653 you will want to know whether the function or the corpus can be improved further. 654 One easy to use metric is, of course, code coverage. 655 You can get the coverage for your corpus like this: 656 657 .. code-block:: console 658 659 ASAN_OPTIONS=coverage=1:html_cov_report=1 ./fuzzer CORPUS_DIR -runs=0 660 661 This will run all tests in the CORPUS_DIR but will not perform any fuzzing. 662 At the end of the process it will dump a single html file with coverage information. 663 See SanitizerCoverage_ for details. 664 665 You may also use other ways to visualize coverage, 666 e.g. using `Clang coverage <http://clang.llvm.org/docs/SourceBasedCodeCoverage.html>`_, 667 but those will require 668 you to rebuild the code with different compiler flags. 669 670 User-supplied mutators 671 ---------------------- 672 673 LibFuzzer allows to use custom (user-supplied) mutators, 674 see FuzzerInterface.h_ 675 676 Startup initialization 677 ---------------------- 678 If the library being tested needs to be initialized, there are several options. 679 680 The simplest way is to have a statically initialized global object inside 681 `LLVMFuzzerTestOneInput` (or in global scope if that works for you): 682 683 .. code-block:: c++ 684 685 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { 686 static bool Initialized = DoInitialization(); 687 ... 688 689 Alternatively, you may define an optional init function and it will receive 690 the program arguments that you can read and modify. Do this **only** if you 691 realy need to access ``argv``/``argc``. 692 693 .. code-block:: c++ 694 695 extern "C" int LLVMFuzzerInitialize(int *argc, char ***argv) { 696 ReadAndMaybeModify(argc, argv); 697 return 0; 698 } 699 700 701 Leaks 702 ----- 703 704 Binaries built with AddressSanitizer_ or LeakSanitizer_ will try to detect 705 memory leaks at the process shutdown. 706 For in-process fuzzing this is inconvenient 707 since the fuzzer needs to report a leak with a reproducer as soon as the leaky 708 mutation is found. However, running full leak detection after every mutation 709 is expensive. 710 711 By default (``-detect_leaks=1``) libFuzzer will count the number of 712 ``malloc`` and ``free`` calls when executing every mutation. 713 If the numbers don't match (which by itself doesn't mean there is a leak) 714 libFuzzer will invoke the more expensive LeakSanitizer_ 715 pass and if the actual leak is found, it will be reported with the reproducer 716 and the process will exit. 717 718 If your target has massive leaks and the leak detection is disabled 719 you will eventually run out of RAM (see the ``-rss_limit_mb`` flag). 720 721 722 Developing libFuzzer 723 ==================== 724 725 Building libFuzzer as a part of LLVM project and running its test requires 726 fresh clang as the host compiler and special CMake configuration: 727 728 .. code-block:: console 729 730 cmake -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=YES -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON /path/to/llvm 731 ninja check-fuzzer 732 733 734 Fuzzing components of LLVM 735 ========================== 736 .. contents:: 737 :local: 738 :depth: 1 739 740 To build any of the LLVM fuzz targets use the build instructions above. 741 742 clang-format-fuzzer 743 ------------------- 744 The inputs are random pieces of C++-like text. 745 746 .. code-block:: console 747 748 ninja clang-format-fuzzer 749 mkdir CORPUS_DIR 750 ./bin/clang-format-fuzzer CORPUS_DIR 751 752 Optionally build other kinds of binaries (ASan+Debug, MSan, UBSan, etc). 753 754 Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23052 755 756 clang-fuzzer 757 ------------ 758 759 The behavior is very similar to ``clang-format-fuzzer``. 760 761 Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23057 762 763 llvm-as-fuzzer 764 -------------- 765 766 Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=24639 767 768 llvm-mc-fuzzer 769 -------------- 770 771 This tool fuzzes the MC layer. Currently it is only able to fuzz the 772 disassembler but it is hoped that assembly, and round-trip verification will be 773 added in future. 774 775 When run in dissassembly mode, the inputs are opcodes to be disassembled. The 776 fuzzer will consume as many instructions as possible and will stop when it 777 finds an invalid instruction or runs out of data. 778 779 Please note that the command line interface differs slightly from that of other 780 fuzzers. The fuzzer arguments should follow ``--fuzzer-args`` and should have 781 a single dash, while other arguments control the operation mode and target in a 782 similar manner to ``llvm-mc`` and should have two dashes. For example: 783 784 .. code-block:: console 785 786 llvm-mc-fuzzer --triple=aarch64-linux-gnu --disassemble --fuzzer-args -max_len=4 -jobs=10 787 788 Buildbot 789 -------- 790 791 A buildbot continuously runs the above fuzzers for LLVM components, with results 792 shown at http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer . 793 794 FAQ 795 ========================= 796 797 Q. Why doesn't libFuzzer use any of the LLVM support? 798 ----------------------------------------------------- 799 800 There are two reasons. 801 802 First, we want this library to be used outside of the LLVM without users having to 803 build the rest of LLVM. This may sound unconvincing for many LLVM folks, 804 but in practice the need for building the whole LLVM frightens many potential 805 users -- and we want more users to use this code. 806 807 Second, there is a subtle technical reason not to rely on the rest of LLVM, or 808 any other large body of code (maybe not even STL). When coverage instrumentation 809 is enabled, it will also instrument the LLVM support code which will blow up the 810 coverage set of the process (since the fuzzer is in-process). In other words, by 811 using more external dependencies we will slow down the fuzzer while the main 812 reason for it to exist is extreme speed. 813 814 Q. What about Windows then? The fuzzer contains code that does not build on Windows. 815 ------------------------------------------------------------------------------------ 816 817 Volunteers are welcome. 818 819 Q. When this Fuzzer is not a good solution for a problem? 820 --------------------------------------------------------- 821 822 * If the test inputs are validated by the target library and the validator 823 asserts/crashes on invalid inputs, in-process fuzzing is not applicable. 824 * Bugs in the target library may accumulate without being detected. E.g. a memory 825 corruption that goes undetected at first and then leads to a crash while 826 testing another input. This is why it is highly recommended to run this 827 in-process fuzzer with all sanitizers to detect most bugs on the spot. 828 * It is harder to protect the in-process fuzzer from excessive memory 829 consumption and infinite loops in the target library (still possible). 830 * The target library should not have significant global state that is not 831 reset between the runs. 832 * Many interesting target libraries are not designed in a way that supports 833 the in-process fuzzer interface (e.g. require a file path instead of a 834 byte array). 835 * If a single test run takes a considerable fraction of a second (or 836 more) the speed benefit from the in-process fuzzer is negligible. 837 * If the target library runs persistent threads (that outlive 838 execution of one test) the fuzzing results will be unreliable. 839 840 Q. So, what exactly this Fuzzer is good for? 841 -------------------------------------------- 842 843 This Fuzzer might be a good choice for testing libraries that have relatively 844 small inputs, each input takes < 10ms to run, and the library code is not expected 845 to crash on invalid inputs. 846 Examples: regular expression matchers, text or binary format parsers, compression, 847 network, crypto. 848 849 Trophies 850 ======== 851 * GLIBC: https://sourceware.org/glibc/wiki/FuzzingLibc 852 853 * MUSL LIBC: `[1] <http://git.musl-libc.org/cgit/musl/commit/?id=39dfd58417ef642307d90306e1c7e50aaec5a35c>`__ `[2] <http://www.openwall.com/lists/oss-security/2015/03/30/3>`__ 854 855 * `pugixml <https://github.com/zeux/pugixml/issues/39>`_ 856 857 * PCRE: Search for "LLVM fuzzer" in http://vcs.pcre.org/pcre2/code/trunk/ChangeLog?view=markup; 858 also in `bugzilla <https://bugs.exim.org/buglist.cgi?bug_status=__all__&content=libfuzzer&no_redirect=1&order=Importance&product=PCRE&query_format=specific>`_ 859 860 * `ICU <http://bugs.icu-project.org/trac/ticket/11838>`_ 861 862 * `Freetype <https://savannah.nongnu.org/search/?words=LibFuzzer&type_of_search=bugs&Search=Search&exact=1#options>`_ 863 864 * `Harfbuzz <https://github.com/behdad/harfbuzz/issues/139>`_ 865 866 * `SQLite <http://www3.sqlite.org/cgi/src/info/088009efdd56160b>`_ 867 868 * `Python <http://bugs.python.org/issue25388>`_ 869 870 * OpenSSL/BoringSSL: `[1] <https://boringssl.googlesource.com/boringssl/+/cb852981cd61733a7a1ae4fd8755b7ff950e857d>`_ `[2] <https://openssl.org/news/secadv/20160301.txt>`_ `[3] <https://boringssl.googlesource.com/boringssl/+/2b07fa4b22198ac02e0cee8f37f3337c3dba91bc>`_ `[4] <https://boringssl.googlesource.com/boringssl/+/6b6e0b20893e2be0e68af605a60ffa2cbb0ffa64>`_ `[5] <https://github.com/openssl/openssl/pull/931/commits/dd5ac557f052cc2b7f718ac44a8cb7ac6f77dca8>`_ `[6] <https://github.com/openssl/openssl/pull/931/commits/19b5b9194071d1d84e38ac9a952e715afbc85a81>`_ 871 872 * `Libxml2 873 <https://bugzilla.gnome.org/buglist.cgi?bug_status=__all__&content=libFuzzer&list_id=68957&order=Importance&product=libxml2&query_format=specific>`_ and `[HT206167] <https://support.apple.com/en-gb/HT206167>`_ (CVE-2015-5312, CVE-2015-7500, CVE-2015-7942) 874 875 * `Linux Kernel's BPF verifier <https://github.com/iovisor/bpf-fuzzer>`_ 876 877 * Capstone: `[1] <https://github.com/aquynh/capstone/issues/600>`__ `[2] <https://github.com/aquynh/capstone/commit/6b88d1d51eadf7175a8f8a11b690684443b11359>`__ 878 879 * file:`[1] <http://bugs.gw.com/view.php?id=550>`__ `[2] <http://bugs.gw.com/view.php?id=551>`__ `[3] <http://bugs.gw.com/view.php?id=553>`__ `[4] <http://bugs.gw.com/view.php?id=554>`__ 880 881 * Radare2: `[1] <https://github.com/revskills?tab=contributions&from=2016-04-09>`__ 882 883 * gRPC: `[1] <https://github.com/grpc/grpc/pull/6071/commits/df04c1f7f6aec6e95722ec0b023a6b29b6ea871c>`__ `[2] <https://github.com/grpc/grpc/pull/6071/commits/22a3dfd95468daa0db7245a4e8e6679a52847579>`__ `[3] <https://github.com/grpc/grpc/pull/6071/commits/9cac2a12d9e181d130841092e9d40fa3309d7aa7>`__ `[4] <https://github.com/grpc/grpc/pull/6012/commits/82a91c91d01ce9b999c8821ed13515883468e203>`__ `[5] <https://github.com/grpc/grpc/pull/6202/commits/2e3e0039b30edaf89fb93bfb2c1d0909098519fa>`__ `[6] <https://github.com/grpc/grpc/pull/6106/files>`__ 884 885 * WOFF2: `[1] <https://github.com/google/woff2/commit/a15a8ab>`__ 886 887 * LLVM: `Clang <https://llvm.org/bugs/show_bug.cgi?id=23057>`_, `Clang-format <https://llvm.org/bugs/show_bug.cgi?id=23052>`_, `libc++ <https://llvm.org/bugs/show_bug.cgi?id=24411>`_, `llvm-as <https://llvm.org/bugs/show_bug.cgi?id=24639>`_, Disassembler: http://reviews.llvm.org/rL247405, http://reviews.llvm.org/rL247414, http://reviews.llvm.org/rL247416, http://reviews.llvm.org/rL247417, http://reviews.llvm.org/rL247420, http://reviews.llvm.org/rL247422. 888 889 .. _pcre2: http://www.pcre.org/ 890 .. _AFL: http://lcamtuf.coredump.cx/afl/ 891 .. _SanitizerCoverage: http://clang.llvm.org/docs/SanitizerCoverage.html 892 .. _SanitizerCoverageTraceDataFlow: http://clang.llvm.org/docs/SanitizerCoverage.html#tracing-data-flow 893 .. _DataFlowSanitizer: http://clang.llvm.org/docs/DataFlowSanitizer.html 894 .. _AddressSanitizer: http://clang.llvm.org/docs/AddressSanitizer.html 895 .. _LeakSanitizer: http://clang.llvm.org/docs/LeakSanitizer.html 896 .. _Heartbleed: http://en.wikipedia.org/wiki/Heartbleed 897 .. _FuzzerInterface.h: https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/FuzzerInterface.h 898 .. _3.7.0: http://llvm.org/releases/3.7.0/docs/LibFuzzer.html 899 .. _building Clang from trunk: http://clang.llvm.org/get_started.html 900 .. _MemorySanitizer: http://clang.llvm.org/docs/MemorySanitizer.html 901 .. _UndefinedBehaviorSanitizer: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html 902 .. _`coverage counters`: http://clang.llvm.org/docs/SanitizerCoverage.html#coverage-counters 903 .. _`caller-callee pairs`: http://clang.llvm.org/docs/SanitizerCoverage.html#caller-callee-coverage 904 .. _BoringSSL: https://boringssl.googlesource.com/boringssl/ 905 .. _`fuzz various parts of LLVM itself`: `Fuzzing components of LLVM`_ 906
