1 JFuzz 2 ===== 3 4 JFuzz is a tool for generating random programs with the objective 5 of fuzz testing the ART infrastructure. Each randomly generated program 6 can be run under various modes of execution, such as using the interpreter, 7 using the optimizing compiler, using an external reference implementation, 8 or using various target architectures. Any difference between the outputs 9 (**divergence**) may indicate a bug in one of the execution modes. 10 11 JFuzz can be combined with DexFuzz to get multi-layered fuzz testing. 12 13 How to run JFuzz 14 ================ 15 16 jfuzz [-s seed] [-d expr-depth] [-l stmt-length] 17 [-i if-nest] [-n loop-nest] [-v] [-h] 18 19 where 20 21 -s : defines a deterministic random seed 22 (randomized using time by default) 23 -d : defines a fuzzing depth for expressions 24 (higher values yield deeper expressions) 25 -l : defines a fuzzing length for statement lists 26 (higher values yield longer statement sequences) 27 -i : defines a fuzzing nest for if/switch statements 28 (higher values yield deeper nested conditionals) 29 -n : defines a fuzzing nest for for/while/do-while loops 30 (higher values yield deeper nested loops) 31 -t : defines a fuzzing nest for try-catch-finally blocks 32 (higher values yield deeper nested try-catch-finally blocks) 33 -v : prints version number and exits 34 -h : prints help and exits 35 36 The current version of JFuzz sends all output to stdout, and uses 37 a fixed testing class named Test. So a typical test run looks as follows. 38 39 jfuzz > Test.java 40 mkdir classes 41 javac -d classes Test.java 42 dx --dex --output=classes.dex classes 43 art -cp classes.dex Test 44 45 How to start JFuzz testing 46 ========================== 47 48 run_jfuzz_test.py 49 [--num_tests=NUM_TESTS] 50 [--device=DEVICE] 51 [--mode1=MODE] [--mode2=MODE] 52 [--report_script=SCRIPT] 53 [--jfuzz_arg=ARG] 54 [--true_divergence] 55 [--dexer=DEXER] 56 [--debug_info] 57 58 where 59 60 --num_tests : number of tests to run (10000 by default) 61 --device : target device serial number (passed to adb -s) 62 --mode1 : m1 63 --mode2 : m2, with m1 != m2, and values one of 64 ri = reference implementation on host (default for m1) 65 hint = Art interpreter on host 66 hopt = Art optimizing on host (default for m2) 67 tint = Art interpreter on target 68 topt = Art optimizing on target 69 --report_script : path to script called for each divergence 70 --jfuzz_arg : argument for jfuzz 71 --true_divergence : don't bisect timeout divergences 72 --dexer=DEXER : use either dx or d8 to obtain dex files 73 --debug_info : include debugging info 74 75 How to start JFuzz nightly testing 76 ================================== 77 78 run_jfuzz_test_nightly.py 79 [--num_proc NUM_PROC] 80 81 where 82 83 --num_proc : number of run_jfuzz_test.py instances to run (8 by default) 84 85 Remaining arguments are passed to run\_jfuzz_test.py. 86 87 How to start J/DexFuzz testing (multi-layered) 88 ============================================== 89 90 run_dex_fuzz_test.py 91 [--num_tests=NUM_TESTS] 92 [--num_inputs=NUM_INPUTS] 93 [--device=DEVICE] 94 [--dexer=DEXER] 95 [--debug_info] 96 97 where 98 99 --num_tests : number of tests to run (10000 by default) 100 --num_inputs : number of JFuzz programs to generate 101 --device : target device serial number (passed to adb -s) 102 --dexer=DEXER : use either dx or d8 to obtain dex files 103 --debug_info : include debugging info 104 105 Background 106 ========== 107 108 Although test suites are extremely useful to validate the correctness of a 109 system and to ensure that no regressions occur, any test suite is necessarily 110 finite in size and scope. Tests typically focus on validating particular 111 features by means of code sequences most programmers would expect. Regression 112 tests often use slightly less idiomatic code sequences, since they reflect 113 problems that were not anticipated originally, but occurred in the field. 114 Still, any test suite leaves the developer wondering whether undetected bugs 115 and flaws still linger in the system. 116 117 Over the years, fuzz testing has gained popularity as a testing technique for 118 discovering such lingering bugs, including bugs that can bring down a system 119 in an unexpected way. Fuzzing refers to feeding a large amount of random data 120 as input to a system in an attempt to find bugs or make it crash. Generation- 121 based fuzz testing constructs random, but properly formatted input data. 122 Mutation-based fuzz testing applies small random changes to existing inputs 123 in order to detect shortcomings in a system. Profile-guided or coverage-guided 124 fuzzing adds a direction to the way these random changes are applied. Multi- 125 layered approaches generate random inputs that are subsequently mutated at 126 various stages of execution. 127 128 The randomness of fuzz testing implies that the size and scope of testing is no 129 longer bounded. Every new run can potentially discover bugs and crashes that were 130 hereto undetected. 131
