Home | History | Annotate | only in /art/tools/jfuzz
Up to higher level directory
NameDateSize
__init__.py06-Dec-2017698
Android.mk06-Dec-2017851
jfuzz.cc06-Dec-201736.2K
README.md06-Dec-20174.7K
run_dex_fuzz_test.py06-Dec-20175.5K
run_jfuzz_test.py06-Dec-201720.6K
run_jfuzz_test_nightly.py06-Dec-20173K

README.md

      1 JFuzz
      2 =====
      3 
      4 JFuzz is a tool for generating random programs with the objective
      5 of fuzz testing the ART infrastructure. Each randomly generated program
      6 can be run under various modes of execution, such as using the interpreter,
      7 using the optimizing compiler, using an external reference implementation,
      8 or using various target architectures. Any difference between the outputs
      9 (**divergence**) may indicate a bug in one of the execution modes.
     10 
     11 JFuzz can be combined with DexFuzz to get multi-layered fuzz testing.
     12 
     13 How to run JFuzz
     14 ================
     15 
     16     jfuzz [-s seed] [-d expr-depth] [-l stmt-length]
     17              [-i if-nest] [-n loop-nest] [-v] [-h]
     18 
     19 where
     20 
     21     -s : defines a deterministic random seed
     22          (randomized using time by default)
     23     -d : defines a fuzzing depth for expressions
     24          (higher values yield deeper expressions)
     25     -l : defines a fuzzing length for statement lists
     26          (higher values yield longer statement sequences)
     27     -i : defines a fuzzing nest for if/switch statements
     28          (higher values yield deeper nested conditionals)
     29     -n : defines a fuzzing nest for for/while/do-while loops
     30          (higher values yield deeper nested loops)
     31     -v : prints version number and exits
     32     -h : prints help and exits
     33 
     34 The current version of JFuzz sends all output to stdout, and uses
     35 a fixed testing class named Test. So a typical test run looks as follows.
     36 
     37     jfuzz > Test.java
     38     jack -cp ${JACK_CLASSPATH} --output-dex . Test.java
     39     art -classpath classes.dex Test
     40 
     41 How to start JFuzz testing
     42 ==========================
     43 
     44     run_jfuzz_test.py
     45                           [--num_tests=NUM_TESTS]
     46                           [--device=DEVICE]
     47                           [--mode1=MODE] [--mode2=MODE]
     48                           [--report_script=SCRIPT]
     49                           [--jfuzz_arg=ARG]
     50                           [--true_divergence]
     51 
     52 where
     53 
     54     --num_tests       : number of tests to run (10000 by default)
     55     --device          : target device serial number (passed to adb -s)
     56     --mode1           : m1
     57     --mode2           : m2, with m1 != m2, and values one of
     58       ri   = reference implementation on host (default for m1)
     59       hint = Art interpreter on host
     60       hopt = Art optimizing on host (default for m2)
     61       tint = Art interpreter on target
     62       topt = Art optimizing on target
     63     --report_script   : path to script called for each divergence
     64     --jfuzz_arg       : argument for jfuzz
     65     --true_divergence : don't bisect timeout divergences
     66 
     67 How to start JFuzz nightly testing
     68 ==================================
     69 
     70     run_jfuzz_test_nightly.py
     71                           [--num_proc NUM_PROC]
     72 
     73 where
     74 
     75     --num_proc      : number of run_jfuzz_test.py instances to run (8 by default)
     76 
     77 Remaining arguments are passed to run\_jfuzz_test.py.
     78 
     79 How to start J/DexFuzz testing (multi-layered)
     80 ==============================================
     81 
     82     run_dex_fuzz_test.py
     83                           [--num_tests=NUM_TESTS]
     84                           [--num_inputs=NUM_INPUTS]
     85                           [--device=DEVICE]
     86 
     87 where
     88 
     89     --num_tests : number of tests to run (10000 by default)
     90     --num_inputs: number of JFuzz programs to generate
     91     --device    : target device serial number (passed to adb -s)
     92 
     93 Background
     94 ==========
     95 
     96 Although test suites are extremely useful to validate the correctness of a
     97 system and to ensure that no regressions occur, any test suite is necessarily
     98 finite in size and scope. Tests typically focus on validating particular
     99 features by means of code sequences most programmers would expect. Regression
    100 tests often use slightly less idiomatic code sequences, since they reflect
    101 problems that were not anticipated originally, but occurred in the field.
    102 Still, any test suite leaves the developer wondering whether undetected bugs
    103 and flaws still linger in the system.
    104 
    105 Over the years, fuzz testing has gained popularity as a testing technique for
    106 discovering such lingering bugs, including bugs that can bring down a system
    107 in an unexpected way. Fuzzing refers to feeding a large amount of random data
    108 as input to a system in an attempt to find bugs or make it crash. Generation-
    109 based fuzz testing constructs random, but properly formatted input data.
    110 Mutation-based fuzz testing applies small random changes to existing inputs
    111 in order to detect shortcomings in a system. Profile-guided or coverage-guided
    112 fuzzing adds a direction to the way these random changes are applied. Multi-
    113 layered approaches generate random inputs that are subsequently mutated at
    114 various stages of execution.
    115 
    116 The randomness of fuzz testing implies that the size and scope of testing is no
    117 longer bounded. Every new run can potentially discover bugs and crashes that were
    118 hereto undetected.
    119