1 Some examples for inject 2 3 inject guarantees the appropriate erroneous return of the specified injection 4 mode (kmalloc,bio,etc) given a call chain and an optional set of predicates. You 5 can also optionally print out the generated BPF program for 6 modification/debugging purposes. 7 8 As a simple example, let's say you wanted to fail all mounts. As of 4.17 we can 9 fail syscalls directly, so let's do that: 10 11 # ./inject.py kmalloc -v 'SyS_mount()' 12 13 The first argument indicates the mode (or what to fail). Appropriate headers are 14 specified, if necessary. The verbosity flag prints the generated program. Note 15 that some syscalls will be available as 'SyS_xyz' and some will be available as 16 'sys_xyz'. This is largely dependent on the number of arguments each syscall 17 takes. 18 19 Trying to mount various filesystems will fail and report an inability to 20 allocate memory, as expected. 21 22 Whenever a predicate is missing, an implicit "(true)" is inserted. The example 23 above can be explicitly written as: 24 25 # ./inject.py kmalloc -v '(true) => SyS_mount()(true)' 26 27 The "(true)" without an associated function is a predicate for the error 28 injection mechanism of the current mode. In the case of kmalloc, the predicate 29 would have access to the arguments of: 30 31 int should_failslab(struct kmem_cache *s, gfp_t gfpflags); 32 33 The bio mode works similarly, with access to the arguments of: 34 35 static noinline int should_fail_bio(struct bio *bio) 36 37 We also note that it's unnecessary to state the arguments of the function if you 38 have no intention to reference them in the associated predicate. 39 40 Now let's say we want to be a bit more specific; suppose you want to fail 41 kmalloc() from mount_subtree() when called from btrfs_mount(). This will fail 42 only btrfs mounts: 43 44 # ./inject.py kmalloc -v 'mount_subtree() => btrfs_mount()' 45 46 Attempting to mount btrfs filesystem during the execution of this command will 47 yield an error, but other filesystems will be fine. 48 49 Next, lets say we want to hit one of the BUG_ONs in fs/btrfs. As of 4.16-rc3, 50 there is a BUG_ON in btrfs_prepare_close_one_device() at fs/btrfs/volumes.c:1002 51 52 To hit this, we can use the following: 53 54 # ./inject.py kmalloc -v 'btrfs_alloc_device() => btrfs_close_devices()' 55 56 While the script was executing, I mounted and unmounted btrfs, causing a 57 segfault on umount(since that satisfied the call path indicated). A look at 58 dmesg will confirm that the erroneous return value injected by the script 59 tripped the BUG_ON, causing a segfault down the line. 60 61 In general, it's worth noting that the required specificity of the call chain is 62 dependent on how much granularity you need. The example above might have 63 performed as expected without the intermediate btrfs_alloc_device, but might 64 have also done something unexpected(an earlier kmalloc could have failed before 65 the one we were targetting). 66 67 For hot paths, the approach outlined above isn't enough. If a path is traversed 68 very often, we can distinguish distinct calls with function arguments. Let's say 69 we want to fail the dentry allocation of a file creatively named 'bananas'. We 70 can do the following: 71 72 # ./inject.py kmalloc -v 'd_alloc_parallel(struct dentry *parent, const struct 73 qstr *name)(STRCMP(name->name, 'bananas'))' 74 75 While this script is executing, any operation that would cause a dentry 76 allocation where the name is 'bananas' fails, as expected. 77 78 Here, since we're referencing a function argument in our predicate, we need to 79 provide the function signature up to the argument we're using. 80 81 To note, STRCMP is a workaround for some rewriter issues. It will take input of 82 the form (x->...->z, 'literal'), and generate some equivalent code that the 83 verifier is more friendly about. It's not horribly robust, but works for the 84 purposes of making string comparisons a bit easier. 85 86 Finally, we briefly demonstrate how to inject bio failures. The mechanism is 87 identical, so any information from above will apply. 88 89 Let's say we want to fail bio requests when the request is to some specific 90 sector. An example use case would be to fail superblock writes in btrfs. For 91 btrfs, we know that there must be a superblock at 65536 bytes, or sector 128. 92 This allows us to run the following: 93 94 # ./inject.py bio -v -I 'linux/blkdev.h' '(({struct gendisk *d = bio->bi_disk; 95 struct disk_part_tbl *tbl = d->part_tbl; struct hd_struct **parts = (void *)tbl + 96 sizeof(struct disk_part_tbl); struct hd_struct **partp = parts + bio->bi_partno; 97 struct hd_struct *p = *partp; dev_t disk = p->__dev.devt; disk == 98 MKDEV(254,16);}) && bio->bi_iter.bi_sector == 128)' 99 100 The predicate in the command above has two parts. The first is a compound 101 statement which shortens to "only if the system is btrfs", but is long due 102 to rewriter/verifier shenanigans. The major/minor information can be found 103 however; I used Python. The second part simply checks the starting 104 address of bi_iter. While executing, this script effectively fails superblock 105 writes to the superblock at sector 128 without affecting other filesystems. 106 107 As an extension to the above, one could easily fail all btrfs superblock writes 108 (we only fail the primary) by calculating the sector number of the mirrors and 109 amending the predicate accordingly. 110 111 Inject also provides a probability option; this allows you to fail the 112 path+predicates some percentage of the time. For example, let's say we want to 113 fail our mounts half the time: 114 115 # ./inject.py kmalloc -v -P 0.01 'SyS_mount()' 116 117 USAGE message: 118 usage: inject.py [-h] [-I header] [-P probability] [-v] {kmalloc,bio} spec 119 120 Fail specified kernel functionality when call chain and predicates are met 121 122 positional arguments: 123 {kmalloc,bio} indicate which base kernel function to fail 124 spec specify call chain 125 126 optional arguments: 127 -h, --help show this help message and exit 128 -I header, --include header 129 additional header files to include in the BPF program 130 -P probability, --probability probability 131 probability that this call chain will fail 132 -v, --verbose print BPF program 133 134 EXAMPLES: 135 # ./inject.py kmalloc -v 'SyS_mount()' 136 Fails all calls to syscall mount 137 # ./inject.py kmalloc -v '(true) => SyS_mount()(true)' 138 Explicit rewriting of above 139 # ./inject.py kmalloc -v 'mount_subtree() => btrfs_mount()' 140 Fails btrfs mounts only 141 # ./inject.py kmalloc -v 'd_alloc_parallel(struct dentry *parent, const struct \ 142 qstr *name)(STRCMP(name->name, 'bananas'))' 143 Fails dentry allocations of files named 'bananas' 144 # ./inject.py kmalloc -v -P 0.01 'SyS_mount()' 145 Fails calls to syscall mount with 1% probability 146
