1 BCC (BPF compiler collection) for Android 2 ========================================= 3 4 Introduction 5 ------------ 6 BCC is a compiler and a toolkit, containing powerful kernel tracing tools that 7 trace at the lowest levels, including adding hooks to functions in kernel space 8 and user space to deeply understand system behavior while being low in 9 overhead. [Here's a presentation with an 10 overview](http://www.joelfernandes.org/resources/bcc-ospm.pdf) and visit [BCC's 11 project page](https://github.com/iovisor/bcc) for the official BCC 12 documentation. 13 14 Quick Start 15 ----------- 16 adeb is the primary vehicle for running BCC on Android. It supports 17 preparing the target Android device with necessary kernel headers, cloning and 18 building BCC on device, and other setup. Take a look a quick look at [adeb 19 README](https://github.com/joelagnel/adeb/blob/master/README.md) so that 20 you're familiar with what it is. 21 22 To download a prebuilt filesystem with BCC already built/installed for an ARM64 23 device, you can just run: 24 ``` 25 adeb prepare --full 26 ``` 27 28 This downloads the FS and also downloads prebuilt kernel headers after 29 detecting your device's kernel version. Running BCC this way may cause a warning 30 at startup since the headers may not be an *exact* match for your kernel's 31 sublevel (only version and patchlevel are matched), however it works well in 32 our testing and could be used, as long as you can tolerate the warning. 33 34 If you would like to setup your own kernel headers and prevent the warning, 35 you can point adeb to the kernel sources which will extract headers from there: 36 ``` 37 adeb prepare --full --kernelsrc /path/to/kernel-source/ 38 ``` 39 For targets other than ARM64, see the [Other Architectures 40 section](https://github.com/joelagnel/adeb/blob/master/BCC.md#other-architectures-other-than-arm64) 41 42 Now to run BCC, just start an adeb shell: `adeb shell`. This uses adb 43 as the backend to start a shell into your adeb environment. Try running 44 `opensnoop` or any of the other BCC tracers to confirm that the setup worked 45 correctly. 46 47 If building your own kernel, following are the kernel requirements: 48 49 You need kernel 4.9 or newer. Anything less needs backports. Your kernel needs 50 to be built with the following config options at the minimum: 51 ``` 52 CONFIG_KPROBES=y 53 CONFIG_KPROBE_EVENT=y 54 CONFIG_BPF_SYSCALL=y 55 ``` 56 Optionally, 57 ``` 58 CONFIG_UPROBES=y 59 CONFIG_UPROBE_EVENT=y 60 ``` 61 Additionally, for the criticalsection BCC tracer to work, you need: 62 ``` 63 CONFIG_DEBUG_PREEMPT=y 64 CONFIG_PREEMPTIRQ_EVENTS=y 65 ``` 66 67 Build BCC during adeb install (Optional) 68 -------------------------------------------- 69 If you would like the latest upstream BCC built and installed on your Android 70 device, you can run: 71 ``` 72 adeb prepare --build --bcc --kernelsrc /path/to/kernel-source/ 73 ``` 74 NOTE: This is a slow process and can take a long time. Since it not only builds 75 BCC but also installs all non-BCC debian packages onto the filesystem and configures them. 76 77 Other Architectures (other than ARM64) 78 ----------------------- 79 By default adeb assumes the target Android device is based on ARM64 80 processor architecture. For other architectures, use the --arch option. For 81 example for x86_64 architecture, run: 82 ``` 83 adeb prepare --arch amd64 --build --bcc --kernelsrc /path/to/kernel-source/ 84 ``` 85 Note: The --download option ignores the --arch flag. This is because we only 86 provide pre-built filesystems for ARM64 at the moment. 87 Note: If you pass --arch, you have to pass --build, because prebuilt 88 filesystems are not available for non-arm64 devices. 89 90 Common Issues 91 ------------- 92 Here are some common issues you may face when running different BCC tools. 93 94 * Issue 1: Headers are missing on the target device. 95 96 Symptom: This will usually result in an error like the following: 97 ``` 98 root@localhost:/# criticalstat 99 100 In file included from <built-in>:2 101 In file included from /virtual/include/bcc/bpf.h:12: 102 In file included from include/linux/types.h:5: 103 include/uapi/linux/types.h:4:10: fatal error: 'asm/types.h' file not found 104 105 #include <asm/types.h> 106 107 ^~~~~~~~~~~~~ 108 1 error generated. 109 Traceback (most recent call last): 110 111 File "./criticalstat.py", line 138, in <module> 112 b = BPF(text=bpf_text) 113 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__ 114 raise Exception("Failed to compile BPF text:\n%s" % text) 115 Exception: Failed to compile BPF text: 116 117 #include <uapi/linux/ptrace.h> 118 #include <uapi/linux/limits.h> 119 #include <linux/sched.h> 120 121 extern char _stext[]; 122 ``` 123 124 * Issue 2: `CONFIG_KPROBES` isn't enabled. 125 126 Symptom: This will result in an error like the following: 127 ``` 128 Traceback (most recent call last): 129 File "/usr/share/bcc/tools/cachetop", line 263, in <module> 130 curses.wrapper(handle_loop, args) 131 File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper 132 return func(stdscr, *args, **kwds) 133 File "/usr/share/bcc/tools/cachetop", line 172, in handle_loop 134 b.attach_kprobe(event="add_to_page_cache_lru", fn_name="do_count") 135 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 543, in 136 attach_kprobe 137 fn = self.load_func(fn_name, BPF.KPROBE) 138 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 355, in 139 load_func 140 (func_name, errstr)) 141 Exception: Failed to load BPF program do_count: Invalid argument 142 ``` 143 144 * Issue 3: `CONFIG_BPF_SYSCALL` isn't enabled. 145 146 Symptom: This may result in a compilation error like the following: 147 ``` 148 root@localhost:/# cachetop 149 Traceback (most recent call last): 150 File "/usr/share/bcc/tools/cachetop", line 263, in <module> 151 curses.wrapper(handle_loop, args) 152 File "/usr/lib/python2.7/curses/wrapper.py", line 43, in wrapper 153 return func(stdscr, *args, **kwds) 154 File "/usr/share/bcc/tools/cachetop", line 171, in handle_loop 155 b = BPF(text=bpf_text) 156 File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__ 157 raise Exception("Failed to compile BPF text:\n%s" % text) 158 Exception: Failed to compile BPF text: 159 160 161 #include <uapi/linux/ptrace.h> 162 struct key_t { 163 u64 ip; 164 u32 pid; 165 u32 uid; 166 char comm[16]; 167 }; 168 169 BPF_HASH(counts, struct key_t); 170 171 int do_count(struct pt_regs *ctx) { 172 struct key_t key = {}; 173 u64 zero = 0 , *val; 174 u64 pid = bpf_get_current_pid_tgid(); 175 u32 uid = bpf_get_current_uid_gid(); 176 177 key.ip = PT_REGS_IP(ctx); 178 key.pid = pid & 0xFFFFFFFF; 179 key.uid = uid & 0xFFFFFFFF; 180 bpf_get_current_comm(&(key.comm), 16); 181 182 val = counts.lookup_or_init(&key, &zero); // update counter 183 (*val)++; 184 return 0; 185 } 186 ``` 187
