1 tdc - Linux Traffic Control (tc) unit testing suite 2 3 Author: Lucas Bates - lucasb (a] mojatatu.com 4 5 tdc is a Python script to load tc unit tests from a separate JSON file and 6 execute them inside a network namespace dedicated to the task. 7 8 9 REQUIREMENTS 10 ------------ 11 12 * Minimum Python version of 3.4. Earlier 3.X versions may work but are not 13 guaranteed. 14 15 * The kernel must have network namespace support 16 17 * The kernel must have veth support available, as a veth pair is created 18 prior to running the tests. 19 20 * The kernel must have the appropriate infrastructure enabled to run all tdc 21 unit tests. See the config file in this directory for minimum required 22 features. As new tests will be added, config options list will be updated. 23 24 * All tc-related features being tested must be built in or available as 25 modules. To check what is required in current setup run: 26 ./tdc.py -c 27 28 Note: 29 In the current release, tdc run will abort due to a failure in setup or 30 teardown commands - which includes not being able to run a test simply 31 because the kernel did not support a specific feature. (This will be 32 handled in a future version - the current workaround is to run the tests 33 on specific test categories that your kernel supports) 34 35 36 BEFORE YOU RUN 37 -------------- 38 39 The path to the tc executable that will be most commonly tested can be defined 40 in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and 41 define the path. 42 43 If you need to test a different tc executable on the fly, you can do so by 44 using the -p option when running tdc: 45 ./tdc.py -p /path/to/tc 46 47 48 RUNNING TDC 49 ----------- 50 51 To use tdc, root privileges are required. This is because the 52 commands being tested must be run as root. The code that enforces 53 execution by root uid has been moved into a plugin (see PLUGIN 54 ARCHITECTURE, below). 55 56 If nsPlugin is linked, all tests are executed inside a network 57 namespace to prevent conflicts within the host. 58 59 Running tdc without any arguments will run all tests. Refer to the section 60 on command line arguments for more information, or run: 61 ./tdc.py -h 62 63 tdc will list the test names as they are being run, and print a summary in 64 TAP (Test Anything Protocol) format when they are done. If tests fail, 65 output captured from the failing test will be printed immediately following 66 the failed test in the TAP output. 67 68 69 OVERVIEW OF TDC EXECUTION 70 ------------------------- 71 72 One run of tests is considered a "test suite" (this will be refined in the 73 future). A test suite has one or more test cases in it. 74 75 A test case has four stages: 76 77 - setup 78 - execute 79 - verify 80 - teardown 81 82 The setup and teardown stages can run zero or more commands. The setup 83 stage does some setup if the test needs it. The teardown stage undoes 84 the setup and returns the system to a "neutral" state so any other test 85 can be run next. These two stages require any commands run to return 86 success, but do not otherwise verify the results. 87 88 The execute and verify stages each run one command. The execute stage 89 tests the return code against one or more acceptable values. The 90 verify stage checks the return code for success, and also compares 91 the stdout with a regular expression. 92 93 Each of the commands in any stage will run in a shell instance. 94 95 96 USER-DEFINED CONSTANTS 97 ---------------------- 98 99 The tdc_config.py file contains multiple values that can be altered to suit 100 your needs. Any value in the NAMES dictionary can be altered without affecting 101 the tests to be run. These values are used in the tc commands that will be 102 executed as part of the test. More will be added as test cases require. 103 104 Example: 105 $TC qdisc add dev $DEV1 ingress 106 107 The NAMES values are used to substitute into the commands in the test cases. 108 109 110 COMMAND LINE ARGUMENTS 111 ---------------------- 112 113 Run tdc.py -h to see the full list of available arguments. 114 115 usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]] 116 [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] [-N] 117 [-d DEVICE] [-P] [-n] [-V] 118 119 Linux TC unit tests 120 121 optional arguments: 122 -h, --help show this help message and exit 123 -p PATH, --path PATH The full path to the tc executable to use 124 -v, --verbose Show the commands that are being run 125 -N, --notap Suppress tap results for command under test 126 -d DEVICE, --device DEVICE 127 Execute the test case in flower category 128 -P, --pause Pause execution just before post-suite stage 129 130 selection: 131 select which test cases: files plus directories; filtered by categories 132 plus testids 133 134 -D DIR [DIR ...], --directory DIR [DIR ...] 135 Collect tests from the specified directory(ies) 136 (default [tc-tests]) 137 -f FILE [FILE ...], --file FILE [FILE ...] 138 Run tests from the specified file(s) 139 -c [CATG [CATG ...]], --category [CATG [CATG ...]] 140 Run tests only from the specified category/ies, or if 141 no category/ies is/are specified, list known 142 categories. 143 -e ID [ID ...], --execute ID [ID ...] 144 Execute the specified test cases with specified IDs 145 146 action: 147 select action to perform on selected test cases 148 149 -l, --list List all test cases, or those only within the 150 specified category 151 -s, --show Display the selected test cases 152 -i, --id Generate ID numbers for new test cases 153 154 netns: 155 options for nsPlugin (run commands in net namespace) 156 157 -n, --namespace 158 Run commands in namespace as specified in tdc_config.py 159 160 valgrind: 161 options for valgrindPlugin (run command under test under Valgrind) 162 163 -V, --valgrind Run commands under valgrind 164 165 166 PLUGIN ARCHITECTURE 167 ------------------- 168 169 There is now a plugin architecture, and some of the functionality that 170 was in the tdc.py script has been moved into the plugins. 171 172 The plugins are in the directory plugin-lib. The are executed from 173 directory plugins. Put symbolic links from plugins to plugin-lib, 174 and name them according to the order you want them to run. 175 176 Example: 177 178 bjb@bee:~/work/tc-testing$ ls -l plugins 179 total 4 180 lrwxrwxrwx 1 bjb bjb 27 Oct 4 16:12 10-rootPlugin.py -> ../plugin-lib/rootPlugin.py 181 lrwxrwxrwx 1 bjb bjb 25 Oct 12 17:55 20-nsPlugin.py -> ../plugin-lib/nsPlugin.py 182 -rwxr-xr-x 1 bjb bjb 0 Sep 29 15:56 __init__.py 183 184 The plugins are a subclass of TdcPlugin, defined in TdcPlugin.py and 185 must be called "SubPlugin" so tdc can find them. They are 186 distinguished from each other in the python program by their module 187 name. 188 189 This base class supplies "hooks" to run extra functions. These hooks are as follows: 190 191 pre- and post-suite 192 pre- and post-case 193 pre- and post-execute stage 194 adjust-command (runs in all stages and receives the stage name) 195 196 The pre-suite hook receives the number of tests and an array of test ids. 197 This allows you to dump out the list of skipped tests in the event of a 198 failure during setup or teardown stage. 199 200 The pre-case hook receives the ordinal number and test id of the current test. 201 202 The adjust-command hook receives the stage id (see list below) and the 203 full command to be executed. This allows for last-minute adjustment 204 of the command. 205 206 The stages are identified by the following strings: 207 208 - pre (pre-suite) 209 - setup 210 - command 211 - verify 212 - teardown 213 - post (post-suite) 214 215 216 To write a plugin, you need to inherit from TdcPlugin in 217 TdcPlugin.py. To use the plugin, you have to put the 218 implementation file in plugin-lib, and add a symbolic link to it from 219 plugins. It will be detected at run time and invoked at the 220 appropriate times. There are a few examples in the plugin-lib 221 directory: 222 223 - rootPlugin.py: 224 implements the enforcement of running as root 225 - nsPlugin.py: 226 sets up a network namespace and runs all commands in that namespace 227 - valgrindPlugin.py 228 runs each command in the execute stage under valgrind, 229 and checks for leaks. 230 This plugin will output an extra test for each test in the test file, 231 one is the existing output as to whether the test passed or failed, 232 and the other is a test whether the command leaked memory or not. 233 (This one is a preliminary version, it may not work quite right yet, 234 but the overall template is there and it should only need tweaks.) 235 - buildebpfPlugin.py: 236 builds all programs in $EBPFDIR. 237 238 239 ACKNOWLEDGEMENTS 240 ---------------- 241 242 Thanks to: 243 244 Jamal Hadi Salim, for providing valuable test cases 245 Keara Leibovitz, who wrote the CLI test driver that I used as a base for the 246 first version of the tc testing suite. This work was presented at 247 Netdev 1.2 Tokyo in October 2016. 248 Samir Hussain, for providing help while I dove into Python for the first time 249 and being a second eye for this code. 250
