Home | History | Annotate | only in /external/linux-kselftest/android
Up to higher level directory
NameDateSize
include/22-Oct-2020
kselftest_test_list.mk22-Oct-20202.6K
patches/22-Oct-2020
README.md22-Oct-202011.4K

README.md

      1 VTS Kselftest Workflow
      2 ======================
      3 
      4 A Guide to Developing Kselftest for VTS/Android
      5 
      6 What is Kselftest?
      7 ------------------
      8 
      9 Kselftest is a collection of tests included within the Linux kernel repository
     10 at tools/testing/selftests. The tests cover a broad range of areas but total
     11 coverage (as of 4.14 anyway) is very spotty. Many of the tests do not apply to
     12 Android platforms.
     13 
     14 As of mid 2018 there are 23 tests in kselftest which run on ARM in VTS.
     15 Almost all of them get executed in both 32-bit and 64-bit mode. There are many
     16 tests which are not enabled because they have dependencies which are not met on
     17 Android platforms.
     18 
     19 How is Kselftest Run in VTS?
     20 ----------------------------
     21 
     22 The Kselftest source is located at external/linux-kselftest in the Android
     23 tree. It is a set of files (tools/testing/selftests/) copied into its own
     24 repository from the Linux kernel repository. This is not an exact mirror of
     25 upstream; there are outstanding changes to kselftest for it to work on Android
     26 which have not yet been pushed upstream (contained in android/patches). In
     27 addition to the kselftest source there is also the VTS wrapper around it. This
     28 is located at test/vts-testcase/kernel/linux_kselftest. Some noteworthy
     29 directories/files:
     30 
     31 * `external/linux-kselftest/Android.bp`: Lists build rules for the kselftest modules built in Soong.
     32 * `external/linux-kselftest/android/`: Contains Android-specific files, aside from Android.[bp, mk] at top level.
     33 * `external/linux-kselftest/android/Android.kselftest.mk`: Lists build rules for the kselftest modules built under make.
     34 * `external/linux-kselftest/android/kselftest_test_list.mk`: Lists all tests that will get pulled into VTS - VTS depends on this list.
     35 * `external/linux-kselftest/android/README`: Lists details on why particular tests are not enabled in Android.
     36 * `external/linux-kselftest/tools/testing/selftests`: Kselftest sources.
     37 * `test/vts-testcase/kernel/linux_kselftest/kselftest_config.py`: Lists the tests included in the staging and stable VTS kselftest suites (VtsKernelLinuxKselftest, VtsKernelLinuxKselftestStaging). If a test is not listed here then it will not be run by VTS.
     38 
     39 To run VTS kselftest it must first be built. VTS is not device specific, you
     40 need not compile it specifically for the device you wish to run it on, assuming
     41 it is the same architecture.
     42 * `. build/envsetup.sh`
     43 * `lunch`
     44 * `make -j vts`
     45 
     46 Before running VTS ensure your host has the required prerequisites (this need
     47 only be done once):
     48 * `. test/vts/script/setup.sh`
     49 
     50 Then open vts-tradefed and run the VTS stable set:
     51 * `vts-tradefed`
     52 * `vts-tf > run vts-kernel -m VtsKernelLinuxKselftest`
     53 
     54 If you regularly work with multiple devices it may be useful to specify the
     55 specific device you wish to run VTS on via the serial number:
     56 * `vts-tf > run vts-kernel -m VtsKernelLinuxKselftest -s 000123456789`
     57 
     58 Or we can run the staging set:
     59 * `vts-tf > run vts-kernel -m VtsKernelLinuxKselftestStaging`
     60 
     61 Or a specific test within the stable set:
     62 * `vts-tf > run vts-kernel -m VtsKernelLinuxKselftest -t net_socket`
     63 
     64 Maybe several:
     65 * `vts-tf > run vts-kernel -m VtsKernelLinuxKselftest -t net_socket,timers_posix_timers`
     66 
     67 Running Kselftest Faster
     68 ------------------------
     69 
     70 Running kselftest tests within VTS can be quite cumbersome, especially if you
     71 are iterating a lot trying to debug something. Build and run kselftest tests
     72 faster by doing
     73 
     74 * `external/linux-kselftest$ mma`
     75 * `external/linux-kselftest$ adb sync data`
     76 
     77 The test cases will be located at `/data/nativetest{64,}/linux-kselftest/`.
     78 
     79 Build just the kselftest test you care about with `make -j kselftest_<testname>` (listed in `external/linux-kselftest/android/kselftest_test_list.mk`).
     80 
     81 What Outstanding Issues Exist?
     82 ------------------------------
     83 
     84 The hotlist for kselftest bugs is
     85 [kselftest-todo](https://buganizer.corp.google.com/hotlists/745928).
     86 
     87 When you begin working on a kselftest bug please assign the bug to yourself so
     88 that others know it is being worked on.
     89 
     90 Testing x86_64
     91 --------------
     92 
     93 It is not advisable to run kselftest tests directly on your host unless you are fully
     94 aware of what the tests will do and are okay with it. These tests may
     95 destabilize your box or cause data loss. If you need to run tests on an x86
     96 platform and are unsure if they are safe you should run them in emulation, in a
     97 virtualized environment, or on a dedicated development x86 platform.
     98 
     99 Sending Fixes Upstream
    100 ----------------------
    101 
    102 Kselftest is part of the upstream Linux kernel and as such you can use the
    103 get_maintainers.pl script there to see who to send patches to. Heres an
    104 example using a commit already upstream:
    105 
    106 ```
    107 smuckle@smuckle:~/repos/linux$ git show 352909b49ba | scripts/get_maintainer.pl
    108 Shuah Khan <shuah (a] kernel.org> (maintainer:KERNEL SELFTEST FRAMEWORK,commit_signer:2/6=33%,authored:2/6=33%,added_lines:2/6=33%,removed_lines:3/6=50%)
    109 Thomas Gleixner <tglx (a] linutronix.de> (commit_signer:2/6=33%)
    110 Greg Kroah-Hartman <gregkh (a] linuxfoundation.org> (commit_signer:1/6=17%,authored:1/6=17%,added_lines:1/6=17%)
    111 "Radim Krm" <rkrcmar (a] redhat.com> (commit_signer:1/6=17%)
    112 Ingo Molnar <mingo (a] kernel.org> (commit_signer:1/6=17%,commit_signer:1/1=100%)
    113 Andy Lutomirski <luto (a] kernel.org> (authored:2/6=33%,added_lines:2/6=33%,removed_lines:2/6=33%,authored:1/1=100%,added_lines:500/500=100%)
    114 "Kirill A. Shutemov" <kirill.shutemov (a] linux.intel.com> (authored:1/6=17%,added_lines:1/6=17%,removed_lines:1/6=17%)
    115 linux-kernel (a] vger.kernel.org (open list)
    116 linux-kselftest (a] vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK)
    117 ```
    118 
    119 In summary patches should be sent to linux-kselftest (a] vger.kernel.org and
    120 linux-kernel (a] vger.kernel.org, Shuah Khan (kselftest maintainer), and anyone
    121 else who has touched the code in question. Standard Linux kernel coding style
    122 and patch rules apply. They should be checkpatch (scripts/checkpatch.pl in the
    123 kernel repository) clean and sent in plain text in canonical patch format. One
    124 easy way to do this is by using git format-patch and git send-email.
    125 
    126 Merging Fixes
    127 -------------
    128 
    129 When possible please merge fixes upstream first. Then cherrypick the change
    130 onto aosp/master in external/linux-kselftest.
    131 
    132 If your change cannot go upstream first for whatever reason, please commit a
    133 patch for it into external/linux-kselftest/android/patches. This allows easier
    134 tracking of the delta with upstream and streamlines upgrades to new kselftest
    135 releases.
    136 
    137 Updating Kselftest
    138 ------------------
    139 
    140 To merge in a new upstream version of kselftest:
    141 1. Do a git merge of the upstream tag with the "ours" policy, dropping all upstream changes. Do not commit yet.
    142 2. Delete tools/testing/selftests and replace it with a copy from the upstream kernel at the same tag as used above.
    143 3. Apply the patches in android/patches/, resolving conflicts as required.
    144 4. Test on all supported kernel versions, ensuring that any tests currently enabled in VTS do not generate new failures.
    145 5. Commit the merge.
    146 
    147 Notes on Individual Tests
    148 -------------------------
    149 
    150 ### bpf/
    151 
    152 The bpf tests depend on the bpf and elf libraries, and several kernel
    153 config options that are not required by Android.
    154 
    155 ### cpufreq/
    156 
    157 Cpufreq is not required by Android.
    158 
    159 ### cpu-hotplug/
    160 
    161 Cpu hotplug is not required by Android.
    162 
    163 ### filesystems/dnotify_test.c
    164 
    165 This test has an endless loop in it. It is not currently run as part of the
    166 kselftest suite.
    167 
    168 ### firmware/
    169 
    170 These tests depend on a kernel module enabled by CONFIG_TEST_FIRMWARE, which is
    171 not required by Android.
    172 
    173 ### ftrace/
    174 
    175 Ftrace is a debug feature and not required by Android.
    176 
    177 ### gpio/
    178 
    179 The gpio test requires a test kernel module and gpiolib, neither of which are
    180 required by Android.
    181 
    182 ### ipc/
    183 
    184 The ipc test requires CONFIG_EXPERT and CONFIG_CHECKPOINT_RESTORE be enabled,
    185 neither of which are required by Android.
    186 
    187 ### kvm/
    188 
    189 KVM is not required by Android.
    190 
    191 ### lib/
    192 
    193 The lib/ tests rely on kernel modules enabled by CONFIG_TEST_PRINTF,
    194 CONFIG_TEST_BITMAP, and CONFIG_PRIME_NUMBERS. None of these are required by
    195 Android.
    196 
    197 ### locking/
    198 
    199 The wait-wound mutex test depends on a kernel module enabled by
    200 CONFIG_WW_MUTEX_SELFTEST which is not required by Android.
    201 
    202 ### media_tests/
    203 
    204 The media_tests depend on /dev/mediaX and /dev/videoX files which may not be
    205 present. They also require manual inspection of dmesg and are not part of
    206 the normal kselftest suite.
    207 
    208 ### membarrier/
    209 
    210 The membarrier test depends on CONFIG_EXPERT and CONFIG_MEMBARRIER being
    211 enabled, neither of which are required by Android.
    212 
    213 ### memfd/
    214 
    215 These tests depend on FUSE which is not present in Android.
    216 
    217 ### memory-hotplug/
    218 
    219 Memory hotplug is not required by Android.
    220 
    221 ### mount/
    222 
    223 The mount test depends on CONFIG_USER_NS which is not required by Android.
    224 
    225 ### mqueue/
    226 
    227 These tests depend on -lrt and -lpopt which are not present in Android.
    228 
    229 ### net/
    230 
    231 The test_bpf.sh test requires a kernel module enabled by CONFIG_TEST_BPF.
    232 The netdevice.sh test operates on an ethernet interface.
    233 The psock_fanout test depends on CONFIG_BPF_SYSCALL which is not required on
    234 Android.
    235 
    236 ### netfilter/
    237 
    238 These tests depend on CONFIG_NET_NS and CONFIG_NF_TABLES_INET, neither of which
    239 are required by Android.
    240 
    241 ### nsfs/
    242 
    243 These tests depend on CONFIG_USER_NS and CONFIG_PID_NS, neither of which are
    244 required by Android.
    245 
    246 ### ntb/
    247 
    248 NTB (non-transparent bridge) is not required by Android.
    249 
    250 ### pstore/
    251 
    252 Pstore is recommended, not required, for Android.
    253 
    254 ### ptp/
    255 
    256 PTP support is not required by Android.
    257 
    258 ### rseq/
    259 
    260 The rseq system call, enabled by CONFIG_RSEQ, is not required by Android.
    261 
    262 ### sigaltstack/
    263 
    264 This is a test for functionality that is not available in bionic (swapcontext()).
    265 
    266 ### static_keys/
    267 
    268 This test depends on a kernel module enabled by CONFIG_TEST_STATIC_KEYS. This
    269 kernel module is not required by Android.
    270 
    271 ### sync/
    272 
    273 These tests verify the legacy sync API which is deprecated on Android.
    274 
    275 ### sysctl
    276 
    277 This test depends on a kernel module enabled by CONFIG_TEST_SYSCTL. This kernel
    278 module is not required by Android.
    279 
    280 ### timers/
    281 
    282 The adjtick test adjusts the system time and takes 2 minutes.
    283 
    284 The change_skew test takes too long to run as part of VTS.
    285 
    286 The clocksource_switch test takes too long to run as part of VTS and also tries
    287 all clocksources.
    288 
    289 The freq_step test relies on clocks having a greater precision than what may be
    290 available.
    291 
    292 The leap-a-day test repeatedly modifies the system date in order to test leap
    293 second behavior. This may disrupt the target environment.
    294 
    295 The leapcrash test adjusts the system date and takes several minutes to run.
    296 
    297 The mqueue_lat test relies on mq_* calls which are not present on Android.
    298 
    299 The rtctest_setdate test modifies the RTC's time and date which may disrupt the
    300 target environment.
    301 
    302 The set-2038 test repeatedly sets the time in order to test edge cases. This
    303 may disrupt the target enviornment. The test also takes a long time.
    304 
    305 The set-tz test sets the timezone which may disrupt the target environment.
    306 
    307 The skew_consistency test takes too long to run as part of VTS.
    308 
    309 ### uevent/
    310 
    311 This test depends on CONFIG_USER_NS which is not required by Android.
    312 
    313 ### user/
    314 
    315 This test depends on a kernel module enabled by CONFIG_TEST_USER_COPY. This
    316 kernel module is not required by Android.
    317 
    318 ### vm/
    319 
    320 The hugepage-mmap, hugepage-shm, compaction, and map_hugetlb tests rely on
    321 hugetlbfs/hugetlb page support which is not required by Android.
    322 
    323 The userfaultfd test relies on the userfaultfd syscall which is not required by
    324 Android.
    325 
    326 ### watchdog/
    327 
    328 The watchdog test depends on CONFIG_WATCHDOG which is not required by Android.
    329 
    330 ### zram/
    331 
    332 The zram test depends on CONFIG_ZSMALLOC and CONFIG_ZRAM which are not required
    333 by Android.
    334