1 ============== 2 Testing libc++ 3 ============== 4 5 .. contents:: 6 :local: 7 8 Getting Started 9 =============== 10 11 libc++ uses LIT to configure and run its tests. The primary way to run the 12 libc++ tests is by using make check-libcxx. However since libc++ can be used 13 in any number of possible configurations it is important to customize the way 14 LIT builds and runs the tests. This guide provides information on how to use 15 LIT directly to test libc++. 16 17 Please see the `Lit Command Guide`_ for more information about LIT. 18 19 .. _LIT Command Guide: http://llvm.org/docs/CommandGuide/lit.html 20 21 Setting up the Environment 22 -------------------------- 23 24 After building libc++ you must setup your environment to test libc++ using 25 LIT. 26 27 #. Create a shortcut to the actual lit executable so that you can invoke it 28 easily from the command line. 29 30 .. code-block:: bash 31 32 $ alias lit='python path/to/llvm/utils/lit/lit.py' 33 34 #. Tell LIT where to find your build configuration. 35 36 .. code-block:: bash 37 38 $ export LIBCXX_SITE_CONFIG=path/to/build-libcxx/test/lit.site.cfg 39 40 Example Usage 41 ------------- 42 43 Once you have your environment set up and you have built libc++ you can run 44 parts of the libc++ test suite by simply running `lit` on a specified test or 45 directory. For example: 46 47 .. code-block:: bash 48 49 $ cd path/to/src/libcxx 50 $ lit -sv test/std/re # Run all of the std::regex tests 51 $ lit -sv test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test 52 $ lit -sv test/std/atomics test/std/threads # Test std::thread and std::atomic 53 54 Sometimes you'll want to change the way LIT is running the tests. Custom options 55 can be specified using the `--param=<name>=<val>` flag. The most common option 56 you'll want to change is the standard dialect (ie -std=c++XX). By default the 57 test suite will select the newest C++ dialect supported by the compiler and use 58 that. However if you want to manually specify the option like so: 59 60 .. code-block:: bash 61 62 $ lit -sv test/std/containers # Run the tests with the newest -std 63 $ lit -sv --param=std=c++03 test/std/containers # Run the tests in C++03 64 65 Occasionally you'll want to add extra compile or link flags when testing. 66 You can do this as follows: 67 68 .. code-block:: bash 69 70 $ lit -sv --param=compile_flags='-Wcustom-warning' 71 $ lit -sv --param=link_flags='-L/custom/library/path' 72 73 Some other common examples include: 74 75 .. code-block:: bash 76 77 # Specify a custom compiler. 78 $ lit -sv --param=cxx_under_test=/opt/bin/g++ test/std 79 80 # Enable warnings in the test suite 81 $ lit -sv --param=enable_warnings=true test/std 82 83 # Use UBSAN when running the tests. 84 $ lit -sv --param=use_sanitizer=Undefined 85 86 87 LIT Options 88 =========== 89 90 :program:`lit` [*options*...] [*filenames*...] 91 92 Command Line Options 93 -------------------- 94 95 To use these options you pass them on the LIT command line as --param NAME or 96 --param NAME=VALUE. Some options have default values specified during CMake's 97 configuration. Passing the option on the command line will override the default. 98 99 .. program:: lit 100 101 .. option:: --cxx_under_test=<path/to/compiler> 102 103 Specify the compiler used to build the tests. 104 105 .. option:: std=<standard version> 106 107 **Values**: c++98, c++03, c++11, c++14, c++1z 108 109 Change the standard version used when building the tests. 110 111 .. option:: --libcxx_site_config=<path/to/lit.site.cfg> 112 113 Specify the site configuration to use when running the tests. This option 114 overrides the enviroment variable LIBCXX_SITE_CONFIG. 115 116 .. option:: --libcxx_headers=<path/to/headers> 117 118 Specify the libc++ headers that are tested. By default the headers in the 119 source tree are used. 120 121 .. option:: --cxx_library_root=<path/to/lib/> 122 123 Specify the directory of the libc++ library to be tested. By default the 124 library folder of the build directory is used. This option cannot be used 125 when use_system_lib is provided. 126 127 128 .. option:: --cxx_runtime_root=<path/to/lib/> 129 130 Specify the directory of the libc++ library to use at runtime. This directory 131 is not added to the linkers search path. This can be used to compile tests 132 against one version of libc++ and run them using another. The default value 133 for this option is `cxx_library_root`. This option cannot be used 134 when use_system_lib is provided. 135 136 .. option:: --use_system_lib=<bool> 137 138 **Default**: False 139 140 Enable or disable testing against the installed version of libc++ library. 141 Note: This does not use the installed headers. 142 143 .. option:: --use_lit_shell=<bool> 144 145 Enable or disable the use of LIT's internal shell in ShTests. If the 146 environment variable LIT_USE_INTERNAL_SHELL is present then that is used as 147 the default value. Otherwise the default value is True on Windows and False 148 on every other platform. 149 150 .. option:: --no_default_flags=<bool> 151 152 **Default**: False 153 154 Disable all default compile and link flags from being added. When this 155 option is used only flags specified using the compile_flags and link_flags 156 will be used. 157 158 .. option:: --compile_flags="<list-of-args>" 159 160 Specify additional compile flags as a space delimited string. 161 Note: This options should not be used to change the standard version used. 162 163 .. option:: --link_flags="<list-of-args>" 164 165 Specify additional link flags as a space delimited string. 166 167 .. option:: --debug_level=<level> 168 169 **Values**: 0, 1 170 171 Enable the use of debug mode. Level 0 enables assertions and level 1 enables 172 assertions and debugging of iterator misuse. 173 174 .. option:: use_sanitizer=<sanitizer name> 175 176 **Values**: Memory, MemoryWithOrigins, Address, Undefined 177 178 Run the tests using the given sanitizer. If LLVM_USE_SANITIZER was given when 179 building libc++ then that sanitizer will be used by default. 180 181 .. option:: color_diagnostics 182 183 Enable the use of colorized compile diagnostics. If the color_diagnostics 184 option is specified or the environment variable LIBCXX_COLOR_DIAGNOSTICS is 185 present then color diagnostics will be enabled. 186 187 188 Environment Variables 189 --------------------- 190 191 .. envvar:: LIBCXX_SITE_CONFIG=<path/to/lit.site.cfg> 192 193 Specify the site configuration to use when running the tests. 194 Also see :option:`libcxx_site_config`. 195 196 .. envvar:: LIBCXX_COLOR_DIAGNOSTICS 197 198 If ``LIBCXX_COLOR_DIAGNOSTICS`` is defined then the test suite will attempt 199 to use color diagnostic outputs from the compiler. 200 Also see :option:`color_diagnostics`. 201
