1 :mod:`unittest` --- Unit testing framework 2 ========================================== 3 4 .. module:: unittest 5 :synopsis: Unit testing framework for Python. 6 7 .. moduleauthor:: Steve Purcell <stephen_purcell (a] yahoo.com> 8 .. sectionauthor:: Steve Purcell <stephen_purcell (a] yahoo.com> 9 .. sectionauthor:: Fred L. Drake, Jr. <fdrake (a] acm.org> 10 .. sectionauthor:: Raymond Hettinger <python (a] rcn.com> 11 12 **Source code:** :source:`Lib/unittest/__init__.py` 13 14 -------------- 15 16 (If you are already familiar with the basic concepts of testing, you might want 17 to skip to :ref:`the list of assert methods <assert-methods>`.) 18 19 The :mod:`unittest` unit testing framework was originally inspired by JUnit 20 and has a similar flavor as major unit testing frameworks in other 21 languages. It supports test automation, sharing of setup and shutdown code 22 for tests, aggregation of tests into collections, and independence of the 23 tests from the reporting framework. 24 25 To achieve this, :mod:`unittest` supports some important concepts in an 26 object-oriented way: 27 28 test fixture 29 A :dfn:`test fixture` represents the preparation needed to perform one or more 30 tests, and any associate cleanup actions. This may involve, for example, 31 creating temporary or proxy databases, directories, or starting a server 32 process. 33 34 test case 35 A :dfn:`test case` is the individual unit of testing. It checks for a specific 36 response to a particular set of inputs. :mod:`unittest` provides a base class, 37 :class:`TestCase`, which may be used to create new test cases. 38 39 test suite 40 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is 41 used to aggregate tests that should be executed together. 42 43 test runner 44 A :dfn:`test runner` is a component which orchestrates the execution of tests 45 and provides the outcome to the user. The runner may use a graphical interface, 46 a textual interface, or return a special value to indicate the results of 47 executing the tests. 48 49 50 .. seealso:: 51 52 Module :mod:`doctest` 53 Another test-support module with a very different flavor. 54 55 `Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_ 56 Kent Beck's original paper on testing frameworks using the pattern shared 57 by :mod:`unittest`. 58 59 `Nose <https://nose.readthedocs.org/en/latest/>`_ and `py.test <http://pytest.org>`_ 60 Third-party unittest frameworks with a lighter-weight syntax for writing 61 tests. For example, ``assert func(10) == 42``. 62 63 `The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_ 64 An extensive list of Python testing tools including functional testing 65 frameworks and mock object libraries. 66 67 `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_ 68 A special-interest-group for discussion of testing, and testing tools, 69 in Python. 70 71 The script :file:`Tools/unittestgui/unittestgui.py` in the Python source distribution is 72 a GUI tool for test discovery and execution. This is intended largely for ease of use 73 for those new to unit testing. For production environments it is 74 recommended that tests be driven by a continuous integration system such as 75 `Buildbot <https://buildbot.net/>`_, `Jenkins <https://jenkins.io/>`_ 76 or `Hudson <http://hudson-ci.org/>`_. 77 78 79 .. _unittest-minimal-example: 80 81 Basic example 82 ------------- 83 84 The :mod:`unittest` module provides a rich set of tools for constructing and 85 running tests. This section demonstrates that a small subset of the tools 86 suffice to meet the needs of most users. 87 88 Here is a short script to test three string methods:: 89 90 import unittest 91 92 class TestStringMethods(unittest.TestCase): 93 94 def test_upper(self): 95 self.assertEqual('foo'.upper(), 'FOO') 96 97 def test_isupper(self): 98 self.assertTrue('FOO'.isupper()) 99 self.assertFalse('Foo'.isupper()) 100 101 def test_split(self): 102 s = 'hello world' 103 self.assertEqual(s.split(), ['hello', 'world']) 104 # check that s.split fails when the separator is not a string 105 with self.assertRaises(TypeError): 106 s.split(2) 107 108 if __name__ == '__main__': 109 unittest.main() 110 111 112 A testcase is created by subclassing :class:`unittest.TestCase`. The three 113 individual tests are defined with methods whose names start with the letters 114 ``test``. This naming convention informs the test runner about which methods 115 represent tests. 116 117 The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an 118 expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse` 119 to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a 120 specific exception gets raised. These methods are used instead of the 121 :keyword:`assert` statement so the test runner can accumulate all test results 122 and produce a report. 123 124 The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you 125 to define instructions that will be executed before and after each test method. 126 They are covered in more detail in the section :ref:`organizing-tests`. 127 128 The final block shows a simple way to run the tests. :func:`unittest.main` 129 provides a command-line interface to the test script. When run from the command 130 line, the above script produces an output that looks like this:: 131 132 ... 133 ---------------------------------------------------------------------- 134 Ran 3 tests in 0.000s 135 136 OK 137 138 Passing the ``-v`` option to your test script will instruct :func:`unittest.main` 139 to enable a higher level of verbosity, and produce the following output:: 140 141 test_isupper (__main__.TestStringMethods) ... ok 142 test_split (__main__.TestStringMethods) ... ok 143 test_upper (__main__.TestStringMethods) ... ok 144 145 ---------------------------------------------------------------------- 146 Ran 3 tests in 0.001s 147 148 OK 149 150 The above examples show the most commonly used :mod:`unittest` features which 151 are sufficient to meet many everyday testing needs. The remainder of the 152 documentation explores the full feature set from first principles. 153 154 155 .. _unittest-command-line-interface: 156 157 Command-Line Interface 158 ---------------------- 159 160 The unittest module can be used from the command line to run tests from 161 modules, classes or even individual test methods:: 162 163 python -m unittest test_module1 test_module2 164 python -m unittest test_module.TestClass 165 python -m unittest test_module.TestClass.test_method 166 167 You can pass in a list with any combination of module names, and fully 168 qualified class or method names. 169 170 Test modules can be specified by file path as well:: 171 172 python -m unittest tests/test_something.py 173 174 This allows you to use the shell filename completion to specify the test module. 175 The file specified must still be importable as a module. The path is converted 176 to a module name by removing the '.py' and converting path separators into '.'. 177 If you want to execute a test file that isn't importable as a module you should 178 execute the file directly instead. 179 180 You can run tests with more detail (higher verbosity) by passing in the -v flag:: 181 182 python -m unittest -v test_module 183 184 When executed without arguments :ref:`unittest-test-discovery` is started:: 185 186 python -m unittest 187 188 For a list of all the command-line options:: 189 190 python -m unittest -h 191 192 .. versionchanged:: 3.2 193 In earlier versions it was only possible to run individual test methods and 194 not modules or classes. 195 196 197 Command-line options 198 ~~~~~~~~~~~~~~~~~~~~ 199 200 :program:`unittest` supports these command-line options: 201 202 .. program:: unittest 203 204 .. cmdoption:: -b, --buffer 205 206 The standard output and standard error streams are buffered during the test 207 run. Output during a passing test is discarded. Output is echoed normally 208 on test fail or error and is added to the failure messages. 209 210 .. cmdoption:: -c, --catch 211 212 :kbd:`Control-C` during the test run waits for the current test to end and then 213 reports all the results so far. A second :kbd:`Control-C` raises the normal 214 :exc:`KeyboardInterrupt` exception. 215 216 See `Signal Handling`_ for the functions that provide this functionality. 217 218 .. cmdoption:: -f, --failfast 219 220 Stop the test run on the first error or failure. 221 222 .. cmdoption:: --locals 223 224 Show local variables in tracebacks. 225 226 .. versionadded:: 3.2 227 The command-line options ``-b``, ``-c`` and ``-f`` were added. 228 229 .. versionadded:: 3.5 230 The command-line option ``--locals``. 231 232 The command line can also be used for test discovery, for running all of the 233 tests in a project or just a subset. 234 235 236 .. _unittest-test-discovery: 237 238 Test Discovery 239 -------------- 240 241 .. versionadded:: 3.2 242 243 Unittest supports simple test discovery. In order to be compatible with test 244 discovery, all of the test files must be :ref:`modules <tut-modules>` or 245 :ref:`packages <tut-packages>` (including :term:`namespace packages 246 <namespace package>`) importable from the top-level directory of 247 the project (this means that their filenames must be valid :ref:`identifiers 248 <identifiers>`). 249 250 Test discovery is implemented in :meth:`TestLoader.discover`, but can also be 251 used from the command line. The basic command-line usage is:: 252 253 cd project_directory 254 python -m unittest discover 255 256 .. note:: 257 258 As a shortcut, ``python -m unittest`` is the equivalent of 259 ``python -m unittest discover``. If you want to pass arguments to test 260 discovery the ``discover`` sub-command must be used explicitly. 261 262 The ``discover`` sub-command has the following options: 263 264 .. program:: unittest discover 265 266 .. cmdoption:: -v, --verbose 267 268 Verbose output 269 270 .. cmdoption:: -s, --start-directory directory 271 272 Directory to start discovery (``.`` default) 273 274 .. cmdoption:: -p, --pattern pattern 275 276 Pattern to match test files (``test*.py`` default) 277 278 .. cmdoption:: -t, --top-level-directory directory 279 280 Top level directory of project (defaults to start directory) 281 282 The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in 283 as positional arguments in that order. The following two command lines 284 are equivalent:: 285 286 python -m unittest discover -s project_directory -p "*_test.py" 287 python -m unittest discover project_directory "*_test.py" 288 289 As well as being a path it is possible to pass a package name, for example 290 ``myproject.subpackage.test``, as the start directory. The package name you 291 supply will then be imported and its location on the filesystem will be used 292 as the start directory. 293 294 .. caution:: 295 296 Test discovery loads tests by importing them. Once test discovery has found 297 all the test files from the start directory you specify it turns the paths 298 into package names to import. For example :file:`foo/bar/baz.py` will be 299 imported as ``foo.bar.baz``. 300 301 If you have a package installed globally and attempt test discovery on 302 a different copy of the package then the import *could* happen from the 303 wrong place. If this happens test discovery will warn you and exit. 304 305 If you supply the start directory as a package name rather than a 306 path to a directory then discover assumes that whichever location it 307 imports from is the location you intended, so you will not get the 308 warning. 309 310 Test modules and packages can customize test loading and discovery by through 311 the `load_tests protocol`_. 312 313 .. versionchanged:: 3.4 314 Test discovery supports :term:`namespace packages <namespace package>`. 315 316 317 .. _organizing-tests: 318 319 Organizing test code 320 -------------------- 321 322 The basic building blocks of unit testing are :dfn:`test cases` --- single 323 scenarios that must be set up and checked for correctness. In :mod:`unittest`, 324 test cases are represented by :class:`unittest.TestCase` instances. 325 To make your own test cases you must write subclasses of 326 :class:`TestCase` or use :class:`FunctionTestCase`. 327 328 The testing code of a :class:`TestCase` instance should be entirely self 329 contained, such that it can be run either in isolation or in arbitrary 330 combination with any number of other test cases. 331 332 The simplest :class:`TestCase` subclass will simply implement a test method 333 (i.e. a method whose name starts with ``test``) in order to perform specific 334 testing code:: 335 336 import unittest 337 338 class DefaultWidgetSizeTestCase(unittest.TestCase): 339 def test_default_widget_size(self): 340 widget = Widget('The widget') 341 self.assertEqual(widget.size(), (50, 50)) 342 343 Note that in order to test something, we use one of the :meth:`assert\*` 344 methods provided by the :class:`TestCase` base class. If the test fails, an 345 exception will be raised, and :mod:`unittest` will identify the test case as a 346 :dfn:`failure`. Any other exceptions will be treated as :dfn:`errors`. 347 348 Tests can be numerous, and their set-up can be repetitive. Luckily, we 349 can factor out set-up code by implementing a method called 350 :meth:`~TestCase.setUp`, which the testing framework will automatically 351 call for every single test we run:: 352 353 import unittest 354 355 class WidgetTestCase(unittest.TestCase): 356 def setUp(self): 357 self.widget = Widget('The widget') 358 359 def test_default_widget_size(self): 360 self.assertEqual(self.widget.size(), (50,50), 361 'incorrect default size') 362 363 def test_widget_resize(self): 364 self.widget.resize(100,150) 365 self.assertEqual(self.widget.size(), (100,150), 366 'wrong size after resize') 367 368 .. note:: 369 The order in which the various tests will be run is determined 370 by sorting the test method names with respect to the built-in 371 ordering for strings. 372 373 If the :meth:`~TestCase.setUp` method raises an exception while the test is 374 running, the framework will consider the test to have suffered an error, and 375 the test method will not be executed. 376 377 Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up 378 after the test method has been run:: 379 380 import unittest 381 382 class WidgetTestCase(unittest.TestCase): 383 def setUp(self): 384 self.widget = Widget('The widget') 385 386 def tearDown(self): 387 self.widget.dispose() 388 389 If :meth:`~TestCase.setUp` succeeded, :meth:`~TestCase.tearDown` will be 390 run whether the test method succeeded or not. 391 392 Such a working environment for the testing code is called a :dfn:`fixture`. 393 394 Test case instances are grouped together according to the features they test. 395 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`, 396 represented by :mod:`unittest`'s :class:`TestSuite` class. In most cases, 397 calling :func:`unittest.main` will do the right thing and collect all the 398 module's test cases for you, and then execute them. 399 400 However, should you want to customize the building of your test suite, 401 you can do it yourself:: 402 403 def suite(): 404 suite = unittest.TestSuite() 405 suite.addTest(WidgetTestCase('test_default_size')) 406 suite.addTest(WidgetTestCase('test_resize')) 407 return suite 408 409 You can place the definitions of test cases and test suites in the same modules 410 as the code they are to test (such as :file:`widget.py`), but there are several 411 advantages to placing the test code in a separate module, such as 412 :file:`test_widget.py`: 413 414 * The test module can be run standalone from the command line. 415 416 * The test code can more easily be separated from shipped code. 417 418 * There is less temptation to change test code to fit the code it tests without 419 a good reason. 420 421 * Test code should be modified much less frequently than the code it tests. 422 423 * Tested code can be refactored more easily. 424 425 * Tests for modules written in C must be in separate modules anyway, so why not 426 be consistent? 427 428 * If the testing strategy changes, there is no need to change the source code. 429 430 431 .. _legacy-unit-tests: 432 433 Re-using old test code 434 ---------------------- 435 436 Some users will find that they have existing test code that they would like to 437 run from :mod:`unittest`, without converting every old test function to a 438 :class:`TestCase` subclass. 439 440 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class. 441 This subclass of :class:`TestCase` can be used to wrap an existing test 442 function. Set-up and tear-down functions can also be provided. 443 444 Given the following test function:: 445 446 def testSomething(): 447 something = makeSomething() 448 assert something.name is not None 449 # ... 450 451 one can create an equivalent test case instance as follows, with optional 452 set-up and tear-down methods:: 453 454 testcase = unittest.FunctionTestCase(testSomething, 455 setUp=makeSomethingDB, 456 tearDown=deleteSomethingDB) 457 458 .. note:: 459 460 Even though :class:`FunctionTestCase` can be used to quickly convert an 461 existing test base over to a :mod:`unittest`\ -based system, this approach is 462 not recommended. Taking the time to set up proper :class:`TestCase` 463 subclasses will make future test refactorings infinitely easier. 464 465 In some cases, the existing tests may have been written using the :mod:`doctest` 466 module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can 467 automatically build :class:`unittest.TestSuite` instances from the existing 468 :mod:`doctest`\ -based tests. 469 470 471 .. _unittest-skipping: 472 473 Skipping tests and expected failures 474 ------------------------------------ 475 476 .. versionadded:: 3.1 477 478 Unittest supports skipping individual test methods and even whole classes of 479 tests. In addition, it supports marking a test as an "expected failure," a test 480 that is broken and will fail, but shouldn't be counted as a failure on a 481 :class:`TestResult`. 482 483 Skipping a test is simply a matter of using the :func:`skip` :term:`decorator` 484 or one of its conditional variants. 485 486 Basic skipping looks like this:: 487 488 class MyTestCase(unittest.TestCase): 489 490 @unittest.skip("demonstrating skipping") 491 def test_nothing(self): 492 self.fail("shouldn't happen") 493 494 @unittest.skipIf(mylib.__version__ < (1, 3), 495 "not supported in this library version") 496 def test_format(self): 497 # Tests that work for only a certain version of the library. 498 pass 499 500 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") 501 def test_windows_support(self): 502 # windows specific testing code 503 pass 504 505 This is the output of running the example above in verbose mode:: 506 507 test_format (__main__.MyTestCase) ... skipped 'not supported in this library version' 508 test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping' 509 test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows' 510 511 ---------------------------------------------------------------------- 512 Ran 3 tests in 0.005s 513 514 OK (skipped=3) 515 516 Classes can be skipped just like methods:: 517 518 @unittest.skip("showing class skipping") 519 class MySkippedTestCase(unittest.TestCase): 520 def test_not_run(self): 521 pass 522 523 :meth:`TestCase.setUp` can also skip the test. This is useful when a resource 524 that needs to be set up is not available. 525 526 Expected failures use the :func:`expectedFailure` decorator. :: 527 528 class ExpectedFailureTestCase(unittest.TestCase): 529 @unittest.expectedFailure 530 def test_fail(self): 531 self.assertEqual(1, 0, "broken") 532 533 It's easy to roll your own skipping decorators by making a decorator that calls 534 :func:`skip` on the test when it wants it to be skipped. This decorator skips 535 the test unless the passed object has a certain attribute:: 536 537 def skipUnlessHasattr(obj, attr): 538 if hasattr(obj, attr): 539 return lambda func: func 540 return unittest.skip("{!r} doesn't have {!r}".format(obj, attr)) 541 542 The following decorators implement test skipping and expected failures: 543 544 .. decorator:: skip(reason) 545 546 Unconditionally skip the decorated test. *reason* should describe why the 547 test is being skipped. 548 549 .. decorator:: skipIf(condition, reason) 550 551 Skip the decorated test if *condition* is true. 552 553 .. decorator:: skipUnless(condition, reason) 554 555 Skip the decorated test unless *condition* is true. 556 557 .. decorator:: expectedFailure 558 559 Mark the test as an expected failure. If the test fails when run, the test 560 is not counted as a failure. 561 562 .. exception:: SkipTest(reason) 563 564 This exception is raised to skip a test. 565 566 Usually you can use :meth:`TestCase.skipTest` or one of the skipping 567 decorators instead of raising this directly. 568 569 Skipped tests will not have :meth:`~TestCase.setUp` or :meth:`~TestCase.tearDown` run around them. 570 Skipped classes will not have :meth:`~TestCase.setUpClass` or :meth:`~TestCase.tearDownClass` run. 571 Skipped modules will not have :func:`setUpModule` or :func:`tearDownModule` run. 572 573 574 .. _subtests: 575 576 Distinguishing test iterations using subtests 577 --------------------------------------------- 578 579 .. versionadded:: 3.4 580 581 When some of your tests differ only by a some very small differences, for 582 instance some parameters, unittest allows you to distinguish them inside 583 the body of a test method using the :meth:`~TestCase.subTest` context manager. 584 585 For example, the following test:: 586 587 class NumbersTest(unittest.TestCase): 588 589 def test_even(self): 590 """ 591 Test that numbers between 0 and 5 are all even. 592 """ 593 for i in range(0, 6): 594 with self.subTest(i=i): 595 self.assertEqual(i % 2, 0) 596 597 will produce the following output:: 598 599 ====================================================================== 600 FAIL: test_even (__main__.NumbersTest) (i=1) 601 ---------------------------------------------------------------------- 602 Traceback (most recent call last): 603 File "subtests.py", line 32, in test_even 604 self.assertEqual(i % 2, 0) 605 AssertionError: 1 != 0 606 607 ====================================================================== 608 FAIL: test_even (__main__.NumbersTest) (i=3) 609 ---------------------------------------------------------------------- 610 Traceback (most recent call last): 611 File "subtests.py", line 32, in test_even 612 self.assertEqual(i % 2, 0) 613 AssertionError: 1 != 0 614 615 ====================================================================== 616 FAIL: test_even (__main__.NumbersTest) (i=5) 617 ---------------------------------------------------------------------- 618 Traceback (most recent call last): 619 File "subtests.py", line 32, in test_even 620 self.assertEqual(i % 2, 0) 621 AssertionError: 1 != 0 622 623 Without using a subtest, execution would stop after the first failure, 624 and the error would be less easy to diagnose because the value of ``i`` 625 wouldn't be displayed:: 626 627 ====================================================================== 628 FAIL: test_even (__main__.NumbersTest) 629 ---------------------------------------------------------------------- 630 Traceback (most recent call last): 631 File "subtests.py", line 32, in test_even 632 self.assertEqual(i % 2, 0) 633 AssertionError: 1 != 0 634 635 636 .. _unittest-contents: 637 638 Classes and functions 639 --------------------- 640 641 This section describes in depth the API of :mod:`unittest`. 642 643 644 .. _testcase-objects: 645 646 Test cases 647 ~~~~~~~~~~ 648 649 .. class:: TestCase(methodName='runTest') 650 651 Instances of the :class:`TestCase` class represent the logical test units 652 in the :mod:`unittest` universe. This class is intended to be used as a base 653 class, with specific tests being implemented by concrete subclasses. This class 654 implements the interface needed by the test runner to allow it to drive the 655 tests, and methods that the test code can use to check for and report various 656 kinds of failure. 657 658 Each instance of :class:`TestCase` will run a single base method: the method 659 named *methodName*. 660 In most uses of :class:`TestCase`, you will neither change 661 the *methodName* nor reimplement the default ``runTest()`` method. 662 663 .. versionchanged:: 3.2 664 :class:`TestCase` can be instantiated successfully without providing a 665 *methodName*. This makes it easier to experiment with :class:`TestCase` 666 from the interactive interpreter. 667 668 :class:`TestCase` instances provide three groups of methods: one group used 669 to run the test, another used by the test implementation to check conditions 670 and report failures, and some inquiry methods allowing information about the 671 test itself to be gathered. 672 673 Methods in the first group (running the test) are: 674 675 .. method:: setUp() 676 677 Method called to prepare the test fixture. This is called immediately 678 before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, 679 any exception raised by this method will be considered an error rather than 680 a test failure. The default implementation does nothing. 681 682 683 .. method:: tearDown() 684 685 Method called immediately after the test method has been called and the 686 result recorded. This is called even if the test method raised an 687 exception, so the implementation in subclasses may need to be particularly 688 careful about checking internal state. Any exception, other than 689 :exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be 690 considered an additional error rather than a test failure (thus increasing 691 the total number of reported errors). This method will only be called if 692 the :meth:`setUp` succeeds, regardless of the outcome of the test method. 693 The default implementation does nothing. 694 695 696 .. method:: setUpClass() 697 698 A class method called before tests in an individual class run. 699 ``setUpClass`` is called with the class as the only argument 700 and must be decorated as a :func:`classmethod`:: 701 702 @classmethod 703 def setUpClass(cls): 704 ... 705 706 See `Class and Module Fixtures`_ for more details. 707 708 .. versionadded:: 3.2 709 710 711 .. method:: tearDownClass() 712 713 A class method called after tests in an individual class have run. 714 ``tearDownClass`` is called with the class as the only argument 715 and must be decorated as a :meth:`classmethod`:: 716 717 @classmethod 718 def tearDownClass(cls): 719 ... 720 721 See `Class and Module Fixtures`_ for more details. 722 723 .. versionadded:: 3.2 724 725 726 .. method:: run(result=None) 727 728 Run the test, collecting the result into the :class:`TestResult` object 729 passed as *result*. If *result* is omitted or ``None``, a temporary 730 result object is created (by calling the :meth:`defaultTestResult` 731 method) and used. The result object is returned to :meth:`run`'s 732 caller. 733 734 The same effect may be had by simply calling the :class:`TestCase` 735 instance. 736 737 .. versionchanged:: 3.3 738 Previous versions of ``run`` did not return the result. Neither did 739 calling an instance. 740 741 .. method:: skipTest(reason) 742 743 Calling this during a test method or :meth:`setUp` skips the current 744 test. See :ref:`unittest-skipping` for more information. 745 746 .. versionadded:: 3.1 747 748 749 .. method:: subTest(msg=None, **params) 750 751 Return a context manager which executes the enclosed code block as a 752 subtest. *msg* and *params* are optional, arbitrary values which are 753 displayed whenever a subtest fails, allowing you to identify them 754 clearly. 755 756 A test case can contain any number of subtest declarations, and 757 they can be arbitrarily nested. 758 759 See :ref:`subtests` for more information. 760 761 .. versionadded:: 3.4 762 763 764 .. method:: debug() 765 766 Run the test without collecting the result. This allows exceptions raised 767 by the test to be propagated to the caller, and can be used to support 768 running tests under a debugger. 769 770 .. _assert-methods: 771 772 The :class:`TestCase` class provides several assert methods to check for and 773 report failures. The following table lists the most commonly used methods 774 (see the tables below for more assert methods): 775 776 +-----------------------------------------+-----------------------------+---------------+ 777 | Method | Checks that | New in | 778 +=========================================+=============================+===============+ 779 | :meth:`assertEqual(a, b) | ``a == b`` | | 780 | <TestCase.assertEqual>` | | | 781 +-----------------------------------------+-----------------------------+---------------+ 782 | :meth:`assertNotEqual(a, b) | ``a != b`` | | 783 | <TestCase.assertNotEqual>` | | | 784 +-----------------------------------------+-----------------------------+---------------+ 785 | :meth:`assertTrue(x) | ``bool(x) is True`` | | 786 | <TestCase.assertTrue>` | | | 787 +-----------------------------------------+-----------------------------+---------------+ 788 | :meth:`assertFalse(x) | ``bool(x) is False`` | | 789 | <TestCase.assertFalse>` | | | 790 +-----------------------------------------+-----------------------------+---------------+ 791 | :meth:`assertIs(a, b) | ``a is b`` | 3.1 | 792 | <TestCase.assertIs>` | | | 793 +-----------------------------------------+-----------------------------+---------------+ 794 | :meth:`assertIsNot(a, b) | ``a is not b`` | 3.1 | 795 | <TestCase.assertIsNot>` | | | 796 +-----------------------------------------+-----------------------------+---------------+ 797 | :meth:`assertIsNone(x) | ``x is None`` | 3.1 | 798 | <TestCase.assertIsNone>` | | | 799 +-----------------------------------------+-----------------------------+---------------+ 800 | :meth:`assertIsNotNone(x) | ``x is not None`` | 3.1 | 801 | <TestCase.assertIsNotNone>` | | | 802 +-----------------------------------------+-----------------------------+---------------+ 803 | :meth:`assertIn(a, b) | ``a in b`` | 3.1 | 804 | <TestCase.assertIn>` | | | 805 +-----------------------------------------+-----------------------------+---------------+ 806 | :meth:`assertNotIn(a, b) | ``a not in b`` | 3.1 | 807 | <TestCase.assertNotIn>` | | | 808 +-----------------------------------------+-----------------------------+---------------+ 809 | :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 3.2 | 810 | <TestCase.assertIsInstance>` | | | 811 +-----------------------------------------+-----------------------------+---------------+ 812 | :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 3.2 | 813 | <TestCase.assertNotIsInstance>` | | | 814 +-----------------------------------------+-----------------------------+---------------+ 815 816 All the assert methods accept a *msg* argument that, if specified, is used 817 as the error message on failure (see also :data:`longMessage`). 818 Note that the *msg* keyword argument can be passed to :meth:`assertRaises`, 819 :meth:`assertRaisesRegex`, :meth:`assertWarns`, :meth:`assertWarnsRegex` 820 only when they are used as a context manager. 821 822 .. method:: assertEqual(first, second, msg=None) 823 824 Test that *first* and *second* are equal. If the values do not 825 compare equal, the test will fail. 826 827 In addition, if *first* and *second* are the exact same type and one of 828 list, tuple, dict, set, frozenset or str or any type that a subclass 829 registers with :meth:`addTypeEqualityFunc` the type-specific equality 830 function will be called in order to generate a more useful default 831 error message (see also the :ref:`list of type-specific methods 832 <type-specific-methods>`). 833 834 .. versionchanged:: 3.1 835 Added the automatic calling of type-specific equality function. 836 837 .. versionchanged:: 3.2 838 :meth:`assertMultiLineEqual` added as the default type equality 839 function for comparing strings. 840 841 842 .. method:: assertNotEqual(first, second, msg=None) 843 844 Test that *first* and *second* are not equal. If the values do 845 compare equal, the test will fail. 846 847 .. method:: assertTrue(expr, msg=None) 848 assertFalse(expr, msg=None) 849 850 Test that *expr* is true (or false). 851 852 Note that this is equivalent to ``bool(expr) is True`` and not to ``expr 853 is True`` (use ``assertIs(expr, True)`` for the latter). This method 854 should also be avoided when more specific methods are available (e.g. 855 ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they 856 provide a better error message in case of failure. 857 858 859 .. method:: assertIs(first, second, msg=None) 860 assertIsNot(first, second, msg=None) 861 862 Test that *first* and *second* evaluate (or don't evaluate) to the 863 same object. 864 865 .. versionadded:: 3.1 866 867 868 .. method:: assertIsNone(expr, msg=None) 869 assertIsNotNone(expr, msg=None) 870 871 Test that *expr* is (or is not) ``None``. 872 873 .. versionadded:: 3.1 874 875 876 .. method:: assertIn(first, second, msg=None) 877 assertNotIn(first, second, msg=None) 878 879 Test that *first* is (or is not) in *second*. 880 881 .. versionadded:: 3.1 882 883 884 .. method:: assertIsInstance(obj, cls, msg=None) 885 assertNotIsInstance(obj, cls, msg=None) 886 887 Test that *obj* is (or is not) an instance of *cls* (which can be a 888 class or a tuple of classes, as supported by :func:`isinstance`). 889 To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`. 890 891 .. versionadded:: 3.2 892 893 894 895 It is also possible to check the production of exceptions, warnings, and 896 log messages using the following methods: 897 898 +---------------------------------------------------------+--------------------------------------+------------+ 899 | Method | Checks that | New in | 900 +=========================================================+======================================+============+ 901 | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | | 902 | <TestCase.assertRaises>` | | | 903 +---------------------------------------------------------+--------------------------------------+------------+ 904 | :meth:`assertRaisesRegex(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 3.1 | 905 | <TestCase.assertRaisesRegex>` | and the message matches regex *r* | | 906 +---------------------------------------------------------+--------------------------------------+------------+ 907 | :meth:`assertWarns(warn, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | 908 | <TestCase.assertWarns>` | | | 909 +---------------------------------------------------------+--------------------------------------+------------+ 910 | :meth:`assertWarnsRegex(warn, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | 911 | <TestCase.assertWarnsRegex>` | and the message matches regex *r* | | 912 +---------------------------------------------------------+--------------------------------------+------------+ 913 | :meth:`assertLogs(logger, level) | The ``with`` block logs on *logger* | 3.4 | 914 | <TestCase.assertLogs>` | with minimum *level* | | 915 +---------------------------------------------------------+--------------------------------------+------------+ 916 917 .. method:: assertRaises(exception, callable, *args, **kwds) 918 assertRaises(exception, msg=None) 919 920 Test that an exception is raised when *callable* is called with any 921 positional or keyword arguments that are also passed to 922 :meth:`assertRaises`. The test passes if *exception* is raised, is an 923 error if another exception is raised, or fails if no exception is raised. 924 To catch any of a group of exceptions, a tuple containing the exception 925 classes may be passed as *exception*. 926 927 If only the *exception* and possibly the *msg* arguments are given, 928 return a context manager so that the code under test can be written 929 inline rather than as a function:: 930 931 with self.assertRaises(SomeException): 932 do_something() 933 934 When used as a context manager, :meth:`assertRaises` accepts the 935 additional keyword argument *msg*. 936 937 The context manager will store the caught exception object in its 938 :attr:`exception` attribute. This can be useful if the intention 939 is to perform additional checks on the exception raised:: 940 941 with self.assertRaises(SomeException) as cm: 942 do_something() 943 944 the_exception = cm.exception 945 self.assertEqual(the_exception.error_code, 3) 946 947 .. versionchanged:: 3.1 948 Added the ability to use :meth:`assertRaises` as a context manager. 949 950 .. versionchanged:: 3.2 951 Added the :attr:`exception` attribute. 952 953 .. versionchanged:: 3.3 954 Added the *msg* keyword argument when used as a context manager. 955 956 957 .. method:: assertRaisesRegex(exception, regex, callable, *args, **kwds) 958 assertRaisesRegex(exception, regex, msg=None) 959 960 Like :meth:`assertRaises` but also tests that *regex* matches 961 on the string representation of the raised exception. *regex* may be 962 a regular expression object or a string containing a regular expression 963 suitable for use by :func:`re.search`. Examples:: 964 965 self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", 966 int, 'XYZ') 967 968 or:: 969 970 with self.assertRaisesRegex(ValueError, 'literal'): 971 int('XYZ') 972 973 .. versionadded:: 3.1 974 under the name ``assertRaisesRegexp``. 975 976 .. versionchanged:: 3.2 977 Renamed to :meth:`assertRaisesRegex`. 978 979 .. versionchanged:: 3.3 980 Added the *msg* keyword argument when used as a context manager. 981 982 983 .. method:: assertWarns(warning, callable, *args, **kwds) 984 assertWarns(warning, msg=None) 985 986 Test that a warning is triggered when *callable* is called with any 987 positional or keyword arguments that are also passed to 988 :meth:`assertWarns`. The test passes if *warning* is triggered and 989 fails if it isn't. Any exception is an error. 990 To catch any of a group of warnings, a tuple containing the warning 991 classes may be passed as *warnings*. 992 993 If only the *warning* and possibly the *msg* arguments are given, 994 return a context manager so that the code under test can be written 995 inline rather than as a function:: 996 997 with self.assertWarns(SomeWarning): 998 do_something() 999 1000 When used as a context manager, :meth:`assertWarns` accepts the 1001 additional keyword argument *msg*. 1002 1003 The context manager will store the caught warning object in its 1004 :attr:`warning` attribute, and the source line which triggered the 1005 warnings in the :attr:`filename` and :attr:`lineno` attributes. 1006 This can be useful if the intention is to perform additional checks 1007 on the warning caught:: 1008 1009 with self.assertWarns(SomeWarning) as cm: 1010 do_something() 1011 1012 self.assertIn('myfile.py', cm.filename) 1013 self.assertEqual(320, cm.lineno) 1014 1015 This method works regardless of the warning filters in place when it 1016 is called. 1017 1018 .. versionadded:: 3.2 1019 1020 .. versionchanged:: 3.3 1021 Added the *msg* keyword argument when used as a context manager. 1022 1023 1024 .. method:: assertWarnsRegex(warning, regex, callable, *args, **kwds) 1025 assertWarnsRegex(warning, regex, msg=None) 1026 1027 Like :meth:`assertWarns` but also tests that *regex* matches on the 1028 message of the triggered warning. *regex* may be a regular expression 1029 object or a string containing a regular expression suitable for use 1030 by :func:`re.search`. Example:: 1031 1032 self.assertWarnsRegex(DeprecationWarning, 1033 r'legacy_function\(\) is deprecated', 1034 legacy_function, 'XYZ') 1035 1036 or:: 1037 1038 with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): 1039 frobnicate('/etc/passwd') 1040 1041 .. versionadded:: 3.2 1042 1043 .. versionchanged:: 3.3 1044 Added the *msg* keyword argument when used as a context manager. 1045 1046 .. method:: assertLogs(logger=None, level=None) 1047 1048 A context manager to test that at least one message is logged on 1049 the *logger* or one of its children, with at least the given 1050 *level*. 1051 1052 If given, *logger* should be a :class:`logging.Logger` object or a 1053 :class:`str` giving the name of a logger. The default is the root 1054 logger, which will catch all messages. 1055 1056 If given, *level* should be either a numeric logging level or 1057 its string equivalent (for example either ``"ERROR"`` or 1058 :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. 1059 1060 The test passes if at least one message emitted inside the ``with`` 1061 block matches the *logger* and *level* conditions, otherwise it fails. 1062 1063 The object returned by the context manager is a recording helper 1064 which keeps tracks of the matching log messages. It has two 1065 attributes: 1066 1067 .. attribute:: records 1068 1069 A list of :class:`logging.LogRecord` objects of the matching 1070 log messages. 1071 1072 .. attribute:: output 1073 1074 A list of :class:`str` objects with the formatted output of 1075 matching messages. 1076 1077 Example:: 1078 1079 with self.assertLogs('foo', level='INFO') as cm: 1080 logging.getLogger('foo').info('first message') 1081 logging.getLogger('foo.bar').error('second message') 1082 self.assertEqual(cm.output, ['INFO:foo:first message', 1083 'ERROR:foo.bar:second message']) 1084 1085 .. versionadded:: 3.4 1086 1087 1088 There are also other methods used to perform more specific checks, such as: 1089 1090 +---------------------------------------+--------------------------------+--------------+ 1091 | Method | Checks that | New in | 1092 +=======================================+================================+==============+ 1093 | :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | | 1094 | <TestCase.assertAlmostEqual>` | | | 1095 +---------------------------------------+--------------------------------+--------------+ 1096 | :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | | 1097 | <TestCase.assertNotAlmostEqual>` | | | 1098 +---------------------------------------+--------------------------------+--------------+ 1099 | :meth:`assertGreater(a, b) | ``a > b`` | 3.1 | 1100 | <TestCase.assertGreater>` | | | 1101 +---------------------------------------+--------------------------------+--------------+ 1102 | :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 3.1 | 1103 | <TestCase.assertGreaterEqual>` | | | 1104 +---------------------------------------+--------------------------------+--------------+ 1105 | :meth:`assertLess(a, b) | ``a < b`` | 3.1 | 1106 | <TestCase.assertLess>` | | | 1107 +---------------------------------------+--------------------------------+--------------+ 1108 | :meth:`assertLessEqual(a, b) | ``a <= b`` | 3.1 | 1109 | <TestCase.assertLessEqual>` | | | 1110 +---------------------------------------+--------------------------------+--------------+ 1111 | :meth:`assertRegex(s, r) | ``r.search(s)`` | 3.1 | 1112 | <TestCase.assertRegex>` | | | 1113 +---------------------------------------+--------------------------------+--------------+ 1114 | :meth:`assertNotRegex(s, r) | ``not r.search(s)`` | 3.2 | 1115 | <TestCase.assertNotRegex>` | | | 1116 +---------------------------------------+--------------------------------+--------------+ 1117 | :meth:`assertCountEqual(a, b) | *a* and *b* have the same | 3.2 | 1118 | <TestCase.assertCountEqual>` | elements in the same number, | | 1119 | | regardless of their order | | 1120 +---------------------------------------+--------------------------------+--------------+ 1121 1122 1123 .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None) 1124 assertNotAlmostEqual(first, second, places=7, msg=None, delta=None) 1125 1126 Test that *first* and *second* are approximately (or not approximately) 1127 equal by computing the difference, rounding to the given number of 1128 decimal *places* (default 7), and comparing to zero. Note that these 1129 methods round the values to the given number of *decimal places* (i.e. 1130 like the :func:`round` function) and not *significant digits*. 1131 1132 If *delta* is supplied instead of *places* then the difference 1133 between *first* and *second* must be less or equal to (or greater than) *delta*. 1134 1135 Supplying both *delta* and *places* raises a ``TypeError``. 1136 1137 .. versionchanged:: 3.2 1138 :meth:`assertAlmostEqual` automatically considers almost equal objects 1139 that compare equal. :meth:`assertNotAlmostEqual` automatically fails 1140 if the objects compare equal. Added the *delta* keyword argument. 1141 1142 1143 .. method:: assertGreater(first, second, msg=None) 1144 assertGreaterEqual(first, second, msg=None) 1145 assertLess(first, second, msg=None) 1146 assertLessEqual(first, second, msg=None) 1147 1148 Test that *first* is respectively >, >=, < or <= than *second* depending 1149 on the method name. If not, the test will fail:: 1150 1151 >>> self.assertGreaterEqual(3, 4) 1152 AssertionError: "3" unexpectedly not greater than or equal to "4" 1153 1154 .. versionadded:: 3.1 1155 1156 1157 .. method:: assertRegex(text, regex, msg=None) 1158 assertNotRegex(text, regex, msg=None) 1159 1160 Test that a *regex* search matches (or does not match) *text*. In case 1161 of failure, the error message will include the pattern and the *text* (or 1162 the pattern and the part of *text* that unexpectedly matched). *regex* 1163 may be a regular expression object or a string containing a regular 1164 expression suitable for use by :func:`re.search`. 1165 1166 .. versionadded:: 3.1 1167 under the name ``assertRegexpMatches``. 1168 .. versionchanged:: 3.2 1169 The method ``assertRegexpMatches()`` has been renamed to 1170 :meth:`.assertRegex`. 1171 .. versionadded:: 3.2 1172 :meth:`.assertNotRegex`. 1173 1174 1175 .. method:: assertCountEqual(first, second, msg=None) 1176 1177 Test that sequence *first* contains the same elements as *second*, 1178 regardless of their order. When they don't, an error message listing the 1179 differences between the sequences will be generated. 1180 1181 Duplicate elements are *not* ignored when comparing *first* and 1182 *second*. It verifies whether each element has the same count in both 1183 sequences. Equivalent to: 1184 ``assertEqual(Counter(list(first)), Counter(list(second)))`` 1185 but works with sequences of unhashable objects as well. 1186 1187 .. versionadded:: 3.2 1188 1189 1190 .. _type-specific-methods: 1191 1192 The :meth:`assertEqual` method dispatches the equality check for objects of 1193 the same type to different type-specific methods. These methods are already 1194 implemented for most of the built-in types, but it's also possible to 1195 register new methods using :meth:`addTypeEqualityFunc`: 1196 1197 .. method:: addTypeEqualityFunc(typeobj, function) 1198 1199 Registers a type-specific method called by :meth:`assertEqual` to check 1200 if two objects of exactly the same *typeobj* (not subclasses) compare 1201 equal. *function* must take two positional arguments and a third msg=None 1202 keyword argument just as :meth:`assertEqual` does. It must raise 1203 :data:`self.failureException(msg) <failureException>` when inequality 1204 between the first two parameters is detected -- possibly providing useful 1205 information and explaining the inequalities in details in the error 1206 message. 1207 1208 .. versionadded:: 3.1 1209 1210 The list of type-specific methods automatically used by 1211 :meth:`~TestCase.assertEqual` are summarized in the following table. Note 1212 that it's usually not necessary to invoke these methods directly. 1213 1214 +-----------------------------------------+-----------------------------+--------------+ 1215 | Method | Used to compare | New in | 1216 +=========================================+=============================+==============+ 1217 | :meth:`assertMultiLineEqual(a, b) | strings | 3.1 | 1218 | <TestCase.assertMultiLineEqual>` | | | 1219 +-----------------------------------------+-----------------------------+--------------+ 1220 | :meth:`assertSequenceEqual(a, b) | sequences | 3.1 | 1221 | <TestCase.assertSequenceEqual>` | | | 1222 +-----------------------------------------+-----------------------------+--------------+ 1223 | :meth:`assertListEqual(a, b) | lists | 3.1 | 1224 | <TestCase.assertListEqual>` | | | 1225 +-----------------------------------------+-----------------------------+--------------+ 1226 | :meth:`assertTupleEqual(a, b) | tuples | 3.1 | 1227 | <TestCase.assertTupleEqual>` | | | 1228 +-----------------------------------------+-----------------------------+--------------+ 1229 | :meth:`assertSetEqual(a, b) | sets or frozensets | 3.1 | 1230 | <TestCase.assertSetEqual>` | | | 1231 +-----------------------------------------+-----------------------------+--------------+ 1232 | :meth:`assertDictEqual(a, b) | dicts | 3.1 | 1233 | <TestCase.assertDictEqual>` | | | 1234 +-----------------------------------------+-----------------------------+--------------+ 1235 1236 1237 1238 .. method:: assertMultiLineEqual(first, second, msg=None) 1239 1240 Test that the multiline string *first* is equal to the string *second*. 1241 When not equal a diff of the two strings highlighting the differences 1242 will be included in the error message. This method is used by default 1243 when comparing strings with :meth:`assertEqual`. 1244 1245 .. versionadded:: 3.1 1246 1247 1248 .. method:: assertSequenceEqual(first, second, msg=None, seq_type=None) 1249 1250 Tests that two sequences are equal. If a *seq_type* is supplied, both 1251 *first* and *second* must be instances of *seq_type* or a failure will 1252 be raised. If the sequences are different an error message is 1253 constructed that shows the difference between the two. 1254 1255 This method is not called directly by :meth:`assertEqual`, but 1256 it's used to implement :meth:`assertListEqual` and 1257 :meth:`assertTupleEqual`. 1258 1259 .. versionadded:: 3.1 1260 1261 1262 .. method:: assertListEqual(first, second, msg=None) 1263 assertTupleEqual(first, second, msg=None) 1264 1265 Tests that two lists or tuples are equal. If not, an error message is 1266 constructed that shows only the differences between the two. An error 1267 is also raised if either of the parameters are of the wrong type. 1268 These methods are used by default when comparing lists or tuples with 1269 :meth:`assertEqual`. 1270 1271 .. versionadded:: 3.1 1272 1273 1274 .. method:: assertSetEqual(first, second, msg=None) 1275 1276 Tests that two sets are equal. If not, an error message is constructed 1277 that lists the differences between the sets. This method is used by 1278 default when comparing sets or frozensets with :meth:`assertEqual`. 1279 1280 Fails if either of *first* or *second* does not have a :meth:`set.difference` 1281 method. 1282 1283 .. versionadded:: 3.1 1284 1285 1286 .. method:: assertDictEqual(first, second, msg=None) 1287 1288 Test that two dictionaries are equal. If not, an error message is 1289 constructed that shows the differences in the dictionaries. This 1290 method will be used by default to compare dictionaries in 1291 calls to :meth:`assertEqual`. 1292 1293 .. versionadded:: 3.1 1294 1295 1296 1297 .. _other-methods-and-attrs: 1298 1299 Finally the :class:`TestCase` provides the following methods and attributes: 1300 1301 1302 .. method:: fail(msg=None) 1303 1304 Signals a test failure unconditionally, with *msg* or ``None`` for 1305 the error message. 1306 1307 1308 .. attribute:: failureException 1309 1310 This class attribute gives the exception raised by the test method. If a 1311 test framework needs to use a specialized exception, possibly to carry 1312 additional information, it must subclass this exception in order to "play 1313 fair" with the framework. The initial value of this attribute is 1314 :exc:`AssertionError`. 1315 1316 1317 .. attribute:: longMessage 1318 1319 This class attribute determines what happens when a custom failure message 1320 is passed as the msg argument to an assertXYY call that fails. 1321 ``True`` is the default value. In this case, the custom message is appended 1322 to the end of the standard failure message. 1323 When set to ``False``, the custom message replaces the standard message. 1324 1325 The class setting can be overridden in individual test methods by assigning 1326 an instance attribute, self.longMessage, to ``True`` or ``False`` before 1327 calling the assert methods. 1328 1329 The class setting gets reset before each test call. 1330 1331 .. versionadded:: 3.1 1332 1333 1334 .. attribute:: maxDiff 1335 1336 This attribute controls the maximum length of diffs output by assert 1337 methods that report diffs on failure. It defaults to 80*8 characters. 1338 Assert methods affected by this attribute are 1339 :meth:`assertSequenceEqual` (including all the sequence comparison 1340 methods that delegate to it), :meth:`assertDictEqual` and 1341 :meth:`assertMultiLineEqual`. 1342 1343 Setting ``maxDiff`` to ``None`` means that there is no maximum length of 1344 diffs. 1345 1346 .. versionadded:: 3.2 1347 1348 1349 Testing frameworks can use the following methods to collect information on 1350 the test: 1351 1352 1353 .. method:: countTestCases() 1354 1355 Return the number of tests represented by this test object. For 1356 :class:`TestCase` instances, this will always be ``1``. 1357 1358 1359 .. method:: defaultTestResult() 1360 1361 Return an instance of the test result class that should be used for this 1362 test case class (if no other result instance is provided to the 1363 :meth:`run` method). 1364 1365 For :class:`TestCase` instances, this will always be an instance of 1366 :class:`TestResult`; subclasses of :class:`TestCase` should override this 1367 as necessary. 1368 1369 1370 .. method:: id() 1371 1372 Return a string identifying the specific test case. This is usually the 1373 full name of the test method, including the module and class name. 1374 1375 1376 .. method:: shortDescription() 1377 1378 Returns a description of the test, or ``None`` if no description 1379 has been provided. The default implementation of this method 1380 returns the first line of the test method's docstring, if available, 1381 or ``None``. 1382 1383 .. versionchanged:: 3.1 1384 In 3.1 this was changed to add the test name to the short description 1385 even in the presence of a docstring. This caused compatibility issues 1386 with unittest extensions and adding the test name was moved to the 1387 :class:`TextTestResult` in Python 3.2. 1388 1389 1390 .. method:: addCleanup(function, *args, **kwargs) 1391 1392 Add a function to be called after :meth:`tearDown` to cleanup resources 1393 used during the test. Functions will be called in reverse order to the 1394 order they are added (:abbr:`LIFO (last-in, first-out)`). They 1395 are called with any arguments and keyword arguments passed into 1396 :meth:`addCleanup` when they are added. 1397 1398 If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called, 1399 then any cleanup functions added will still be called. 1400 1401 .. versionadded:: 3.1 1402 1403 1404 .. method:: doCleanups() 1405 1406 This method is called unconditionally after :meth:`tearDown`, or 1407 after :meth:`setUp` if :meth:`setUp` raises an exception. 1408 1409 It is responsible for calling all the cleanup functions added by 1410 :meth:`addCleanup`. If you need cleanup functions to be called 1411 *prior* to :meth:`tearDown` then you can call :meth:`doCleanups` 1412 yourself. 1413 1414 :meth:`doCleanups` pops methods off the stack of cleanup 1415 functions one at a time, so it can be called at any time. 1416 1417 .. versionadded:: 3.1 1418 1419 1420 .. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None) 1421 1422 This class implements the portion of the :class:`TestCase` interface which 1423 allows the test runner to drive the test, but does not provide the methods 1424 which test code can use to check and report errors. This is used to create 1425 test cases using legacy test code, allowing it to be integrated into a 1426 :mod:`unittest`-based test framework. 1427 1428 1429 .. _deprecated-aliases: 1430 1431 Deprecated aliases 1432 ################## 1433 1434 For historical reasons, some of the :class:`TestCase` methods had one or more 1435 aliases that are now deprecated. The following table lists the correct names 1436 along with their deprecated aliases: 1437 1438 ============================== ====================== ====================== 1439 Method Name Deprecated alias Deprecated alias 1440 ============================== ====================== ====================== 1441 :meth:`.assertEqual` failUnlessEqual assertEquals 1442 :meth:`.assertNotEqual` failIfEqual assertNotEquals 1443 :meth:`.assertTrue` failUnless assert\_ 1444 :meth:`.assertFalse` failIf 1445 :meth:`.assertRaises` failUnlessRaises 1446 :meth:`.assertAlmostEqual` failUnlessAlmostEqual assertAlmostEquals 1447 :meth:`.assertNotAlmostEqual` failIfAlmostEqual assertNotAlmostEquals 1448 :meth:`.assertRegex` assertRegexpMatches 1449 :meth:`.assertRaisesRegex` assertRaisesRegexp 1450 ============================== ====================== ====================== 1451 1452 .. deprecated:: 3.1 1453 the fail* aliases listed in the second column. 1454 .. deprecated:: 3.2 1455 the assert* aliases listed in the third column. 1456 .. deprecated:: 3.2 1457 ``assertRegexpMatches`` and ``assertRaisesRegexp`` have been renamed to 1458 :meth:`.assertRegex` and :meth:`.assertRaisesRegex` 1459 1460 1461 .. _testsuite-objects: 1462 1463 Grouping tests 1464 ~~~~~~~~~~~~~~ 1465 1466 .. class:: TestSuite(tests=()) 1467 1468 This class represents an aggregation of individual test cases and test suites. 1469 The class presents the interface needed by the test runner to allow it to be run 1470 as any other test case. Running a :class:`TestSuite` instance is the same as 1471 iterating over the suite, running each test individually. 1472 1473 If *tests* is given, it must be an iterable of individual test cases or other 1474 test suites that will be used to build the suite initially. Additional methods 1475 are provided to add test cases and suites to the collection later on. 1476 1477 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except 1478 they do not actually implement a test. Instead, they are used to aggregate 1479 tests into groups of tests that should be run together. Some additional 1480 methods are available to add tests to :class:`TestSuite` instances: 1481 1482 1483 .. method:: TestSuite.addTest(test) 1484 1485 Add a :class:`TestCase` or :class:`TestSuite` to the suite. 1486 1487 1488 .. method:: TestSuite.addTests(tests) 1489 1490 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` 1491 instances to this test suite. 1492 1493 This is equivalent to iterating over *tests*, calling :meth:`addTest` for 1494 each element. 1495 1496 :class:`TestSuite` shares the following methods with :class:`TestCase`: 1497 1498 1499 .. method:: run(result) 1500 1501 Run the tests associated with this suite, collecting the result into the 1502 test result object passed as *result*. Note that unlike 1503 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to 1504 be passed in. 1505 1506 1507 .. method:: debug() 1508 1509 Run the tests associated with this suite without collecting the 1510 result. This allows exceptions raised by the test to be propagated to the 1511 caller and can be used to support running tests under a debugger. 1512 1513 1514 .. method:: countTestCases() 1515 1516 Return the number of tests represented by this test object, including all 1517 individual tests and sub-suites. 1518 1519 1520 .. method:: __iter__() 1521 1522 Tests grouped by a :class:`TestSuite` are always accessed by iteration. 1523 Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note 1524 that this method may be called several times on a single suite (for 1525 example when counting tests or comparing for equality) so the tests 1526 returned by repeated iterations before :meth:`TestSuite.run` must be the 1527 same for each call iteration. After :meth:`TestSuite.run`, callers should 1528 not rely on the tests returned by this method unless the caller uses a 1529 subclass that overrides :meth:`TestSuite._removeTestAtIndex` to preserve 1530 test references. 1531 1532 .. versionchanged:: 3.2 1533 In earlier versions the :class:`TestSuite` accessed tests directly rather 1534 than through iteration, so overriding :meth:`__iter__` wasn't sufficient 1535 for providing tests. 1536 1537 .. versionchanged:: 3.4 1538 In earlier versions the :class:`TestSuite` held references to each 1539 :class:`TestCase` after :meth:`TestSuite.run`. Subclasses can restore 1540 that behavior by overriding :meth:`TestSuite._removeTestAtIndex`. 1541 1542 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method 1543 is invoked by a :class:`TestRunner` rather than by the end-user test harness. 1544 1545 1546 Loading and running tests 1547 ~~~~~~~~~~~~~~~~~~~~~~~~~ 1548 1549 .. class:: TestLoader() 1550 1551 The :class:`TestLoader` class is used to create test suites from classes and 1552 modules. Normally, there is no need to create an instance of this class; the 1553 :mod:`unittest` module provides an instance that can be shared as 1554 :data:`unittest.defaultTestLoader`. Using a subclass or instance, however, 1555 allows customization of some configurable properties. 1556 1557 :class:`TestLoader` objects have the following attributes: 1558 1559 1560 .. attribute:: errors 1561 1562 A list of the non-fatal errors encountered while loading tests. Not reset 1563 by the loader at any point. Fatal errors are signalled by the relevant 1564 a method raising an exception to the caller. Non-fatal errors are also 1565 indicated by a synthetic test that will raise the original error when 1566 run. 1567 1568 .. versionadded:: 3.5 1569 1570 1571 :class:`TestLoader` objects have the following methods: 1572 1573 1574 .. method:: loadTestsFromTestCase(testCaseClass) 1575 1576 Return a suite of all test cases contained in the :class:`TestCase`\ -derived 1577 :class:`testCaseClass`. 1578 1579 A test case instance is created for each method named by 1580 :meth:`getTestCaseNames`. By default these are the method names 1581 beginning with ``test``. If :meth:`getTestCaseNames` returns no 1582 methods, but the :meth:`runTest` method is implemented, a single test 1583 case is created for that method instead. 1584 1585 1586 .. method:: loadTestsFromModule(module, pattern=None) 1587 1588 Return a suite of all test cases contained in the given module. This 1589 method searches *module* for classes derived from :class:`TestCase` and 1590 creates an instance of the class for each test method defined for the 1591 class. 1592 1593 .. note:: 1594 1595 While using a hierarchy of :class:`TestCase`\ -derived classes can be 1596 convenient in sharing fixtures and helper functions, defining test 1597 methods on base classes that are not intended to be instantiated 1598 directly does not play well with this method. Doing so, however, can 1599 be useful when the fixtures are different and defined in subclasses. 1600 1601 If a module provides a ``load_tests`` function it will be called to 1602 load the tests. This allows modules to customize test loading. 1603 This is the `load_tests protocol`_. The *pattern* argument is passed as 1604 the third argument to ``load_tests``. 1605 1606 .. versionchanged:: 3.2 1607 Support for ``load_tests`` added. 1608 1609 .. versionchanged:: 3.5 1610 The undocumented and unofficial *use_load_tests* default argument is 1611 deprecated and ignored, although it is still accepted for backward 1612 compatibility. The method also now accepts a keyword-only argument 1613 *pattern* which is passed to ``load_tests`` as the third argument. 1614 1615 1616 .. method:: loadTestsFromName(name, module=None) 1617 1618 Return a suite of all test cases given a string specifier. 1619 1620 The specifier *name* is a "dotted name" that may resolve either to a 1621 module, a test case class, a test method within a test case class, a 1622 :class:`TestSuite` instance, or a callable object which returns a 1623 :class:`TestCase` or :class:`TestSuite` instance. These checks are 1624 applied in the order listed here; that is, a method on a possible test 1625 case class will be picked up as "a test method within a test case class", 1626 rather than "a callable object". 1627 1628 For example, if you have a module :mod:`SampleTests` containing a 1629 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test 1630 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the 1631 specifier ``'SampleTests.SampleTestCase'`` would cause this method to 1632 return a suite which will run all three test methods. Using the specifier 1633 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test 1634 suite which will run only the :meth:`test_two` test method. The specifier 1635 can refer to modules and packages which have not been imported; they will 1636 be imported as a side-effect. 1637 1638 The method optionally resolves *name* relative to the given *module*. 1639 1640 .. versionchanged:: 3.5 1641 If an :exc:`ImportError` or :exc:`AttributeError` occurs while traversing 1642 *name* then a synthetic test that raises that error when run will be 1643 returned. These errors are included in the errors accumulated by 1644 self.errors. 1645 1646 1647 .. method:: loadTestsFromNames(names, module=None) 1648 1649 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather 1650 than a single name. The return value is a test suite which supports all 1651 the tests defined for each name. 1652 1653 1654 .. method:: getTestCaseNames(testCaseClass) 1655 1656 Return a sorted sequence of method names found within *testCaseClass*; 1657 this should be a subclass of :class:`TestCase`. 1658 1659 1660 .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) 1661 1662 Find all the test modules by recursing into subdirectories from the 1663 specified start directory, and return a TestSuite object containing them. 1664 Only test files that match *pattern* will be loaded. (Using shell style 1665 pattern matching.) Only module names that are importable (i.e. are valid 1666 Python identifiers) will be loaded. 1667 1668 All test modules must be importable from the top level of the project. If 1669 the start directory is not the top level directory then the top level 1670 directory must be specified separately. 1671 1672 If importing a module fails, for example due to a syntax error, then 1673 this will be recorded as a single error and discovery will continue. If 1674 the import failure is due to :exc:`SkipTest` being raised, it will be 1675 recorded as a skip instead of an error. 1676 1677 If a package (a directory containing a file named :file:`__init__.py`) is 1678 found, the package will be checked for a ``load_tests`` function. If this 1679 exists then it will be called 1680 ``package.load_tests(loader, tests, pattern)``. Test discovery takes care 1681 to ensure that a package is only checked for tests once during an 1682 invocation, even if the load_tests function itself calls 1683 ``loader.discover``. 1684 1685 If ``load_tests`` exists then discovery does *not* recurse into the 1686 package, ``load_tests`` is responsible for loading all tests in the 1687 package. 1688 1689 The pattern is deliberately not stored as a loader attribute so that 1690 packages can continue discovery themselves. *top_level_dir* is stored so 1691 ``load_tests`` does not need to pass this argument in to 1692 ``loader.discover()``. 1693 1694 *start_dir* can be a dotted module name as well as a directory. 1695 1696 .. versionadded:: 3.2 1697 1698 .. versionchanged:: 3.4 1699 Modules that raise :exc:`SkipTest` on import are recorded as skips, 1700 not errors. 1701 Discovery works for :term:`namespace packages <namespace package>`. 1702 Paths are sorted before being imported so that execution order is 1703 the same even if the underlying file system's ordering is not 1704 dependent on file name. 1705 1706 .. versionchanged:: 3.5 1707 Found packages are now checked for ``load_tests`` regardless of 1708 whether their path matches *pattern*, because it is impossible for 1709 a package name to match the default pattern. 1710 1711 1712 The following attributes of a :class:`TestLoader` can be configured either by 1713 subclassing or assignment on an instance: 1714 1715 1716 .. attribute:: testMethodPrefix 1717 1718 String giving the prefix of method names which will be interpreted as test 1719 methods. The default value is ``'test'``. 1720 1721 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` 1722 methods. 1723 1724 1725 .. attribute:: sortTestMethodsUsing 1726 1727 Function to be used to compare method names when sorting them in 1728 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. 1729 1730 1731 .. attribute:: suiteClass 1732 1733 Callable object that constructs a test suite from a list of tests. No 1734 methods on the resulting object are needed. The default value is the 1735 :class:`TestSuite` class. 1736 1737 This affects all the :meth:`loadTestsFrom\*` methods. 1738 1739 1740 .. class:: TestResult 1741 1742 This class is used to compile information about which tests have succeeded 1743 and which have failed. 1744 1745 A :class:`TestResult` object stores the results of a set of tests. The 1746 :class:`TestCase` and :class:`TestSuite` classes ensure that results are 1747 properly recorded; test authors do not need to worry about recording the 1748 outcome of tests. 1749 1750 Testing frameworks built on top of :mod:`unittest` may want access to the 1751 :class:`TestResult` object generated by running a set of tests for reporting 1752 purposes; a :class:`TestResult` instance is returned by the 1753 :meth:`TestRunner.run` method for this purpose. 1754 1755 :class:`TestResult` instances have the following attributes that will be of 1756 interest when inspecting the results of running a set of tests: 1757 1758 1759 .. attribute:: errors 1760 1761 A list containing 2-tuples of :class:`TestCase` instances and strings 1762 holding formatted tracebacks. Each tuple represents a test which raised an 1763 unexpected exception. 1764 1765 .. attribute:: failures 1766 1767 A list containing 2-tuples of :class:`TestCase` instances and strings 1768 holding formatted tracebacks. Each tuple represents a test where a failure 1769 was explicitly signalled using the :meth:`TestCase.assert\*` methods. 1770 1771 .. attribute:: skipped 1772 1773 A list containing 2-tuples of :class:`TestCase` instances and strings 1774 holding the reason for skipping the test. 1775 1776 .. versionadded:: 3.1 1777 1778 .. attribute:: expectedFailures 1779 1780 A list containing 2-tuples of :class:`TestCase` instances and strings 1781 holding formatted tracebacks. Each tuple represents an expected failure 1782 of the test case. 1783 1784 .. attribute:: unexpectedSuccesses 1785 1786 A list containing :class:`TestCase` instances that were marked as expected 1787 failures, but succeeded. 1788 1789 .. attribute:: shouldStop 1790 1791 Set to ``True`` when the execution of tests should stop by :meth:`stop`. 1792 1793 .. attribute:: testsRun 1794 1795 The total number of tests run so far. 1796 1797 .. attribute:: buffer 1798 1799 If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between 1800 :meth:`startTest` and :meth:`stopTest` being called. Collected output will 1801 only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test 1802 fails or errors. Any output is also attached to the failure / error message. 1803 1804 .. versionadded:: 3.2 1805 1806 .. attribute:: failfast 1807 1808 If set to true :meth:`stop` will be called on the first failure or error, 1809 halting the test run. 1810 1811 .. versionadded:: 3.2 1812 1813 .. attribute:: tb_locals 1814 1815 If set to true then local variables will be shown in tracebacks. 1816 1817 .. versionadded:: 3.5 1818 1819 .. method:: wasSuccessful() 1820 1821 Return ``True`` if all tests run so far have passed, otherwise returns 1822 ``False``. 1823 1824 .. versionchanged:: 3.4 1825 Returns ``False`` if there were any :attr:`unexpectedSuccesses` 1826 from tests marked with the :func:`expectedFailure` decorator. 1827 1828 .. method:: stop() 1829 1830 This method can be called to signal that the set of tests being run should 1831 be aborted by setting the :attr:`shouldStop` attribute to ``True``. 1832 :class:`TestRunner` objects should respect this flag and return without 1833 running any additional tests. 1834 1835 For example, this feature is used by the :class:`TextTestRunner` class to 1836 stop the test framework when the user signals an interrupt from the 1837 keyboard. Interactive tools which provide :class:`TestRunner` 1838 implementations can use this in a similar manner. 1839 1840 The following methods of the :class:`TestResult` class are used to maintain 1841 the internal data structures, and may be extended in subclasses to support 1842 additional reporting requirements. This is particularly useful in building 1843 tools which support interactive reporting while tests are being run. 1844 1845 1846 .. method:: startTest(test) 1847 1848 Called when the test case *test* is about to be run. 1849 1850 .. method:: stopTest(test) 1851 1852 Called after the test case *test* has been executed, regardless of the 1853 outcome. 1854 1855 .. method:: startTestRun() 1856 1857 Called once before any tests are executed. 1858 1859 .. versionadded:: 3.1 1860 1861 1862 .. method:: stopTestRun() 1863 1864 Called once after all tests are executed. 1865 1866 .. versionadded:: 3.1 1867 1868 1869 .. method:: addError(test, err) 1870 1871 Called when the test case *test* raises an unexpected exception. *err* is a 1872 tuple of the form returned by :func:`sys.exc_info`: ``(type, value, 1873 traceback)``. 1874 1875 The default implementation appends a tuple ``(test, formatted_err)`` to 1876 the instance's :attr:`errors` attribute, where *formatted_err* is a 1877 formatted traceback derived from *err*. 1878 1879 1880 .. method:: addFailure(test, err) 1881 1882 Called when the test case *test* signals a failure. *err* is a tuple of 1883 the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. 1884 1885 The default implementation appends a tuple ``(test, formatted_err)`` to 1886 the instance's :attr:`failures` attribute, where *formatted_err* is a 1887 formatted traceback derived from *err*. 1888 1889 1890 .. method:: addSuccess(test) 1891 1892 Called when the test case *test* succeeds. 1893 1894 The default implementation does nothing. 1895 1896 1897 .. method:: addSkip(test, reason) 1898 1899 Called when the test case *test* is skipped. *reason* is the reason the 1900 test gave for skipping. 1901 1902 The default implementation appends a tuple ``(test, reason)`` to the 1903 instance's :attr:`skipped` attribute. 1904 1905 1906 .. method:: addExpectedFailure(test, err) 1907 1908 Called when the test case *test* fails, but was marked with the 1909 :func:`expectedFailure` decorator. 1910 1911 The default implementation appends a tuple ``(test, formatted_err)`` to 1912 the instance's :attr:`expectedFailures` attribute, where *formatted_err* 1913 is a formatted traceback derived from *err*. 1914 1915 1916 .. method:: addUnexpectedSuccess(test) 1917 1918 Called when the test case *test* was marked with the 1919 :func:`expectedFailure` decorator, but succeeded. 1920 1921 The default implementation appends the test to the instance's 1922 :attr:`unexpectedSuccesses` attribute. 1923 1924 1925 .. method:: addSubTest(test, subtest, outcome) 1926 1927 Called when a subtest finishes. *test* is the test case 1928 corresponding to the test method. *subtest* is a custom 1929 :class:`TestCase` instance describing the subtest. 1930 1931 If *outcome* is :const:`None`, the subtest succeeded. Otherwise, 1932 it failed with an exception where *outcome* is a tuple of the form 1933 returned by :func:`sys.exc_info`: ``(type, value, traceback)``. 1934 1935 The default implementation does nothing when the outcome is a 1936 success, and records subtest failures as normal failures. 1937 1938 .. versionadded:: 3.4 1939 1940 1941 .. class:: TextTestResult(stream, descriptions, verbosity) 1942 1943 A concrete implementation of :class:`TestResult` used by the 1944 :class:`TextTestRunner`. 1945 1946 .. versionadded:: 3.2 1947 This class was previously named ``_TextTestResult``. The old name still 1948 exists as an alias but is deprecated. 1949 1950 1951 .. data:: defaultTestLoader 1952 1953 Instance of the :class:`TestLoader` class intended to be shared. If no 1954 customization of the :class:`TestLoader` is needed, this instance can be used 1955 instead of repeatedly creating new instances. 1956 1957 1958 .. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \ 1959 buffer=False, resultclass=None, warnings=None, *, tb_locals=False) 1960 1961 A basic test runner implementation that outputs results to a stream. If *stream* 1962 is ``None``, the default, :data:`sys.stderr` is used as the output stream. This class 1963 has a few configurable parameters, but is essentially very simple. Graphical 1964 applications which run test suites should provide alternate implementations. Such 1965 implementations should accept ``**kwargs`` as the interface to construct runners 1966 changes when features are added to unittest. 1967 1968 By default this runner shows :exc:`DeprecationWarning`, 1969 :exc:`PendingDeprecationWarning`, :exc:`ResourceWarning` and 1970 :exc:`ImportWarning` even if they are :ref:`ignored by default 1971 <warning-ignored>`. Deprecation warnings caused by :ref:`deprecated unittest 1972 methods <deprecated-aliases>` are also special-cased and, when the warning 1973 filters are ``'default'`` or ``'always'``, they will appear only once 1974 per-module, in order to avoid too many warning messages. This behavior can 1975 be overridden using Python's :option:`!-Wd` or :option:`!-Wa` options 1976 (see :ref:`Warning control <using-on-warnings>`) and leaving 1977 *warnings* to ``None``. 1978 1979 .. versionchanged:: 3.2 1980 Added the ``warnings`` argument. 1981 1982 .. versionchanged:: 3.2 1983 The default stream is set to :data:`sys.stderr` at instantiation time rather 1984 than import time. 1985 1986 .. versionchanged:: 3.5 1987 Added the tb_locals parameter. 1988 1989 .. method:: _makeResult() 1990 1991 This method returns the instance of ``TestResult`` used by :meth:`run`. 1992 It is not intended to be called directly, but can be overridden in 1993 subclasses to provide a custom ``TestResult``. 1994 1995 ``_makeResult()`` instantiates the class or callable passed in the 1996 ``TextTestRunner`` constructor as the ``resultclass`` argument. It 1997 defaults to :class:`TextTestResult` if no ``resultclass`` is provided. 1998 The result class is instantiated with the following arguments:: 1999 2000 stream, descriptions, verbosity 2001 2002 .. method:: run(test) 2003 2004 This method is the main public interface to the `TextTestRunner`. This 2005 method takes a :class:`TestSuite` or :class:`TestCase` instance. A 2006 :class:`TestResult` is created by calling 2007 :func:`_makeResult` and the test(s) are run and the 2008 results printed to stdout. 2009 2010 2011 .. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, \ 2012 testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, \ 2013 failfast=None, catchbreak=None, buffer=None, warnings=None) 2014 2015 A command-line program that loads a set of tests from *module* and runs them; 2016 this is primarily for making test modules conveniently executable. 2017 The simplest use for this function is to include the following line at the 2018 end of a test script:: 2019 2020 if __name__ == '__main__': 2021 unittest.main() 2022 2023 You can run tests with more detailed information by passing in the verbosity 2024 argument:: 2025 2026 if __name__ == '__main__': 2027 unittest.main(verbosity=2) 2028 2029 The *defaultTest* argument is either the name of a single test or an 2030 iterable of test names to run if no test names are specified via *argv*. If 2031 not specified or ``None`` and no test names are provided via *argv*, all 2032 tests found in *module* are run. 2033 2034 The *argv* argument can be a list of options passed to the program, with the 2035 first element being the program name. If not specified or ``None``, 2036 the values of :data:`sys.argv` are used. 2037 2038 The *testRunner* argument can either be a test runner class or an already 2039 created instance of it. By default ``main`` calls :func:`sys.exit` with 2040 an exit code indicating success or failure of the tests run. 2041 2042 The *testLoader* argument has to be a :class:`TestLoader` instance, 2043 and defaults to :data:`defaultTestLoader`. 2044 2045 ``main`` supports being used from the interactive interpreter by passing in the 2046 argument ``exit=False``. This displays the result on standard output without 2047 calling :func:`sys.exit`:: 2048 2049 >>> from unittest import main 2050 >>> main(module='test_module', exit=False) 2051 2052 The *failfast*, *catchbreak* and *buffer* parameters have the same 2053 effect as the same-name `command-line options`_. 2054 2055 The *warnings* argument specifies the :ref:`warning filter <warning-filter>` 2056 that should be used while running the tests. If it's not specified, it will 2057 remain ``None`` if a :option:`!-W` option is passed to :program:`python` 2058 (see :ref:`Warning control <using-on-warnings>`), 2059 otherwise it will be set to ``'default'``. 2060 2061 Calling ``main`` actually returns an instance of the ``TestProgram`` class. 2062 This stores the result of the tests run as the ``result`` attribute. 2063 2064 .. versionchanged:: 3.1 2065 The *exit* parameter was added. 2066 2067 .. versionchanged:: 3.2 2068 The *verbosity*, *failfast*, *catchbreak*, *buffer* 2069 and *warnings* parameters were added. 2070 2071 .. versionchanged:: 3.4 2072 The *defaultTest* parameter was changed to also accept an iterable of 2073 test names. 2074 2075 2076 load_tests Protocol 2077 ################### 2078 2079 .. versionadded:: 3.2 2080 2081 Modules or packages can customize how tests are loaded from them during normal 2082 test runs or test discovery by implementing a function called ``load_tests``. 2083 2084 If a test module defines ``load_tests`` it will be called by 2085 :meth:`TestLoader.loadTestsFromModule` with the following arguments:: 2086 2087 load_tests(loader, standard_tests, pattern) 2088 2089 where *pattern* is passed straight through from ``loadTestsFromModule``. It 2090 defaults to ``None``. 2091 2092 It should return a :class:`TestSuite`. 2093 2094 *loader* is the instance of :class:`TestLoader` doing the loading. 2095 *standard_tests* are the tests that would be loaded by default from the 2096 module. It is common for test modules to only want to add or remove tests 2097 from the standard set of tests. 2098 The third argument is used when loading packages as part of test discovery. 2099 2100 A typical ``load_tests`` function that loads tests from a specific set of 2101 :class:`TestCase` classes may look like:: 2102 2103 test_cases = (TestCase1, TestCase2, TestCase3) 2104 2105 def load_tests(loader, tests, pattern): 2106 suite = TestSuite() 2107 for test_class in test_cases: 2108 tests = loader.loadTestsFromTestCase(test_class) 2109 suite.addTests(tests) 2110 return suite 2111 2112 If discovery is started in a directory containing a package, either from the 2113 command line or by calling :meth:`TestLoader.discover`, then the package 2114 :file:`__init__.py` will be checked for ``load_tests``. If that function does 2115 not exist, discovery will recurse into the package as though it were just 2116 another directory. Otherwise, discovery of the package's tests will be left up 2117 to ``load_tests`` which is called with the following arguments:: 2118 2119 load_tests(loader, standard_tests, pattern) 2120 2121 This should return a :class:`TestSuite` representing all the tests 2122 from the package. (``standard_tests`` will only contain tests 2123 collected from :file:`__init__.py`.) 2124 2125 Because the pattern is passed into ``load_tests`` the package is free to 2126 continue (and potentially modify) test discovery. A 'do nothing' 2127 ``load_tests`` function for a test package would look like:: 2128 2129 def load_tests(loader, standard_tests, pattern): 2130 # top level directory cached on loader instance 2131 this_dir = os.path.dirname(__file__) 2132 package_tests = loader.discover(start_dir=this_dir, pattern=pattern) 2133 standard_tests.addTests(package_tests) 2134 return standard_tests 2135 2136 .. versionchanged:: 3.5 2137 Discovery no longer checks package names for matching *pattern* due to the 2138 impossibility of package names matching the default pattern. 2139 2140 2141 2142 Class and Module Fixtures 2143 ------------------------- 2144 2145 Class and module level fixtures are implemented in :class:`TestSuite`. When 2146 the test suite encounters a test from a new class then :meth:`tearDownClass` 2147 from the previous class (if there is one) is called, followed by 2148 :meth:`setUpClass` from the new class. 2149 2150 Similarly if a test is from a different module from the previous test then 2151 ``tearDownModule`` from the previous module is run, followed by 2152 ``setUpModule`` from the new module. 2153 2154 After all the tests have run the final ``tearDownClass`` and 2155 ``tearDownModule`` are run. 2156 2157 Note that shared fixtures do not play well with [potential] features like test 2158 parallelization and they break test isolation. They should be used with care. 2159 2160 The default ordering of tests created by the unittest test loaders is to group 2161 all tests from the same modules and classes together. This will lead to 2162 ``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and 2163 module. If you randomize the order, so that tests from different modules and 2164 classes are adjacent to each other, then these shared fixture functions may be 2165 called multiple times in a single test run. 2166 2167 Shared fixtures are not intended to work with suites with non-standard 2168 ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to 2169 support shared fixtures. 2170 2171 If there are any exceptions raised during one of the shared fixture functions 2172 the test is reported as an error. Because there is no corresponding test 2173 instance an ``_ErrorHolder`` object (that has the same interface as a 2174 :class:`TestCase`) is created to represent the error. If you are just using 2175 the standard unittest test runner then this detail doesn't matter, but if you 2176 are a framework author it may be relevant. 2177 2178 2179 setUpClass and tearDownClass 2180 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2181 2182 These must be implemented as class methods:: 2183 2184 import unittest 2185 2186 class Test(unittest.TestCase): 2187 @classmethod 2188 def setUpClass(cls): 2189 cls._connection = createExpensiveConnectionObject() 2190 2191 @classmethod 2192 def tearDownClass(cls): 2193 cls._connection.destroy() 2194 2195 If you want the ``setUpClass`` and ``tearDownClass`` on base classes called 2196 then you must call up to them yourself. The implementations in 2197 :class:`TestCase` are empty. 2198 2199 If an exception is raised during a ``setUpClass`` then the tests in the class 2200 are not run and the ``tearDownClass`` is not run. Skipped classes will not 2201 have ``setUpClass`` or ``tearDownClass`` run. If the exception is a 2202 :exc:`SkipTest` exception then the class will be reported as having been skipped 2203 instead of as an error. 2204 2205 2206 setUpModule and tearDownModule 2207 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2208 2209 These should be implemented as functions:: 2210 2211 def setUpModule(): 2212 createConnection() 2213 2214 def tearDownModule(): 2215 closeConnection() 2216 2217 If an exception is raised in a ``setUpModule`` then none of the tests in the 2218 module will be run and the ``tearDownModule`` will not be run. If the exception is a 2219 :exc:`SkipTest` exception then the module will be reported as having been skipped 2220 instead of as an error. 2221 2222 2223 Signal Handling 2224 --------------- 2225 2226 .. versionadded:: 3.2 2227 2228 The :option:`-c/--catch <unittest -c>` command-line option to unittest, 2229 along with the ``catchbreak`` parameter to :func:`unittest.main()`, provide 2230 more friendly handling of control-C during a test run. With catch break 2231 behavior enabled control-C will allow the currently running test to complete, 2232 and the test run will then end and report all the results so far. A second 2233 control-c will raise a :exc:`KeyboardInterrupt` in the usual way. 2234 2235 The control-c handling signal handler attempts to remain compatible with code or 2236 tests that install their own :const:`signal.SIGINT` handler. If the ``unittest`` 2237 handler is called but *isn't* the installed :const:`signal.SIGINT` handler, 2238 i.e. it has been replaced by the system under test and delegated to, then it 2239 calls the default handler. This will normally be the expected behavior by code 2240 that replaces an installed handler and delegates to it. For individual tests 2241 that need ``unittest`` control-c handling disabled the :func:`removeHandler` 2242 decorator can be used. 2243 2244 There are a few utility functions for framework authors to enable control-c 2245 handling functionality within test frameworks. 2246 2247 .. function:: installHandler() 2248 2249 Install the control-c handler. When a :const:`signal.SIGINT` is received 2250 (usually in response to the user pressing control-c) all registered results 2251 have :meth:`~TestResult.stop` called. 2252 2253 2254 .. function:: registerResult(result) 2255 2256 Register a :class:`TestResult` object for control-c handling. Registering a 2257 result stores a weak reference to it, so it doesn't prevent the result from 2258 being garbage collected. 2259 2260 Registering a :class:`TestResult` object has no side-effects if control-c 2261 handling is not enabled, so test frameworks can unconditionally register 2262 all results they create independently of whether or not handling is enabled. 2263 2264 2265 .. function:: removeResult(result) 2266 2267 Remove a registered result. Once a result has been removed then 2268 :meth:`~TestResult.stop` will no longer be called on that result object in 2269 response to a control-c. 2270 2271 2272 .. function:: removeHandler(function=None) 2273 2274 When called without arguments this function removes the control-c handler 2275 if it has been installed. This function can also be used as a test decorator 2276 to temporarily remove the handler whilst the test is being executed:: 2277 2278 @unittest.removeHandler 2279 def test_signal_handling(self): 2280 ... 2281