Home | History | Annotate | Download | only in testsuite 
      1 This document attempts to point out some best practices that prove to be helpful
      2 when building new test cases in the tot/test directory.  Everyone is welcomed to
      3 add/modify contents into this file.
      4 
      5 o Do not use hard-coded line numbers in your test case.  Instead, try to tag the
      6   line with some distinguishing pattern, and use the function line_number()
      7   defined in lldbtest.py which takes filename and string_to_match as arguments
      8   and returns the line number.
      9 
     10 As an example, take a look at test/breakpoint_conditions/main.c which has these
     11 two lines:
     12 
     13         return c(val); // Find the line number of c's parent call here.
     14 
     15 and
     16 
     17     return val + 3; // Find the line number of function "c" here.
     18 
     19 The Python test case TestBreakpointConditions.py uses the comment strings to
     20 find the line numbers during setUp(self) and use them later on to verify that
     21 the correct breakpoint is being stopped on and that its parent frame also has
     22 the correct line number as intended through the breakpoint condition.
     23 
     24 o Take advantage of the unittest framework's decorator features to properly
     25   mark your test class or method for platform-specific tests.
     26 
     27 As an example, take a look at test/forward/TestForwardDeclaration.py which has
     28 these lines:
     29 
     30     @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
     31     def test_with_dsym_and_run_command(self):
     32         """Display *bar_ptr when stopped on a function with forward declaration of struct bar."""
     33         self.buildDsym()
     34         self.forward_declaration()
     35 
     36 This tells the test harness that unless we are running "darwin", the test should
     37 be skipped.  This is because we are asking to build the binaries with dsym debug
     38 info, which is only available on the darwin platforms.
     39 
     40 o Cleanup after yourself.  A classic example of this can be found in test/types/
     41   TestFloatTypes.py:
     42 
     43     def test_float_types_with_dsym(self):
     44         """Test that float-type variables are displayed correctly."""
     45         d = {'CXX_SOURCES': 'float.cpp'}
     46         self.buildDsym(dictionary=d)
     47         self.setTearDownCleanup(dictionary=d)
     48         self.float_type()
     49 
     50     ...
     51 
     52     def test_double_type_with_dsym(self):
     53         """Test that double-type variables are displayed correctly."""
     54         d = {'CXX_SOURCES': 'double.cpp'}
     55         self.buildDsym(dictionary=d)
     56         self.setTearDownCleanup(dictionary=d)
     57         self.double_type()
     58 
     59 This tests different data structures composed of float types to verify that what
     60 the debugger prints out matches what the compiler does for different variables
     61 of these types.  We're using a dictionary to pass the build parameters to the
     62 build system.  After a particular test instance is done, it is a good idea to
     63 clean up the files built.  This eliminates the chance that some leftover files
     64 can interfere with the build phase for the next test instance and render it
     65 invalid.
     66 
     67 TestBase.setTearDownCleanup(self, dictionary) defined in lldbtest.py is created
     68 to cope with this use case by taking the same build parameters in order to do
     69 the cleanup when we are finished with a test instance, during
     70 TestBase.tearDown(self).
     71 
     72 o Class-wise cleanup after yourself.
     73 
     74 TestBase.tearDownClass(cls) provides a mechanism to invoke the platform-specific
     75 cleanup after finishing with a test class. A test class can have more than one
     76 test methods, so the tearDownClass(cls) method gets run after all the test
     77 methods have been executed by the test harness.
     78 
     79 The default cleanup action performed by the plugins/darwin.py module invokes the
     80 "make clean" os command.
     81 
     82 If this default cleanup is not enough, individual class can provide an extra
     83 cleanup hook with a class method named classCleanup , for example,
     84 in test/breakpoint_command/TestBreakpointCommand.py:
     85 
     86     @classmethod
     87     def classCleanup(cls):
     88         system(["/bin/sh", "-c", "rm -f output.txt"])
     89 
     90 The 'output.txt' file gets generated during the test run, so it makes sense to
     91 explicitly spell out the action in the same TestBreakpointCommand.py file to do
     92 the cleanup instead of artificially adding it as part of the default cleanup
     93 action which serves to cleanup those intermediate and a.out files. 
     94