1 SECTIONS: 2 1. Overview 3 2. Framework Functionality 4 3. Building and Running the Tests 5 6 1. Overview 7 ------------ 8 This document describes how to run the tests in the POSIX Test Suite. 9 10 Our framework currently has the ability to build and run conformance, 11 functional, and stress tests. All tests are built with make all, but 12 certain care should be used when running the stress test suite as the 13 tests may leave the system in an indeterminate state. 14 15 2. Framework Functionality 16 ---------------------------------------------------- 17 18 * Conformance Tests 19 The build and execution process varies for conformance tests. 20 21 For definitions tests, the build and execution process is the same since 22 the pass/fail criterion is determine by whether or not the test compiles. 23 A definitions test will be compiled, not linked, by the toolchain. 24 25 For all other tests, they will have one test for the compile, one for the 26 link, and one for the execution of the test. Successful tests return 27 PTS_PASS, which the interprets as success. All other return values are 28 considered failures [For more info, see HOWTO_ResultCodes]. 29 30 * Functional/Stress Tests 31 Functional and stress tests have their own Makefile and method for running, 32 and the framework merely calls these mechanisms. To build functional and 33 stress tests, the framework calls the main Makefile for each functional 34 area. To run these tests, the framework calls the run.sh file for each 35 functional area. 36 37 3. Building and Running the Tests 38 ---------------------------------- 39 40 ** See the BUILD file for info on how to set up the build for the specific 41 area you are testing (threads, mqs, semaphores, etc.), as well as setup the 42 build specific to your environment. 43 44 To build and run the tests, you should be in the main posix test suite 45 directory. 46 47 From there, execute: 48 # make all 49 50 This will build all of the conformance, functional, and stress tests. 51 52 To disable known failures on Linux do make filter-known-fails. 53 54 * Conformance-specific items * 55 56 To run conformance tests for a specific directory, just cd to the directory 57 and run make all test. 58 59 Example: 60 # cd conformance/definitions/time_h ; make all test 61 62 To just build conformance tests, run: 63 # make conformance-all 64 65 To just run conformance tests, run: 66 # make conformance-test 67 68 * Functional/Stress-specific items * 69 70 To run only functional tests, run: 71 # make functional-test 72 73 To run only stress tests, run: 74 # make stress-test 75 76 To make only functional tests, run: 77 # make functional-all 78 79 To make only stress tests, run: 80 # make stress-all 81 82 To skip known failures on Linux, run: 83 # make filter-known-fails 84 # make test 85 86 4. Running POSIX Option Group Feature Tests 87 ----------------------------------------------------- 88 89 There are several POSIX option groups available that can be tested with this 90 test suite. The available groups are: 91 92 AIO | Asynchonous I/O 93 MEM | Memory 94 MSG | IPC 95 SEM | Semaphores 96 SIG | Signals 97 THR | Threading 98 TMR | Timers 99 TPS | Thread Execution Scheduling 100 101 You can run the tests by executing: 102 103 bin/run-posix-option-group-test.sh [OPTION-GROUP] 104 105 or run all of the tests by executing: 106 107 bin/run-all-posix-option-group-tests.sh 108 109 More details about these features can be found on the OpenGroup site 110 (http://www.opengroup.org). 111 112 Contributors: rusty.lynch REMOVE-THIS AT intel DOT com 113 julie.n.fleischer REMOVE-THIS AT intel DOT com 114 rolla.n.selbak REMOVE-THIS AT intel DOT com 115 yaneurabeya REMOVE-THIS AT gmail DOT com 116
