1 This document defines the plan for functional testing of POSIX Clocks 2 and Timers. 3 4 First a list of the features that will and will not be tested is given. 5 6 Then, an attempt is made to break down the features into possible testing 7 scenarios. Not all of these may be implemented. The assertions.xml file 8 will have the additional detail on which were chosen for implementation. 9 10 Features To Be Tested 11 ===================== 12 The following features will be tested: 13 clocks - Testing that clocks can accurately keep time in certain scenarios. 14 sleep - Testing that processes can go to sleep for specified periods of time. 15 timers - The following types of timers will be tested. 16 absolute timer, no repeat think: alarm clock 17 relative timer, no repeat think: stopwatch 18 absolute timer, repeats think: grandfather clock 19 relative timer, repeats think: repeating stopwatch 20 Also, timer overruns will be counted. 21 22 Features Not To Be Tested 23 ========================= 24 API testing of the clocks/timers functions is considered to be done by 25 the conformance section. 26 Load and other stress testing will be done in the stress section. 27 A few error handling tests will be created, but the majority will not for 28 this initial release. (Over time, these may be added from regression 29 tests, or regression tests may suggest additional error handling tests to 30 create.) 31 32 Approach 33 ======== 34 In order to generate the test cases listed below, the following approaches 35 were taken: 36 - If it was not covered in the conformance area fully, determined the 37 correct behavior of the features listed above and generated test scenarios 38 to verify this behavior happened. 39 - Determined certain critical values that may cause failures to appear and 40 generated test scenarios setting clocks or timers to these critical 41 values. 42 - Determined race conditions that could appear or time dependencies for 43 things happening and generated test cases to expose these issues. 44 45 Test Case Scenarios 46 =================== 47 Some potential test case scenarios are listed after each functional 48 area. 49 50 Clocks 51 ------ 52 - Set clock time to: 0, a large number, all the dates that were 53 critical for Y2K testing, set to time before daylight savings (and then 54 after daylight savings). ==> This is probably multiple test cases. 55 - Call clock_gettime() mulitple times and ensure the time is always 56 increasing (call also at nsec increments to ensure both nsec and sec 57 are always increasing). 58 - Ensure we can tell time within acceptable jitter (over some period of time). 59 - Test for a few different clocks. 60 - Test after setting the time. 61 - Send a signal while setting the time. 62 - Kill the process (example of previous with SIGKILL) when setting the time. 63 - Have a timer expire when setting the time. 64 - Set the time from two different processes. 65 66 Sleep 67 ----- 68 - Mimic the non-repeating timers tests as they apply to sleep. 69 70 Timers 71 ------ 72 Non-repeating (relative and absolute) 73 - Set timer expiration to: 0, a large number, all the dates that were 74 critical for Y2K testing, set to time before daylight savings (and then 75 after daylight savings). ==> This is probably multiple test cases. Some 76 are absolute timers only. 77 - Have the clock change underneath a timer and ensure it expires 78 correctly. 79 - [Relative only] Set up multiple timers to expire after the same duration 80 and ensure they expire in the order set up. (May need to nanosleep 81 for CLOCKRES nanoseconds in between set ups to get correct results. Also, 82 even if this fails, not necessarily a "bug" since many factors influence 83 expiration time.) 84 - Test out all the various events that can be sent for a timer expiration. 85 - Set two timers to expire at the same time, different events, in the 86 same process. 87 - Set up two timers to expire for the same time in two different processes. 88 - Set up one repeating timer to expire after X time and one non-repeating 89 to expire after 2*X time. 90 91 Repeating (relative and absolute) 92 - Perform all non-repeating timer tests on repeating timers. 93 - Ensure that these are still going off at the same time, over time. 94 95 Overruns 96 - Have multiple timers in a process blocked and ensure overrun count 97 for each is correct. 98 - Have a repeating timer blocked for a period of time and ensure overrun 99 count is correct. 100 - Ensure overrun count and timer_gettime() return correct values when 101 SIGEV_NONE is used in ev.sigev_notify. 102 - Try to call timer_getoverrun() as an overrun is happening. 103 104 Contributors: julie.n.fleischer REMOVE-THIS AT intel DOT com 105 geoffrey.r.gustafson REMOVE-THIS AT intel DOT com 106 rolla.n.selbak REMOVE-THIS AT intel DOT com 107 majid.awad REMOVE-THIS AT intel DOT com 108 salwan.searty REMOVE-THIS AT intel DOT com 109 george REMOVE-THIS AT mvista DOT com 110
