1 <?xml version="1.0" standalone="no"?> 2 <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" 3 "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" 4 [ 5 ]> 6 7 <article id="index"> 8 <articleinfo> 9 <title>D-Bus Test Plan</title> 10 <date>14 February 2003</date> 11 <authorgroup> 12 <author> 13 <firstname>Anders</firstname> 14 <surname>Carlsson</surname> 15 <affiliation> 16 <orgname>CodeFactory AB</orgname> 17 <address><email>andersca (a] codefactory.se</email></address> 18 </affiliation> 19 </author> 20 </authorgroup> 21 </articleinfo> 22 <sect1 id="introduction"> 23 <title>Introduction</title> 24 <para> 25 This document tries to explain the details of the test plan for D-Bus 26 </para> 27 <sect2 id="importance-of-testing"> 28 <title>The importance of testing</title> 29 <para> 30 As with any big library or program, testing is important. It 31 can help find bugs and regressions and make the code better 32 overall. 33 </para> 34 <para> 35 D-Bus is a large and complex piece of software (about 25,000 36 lines of code for the client library, and 2,500 lines of code 37 for the bus daemon) and it's therefore important to try to make sure 38 that all parts of the software is functioning correctly. 39 </para> 40 <para> 41 D-Bus can be built with support for testing by passing 42 <literal>--enable-tests</literal>. to the configure script. It 43 is recommended that production systems build without testing 44 since that reduces the D-Bus client library size. 45 </para> 46 </sect2> 47 </sect1> 48 <sect1 id="client-library"> 49 <title>Testing the D-Bus client library</title> 50 <para> 51 The tests for the client library consist of the dbus-test 52 program which is a unit test for all aspects of the client 53 library. Whenever a bug in the client library is found and 54 fixed, a test is added to make sure that the bug won't occur again. 55 </para> 56 <sect2 id="data-structures"> 57 <title>Data Structures</title> 58 <para> 59 The D-Bus client library consists of some data structures that 60 are used internally; a linked list class, a hashtable class and 61 a string class. All aspects of those are tested by dbus-test. 62 </para> 63 </sect2> 64 <sect2 id="message-loader"> 65 <title>Message loader</title> 66 <para> 67 The message loader is the part of D-Bus that takes messages in 68 raw character form and parses them, turning them into DBusMessages. 69 </para> 70 <para> 71 This is one of the parts of D-Bus that 72 <emphasis>must</emphasis> be absolutely bug-free and 73 robust. The message loader should be able to handle invalid 74 and incomplete messages without crashing. Not doing so is a 75 serious issue and can easily result in D-Bus being exploitable 76 to DoS attacks. 77 </para> 78 <para> 79 To solve these problems, there is a testing feature called the 80 Message Builder. The message builder can take a serialized 81 message in string-form and convert it into a raw character 82 string which can then be loaded by the message loader. 83 </para> 84 <figure> 85 <title>Example of a message in string form</title> 86 <programlisting> 87 # Standard org.freedesktop.DBus.Hello message 88 89 VALID_HEADER 90 FIELD_NAME name 91 TYPE STRING 92 STRING 'org.freedesktop.DBus.Hello' 93 FIELD_NAME srvc 94 TYPE STRING 95 STRING 'org.freedesktop.DBus' 96 ALIGN 8 97 END_LENGTH Header 98 START_LENGTH Body 99 END_LENGTH Body 100 </programlisting> 101 </figure> 102 <para> 103 The file format of messages in string form is documented in 104 the D-Bus Reference Manual. 105 </para> 106 <para> 107 The message test part of dbus-test is using the message 108 builder to build different kinds of messages, both valid, 109 invalid, and invalid ones, to make sure that the loader won't 110 crash or leak memory of any of those, and that the loader 111 knows if a message is valid or not. 112 </para> 113 <para> 114 There is also a test program called 115 <literal>break-loader</literal> that loads a message in 116 string-form into raw character form using the message 117 builder. It then randomly changes the message, it can for 118 example replace single bytes of data or modify the length of 119 the message. This is to simulate network errors. The 120 break-loader program saves all the messages leading to errors 121 so it can easily be run for a long period of time. 122 </para> 123 </sect2> 124 <sect2 id="authentication"> 125 <title>Authentication</title> 126 <para> 127 For testing authentication, there is a testing feature that 128 can read authentication sequences from a file and play them 129 back to a dummy server and client to make sure that 130 authentication is working according to the specification. 131 </para> 132 <figure> 133 <title>Example of an authentication script</title> 134 <programlisting> 135 ## this tests a successful auth of type EXTERNAL 136 137 SERVER 138 SEND 'AUTH EXTERNAL USERNAME_HEX' 139 EXPECT_COMMAND OK 140 EXPECT_STATE WAITING_FOR_INPUT 141 SEND 'BEGIN' 142 EXPECT_STATE AUTHENTICATED 143 </programlisting> 144 </figure> 145 </sect2> 146 </sect1> 147 <sect1 id="daemon"> 148 <title>Testing the D-Bus bus daemon</title> 149 <para> 150 Since the D-Bus bus daemon is using the D-Bus client library it 151 will benefit from all tests done on the client library, but 152 there is still the issue of testing client-server communication. 153 This is more complicated since it it may require another process 154 running. 155 </para> 156 <sect2 id="debug-transport"> 157 <title>The debug transport</title> 158 <para> 159 In D-Bus, a <emphasis>transport</emphasis> is a class that 160 handles sending and receiving raw data over a certain 161 medium. The transport that is used most in D-Bus is the UNIX 162 transport with sends and recevies data over a UNIX socket. A 163 transport that tunnels data through X11 client messages is 164 also under development. 165 </para> 166 <para> 167 The D-Bus debug transport is a specialized transport that 168 works in-process. This means that a client and server that 169 exists in the same process can talk to eachother without using 170 a socket. 171 </para> 172 </sect2> 173 <sect2 id="bus-test"> 174 <title>The bus-test program</title> 175 <para> 176 The bus-test program is a program that is used to test various 177 parts of the D-Bus bus daemon; robustness and that it conforms 178 to the specifications. 179 </para> 180 <para> 181 The test program has the necessary code from the bus daemon 182 linked in, and it uses the debug transport for 183 communication. This means that the bus daemon code can be 184 tested without the real bus actually running, which makes 185 testing easier. 186 </para> 187 <para> 188 The bus-test program should test all major features of the 189 bus, such as service registration, notification when things 190 occurs and message matching. 191 </para> 192 </sect2> 193 </sect1> 194 <sect1 id="other-tests"> 195 <title>Other tests</title> 196 197 <sect2 id="oom-robustness"> 198 <title>Out-Of-Memory robustness</title> 199 <para> 200 Since D-Bus should be able to be used in embedded devices, and 201 also as a system service, it should be able to cope with 202 low-memory situations without exiting or crashing. 203 </para> 204 <para> 205 In practice, this means that both the client and server code 206 must be able to handle dbus_malloc returning NULL. 207 </para> 208 <para> 209 To test this, two environment variables 210 exist. <literal>DBUS_MALLOC_FAIL_NTH</literal> will make every 211 nth call to dbus_malloc return NULL, and 212 <literal>DBUS_MALLOC_FAIL_GREATER_THAN</literal> will make any 213 dbus_malloc call with a request for more than the specified 214 number of bytes fail. 215 </para> 216 </sect2> 217 218 <sect2 id="leaks-and-other-stuff"> 219 <title>Memory leaks and code robustness</title> 220 <para> 221 Naturally there are some things that tests can't be written 222 for, for example things like memory leaks and out-of-bounds 223 memory reading or writing. 224 </para> 225 <para> 226 Luckily there exists good tools for catching such errors. One 227 free good tool is <ulink url="http://devel-home.kde.org/~sewardj/">Valgrind</ulink>, which runs the program in a 228 virtual CPU which makes catching errors easy. All test programs can be run under Valgrind, 229 </para> 230 </sect2> 231 </sect1> 232 </article> 233
