1 This file describes some special Python build types enabled via compile-time 2 preprocessor defines. 3 4 IMPORTANT: if you want to build a debug-enabled Python, it is recommended that 5 you use ``./configure --with-pydebug``, rather than the options listed here. 6 7 However, if you wish to define some of these options individually, it is best 8 to define them in the EXTRA_CFLAGS make variable; 9 ``make EXTRA_CFLAGS="-DPy_REF_DEBUG"``. 10 11 12 Py_REF_DEBUG 13 ------------ 14 15 Turn on aggregate reference counting. This arranges that extern _Py_RefTotal 16 hold a count of all references, the sum of ob_refcnt across all objects. In a 17 debug-mode build, this is where the "8288" comes from in 18 19 >>> 23 20 23 21 [8288 refs] 22 >>> 23 24 Note that if this count increases when you're not storing away new objects, 25 there's probably a leak. Remember, though, that in interactive mode the special 26 name "_" holds a reference to the last result displayed! 27 28 Py_REF_DEBUG also checks after every decref to verify that the refcount hasn't 29 gone negative, and causes an immediate fatal error if it has. 30 31 Special gimmicks: 32 33 sys.gettotalrefcount() 34 Return current total of all refcounts. 35 36 37 Py_TRACE_REFS 38 ------------- 39 40 Turn on heavy reference debugging. This is major surgery. Every PyObject grows 41 two more pointers, to maintain a doubly-linked list of all live heap-allocated 42 objects. Most built-in type objects are not in this list, as they're statically 43 allocated. Starting in Python 2.3, if COUNT_ALLOCS (see below) is also defined, 44 a static type object T does appear in this list if at least one object of type T 45 has been created. 46 47 Note that because the fundamental PyObject layout changes, Python modules 48 compiled with Py_TRACE_REFS are incompatible with modules compiled without it. 49 50 Py_TRACE_REFS implies Py_REF_DEBUG. 51 52 Special gimmicks: 53 54 sys.getobjects(max[, type]) 55 Return list of the (no more than) max most-recently allocated objects, most 56 recently allocated first in the list, least-recently allocated last in the 57 list. max=0 means no limit on list length. If an optional type object is 58 passed, the list is also restricted to objects of that type. The return 59 list itself, and some temp objects created just to call sys.getobjects(), 60 are excluded from the return list. Note that the list returned is just 61 another object, though, so may appear in the return list the next time you 62 call getobjects(); note that every object in the list is kept alive too, 63 simply by virtue of being in the list. 64 65 envvar PYTHONDUMPREFS 66 If this envvar exists, Py_Finalize() arranges to print a list of all 67 still-live heap objects. This is printed twice, in different formats, 68 before and after Py_Finalize has cleaned up everything it can clean up. The 69 first output block produces the repr() of each object so is more 70 informative; however, a lot of stuff destined to die is still alive then. 71 The second output block is much harder to work with (repr() can't be invoked 72 anymore -- the interpreter has been torn down too far), but doesn't list any 73 objects that will die. The tool script combinerefs.py can be run over this 74 to combine the info from both output blocks. The second output block, and 75 combinerefs.py, were new in Python 2.3b1. 76 77 78 PYMALLOC_DEBUG 79 -------------- 80 81 When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_ 82 memory routines are handled by Python's own small-object allocator, while calls 83 to the PyMem_ memory routines are directed to the system malloc/ realloc/free. 84 If PYMALLOC_DEBUG is also defined, calls to both PyObject_ and PyMem_ memory 85 routines are directed to a special debugging mode of Python's small-object 86 allocator. 87 88 This mode fills dynamically allocated memory blocks with special, recognizable 89 bit patterns, and adds debugging info on each end of dynamically allocated 90 memory blocks. The special bit patterns are: 91 92 #define CLEANBYTE 0xCB /* clean (newly allocated) memory */ 93 #define DEADBYTE 0xDB /* dead (newly freed) memory */ 94 #define FORBIDDENBYTE 0xFB /* forbidden -- untouchable bytes */ 95 96 Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit 97 ASCII strings. 98 99 Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N bytes 100 requested. The memory layout is like so, where p represents the address 101 returned by a malloc-like or realloc-like function (p[i:j] means the slice of 102 bytes from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment of 103 negative indices differs from a Python slice): 104 105 p[-2*S:-S] 106 Number of bytes originally asked for. This is a size_t, big-endian (easier 107 to read in a memory dump). 108 p[-S:0] 109 Copies of FORBIDDENBYTE. Used to catch under- writes and reads. 110 p[0:N] 111 The requested memory, filled with copies of CLEANBYTE, used to catch 112 reference to uninitialized memory. When a realloc-like function is called 113 requesting a larger memory block, the new excess bytes are also filled with 114 CLEANBYTE. When a free-like function is called, these are overwritten with 115 DEADBYTE, to catch reference to freed memory. When a realloc- like function 116 is called requesting a smaller memory block, the excess old bytes are also 117 filled with DEADBYTE. 118 p[N:N+S] 119 Copies of FORBIDDENBYTE. Used to catch over- writes and reads. 120 p[N+S:N+2*S] 121 A serial number, incremented by 1 on each call to a malloc-like or 122 realloc-like function. Big-endian size_t. If "bad memory" is detected 123 later, the serial number gives an excellent way to set a breakpoint on the 124 next run, to capture the instant at which this block was passed out. The 125 static function bumpserialno() in obmalloc.c is the only place the serial 126 number is incremented, and exists so you can set such a breakpoint easily. 127 128 A realloc-like or free-like function first checks that the FORBIDDENBYTEs at 129 each end are intact. If they've been altered, diagnostic output is written to 130 stderr, and the program is aborted via Py_FatalError(). The other main failure 131 mode is provoking a memory error when a program reads up one of the special bit 132 patterns and tries to use it as an address. If you get in a debugger then and 133 look at the object, you're likely to see that it's entirely filled with 0xDB 134 (meaning freed memory is getting used) or 0xCB (meaning uninitialized memory is 135 getting used). 136 137 Note that PYMALLOC_DEBUG requires WITH_PYMALLOC. 138 139 Special gimmicks: 140 141 envvar PYTHONMALLOCSTATS 142 If this envvar exists, a report of pymalloc summary statistics is printed to 143 stderr whenever a new arena is allocated, and also by Py_Finalize(). 144 145 Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t). 146 Before it was 16 on all boxes, reflecting that Python couldn't make use of 147 allocations >= 2**32 bytes even on 64-bit boxes before 2.5. 148 149 150 Py_DEBUG 151 -------- 152 153 This is what is generally meant by "a debug build" of Python. 154 155 Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and PYMALLOC_DEBUG (if 156 WITH_PYMALLOC is enabled). In addition, C assert()s are enabled (via the C way: 157 by not defining NDEBUG), and some routines do additional sanity checks inside 158 "#ifdef Py_DEBUG" blocks. 159 160 161 COUNT_ALLOCS 162 ------------ 163 164 Each type object grows three new members: 165 166 /* Number of times an object of this type was allocated. */ 167 int tp_allocs; 168 169 /* Number of times an object of this type was deallocated. */ 170 int tp_frees; 171 172 /* Highwater mark: the maximum value of tp_allocs - tp_frees so 173 * far; or, IOW, the largest number of objects of this type alive at 174 * the same time. 175 */ 176 int tp_maxalloc; 177 178 Allocation and deallocation code keeps these counts up to date. Py_Finalize() 179 displays a summary of the info returned by sys.getcounts() (see below), along 180 with assorted other special allocation counts (like the number of tuple 181 allocations satisfied by a tuple free-list, the number of 1-character strings 182 allocated, etc). 183 184 Before Python 2.2, type objects were immortal, and the COUNT_ALLOCS 185 implementation relies on that. As of Python 2.2, heap-allocated type/ class 186 objects can go away. COUNT_ALLOCS can blow up in 2.2 and 2.2.1 because of this; 187 this was fixed in 2.2.2. Use of COUNT_ALLOCS makes all heap-allocated type 188 objects immortal, except for those for which no object of that type is ever 189 allocated. 190 191 Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCS 192 arranges to ensure that the type object for each allocated object appears in the 193 doubly-linked list of all objects maintained by Py_TRACE_REFS. 194 195 Special gimmicks: 196 197 sys.getcounts() 198 Return a list of 4-tuples, one entry for each type object for which at least 199 one object of that type was allocated. Each tuple is of the form: 200 201 (tp_name, tp_allocs, tp_frees, tp_maxalloc) 202 203 Each distinct type object gets a distinct entry in this list, even if two or 204 more type objects have the same tp_name (in which case there's no way to 205 distinguish them by looking at this list). The list is ordered by time of 206 first object allocation: the type object for which the first allocation of 207 an object of that type occurred most recently is at the front of the list. 208 209 210 LLTRACE 211 ------- 212 213 Compile in support for Low Level TRACE-ing of the main interpreter loop. 214 215 When this preprocessor symbol is defined, before PyEval_EvalFrame executes a 216 frame's code it checks the frame's global namespace for a variable 217 "__lltrace__". If such a variable is found, mounds of information about what 218 the interpreter is doing are sprayed to stdout, such as every opcode and opcode 219 argument and values pushed onto and popped off the value stack. 220 221 Not useful very often, but very useful when needed. 222 223 224 CALL_PROFILE 225 ------------ 226 227 Count the number of function calls executed. 228 229 When this symbol is defined, the ceval mainloop and helper functions count the 230 number of function calls made. It keeps detailed statistics about what kind of 231 object was called and whether the call hit any of the special fast paths in the 232 code. 233 234 235 WITH_TSC 236 -------- 237 238 Super-lowlevel profiling of the interpreter. When enabled, the sys module grows 239 a new function: 240 241 settscdump(bool) 242 If true, tell the Python interpreter to dump VM measurements to stderr. If 243 false, turn off dump. The measurements are based on the processor's 244 time-stamp counter. 245 246 This build option requires a small amount of platform specific code. Currently 247 this code is present for linux/x86 and any PowerPC platform that uses GCC 248 (i.e. OS X and linux/ppc). 249 250 On the PowerPC the rate at which the time base register is incremented is not 251 defined by the architecture specification, so you'll need to find the manual for 252 your specific processor. For the 750CX, 750CXe and 750FX (all sold as the G3) 253 we find: 254 255 The time base counter is clocked at a frequency that is one-fourth that of 256 the bus clock. 257 258 This build is enabled by the --with-tsc flag to configure. 259
