1 Coverage.py TODO 2 3 Key: 4 * Heading 5 - Not done yet. 6 + Done. 7 x Not going to do. 8 9 * 4.0 10 11 - What defaults should change? 12 x --source = . ? 13 x --branch = True ? 14 15 - Remove 2.3, 2.4, 2.5 limitations 16 + set, sorted, reversed, rpartition 17 + generator expressions 18 + decorators 19 + collections.defaultdict 20 + .startswith((,)) 21 + "with" statements 22 - .format() ? 23 + try/except/finally 24 + with assertRaises 25 + addCleanup instead of tearDown 26 + exec statement can look like a function in py2 (since when?) 27 - runpy ? 28 + we can use "except ExcClass as e:" 29 30 - Plugins 31 + Clean up 32 + implement plugin support in CTracer 33 + remove plugin support from PyTracer 34 x add services: 35 - filelocator 36 - warning 37 - dynamic_source_filename: return should be a canonical path 38 - update the omit test to use "quux*" instead of "*quux*" 39 + docs 40 + Make reports use filenames, not module names 41 - documentation 42 - test helpers 43 + cov.config["run:branch"] api (well, coverage.get_option etc) 44 + "added in 4.0" 45 - tweaks to theme? 46 - Plugins! 47 Once per process 48 Once per file 49 - create a file tracer 50 - call its has_dynamic_source_file() 51 Once per call 52 Once per line 53 - build process 54 - don't publish to nedbat.com any more (but still need the sample html reports) 55 + don't need .px tooling 56 - write a new nedbat.com/code/coverage page. 57 - all doc links should point to rtfd 58 + Remove code only run on <2.6 59 + Change data file to json 60 + Create data api 61 + gevent, etc. 62 + Remove the old command-line syntax 63 + A pain, b/c of the structure of the tests. 64 + BTW: make an easier way to write those tests. 65 66 - tests 67 - test the kit has the right contents 68 - test the kit installs the right stuff 69 70 71 * --source stuff: 72 + warn if a package is never found. 73 + warn if no data was collected 74 - tie --source into reporting 75 76 * Soon 77 78 + Better omit handling that ignores files during measurement. 79 - Deal with ~ in specified paths correctly. 80 + while TRUE claims to be partial. 81 + A way to mark lines as partial branches, with a regex? 82 + Default to "while True:", "while 1:" 83 + HTML keyboard short cuts 84 85 86 * 3.2 87 88 + Some kind of indication in the HTML where yellow lines aren't going. 89 - Profile the reporting code: it's REALLY slow. 90 - parser is doing some redundant work. 91 + Analysis class should do rolling up of stats also (actually Numbers) 92 + Update docs for --branch. 93 x self.coverage.data.has_arcs is ugly. 94 + Branches that never jump to nocover lines shouldn't be marked as partial. 95 (see top of test_cogapp for examples) 96 + Maybe turning off yellow lines should make those lines green? 97 + A missing branch to leave the function shows an annotation of -1. Now "exit". 98 + XML report needs to get branch information. 99 + Add branch info to "coverage debug data" 100 + Polish up the help, and double-check the docs. 101 102 103 * Speed 104 105 + C extension collector 106 - bitvector in trace extension. 107 - Ignore certain modules 108 + Record linenos rather than (file,lineno) pairs in tracer. 109 x Tricky swapping of collector like figleaf, pycov, et al. (Don't need to do 110 this with C collector). 111 - Seems like there should be a faster way to manage all the line number sets in 112 CodeParser.raw_parse. 113 - If tracing, canonical_filename_cache overlaps with should_trace_cache. Skip 114 canonical_filename_cache. Maybe it isn't even worth it... 115 - Would pre-allocating line number integers make the C tracer faster? It would 116 use less memory. 117 118 119 * Accuracy 120 121 - Record magic number of module to ensure code hasn't changed 122 - Record version of coverage data file, so we can update what's stored there. 123 - Record options in coverage data file, so multiple runs are certain to make 124 sense together. 125 - Do I still need the lines in annotate_file that deal specially with "else"? 126 127 128 * Power 129 130 + Branch coverage 131 Titus' idea: 132 1: if a: 133 2: b = 2 134 3: c = 3 135 if the coverage data shows 1,2,3, it was if-then. if it's 1,3, then the 136 missing else was executed. 137 + API for getting coverage data. 138 - Instruction tracing instead of line tracing. 139 - Path tracing (how does this even work?) 140 - Count execution of lines 141 - Track callers of functions (ala std module trace) 142 - Method/Class/Module coverage reporting. 143 - .coverage files that can be kept separate, rather than accumulated. 144 - test/coverage map: http://rbtcollins.wordpress.com/2009/09/16/back-from-hiatus/ 145 - Similar to figleaf's sections. 146 147 148 * Convenience 149 150 - Command line modules should also be directories, meaning all the modules in that 151 directory. 152 - Why can't a morf also be a string, the name of a module? 153 - ignore by module as well as file? 154 + Use a .coveragerc file to control coverage.py without the programmatic API. 155 - Add a --data switch to explicitly control the data file on the command line. 156 x Why can't you specify execute (-x) and report (-r) in the same invocation? 157 Maybe just because -x needs the rest of the command line? 158 + Support 2.3 - 3.1! 159 http://pythonology.blogspot.com/2009/02/making-code-run-on-python-20-through-30.html 160 http://www.rfk.id.au/blog/entry/preparing-pyenchant-for-python-3 161 http://pydev.blogspot.com/2008/11/making-code-work-in-python-2-and-3.html 162 + Explicitly set pickle protocol to 2. 163 - An inference mode that marks lines as executed if they "must have been" executed: 164 class definitions, etc, when coverage is started after the class is defined. 165 - Different categories of exclude pragma? So you can enable and disable them 166 from the command line, to reconsider exclusions. 167 + Reporting on files never touched by coverage.py (package completeness) 168 - A setup.py command? http://jeetworks.org/node/50 169 - Deltas: indicate the change in coverage percentage from the last run. 170 + Show lines missing rather than lines run in the reporting, since that's what 171 you need to focus on. 172 173 174 * Beauty 175 176 + HTML report 177 - Colored bars indicating coverage per file. 178 - Package navigation. 179 - Rolled-up statistics. 180 - Some way to focus in on red and yellow 181 - Show only lines near highlights? 182 + Jump to next highlight? 183 + Keyboard navigation: j and k. 184 - Cookie for changes to pyfile.html state. 185 + Clickable column headers on the index page. 186 + Syntax coloring in HTML report. 187 + Dynamic effects in HTML report. 188 + Footer in reports pointing to coverage.py home page. 189 + Baseline grid for linenumber font. 190 + Separate out css and HTML. 191 + Does it work right with utf-8 source files? http://www.python.org/dev/peps/pep-0263/ 192 - Use vim modeline to determine tab width: http://vimdoc.sourceforge.net/htmldoc/options.html#modeline 193 194 195 * Community 196 197 + New docs, rather than pointing to Gareth's 198 + Min python version is 2.3. 199 - Three phases of work: 200 - Collection 201 - Analysis 202 - Reporting 203 - Distinction between: 204 - ignore (files not to collect) 205 - exclude (lines not to report as missed) 206 - omit (files not to report) 207 - Changes from coverage.py 2.x: 208 - Bare "except:" lines now count as executable code. 209 - Double function decorators: all decorator lines count as executable code. 210 x Document the .coverage file format. 211 + HTML reporting. 212 - Much more detail about what's in the report. 213 - References between pages are off: 214 - They have <em> tags around them. 215 - They use #anchors that don't survive the px->html conversion. 216 + Be sure --help text is complete (-i is missing). 217 + Host the project somewhere with a real bug tracker: bitbucket.org 218 + Point discussion to TIP 219 - PEP 8 compliance? 220 221 222 * Programmability 223 224 + Don't use sys.exit in CoverageScript. 225 + Remove singleton 226 + Initialization of instance variables in the class. 227 228 229 * Installation 230 231 x How will coverage.py package install over coverage.py module? 232 x pip can't install it: it reads the coverage.py html page, and finds the kit link, 233 but then can't handle the root-relative link. 234 235 236 * Modernization 237 238 + Decide on minimum supported version 239 + 2.3 240 + Get rid of the basestring protection 241 + Use enumerate 242 + Use sets instead of dicts 243 + Switch from getopt to optparse. 244 + Get rid of the recursive nonsense. 245 + Docstrings. 246 + Remove huge document-style comments. 247 - Better names: 248 + self.cache -> self.cache_filename -> CoverageData.filename 249 + self.usecache -> CoverageData.use_file 250 - More classes: 251 - Module munging 252 + Coverage data files 253 + Why are some imports at the top of the file, and some in functions? 254 + Get rid of sys.exitfunc use. 255 + True and False (with no backward adaptation: the constants are new in 2.2.1) 256 + Get rid of compiler module 257 + In analyzing code 258 + In test_coverage.py 259 + Style: 260 + lineno 261 + filename 262 263 264 * Correctness 265 266 - What does -p (parallel mode) mean with -e (erase data)? 267 268 269 * Tests 270 271 + Switch to a real test runner, like nose. 272 + Test both the C trace function and the Python trace function. 273 + parser.py has no direct tests. 274 + Tests about the .coverage file. 275 + Tests about the --long-form of arguments. 276 + Tests about overriding the .coverage filename. 277 - Tests about parallel mode. 278 + Tests about assigning a multi-line string. 279 - Tests about tricky docstrings. 280 + Coverage test coverage.py! 281 - Tests that tracing stops after calling stop() 282 - More intensive thread testing. 283 x Tests about the "import __main__" in cmdline.py 284 + What happens if the -x script raises an exception? 285 - Test that the kit has all the proper contents. 286