summaryrefslogtreecommitdiff
path: root/TODO.txt
blob: 0062dbe696086d0e7e9c68d62e0d7c8ce844b723 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
Coverage TODO

Key:
    * Heading
    - Not done yet.
    + Done.
    x Not going to do.

* 4.0

- What defaults should change?
    - --source = . ?
    x --branch = True ?

- Remove 2.3, 2.4, 2.5 limitations
    + set, sorted, reversed, rpartition
    + generator expressions
    + decorators
    + collections.defaultdict
    + .startswith((,))
    + "with" statements
    - .format() ?
    + try/except/finally
    + with assertRaises
    + addCleanup instead of tearDown
    + exec statement can look like a function in py2 (since when?)
    - runpy ?
    + we can use "except ExcClass as e:"

- Plugins
    - Clean up
    + implement plugin support in CTracer
    + remove plugin support from PyTracer
    - add services:
        - filelocator
        - warning
    - dynamic_source_filename: return should be a canonical path
        - update the omit test to use "quux*" instead of "*quux*"
    - docs
+ Make reports use filenames, not module names
- documentation
    - test helpers
    - cov.config["run:branch"] api
    - "added in 4.0"
+ Remove code only run on <2.6
- Change data file to json
- Create data api
+ gevent, etc.
+ Remove the old command-line syntax
    + A pain, b/c of the structure of the tests.
    + BTW: make an easier way to write those tests.

- Documentation
    - Plugins!
        Once per process
        Once per file
            - create a file tracer
            - call its has_dynamic_source_file()
        Once per call
        Once per line


* --source stuff:
    + warn if a package is never found.
    + warn if no data was collected
    - tie --source into reporting

* Soon

+ Better omit handling that ignores files during measurement.
    - Deal with ~ in specified paths correctly.
+ while TRUE claims to be partial.
    + A way to mark lines as partial branches, with a regex?
        + Default to "while True:", "while 1:"
+ HTML keyboard short cuts


* 3.2

+ Some kind of indication in the HTML where yellow lines aren't going.
- Profile the reporting code: it's REALLY slow.
    - parser is doing some redundant work.
+ Analysis class should do rolling up of stats also (actually Numbers)
+ Update docs for --branch.
- self.coverage.data.has_arcs is ugly.
+ Branches that never jump to nocover lines shouldn't be marked as partial.
    (see top of test_cogapp for examples)
+ Maybe turning off yellow lines should make those lines green?
+ A missing branch to leave the function shows an annotation of -1. Now "exit".
+ XML report needs to get branch information.
+ Add branch info to "coverage debug data"
+ Polish up the help, and double-check the docs.


* Speed

+ C extension collector
- bitvector in trace extension.
- Ignore certain modules
+ Record linenos rather than (file,lineno) pairs in tracer.
x Tricky swapping of collector like figleaf, pycov, et al. (Don't need to do
    this with C collector).
- Seems like there should be a faster way to manage all the line number sets in
    CodeParser.raw_parse.
- If tracing, canonical_filename_cache overlaps with should_trace_cache.  Skip
    canonical_filename_cache. Maybe it isn't even worth it...
- Would pre-allocating line number integers make the C tracer faster? It would
    use less memory.


* Accuracy

- Record magic number of module to ensure code hasn't changed
- Record version of coverage data file, so we can update what's stored there.
- Record options in coverage data file, so multiple runs are certain to make
    sense together.
- Do I still need the lines in annotate_file that deal specially with "else"?


* Power

+ Branch coverage
    Titus' idea:
        1: if a:
        2:     b = 2
        3: c = 3
    if the coverage data shows 1,2,3, it was if-then.  if it's 1,3, then the
    missing else was executed.
- API for getting coverage data.
- Instruction tracing instead of line tracing.
- Path tracing (how does this even work?)
- Count execution of lines
- Track callers of functions (ala std module trace)
- Method/Class/Module coverage reporting.
- .coverage files that can be kept separate, rather than accumulated.
- test/coverage map: http://rbtcollins.wordpress.com/2009/09/16/back-from-hiatus/
    - Similar to figleaf's sections.


* Convenience

- Command line modules should also be directories, meaning all the modules in that
    directory.
- Why can't a morf also be a string, the name of a module?
- ignore by module as well as file?
+ Use a .coveragerc file to control Coverage without the programmatic API.
- Add a --data switch to explicitly control the data file on the command line.
x Why can't you specify execute (-x) and report (-r) in the same invocation?
    Maybe just because -x needs the rest of the command line?
+ Support 2.3 - 3.1!
    http://pythonology.blogspot.com/2009/02/making-code-run-on-python-20-through-30.html
    http://www.rfk.id.au/blog/entry/preparing-pyenchant-for-python-3
    http://pydev.blogspot.com/2008/11/making-code-work-in-python-2-and-3.html
    + Explicitly set pickle protocol to 2.
- An inference mode that marks lines as executed if they "must have been" executed:
    class definitions, etc, when coverage is started after the class is defined.
- Different categories of exclude pragma? So you can enable and disable them
    from the command line, to reconsider exclusions.
+ Reporting on files never touched by coverage (package completeness)
- A setup.py command? http://jeetworks.org/node/50
- Deltas: indicate the change in coverage percentage from the last run.
+ Show lines missing rather than lines run in the reporting, since that's what
  you need to focus on.


* Beauty

+ HTML report
    - Colored bars indicating coverage per file.
    - Package navigation.
    - Rolled-up statistics.
    - Some way to focus in on red and yellow
        - Show only lines near highlights?
        + Jump to next highlight?
            + Keyboard navigation: j and k.
    - Cookie for changes to pyfile.html state.
    + Clickable column headers on the index page.
    + Syntax coloring in HTML report.
    + Dynamic effects in HTML report.
    + Footer in reports pointing to coverage home page.
    + Baseline grid for linenumber font.
    + Separate out css and HTML.
    + Does it work right with utf-8 source files? http://www.python.org/dev/peps/pep-0263/
    - Use vim modeline to determine tab width: http://vimdoc.sourceforge.net/htmldoc/options.html#modeline


* Community

+ New docs, rather than pointing to Gareth's
    + Min python version is 2.3.
    - Three phases of work:
        - Collection
        - Analysis
        - Reporting
    - Distinction between:
        - ignore (files not to collect)
        - exclude (lines not to report as missed)
        - omit (files not to report)
    - Changes from coverage 2.x:
        - Bare "except:" lines now count as executable code.
        - Double function decorators: all decorator lines count as executable code.
    - Document the .coverage file format.
    + HTML reporting.
        - Much more detail about what's in the report.
    - References between pages are off:
        - They have <em> tags around them.
        - They use #anchors that don't survive the px->html conversion.
+ Be sure --help text is complete (-i is missing).
+ Host the project somewhere with a real bug tracker: bitbucket.org
+ Point discussion to TIP
- PEP 8 compliance?


* Programmability

+ Don't use sys.exit in CoverageScript.
+ Remove singleton
    + Initialization of instance variables in the class.


* Installation

x How will Coverage package install over coverage.py module?
x pip can't install it: it reads the coverage html page, and finds the kit link,
    but then can't handle the root-relative link.


* Modernization

+ Decide on minimum supported version
    + 2.3
    + Get rid of the basestring protection
    + Use enumerate
    + Use sets instead of dicts
+ Switch from getopt to optparse.
+ Get rid of the recursive nonsense.
+ Docstrings.
+ Remove huge document-style comments.
- Better names:
    + self.cache -> self.cache_filename -> CoverageData.filename
    + self.usecache -> CoverageData.use_file
- More classes:
    - Module munging
    + Coverage data files
+ Why are some imports at the top of the file, and some in functions?
+ Get rid of sys.exitfunc use.
+ True and False (with no backward adaptation: the constants are new in 2.2.1)
+ Get rid of compiler module
    + In analyzing code
    + In test_coverage.py
+ Style:
    + lineno
    + filename


* Correctness

- What does -p (parallel mode) mean with -e (erase data)?


* Tests

+ Switch to a real test runner, like nose.
+ Test both the C trace function and the Python trace function.
+ parser.py has no direct tests.
- Tests about the .coverage file.
+ Tests about the --long-form of arguments.
+ Tests about overriding the .coverage filename.
- Tests about parallel mode.
+ Tests about assigning a multi-line string.
- Tests about tricky docstrings.
+ Coverage test Coverage!
- Tests that tracing stops after calling stop()
- More intensive thread testing.
x Tests about the "import __main__" in cmdline.py
+ What happens if the -x script raises an exception?
- Test that the kit has all the proper contents.