1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
|
Coverage TODO
* --source stuff:
- warn if a package is never found.
- warn if no data was collected
- tie --source into reporting
* plugin work
+ add --cover-include.
+ add --cover-rcfile.
- maybe remove a bunch of options.
* 3.3
- Config file
+ A command switch to point to the rc file.
- Write docs
* Soon
- Ship the nose and py.test plugin.
+ Better omit handling that ignores files during measurement.
- Deal with ~ in specified paths correctly.
- while TRUE claims to be partial.
- A way to mark lines as partial branches, with a regex?
- Default to "while True:", "while 1:"
- HTML keyboard short cuts
* 3.2
+ Some kind of indication in the HTML where yellow lines aren't going.
- Profile the reporting code: it's REALLY slow.
- parser is doing some redundant work.
+ Analysis class should do rolling up of stats also (actually Numbers)
+ Update docs for --branch.
- self.coverage.data.has_arcs is ugly.
+ Branches that never jump to nocover lines shouldn't be marked as partial.
(see top of test_cogapp for examples)
+ Maybe turning off yellow lines should make those lines green?
+ A missing branch to leave the function shows an annotation of -1. Now "exit".
+ XML report needs to get branch information.
+ Add branch info to "coverage debug data"
+ Polish up the help, and double-check the docs.
* Test-runner plugin
- Support distribute/3k installation, register py.test-entrypoint there
as well as py-1.1.1 is 2.4-3.1 compatible.
- Skip registering plugin entrypoints if not running on Jython.
- Update docs to include a section about the plugin.
* Speed
+ C extension collector
- bitvector in trace extension.
- Ignore certain modules
+ Record linenos rather than (file,lineno) pairs in tracer.
x Tricky swapping of collector like figleaf, pycov, et al. (Don't need to do
this with C collector).
- Seems like there should be a faster way to manage all the line number sets in
CodeParser.raw_parse.
- If tracing, canonical_filename_cache overlaps with should_trace_cache. Skip
canonical_filename_cache. Maybe it isn't even worth it...
- Would pre-allocating line number integers make the C tracer faster? It would
use less memory.
* Accuracy
- Record magic number of module to ensure code hasn't changed
- Record version of coverage data file, so we can update what's stored there.
- Record options in coverage data file, so multiple runs are certain to make
sense together.
- Do I still need the lines in annotate_file that deal specially with "else"?
* Power
+ Branch coverage
Titus' idea:
1: if a:
2: b = 2
3: c = 3
if the coverage data shows 1,2,3, it was if-then. if it's 1,3, then the
missing else was executed.
- API for getting coverage data.
- Instruction tracing instead of line tracing.
- Path tracing (how does this even work?)
- Count execution of lines
- Track callers of functions (ala std module trace)
- Method/Class/Module coverage reporting.
- .coverage files that can be kept separate, rather than accumulated.
- test/coverage map: http://rbtcollins.wordpress.com/2009/09/16/back-from-hiatus/
- Similar to figleaf's sections.
* Convenience
- Command line modules should also be directories, meaning all the modules in that
directory.
- Why can't a morf also be a string, the name of a module?
- ignore by module as well as file?
+ Use a .coveragerc file to control Coverage without the programmatic API.
- Own the nose --with-cover plugin.
- Add a --data switch to explicitly control the data file on the command line.
x Why can't you specify execute (-x) and report (-r) in the same invocation?
Maybe just because -x needs the rest of the command line?
+ Support 2.3 - 3.1!
http://pythonology.blogspot.com/2009/02/making-code-run-on-python-20-through-30.html
http://www.rfk.id.au/blog/entry/preparing-pyenchant-for-python-3
http://pydev.blogspot.com/2008/11/making-code-work-in-python-2-and-3.html
+ Explicitly set pickle protocol to 2.
- An inference mode that marks lines as executed if they "must have been" executed:
class definitions, etc, when coverage is started after the class is defined.
- Different categories of exclude pragma? So you can enable and disable them
from the command line, to reconsider exclusions.
- Reporting on files never touched by coverage (package completeness)
- A setup.py command? http://jeetworks.org/node/50
- Deltas: indicate the change in coverage percentage from the last run.
+ Show lines missing rather than lines run in the reporting, since that's what
you need to focus on.
* Beauty
+ HTML report
- Colored bars indicating coverage per file.
- Package navigation.
- Rolled-up statistics.
- Some way to focus in on red and yellow
- Show only lines near highlights?
- Jump to next highlight?
- Keyboard navigation: N and P
- Cookie for changes to pyfile.html state.
+ Clickable column headers on the index page.
+ Syntax coloring in HTML report.
+ Dynamic effects in HTML report.
+ Footer in reports pointing to coverage home page.
+ Baseline grid for linenumber font.
+ Separate out css and HTML.
- Does it work right with utf-8 source files? http://www.python.org/dev/peps/pep-0263/
- Use vim modeline to determine tab width: http://vimdoc.sourceforge.net/htmldoc/options.html#modeline
* Community
+ New docs, rather than pointing to Gareth's
+ Min python version is 2.3.
- Three phases of work:
- Collection
- Analysis
- Reporting
- Distinction between:
- ignore (files not to collect)
- exclude (lines not to report as missed)
- omit (files not to report)
- Changes from coverage 2.x:
- Bare "except:" lines now count as executable code.
- Double function decorators: all decorator lines count as executable code.
- Document the .coverage file format.
+ HTML reporting.
- Much more detail about what's in the report.
- References between pages are off:
- They have <em> tags around them.
- They use #anchors that don't survive the px->html conversion.
+ Be sure --help text is complete (-i is missing).
+ Host the project somewhere with a real bug tracker: bitbucket.org
+ Point discussion to TIP
- PEP 8 compliance?
* Programmability
+ Don't use sys.exit in CoverageScript.
+ Remove singleton
+ Initialization of instance variables in the class.
* Installation
- How will Coverage package install over coverage.py module?
- pip can't install it: it reads the coverage html page, and finds the kit link,
but then can't handle the root-relative link.
* Modernization
+ Decide on minimum supported version
+ 2.3
+ Get rid of the basestring protection
+ Use enumerate
+ Use sets instead of dicts
+ Switch from getopt to optparse.
+ Get rid of the recursive nonsense.
+ Docstrings.
+ Remove huge document-style comments.
- Better names:
+ self.cache -> self.cache_filename -> CoverageData.filename
+ self.usecache -> CoverageData.use_file
- More classes:
- Module munging
+ Coverage data files
+ Why are some imports at the top of the file, and some in functions?
+ Get rid of sys.exitfunc use.
+ True and False (with no backward adaptation: the constants are new in 2.2.1)
+ Get rid of compiler module
+ In analyzing code
+ In test_coverage.py
+ Style:
+ lineno
+ filename
* Correctness
- What does -p (parallel mode) mean with -e (erase data)?
* Tests
+ Switch to a real test runner, like nose.
+ Test both the C trace function and the Python trace function.
+ parser.py has no direct tests.
- Tests about the .coverage file.
+ Tests about the --long-form of arguments.
+ Tests about overriding the .coverage filename.
- Tests about parallel mode.
+ Tests about assigning a multi-line string.
- Tests about tricky docstrings.
+ Coverage test Coverage!
- Tests that tracing stops after calling stop()
- More intensive thread testing.
x Tests about the "import __main__" in cmdline.py
+ What happens if the -x script raises an exception?
- Test that the kit has all the proper contents.
|
