1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
|
DOCUMENTATION
CHANGELOG
NEWS
sep. upload of index for 0.10 -- link from 0.9 index
sanity check all docstrings
FEATURES
new plugin:
testid
-- needs loadTestsFromNames hook
rename Failure and split into subclasses for Import and other, and make
it optionally include the name of the file being considered so that
Failure test logs can be more informative
* plugin methods to add
before/after Context [in suite, call with parent?
or in lazy load situations only?]
loadTestsFromName
when result prints final result, patch in error classes, eg,
FAILED (todo=10)
or
OK (skipped=2)
** this can only be done in nose test runner
* loader
- support module.callable in addition to module:callable names
- loadTestsFromTestCase -- override to add plugin call
- loadTestsFromNames -- plugin hook
BUGS
--exclude doesn't seem to work? in some cases?
-- when run vs spine suite, makeTest seemingly was called on an object
that should not have passed the selector --
sqlalchemy.sql._FunctionGateway, which doesn't match testmatch and
isn't a testcase subclass -- need to look into it further.
CHORES
make setup work with & w/out setuptools
MERGE merge trunk -> 0.10 from r9 to pick up trunk changes/fixes
fix builtin plugins:
isolation * post merge
- need loadTestFromNames hook to make this work -- must be able to take over
completely and be (or use internally at least) a generator. This is
needed because the plugin needs to isolate a import time, not just
test-run time.
TESTS NEEDED
deprecated at module/module setup
error at module/module setup
plugin api -- comprehensive integration test that runs a suite with a plugin
that tracks calls and ensures that all calls are called with proper
arguments
PROFILE
need to profile -- on 250 tests with discovery, 0.10-dev is ~ 1 second
slower than 0.9. Profile and optimize.
older notes:
not sure how this one can be made to work at the module level, since
import and testrunning take place separately now -- mods a, b, c are all
imported before tests are run in a. probably best to have it work at the
directory and context levels -- that will get pretty close to what it
does in 0.9
it can be made to work only in surrounding contexts that are generators:
hook before
load and yield suite
hook after
needed in 2 places -- loadTestsFromDir, loadTestsFromNames.
So loadTestsFromNames will have to become lazy -- what impact will that
have on unittest compatibility, etc? can it just return a lazysuite?
the problem with loadTestsFromNames is that context teardown requires
that for a given context, all tests within the context are known before
teardown of the first suite within the context. Say you have a package of
tests, a, with subpackages a.b and a.c. User wants to run tests a.b.test
and a.c.test:
nosetests a.b.test a.c.test
'a' and 'b' setup and 'b' teardown run naturally with the 1st test
'c' setup and teardown run natually with the 2nd test. But when, or
how, does 'a' teardown run? If all tests are known, then a.c.test can
run a teardown, because it knows that no more tests in 'a' are going
to run. But if the 2 tests are loaded lazily -- as is REQUIRED for
the isolation plugin -- then 1st test will mistakenly run 'a' teardown,
since it doesn't know as it is torn down that another test in the
same context is about to load.
can't fix this by a call at the end of the loading process, since then
context teardowns may interleave -- 'a' and 'b' may be present at once
can only fix it by forcing full context load & unload around each
test WHEN USING loadTestFromNames -- this is inefficient, but unavoidable
without controlling the *order* of tests passed to loadTestsFromNames.
|
