1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
|
DOCUMENTATION
CHANGELOG
NEWS
sep. upload of index for 0.10 -- link from 0.9 index
sanity check all docstrings
FEATURES
rename Failure and split into subclasses for Import and other, and make
it optionally include the name of the file being considered so that
Failure test logs can be more informative
when result prints final result, patch in error classes, eg,
FAILED (todo=10)
or
OK (skipped=2)
** this can only be done in nose test runner
* loader
- support module.callable in addition to module:callable names
- loadTestsFromTestCase -- override to add plugin call
* result/proxy
- don't use sep proxy obj for each test? -- see under PROFILE
BUGS
-- when run vs spine suite, makeTest seemingly was called on an object
that should not have passed the selector --
sqlalchemy.sql._FunctionGateway, which doesn't match testmatch and
isn't a testcase subclass -- need to look into it further.
-- incorrectly defined test class generator method causes failure of whole
test run:
_TextTestResult.printErrors(self)
File "unittest.py", line 667, in printErrors
self.printErrorList('ERROR', self.errors)
File "unittest.py", line 673, in printErrorList
self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
File "unittest.py", line 633, in getDescription
return test.shortDescription() or str(test)
AttributeError: 'ContextSuite' object has no attribute 'shortDescription'
need to add shortDescription() to suite?
2.3 COMPAT
-- 2.3 doctest is just too different. Include 2.5's doctest (under some other
name -- dtcompat.py ? and use that iff SKIP is missing from the
system doctest module
-- fixtures aren't running
-- result patching doesn't seem to be working
CHORES
make setup work with & w/out setuptools
TESTS NEEDED
deprecated at module/module setup
error at module/module setup
plugin api -- comprehensive integration test that runs a suite with a plugin
that tracks calls and ensures that all calls are called with proper
arguments
testid
-- needs tests with doctests
-- doctests from non module files
doctest
-- needs tests from non-module files
PROFILE
need to profile -- on 250 tests with discovery, 0.10-dev is ~ 1 second
slower than 0.9. Profile and optimize.
one possiblity -- instead of instantiating a result proxy for every
test, tag each test with a weakref to the nose.case.Test that wraps
it.
older notes:
not sure how this one can be made to work at the module level, since
import and testrunning take place separately now -- mods a, b, c are all
imported before tests are run in a. probably best to have it work at the
directory and context levels -- that will get pretty close to what it
does in 0.9
it can be made to work only in surrounding contexts that are generators:
hook before
load and yield suite
hook after
needed in 2 places -- loadTestsFromDir, loadTestsFromNames.
So loadTestsFromNames will have to become lazy -- what impact will that
have on unittest compatibility, etc? can it just return a lazysuite?
the problem with loadTestsFromNames is that context teardown requires
that for a given context, all tests within the context are known before
teardown of the first suite within the context. Say you have a package of
tests, a, with subpackages a.b and a.c. User wants to run tests a.b.test
and a.c.test:
nosetests a.b.test a.c.test
'a' and 'b' setup and 'b' teardown run naturally with the 1st test
'c' setup and teardown run natually with the 2nd test. But when, or
how, does 'a' teardown run? If all tests are known, then a.c.test can
run a teardown, because it knows that no more tests in 'a' are going
to run. But if the 2 tests are loaded lazily -- as is REQUIRED for
the isolation plugin -- then 1st test will mistakenly run 'a' teardown,
since it doesn't know as it is torn down that another test in the
same context is about to load.
can't fix this by a call at the end of the loading process, since then
context teardowns may interleave -- 'a' and 'b' may be present at once
can only fix it by forcing full context load & unload around each
test WHEN USING loadTestFromNames -- this is inefficient, but unavoidable
without controlling the *order* of tests passed to loadTestsFromNames.
|
