1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
|
subunit: extensions to python unittest to get test results from subprocesses.
Copyright (C) 2005 Robert Collins <robertc@robertcollins.net>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Subunit is attempting to extend unittest with a clean and simple api to
run arbitrary external test suites and return the results to standard
python unittest.
Using subunit:
1) As a runner for external tests (potentially in other languages)
2) As a process boundary for unittest TestCases to prevent them fiddling with
in-process state (i.e. singletons).
3) As a wrapper around a TestCase (or Suite) to run a group of tests
externally.
1) As a runner for external tests
=================================
This is supported on all platforms with python 2.4.
For each test script you want to run, declare a ExecTestCase with one
or more tests whose docstring defines the script to run:
import subunit
import unittest
class TestCProgram(subunit.ExecTestCase):
def test_script_one(self):
"""./bin/script_one"""
def test_script_two(self):
"""./bin/script_two"""
# Yes, the test prefix on the methods matters.
# Then run this in whatever normal way you would run python unittests.
# If you don't have a specific test runner, you can run it using the
# default one in unittest.py:
if __name__ == '__main__':
unittest.main()
2) As a process boundary for unittest TestCases
===============================================
This is currently supported only on platforms
that support os.fork(), which allows us to
transparently introduce a process boundary
without affecting manual test parameterisation.
*** TODO explain in more detail and sketch out
*** a solution for win32
Just import subunit and derive your test cases
from subunit.IsolatedTestCase:
import subunit
class TestFoo(subunit.IsolatedTestCase):
def test_something_globally(self):
SomethingGlobal.do()
self.assertEqual(SomethingGlobal.value(), 3)
# the run() method of IsolatedTestCase will intercept the
# test execution, fork() python to create a new process,
# then run the test and report the results back to the parent
# process.
# you run this in the normal way you run test cases.
3) As a wrapper around a TestCase to run a group of tests externally.
=====================================================================
import subunit
import unittest
class TestFoo(unittest.TestCase):
def test_foo(self):
...
def test_suite():
result = subunit.IsolatedTestSuite()
loader = unittest.TestLoader()
result.addTestCase(loader.loadTestsFromName(__name__))
return result
# you can test the result of test_suite() as follows (or in any normal python
# manner.
runner = unittest.TextTestRunner(verbosity=2)
runner.run(test_suite())
# enjoy.
Some requirements:
The shape of the external unittest should not need to be known a-priori.
After the test has run, tests should still exist as discrete objects, so that
anything taking a reference to them doesn't get 50 copies of the same object.
TEST: test foo works
SUCCESS: test foo works.
TEST: tar a file.
FAILURE: tar a file. [
..
].. space is eaten.
foo.c:34 WARNING foo is not defined.
]
a writeln to stdout
===========
.F
a writeln to stdout
========================
FAILURE: tar a file.
-------------------
..
].. space is eatern.
foo.c:34 WARNING foo is not defined.
========================
control protocol:
test|testing|test:|testing: test label
success|success:|successful|successful: test label
failure test label
failure: test label
failure test label [
...
]
failure: test label [
...
]
error: test label
error: test label [
]
unexpected output on stdout -> stdout.
exit w/0 or last test -> error
TODO:
def run:
do a fork,
this process runs server
child runs client and calls self.run() with a SubprocessTestResult
|
