1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
|
% Lorry Controller WEBAPP integration test suite
% Codethink Ltd
Introduction
============
This is an integration test suite for the WEBAPP component of Lorry
Controller. It is implemented using the [yarn] tool and uses a style
of automated testing called "scenario testing" by the tool authors.
[yarn]: http://liw.fi/cmdtest/README.yarn/
As an example, here is a scenario that verifies that the Lorry
Controller WEBAPP can be started at all:
SCENARIO WEBAPP can be started at all
WHEN WEBAPP --help is requested
THEN WEBAPP --help exited with a zero exit code
A scenario consists of a sequence of steps that can be executed by a
computer. The steps are then defined using IMPLEMENTS:
IMPLEMENTS WHEN WEBAPP --help is requested
if "$SRCDIR/lorry-controller-webapp" --help
then
exit=0
else
exit=$?
fi
echo "$exit" > "$DATADIR/webapp.exit"
And another:
IMPLEMENTS THEN WEBAPP --help exited with a zero exit code
grep -Fx 0 "$DATADIR/webapp.exit"
Yarn will run each scenario in the order it finds them. If all steps
in a scenario succeed, the scenario succeeds.
Scenarios, though not their implementations, are intended to be
understandable by people who aren't programmers, though some
understanding of the technology is required.
For more information, see the documentation for yarn.
Test environment and setup
==========================
In this chapter, we discuss how the environment is set up for tests to
run in. Yarn provides a temporary directory in which tests can create
temporary directories, and sets the environment variable `$DATADIR` to
point at that directory. Yarn also deletes the directory and all of
its contents at the end, so the test suite itself does not need to do
that.
We put several files into `$DATADIR`.
* The WEBAPP STATEDB database file.
* Responses from HTTP queries to WEBAPP.
* PID of the running WEBAPP.
The purpose of each file is documented with the IMPLEMENTS sections
that use it, typically with the one that creates it.
Since many scenarios will start an instance of WEBAPP, they also need
to make sure it gets killed. There are steps for these (`GIVEN a
running WEBAPP` and `FINALLY WEBAPP is terminated`), which MUST be
used as a pair in each scenario: having only one of these steps is
always a bug in the scenario, whereas having neither is OK.
WEBAPP has stores its persistent state in STATEDB, which is an Sqlite
database on disk. Our tests do _not_ touch it directly, only via WEBAPP,
so that we do not encode in our tests internals of the database, such
as the database schema. We do not care: we only care that WEBAPP
works, and the database schema of STATEDB is _not_ a public interface.
|
