1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
|
Automated testing
=================
Introduction
------------
The bash-completion package contains an automated test suite. Running the
tests should help verifying that bash-completion works as expected. The tests
are also very helpful in uncovering software regressions at an early stage.
The original, "legacy" bash-completion test suite is written on top of the
http://www.gnu.org/software/dejagnu/[DejaGnu] testing framework. DejaGnu is
written in http://expect.nist.gov[Expect], which in turn uses
http://tcl.sourceforge.net[Tcl] -- Tool command language.
Most of the test framework has been ported over to use
https://pytest.org/[pytest] and https://pexpect.readthedocs.io/[pexpect].
Eventually, all of it should be ported.
Coding Style Guide
------------------
For the Python part, all of it is formatted using
https://github.com/ambv/black[Black], and we also run
http://flake8.pycqa.org/[Flake8] on it.
The legacy test suite tries to adhere to this
http://wiki.tcl.tk/708[Tcl Style Guide].
Installing dependencies
-----------------------
Installing dependencies should be easy using your local package manager or
`pip`. Python 3.4 or newer is required, and the rest of the Python package
dependencies are specified in the `test/requirements.txt` file. If using `pip`,
this file can be fed directly to it, e.g. like:
------------------------------------
pip install -r test/requirements.txt
------------------------------------
Debian/Ubuntu
~~~~~~~~~~~~~
On Debian/Ubuntu you can use `apt-get`:
-------------
sudo apt-get install python3-pytest python3-pexpect dejagnu tcllib
-------------
This should also install the necessary dependencies. Only Debian testing
(buster) and Ubuntu 18.10 (cosmic) and later have an appropriate version
of pytest in the repositories.
Fedora/RHEL/CentOS
~~~~~~~~~~~~~~~~~~
On Fedora and RHEL/CentOS (with EPEL) you can try `yum` or `dnf`:
-------------
sudo yum install python3-pytest python3-pexpect dejagnu tcllib
-------------
This should also install the necessary dependencies. At time of writing, only
Fedora 29 comes with recent enough pytest.
Structure
---------
Pytest tests are in the `t/` subdirectory, with `t/test_\*.py` being
completion tests, and `t/unit/test_unit_\*.py` unit tests.
Legacy tests are grouped into different areas, called _tool_ in DejaGnu:
*completion*::
Functional tests per completion.
*install*::
Functional tests for installation and caching of the main bash-completion
package.
*unit*::
Unit tests for bash-completion helper functions.
Running the tests
-----------------
Python based tests are run by calling `pytest` on the desired test directories
or individual files, for example in the project root directory:
-----------------------
pytest test/t
-----------------------
Legacy tests are run by calling `runtest` command in the test directory:
-----------------------
runtest --outdir log --tool completion
runtest --outdir log --tool install
runtest --outdir log --tool unit
-----------------------
The commands above are already wrapped up in shell scripts within the `test`
directory:
-----------------------
./runCompletion
./runInstall
./runUnit
-----------------------
To run a particular test, specify file name of your test as an argument to
`runCompletion` script:
-----------------------
./runCompletion ssh.exp
-----------------------
That will run `test/completion/ssh.exp`.
See `test/docker/docker-script.sh` for how and what we run and test in CI.
Specifying bash binary
~~~~~~~~~~~~~~~~~~~~~~
The test suite standard uses `bash` as found in PATH. Export the
`bashcomp_bash` environment variable with a path to another bash executable if
you want to test against something else.
Maintenance
-----------
Adding a completion test
~~~~~~~~~~~~~~~~~~~~~~~~
You can run `cd test && ./generate cmd` to add a test for the `cmd`
command. Additional arguments will be passed to the first generated test case.
This will add the `test/t/test_cmd.py` file with a very basic test, and add it
to `test/t/Makefile.am`. Add additional tests to the generated file.
Rationale
---------
Naming conventions
~~~~~~~~~~~~~~~~~~
Test suite or testsuite
^^^^^^^^^^^^^^^^^^^^^^^
The primary Wikipedia page is called
http://en.wikipedia.org/wiki/Test_suite[test suite] and not testsuite, so
that's what this document sticks to.
script/generate
^^^^^^^^^^^^^^^
The name and location of this code generation script come from Ruby on Rails'
http://en.wikibooks.org/wiki/Ruby_on_Rails/Tools/Generators[script/generate].
|
