diff options
author | Adam Spiers <aspiers@suse.com> | 2019-08-19 11:31:59 +0100 |
---|---|---|
committer | Adam Spiers <aspiers@suse.com> | 2019-08-19 17:48:39 +0100 |
commit | 5df748b2eda5fcf1db4d64d7b19138aba07ba1ea (patch) | |
tree | d4cada9eb32e23901d924475a10c69abdc1fc7d4 /HACKING.rst | |
parent | ee6b69cadc75219aa53bdb91bb71a8b0cedd215b (diff) | |
download | nova-5df748b2eda5fcf1db4d64d7b19138aba07ba1ea.tar.gz |
Make it easier to run a selection of tests relevant to ongoing work
During development of a new git commit, locally running a whole unit
or functional test suite to check every minor code change is
prohibitively expensive. For maximum developer productivity and
happiness, it's generally desirable to make the feedback loop of the
traditional red/green cycle as quick as possible.
So add run-tests-for-diff.sh and run-tests.py to the tools/
subdirectory, using a few tricks as explained below to help with this.
run-tests.py takes a list of files on STDIN, filters the list for
tests which can be run in the current tox virtualenv, and then runs
them with the correct stestr options.
run-tests-for-diff.sh is a simple wrapper around run-tests.py which
determines which tests to run using output from "git diff". This
allows running only the test files changed/added in the working tree:
tools/run-tests-for-diff.sh
or by a single commit:
tools/run-tests-for-diff.sh mybranch^!
or a range of commits, e.g. a branch containing a whole patch series
for a blueprint:
tools/run-tests-for-diff.sh gerrit/master..bp/my-blueprint
It supports the same "-HEAD" invocation syntax as flake8wrap.sh (as
used by the "fast8" tox environment):
tools/run-tests-for-diff.sh -HEAD
run-tests.py uses two tricks to make test runs as quick as possible:
1. It's (already) possible to speed up running of tests by
source'ing the "activate" file for the desired tox virtualenv,
e.g.
source .tox/py36/bin/activate
and then running stestr directly. This saves a few seconds by
skipping the overhead introduced by running tox.
2. When only one test file needs to be run, specifying the -n option
to stestr will skip the costly test discovery phase, saving
several more valuable seconds.
Future commits could build on top of this work, harnessing a framework
such as watchdog / watchmedo[0] or Guard[1] in order to automatically
run relevant tests every time your editor saves changes to a .py file.
[0] https://github.com/gorakhargosh/watchdog - Python-based
[1] https://guardgem.org - probably best in class, but Ruby-based so
maybe unacceptable for use within Nova.
Change-Id: I9a9bda5d29bbb8d8d77f769cd1abf7c42a18c36b
Diffstat (limited to 'HACKING.rst')
-rw-r--r-- | HACKING.rst | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/HACKING.rst b/HACKING.rst index 16664f02fd..f47c8d32f5 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -117,6 +117,30 @@ command directly. Running ``stestr run`` will run the entire test suite. tests in parallel). More information about stestr can be found at: http://stestr.readthedocs.io/ +Since when testing locally, running the entire test suite on a regular +basis is prohibitively expensive, the ``tools/run-tests-for-diff.sh`` +script is provided as a convenient way to run selected tests using +output from ``git diff``. For example, this allows running only the +test files changed/added in the working tree:: + + tools/run-tests-for-diff.sh + +However since it passes its arguments directly to ``git diff``, tests +can be selected in lots of other interesting ways, e.g. it can run all +tests affected by a single commit at the tip of a given branch:: + + tools/run-tests-for-diff.sh mybranch^! + +or all those affected by a range of commits, e.g. a branch containing +a whole patch series for a blueprint:: + + tools/run-tests-for-diff.sh gerrit/master..bp/my-blueprint + +It supports the same ``-HEAD`` invocation syntax as ``flake8wrap.sh`` +(as used by the ``fast8`` tox environment):: + + tools/run-tests-for-diff.sh -HEAD + By default tests log at ``INFO`` level. It is possible to make them log at ``DEBUG`` level by exporting the ``OS_DEBUG`` environment variable to ``True``. |