From 8ae04283213f14c711922fab8e55a1cdc97903ce Mon Sep 17 00:00:00 2001 From: Tristan Van Berkom Date: Wed, 26 Dec 2018 20:18:08 -0500 Subject: CONTRIBUTING.rst: Updated to reflect running tests using tox. --- CONTRIBUTING.rst | 52 ++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 40 insertions(+), 12 deletions(-) (limited to 'CONTRIBUTING.rst') diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 12f61fc5f..e80fd3af4 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -1478,48 +1478,76 @@ Don't get lost in the docs if you don't need to, follow existing examples instea Running tests ~~~~~~~~~~~~~ -To run the tests, just type:: +We use `tox `_ as a frontend run the tests which +are implemented using `pytest `_. To run the tests, simply +navigate to the toplevel directory of your buildstream checkout and run:: - ./setup.py test + tox -At the toplevel. +By default, the test suite will be run against every supported python version +found on your host. If you have multiple python versions installed, you may +want to run tests against only one version and you can do that using the ``-e`` +option when running tox:: -When debugging a test, it can be desirable to see the stdout -and stderr generated by a test, to do this use the ``--addopts`` -function to feed arguments to pytest as such:: + tox -e py37 - ./setup.py test --addopts -s +The output of all failing tests will always be printed in the summary, but +if you want to observe the stdout and stderr generated by a passing test, +you can pass the ``-s`` option to pytest as such:: + + tox -- -s + +.. tip:: + + The ``-s`` option is `a pytest option `_. + + Any options specified before the ``--`` separator are consumed by ``tox``, + and any options after the ``--`` separator will be passed along to pytest. You can always abort on the first failure by running:: - ./setup.py test --addopts -x + tox -- -x If you want to run a specific test or a group of tests, you can specify a prefix to match. E.g. if you want to run all of the frontend tests you can do:: - ./setup.py test --addopts 'tests/frontend/' + tox -- tests/frontend/ Specific tests can be chosen by using the :: delimeter after the test module. If you wanted to run the test_build_track test within frontend/buildtrack.py you could do:: - ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' + tox -- tests/frontend/buildtrack.py::test_build_track We also have a set of slow integration tests that are disabled by default - you will notice most of them marked with SKIP in the pytest output. To run them, you can use:: - ./setup.py test --addopts '--integration' + tox -- --integration By default, buildstream also runs pylint on all files. Should you want to run just pylint (these checks are a lot faster), you can do so with:: - ./setup.py test --addopts '-m pylint' + tox -- -m pylint Alternatively, any IDE plugin that uses pytest should automatically detect the ``.pylintrc`` in the project's root directory. +.. note:: + + While using ``tox`` is practical for developers running tests in + more predictable execution environments, it is still possible to + execute the test suite against a specific installation environment + using pytest directly:: + + ./setup.py test + + Specific options can be passed to ``pytest`` using the ``--addopts`` + option:: + + ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track' + Adding tests ~~~~~~~~~~~~ -- cgit v1.2.1