diff options
| author | Tristan Van Berkom <tristan.vanberkom@codethink.co.uk> | 2019-01-08 14:22:58 -0500 |
|---|---|---|
| committer | Tristan Van Berkom <tristan.vanberkom@codethink.co.uk> | 2019-01-09 14:51:14 -0500 |
| commit | f29a099580cd311517701da4bd125fa83c193f66 (patch) | |
| tree | a213ed0d239634fab2c28ea75bbbb2eaa287c1a1 /.gitlab-ci.yml | |
| parent | ed8bc5bc7c0882c67ac925e4b042996f983ba3a5 (diff) | |
| download | buildstream-f29a099580cd311517701da4bd125fa83c193f66.tar.gz | |
Support running test environments in parallel with `detox`
This patch namespaces the test temp directory and the output
coverage report file with the name of the environment under test,
such that separately run tests do not access the same files.
When running tests without tox, directly through setup.py,
then the tmp directory will still be `./tmp`.
* .gitignore: Added new .coverage-reports/ directory
* .gitlab-ci.yml: Rely on tox to combine and report coverage, only
tell tox about the COVERAGE_PREFIX so that results can be namespaced
by CI job name.
This change also publishes the sources and final combined `.coverage`
file in an output gitlab artifact for inspection, and lists some missing
dependencies to the `coverage` job.
* tox.ini: Add comments and refactor main [testenv] section so that
other environments dont inherit too much unrelated cruft.
Generate the coverate reports in the respective {envtmpdir} so that
all per-process coverage files are prefixed with a full path, ensuring
that concurrent runs don't mix reports and addressing concerns
raised in #844.
Also implemented new `tox -e coverage` environment to combine
any found coverage and print a report.
* .coveragerc: Omit .tox/ directory from coverage stats
Fixes issue #844
Diffstat (limited to '.gitlab-ci.yml')
| -rw-r--r-- | .gitlab-ci.yml | 28 |
1 files changed, 11 insertions, 17 deletions
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9706037a4..78e725da8 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -13,6 +13,7 @@ variables: PYTEST_ADDOPTS: "--color=yes" INTEGRATION_CACHE: "${CI_PROJECT_DIR}/cache/integration-cache" TEST_COMMAND: "tox -- --color=yes --integration" + COVERAGE_PREFIX: "${CI_JOB_NAME}." ##################################################### @@ -24,9 +25,6 @@ variables: .tests-template: &tests stage: test - variables: - COVERAGE_DIR: coverage-linux - before_script: # Diagnostics - mount @@ -40,14 +38,11 @@ variables: - su buildstream -c "${TEST_COMMAND}" after_script: - # Collect our reports - - mkdir -p ${COVERAGE_DIR} - - cp .coverage ${COVERAGE_DIR}/coverage."${CI_JOB_NAME}" except: - schedules artifacts: paths: - - ${COVERAGE_DIR} + - .coverage-reports tests-debian-9: image: buildstream/testsuite-debian:9-5da27168-32c47d1c @@ -83,7 +78,6 @@ tests-unix: <<: *tests variables: BST_FORCE_BACKEND: "unix" - COVERAGE_DIR: coverage-unix script: @@ -239,22 +233,22 @@ coverage: stage: post coverage: '/TOTAL +\d+ +\d+ +(\d+\.\d+)%/' script: - - pip3 install -r requirements/requirements.txt -r requirements/dev-requirements.txt - - pip3 install --no-index . - - mkdir report - - cd report - - cp ../coverage-unix/coverage.* . - - cp ../coverage-linux/coverage.* . - - ls coverage.* - - coverage combine --rcfile=../.coveragerc -a coverage.* - - coverage report --rcfile=../.coveragerc -m + - cp -a .coverage-reports/ ./coverage-sources + - tox -e coverage + - cp -a .coverage-reports/ ./coverage-report dependencies: - tests-debian-9 - tests-fedora-27 - tests-fedora-28 + - tests-fedora-missing-deps + - tests-ubuntu-18.04 - tests-unix except: - schedules + artifacts: + paths: + - coverage-sources/ + - coverage-report/ # Deploy, only for merges which land on master branch. # |