summaryrefslogtreecommitdiff
path: root/testsuite/driver
diff options
context:
space:
mode:
authorBen Gamari <ben@smart-cactus.org>2018-12-16 10:43:11 -0500
committerMarge Bot <ben+marge-bot@smart-cactus.org>2019-10-17 01:03:11 -0400
commitb15a7fb8aa022bfdf3286d65227061c19a4acf28 (patch)
tree95e8f4dcf22302e6ed14ab8a98eabd6ea41505af /testsuite/driver
parent42ebc3f67c8def0f941792b51bd27270ec8e8d88 (diff)
downloadhaskell-b15a7fb8aa022bfdf3286d65227061c19a4acf28.tar.gz
testsuite: Ensure that makefile tests get run
Previously `makefile_test` and `run_command` tests could easily end up in a situation where they wouldn't be run if the user used the `only_ways` modifier. The reason is to build the set of a ways to run the test in we first start with a candidate set determined by the test type (e.g. `makefile_test`, `compile_run`, etc.) and then filter that set with the constraints given by the test's modifiers. `makefile_test` and `run_command` tests' candidate sets were simply `{normal}`, and consequently most uses of `only_ways` would result in the test being never run. To avoid this we rather use all ways as the candidate sets for these test types. This may result in a few more testcases than we would like (given that some `run_command` tests are insensitive to way) but this can be fixed by adding modifiers and we would much rather run too many tests than too few. This fixes #16042 and a number of other tests afflicted by the same issue. However, there were a few cases that required special attention: * `T14028` is currently failing and is therefore marked as broken due to #17300 * `T-signals-child` is fragile in the `threaded1` and `threaded2` ways (tracked in #17307)
Diffstat (limited to 'testsuite/driver')
-rw-r--r--testsuite/driver/testlib.py5
1 files changed, 5 insertions, 0 deletions
diff --git a/testsuite/driver/testlib.py b/testsuite/driver/testlib.py
index 36aaf38076..7fe0630094 100644
--- a/testsuite/driver/testlib.py
+++ b/testsuite/driver/testlib.py
@@ -829,6 +829,11 @@ def test_common_work(watcher: testutil.Watcher,
all_ways = [WayName('ghci')]
else:
all_ways = []
+ elif func in [makefile_test, run_command]:
+ # makefile tests aren't necessarily runtime or compile-time
+ # specific. Assume we can run them in all ways. See #16042 for what
+ # happened previously.
+ all_ways = config.compile_ways + config.run_ways
else:
all_ways = [WayName('normal')]