summaryrefslogtreecommitdiff
path: root/testsuite/driver/testlib.py
Commit message (Collapse)AuthorAgeFilesLines
* Change CaseAlt and LambdaExpr to FunRhs in deriving Foldable and Traversable ↵Artyom Kuznetsov2021-10-261-0/+6
| | | | (#20496)
* hadrian, testsuite: Teach Hadrian to query the testsuite driver for dependenciesZubin Duggal2021-10-131-0/+16
| | | | Issues #19072, #17728, #20176
* testsuite: Clean up dynlib support predicatesBen Gamari2021-10-121-2/+15
| | | | | | | | | | | | | | | | | | Previously it was unclear whether req_shared_libs should require: * that the platform supports dynamic library loading, * that GHC supports dynamic linking of Haskell code, or * that the dyn way libraries were built Clarify by splitting the predicate into two: * `req_dynamic_lib_support` demands that the platform support dynamic linking * `req_dynamic_hs` demands that the GHC support dynamic linking of Haskell code on the target platform Naturally `req_dynamic_hs` cannot be true unless `req_dynamic_lib_support` is also true.
* testsuite: Fix overzealous command-line manglingBen Gamari2021-10-121-1/+1
| | | | | Previously this attempt at suppressing make's -s flag would mangle otherwise valid arguments.
* Improve overlap error for polykinded constraintssheaf2021-10-061-1/+3
| | | | | | | | | | | | | | There were two problems around `mkDictErr`: 1. An outdated call to `flattenTys` meant that we missed out on some instances. As we no longer flatten type-family applications, the logic is obsolete and can be removed. 2. We reported "out of scope" errors in a poly-kinded situation because `BoxedRep` and `Lifted` were considered out of scope. We fix this by using `pretendNameIsInScope`. fixes #20465
* hadrian: Use ghc version as suffix for all executablesMatthew Pickering2021-08-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` [matt@nixos:~/ghc-unique-spin]$ ls _build/bindist/ghc-9.3.20210813-x86_64-unknown-linux/bin/ ghc haddock runghc ghc-9.3.20210813 haddock-ghc-9.3.20210813 runghc-9.3.20210813 ghc-iserv hp2ps runhaskell ghc-iserv-dyn hp2ps-ghc-9.3.20210813 runhaskell-9.3.20210813 ghc-iserv-dyn-ghc-9.3.20210813 hpc unlit ghc-iserv-ghc-9.3.20210813 hpc-ghc-9.3.20210813 unlit-ghc-9.3.20210813 ghc-pkg hsc2hs ghc-pkg-9.3.20210813 hsc2hs-ghc-9.3.20210813 [matt@nixos:~/ghc-unique-spin]$ ls _build/bindist/ghc-9.3.20210813-x86_64-unknown-linux/wrappers/ ghc ghc-pkg-9.3.20210813 hpc runghc-9.3.20210813 ghc-9.3.20210813 haddock hpc-ghc-9.3.20210813 runhaskell ghci haddock-ghc-9.3.20210813 hsc2hs runhaskell-9.3.20210813 ghci-9.3.20210813 hp2ps hsc2hs-ghc-9.3.20210813 ghc-pkg hp2ps-ghc-9.3.20210813 runghc ``` See the discussion on #19571 where we decided that it was most sensible to use the same version number as a suffix for all executables. For those whose version number is different to normal (for example, haddock as it's own versioning scheme) the additional "ghc" suffix is used. Cabal already knows to look for this suffix so should work nicely with existing tooling.
* packaging: Create both versioned and unversioned executablesMatthew Pickering2021-07-271-0/+6
| | | | | | | | | Before we would just copy the unversioned executable into the bindist. Now the actual executable is copied into the bindist and a version suffix is added. Then a wrapper or symlink is added which points to the versioned executable. Fixes #20074
* testsuite: Don't try to run tests with missing librariesMatthew Pickering2021-06-231-6/+6
| | | | | | | | | | | | | | | | As noticed by sgraf, we were still running reqlib tests, even if the library was not available. The reasons for this were not clear to me as they would never work and it was causing some issues with empty stderr files being generated if you used --test-accept. Now if the required library is not there, the test is just skipped, and a counter increased to mark the fact. Perhaps in the future it would be nicer to explicitly record why certain tests are skipped. Missing libraries causing a skip is a special case at the moment. Fixes #20005
* [testsuite] fix T13702 with clangMoritz Angermann2021-06-051-0/+5
|
* Driver Rework PatchMatthew Pickering2021-06-031-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch comprises of four different but closely related ideas. The net result is fixing a large number of open issues with the driver whilst making it simpler to understand. 1. Use the hash of the source file to determine whether the source file has changed or not. This makes the recompilation checking more robust to modern build systems which are liable to copy files around changing their modification times. 2. Remove the concept of a "stable module", a stable module was one where the object file was older than the source file, and all transitive dependencies were also stable. Now we don't rely on the modification time of the source file, the notion of stability is moot. 3. Fix TH/plugin recompilation after the removal of stable modules. The TH recompilation check used to rely on stable modules. Now there is a uniform and simple way, we directly track the linkables which were loaded into the interpreter whilst compiling a module. This is an over-approximation but more robust wrt package dependencies changing. 4. Fix recompilation checking for dynamic object files. Now we actually check if the dynamic object file exists when compiling with -dynamic-too Fixes #19774 #19771 #19758 #17434 #11556 #9121 #8211 #16495 #7277 #16093
* testsuite: Don't copy .hi-boot and .o-boot files into temp dirMatthew Pickering2021-05-191-1/+3
|
* [testsuite] filter out superfluous dylib warningsMoritz Angermann2021-05-071-3/+18
| | | | (cherry picked from commit 33c4d497545559a38bd8d1caf6c94e5e2a77647b)
* [testlib/driver] denoiseMoritz Angermann2021-05-071-4/+8
| | | | | | | | | | this prevents the testlib/driver to be overly noisy, and will also kill some noise produiced by the aarch64-darwin cc (for now). Fixing sysctl, will allow us to run the test's properly in a nix-shell on aarch64-darwin (cherry picked from commit 5109e87e13ab45d799db2013535f54ca35f1f4dc)
* test driver: Make sure RESIDENCY_OPTS is passed for 'all' perf testsMatthew Pickering2021-05-051-11/+28
| | | | | | | | | | | | Fixes #19731 ------------------------- Metric Decrease: T11545 Metric Increase: T12545 T15304 -------------------------
* Revert "[ci/arm/darwin/testsuite] Forwards ports from GHC-8.10"Ben Gamari2021-04-051-25/+6
| | | | This reverts commit 0cbdba2768d84a0f6832ae5cf9ea1e98efd739da.
* testsuite: Check test stats only after test correctnessBen Gamari2021-04-021-5/+5
| | | | | | | | | Ticket #19576 noted that a test that failed in correctness (e.g. due to stderr mismatch) *and* failed due to a metrics change would report misleading stats. This was due to the testsuite driver *first* checking stats, before checking for correctness. Fix this. Closes #19576.
* testsuite: Make passFail a booleanBen Gamari2021-04-021-21/+6
|
* [testlib] ignore strip warningsMoritz Angermann2021-03-291-0/+4
|
* [ci/arm/darwin/testsuite] Forwards ports from GHC-8.10Moritz Angermann2021-03-211-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a set of forward ports (cherry-picks) from 8.10 - a7d22795ed [ci] Add support for building on aarch64-darwin - 5109e87e13 [testlib/driver] denoise - 307d34945b [ci] default value for CONFIGURE_ARGS - 10a18cb4e0 [testsuite] mark ghci056 as fragile - 16c13d5acf [ci] Default value for MAKE_ARGS - ab571457b9 [ci/build] Copy config.sub around - 251892b98f [ci/darwin] bump nixpkgs rev - 5a6c36ecb4 [testsuite/darwin] fix conc059 - aae95ef0c9 [ci] add timing info - 3592d1104c [Aarch64] No div-by-zero; disable test. - 57671071ad [Darwin] mark stdc++ tests as broken - 33c4d49754 [testsuite] filter out superfluous dylib warnings - 4bea83afec [ci/nix-shell] Add Foundation and Security - 6345530062 [testsuite/json2] Fix failure with LLVM backends - c3944bc89d [ci/nix-shell] [Darwin] Stop the ld warnings about libiconv. - b821fcc714 [testsuite] static001 is not broken anymore. - f7062e1b0c [testsuite/arm64] fix section_alignment - 820b076698 [darwin] stop the DYLD_LIBRARY_PATH madness - 07b1af0362 [ci/nix-shell] uniquify NIX_LDFLAGS{_FOR_TARGET} As well as a few additional fixups needed to make this block compile: - Fixup all.T - Set CROSS_TARGET, BROKEN_TESTS, XZ, RUNTEST_ARGS, default value. - [ci] shell.nix bump happy
* gitlab-ci: Ignore performance improvements in marge jobsBen Gamari2021-03-191-12/+13
| | | | | | | | Currently we have far too many merge failures due to cumulative performance improvements. Avoid this by accepting metric decreases in marge-bot jobs. Fixes #19562.
* testsuite: Make --ignore-perf-tests more expressiveBen Gamari2021-03-191-4/+9
| | | | Allow skipping of only increases/decreases.
* testsuite: Introduce flag to ignore performance failuresBen Gamari2021-02-241-1/+1
| | | | Needed by #19025.
* Test Driver: Tweak interval of test reportingMatthew Pickering2021-02-181-5/+12
| | | | | | | | | | | Rather than just display every 100 tests, work out how many to display based on the total number of tests. This improves the experience when running a small number of tests. For [0..100] - Report every test [100..1000] - Report every 10 tests [1000..10000] - Report every 100 tests and so on..
* Reduce default test verbosityMatthew Pickering2021-01-281-1/+2
|
* Deprecate -h flagMatthew Pickering2021-01-271-4/+4
| | | | | | | | | | It is confusing that it defaults to two different things depending on whether we are in the profiling way or not. Use -hc if you have a profiling build Use -hT if you have a normal build Fixes #19031
* Add some additional information to the fail message based on exit codeHécate2021-01-171-2/+16
|
* Make proper fixed-width number literalsSylvain Henry2021-01-021-1/+1
| | | | | | | | (Progress towards #11953, #17377, #17375) Besides being nicer to use, this also will allow for better constant folding for the fixed-width types, on par with what `Int#` and `Word#` have today.
* [Sized Cmm] properly retain sizes.Moritz Angermann2020-11-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces all Word<N> = W<N># Word# and Int<N> = I<N># Int# with Word<N> = W<N># Word<N># and Int<N> = I<N># Int<N>#, thus providing us with properly sized primitives in the codegenerator instead of pretending they are all full machine words. This came up when implementing darwinpcs for arm64. The darwinpcs reqires us to pack function argugments in excess of registers on the stack. While most procedure call standards (pcs) assume arguments are just passed in 8 byte slots; and thus the caller does not know the exact signature to make the call, darwinpcs requires us to adhere to the prototype, and thus have the correct sizes. If we specify CInt in the FFI call, it should correspond to the C int, and not just be Word sized, when it's only half the size. This does change the expected output of T16402 but the new result is no less correct as it eliminates the narrowing (instead of the `and` as was previously done). Bumps the array, bytestring, text, and binary submodules. Co-Authored-By: Ben Gamari <ben@well-typed.com> Metric Increase: T13701 T14697
* CmmToLlvm: Declare signature for memcmpwip/angerman/arm64Ben Gamari2020-11-241-0/+10
| | | | | | Otherwise `opt` fails with: error: use of undefined value '@memcmp$def'
* Merge remote-tracking branch 'origin/wip/tsan/all'Ben Gamari2020-11-081-0/+7
|\
| * testsuite: Skip high memory usage tests with TSANBen Gamari2020-10-241-0/+4
| | | | | | | | | | ThreadSanitizer significantly increases the memory footprint of tests, so much so that it can send machines into OOM.
| * testsuite: Mark setnumcapabilities001 as broken with TSANGHC GitLab CI2020-10-241-0/+3
| | | | | | | | Due to #18808.
* | Testsuite: Support for user supplied package dbsAndreas Klebinger2020-11-041-1/+14
| | | | | | | | | | | | | | | | | | We can now supply additional package dbs to the testsuite. For make the package db can be supplied by passing PACKAGE_DB=/path/to/db. In the testsuite driver it's passed via the --test-package-db argument.
* | testsuite: Add --top flag to driverGHC GitLab CI2020-11-021-1/+1
| | | | | | | | | | This allows us to make `config.top` a proper Path. Previously it was a str, which caused the Ghostscript detection logic to break.
* | Use config.run_ways for multi_compile_and_run testsDavid Eichmann2020-10-271-1/+1
|/
* testsuite: Fix WinIO error message normalizationGHC GitLab CI2020-09-201-4/+6
| | | | This wasn't being applied to stderr.
* testsuite: Allow baseline commit to be set explicitlyBen Gamari2020-08-181-1/+2
|
* testsuite: Normalise WinIO error message differencesBen Gamari2020-07-261-0/+24
| | | | | Previously the old Windows IO manager threw different errors than WinIO. We now canonicalise these to the WinIO errors.
* Add test for #18064Sylvain Henry2020-07-221-0/+5
| | | | It has been fixed by 0effc57d48ace6b719a9f4cbeac67c95ad55010b
* Add tests for #17920Sylvain Henry2020-06-231-2/+1
| | | | | | Metric Decrease: T12150 T12234
* Update testsuiteSylvain Henry2020-06-171-7/+7
| | | | | | | | | | | | | | * support detection of slow ghc-bignum backend (to replace the detection of integer-simple use). There are still some test cases that the native backend doesn't handle efficiently enough. * remove tests for GMP only functions that have been removed from ghc-bignum * fix test results showing dependent packages (e.g. integer-gmp) or showing suggested instances * fix test using Integer/Natural API or showing internal names
* testsuite: Refactor ghostscript detectionBen Gamari2020-06-011-24/+31
| | | | Tamar reported that he saw crashes due to unhandled exceptions.
* Implement cstringLength# and FinalPtrAndrew Martin2020-05-231-0/+20
| | | | | | | | | | | | | | | | | | | | This function and its accompanying rule resolve issue #5218. A future PR to the bytestring library will make the internal Data.ByteString.Internal.unsafePackAddress compute string length with cstringLength#. This will improve the status quo because it is eligible for constant folding. Additionally, introduce a new data constructor to ForeignPtrContents named FinalPtr. This additional data constructor, when used in the IsString instance for ByteString, leads to more Core-to-Core optimization opportunities, fewer runtime allocations, and smaller binaries. Also, this commit re-exports all the functions from GHC.CString (including cstringLength#) in GHC.Exts. It also adds a new test driver. This test driver is used to perform substring matches on Core that is dumped after all the simplifier passes. In this commit, it is used to check that constant folding of cstringLength# works.
* testsuite: Don't attempt to read .std{err,out} files if they don't existBen Gamari2020-04-231-2/+10
| | | | | | | | Simon reports that he was previously seeing framework failures due to an attempt to read the non-existing T13456.stderr. While I don't know exactly what this is due to, it does seem like a non-existing .std{out,err} file should be equivalent to an empty file. Teach the testsuite driver to treat it as such.
* testsuite: Move no_lint to the top level, tweak hie002Ömer Sinan Ağacan2020-04-101-0/+6
| | | | | | | | | | | | | | | | | | | | | | | - We don't want to benchmark linting so disable lints in hie002 perf test - Move no_lint to the top-level to be able to use it in tests other than those in `testsuite/tests/perf/compiler`. - Filter out -dstg-lint in no_lint. - hie002 allocation numbers on 32-bit are unstable, so skip it on 32-bit Metric Decrease: hie002 ManyConstructors T12150 T12234 T13035 T1969 T4801 T9233 T9961
* testsuite: Allow tests to be marked as broken on the command lineBen Gamari2020-02-261-0/+3
| | | | This allows us to work-around distribution-specific breakage easily.
* Fix testsuite driver output (#17847)Vladislav Zavialov2020-02-181-1/+1
|
* testsuite: Assert the opsys names are knownBen Gamari2020-02-141-0/+11
| | | | | | Previously opsys would take any string. This meant it was very easy for a typo to silently render the predicate ineffective. Fix this by checking the given operating system name against a list of known values.
* testlib: Extend existing *_opts in extra_*_optsÖmer Sinan Ağacan2020-02-051-4/+6
| | | | | | | | | | | | Previously we'd override the existing {run,hc} opts in extra_{run,hc}_opts, which caused flakiness in T1969, see #17712. extra_{run,hc}_opts now extends {run,hc} opts, instead of overriding. Also we shrank the allocation area for T1969 in order to increase residency sampling frequency. Fixes #17712
* testsuite: Don't crash on encoding failure in printBen Gamari2020-01-311-8/+12
| | | | | | If the user doesn't use a Unicode locale then the testsuite driver would previously throw framework failures due to encoding failures. We now rather use the `replace` error-handling strategy.
View Lorry Controller Status