summaryrefslogtreecommitdiff
path: root/testsuite/driver
Commit message (Collapse)AuthorAgeFilesLines
* Change CaseAlt and LambdaExpr to FunRhs in deriving Foldable and Traversable ↵Artyom Kuznetsov2021-10-261-0/+6
| | | | (#20496)
* ci: test in-tree compiler in hadrianZubin Duggal2021-10-132-4/+5
|
* hadrian, testsuite: Teach Hadrian to query the testsuite driver for dependenciesZubin Duggal2021-10-133-0/+36
| | | | Issues #19072, #17728, #20176
* testsuite: Clean up dynlib support predicatesBen Gamari2021-10-123-7/+35
| | | | | | | | | | | | | | | | | | Previously it was unclear whether req_shared_libs should require: * that the platform supports dynamic library loading, * that GHC supports dynamic linking of Haskell code, or * that the dyn way libraries were built Clarify by splitting the predicate into two: * `req_dynamic_lib_support` demands that the platform support dynamic linking * `req_dynamic_hs` demands that the GHC support dynamic linking of Haskell code on the target platform Naturally `req_dynamic_hs` cannot be true unless `req_dynamic_lib_support` is also true.
* testsuite: Fix overzealous command-line manglingBen Gamari2021-10-121-1/+1
| | | | | Previously this attempt at suppressing make's -s flag would mangle otherwise valid arguments.
* Improve overlap error for polykinded constraintssheaf2021-10-061-1/+3
| | | | | | | | | | | | | | There were two problems around `mkDictErr`: 1. An outdated call to `flattenTys` meant that we missed out on some instances. As we no longer flatten type-family applications, the logic is obsolete and can be removed. 2. We reported "out of scope" errors in a poly-kinded situation because `BoxedRep` and `Lifted` were considered out of scope. We fix this by using `pretendNameIsInScope`. fixes #20465
* hadrian: Use ghc version as suffix for all executablesMatthew Pickering2021-08-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ``` [matt@nixos:~/ghc-unique-spin]$ ls _build/bindist/ghc-9.3.20210813-x86_64-unknown-linux/bin/ ghc haddock runghc ghc-9.3.20210813 haddock-ghc-9.3.20210813 runghc-9.3.20210813 ghc-iserv hp2ps runhaskell ghc-iserv-dyn hp2ps-ghc-9.3.20210813 runhaskell-9.3.20210813 ghc-iserv-dyn-ghc-9.3.20210813 hpc unlit ghc-iserv-ghc-9.3.20210813 hpc-ghc-9.3.20210813 unlit-ghc-9.3.20210813 ghc-pkg hsc2hs ghc-pkg-9.3.20210813 hsc2hs-ghc-9.3.20210813 [matt@nixos:~/ghc-unique-spin]$ ls _build/bindist/ghc-9.3.20210813-x86_64-unknown-linux/wrappers/ ghc ghc-pkg-9.3.20210813 hpc runghc-9.3.20210813 ghc-9.3.20210813 haddock hpc-ghc-9.3.20210813 runhaskell ghci haddock-ghc-9.3.20210813 hsc2hs runhaskell-9.3.20210813 ghci-9.3.20210813 hp2ps hsc2hs-ghc-9.3.20210813 ghc-pkg hp2ps-ghc-9.3.20210813 runghc ``` See the discussion on #19571 where we decided that it was most sensible to use the same version number as a suffix for all executables. For those whose version number is different to normal (for example, haddock as it's own versioning scheme) the additional "ghc" suffix is used. Cabal already knows to look for this suffix so should work nicely with existing tooling.
* testsuite: Add more debug output on failure to call ghc-pkgMatthew Pickering2021-07-271-1/+1
|
* packaging: Create both versioned and unversioned executablesMatthew Pickering2021-07-271-0/+6
| | | | | | | | | Before we would just copy the unversioned executable into the bindist. Now the actual executable is copied into the bindist and a version suffix is added. Then a wrapper or symlink is added which points to the versioned executable. Fixes #20074
* testsuite: Don't try to run tests with missing librariesMatthew Pickering2021-06-232-7/+7
| | | | | | | | | | | | | | | | As noticed by sgraf, we were still running reqlib tests, even if the library was not available. The reasons for this were not clear to me as they would never work and it was causing some issues with empty stderr files being generated if you used --test-accept. Now if the required library is not there, the test is just skipped, and a counter increased to mark the fact. Perhaps in the future it would be nicer to explicitly record why certain tests are skipped. Missing libraries causing a skip is a special case at the moment. Fixes #20005
* [testsuite] fix T13702 with clangMoritz Angermann2021-06-051-0/+5
|
* Driver Rework PatchMatthew Pickering2021-06-033-7/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch comprises of four different but closely related ideas. The net result is fixing a large number of open issues with the driver whilst making it simpler to understand. 1. Use the hash of the source file to determine whether the source file has changed or not. This makes the recompilation checking more robust to modern build systems which are liable to copy files around changing their modification times. 2. Remove the concept of a "stable module", a stable module was one where the object file was older than the source file, and all transitive dependencies were also stable. Now we don't rely on the modification time of the source file, the notion of stability is moot. 3. Fix TH/plugin recompilation after the removal of stable modules. The TH recompilation check used to rely on stable modules. Now there is a uniform and simple way, we directly track the linkables which were loaded into the interpreter whilst compiling a module. This is an over-approximation but more robust wrt package dependencies changing. 4. Fix recompilation checking for dynamic object files. Now we actually check if the dynamic object file exists when compiling with -dynamic-too Fixes #19774 #19771 #19758 #17434 #11556 #9121 #8211 #16495 #7277 #16093
* testsuite: Don't copy .hi-boot and .o-boot files into temp dirMatthew Pickering2021-05-191-1/+3
|
* [darwin] stop the DYLD_LIBRARY_PATH madnessMoritz Angermann2021-05-071-5/+2
| | | | | | | | this causes *significant* slowdown on macOS as the linker ends up looking through all the paths. Slowdown can be as bad as 100% or more. (cherry picked from commit 820b0766984d42c06c977a6c32da75c429106f7f)
* [testsuite] filter out superfluous dylib warningsMoritz Angermann2021-05-071-3/+18
| | | | (cherry picked from commit 33c4d497545559a38bd8d1caf6c94e5e2a77647b)
* [testlib/driver] denoiseMoritz Angermann2021-05-072-6/+11
| | | | | | | | | | this prevents the testlib/driver to be overly noisy, and will also kill some noise produiced by the aarch64-darwin cc (for now). Fixing sysctl, will allow us to run the test's properly in a nix-shell on aarch64-darwin (cherry picked from commit 5109e87e13ab45d799db2013535f54ca35f1f4dc)
* test driver: Make sure RESIDENCY_OPTS is passed for 'all' perf testsMatthew Pickering2021-05-052-12/+33
| | | | | | | | | | | | Fixes #19731 ------------------------- Metric Decrease: T11545 Metric Increase: T12545 T15304 -------------------------
* tests: Allow --skip-perf-tests/--only-perf-tests to be used with ↵Matthew Pickering2021-04-101-1/+1
| | | | --ignore-perf-failures
* Revert "[ci/arm/darwin/testsuite] Forwards ports from GHC-8.10"Ben Gamari2021-04-053-30/+13
| | | | This reverts commit 0cbdba2768d84a0f6832ae5cf9ea1e98efd739da.
* testsuite: Check test stats only after test correctnessBen Gamari2021-04-021-5/+5
| | | | | | | | | Ticket #19576 noted that a test that failed in correctness (e.g. due to stderr mismatch) *and* failed due to a metrics change would report misleading stats. This was due to the testsuite driver *first* checking stats, before checking for correctness. Fix this. Closes #19576.
* testsuite: Make passFail a booleanBen Gamari2021-04-022-24/+13
|
* [testlib] ignore strip warningsMoritz Angermann2021-03-291-0/+4
|
* [ci/arm/darwin/testsuite] Forwards ports from GHC-8.10Moritz Angermann2021-03-213-13/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a set of forward ports (cherry-picks) from 8.10 - a7d22795ed [ci] Add support for building on aarch64-darwin - 5109e87e13 [testlib/driver] denoise - 307d34945b [ci] default value for CONFIGURE_ARGS - 10a18cb4e0 [testsuite] mark ghci056 as fragile - 16c13d5acf [ci] Default value for MAKE_ARGS - ab571457b9 [ci/build] Copy config.sub around - 251892b98f [ci/darwin] bump nixpkgs rev - 5a6c36ecb4 [testsuite/darwin] fix conc059 - aae95ef0c9 [ci] add timing info - 3592d1104c [Aarch64] No div-by-zero; disable test. - 57671071ad [Darwin] mark stdc++ tests as broken - 33c4d49754 [testsuite] filter out superfluous dylib warnings - 4bea83afec [ci/nix-shell] Add Foundation and Security - 6345530062 [testsuite/json2] Fix failure with LLVM backends - c3944bc89d [ci/nix-shell] [Darwin] Stop the ld warnings about libiconv. - b821fcc714 [testsuite] static001 is not broken anymore. - f7062e1b0c [testsuite/arm64] fix section_alignment - 820b076698 [darwin] stop the DYLD_LIBRARY_PATH madness - 07b1af0362 [ci/nix-shell] uniquify NIX_LDFLAGS{_FOR_TARGET} As well as a few additional fixups needed to make this block compile: - Fixup all.T - Set CROSS_TARGET, BROKEN_TESTS, XZ, RUNTEST_ARGS, default value. - [ci] shell.nix bump happy
* gitlab-ci: Ignore performance improvements in marge jobsBen Gamari2021-03-191-12/+13
| | | | | | | | Currently we have far too many merge failures due to cumulative performance improvements. Avoid this by accepting metric decreases in marge-bot jobs. Fixes #19562.
* testsuite: Make --ignore-perf-tests more expressiveBen Gamari2021-03-193-7/+20
| | | | Allow skipping of only increases/decreases.
* testsuite: Introduce flag to ignore performance failuresBen Gamari2021-02-243-1/+6
| | | | Needed by #19025.
* Test Driver: Tweak interval of test reportingMatthew Pickering2021-02-181-5/+12
| | | | | | | | | | | Rather than just display every 100 tests, work out how many to display based on the total number of tests. This improves the experience when running a small number of tests. For [0..100] - Report every test [100..1000] - Report every 10 tests [1000..10000] - Report every 100 tests and so on..
* Fix typosBrian Wignall2021-02-062-2/+2
|
* Reduce default test verbosityMatthew Pickering2021-01-283-3/+4
|
* Deprecate -h flagMatthew Pickering2021-01-271-4/+4
| | | | | | | | | | It is confusing that it defaults to two different things depending on whether we are in the profiling way or not. Use -hc if you have a profiling build Use -hT if you have a normal build Fixes #19031
* Add some additional information to the fail message based on exit codeHécate2021-01-171-2/+16
|
* Make proper fixed-width number literalsSylvain Henry2021-01-021-1/+1
| | | | | | | | (Progress towards #11953, #17377, #17375) Besides being nicer to use, this also will allow for better constant folding for the fixed-width types, on par with what `Int#` and `Word#` have today.
* Bump the # of commits searched for perf baselineRichard Eisenberg2020-12-011-1/+1
| | | | | | | | The previous value of 75 meant that a feature branch with more than 75 commits would get spurious CI passes. This affects #18692, but does not fix that ticket, because if a baseline cannot be found, we should fail, not succeed.
* [Sized Cmm] properly retain sizes.Moritz Angermann2020-11-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces all Word<N> = W<N># Word# and Int<N> = I<N># Int# with Word<N> = W<N># Word<N># and Int<N> = I<N># Int<N>#, thus providing us with properly sized primitives in the codegenerator instead of pretending they are all full machine words. This came up when implementing darwinpcs for arm64. The darwinpcs reqires us to pack function argugments in excess of registers on the stack. While most procedure call standards (pcs) assume arguments are just passed in 8 byte slots; and thus the caller does not know the exact signature to make the call, darwinpcs requires us to adhere to the prototype, and thus have the correct sizes. If we specify CInt in the FFI call, it should correspond to the C int, and not just be Word sized, when it's only half the size. This does change the expected output of T16402 but the new result is no less correct as it eliminates the narrowing (instead of the `and` as was previously done). Bumps the array, bytestring, text, and binary submodules. Co-Authored-By: Ben Gamari <ben@well-typed.com> Metric Increase: T13701 T14697
* CmmToLlvm: Declare signature for memcmpwip/angerman/arm64Ben Gamari2020-11-241-0/+10
| | | | | | Otherwise `opt` fails with: error: use of undefined value '@memcmp$def'
* Merge remote-tracking branch 'origin/wip/tsan/all'Ben Gamari2020-11-082-0/+10
|\
| * testsuite: Skip high memory usage tests with TSANBen Gamari2020-10-241-0/+4
| | | | | | | | | | ThreadSanitizer significantly increases the memory footprint of tests, so much so that it can send machines into OOM.
| * testsuite: Mark setnumcapabilities001 as broken with TSANGHC GitLab CI2020-10-242-0/+6
| | | | | | | | Due to #18808.
* | Testsuite: Support for user supplied package dbsAndreas Klebinger2020-11-043-1/+21
| | | | | | | | | | | | | | | | | | We can now supply additional package dbs to the testsuite. For make the package db can be supplied by passing PACKAGE_DB=/path/to/db. In the testsuite driver it's passed via the --test-package-db argument.
* | testsuite: Add --top flag to driverGHC GitLab CI2020-11-023-3/+8
| | | | | | | | | | This allows us to make `config.top` a proper Path. Previously it was a str, which caused the Ghostscript detection logic to break.
* | Use config.run_ways for multi_compile_and_run testsDavid Eichmann2020-10-271-1/+1
|/
* testsuite: Sort metrics by metric typeBen Gamari2020-10-161-1/+15
| | | | Closes #18838.
* testsuite: Allow whitespace before "Metric (in|de)crease"Ben Gamari2020-10-071-1/+1
| | | | | | | Several people have struggled with metric change annotations in their commit messages not being recognized due to the fact that GitLab's job log inserts a space at the beginning of each line. Teach the regular expression to accept this whitespace.
* testsuite: Fix WinIO error message normalizationGHC GitLab CI2020-09-201-4/+6
| | | | This wasn't being applied to stderr.
* Enhance metrics outputSylvain Henry2020-09-152-10/+11
|
* Make sure we can read past perf notesKrzysztof Gogolewski2020-09-121-1/+9
| | | | See #18656.
* testsuite: Output performance test results in tabular formatDaishi Nakajima2020-09-083-35/+98
| | | | | | | | | | | | | | | | | this was suggested in #18417. Change the print format of the values. * Shorten commit hash * Reduce precision of the "Value" field * Shorten metrics name * e.g. runtime/bytes allocated -> run/alloc * Shorten "MetricsChange" * e.g. unchanged -> unch, increased -> incr And, print the baseline environment if there are baselines that were measured in a different environment than the current environment. If all "Baseline commit" are the same, print it once.
* testsuite: Only run llvm ways if llc is availableBen Gamari2020-08-181-0/+3
| | | | | | | | | As noted in #18560, we previously would always run the LLVM ways since `configure` would set `SettingsLlcCommand` to something non-null when it otherwise couldn't find the `llc` executable. Now we rather probe for the existence of the `llc` executable in the testsuite driver. Fixes #18560.
* testsuite: Allow baseline commit to be set explicitlyBen Gamari2020-08-184-15/+32
|
* testsuite: Normalise WinIO error message differencesBen Gamari2020-07-261-0/+24
| | | | | Previously the old Windows IO manager threw different errors than WinIO. We now canonicalise these to the WinIO errors.
View Lorry Controller Status