summaryrefslogtreecommitdiff
path: root/testsuite/driver/testlib.py
Commit message (Collapse)AuthorAgeFilesLines
* testlib: Extend existing *_opts in extra_*_optsÖmer Sinan Ağacan2020-02-051-4/+6
| | | | | | | | | | | | Previously we'd override the existing {run,hc} opts in extra_{run,hc}_opts, which caused flakiness in T1969, see #17712. extra_{run,hc}_opts now extends {run,hc} opts, instead of overriding. Also we shrank the allocation area for T1969 in order to increase residency sampling frequency. Fixes #17712
* testsuite: Don't crash on encoding failure in printBen Gamari2020-01-311-8/+12
| | | | | | If the user doesn't use a Unicode locale then the testsuite driver would previously throw framework failures due to encoding failures. We now rather use the `replace` error-handling strategy.
* testsuite: Preserve more information in framework failuresBen Gamari2020-01-201-1/+1
| | | | | Namely print the entire exception in hopes that this will help track down #17649.
* Fix typos, via a Levenshtein-style correctorBrian Wignall2020-01-041-1/+1
|
* Implement pointer tagging for big families (#14373)Gabor Greif2019-12-061-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Formerly we punted on these and evaluated constructors always got a tag of 1. We now cascade switches because we have to check the tag first and when it is MAX_PTR_TAG then get the precise tag from the info table and switch on that. The only technically tricky part is that the default case needs (logical) duplication. To do this we emit an extra label for it and branch to that from the second switch. This avoids duplicated codegen. Here's a simple example of the new code gen: data D = D1 | D2 | D3 | D4 | D5 | D6 | D7 | D8 On a 64-bit system previously all constructors would be tagged 1. With the new code gen D7 and D8 are tagged 7: [Lib.D7_con_entry() { ... {offset c1eu: // global R1 = R1 + 7; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; } }] [Lib.D8_con_entry() { ... {offset c1ez: // global R1 = R1 + 7; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; } }] When switching we now look at the info table only when the tag is 7. For example, if we derive Enum for the type above, the Cmm looks like this: c2Le: _s2Js::P64 = R1; _c2Lq::P64 = _s2Js::P64 & 7; switch [1 .. 7] _c2Lq::P64 { case 1 : goto c2Lk; case 2 : goto c2Ll; case 3 : goto c2Lm; case 4 : goto c2Ln; case 5 : goto c2Lo; case 6 : goto c2Lp; case 7 : goto c2Lj; } // Read info table for tag c2Lj: _c2Lv::I64 = %MO_UU_Conv_W32_W64(I32[I64[_s2Js::P64 & (-8)] - 4]); if (_c2Lv::I64 != 6) goto c2Lu; else goto c2Lt; Generated Cmm sizes do not change too much, but binaries are very slightly larger, due to the fact that the new instructions are longer in encoded form. E.g. previously entry code for D8 above would be 00000000000001c0 <Lib_D8_con_info>: 1c0: 48 ff c3 inc %rbx 1c3: ff 65 00 jmpq *0x0(%rbp) With this patch 00000000000001d0 <Lib_D8_con_info>: 1d0: 48 83 c3 07 add $0x7,%rbx 1d4: ff 65 00 jmpq *0x0(%rbp) This is one byte longer. Secondly, reading info table directly and then switching is shorter _c1co: movq -1(%rbx),%rax movl -4(%rax),%eax // Switch on info table tag jmp *_n1d5(,%rax,8) than doing the same switch, and then for the tag 7 doing another switch: // When tag is 7 _c1ct: andq $-8,%rbx movq (%rbx),%rax movl -4(%rax),%eax // Switch on info table tag ... Some changes of binary sizes in actual programs: - In NoFib the worst case is 0.1% increase in benchmark "parser" (see NoFib results below). All programs get slightly larger. - Stage 2 compiler size does not change. - In "containers" (the library) size of all object files increases 0.0005%. Size of the test program "bitqueue-properties" increases 0.03%. nofib benchmarks kindly provided by Ömer (@osa1): NoFib Results ============= -------------------------------------------------------------------------------- Program Size Allocs Instrs Reads Writes -------------------------------------------------------------------------------- CS +0.0% 0.0% -0.0% -0.0% -0.0% CSD +0.0% 0.0% 0.0% +0.0% +0.0% FS +0.0% 0.0% 0.0% +0.0% 0.0% S +0.0% 0.0% -0.0% 0.0% 0.0% VS +0.0% 0.0% -0.0% +0.0% +0.0% VSD +0.0% 0.0% -0.0% +0.0% -0.0% VSM +0.0% 0.0% 0.0% 0.0% 0.0% anna +0.0% 0.0% +0.1% -0.9% -0.0% ansi +0.0% 0.0% -0.0% +0.0% +0.0% atom +0.0% 0.0% 0.0% 0.0% 0.0% awards +0.0% 0.0% -0.0% +0.0% 0.0% banner +0.0% 0.0% -0.0% +0.0% 0.0% bernouilli +0.0% 0.0% +0.0% +0.0% +0.0% binary-trees +0.0% 0.0% -0.0% -0.0% -0.0% boyer +0.0% 0.0% +0.0% 0.0% -0.0% boyer2 +0.0% 0.0% +0.0% 0.0% -0.0% bspt +0.0% 0.0% +0.0% +0.0% 0.0% cacheprof +0.0% 0.0% +0.1% -0.8% 0.0% calendar +0.0% 0.0% -0.0% +0.0% -0.0% cichelli +0.0% 0.0% +0.0% 0.0% 0.0% circsim +0.0% 0.0% -0.0% -0.1% -0.0% clausify +0.0% 0.0% +0.0% +0.0% 0.0% comp_lab_zift +0.0% 0.0% +0.0% 0.0% -0.0% compress +0.0% 0.0% +0.0% +0.0% 0.0% compress2 +0.0% 0.0% 0.0% 0.0% 0.0% constraints +0.0% 0.0% -0.0% -0.0% -0.0% cryptarithm1 +0.0% 0.0% +0.0% 0.0% 0.0% cryptarithm2 +0.0% 0.0% +0.0% -0.0% 0.0% cse +0.0% 0.0% +0.0% +0.0% 0.0% digits-of-e1 +0.0% 0.0% -0.0% -0.0% -0.0% digits-of-e2 +0.0% 0.0% +0.0% -0.0% -0.0% dom-lt +0.0% 0.0% +0.0% +0.0% 0.0% eliza +0.0% 0.0% -0.0% +0.0% 0.0% event +0.0% 0.0% -0.0% -0.0% -0.0% exact-reals +0.0% 0.0% +0.0% +0.0% +0.0% exp3_8 +0.0% 0.0% -0.0% -0.0% -0.0% expert +0.0% 0.0% +0.0% +0.0% +0.0% fannkuch-redux +0.0% 0.0% +0.0% 0.0% 0.0% fasta +0.0% 0.0% -0.0% -0.0% -0.0% fem +0.0% 0.0% +0.0% +0.0% +0.0% fft +0.0% 0.0% +0.0% -0.0% -0.0% fft2 +0.0% 0.0% +0.0% +0.0% +0.0% fibheaps +0.0% 0.0% +0.0% +0.0% 0.0% fish +0.0% 0.0% +0.0% +0.0% 0.0% fluid +0.0% 0.0% +0.0% +0.0% +0.0% fulsom +0.0% 0.0% +0.0% -0.0% +0.0% gamteb +0.0% 0.0% +0.0% -0.0% -0.0% gcd +0.0% 0.0% +0.0% +0.0% 0.0% gen_regexps +0.0% 0.0% +0.0% -0.0% -0.0% genfft +0.0% 0.0% -0.0% -0.0% -0.0% gg +0.0% 0.0% 0.0% -0.0% 0.0% grep +0.0% 0.0% +0.0% +0.0% +0.0% hidden +0.0% 0.0% +0.0% -0.0% -0.0% hpg +0.0% 0.0% +0.0% -0.1% -0.0% ida +0.0% 0.0% +0.0% -0.0% -0.0% infer +0.0% 0.0% -0.0% -0.0% -0.0% integer +0.0% 0.0% -0.0% -0.0% -0.0% integrate +0.0% 0.0% 0.0% +0.0% 0.0% k-nucleotide +0.0% 0.0% -0.0% -0.0% -0.0% kahan +0.0% 0.0% -0.0% -0.0% -0.0% knights +0.0% 0.0% +0.0% -0.0% -0.0% lambda +0.0% 0.0% +1.2% -6.1% -0.0% last-piece +0.0% 0.0% +0.0% -0.0% -0.0% lcss +0.0% 0.0% +0.0% -0.0% -0.0% life +0.0% 0.0% +0.0% -0.0% -0.0% lift +0.0% 0.0% +0.0% +0.0% 0.0% linear +0.0% 0.0% +0.0% +0.0% +0.0% listcompr +0.0% 0.0% -0.0% -0.0% -0.0% listcopy +0.0% 0.0% -0.0% -0.0% -0.0% maillist +0.0% 0.0% +0.0% -0.0% -0.0% mandel +0.0% 0.0% +0.0% +0.0% +0.0% mandel2 +0.0% 0.0% +0.0% +0.0% -0.0% mate +0.0% 0.0% +0.0% +0.0% +0.0% minimax +0.0% 0.0% -0.0% +0.0% -0.0% mkhprog +0.0% 0.0% +0.0% +0.0% +0.0% multiplier +0.0% 0.0% 0.0% +0.0% -0.0% n-body +0.0% 0.0% +0.0% -0.0% -0.0% nucleic2 +0.0% 0.0% +0.0% +0.0% -0.0% para +0.0% 0.0% +0.0% +0.0% +0.0% paraffins +0.0% 0.0% +0.0% +0.0% +0.0% parser +0.1% 0.0% +0.4% -1.7% -0.0% parstof +0.0% 0.0% -0.0% -0.0% -0.0% pic +0.0% 0.0% +0.0% 0.0% -0.0% pidigits +0.0% 0.0% -0.0% -0.0% -0.0% power +0.0% 0.0% +0.0% -0.0% -0.0% pretty +0.0% 0.0% +0.0% +0.0% +0.0% primes +0.0% 0.0% +0.0% 0.0% 0.0% primetest +0.0% 0.0% +0.0% +0.0% +0.0% prolog +0.0% 0.0% +0.0% +0.0% +0.0% puzzle +0.0% 0.0% +0.0% +0.0% +0.0% queens +0.0% 0.0% 0.0% +0.0% +0.0% reptile +0.0% 0.0% +0.0% +0.0% 0.0% reverse-complem +0.0% 0.0% -0.0% -0.0% -0.0% rewrite +0.0% 0.0% +0.0% 0.0% -0.0% rfib +0.0% 0.0% +0.0% +0.0% +0.0% rsa +0.0% 0.0% +0.0% +0.0% +0.0% scc +0.0% 0.0% +0.0% +0.0% +0.0% sched +0.0% 0.0% +0.0% +0.0% +0.0% scs +0.0% 0.0% +0.0% +0.0% 0.0% simple +0.0% 0.0% +0.0% +0.0% +0.0% solid +0.0% 0.0% +0.0% +0.0% 0.0% sorting +0.0% 0.0% +0.0% -0.0% 0.0% spectral-norm +0.0% 0.0% -0.0% -0.0% -0.0% sphere +0.0% 0.0% +0.0% -1.0% 0.0% symalg +0.0% 0.0% +0.0% +0.0% +0.0% tak +0.0% 0.0% +0.0% +0.0% +0.0% transform +0.0% 0.0% +0.4% -1.3% +0.0% treejoin +0.0% 0.0% +0.0% -0.0% 0.0% typecheck +0.0% 0.0% -0.0% +0.0% 0.0% veritas +0.0% 0.0% +0.0% -0.1% +0.0% wang +0.0% 0.0% +0.0% +0.0% +0.0% wave4main +0.0% 0.0% +0.0% 0.0% -0.0% wheel-sieve1 +0.0% 0.0% +0.0% +0.0% +0.0% wheel-sieve2 +0.0% 0.0% +0.0% +0.0% 0.0% x2n1 +0.0% 0.0% +0.0% +0.0% 0.0% -------------------------------------------------------------------------------- Min +0.0% 0.0% -0.0% -6.1% -0.0% Max +0.1% 0.0% +1.2% +0.0% +0.0% Geometric Mean +0.0% -0.0% +0.0% -0.1% -0.0% NoFib GC Results ================ -------------------------------------------------------------------------------- Program Size Allocs Instrs Reads Writes -------------------------------------------------------------------------------- circsim +0.0% 0.0% -0.0% -0.0% -0.0% constraints +0.0% 0.0% -0.0% 0.0% -0.0% fibheaps +0.0% 0.0% 0.0% -0.0% -0.0% fulsom +0.0% 0.0% 0.0% -0.6% -0.0% gc_bench +0.0% 0.0% 0.0% 0.0% -0.0% hash +0.0% 0.0% -0.0% -0.0% -0.0% lcss +0.0% 0.0% 0.0% -0.0% 0.0% mutstore1 +0.0% 0.0% 0.0% -0.0% -0.0% mutstore2 +0.0% 0.0% +0.0% -0.0% -0.0% power +0.0% 0.0% -0.0% 0.0% -0.0% spellcheck +0.0% 0.0% -0.0% -0.0% -0.0% -------------------------------------------------------------------------------- Min +0.0% 0.0% -0.0% -0.6% -0.0% Max +0.0% 0.0% +0.0% 0.0% 0.0% Geometric Mean +0.0% +0.0% +0.0% -0.1% +0.0% Fixes #14373 These performance regressions appear to be a fluke in CI. See the discussion in !1742 for details. Metric Increase: T6048 T12234 T12425 Naperian T12150 T5837 T13035
* testsuite: Make performance metric summary more readableBen Gamari2019-12-051-2/+2
| | | | Along with some refactoring.
* testsuite: Factor out terminal coloringBen Gamari2019-12-051-11/+10
|
* Fix typos, using Wikipedia list of common typosBrian Wignall2019-11-281-1/+1
|
* testsuite: don't collect compiler stats in collect_runtime_residencyAlp Mestanogullari2019-11-121-1/+1
| | | | | | | | | | | | | | | | | | | We instead want to collect the runtime stats (with collect_stats, instead of collect_compiler_stats). This should fix a number of perf tests failures we have been seeing, where we suddenly started measuring metrics we didn't intend to measure, which tend to fall outside of the acceptance window. Metric Decrease: lazy-bs-alloc T3586 Metric Increase: space_leak_001 T4801 T5835 T12791
* testsuite: Don't check_stats at runtime if not requestedwip/T17387Ben Gamari2019-11-101-4/+9
| | | | | | Previously we would call check_stats to check the runtime metrics even if the test definition hadn't requested it. This would result in an error since the .stats file doesn't exist.
* testsuite: Use small allocation area when measuring residencyBen Gamari2019-11-101-0/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | As suggested in #17387; this helps reduce the variance in our residency sampling. Metric Increase: T10370 T3586 lazy-bs-alloc Metric Decrease 'compile_time/peak_megabytes_allocated': T1969 Metric Decrease 'runtime/bytes allocated': space_leak_001 Metric Increase 'compile_time/bytes allocated': T1969 Metric Increase 'runtime/peak_megabytes_allocated': space_leak_001 Metric Decrease: T3064 T9675
* testsuite: Fix Windows cleanup pathBen Gamari2019-11-091-2/+2
| | | | This was a regression introduced with the Path refactoring.
* Testsuite: Introduce req_rts_linkerStefan Schulze Frielinghaus2019-11-081-0/+4
| | | | | Some tests depend on the RTS linker. Introduce a modifier to skip such tests, in case the RTS linker is not available.
* CI: Always dump performance metrics.David Eichmann2019-10-221-3/+3
|
* testsuite: Assert that testsuite ways are knownBen Gamari2019-10-171-28/+49
| | | | | This ensures that all testsuite way names given to `omit_ways`, `only_ways`, etc. are known ways.
* testsuite: Ensure that makefile tests get runBen Gamari2019-10-171-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously `makefile_test` and `run_command` tests could easily end up in a situation where they wouldn't be run if the user used the `only_ways` modifier. The reason is to build the set of a ways to run the test in we first start with a candidate set determined by the test type (e.g. `makefile_test`, `compile_run`, etc.) and then filter that set with the constraints given by the test's modifiers. `makefile_test` and `run_command` tests' candidate sets were simply `{normal}`, and consequently most uses of `only_ways` would result in the test being never run. To avoid this we rather use all ways as the candidate sets for these test types. This may result in a few more testcases than we would like (given that some `run_command` tests are insensitive to way) but this can be fixed by adding modifiers and we would much rather run too many tests than too few. This fixes #16042 and a number of other tests afflicted by the same issue. However, there were a few cases that required special attention: * `T14028` is currently failing and is therefore marked as broken due to #17300 * `T-signals-child` is fragile in the `threaded1` and `threaded2` ways (tracked in #17307)
* Use os.devnull instead of '/dev/null' in the testsuite driver.Andreas Klebinger2019-08-141-4/+26
| | | | | | | | | | | | The later caused issues on windows by being translated into "\\dev\\null" and python then trying to open this non-existant file. So we now use os.devnull inside python and convert it to "/dev/null" when calling out to the shell, which is bound to run in a unix like environment. This fixes an issue a test producing unexpected stderr output failed with a framework failure instead of showing a diff of the output.
* testsuite: Rework tracking of fragile testsBen Gamari2019-08-041-5/+14
| | | | | Breaks fragile tests into two groups, allowing us to easily preserve stdout/stderr of failing fragile tests.
* testsuite: Fix some ints used as boolsBen Gamari2019-07-181-10/+10
|
* testsuite: Print output from hp2psBen Gamari2019-07-181-1/+1
|
* testsuite: Fix req_thBen Gamari2019-07-181-1/+2
|
* testsuite: More type checking fixesBen Gamari2019-07-181-59/+112
|
* Testsuite tweaks and refactoringÖmer Sinan Ağacan2019-07-091-8/+10
| | | | | | | | | | | | | | | | - Rename requires_th to req_th for consistency with other req functions (e.g. req_interp, req_profiling etc.) - req_th (previously requires_th) now checks for interpreter (via req_interp). With this running TH tests are skipped when running the test suite with stage=1. - Test tweaks: - T9360a, T9360b: Use req_interp - recomp009, T13938, RAE_T32a: Use req_th - Fix check-makefiles linter: it now looks for Makefiles instead of .T files (which are actually Python files)
* testsuite: Fix #16818Ben Gamari2019-07-091-3/+10
| | | | | Renames performance metrics to include whether they are compile-time or runtime metrics.
* testsuite: Run and report on fragile testsBen Gamari2019-06-261-8/+17
| | | | | | This allows us to run (but ignore the result of) fragile testcases. Hopefully this should allow us to more easily spot when a fragile test becomes un-fragile.
* testsuite: A major revamp of the driverBen Gamari2019-06-251-283/+324
| | | | | | | | | This tries to put the testsuite driver into a slightly more maintainable condition: * Add type annotations where easily done * Use pathlib.Path instead of str paths * Make it pass the mypy typechecker
* testsuite: Add assertions that way lists are in fact listsBen Gamari2019-06-151-0/+6
| | | | | | Previously there were a few cases where operations like `omit_ways` were incorrectly passed a single way (e.g. `omit_ways('threaded2')`). This won't work as the author expected.
* testsuite: Don't run tests requiring TH in profasm way when GhcDynamicBen Gamari2019-06-121-0/+8
| | | | | | | | | | Since we can't load profiled objects when GhcDynamic==YES. Affects: * T16737 * T16384 * T16718 * T16619 * T16190
* testsuite: Fix omit_ways usageBen Gamari2019-06-121-0/+1
| | | | omit_ways expects a list but this was broken in several cases.
* testsuite: Fix fragile_for test modifierBen Gamari2019-06-121-3/+3
|
* TestRunner: Added --chart to display a chart of performance testsDavid Eichmann2019-06-041-4/+1
| | | | | | | | | | | | | This uses the Chart.js javascript library. Everything is put into a standalone .html file and opened with the default browser. I also simplified the text output to use the same data as the chart. You can now use a commit range with git's ".." syntax. The --ci option will use results from CI (you'll need to fetch them first): $ git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf $ python3 testsuite/driver/perf_notes.py --ci --chart --test-env x86_64-darwin --test-name T9630 master~500..master
* testsuite: introduce 'static_stats' testsAlp Mestanogullari2019-05-291-5/+11
| | | | | | | | | | | | | | | | They are a particular type of perf tests. This patch introduces a 'stats_files_dir' configuration field in the testsuite driver where all haddock timing files (and possibly others in the future) are assumed to live. We also change both the Make and Hadrian build systems to pass respectively $(TOP)/testsuite/tests/perf/haddock/ and <build root>/stage1/haddock-timing-files/ as the value of that new configuration field, and to generate the timing files in those directories in the first place while generating documentation with haddock. This new test type can be seen as one dedicated to examining stats files that are generated while building a GHC distribution. This also lets us get rid of the 'extra_files' directives in the all.T entries for haddock.base, haddock.Cabal and haddock.compiler.
* Update terminal title while running test-suiteOleg Grenrus2019-05-141-3/+9
| | | | | Useful progress indicator even when `make test VERBOSE=1`, and when you do something else, but have terminal title visible.
* codegen: fix memset unroll for small bytearrays, add 64-bit setsArtem Pyanykh2019-04-091-2/+57
| | | | | | | | | | | | | | | | | | | | | | Fixes #16052 When the offset in `setByteArray#` is statically known, we can provide better alignment guarantees then just 1 byte. Also, memset can now do 64-bit wide sets. The current memset intrinsic is not optimal however and can be improved for the case when we know that we deal with (baseAddress at known alignment) + offset For instance, on 64-bit `setByteArray# s 1# 23# 0#` given that bytearray is 8 bytes aligned could be unrolled into `movb, movw, movl, movq, movq`; but currently it is `movb x23` since alignment of 1 is all we can embed into MO_Memset op.
* testsuite: Show exit code of GHCi tests on failureÖmer Sinan Ağacan2019-04-081-1/+1
|
* Print test suite results ("unexpected failures" etc.) in sorted orderÖmer Sinan Ağacan2019-03-201-1/+1
| | | | Fixes #16425
* testsuite: Display observed exit code on failure due to bad exit codeBen Gamari2019-03-201-1/+1
|
* Update Trac ticket URLs to point to GitLabRyan Scott2019-03-151-2/+2
| | | | | This moves all URL references to Trac tickets to their corresponding GitLab counterparts.
* testsuite: Introduce fragile modifierBen Gamari2019-03-061-0/+24
| | | | | | | | | | Now since we have been a bit more stringent in testsuite cleanliness we have been marking a lot of tests as fragile using the `skip` modifier. However, this unfortunately means that we lose the association with the ticket number documenting the fragility. Here we introduce `fragile` and `fragile_for` to retain this information.
* testsuite: Fix whitespace in hp2ps error messageBen Gamari2019-02-231-1/+1
|
* Exit with exit code 1 when tests unexpectedly passMatthew Pickering2019-02-231-0/+1
| | | | | | This was causing gitlab to not report from builds as failing. It also highlighted a problem with the LLVM tests where some of the external interpreter tests are failing.
* Fix test runner crash when not in a git repoDavid Eichmann2019-02-211-2/+3
| | | | Respect `inside_git_repo()` when checking performance stats.
* Testsuite: implement use_specs.Tamar Christina2019-02-171-0/+26
|
* Fix and Reapply "Performance tests: recover a baseline from ancestor commits ↵David Eichmann2019-02-161-43/+27
| | | | and CI results."
* testsuite: Report stdout and stderr in JUnit outputBen Gamari2019-02-101-21/+37
| | | | | | | | | | This patch makes the JUnit output more useful as now we also report the stdout/stderr in the message which can be used to quickly identify why a test is failing without downloading the log. This also introduces TestResult, previously we were simply passing around tuples, making things the implementation rather difficult to follow and harder to extend.
* testsuite: Mark print037 as broken when GHC is built with LLVMBen Gamari2019-02-041-0/+9
| | | | As noted in #16205 this configuration reliably segfaults.
* Revert "Performance tests: recover a baseline from ancestor commits and CI ↵Ben Gamari2019-01-311-26/+43
| | | | | | | | | results." Unfortunately this has broken all future commits due to spurious(?) performance changes which I have been unable to work around. This reverts commit cc2261d42f6a954d88e355aaad41f001f65c95da.
* Performance tests: recover a baseline from ancestor commits and CI results.David Eichmann2019-01-301-43/+26
| | | | gitlab-ci: push performance metrics as git notes to the "GHC Performance Notes" repository.
* testsuite: Use makefile_testBen Gamari2019-01-301-1/+1
| | | | | This eliminates most uses of run_command in the testsuite in favor of the more structured makefile_test.
* testsuite: Introduce makefile_testBen Gamari2019-01-301-0/+7
|