summaryrefslogtreecommitdiff
path: root/testsuite/driver/testlib.py
Commit message (Collapse)AuthorAgeFilesLines
* [testsuite] filter out superfluous dylib warningsMoritz Angermann2021-05-071-3/+18
| | | | (cherry picked from commit 33c4d497545559a38bd8d1caf6c94e5e2a77647b)
* [testlib/driver] denoiseMoritz Angermann2021-05-071-4/+8
| | | | | | | | | | this prevents the testlib/driver to be overly noisy, and will also kill some noise produiced by the aarch64-darwin cc (for now). Fixing sysctl, will allow us to run the test's properly in a nix-shell on aarch64-darwin (cherry picked from commit 5109e87e13ab45d799db2013535f54ca35f1f4dc)
* test driver: Make sure RESIDENCY_OPTS is passed for 'all' perf testsMatthew Pickering2021-05-051-11/+28
| | | | | | | | | | | | Fixes #19731 ------------------------- Metric Decrease: T11545 Metric Increase: T12545 T15304 -------------------------
* Revert "[ci/arm/darwin/testsuite] Forwards ports from GHC-8.10"Ben Gamari2021-04-051-25/+6
| | | | This reverts commit 0cbdba2768d84a0f6832ae5cf9ea1e98efd739da.
* testsuite: Check test stats only after test correctnessBen Gamari2021-04-021-5/+5
| | | | | | | | | Ticket #19576 noted that a test that failed in correctness (e.g. due to stderr mismatch) *and* failed due to a metrics change would report misleading stats. This was due to the testsuite driver *first* checking stats, before checking for correctness. Fix this. Closes #19576.
* testsuite: Make passFail a booleanBen Gamari2021-04-021-21/+6
|
* [testlib] ignore strip warningsMoritz Angermann2021-03-291-0/+4
|
* [ci/arm/darwin/testsuite] Forwards ports from GHC-8.10Moritz Angermann2021-03-211-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a set of forward ports (cherry-picks) from 8.10 - a7d22795ed [ci] Add support for building on aarch64-darwin - 5109e87e13 [testlib/driver] denoise - 307d34945b [ci] default value for CONFIGURE_ARGS - 10a18cb4e0 [testsuite] mark ghci056 as fragile - 16c13d5acf [ci] Default value for MAKE_ARGS - ab571457b9 [ci/build] Copy config.sub around - 251892b98f [ci/darwin] bump nixpkgs rev - 5a6c36ecb4 [testsuite/darwin] fix conc059 - aae95ef0c9 [ci] add timing info - 3592d1104c [Aarch64] No div-by-zero; disable test. - 57671071ad [Darwin] mark stdc++ tests as broken - 33c4d49754 [testsuite] filter out superfluous dylib warnings - 4bea83afec [ci/nix-shell] Add Foundation and Security - 6345530062 [testsuite/json2] Fix failure with LLVM backends - c3944bc89d [ci/nix-shell] [Darwin] Stop the ld warnings about libiconv. - b821fcc714 [testsuite] static001 is not broken anymore. - f7062e1b0c [testsuite/arm64] fix section_alignment - 820b076698 [darwin] stop the DYLD_LIBRARY_PATH madness - 07b1af0362 [ci/nix-shell] uniquify NIX_LDFLAGS{_FOR_TARGET} As well as a few additional fixups needed to make this block compile: - Fixup all.T - Set CROSS_TARGET, BROKEN_TESTS, XZ, RUNTEST_ARGS, default value. - [ci] shell.nix bump happy
* gitlab-ci: Ignore performance improvements in marge jobsBen Gamari2021-03-191-12/+13
| | | | | | | | Currently we have far too many merge failures due to cumulative performance improvements. Avoid this by accepting metric decreases in marge-bot jobs. Fixes #19562.
* testsuite: Make --ignore-perf-tests more expressiveBen Gamari2021-03-191-4/+9
| | | | Allow skipping of only increases/decreases.
* testsuite: Introduce flag to ignore performance failuresBen Gamari2021-02-241-1/+1
| | | | Needed by #19025.
* Test Driver: Tweak interval of test reportingMatthew Pickering2021-02-181-5/+12
| | | | | | | | | | | Rather than just display every 100 tests, work out how many to display based on the total number of tests. This improves the experience when running a small number of tests. For [0..100] - Report every test [100..1000] - Report every 10 tests [1000..10000] - Report every 100 tests and so on..
* Reduce default test verbosityMatthew Pickering2021-01-281-1/+2
|
* Deprecate -h flagMatthew Pickering2021-01-271-4/+4
| | | | | | | | | | It is confusing that it defaults to two different things depending on whether we are in the profiling way or not. Use -hc if you have a profiling build Use -hT if you have a normal build Fixes #19031
* Add some additional information to the fail message based on exit codeHécate2021-01-171-2/+16
|
* Make proper fixed-width number literalsSylvain Henry2021-01-021-1/+1
| | | | | | | | (Progress towards #11953, #17377, #17375) Besides being nicer to use, this also will allow for better constant folding for the fixed-width types, on par with what `Int#` and `Word#` have today.
* [Sized Cmm] properly retain sizes.Moritz Angermann2020-11-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces all Word<N> = W<N># Word# and Int<N> = I<N># Int# with Word<N> = W<N># Word<N># and Int<N> = I<N># Int<N>#, thus providing us with properly sized primitives in the codegenerator instead of pretending they are all full machine words. This came up when implementing darwinpcs for arm64. The darwinpcs reqires us to pack function argugments in excess of registers on the stack. While most procedure call standards (pcs) assume arguments are just passed in 8 byte slots; and thus the caller does not know the exact signature to make the call, darwinpcs requires us to adhere to the prototype, and thus have the correct sizes. If we specify CInt in the FFI call, it should correspond to the C int, and not just be Word sized, when it's only half the size. This does change the expected output of T16402 but the new result is no less correct as it eliminates the narrowing (instead of the `and` as was previously done). Bumps the array, bytestring, text, and binary submodules. Co-Authored-By: Ben Gamari <ben@well-typed.com> Metric Increase: T13701 T14697
* CmmToLlvm: Declare signature for memcmpwip/angerman/arm64Ben Gamari2020-11-241-0/+10
| | | | | | Otherwise `opt` fails with: error: use of undefined value '@memcmp$def'
* Merge remote-tracking branch 'origin/wip/tsan/all'Ben Gamari2020-11-081-0/+7
|\
| * testsuite: Skip high memory usage tests with TSANBen Gamari2020-10-241-0/+4
| | | | | | | | | | ThreadSanitizer significantly increases the memory footprint of tests, so much so that it can send machines into OOM.
| * testsuite: Mark setnumcapabilities001 as broken with TSANGHC GitLab CI2020-10-241-0/+3
| | | | | | | | Due to #18808.
* | Testsuite: Support for user supplied package dbsAndreas Klebinger2020-11-041-1/+14
| | | | | | | | | | | | | | | | | | We can now supply additional package dbs to the testsuite. For make the package db can be supplied by passing PACKAGE_DB=/path/to/db. In the testsuite driver it's passed via the --test-package-db argument.
* | testsuite: Add --top flag to driverGHC GitLab CI2020-11-021-1/+1
| | | | | | | | | | This allows us to make `config.top` a proper Path. Previously it was a str, which caused the Ghostscript detection logic to break.
* | Use config.run_ways for multi_compile_and_run testsDavid Eichmann2020-10-271-1/+1
|/
* testsuite: Fix WinIO error message normalizationGHC GitLab CI2020-09-201-4/+6
| | | | This wasn't being applied to stderr.
* testsuite: Allow baseline commit to be set explicitlyBen Gamari2020-08-181-1/+2
|
* testsuite: Normalise WinIO error message differencesBen Gamari2020-07-261-0/+24
| | | | | Previously the old Windows IO manager threw different errors than WinIO. We now canonicalise these to the WinIO errors.
* Add test for #18064Sylvain Henry2020-07-221-0/+5
| | | | It has been fixed by 0effc57d48ace6b719a9f4cbeac67c95ad55010b
* Add tests for #17920Sylvain Henry2020-06-231-2/+1
| | | | | | Metric Decrease: T12150 T12234
* Update testsuiteSylvain Henry2020-06-171-7/+7
| | | | | | | | | | | | | | * support detection of slow ghc-bignum backend (to replace the detection of integer-simple use). There are still some test cases that the native backend doesn't handle efficiently enough. * remove tests for GMP only functions that have been removed from ghc-bignum * fix test results showing dependent packages (e.g. integer-gmp) or showing suggested instances * fix test using Integer/Natural API or showing internal names
* testsuite: Refactor ghostscript detectionBen Gamari2020-06-011-24/+31
| | | | Tamar reported that he saw crashes due to unhandled exceptions.
* Implement cstringLength# and FinalPtrAndrew Martin2020-05-231-0/+20
| | | | | | | | | | | | | | | | | | | | This function and its accompanying rule resolve issue #5218. A future PR to the bytestring library will make the internal Data.ByteString.Internal.unsafePackAddress compute string length with cstringLength#. This will improve the status quo because it is eligible for constant folding. Additionally, introduce a new data constructor to ForeignPtrContents named FinalPtr. This additional data constructor, when used in the IsString instance for ByteString, leads to more Core-to-Core optimization opportunities, fewer runtime allocations, and smaller binaries. Also, this commit re-exports all the functions from GHC.CString (including cstringLength#) in GHC.Exts. It also adds a new test driver. This test driver is used to perform substring matches on Core that is dumped after all the simplifier passes. In this commit, it is used to check that constant folding of cstringLength# works.
* testsuite: Don't attempt to read .std{err,out} files if they don't existBen Gamari2020-04-231-2/+10
| | | | | | | | Simon reports that he was previously seeing framework failures due to an attempt to read the non-existing T13456.stderr. While I don't know exactly what this is due to, it does seem like a non-existing .std{out,err} file should be equivalent to an empty file. Teach the testsuite driver to treat it as such.
* testsuite: Move no_lint to the top level, tweak hie002Ömer Sinan Ağacan2020-04-101-0/+6
| | | | | | | | | | | | | | | | | | | | | | | - We don't want to benchmark linting so disable lints in hie002 perf test - Move no_lint to the top-level to be able to use it in tests other than those in `testsuite/tests/perf/compiler`. - Filter out -dstg-lint in no_lint. - hie002 allocation numbers on 32-bit are unstable, so skip it on 32-bit Metric Decrease: hie002 ManyConstructors T12150 T12234 T13035 T1969 T4801 T9233 T9961
* testsuite: Allow tests to be marked as broken on the command lineBen Gamari2020-02-261-0/+3
| | | | This allows us to work-around distribution-specific breakage easily.
* Fix testsuite driver output (#17847)Vladislav Zavialov2020-02-181-1/+1
|
* testsuite: Assert the opsys names are knownBen Gamari2020-02-141-0/+11
| | | | | | Previously opsys would take any string. This meant it was very easy for a typo to silently render the predicate ineffective. Fix this by checking the given operating system name against a list of known values.
* testlib: Extend existing *_opts in extra_*_optsÖmer Sinan Ağacan2020-02-051-4/+6
| | | | | | | | | | | | Previously we'd override the existing {run,hc} opts in extra_{run,hc}_opts, which caused flakiness in T1969, see #17712. extra_{run,hc}_opts now extends {run,hc} opts, instead of overriding. Also we shrank the allocation area for T1969 in order to increase residency sampling frequency. Fixes #17712
* testsuite: Don't crash on encoding failure in printBen Gamari2020-01-311-8/+12
| | | | | | If the user doesn't use a Unicode locale then the testsuite driver would previously throw framework failures due to encoding failures. We now rather use the `replace` error-handling strategy.
* testsuite: Preserve more information in framework failuresBen Gamari2020-01-201-1/+1
| | | | | Namely print the entire exception in hopes that this will help track down #17649.
* Fix typos, via a Levenshtein-style correctorBrian Wignall2020-01-041-1/+1
|
* Implement pointer tagging for big families (#14373)Gabor Greif2019-12-061-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Formerly we punted on these and evaluated constructors always got a tag of 1. We now cascade switches because we have to check the tag first and when it is MAX_PTR_TAG then get the precise tag from the info table and switch on that. The only technically tricky part is that the default case needs (logical) duplication. To do this we emit an extra label for it and branch to that from the second switch. This avoids duplicated codegen. Here's a simple example of the new code gen: data D = D1 | D2 | D3 | D4 | D5 | D6 | D7 | D8 On a 64-bit system previously all constructors would be tagged 1. With the new code gen D7 and D8 are tagged 7: [Lib.D7_con_entry() { ... {offset c1eu: // global R1 = R1 + 7; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; } }] [Lib.D8_con_entry() { ... {offset c1ez: // global R1 = R1 + 7; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; } }] When switching we now look at the info table only when the tag is 7. For example, if we derive Enum for the type above, the Cmm looks like this: c2Le: _s2Js::P64 = R1; _c2Lq::P64 = _s2Js::P64 & 7; switch [1 .. 7] _c2Lq::P64 { case 1 : goto c2Lk; case 2 : goto c2Ll; case 3 : goto c2Lm; case 4 : goto c2Ln; case 5 : goto c2Lo; case 6 : goto c2Lp; case 7 : goto c2Lj; } // Read info table for tag c2Lj: _c2Lv::I64 = %MO_UU_Conv_W32_W64(I32[I64[_s2Js::P64 & (-8)] - 4]); if (_c2Lv::I64 != 6) goto c2Lu; else goto c2Lt; Generated Cmm sizes do not change too much, but binaries are very slightly larger, due to the fact that the new instructions are longer in encoded form. E.g. previously entry code for D8 above would be 00000000000001c0 <Lib_D8_con_info>: 1c0: 48 ff c3 inc %rbx 1c3: ff 65 00 jmpq *0x0(%rbp) With this patch 00000000000001d0 <Lib_D8_con_info>: 1d0: 48 83 c3 07 add $0x7,%rbx 1d4: ff 65 00 jmpq *0x0(%rbp) This is one byte longer. Secondly, reading info table directly and then switching is shorter _c1co: movq -1(%rbx),%rax movl -4(%rax),%eax // Switch on info table tag jmp *_n1d5(,%rax,8) than doing the same switch, and then for the tag 7 doing another switch: // When tag is 7 _c1ct: andq $-8,%rbx movq (%rbx),%rax movl -4(%rax),%eax // Switch on info table tag ... Some changes of binary sizes in actual programs: - In NoFib the worst case is 0.1% increase in benchmark "parser" (see NoFib results below). All programs get slightly larger. - Stage 2 compiler size does not change. - In "containers" (the library) size of all object files increases 0.0005%. Size of the test program "bitqueue-properties" increases 0.03%. nofib benchmarks kindly provided by Ömer (@osa1): NoFib Results ============= -------------------------------------------------------------------------------- Program Size Allocs Instrs Reads Writes -------------------------------------------------------------------------------- CS +0.0% 0.0% -0.0% -0.0% -0.0% CSD +0.0% 0.0% 0.0% +0.0% +0.0% FS +0.0% 0.0% 0.0% +0.0% 0.0% S +0.0% 0.0% -0.0% 0.0% 0.0% VS +0.0% 0.0% -0.0% +0.0% +0.0% VSD +0.0% 0.0% -0.0% +0.0% -0.0% VSM +0.0% 0.0% 0.0% 0.0% 0.0% anna +0.0% 0.0% +0.1% -0.9% -0.0% ansi +0.0% 0.0% -0.0% +0.0% +0.0% atom +0.0% 0.0% 0.0% 0.0% 0.0% awards +0.0% 0.0% -0.0% +0.0% 0.0% banner +0.0% 0.0% -0.0% +0.0% 0.0% bernouilli +0.0% 0.0% +0.0% +0.0% +0.0% binary-trees +0.0% 0.0% -0.0% -0.0% -0.0% boyer +0.0% 0.0% +0.0% 0.0% -0.0% boyer2 +0.0% 0.0% +0.0% 0.0% -0.0% bspt +0.0% 0.0% +0.0% +0.0% 0.0% cacheprof +0.0% 0.0% +0.1% -0.8% 0.0% calendar +0.0% 0.0% -0.0% +0.0% -0.0% cichelli +0.0% 0.0% +0.0% 0.0% 0.0% circsim +0.0% 0.0% -0.0% -0.1% -0.0% clausify +0.0% 0.0% +0.0% +0.0% 0.0% comp_lab_zift +0.0% 0.0% +0.0% 0.0% -0.0% compress +0.0% 0.0% +0.0% +0.0% 0.0% compress2 +0.0% 0.0% 0.0% 0.0% 0.0% constraints +0.0% 0.0% -0.0% -0.0% -0.0% cryptarithm1 +0.0% 0.0% +0.0% 0.0% 0.0% cryptarithm2 +0.0% 0.0% +0.0% -0.0% 0.0% cse +0.0% 0.0% +0.0% +0.0% 0.0% digits-of-e1 +0.0% 0.0% -0.0% -0.0% -0.0% digits-of-e2 +0.0% 0.0% +0.0% -0.0% -0.0% dom-lt +0.0% 0.0% +0.0% +0.0% 0.0% eliza +0.0% 0.0% -0.0% +0.0% 0.0% event +0.0% 0.0% -0.0% -0.0% -0.0% exact-reals +0.0% 0.0% +0.0% +0.0% +0.0% exp3_8 +0.0% 0.0% -0.0% -0.0% -0.0% expert +0.0% 0.0% +0.0% +0.0% +0.0% fannkuch-redux +0.0% 0.0% +0.0% 0.0% 0.0% fasta +0.0% 0.0% -0.0% -0.0% -0.0% fem +0.0% 0.0% +0.0% +0.0% +0.0% fft +0.0% 0.0% +0.0% -0.0% -0.0% fft2 +0.0% 0.0% +0.0% +0.0% +0.0% fibheaps +0.0% 0.0% +0.0% +0.0% 0.0% fish +0.0% 0.0% +0.0% +0.0% 0.0% fluid +0.0% 0.0% +0.0% +0.0% +0.0% fulsom +0.0% 0.0% +0.0% -0.0% +0.0% gamteb +0.0% 0.0% +0.0% -0.0% -0.0% gcd +0.0% 0.0% +0.0% +0.0% 0.0% gen_regexps +0.0% 0.0% +0.0% -0.0% -0.0% genfft +0.0% 0.0% -0.0% -0.0% -0.0% gg +0.0% 0.0% 0.0% -0.0% 0.0% grep +0.0% 0.0% +0.0% +0.0% +0.0% hidden +0.0% 0.0% +0.0% -0.0% -0.0% hpg +0.0% 0.0% +0.0% -0.1% -0.0% ida +0.0% 0.0% +0.0% -0.0% -0.0% infer +0.0% 0.0% -0.0% -0.0% -0.0% integer +0.0% 0.0% -0.0% -0.0% -0.0% integrate +0.0% 0.0% 0.0% +0.0% 0.0% k-nucleotide +0.0% 0.0% -0.0% -0.0% -0.0% kahan +0.0% 0.0% -0.0% -0.0% -0.0% knights +0.0% 0.0% +0.0% -0.0% -0.0% lambda +0.0% 0.0% +1.2% -6.1% -0.0% last-piece +0.0% 0.0% +0.0% -0.0% -0.0% lcss +0.0% 0.0% +0.0% -0.0% -0.0% life +0.0% 0.0% +0.0% -0.0% -0.0% lift +0.0% 0.0% +0.0% +0.0% 0.0% linear +0.0% 0.0% +0.0% +0.0% +0.0% listcompr +0.0% 0.0% -0.0% -0.0% -0.0% listcopy +0.0% 0.0% -0.0% -0.0% -0.0% maillist +0.0% 0.0% +0.0% -0.0% -0.0% mandel +0.0% 0.0% +0.0% +0.0% +0.0% mandel2 +0.0% 0.0% +0.0% +0.0% -0.0% mate +0.0% 0.0% +0.0% +0.0% +0.0% minimax +0.0% 0.0% -0.0% +0.0% -0.0% mkhprog +0.0% 0.0% +0.0% +0.0% +0.0% multiplier +0.0% 0.0% 0.0% +0.0% -0.0% n-body +0.0% 0.0% +0.0% -0.0% -0.0% nucleic2 +0.0% 0.0% +0.0% +0.0% -0.0% para +0.0% 0.0% +0.0% +0.0% +0.0% paraffins +0.0% 0.0% +0.0% +0.0% +0.0% parser +0.1% 0.0% +0.4% -1.7% -0.0% parstof +0.0% 0.0% -0.0% -0.0% -0.0% pic +0.0% 0.0% +0.0% 0.0% -0.0% pidigits +0.0% 0.0% -0.0% -0.0% -0.0% power +0.0% 0.0% +0.0% -0.0% -0.0% pretty +0.0% 0.0% +0.0% +0.0% +0.0% primes +0.0% 0.0% +0.0% 0.0% 0.0% primetest +0.0% 0.0% +0.0% +0.0% +0.0% prolog +0.0% 0.0% +0.0% +0.0% +0.0% puzzle +0.0% 0.0% +0.0% +0.0% +0.0% queens +0.0% 0.0% 0.0% +0.0% +0.0% reptile +0.0% 0.0% +0.0% +0.0% 0.0% reverse-complem +0.0% 0.0% -0.0% -0.0% -0.0% rewrite +0.0% 0.0% +0.0% 0.0% -0.0% rfib +0.0% 0.0% +0.0% +0.0% +0.0% rsa +0.0% 0.0% +0.0% +0.0% +0.0% scc +0.0% 0.0% +0.0% +0.0% +0.0% sched +0.0% 0.0% +0.0% +0.0% +0.0% scs +0.0% 0.0% +0.0% +0.0% 0.0% simple +0.0% 0.0% +0.0% +0.0% +0.0% solid +0.0% 0.0% +0.0% +0.0% 0.0% sorting +0.0% 0.0% +0.0% -0.0% 0.0% spectral-norm +0.0% 0.0% -0.0% -0.0% -0.0% sphere +0.0% 0.0% +0.0% -1.0% 0.0% symalg +0.0% 0.0% +0.0% +0.0% +0.0% tak +0.0% 0.0% +0.0% +0.0% +0.0% transform +0.0% 0.0% +0.4% -1.3% +0.0% treejoin +0.0% 0.0% +0.0% -0.0% 0.0% typecheck +0.0% 0.0% -0.0% +0.0% 0.0% veritas +0.0% 0.0% +0.0% -0.1% +0.0% wang +0.0% 0.0% +0.0% +0.0% +0.0% wave4main +0.0% 0.0% +0.0% 0.0% -0.0% wheel-sieve1 +0.0% 0.0% +0.0% +0.0% +0.0% wheel-sieve2 +0.0% 0.0% +0.0% +0.0% 0.0% x2n1 +0.0% 0.0% +0.0% +0.0% 0.0% -------------------------------------------------------------------------------- Min +0.0% 0.0% -0.0% -6.1% -0.0% Max +0.1% 0.0% +1.2% +0.0% +0.0% Geometric Mean +0.0% -0.0% +0.0% -0.1% -0.0% NoFib GC Results ================ -------------------------------------------------------------------------------- Program Size Allocs Instrs Reads Writes -------------------------------------------------------------------------------- circsim +0.0% 0.0% -0.0% -0.0% -0.0% constraints +0.0% 0.0% -0.0% 0.0% -0.0% fibheaps +0.0% 0.0% 0.0% -0.0% -0.0% fulsom +0.0% 0.0% 0.0% -0.6% -0.0% gc_bench +0.0% 0.0% 0.0% 0.0% -0.0% hash +0.0% 0.0% -0.0% -0.0% -0.0% lcss +0.0% 0.0% 0.0% -0.0% 0.0% mutstore1 +0.0% 0.0% 0.0% -0.0% -0.0% mutstore2 +0.0% 0.0% +0.0% -0.0% -0.0% power +0.0% 0.0% -0.0% 0.0% -0.0% spellcheck +0.0% 0.0% -0.0% -0.0% -0.0% -------------------------------------------------------------------------------- Min +0.0% 0.0% -0.0% -0.6% -0.0% Max +0.0% 0.0% +0.0% 0.0% 0.0% Geometric Mean +0.0% +0.0% +0.0% -0.1% +0.0% Fixes #14373 These performance regressions appear to be a fluke in CI. See the discussion in !1742 for details. Metric Increase: T6048 T12234 T12425 Naperian T12150 T5837 T13035
* testsuite: Make performance metric summary more readableBen Gamari2019-12-051-2/+2
| | | | Along with some refactoring.
* testsuite: Factor out terminal coloringBen Gamari2019-12-051-11/+10
|
* Fix typos, using Wikipedia list of common typosBrian Wignall2019-11-281-1/+1
|
* testsuite: don't collect compiler stats in collect_runtime_residencyAlp Mestanogullari2019-11-121-1/+1
| | | | | | | | | | | | | | | | | | | We instead want to collect the runtime stats (with collect_stats, instead of collect_compiler_stats). This should fix a number of perf tests failures we have been seeing, where we suddenly started measuring metrics we didn't intend to measure, which tend to fall outside of the acceptance window. Metric Decrease: lazy-bs-alloc T3586 Metric Increase: space_leak_001 T4801 T5835 T12791
* testsuite: Don't check_stats at runtime if not requestedwip/T17387Ben Gamari2019-11-101-4/+9
| | | | | | Previously we would call check_stats to check the runtime metrics even if the test definition hadn't requested it. This would result in an error since the .stats file doesn't exist.
* testsuite: Use small allocation area when measuring residencyBen Gamari2019-11-101-0/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | As suggested in #17387; this helps reduce the variance in our residency sampling. Metric Increase: T10370 T3586 lazy-bs-alloc Metric Decrease 'compile_time/peak_megabytes_allocated': T1969 Metric Decrease 'runtime/bytes allocated': space_leak_001 Metric Increase 'compile_time/bytes allocated': T1969 Metric Increase 'runtime/peak_megabytes_allocated': space_leak_001 Metric Decrease: T3064 T9675
* testsuite: Fix Windows cleanup pathBen Gamari2019-11-091-2/+2
| | | | This was a regression introduced with the Path refactoring.
* Testsuite: Introduce req_rts_linkerStefan Schulze Frielinghaus2019-11-081-0/+4
| | | | | Some tests depend on the RTS linker. Introduce a modifier to skip such tests, in case the RTS linker is not available.
View Lorry Controller Status