| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we'd override the existing {run,hc} opts in
extra_{run,hc}_opts, which caused flakiness in T1969, see #17712.
extra_{run,hc}_opts now extends {run,hc} opts, instead of overriding.
Also we shrank the allocation area for T1969 in order to increase
residency sampling frequency.
Fixes #17712
|
|
|
|
|
|
| |
If the user doesn't use a Unicode locale then the testsuite driver would
previously throw framework failures due to encoding failures. We now
rather use the `replace` error-handling strategy.
|
|
|
|
|
| |
Namely print the entire exception in hopes that this will help track
down #17649.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Formerly we punted on these and evaluated constructors always got a tag
of 1.
We now cascade switches because we have to check the tag first and when
it is MAX_PTR_TAG then get the precise tag from the info table and
switch on that. The only technically tricky part is that the default
case needs (logical) duplication. To do this we emit an extra label for
it and branch to that from the second switch. This avoids duplicated
codegen.
Here's a simple example of the new code gen:
data D = D1 | D2 | D3 | D4 | D5 | D6 | D7 | D8
On a 64-bit system previously all constructors would be tagged 1. With
the new code gen D7 and D8 are tagged 7:
[Lib.D7_con_entry() {
...
{offset
c1eu: // global
R1 = R1 + 7;
call (P64[Sp])(R1) args: 8, res: 0, upd: 8;
}
}]
[Lib.D8_con_entry() {
...
{offset
c1ez: // global
R1 = R1 + 7;
call (P64[Sp])(R1) args: 8, res: 0, upd: 8;
}
}]
When switching we now look at the info table only when the tag is 7. For
example, if we derive Enum for the type above, the Cmm looks like this:
c2Le:
_s2Js::P64 = R1;
_c2Lq::P64 = _s2Js::P64 & 7;
switch [1 .. 7] _c2Lq::P64 {
case 1 : goto c2Lk;
case 2 : goto c2Ll;
case 3 : goto c2Lm;
case 4 : goto c2Ln;
case 5 : goto c2Lo;
case 6 : goto c2Lp;
case 7 : goto c2Lj;
}
// Read info table for tag
c2Lj:
_c2Lv::I64 = %MO_UU_Conv_W32_W64(I32[I64[_s2Js::P64 & (-8)] - 4]);
if (_c2Lv::I64 != 6) goto c2Lu; else goto c2Lt;
Generated Cmm sizes do not change too much, but binaries are very
slightly larger, due to the fact that the new instructions are longer in
encoded form. E.g. previously entry code for D8 above would be
00000000000001c0 <Lib_D8_con_info>:
1c0: 48 ff c3 inc %rbx
1c3: ff 65 00 jmpq *0x0(%rbp)
With this patch
00000000000001d0 <Lib_D8_con_info>:
1d0: 48 83 c3 07 add $0x7,%rbx
1d4: ff 65 00 jmpq *0x0(%rbp)
This is one byte longer.
Secondly, reading info table directly and then switching is shorter
_c1co:
movq -1(%rbx),%rax
movl -4(%rax),%eax
// Switch on info table tag
jmp *_n1d5(,%rax,8)
than doing the same switch, and then for the tag 7 doing another switch:
// When tag is 7
_c1ct:
andq $-8,%rbx
movq (%rbx),%rax
movl -4(%rax),%eax
// Switch on info table tag
...
Some changes of binary sizes in actual programs:
- In NoFib the worst case is 0.1% increase in benchmark "parser" (see
NoFib results below). All programs get slightly larger.
- Stage 2 compiler size does not change.
- In "containers" (the library) size of all object files increases
0.0005%. Size of the test program "bitqueue-properties" increases
0.03%.
nofib benchmarks kindly provided by Ömer (@osa1):
NoFib Results
=============
--------------------------------------------------------------------------------
Program Size Allocs Instrs Reads Writes
--------------------------------------------------------------------------------
CS +0.0% 0.0% -0.0% -0.0% -0.0%
CSD +0.0% 0.0% 0.0% +0.0% +0.0%
FS +0.0% 0.0% 0.0% +0.0% 0.0%
S +0.0% 0.0% -0.0% 0.0% 0.0%
VS +0.0% 0.0% -0.0% +0.0% +0.0%
VSD +0.0% 0.0% -0.0% +0.0% -0.0%
VSM +0.0% 0.0% 0.0% 0.0% 0.0%
anna +0.0% 0.0% +0.1% -0.9% -0.0%
ansi +0.0% 0.0% -0.0% +0.0% +0.0%
atom +0.0% 0.0% 0.0% 0.0% 0.0%
awards +0.0% 0.0% -0.0% +0.0% 0.0%
banner +0.0% 0.0% -0.0% +0.0% 0.0%
bernouilli +0.0% 0.0% +0.0% +0.0% +0.0%
binary-trees +0.0% 0.0% -0.0% -0.0% -0.0%
boyer +0.0% 0.0% +0.0% 0.0% -0.0%
boyer2 +0.0% 0.0% +0.0% 0.0% -0.0%
bspt +0.0% 0.0% +0.0% +0.0% 0.0%
cacheprof +0.0% 0.0% +0.1% -0.8% 0.0%
calendar +0.0% 0.0% -0.0% +0.0% -0.0%
cichelli +0.0% 0.0% +0.0% 0.0% 0.0%
circsim +0.0% 0.0% -0.0% -0.1% -0.0%
clausify +0.0% 0.0% +0.0% +0.0% 0.0%
comp_lab_zift +0.0% 0.0% +0.0% 0.0% -0.0%
compress +0.0% 0.0% +0.0% +0.0% 0.0%
compress2 +0.0% 0.0% 0.0% 0.0% 0.0%
constraints +0.0% 0.0% -0.0% -0.0% -0.0%
cryptarithm1 +0.0% 0.0% +0.0% 0.0% 0.0%
cryptarithm2 +0.0% 0.0% +0.0% -0.0% 0.0%
cse +0.0% 0.0% +0.0% +0.0% 0.0%
digits-of-e1 +0.0% 0.0% -0.0% -0.0% -0.0%
digits-of-e2 +0.0% 0.0% +0.0% -0.0% -0.0%
dom-lt +0.0% 0.0% +0.0% +0.0% 0.0%
eliza +0.0% 0.0% -0.0% +0.0% 0.0%
event +0.0% 0.0% -0.0% -0.0% -0.0%
exact-reals +0.0% 0.0% +0.0% +0.0% +0.0%
exp3_8 +0.0% 0.0% -0.0% -0.0% -0.0%
expert +0.0% 0.0% +0.0% +0.0% +0.0%
fannkuch-redux +0.0% 0.0% +0.0% 0.0% 0.0%
fasta +0.0% 0.0% -0.0% -0.0% -0.0%
fem +0.0% 0.0% +0.0% +0.0% +0.0%
fft +0.0% 0.0% +0.0% -0.0% -0.0%
fft2 +0.0% 0.0% +0.0% +0.0% +0.0%
fibheaps +0.0% 0.0% +0.0% +0.0% 0.0%
fish +0.0% 0.0% +0.0% +0.0% 0.0%
fluid +0.0% 0.0% +0.0% +0.0% +0.0%
fulsom +0.0% 0.0% +0.0% -0.0% +0.0%
gamteb +0.0% 0.0% +0.0% -0.0% -0.0%
gcd +0.0% 0.0% +0.0% +0.0% 0.0%
gen_regexps +0.0% 0.0% +0.0% -0.0% -0.0%
genfft +0.0% 0.0% -0.0% -0.0% -0.0%
gg +0.0% 0.0% 0.0% -0.0% 0.0%
grep +0.0% 0.0% +0.0% +0.0% +0.0%
hidden +0.0% 0.0% +0.0% -0.0% -0.0%
hpg +0.0% 0.0% +0.0% -0.1% -0.0%
ida +0.0% 0.0% +0.0% -0.0% -0.0%
infer +0.0% 0.0% -0.0% -0.0% -0.0%
integer +0.0% 0.0% -0.0% -0.0% -0.0%
integrate +0.0% 0.0% 0.0% +0.0% 0.0%
k-nucleotide +0.0% 0.0% -0.0% -0.0% -0.0%
kahan +0.0% 0.0% -0.0% -0.0% -0.0%
knights +0.0% 0.0% +0.0% -0.0% -0.0%
lambda +0.0% 0.0% +1.2% -6.1% -0.0%
last-piece +0.0% 0.0% +0.0% -0.0% -0.0%
lcss +0.0% 0.0% +0.0% -0.0% -0.0%
life +0.0% 0.0% +0.0% -0.0% -0.0%
lift +0.0% 0.0% +0.0% +0.0% 0.0%
linear +0.0% 0.0% +0.0% +0.0% +0.0%
listcompr +0.0% 0.0% -0.0% -0.0% -0.0%
listcopy +0.0% 0.0% -0.0% -0.0% -0.0%
maillist +0.0% 0.0% +0.0% -0.0% -0.0%
mandel +0.0% 0.0% +0.0% +0.0% +0.0%
mandel2 +0.0% 0.0% +0.0% +0.0% -0.0%
mate +0.0% 0.0% +0.0% +0.0% +0.0%
minimax +0.0% 0.0% -0.0% +0.0% -0.0%
mkhprog +0.0% 0.0% +0.0% +0.0% +0.0%
multiplier +0.0% 0.0% 0.0% +0.0% -0.0%
n-body +0.0% 0.0% +0.0% -0.0% -0.0%
nucleic2 +0.0% 0.0% +0.0% +0.0% -0.0%
para +0.0% 0.0% +0.0% +0.0% +0.0%
paraffins +0.0% 0.0% +0.0% +0.0% +0.0%
parser +0.1% 0.0% +0.4% -1.7% -0.0%
parstof +0.0% 0.0% -0.0% -0.0% -0.0%
pic +0.0% 0.0% +0.0% 0.0% -0.0%
pidigits +0.0% 0.0% -0.0% -0.0% -0.0%
power +0.0% 0.0% +0.0% -0.0% -0.0%
pretty +0.0% 0.0% +0.0% +0.0% +0.0%
primes +0.0% 0.0% +0.0% 0.0% 0.0%
primetest +0.0% 0.0% +0.0% +0.0% +0.0%
prolog +0.0% 0.0% +0.0% +0.0% +0.0%
puzzle +0.0% 0.0% +0.0% +0.0% +0.0%
queens +0.0% 0.0% 0.0% +0.0% +0.0%
reptile +0.0% 0.0% +0.0% +0.0% 0.0%
reverse-complem +0.0% 0.0% -0.0% -0.0% -0.0%
rewrite +0.0% 0.0% +0.0% 0.0% -0.0%
rfib +0.0% 0.0% +0.0% +0.0% +0.0%
rsa +0.0% 0.0% +0.0% +0.0% +0.0%
scc +0.0% 0.0% +0.0% +0.0% +0.0%
sched +0.0% 0.0% +0.0% +0.0% +0.0%
scs +0.0% 0.0% +0.0% +0.0% 0.0%
simple +0.0% 0.0% +0.0% +0.0% +0.0%
solid +0.0% 0.0% +0.0% +0.0% 0.0%
sorting +0.0% 0.0% +0.0% -0.0% 0.0%
spectral-norm +0.0% 0.0% -0.0% -0.0% -0.0%
sphere +0.0% 0.0% +0.0% -1.0% 0.0%
symalg +0.0% 0.0% +0.0% +0.0% +0.0%
tak +0.0% 0.0% +0.0% +0.0% +0.0%
transform +0.0% 0.0% +0.4% -1.3% +0.0%
treejoin +0.0% 0.0% +0.0% -0.0% 0.0%
typecheck +0.0% 0.0% -0.0% +0.0% 0.0%
veritas +0.0% 0.0% +0.0% -0.1% +0.0%
wang +0.0% 0.0% +0.0% +0.0% +0.0%
wave4main +0.0% 0.0% +0.0% 0.0% -0.0%
wheel-sieve1 +0.0% 0.0% +0.0% +0.0% +0.0%
wheel-sieve2 +0.0% 0.0% +0.0% +0.0% 0.0%
x2n1 +0.0% 0.0% +0.0% +0.0% 0.0%
--------------------------------------------------------------------------------
Min +0.0% 0.0% -0.0% -6.1% -0.0%
Max +0.1% 0.0% +1.2% +0.0% +0.0%
Geometric Mean +0.0% -0.0% +0.0% -0.1% -0.0%
NoFib GC Results
================
--------------------------------------------------------------------------------
Program Size Allocs Instrs Reads Writes
--------------------------------------------------------------------------------
circsim +0.0% 0.0% -0.0% -0.0% -0.0%
constraints +0.0% 0.0% -0.0% 0.0% -0.0%
fibheaps +0.0% 0.0% 0.0% -0.0% -0.0%
fulsom +0.0% 0.0% 0.0% -0.6% -0.0%
gc_bench +0.0% 0.0% 0.0% 0.0% -0.0%
hash +0.0% 0.0% -0.0% -0.0% -0.0%
lcss +0.0% 0.0% 0.0% -0.0% 0.0%
mutstore1 +0.0% 0.0% 0.0% -0.0% -0.0%
mutstore2 +0.0% 0.0% +0.0% -0.0% -0.0%
power +0.0% 0.0% -0.0% 0.0% -0.0%
spellcheck +0.0% 0.0% -0.0% -0.0% -0.0%
--------------------------------------------------------------------------------
Min +0.0% 0.0% -0.0% -0.6% -0.0%
Max +0.0% 0.0% +0.0% 0.0% 0.0%
Geometric Mean +0.0% +0.0% +0.0% -0.1% +0.0%
Fixes #14373
These performance regressions appear to be a fluke in CI. See the
discussion in !1742 for details.
Metric Increase:
T6048
T12234
T12425
Naperian
T12150
T5837
T13035
|
|
|
|
| |
Along with some refactoring.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We instead want to collect the runtime stats (with collect_stats, instead of
collect_compiler_stats).
This should fix a number of perf tests failures we have been seeing, where
we suddenly started measuring metrics we didn't intend to measure, which
tend to fall outside of the acceptance window.
Metric Decrease:
lazy-bs-alloc
T3586
Metric Increase:
space_leak_001
T4801
T5835
T12791
|
|
|
|
|
|
| |
Previously we would call check_stats to check the runtime metrics even
if the test definition hadn't requested it. This would result in an
error since the .stats file doesn't exist.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As suggested in #17387; this helps reduce the variance in our residency
sampling.
Metric Increase:
T10370
T3586
lazy-bs-alloc
Metric Decrease 'compile_time/peak_megabytes_allocated':
T1969
Metric Decrease 'runtime/bytes allocated':
space_leak_001
Metric Increase 'compile_time/bytes allocated':
T1969
Metric Increase 'runtime/peak_megabytes_allocated':
space_leak_001
Metric Decrease:
T3064
T9675
|
|
|
|
| |
This was a regression introduced with the Path refactoring.
|
|
|
|
|
| |
Some tests depend on the RTS linker. Introduce a modifier to skip such
tests, in case the RTS linker is not available.
|
| |
|
|
|
|
|
| |
This ensures that all testsuite way names given to `omit_ways`,
`only_ways`, etc. are known ways.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously `makefile_test` and `run_command` tests could easily end up
in a situation where they wouldn't be run if the user used the
`only_ways` modifier. The reason is to build the set of a ways to run
the test in we first start with a candidate set determined by the test
type (e.g. `makefile_test`, `compile_run`, etc.) and then filter that
set with the constraints given by the test's modifiers.
`makefile_test` and `run_command` tests' candidate sets were simply
`{normal}`, and consequently most uses of `only_ways` would result in
the test being never run.
To avoid this we rather use all ways as the candidate sets for these
test types. This may result in a few more testcases than we would like
(given that some `run_command` tests are insensitive to way) but this
can be fixed by adding modifiers and we would much rather run too many
tests than too few.
This fixes #16042 and a number of other tests afflicted by the same issue.
However, there were a few cases that required special attention:
* `T14028` is currently failing and is therefore marked as broken due
to #17300
* `T-signals-child` is fragile in the `threaded1` and `threaded2` ways
(tracked in #17307)
|
|
|
|
|
|
|
|
|
|
|
|
| |
The later caused issues on windows by being translated into
"\\dev\\null" and python then trying to open this non-existant file.
So we now use os.devnull inside python and convert it to "/dev/null"
when calling out to the shell, which is bound to run in a unix like
environment.
This fixes an issue a test producing unexpected stderr output failed
with a framework failure instead of showing a diff of the output.
|
|
|
|
|
| |
Breaks fragile tests into two groups, allowing us to easily preserve
stdout/stderr of failing fragile tests.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Rename requires_th to req_th for consistency with other req functions
(e.g. req_interp, req_profiling etc.)
- req_th (previously requires_th) now checks for interpreter (via
req_interp). With this running TH tests are skipped when running the
test suite with stage=1.
- Test tweaks:
- T9360a, T9360b: Use req_interp
- recomp009, T13938, RAE_T32a: Use req_th
- Fix check-makefiles linter: it now looks for Makefiles instead of .T
files (which are actually Python files)
|
|
|
|
|
| |
Renames performance metrics to include whether they are compile-time or
runtime metrics.
|
|
|
|
|
|
| |
This allows us to run (but ignore the result of) fragile testcases.
Hopefully this should allow us to more easily spot when a fragile test
becomes un-fragile.
|
|
|
|
|
|
|
|
|
| |
This tries to put the testsuite driver into a slightly more maintainable
condition:
* Add type annotations where easily done
* Use pathlib.Path instead of str paths
* Make it pass the mypy typechecker
|
|
|
|
|
|
| |
Previously there were a few cases where operations like `omit_ways`
were incorrectly passed a single way (e.g. `omit_ways('threaded2')`).
This won't work as the author expected.
|
|
|
|
|
|
|
|
|
|
| |
Since we can't load profiled objects when GhcDynamic==YES. Affects:
* T16737
* T16384
* T16718
* T16619
* T16190
|
|
|
|
| |
omit_ways expects a list but this was broken in several cases.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This uses the Chart.js javascript library.
Everything is put into a standalone .html file and opened with the
default browser.
I also simplified the text output to use the same data as the chart.
You can now use a commit range with git's ".." syntax.
The --ci option will use results from CI (you'll need to fetch them
first):
$ git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf
$ python3 testsuite/driver/perf_notes.py --ci --chart --test-env x86_64-darwin --test-name T9630 master~500..master
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
They are a particular type of perf tests. This patch introduces a
'stats_files_dir' configuration field in the testsuite driver where all
haddock timing files (and possibly others in the future) are assumed to live.
We also change both the Make and Hadrian build systems to pass respectively
$(TOP)/testsuite/tests/perf/haddock/ and
<build root>/stage1/haddock-timing-files/ as the value of that new
configuration field, and to generate the timing files in those directories
in the first place while generating documentation with haddock.
This new test type can be seen as one dedicated to examining stats files that
are generated while building a GHC distribution. This also lets us get rid of
the 'extra_files' directives in the all.T entries for haddock.base,
haddock.Cabal and haddock.compiler.
|
|
|
|
|
| |
Useful progress indicator even when `make test VERBOSE=1`,
and when you do something else, but have terminal title visible.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes #16052
When the offset in `setByteArray#` is statically known, we can provide
better alignment guarantees then just 1 byte.
Also, memset can now do 64-bit wide sets.
The current memset intrinsic is not optimal however and can be
improved for the case when we know that we deal with
(baseAddress at known alignment) + offset
For instance, on 64-bit
`setByteArray# s 1# 23# 0#`
given that bytearray is 8 bytes aligned could be unrolled into
`movb, movw, movl, movq, movq`; but currently it is
`movb x23` since alignment of 1 is all we can embed into MO_Memset op.
|
| |
|
|
|
|
| |
Fixes #16425
|
| |
|
|
|
|
|
| |
This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.
|
|
|
|
|
|
|
|
|
|
| |
Now since we have been a bit more stringent in testsuite cleanliness we
have been marking a lot of tests as fragile using the `skip` modifier.
However, this unfortunately means that we lose the association with the
ticket number documenting the fragility.
Here we introduce `fragile` and `fragile_for` to retain this
information.
|
| |
|
|
|
|
|
|
| |
This was causing gitlab to not report from builds as failing. It also
highlighted a problem with the LLVM tests where some of the external
interpreter tests are failing.
|
|
|
|
| |
Respect `inside_git_repo()` when checking performance stats.
|
| |
|
|
|
|
| |
and CI results."
|
|
|
|
|
|
|
|
|
|
| |
This patch makes the JUnit output more useful as now we also report the
stdout/stderr in the message which can be used to quickly identify why a
test is failing without downloading the log.
This also introduces TestResult,
previously we were simply passing around tuples, making things the
implementation rather difficult to follow and harder to extend.
|
|
|
|
| |
As noted in #16205 this configuration reliably segfaults.
|
|
|
|
|
|
|
|
|
| |
results."
Unfortunately this has broken all future commits due to spurious(?)
performance changes which I have been unable to work around.
This reverts commit cc2261d42f6a954d88e355aaad41f001f65c95da.
|
|
|
|
| |
gitlab-ci: push performance metrics as git notes to the "GHC Performance Notes" repository.
|
|
|
|
|
| |
This eliminates most uses of run_command in the testsuite in favor of the more
structured makefile_test.
|
| |
|