| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
They are a particular type of perf tests. This patch introduces a
'stats_files_dir' configuration field in the testsuite driver where all
haddock timing files (and possibly others in the future) are assumed to live.
We also change both the Make and Hadrian build systems to pass respectively
$(TOP)/testsuite/tests/perf/haddock/ and
<build root>/stage1/haddock-timing-files/ as the value of that new
configuration field, and to generate the timing files in those directories
in the first place while generating documentation with haddock.
This new test type can be seen as one dedicated to examining stats files that
are generated while building a GHC distribution. This also lets us get rid of
the 'extra_files' directives in the all.T entries for haddock.base,
haddock.Cabal and haddock.compiler.
|
|
|
|
|
|
| |
We attempt to get 75 commit hashes via `git log`, but this only gave 10
hashes in a CI run (see #16662). Better logging may help solve this
error if it occurs again in the future.
|
|
|
|
|
| |
Useful progress indicator even when `make test VERBOSE=1`,
and when you do something else, but have terminal title visible.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes #16052
When the offset in `setByteArray#` is statically known, we can provide
better alignment guarantees then just 1 byte.
Also, memset can now do 64-bit wide sets.
The current memset intrinsic is not optimal however and can be
improved for the case when we know that we deal with
(baseAddress at known alignment) + offset
For instance, on 64-bit
`setByteArray# s 1# 23# 0#`
given that bytearray is 8 bytes aligned could be unrolled into
`movb, movw, movl, movq, movq`; but currently it is
`movb x23` since alignment of 1 is all we can embed into MO_Memset op.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This moves all URL references to Trac Wiki to their corresponding
GitLab counterparts.
This substitution is classified as follows:
1. Automated substitution using sed with Ben's mapping rule [1]
Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy...
New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy...
2. Manual substitution for URLs containing `#` index
Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy...#Zzz
New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy...#zzz
3. Manual substitution for strings starting with `Commentary`
Old: Commentary/XxxYyy...
New: commentary/xxx-yyy...
See also !539
[1]: https://gitlab.haskell.org/bgamari/gitlab-migration/blob/master/wiki-mapping.json
|
|
|
|
| |
Fixes #16425
|
| |
|
|
|
|
|
| |
This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.
|
|
|
|
|
|
|
|
|
|
| |
Now since we have been a bit more stringent in testsuite cleanliness we
have been marking a lot of tests as fragile using the `skip` modifier.
However, this unfortunately means that we lose the association with the
ticket number documenting the fragility.
Here we introduce `fragile` and `fragile_for` to retain this
information.
|
|
|
|
|
|
| |
expected changes on the current commit.
Trac #16359
|
|
|
|
| |
Python's split() function is used to split on all white space.
|
| |
|
|
|
|
|
|
| |
This was causing gitlab to not report from builds as failing. It also
highlighted a problem with the LLVM tests where some of the external
interpreter tests are failing.
|
| |
|
|
|
|
| |
Respect `inside_git_repo()` when checking performance stats.
|
| |
|
| |
|
|
|
|
| |
and CI results."
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This patch makes the JUnit output more useful as now we also report the
stdout/stderr in the message which can be used to quickly identify why a
test is failing without downloading the log.
This also introduces TestResult,
previously we were simply passing around tuples, making things the
implementation rather difficult to follow and harder to extend.
|
|
|
|
| |
As noted in #16205 this configuration reliably segfaults.
|
|
|
|
|
|
|
|
|
| |
results."
Unfortunately this has broken all future commits due to spurious(?)
performance changes which I have been unable to work around.
This reverts commit cc2261d42f6a954d88e355aaad41f001f65c95da.
|
|
|
|
| |
gitlab-ci: push performance metrics as git notes to the "GHC Performance Notes" repository.
|
|
|
|
|
| |
This eliminates most uses of run_command in the testsuite in favor of the more
structured makefile_test.
|
| |
|
|
|
|
| |
This reverts commit 76c8fd674435a652c75a96c85abbf26f1f221876.
|
| |
|
|
|
|
|
| |
Otherwise the testsuite driver crashes when run multiple times with CLEANUP=NO
on a test containing such extra_files.
|
|
|
|
|
|
|
| |
Previously testing code-generation for ISA extensions was nearly impossible
since we had no ability to determine whether the host supports the needed
extension. Here we fix this by introducing a simple /proc/cpuinfo-based
testsuite predicate. We really ought to
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: monoidal
Reviewed By: monoidal
Subscribers: monoidal, rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5056
|
|
|
|
|
|
|
|
|
|
|
|
| |
A bunch of tests for `integer-simple` were now broken for a foolish reason:
unlike the `integer-gmp` case, there is no CorePrep optimization for turning
small integers directly into applications of `S#`.
Rather than port this optimization to `integer-simple` (which would involve
moving a bunch of `integer-simple` names into `PrelNames`), I switched
as many tests as possible to use `Int`.
The printing of `Integer` is already tested in `print037`.
|
| |
|
|
|
|
|
|
|
| |
... for testing presence of NCG
This commit adds a criterion for checking whether we can expect sensible
output from --ddump-asm.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Mark arith011 as broken with integer-simple
As noted in #16091, arith011 fails when run against integer-simple with a
"divide by zero" exception. This suggests that integer-gmp and integer-simple
are handling division by zero differently.
* This also fixes broken_without_gmp; the lack of types made the previous
failure silent, sadly. Improves situation of #16043.
* Mark several tests implicitly depending upon integer-gmp as broken
with integer-simple. These expect to see Core coming from integer-gmp,
which breaks with integer-simple.
* Increase runtime timeout multiplier of T11627a with integer-simple
I previously saw that T11627a timed out in all profiling ways when run against
integer-simple. I suspect this is due to integer-simple's rather verbose heap
representation. Let's see whether increasing the runtime timeout helps.
Fixes test for #11627.
This is all in service of fixing #16043.
|
|
|
|
|
|
|
| |
This reverts commit e59439af3222d151918ad1ad2a03942ce9e6a1ff.
This is causing unexpected failures in some test ways. Further proof
that no change is too trivial for CI.
|
|
|
|
|
| |
The lack of types made the previous failure silent, sadly. Improves
situation of #16043.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a variety of testsuite failures with integer-simple of the form
```
--- typecheck/should_fail/tcfail072.run/tcfail072.stderr.normalised
+++ typecheck/should_fail/tcfail072.run/tcfail072.comp.stderr.normalised
@@ -12,7 +12,7 @@
-- Defined in ‘integer-<IMPL>-<VERSION>:GHC.Integer.Type’
instance Ord () -- Defined in ‘GHC.Classes’
...plus 21 others
- ...plus three instances involving out-of-scope types
+ ...plus two instances involving out-of-scope types
(use -fprint-potential-instances to see them all)
In the expression: g A
In an equation for ‘g’: g (B _ _) = g A
```
In service of fixing #16043.
|
|
|
|
| |
Towards fixing #16043.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: bgamari, tdammers
Reviewed By: bgamari, tdammers
Subscribers: rwbarton, carter
GHC Trac Issues: #15924
Differential Revision: https://phabricator.haskell.org/D5368
|
|
|
|
|
| |
Git status is extremely expensive for this task. We instead use `git rev-parse
HEAD` and throw away the output to ensure we don't spam the user.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: bgamari, tdammers, osa1
Reviewed By: tdammers
Subscribers: osa1, tdammers, rwbarton, carter
GHC Trac Issues: #15923
Differential Revision: https://phabricator.haskell.org/D5367
|
|
|
|
|
|
|
|
|
|
| |
Test Plan: ./validate
Reviewers: bgamari, simonmar
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5362
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The option `confdir` (used in GhostScript test) was set correctly
via `--config` in `test.mk` and incorrectly via `config/ghc`.
AFAICT, some time ago this was working because the
incorrect assignment was done first, and later it broke.
Hardian doesn't pass `confdir`. I removed `confdir` and use
`config.top` to determine the directory of the
`good.ps` and `bad.ps` files. This is simpler.
I also removed some redundant assignments in `config/ghc`.
Test Plan: manually set config.have_profiling and make test
Reviewers: bgamari
Subscribers: rwbarton, carter
GHC Trac Issues: #15856
Differential Revision: https://phabricator.haskell.org/D5298
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch makes the following improvement:
- Automatically records test metrics (per test environment) so that
the programmer need not supply nor update expected values in *.T
files.
- On expected metric changes, the programmer need only indicate the
direction of change in the git commit message.
- Provides a simple python tool "perf_notes.py" to compare metrics
over time.
Issues:
- Using just the previous commit allows performance to drift with each
commit.
- Currently we allow drift as we have a preference for minimizing
false positives.
- Some possible alternatives include:
- Use metrics from a fixed commit per test: the last commit that
allowed a change in performance (else the oldest metric)
- Or use some sort of aggregate since the last commit that allowed
a change in performance (else all available metrics)
- These alternatives may result in a performance issue (with the
test driver) having to heavily search git commits/notes.
- Run locally, performance tests will trivially pass unless the tests
were run locally on the previous commit. This is often not the case
e.g. after pulling recent changes.
Previously, *.T files contain statements such as:
```
stats_num_field('peak_megabytes_allocated', (2, 1))
compiler_stats_num_field('bytes allocated',
[(wordsize(64), 165890392, 10)])
```
This required the programmer to give the expected values and a tolerance
deviation (percentage). With this patch, the above statements are
replaced with:
```
collect_stats('peak_megabytes_allocated', 5)
collect_compiler_stats('bytes allocated', 10)
```
So that programmer must only enter which metrics to test and a tolerance
deviation. No expected value is required. CircleCI will then run the
tests per test environment and record the metrics to a git note for that
commit and push them to the git.haskell.org ghc repo. Metrics will be
compared to the previous commit. If they are different by the tolerance
deviation from the *.T file, then the corresponding test will fail. By
adding to the git commit message e.g.
```
# Metric (In|De)crease <metric(s)> <options>: <tests>
Metric Increase ['bytes allocated', 'peak_megabytes_allocated'] \
(test_env='linux_x86', way='default'):
Test012, Test345
Metric Decrease 'bytes allocated':
Test678
Metric Increase:
Test711
```
This will allow the noted changes (letting the test pass). Note that by
omitting metrics or options, the change will apply to all possible
metrics/options (i.e. in the above, an increase for all metrics in all
test environments is allowed for Test711)
phabricator will use the message in the description
Reviewers: bgamari, hvr
Reviewed By: bgamari
Subscribers: rwbarton, carter
GHC Trac Issues: #12758
Differential Revision: https://phabricator.haskell.org/D5059
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: Just as it says on the tin.
Test Plan: Validate
Reviewers: bgamari, osa1
Reviewed By: osa1
Subscribers: osa1, monoidal, rwbarton, thomie, carter
Differential Revision: https://phabricator.haskell.org/D5010
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
- Avoid import *; this helps tools such as pyflakes.
The last occurrence in runtests.py is not easy to remove
as it's used by .T files.
- Use False/True instead of 0/1.
Test Plan: validate
Reviewers: bgamari, thomie, simonmar
Reviewed By: thomie
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5062
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
- remove clean_cmd
- framework_failures was undefined
- times_file was not used
- if_verbose_dump was called only when verbose >= 1; remove the check
- simplify normalise_whitespace
Test Plan: validate
Reviewers: bgamari, thomie
Reviewed By: thomie
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5061
|