| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This re-applies {D5195} with fixes for i386:
* Fix unused label warnings, see {D5230} or {D5273}
* Fix a silly bug introduced by moving `#if`
{P190}
Add a RTS option -xp to load PIC object anywhere in address space. We do
this by relaxing the requirement of <0x80000000 result of
`mmapForLinker` and implying USE_CONTIGUOUS_MMAP.
We also need to change calls to `ocInit` and `ocGetNames` to avoid
dangling pointers when the address of `oc->image` is changed by
`ocAllocateSymbolExtra`.
Test Plan:
See {D5195}, also test under i386:
```
$ uname -a
Linux watashi-arch32 4.18.5-arch1-1.0-ARCH #1 SMP PREEMPT Tue Aug 28
20:45:30 CEST 2018 i686 GNU/Linux
$ cd testsuite/tests/th/ && make test
...
```
will run `./validate` on stacked diff.
Reviewers: simonmar, bgamari, alpmestan, trommler, hvr, erikd
Reviewed By: simonmar
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5289
|
|
|
|
| |
This reverts commit 76c8fd674435a652c75a96c85abbf26f1f221876.
|
| |
|
| |
|
|
|
|
|
|
| |
Complete b7b6617a90824303daf555c817f538cd9c792671 (see ticket #15086) to
actually enable profiling by closure type in the profiling rts. I.e.
+RTS -p -hT is now accepted.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add default value
- Update the help line as multiple short lines instead of one long line,
for consistency with other lines.
Reviewers: simonmar, bgamari, erikd
Reviewed By: bgamari
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5445
|
|
|
|
|
|
|
|
|
|
|
|
| |
Test Plan: `./inplace/bin/ghc-stage2 +RTS --help"
Reviewers: bgamari, sgraf, erikd
Reviewed By: sgraf
Subscribers: adamse, rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5428
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces the `+RTS -ol` flag, which allows user to specify the
destination file for eventlog output.
Test Plan: Validate with included test
Reviewers: simonmar, erikd
Reviewed By: simonmar
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5293
|
|
|
|
| |
This reverts commit 5403a8636fe82f971234873564f3a05393b89b7a.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a RTS option -xp to load PIC object anywhere in address space. We do
this by relaxing the requirement of <0x80000000 result of
`mmapForLinker` and implying USE_CONTIGUOUS_MMAP.
We also need to change calls to `ocInit` and `ocGetNames` to avoid
dangling pointers when the address of `oc->image` is changed by
`ocAllocateSymbolExtra`.
Test Plan:
```
$ uname -a
Linux localhost 4.18.8-arch1-1-ARCH #1 SMP PREEMPT Sat Sep 15 20:34:48
UTC 2018 x86_64 GNU/Linux
$ cat mk/build.mk
DYNAMIC_GHC_PROGRAMS = NO
DYNAMIC_BY_DEFAULT = NO
GhcRTSWays += thr_debug
EXTRA_HC_OPTS += -debug
WAY_p_HC_OPTS += -fPIC -fexternal-dynamic-refs
$ inplace/bin/ghc-stage2 --interactive -prof +RTS -xp
GHCi, version 8.7.20180928: http://www.haskell.org/ghc/ :? for help
ghc-stage2: R_X86_64_32 relocation out of range:
ghczmprim_GHCziTypes_ZMZN_closure = 7f690bffab59
Recompile
/data/users/watashi/ghc/libraries/ghc-prim/dist-install/build/HSghc-prim
-0.5.3.o with -fPIC -fexternal-dynamic-refs.
ghc-stage2: unable to load package `ghc-prim-0.5.3'
$ strace -f -e open,mmap inplace/bin/ghc-stage2 --interactive -prof
-fexternal-interpreter -opti+RTS -opti-xp
...
[pid 1355283]
open("/data/users/watashi/ghc/libraries/base/dist-install/build/libHSbas
e-4.12.0.0_p.a", O_RDONLY) = 14
[pid 1355283] mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a84842000
[pid 1355283]
open("/data/users/watashi/ghc/libraries/base/dist-install/build/libHSbas
e-4.12.0.0_p.a", O_RDONLY) = 14
[pid 1355283] mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a84676000
...
Prelude> System.Posix.Process.getProcessID
...
[pid 1355283]
open("/data/users/watashi/ghc/libraries/unix/dist-install/build/libHSuni
x-2.7.2.2_p.a", O_RDONLY) = 14
[pid 1355283] mmap(NULL, 45056, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a67d60000
[pid 1355283]
open("/data/users/watashi/ghc/libraries/unix/dist-install/build/libHSuni
x-2.7.2.2_p.a", O_RDONLY) = 14
[pid 1355283] mmap(NULL, 57344, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6a67d52000
...
```
```
$ uname -a
Darwin watashis-iMac.local 18.0.0 Darwin Kernel Version 18.0.0: Wed Aug
22 20:13:40 PDT 2018; root:xnu-4903.201.2~1/RELEASE_X86_64 x86_64
$ mv
/Users/watashi/gao/ghc/libraries/integer-gmp/dist-install/build/HSintege
r-gmp-1.0.2.0.o{,._DISABLE_GHC_ISSUE_15105}
$ inplace/bin/ghc-stage2 --interactive +RTS -xp
GHCi, version 8.7.20181003: http://www.haskell.org/ghc/ :? for help
Prelude> System.Posix.Process.getProcessID
42791
Prelude> Data.Set.fromList [1 .. 10]
fromList [1,2,3,4,5,6,7,8,9,10]
Prelude>
Leaving GHCi.
$ inplace/bin/ghc-stage2 --interactive -prof -fexternal-interpreter
GHCi, version 8.7.20181003: http://www.haskell.org/ghc/ :? for help
Prelude> System.Posix.Process.getProcessID
42806
Prelude> Data.Set.fromList [1 .. 10]
fromList [1,2,3,4,5,6,7,8,9,10]
Prelude>
Leaving GHCi.
```
Also test with something that used to hit the 2Gb limit and it loads
and runs without problem.
Reviewers: simonmar, bgamari, angerman, Phyx, hvr, erikd
Reviewed By: simonmar
Subscribers: rwbarton, carter
Differential Revision: https://phabricator.haskell.org/D5195
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add an additional line to the output of +RTS --info. It shows the value
of the flag -with-rtsopts provided at compile/link time.
Test Plan: make test TESTS="T15261a T15261b"
Reviewers: hvr, erikd, dfeuer, thomie, austin, bgamari, simonmar, osa1,
monoidal
Reviewed By: osa1, monoidal
Subscribers: osa1, rwbarton, carter
GHC Trac Issues: #15261
Differential Revision: https://phabricator.haskell.org/D5053
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Test Plan: Validate, run program with `+RTS --numa` without libnuma
support compiled in
Reviewers: erikd, simonmar
Subscribers: thomie, carter
GHC Trac Issues: #14956
Differential Revision: https://phabricator.haskell.org/D4556
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we inexplicably disabled support for `-hT` profiling in the profiled
way. Admittedly, there are relatively few cases where one would prefer -hT to
`-hd`, but the option should nevertheless be available for the sake of
consistency.
Note that this does mean that there is a bit of an inconsistency in the behavior
of `-h`: in the profiled way `-h` behaves like `-hc` whereas in the non-profiled
way it defaults to `-hT`.
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: erikd, simonmar
Subscribers: thomie, carter
GHC Trac Issues: #15086
Differential Revision: https://phabricator.haskell.org/D4643
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This shims out fopen and sopen so that they use modern APIs under the hood
along with namespaced paths.
This lifts the MAX_PATH restrictions from Haskell programs and makes the new
limit ~32k.
There are only some slight caveats that have been documented.
Some utilities have not been upgraded such as lndir, since all these things are
different cabal packages I have been forced to copy the source in different places
which is less than ideal. But it's the only way to keep sdist working.
Test Plan: ./validate
Reviewers: hvr, bgamari, erikd, simonmar
Reviewed By: bgamari
Subscribers: rwbarton, thomie, carter
GHC Trac Issues: #10822
Differential Revision: https://phabricator.haskell.org/D4416
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The existing internal counters:
* gc_alloc_block_sync
* whitehole_spin
* gen[g].sync
* gen[1].sync
are now not shown in the -s report unless --internal-counters is also passed.
If --internal-counters is passed we now show the counters above, reformatted, as
well as several other counters. In particular, we now count the yieldThread()
calls that SpinLocks do as well as their spins.
The added counters are:
* gc_spin (spin and yield)
* mut_spin (spin and yield)
* whitehole_threadPaused (spin only)
* whitehole_executeMessage (spin only)
* whitehole_lockClosure (spin only)
* waitForGcThreadsd (spin and yield)
As well as the following, which are not SpinLock-like things:
* any_work
* do_work
* scav_find_work
See the Note for descriptions of what these counters are.
We add busy_wait_nops in these loops along with the counter increment where it
was absent.
Old internal counters output:
```
gc_alloc_block_sync: 0
whitehole_gc_spin: 0
gen[0].sync: 0
gen[1].sync: 0
```
New internal counters output:
```
Internal Counters:
Spins Yields
gc_alloc_block_sync 323 0
gc_spin 9016713 752
mut_spin 57360944 47716
whitehole_gc 0 n/a
whitehole_threadPaused 0 n/a
whitehole_executeMessage 0 n/a
whitehole_lockClosure 0 0
waitForGcThreads 2 415
gen[0].sync 6 0
gen[1].sync 1 0
any_work 2017
no_work 2014
scav_find_work 1004
```
Test Plan:
./validate
Check it builds with #define PROF_SPIN removed from includes/rts/Config.h
Reviewers: bgamari, erikd, simonmar, hvr
Reviewed By: simonmar
Subscribers: rwbarton, thomie, carter
GHC Trac Issues: #3553, #9221
Differential Revision: https://phabricator.haskell.org/D4302
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: bgamari, erikd, simonmar
Reviewed By: bgamari
Subscribers: rwbarton, thomie, carter
Differential Revision: https://phabricator.haskell.org/D4441
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This was presumably a vestige of the days when the profiled RTS couldn't
run threaded. Fixes #14545.
Test Plan: simonmar
Reviewers: erikd, simonmar
Reviewed By: simonmar
Subscribers: rwbarton, thomie, carter
GHC Trac Issues: #14545
Differential Revision: https://phabricator.haskell.org/D4245
|
|
|
|
|
|
|
|
|
|
| |
Reviewers: erikd, simonmar, hvr
Reviewed By: hvr
Subscribers: hvr, rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D4238
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
GC sync is the time between a GC being intiated and all the mutator
threads finally stopping so that the GC can start. Problems that cause
the GC sync to be delayed are hard to find and can cause dramatic
slowdowns for heavily parallel programs.
The new flag --long-gc-sync=<time> helps by emitting a warning and
calling a user-overridable hook when the GC sync time exceeds the
specified threshold. A debugger can be used to set a breakpoint when
this happens and inspect the stacks of threads to find the culprit.
Test Plan:
```
$ ./inplace/bin/ghc-stage2 +RTS --long-gc-sync=0.0000001 -S
Alloc Copied Live GC GC TOT TOT Page Flts
bytes bytes bytes user elap user elap
1135856 51144 153736 0.000 0.000 0.002 0.002 0 0 (Gen: 0)
1034760 94704 188752 0.000 0.000 0.002 0.002 0 0 (Gen: 0)
1038888 134832 228888 0.009 0.009 0.011 0.011 0 0 (Gen: 1)
1025288 90128 235184 0.000 0.000 0.012 0.012 0 0 (Gen: 0)
1049088 130080 333984 0.000 0.000 0.013 0.013 0 0 (Gen: 0)
Warning: waited 0us for GC sync
1034424 73360 331976 0.000 0.000 0.013 0.013 0 0 (Gen: 0)
```
Also tested on a real production problem.
Reviewers: niteria, bgamari, erikd
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D4193
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This patch adds the ability to generate stack traces on crashes for Windows.
When running in the interpreter this attempts to use symbol information from
the interpreter and information we know about the loaded object files to
resolve addresses to symbols.
When running compiled it doesn't have this information and then defaults
to using symbol information from PDB files. Which for now means only
files compiled with ICC or MSVC will show traces compiled.
But I have a future patch that may address this shortcoming.
Also since I don't know how to walk a pure haskell stack, I can for now
only show the last entry. I'm hoping to figure out how Apply.cmm works to
be able to walk the stalk and give more entries for pure haskell code.
In GHCi
```
$ echo main | inplace/bin/ghc-stage2.exe --interactive ./testsuite/tests/rts/derefnull.hs
GHCi, version 8.3.20170830: http://www.haskell.org/ghc/ :? for help
Ok, 1 module loaded.
Prelude Main>
Access violation in generated code when reading 0x0
Attempting to reconstruct a stack trace...
Frame Code address
* 0x77cde10 0xc370229 E:\..\base\dist-install\build\HSbase-4.10.0.0.o+0x190031
(base_ForeignziStorable_zdfStorableInt4_info+0x3f)
```
and compiled
```
Access violation in generated code when reading 0x0
Attempting to reconstruct a stack trace...
Frame Code address
* 0xf0dbd0 0x40bb01 E:\..\rts\derefnull.run\derefnull.exe+0xbb01
```
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar
Reviewed By: bgamari
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3913
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's often hard to debug things like segfaults on Windows,
mostly because gdb isn't always of use and users don't know
how to effectively use it.
This patch provides a way to create a crash drump by passing
`+RTS --generate-crash-dumps` as an option. If any unhandled
exception is triggered a dump is made that contains enough
information to be able to diagnose things successfully.
Currently the created dumps are a bit big because I include
all registers, code and threads information.
This looks like
```
$ testsuite/tests/rts/derefnull.run/derefnull.exe +RTS
--generate-crash-dumps
Access violation in generated code when reading 0000000000000000
Crash dump created. Dump written to:
E:\msys64\tmp\ghc-20170901-220250-11216-16628.dmp
```
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar
Reviewed By: bgamari, simonmar
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3912
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Exception handling on Windows is unfortunately a bit complicated.
But essentially the VEH Handlers we currently have are running too
early.
This was a problem as it ran so early it also swallowed C++ exceptions
and other software exceptions which the system could have very well
recovered from.
So instead we use a sequence of chains to for the exception handlers to
run as late as possible. You really can't get any later than this.
Please read the comment in the patch for more details.
I'm also providing a switch to allow people to turn off the exception
handling entirely. In case it does present a problem with their code.
(Reverted and recommitted to fix authorship information)
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar
Reviewed By: bgamari
Subscribers: rwbarton, thomie
GHC Trac Issues: #13911, #12110
Differential Revision: https://phabricator.haskell.org/D3911
|
|
|
|
|
|
| |
Reverting to fix authorship of commit.
This reverts commit 1825cbdbdf08ed4bd6fd6794852596078953298a.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Exception handling on Windows is unfortunately a bit complicated.
But essentially the VEH Handlers we currently have are running too
early.
This was a problem as it ran so early it also swallowed C++ exceptions
and other software exceptions which the system could have very well
recovered from.
So instead we use a sequence of chains to for the exception handlers to
run as late as possible. You really can't get any later than this.
Please read the comment in the patch for more details.
I'm also providing a switch to allow people to turn off the exception
handling entirely. In case it does present a problem with their code.
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar
Reviewed By: bgamari
Subscribers: rwbarton, thomie
GHC Trac Issues: #13911, #12110
Differential Revision: https://phabricator.haskell.org/D3911
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
db2a667655506c43dd3c8260d29031bde55f1bee added `-po` option, but
the part that parses it was missing.
Test Plan:
On a simple file:
```
./inplace/bin/ghc-stage2 A.hs -prof -main-is A; ./A +RTS -P -potest
```
produced test.prof file and didn't produce A.prof file.
```
./A +RTS -P
```
produced A.prof file
Reviewers: simonmar, bgamari, austin, erikd
Reviewed By: bgamari
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3946
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This sprintf is safe thanks to the guarantees on the format strings that
we pass to it. Well, almost. The GR_FILENAME_FMT_GUM format would not
have satisfied them if it was still used.
If someone makes a mistake that's a potential privilege escalation,
so I think it's reasonable to switch to snprintf to protect against
that remote possibility.
Test Plan: it builds, CI
Reviewers: simonmar, bgamari, austin, erikd
Reviewed By: bgamari
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3944
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These ignore commandline arguments for ignore and commandline as well as
GHCRTS arguments for ignoreAll. Passing RTS flags given on the command
line along to the program by simply skipping processing of these flags
by the RTS.
This fixes #12870.
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar
Reviewed By: simonmar
Subscribers: Phyx, rwbarton, thomie
GHC Trac Issues: #12870
Differential Revision: https://phabricator.haskell.org/D3740
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
Get utf8 encoded arguments before we call hs_init and use them
instead of ignoring hs_init arguments. This reduces differing
behaviour of the RTS between windows and linux and simplifies
the code involved.
A few testcases were changed to expect the same result on windows
as on linux after the changes.
This fixes #13940.
Test Plan: ./validate
Reviewers: austin, hvr, bgamari, erikd, simonmar, Phyx
Subscribers: Phyx, rwbarton, thomie
GHC Trac Issues: #13940
Differential Revision: https://phabricator.haskell.org/D3739
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems that 12ad4d417b89462ba8e19a3c7772a931b3a93f0e enabled
collection by default as its needs stats.allocated_bytes to determine
whether the program has exceeded its grace limit.
However, enabling stats also enables some potentially expensive times
checks. In general GC statistics should be cheap to compute (relative
to the GC itself), so now we always compute them. This allows us to once
again disable giveStats by default.
Fixes #13864.
Reviewers: simonmar, austin, erikd
Reviewed By: simonmar
Subscribers: rwbarton, thomie
GHC Trac Issues: #13864
Differential Revision: https://phabricator.haskell.org/D3669
|
|
|
|
|
|
|
|
|
|
|
|
| |
Test Plan: validate
Reviewers: austin, bgamari, erikd, simonmar
Reviewed By: bgamari, simonmar
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3648
|
|
|
|
| |
Our new CPP linter enforces this.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a RTS option, -po, which allows the user to override the stem
used to form the output file names of the heap profile and cost center summary.
It's a bit unclear to me whether this is really the interface we want.
Alternatively we could just allow the user to specify the `.hp` and `.prof` file
names separately. This would arguably be a bit more straightforward and would
allow the user to name JSON output with an appropriate `.json` suffix if they so
desired. However, this would come at the cost of taking more of the option
space, which is a somewhat precious commodity.
Test Plan: Validate, try using `-po` RTS option
Reviewers: simonmar, austin, erikd
Reviewed By: simonmar
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D3182
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
getArgs didn't match the treatmeant of -- in the RTS leading to
inconsistencies between behavior on Windows and other platforms. See #13287.
Reviewers: austin, hvr, bgamari, erikd, simonmar, rwbarton
Reviewed By: bgamari, rwbarton
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D3147
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces a JSON output format for cost-centre profiler reports.
It's not clear whether this is really something we want to introduce
given that we may also move to a more Haskell-driven output pipeline in
the future, but I nevertheless found this helpful, so I thought I would
put it up.
Test Plan: Compile a program with `-prof -fprof-auto`; run with `+RTS
-pj`
Reviewers: austin, erikd, simonmar
Reviewed By: simonmar
Subscribers: duncan, maoe, thomie, simonmar
Differential Revision: https://phabricator.haskell.org/D3132
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently eventlog data is always written to a file `progname.eventlog`.
This patch introduces the `flushEventLog` field in `RtsConfig` which
allows to customize the writing of eventlog data.
One possible scenario is the ongoing live-profile-monitor effort by
@NCrashed which slurps all eventlog data through `fluchEventLog`.
`flushEventLog` takes a buffer with eventlog data and its size and
returns `false` (0) in case eventlog data could not be procesed.
Reviewers: simonmar, austin, erikd, bgamari
Reviewed By: simonmar, bgamari
Subscribers: qnikst, thomie, NCrashed
Differential Revision: https://phabricator.haskell.org/D2934
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This changes heap overflow to throw a HeapOverflow exception instead of
killing the process.
Test Plan: GHC CI
Reviewers: simonmar, austin, hvr, erikd, bgamari
Reviewed By: simonmar, bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2790
GHC Trac Issues: #1791
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
This commit makes various improvements and addresses some issues with
Compact Regions (aka Compact Normal Forms).
This was the most important thing I wanted to fix. Compaction
previously prevented GC from running until it was complete, which
would be a problem in a multicore setting. Now, we compact using a
hand-written Cmm routine that can be interrupted at any point. When a
GC is triggered during a sharing-enabled compaction, the GC has to
traverse and update the hash table, so this hash table is now stored
in the StgCompactNFData object.
Previously, compaction consisted of a deepseq using the NFData class,
followed by a traversal in C code to copy the data. This is now done
in a single pass with hand-written Cmm (see rts/Compact.cmm). We no
longer use the NFData instances, instead the Cmm routine evaluates
components directly as it compacts.
The new compaction is about 50% faster than the old one with no
sharing, and a little faster on average with sharing (the cost of the
hash table dominates when we're doing sharing).
Static objects that don't (transitively) refer to any CAFs don't need
to be copied into the compact region. In particular this means we
often avoid copying Char values and small Int values, because these
are static closures in the runtime.
Each Compact# object can support a single compactAdd# operation at any
given time, so the Data.Compact library now enforces mutual exclusion
using an MVar stored in the Compact object.
We now get exceptions rather than killing everything with a barf()
when we encounter an object that cannot be compacted (a function, or a
mutable object). We now also detect pinned objects, which can't be
compacted either.
The Data.Compact API has been refactored and cleaned up. A new
compactSize operation returns the size (in bytes) of the compact
object.
Most of the documentation is in the Haddock docs for the compact
library, which I've expanded and improved here.
Various comments in the code have been improved, especially the main
Note [Compact Normal Forms] in rts/sm/CNF.c.
I've added a few tests, and expanded a few of the tests that were
there. We now also run the tests with GHCi, and in a new test way
that enables sanity checking (+RTS -DS).
There's a benchmark in libraries/compact/tests/compact_bench.hs for
measuring compaction speed and comparing sharing vs. no sharing.
The field totalDataW in StgCompactNFData was unnecessary.
Test Plan:
* new unit tests
* validate
* tested manually that we can compact Data.Aeson data
Reviewers: gcampax, bgamari, ezyang, austin, niteria, hvr, erikd
Subscribers: thomie, simonpj
Differential Revision: https://phabricator.haskell.org/D2751
GHC Trac Issues: #12455
|
|
|
|
|
|
|
|
|
|
|
|
| |
Test Plan: Validate on lots of platforms
Reviewers: erikd, simonmar, austin
Reviewed By: erikd, simonmar
Subscribers: michalt, thomie
Differential Revision: https://phabricator.haskell.org/D2699
|
|
|
|
|
|
| |
It's entirely reasonable to set +RTS -qn without setting -N, because the
program might later call setNumCapabilities. If we disallow it, there's
no way to use -qn on programs that use setNumCapabilities.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Nursery chunks help reduce the cost of GC when capabilities are unevenly
loaded, by ensuring that we use more of the available nursery.
The rationale for enabling this at -A16m is that any negative effects
due to loss of cache locality are less likely to be an issue at -A16m
and above. It's a conservative guess. If we had a lot of benchmark
data we could probably do better.
Results for nofib/parallel at -N4 -A32m with and without -n4m:
```
------------------------------------------------------------------------
Program Size Allocs Runtime Elapsed TotalMem
------------------------------------------------------------------------
blackscholes 0.0% -9.5% -9.0% -15.0% -2.2%
coins 0.0% -4.7% -3.6% -0.6% -13.6%
mandel 0.0% -0.3% +7.7% +13.1% +0.1%
matmult 0.0% +1.5% +10.0% +7.7% +0.1%
nbody 0.0% -4.1% -2.9% 0.085 0.0%
parfib 0.0% -1.4% +1.0% +1.5% +0.2%
partree 0.0% -0.3% +0.8% +2.9% -0.8%
prsa 0.0% -0.5% -2.1% -7.6% 0.0%
queens 0.0% -3.2% -1.4% +2.2% +1.3%
ray 0.0% -5.6% -14.5% -7.6% +0.8%
sumeuler 0.0% -0.4% +2.4% +1.1% 0.0%
------------------------------------------------------------------------
Min 0.0% -9.5% -14.5% -15.0% -13.6%
Max 0.0% +1.5% +10.0% +13.1% +1.3%
Geometric Mean +0.0% -2.6% -1.3% -0.5% -1.4%
```
Not conclusive, but slightly better. This matters a lot more when you
have more cores.
Test Plan: validate, nofib/paralel
Reviewers: niteria, ezyang, nh2, trofi, austin, erikd, bgamari
Reviewed By: bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2581
GHC Trac Issues: #9221
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes the leak with `setProgArgv`. The problem was
that `setProgArgv` would not free the objects pointed
to by `prog_argc`, `prog_argv` when the globals were
changed resulting in a leak.
The only strictly necessary change is in `rts/RtsFlags.c`, but
the code in `System.Environment` was a bit confusing and not
exception safe, so I refactored it.
Test Plan: ./validate
Reviewers: simonmar, ezyang, austin, hvr, bgamari, erikd
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2576
GHC Trac Issues: #12664
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
We stumbled upon a case where an external library (OpenCL) does not work
if a specific address (0x200000000) is taken.
It so happens that `osReserveHeapMemory` starts trying to mmap at 0x200000000:
```
void *hint = (void*)((W_)8 * (1 << 30) + attempt * BLOCK_SIZE);
at = osTryReserveHeapMemory(*len, hint);
```
This makes it impossible to use Haskell programs compiled with GHC 8
with C functions that use OpenCL.
See this example ​https://github.com/chpatrick/oclwtf for a repro.
This patch allows the user to work around this kind of behavior outside
our control by letting the user override the starting address through an
RTS command line flag.
Reviewers: bgamari, Phyx, simonmar, erikd, austin
Reviewed By: Phyx, simonmar
Subscribers: rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D2513
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is long overdue.
Perhaps 1MB is a little on the skinny size, but this is based on
* A lot of commodity dual-core desktop processors have 3MB L3 cache
* We're traditionally quite frugal with memory by default
Test Plan: validate
Reviewers: erikd, bgamari, hvr, austin, rwbarton, ezyang
Reviewed By: ezyang
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2496
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Parallel GC does not scan large allocation area (-A)
effectively as it does not do work stealing from nursery
by default.
That leads to large imbalance when only one of threads
overflows allocation area: most of GC threads finish
quickly (as there is not much to collect) and sit idle
waiting while single GC thread finishes scan of single
allocation area for that thread.
The patch enables work stealing for (equivalent of -qb0)
allocation area of -A32M or higher.
Tested on a highlighting-kate package from Trac #9221
On 8-core machine the difference is around 5% faster
of wall-clock time. On 24-core VM the speedup is 20%.
Signed-off-by: Sergei Trofimovich <siarheit@google.com>
Test Plan: measured wall time and GC parallelism on highlighting-kate build
Reviewers: austin, bgamari, erikd, simonmar
Reviewed By: bgamari, simonmar
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2483
GHC Trac Issues: #9221
|
|
|
|
|
| |
- Move the numaMap and nNumaNodes out of RtsFlags to Capability.c
- Add a test to tests/rts
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary:
The aim here is to reduce the number of remote memory accesses on
systems with a NUMA memory architecture, typically multi-socket servers.
Linux provides a NUMA API for doing two things:
* Allocating memory local to a particular node
* Binding a thread to a particular node
When given the +RTS --numa flag, the runtime will
* Determine the number of NUMA nodes (N) by querying the OS
* Assign capabilities to nodes, so cap C is on node C%N
* Bind worker threads on a capability to the correct node
* Keep a separate free lists in the block layer for each node
* Allocate the nursery for a capability from node-local memory
* Allocate blocks in the GC from node-local memory
For example, using nofib/parallel/queens on a 24-core 2-socket machine:
```
$ ./Main 15 +RTS -N24 -s -A64m
Total time 173.960s ( 7.467s elapsed)
$ ./Main 15 +RTS -N24 -s -A64m --numa
Total time 150.836s ( 6.423s elapsed)
```
The biggest win here is expected to be allocating from node-local
memory, so that means programs using a large -A value (as here).
According to perf, on this program the number of remote memory accesses
were reduced by more than 50% by using `--numa`.
Test Plan:
* validate
* There's a new flag --debug-numa=<n> that pretends to do NUMA without
actually making the OS calls, which is useful for testing the code
on non-NUMA systems.
* TODO: I need to add some unit tests
Reviewers: erikd, austin, rwbarton, ezyang, bgamari, hvr, niteria
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2199
|