summaryrefslogtreecommitdiff
path: root/rts
Commit message (Collapse)AuthorAgeFilesLines
* Set `USE_MMAP` at configure timeErik de Castro Lopo2016-05-171-35/+19
| | | | | | | | | | | | | | | Summary: The `USE_MMAP` macro is used in the run time linker and was being set with some really ugly CPP hackery. Setting in the configure script is much neater. Reviewers: rwbarton, hvr, austin, simonmar, bgamari Reviewed By: hvr, bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2223
* Fix comments about scavenging WEAK objectsTakano Akio2016-05-122-7/+5
| | | | | | | | | | | | | | | | | | This is a follow-up of D2189. If fixes some comments, deletes a section in the User's Guide about the bug, and updates .mailmap as suggested on the WorkinConventions wiki page. Test Plan: It compiles. Reviewers: austin, simonmar, bgamari Reviewed By: bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2202 GHC Trac Issues: #11108
* RtsUtils: Use `size_t` instead of `int` where appropriateErik de Castro Lopo2016-05-122-16/+12
| | | | | | | | | | | | | | | | | | | Functions like `stgMallocBytes` take a size parameter which was of type `int`, but is commonly used as `stgMallocBytes (sizeof (...))`. This is problematic because the `sizeof` operator returns `size_t` so that on 64 bit systems, in this common use case the `size_t` parameter would be truncated to 32 bits when passed to `stgMallocBytes` where it was cast back to `size_t`. Test Plan: Validate on Linux, OS X and Windows Reviewers: austin, hvr, bgamari, simonmar, hsyl20 Reviewed By: hvr, bgamari, simonmar, hsyl20 Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2201
* rts: Make function pointer parameters `const` where possibleErik de Castro Lopo2016-05-127-15/+15
| | | | | | | | | | | | | | | | If a function takes a pointer parameter and doesn't update what the pointer points to, we can add `const` to the parameter declaration to document that no updates occur. Test Plan: Validate on Linux, OS X and Windows Reviewers: austin, Phyx, bgamari, simonmar, hsyl20 Reviewed By: bgamari, simonmar, hsyl20 Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2200
* Use StgHalfWord instead of a CPP #ifTomas Carnecky2016-05-121-8/+3
| | | | | | | | | | Reviewers: simonmar, bgamari, austin Reviewed By: bgamari, austin Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2183
* Fix ASSERT failure and re-enable setnumcapabilities001Simon Marlow2016-05-111-2/+5
| | | | | | | | | | | | The assertion failure was fairly benign, I think, but this fixes it. I've been running the test repeatedly for the last 30 mins and it hasn't triggered. There are other problems exposed by this test (see #12038), but I've worked around those in the test itself for now. I also copied the relevant bits of the parallel library here so that we don't need parallel for the test to run.
* Handle promotion failures when scavenging a WEAK (#11108)Takano Akio2016-05-113-3/+42
| | | | | | | | | | | | | | | | | | | | | | Previously, we ignored promotion failures when evacuating fields of a WEAK object. When a failure happens, this resulted in an WEAK object pointing to another object in a younger generation, causing crashes. I used the test case from #11746 to check that the fix is working. However I haven't managed to produce a test case that quickly reproduces the issue. Test Plan: ./validate Reviewers: austin, bgamari, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2189 GHC Trac Issues: #11108
* rts: Fix C compiler warnings on WindowsErik de Castro Lopo2016-05-112-9/+13
| | | | | | | | | | | | | | | | Summary: Specifcally we want the MinGW compiler to use ISO print format specfifiers. Test Plan: Validate on Linux, OS X and Windows Reviewers: Phyx, austin, bgamari, simonmar Reviewed By: bgamari, simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2192
* RtsFlags: Make `mallocFailHook` const correctErik de Castro Lopo2016-05-112-2/+2
| | | | | | | | | | | | Test Plan: Validate Reviewers: hvr, austin, bgamari, simonmar Reviewed By: bgamari, simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2193
* Fix a crash in requestSync()Simon Marlow2016-05-101-9/+14
| | | | | | | | It was possible for a thread to read invalid memory after a conflict when multiple threads were synchronising. I haven't been successful in constructing a test case that triggers this, but we have some internal code that ran into it.
* Use stdint types for Stg{Word,Int}{8,16,32,64}Tomas Carnecky2016-05-103-5/+5
| | | | | | | | | | | | | | | | | | We can't define Stg{Int,Word} in terms of {,u}intptr_t because STG depends on them being the exact same size as void*, and {,u}intptr_t does not make that guarantee. Furthermore, we also need to define StgHalf{Int,Word}, so the preprocessor if needs to stay. But we can at least keep it in a single place instead of repeating it in various files. Also define STG_{INT,WORD}{8,16,32,64}_{MIN,MAX} and use it in HsFFI.h, further reducing the need for CPP in other files. Reviewers: austin, bgamari, simonmar, hvr, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2182
* rts/ProfHeap.c: Use `ssize_t` instead of `long`.Erik de Castro Lopo2016-05-081-21/+22
| | | | | | | | | | | | | | | On x64 Windows `sizeof long` is 4 which could easily overflow resulting in incorrect heap profiling results. This change does not affect either Linux or OS X where `sizeof long` == `sizeof ssize_t` regardless of machine word size. Test Plan: Validate on Linux and Windows Reviewers: hsyl20, bgamari, simonmar, austin Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2177
* rts: Replace `nat` with `uint32_t`Erik de Castro Lopo2016-05-0571-528/+537
| | | | | | | | | | | | The `nat` type was an alias for `unsigned int` with a comment saying it was at least 32 bits. We keep the typedef in case client code is using it but mark it as deprecated. Test Plan: Validated on Linux, OS X and Windows Reviewers: simonmar, austin, thomie, hvr, bgamari, hsyl20 Differential Revision: https://phabricator.haskell.org/D2166
* Don't STATIC_INLINE giveCapabilityToTaskSimon Marlow2016-05-041-1/+1
| | | | This causes errors with some versions of gcc (4.4.7 here).
* schedulePushWork: avoid unnecessary wakeupsSimon Marlow2016-05-041-7/+25
| | | | | | | | | | | | | | | | | | | | This function had some pathalogically bad behaviour: if we had 2 threads on the current capability and 23 other idle capabilities, we would * grab all 23 capabilities * migrate one Haskell thread to one of them * wake up a worker on *all* 23 other capabilities. This lead to a lot of unnecessary wakeups when using large -N values. Now, we * Count how many capabilities we need to wake up * Start from cap->no+1, so that we don't overload low-numbered capabilities * Only wake up capabilities that we migrated a thread to (unless we have sparks to steal) This results in a pretty dramatic improvement in our production system.
* Add +RTS -AL<size>Simon Marlow2016-05-042-15/+27
| | | | | | | | | | | | | | | +RTS -AL<size> controls the total size of large objects that can be allocated before a GC is triggered. Previously this was always just the value of -A, and the limit mainly existed to prevent runaway allocation in pathalogical programs that allocate a lot of large objects. However, since the limit is shared between all cores, on a large multicore the default becomes more restrictive, and can end up triggering GC well before it would normally have been. Arguably a better default would be A*N, but this is probably excessive. Adding a flag lets you choose, and I've left the default as it was. See docs for usage.
* Allow limiting the number of GC threads (+RTS -qn<n>)Simon Marlow2016-05-046-121/+271
| | | | | | | | | | | | | | | | | | This allows the GC to use fewer threads than the number of capabilities. At each GC, we choose some of the capabilities to be "idle", which means that the thread running on that capability (if any) will sleep for the duration of the GC, and the other threads will do its work. We choose capabilities that are already idle (if any) to be the idle capabilities. The idea is that this helps in the following situation: * We want to use a large -N value so as to make use of hyperthreaded cores * We use a large heap size, so GC is infrequent * But we don't want to use all -N threads in the GC, because that thrashes the memory too much. See docs for usage.
* Cleanups related to MAX_FREE_LISTSimon Marlow2016-05-021-19/+24
| | | | | | | | | | | | | | | | - Rename to the (more correct) NUM_FREE_LISTS - NUM_FREE_LISTS should be derived from the block and mblock sizes, not defined manually. It was actually too large by one, which caused a little bit of (benign) extra work in the form of a redundant loop iteration in some cases. - Add some ASSERTs for input preconditions to log_2() and log_2_ceil() - Fix some comments - Fix usage in allocLargeChunk, to account for the fact that log_2_ceil() can return NUM_FREE_LISTS.
* Revert "Revert "Use __builtin_clz() to implement log_1()""U-THEFACEBOOK\smarlow2016-05-021-11/+27
| | | | | | | This reverts commit 546f24e4f8a7c086b1e5afcdda624176610cbcf8. And adds a fix for Windows: we need to use __builtin_clzll() rather than __builtin_clzl(), because StgWord is unsigned long long on Windows.
* rts/itimer/pthread: Stop timer when ticker is stoppedBen Gamari2016-05-012-36/+74
| | | | | | | | | | | | | | | | | This reworks the pthread-based itimer implementation to disarm the timer when events aren't needed. Thanks to hsyl20 for the nice design. Test Plan: Validate Reviewers: hsyl20, simonmar, austin Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2131 GHC Trac Issues: #1623, #11965
* Kill Itimer.hBen Gamari2016-05-015-16/+0
| | | | It declared nothing.
* Catch errors from timerfd_settimeBen Gamari2016-05-011-1/+4
|
* rts: Split up Itimer.cBen Gamari2016-05-016-299/+407
| | | | | | | | | | | | | | | | | This shouldn't have any functional changes. It merely splits up what are essentially three distinct codepaths which are melding together with CPP. At the moment I merely #include the implementation to use with CPP although this really feels very yucky. Reviewers: erikd, austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2130
* rts: Close livelock window due to rapid ticker enable/disableBen Gamari2016-05-011-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes #11830, where the RTS would livelock if run with `-I0` due to a regression introduced by bbdc52f3a6e6a28e209fb8f65699121d4ef3a4e3. The reason for this is that the new codepath introduced a subtle race condition: 1. one thread could request that the ticker stop and would block until the ticker in fact stopped 2. meanwhile, another thread could sneak in and restart the ticker this was implemented in such a way where thread (1) would end up blocked forever. The solution here is to simply not block. The worst that will happen is that timer fires again, but is ignored since the ticker is stopped. Test Plan: Validate, try reproduction case in #11830. Need to find a nice testcase. Reviewers: simonmar, erikd, hsyl20, austin Reviewed By: erikd, hsyl20 Subscribers: erikd, thomie Differential Revision: https://phabricator.haskell.org/D2129 GHC Trac Issues: #11830
* RTS: delete BlockedOnGA* + dead codeThomas Miedema2016-04-291-4/+0
| | | | | | | | Some old stuff related to the PAR way. Reviewed by: austin, simonmar Differential Revision: https://phabricator.haskell.org/D2137
* Linker: Fix implicit function declaration warning on OS XErik de Castro Lopo2016-04-291-4/+1
| | | | | | | | | | | | Introduced in commit 177aec697b3. Test Plan: Validate on OSX and Linux. Reviewers: austin, bgamari, hvr Subscribers: goldfire, thomie Differential Revision: https://phabricator.haskell.org/D2140
* rts/LdvProfile.c: Fix NULL dereference on shutdownErik de Castro Lopo2016-04-291-0/+3
| | | | | | | | | | | | | | Test Plan: validate Reviewers: carter, austin, simonmar, bgamari Reviewed By: simonmar, bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2154 GHC Trac Issues: #11978
* Revert "Use __builtin_clz() to implement log_2()"Simon Peyton Jones2016-04-281-21/+11
| | | | This reverts commit 24864ba5587c1a0447beabae90529e8bb4fa117a.
* Just comments & reformattingSimon Marlow2016-04-261-35/+21
|
* Use __builtin_clz() to implement log_2()Simon Marlow2016-04-261-11/+21
| | | | A microoptimisation in the block allocator.
* RTS: Add setInCallCapability()Simon Marlow2016-04-263-14/+31
| | | | | | | | This allows an OS thread to specify which capability it should run on when it makes a call into Haskell. It is intended for a fairly specialised use case, when the client wants to have tighter control over the mapping between OS threads and Capabilities - perhaps 1:1 correspondence, for example.
* rts: Don't use strndupBen Gamari2016-04-183-1/+15
| | | | | | | | Reviewers: austin Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2125
* rts: Limit maximum backtrace depthBen Gamari2016-04-181-3/+13
| | | | | This prevents us from entering an infinite loop in the event of a hitting bad unwinding information.
* Linker: Clean up #if USE_MMAP usageErik de Castro Lopo2016-04-171-58/+52
| | | | | | | | | | | | | | | Test Plan: - Run tests on x86_64/linux, x86_64/darwin and powerpc/linux - Cross compile rts/Linker.c with the i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc Linux to Windows cross-compilers. Reviewers: hvr, austin, thomie, bgamari, simonmar, Phyx Reviewed By: Phyx Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1437
* Check CCS tree for pointers into shared object during checkUnloadAndrew Farmer2016-04-171-0/+31
| | | | | | | | | | | | | | | | | | | Prevent shared objects from being unloaded if cost centre stacks point at the object. This will prevent segfault in #11776, but also prevents objects from ever being unloaded when profiling. Pruning CCS tree will enable that in another diff. Test Plan: make TEST=linker_profiled, examine linker_profiled.run.stderr Reviewers: austin, simonmar, bgamari Reviewed By: simonmar, bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2069 GHC Trac Issues: #11776
* Add Windows import library support to the Runtime LinkerTamar Christina2016-04-172-137/+373
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Import libraries are files ending in `.dll.a` and `.lib` depending on which compiler creates them (GCC, vs MSVC). Import Libraries are standard `archive` files that contain object files. These object files can have two different formats: 1) The normal COFF Object format for object files (contains all ascii data and very little program code, so do not try to execute.) 2) "short import" format which just contains a symbol name and the dll in which the symbol can be found. Import Libraries are useful for two things: 1) Allowing applications that don't support dynamic linking to link against the import lib (non-short format) which then makes calls into the DLL by loading it at runtime. 2) Allow linking of mutually recursive dlls. if `A.DLL` requires `B.DLL` and vice versa, import libs can be used to break the cycle as they can be created from the expected exports of the DLLs. A side effect of having these two capabilities is that Import libs are often used to hide specific versions of DLLs behind a non-versioned import lib. e.g. GCC_S.a (non-conventional import lib) will point to the correct `libGCC` DLL. With this support Windows Haskell files can now just link to `-lGCC_S` and not have to worry about what the actual name of libGCC is. Also third party libraries such as `icuuc` use import libs to forward to versioned DLLs. e.g. `icuuc.lib` points to `icuuc51.dll` etc. Test Plan: ./validate Two new tests added T11072gcc T11072msvc Two binary files have been added to the test folder because the "short" import library format doesn't seem to be creatable via `dlltool` and requires Microsoft's `lib.exe`. Reviewers: bgamari, RyanGlScott, erikd, goldfire, austin, hvr Reviewed By: RyanGlScott, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1696 GHC Trac Issues: #11072
* Rework CC/CC_STAGE0 handling in `configure.ac`Herbert Valerio Riedel2016-04-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than using the non-standard/idiomatic `--with-{gcc,clang}=...` scheme use the `CC=...` style scheme. The basic idea is to have Autoconf's CC/CFLAG/CPPFLAG apply to stage{1,2,3}, while having a separate _STAGE0 set of env-vars denote the bootstrap-toolchain flags/programs. This should be simpler, less confusing, and somewhat more in line with Autoconf's idioms (allowing us to reuse more of Autoconf rather than (re)inventing our own confusing non-standard m4 macros to do stuff that Autoconf could almost do already for us) Morever, expose CC_STAGE0 as a so-called "precious" variable. So now we can better control which bootstrapping gcc is used (by default the one used by the stage0 ghc, unless CC_STAGE0 is overriden) ``` Some influential environment variables: CC_STAGE0 C compiler command (bootstrap) CC C compiler command CFLAGS C compiler flags ... Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. ``` Test Plan: I've tested that cross-compiling with `--target=powerpc-linux-gnu` still works, and tried a few variants of settting `CC=` and `CC_STAGE0=`; `./validate` passed as well Reviewers: erikd, austin, bgamari, simonmar Reviewed By: simonmar Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D2078
* rts/RetainerProfile: Remove unused localBen Gamari2016-04-161-2/+0
| | | | Reported in #11777.
* Linker: Fix signedness mismatchBen Gamari2016-04-152-3/+3
| | | | | | | | | | | | | | Test Plan: Validate on OS X Reviewers: erikd, austin, Phyx Reviewed By: austin, Phyx Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D2110 GHC Trac Issues: #11828
* Allocate blocks in the GC in batchesSimon Marlow2016-04-123-30/+29
| | | | | | | | | | | | | | | | | Avoids contention for the block allocator lock in the GC; this can be seen in the gc_alloc_block_sync counter emitted by +RTS -s. I experimented with this a while ago, and there was already commented-out code for it in GCUtils.c, but I've now improved it so that it doesn't result in significantly worse memory usage. * The old method of putting spare blocks on ws->part_list was wasteful, the spare blocks are now shared between all generations and retained between GCs. * repeated allocGroup() results in fragmentation, so I switched to using allocLargeChunk() instead which is fragmentation-friendly; we already use it for the same reason in nursery allocation.
* Cache the size of part_list/scavd_list (#11783)Simon Marlow2016-04-124-9/+20
| | | | | | | | | After a parallel GC, it is possible to have a long list of blocks in ws->part_list, if we did a lot of work stealing but didn't fill up the blocks we stole. These blocks persist until the next load-balanced GC, which might be a long time, and during every GC we were traversing this list to find its size. The fix is to maintain the size all the time, so we don't have to compute it.
* Small simplification (#11777)Simon Marlow2016-04-121-5/+1
| | | | | DEAD_WEAK used to have a different layout, see d61c623ed6b2d352474a7497a65015dbf6a72e12
* Add linker notesTamar Christina2016-04-111-0/+13
| | | | | | | | | | | | Summary: Add linker notes following #11223 and D1805 Reviewers: austin, bgamari, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2102 GHC Trac Issues: #11223
* Change runtime linker to perform lazy loading of symbols/sectionsTamar Christina2016-04-113-411/+376
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Runtime Linker is currently eagerly loading all object files on all platforms which do not use the system linker for `GHCi`. The problem with this approach is that it requires all symbols to be found. Even those of functions never used/called. This makes the number of libraries required to link things like `mingwex` quite high. To work around this the `rts` was relying on a trick. It itself was compiled with `MingW64-w`'s `GCC`. So it was already linked against `mingwex`. As such, it re-exported the symbols from itself. While this worked it made it impossible to link against `mingwex` in user libraries. And with this means no `C99` code could ever run in `GHCi` on Windows without having the required symbols re-exported from the rts. Consequently this rules out a large number of packages on Windows. SDL2, HMatrix etc. After talking with @rwbarton I have taken the approach of loading entire object files when a symbol is needed instead of doing the dependency tracking on a per symbol basis. This is a lot less fragile and a lot less complicated to implement. The changes come down to the following steps: 1) modify the linker to and introduce a new state for ObjectCode: `Needed`. A Needed object is one that is required for the linking to succeed. The initial set consists of all Object files passed as arguments to the link. 2) Change `ObjectCode`'s to be indexed but not initialized or resolved. This means we know where we would load the symbols, but haven't actually done so. 3) Mark any `ObjectCode` belonging to `.o` passed as argument as required: ObjectState `NEEDED`. 4) During `Resolve` object calls, mark all `ObjectCode` containing the required symbols as `NEEDED` 5) During `lookupSymbol` lookups, (which is called from `linkExpr` and `linkDecl` in `GHCI.hs`) is the symbol is in a not-yet-loaded `ObjectCode` then load the `ObjectCode` on demand and return the address of the symbol. Otherwise produce an unresolved symbols error as expected. 6) On `unloadObj` we then change the state of the object and remove it's symbols from the `reqSymHash` table so it can be reloaded. This change affects all platforms and OSes which use the runtime linker. It seems there are no real perf tests for `GHCi`, but performance shouldn't be impacted much. We gain a lot of time not loading all `obj` files, and we lose some time in `lookupSymbol` when we're finding sections that have to be loaded. The actual finding itself is O(1) (Assuming the hashtnl is perfect) It also consumes slighly more memory as instead of storing just the address of a symbol I also store some other information, like if the symbol is weak or not. This change will break any packages relying on renamed POSIX functions that were re-named and re-exported by the rts. Any packages following the proper naming for functions as found on MSDN will work fine. Test Plan: ./validate on all platforms which use the Runtime linker. Reviewers: thomie, rwbarton, simonmar, erikd, bgamari, austin, hvr Reviewed By: erikd Subscribers: kgardas, gridaphobe, RyanGlScott, simonmar, rwbarton, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D1805 GHC Trac Issues: #11223
* RtsFlags: Un-constify temporary bufferBen Gamari2016-04-101-1/+1
| | | | | Otherwise we get a const-ness mismatch when we free the buffer, which for some reason gcc 5.3 didn't notice.
* Remove spurious STG_UNUSED annotationTomas Carnecky2016-04-101-5/+1
| | | | | | | | | | Reviewers: austin, bgamari Reviewed By: bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2096
* Refactor comments about shutdownSimon Marlow2016-04-102-18/+21
|
* rts: Fix parsing of profiler selectorsBen Gamari2016-04-071-9/+11
| | | | | | | | | | | | | | | 69822f0c5b67161b4d7558081bc94f6f3a7c5dbb broke this as it held on to a reference into the `arg` string, which is later freed. Humbug. Test Plan: Try using filtering Reviewers: austin Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2090 GHC Trac Issues: #11810
* runtime: replace hw.ncpu with hw.logicalcpu for Mac OS XEugene Akentyev2016-04-071-2/+8
| | | | | | | | Reviewed By: erikd, austin Differential Revision: https://phabricator.haskell.org/D2082 GHC Trac Issues: #8594
* rts: Make StablePtr derefs thread-safe (#10296)Jason Eisenberg2016-04-041-4/+72
| | | | | | | | | | | | | | | Stable pointers can now be safely dereferenced while the stable pointer table is simultaneously being enlarged. Test Plan: ./validate Reviewers: ezyang, austin, bgamari, simonmar Subscribers: carter, thomie Differential Revision: https://phabricator.haskell.org/D2031 GHC Trac Issues: #10296
View Lorry Controller Status