summaryrefslogtreecommitdiff
path: root/rts/LinkerInternals.h
Commit message (Collapse)AuthorAgeFilesLines
* linker: store entire link map and use it.Tamar Christina2018-12-041-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This fixes a corner case in which we have seen the symbol multiple times in different static libraries, but due to a depencency we end up loading the symbol from a library other than the first one. Previously the runtime linker would only track symbols from the first library and did not store the full link map. In this case it was unable to find the address for the symbols in the second library during delay loading. This change stores the address of all symbols seen so a full link map is generated, such that when we make a different decision later than what was expected we're able to still correctly load the library. Test Plan: ./validate, new testcase T15894 Reviewers: angerman, bgamari, erikd, simonmar Reviewed By: bgamari Subscribers: rwbarton, carter GHC Trac Issues: #15894 Differential Revision: https://phabricator.haskell.org/D5353
* Revert "Allocate bss section within proper range of other sections"Ben Gamari2018-10-301-4/+0
| | | | | | | | This reverts commit e019ec94f12268dd92ea5d5204e9e57e7ebf10ca. This sadly breaks the external interpreter on i386. For instance, see https://circleci.com/gh/ghc/ghc/10925.
* Allocate bss section within proper range of other sectionsZejun Wu2018-10-151-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allocate bss section within proper range of other sections: * when `+RTS -xp` is passed, allocate it contiguously as we did for jump islands * when we mmap the code to lower 2Gb, we should allocate bss section there too This depends on {D5195} Test Plan: 1. `./validate` 2. with ``` DYNAMIC_GHC_PROGRAMS = NO DYNAMIC_BY_DEFAULT = NO ``` `TEST="T15729" make test` passed in both linux and macos. 3. Also test in a use case where we used to encouter error like: ``` ghc-iserv-prof: R_X86_64_PC32 relocation out of range: (noname) = b90282ba ``` and now, everything works fine. Reviewers: simonmar, bgamari, angerman, erikd Reviewed By: simonmar Subscribers: rwbarton, carter GHC Trac Issues: #15729 Differential Revision: https://phabricator.haskell.org/D5219
* Updated PE linker, section alignment and cleanup.Tamar Christina2018-09-171-25/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch is to address a couple of short comings of the PE linker. The first thing it does is properly honor section alignments, so SSE code will work reliably. While doing this I've also changed how it reads and stores ObjectFile information. Previously the entire object file was read in and treated as one blob, including headers, symbol tables etc. Now the ObjectFile is read in but stored in chunks, tables go into a temporary info struct and code/data into a new private heap. This allows me to free all meta data once we're done relocating. Which means we can reclaim this memory. As I've mentioned above I've also moved from using VirtualAlloc to HeapAlloc. The reason is VirtualAlloc is meant to be used for more low level memory allocation, it's very fast because it can only allocate whole blocks, (64k) by default, and the memory must be paged (4k) aligned. So when you ask for e.g. 30k of memory, you're given a whole block where 34k will be wasted memory. Nothing else can ever access that untill you free the 30k. One downside of HeapAlloc is that you're not in control of how the heap grows, and heap memory is always committed. So it's harder to tell how much we're actually using now. Another big upside of splitting off the ObjectCode tables to info structs is that I can adjust them, so that later addressings can just use array subscripts to index into them. This simplifies the code a lot and a lot of complicated casts and indexing can be removed. Leaving less and more simple code. This patch doesn't fix the memprotection but it doesn't regress it either. It does however make the next changes smaller and fixes the alignments. Test Plan: ./validate , new test T13617 Reviewers: bgamari, erikd, simonmar, hvr, angerman Reviewed By: angerman Subscribers: nickkuk, carter, RyanGlScott, rwbarton, thomie GHC Trac Issues: #13617 Differential Revision: https://phabricator.haskell.org/D3915
* Add stack traces on crashes on WindowsTamar Christina2017-10-221-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This patch adds the ability to generate stack traces on crashes for Windows. When running in the interpreter this attempts to use symbol information from the interpreter and information we know about the loaded object files to resolve addresses to symbols. When running compiled it doesn't have this information and then defaults to using symbol information from PDB files. Which for now means only files compiled with ICC or MSVC will show traces compiled. But I have a future patch that may address this shortcoming. Also since I don't know how to walk a pure haskell stack, I can for now only show the last entry. I'm hoping to figure out how Apply.cmm works to be able to walk the stalk and give more entries for pure haskell code. In GHCi ``` $ echo main | inplace/bin/ghc-stage2.exe --interactive ./testsuite/tests/rts/derefnull.hs GHCi, version 8.3.20170830: http://www.haskell.org/ghc/ :? for help Ok, 1 module loaded. Prelude Main> Access violation in generated code when reading 0x0 Attempting to reconstruct a stack trace... Frame Code address * 0x77cde10 0xc370229 E:\..\base\dist-install\build\HSbase-4.10.0.0.o+0x190031 (base_ForeignziStorable_zdfStorableInt4_info+0x3f) ``` and compiled ``` Access violation in generated code when reading 0x0 Attempting to reconstruct a stack trace... Frame Code address * 0xf0dbd0 0x40bb01 E:\..\rts\derefnull.run\derefnull.exe+0xbb01 ``` Test Plan: ./validate Reviewers: austin, hvr, bgamari, erikd, simonmar Reviewed By: bgamari Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3913
* [RTS] Add getObjectLoadStatusBartosz Nitka2017-09-181-9/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | This adds a function to the RTS linker API which lets the user check the status of dynamically linked objects. It was initially proposed by @afarmer in D2068. It's useful for testing the linker and also for detecting retention problems in production. It takes a path, because it's easier to use path as key instead of producing some stable handle. It returns an enum instead of bool, because I see no reason for destroying information. All the complexity is already out in the open, so there's nothing to save the users from. Test Plan: ./validate Reviewers: simonmar, Phyx, bgamari, austin, erikd Reviewed By: Phyx, bgamari Subscribers: rwbarton, afarmer, thomie Differential Revision: https://phabricator.haskell.org/D3963
* rts: Fix references to Note [BFD import library]Ben Gamari2017-08-251-1/+1
| | | | | | | | Reviewers: austin, erikd, simonmar Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3884
* [linker] fix armv7 & add aarch64Moritz Angermann2017-06-081-10/+1
| | | | | | | | | | | | | | | This adds Global Offset Table logic, as well as PLT like logic for armv7 and aarch64; which replaces the preexisting symbolExtras logic, by placing the PLT tables next to the separtely loaded sections. This is needed to ensure that the symbol stubs are in range. Reviewers: bgamari, austin, erikd, simonmar Reviewed By: bgamari Subscribers: Ericson2314, ryantrinkle, rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3448
* Better import library support for WindowsTamar Christina2017-06-021-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The import library support added for 7.10.3 was only a partial one. This support was predicated on using file extensions to determine whether or not a library was an import library. It also couldn't handle libraries with multiple dll pointers. This is a rewrite of that patch and fully integrating it into the normal archive parsing and loading routines. This solves a host of issues, among others allowing us to finally use `-lgcc_s`. This also fixes a problem with our previous implementation, where we just loaded the DLL and moved on. Doing this had the potential of using the wrong symbol at resolve time. Say a DLL already loaded (A.dll) has symbol a exported (dependency of another dll perhaps). We find an import library `B.lib` explicitly defining an export of `a`. we load `B.dll` but this gets put after `A.dll`, at resolve time we would use the value from `A` instead of `B` which is what we wanted. Test Plan: ./valide and make test TEST=13606 Reviewers: austin, bgamari, erikd, simonmar Reviewed By: bgamari Subscribers: rwbarton, RyanGlScott, thomie, #ghc_windows_task_force GHC Trac Issues: #13606, #12499, #12498 Differential Revision: https://phabricator.haskell.org/D3513
* Fix iossimulatorMoritz Angermann2017-05-111-0/+5
| | | | | | | | | | | | | | | | | The introduction of the aarch64 linker for iOS forgot that the ios simulator was still using the x86_64/mach-o linker, which requires the use of symbol extras. Until this is overhauled (see #13678), we should revert to the symbol extras logic for x86_64-apple-ios Reviewers: austin, bgamari, erikd, simonmar Reviewed By: simonmar Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3556
* Abandon typedefing the {Section,ObjectCode}FormatInfo structsGabor Greif2017-05-041-9/+22
| | | | | | | | | | | | | | | | | | | | | | Summary: This is a follow-up to @angerman 's refactoring for ELF that happened with e5e8646d3c6af82549b55fbee6764b087144a7ec My previous commit a6675a93efe7cae2f206508047a39e73ce4e92a5 corrected a typedef redefinition issue with GCC v4.4 (which is pervasive with RHEL 6). Now the problem has resurfaced. Instead of dancing after the different compiler's pipe, I decided to eliminate the typedefs altogether and refer to the struct namespace explicitly. Added a note to describe why typedefs are not applied on customisable structs. Reviewers: austin, bgamari, erikd, simonmar Subscribers: rwbarton, thomie, angerman Differential Revision: https://phabricator.haskell.org/D3527
* Enable new warning for fragile/incorrect CPP #if usageErik de Castro Lopo2017-04-281-1/+2
| | | | | | | | | | | | | | | | The C code in the RTS now gets built with `-Wundef` and the Haskell code (stages 1 and 2 only) with `-Wcpp-undef`. We now get warnings whereever `#if` is used on undefined identifiers. Test Plan: Validate on Linux and Windows Reviewers: austin, angerman, simonmar, bgamari, Phyx Reviewed By: bgamari Subscribers: thomie, snowleopard Differential Revision: https://phabricator.haskell.org/D3278
* [linker] Adds ElfTypesMoritz Angermann2017-04-231-8/+7
| | | | | | | | | | | | | | This diff introduces ElfTypes similar to provide the linker code with a richer data structure, similar to the approach taken for mach-o already. Reviewers: bgamari, austin, erikd, simonmar Reviewed By: simonmar Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3445
* cpp: Use #pragma once instead of #ifndef guardsBen Gamari2017-04-231-3/+1
| | | | | | | | | | | | | | This both says what we mean and silences a bunch of spurious CPP linting warnings. This pragma is supported by all CPP implementations which we support. Reviewers: austin, erikd, simonmar, hvr Reviewed By: simonmar Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3482
* Revert "Enable new warning for fragile/incorrect CPP #if usage"Ben Gamari2017-04-051-6/+5
| | | | | | | | This is causing too much platform dependent breakage at the moment. We will need a more rigorous testing strategy before this can be merged again. This reverts commit 7e340c2bbf4a56959bd1e95cdd1cfdb2b7e537c2.
* rts: Fix lingering #ifsBen Gamari2017-04-041-1/+1
| | | | These were missed in D3278.
* Enable new warning for fragile/incorrect CPP #if usageErik de Castro Lopo2017-04-051-5/+6
| | | | | | | | | | | | | | | | The C code in the RTS now gets built with `-Wundef` and the Haskell code (stages 1 and 2 only) with `-Wcpp-undef`. We now get warnings whereever `#if` is used on undefined identifiers. Test Plan: Validate on Linux and Windows Reviewers: austin, angerman, simonmar, bgamari, Phyx Reviewed By: bgamari Subscribers: thomie, snowleopard Differential Revision: https://phabricator.haskell.org/D3278
* Don't redefine typedef namesGabor Greif2017-03-271-4/+4
| | | | | | | | | | instead define the structs referred to by - SectionFormatInfo - ObjectCodeFormatInfo that were only forward-declared earlier. This fixes redefinition errors with gcc4.4
* rts linker: Introduce MachOTypesMoritz Angermann2017-03-261-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This diff introduces MachOTypes, to reduce the need to typing `struct` all the time. It also coaleces the 64 and non 64 structs. It also adds additional fiedls to the object code structure for macho, which makes working with macho object code much simpler and requires less passing around of variabls or address recomputation for the header, symbol table, etc... Furthermore this diff introduces a type for a linked list of stubs. I had to move the #ifdef from the bottom of the file up, to be able to extend the object code structure conditional on the use of the macho file format. This is just one of the pieces for the rts linker support for ios (aarch64-macho) --- The following diagram and legend tries to explain the dependencies a bit: ``` .- D3240 v D3255 <- D3252 <- D3251 <- This ^ '- D3238 ``` - In D3238 we started allowing preloading object code with mmap in iOS, where we can't have r+w+x. - In D3239 we introduced a richer extension of the object code data type to make working with mach-o files easier. - In D3240 we set the stage to allow loading archives (.a) on iOS - In D3251 we added init and deinit functions to populate and depopulate the enriched object code data structure for mach-o files. - In D3252 we refactored most of the MachO.c file to use the new types and data structure. - in D3255 we finally introduce the aarch64-mach-o linker. Reviewers: austin, erikd, simonmar, rwbarton, bgamari Subscribers: rwbarton, thomie, ryantrinkle Differential Revision: https://phabricator.haskell.org/D3239
* Define MAP_ANONYMOUS on systems that only provide MAP_ANONGracjan Polak2016-12-231-0/+10
| | | | | | | | | | | | Reviewers: simonmar, erikd, austin, bgamari Reviewed By: bgamari Subscribers: gracjan, rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D2881 GHC Trac Issues: #13005
* Don't barf() on failures in loadArchive()Ben Gamari2016-12-071-6/+1
| | | | | | | | | | | | | | | | | | | This patch replaces calls to barf() in loadArchive() with proper error handling. Test Plan: GHC CI Reviewers: rwbarton, erikd, hvr, austin, simonmar, bgamari Reviewed By: bgamari Subscribers: thomie Tags: #ghc Differential Revision: https://phabricator.haskell.org/D2652 GHC Trac Issues: #12388
* Use C99's boolBen Gamari2016-11-291-1/+1
| | | | | | | | | | | | Test Plan: Validate on lots of platforms Reviewers: erikd, simonmar, austin Reviewed By: erikd, simonmar Subscribers: michalt, thomie Differential Revision: https://phabricator.haskell.org/D2699
* Fix windows validateTamar Christina2016-11-171-1/+1
| | | | | | | | | | | | | | | | | Summary: mark myindex as inline only and hide it from profilers. Also prevent this function from being used any other way than just for inlining. Test Plan: ./validate Reviewers: austin, erikd, simonmar, bgamari Reviewed By: bgamari Subscribers: thomie, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D2715
* Don't use mmap symbols when !RTS_LINKER_USE_MMAPShea Levy2016-11-151-0/+10
| | | | | | | | | | | | | | | | | | | Some usages of symbols from sys/mman.h are guarded by RTS_LINKER_USE_MMAP by C conditionals, not CPP conditionals. Since those branches are dead anyway when !RTS_LINKER_USE_MMAP, we just stub out the relevant symbols rather than increasing CPP branching. Fixes #12839. Reviewers: simonmar, austin, bgamari, erikd Reviewed By: simonmar, bgamari, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2710 GHC Trac Issues: #12839
* rts: Fix references to UCharBen Gamari2016-11-131-4/+5
| | | | | | | | | | | | Test Plan: Validate on Windows. Reviewers: erikd, austin, simonmar Reviewed By: erikd Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D2700
* rts: Fix LoadArchive on OS XBen Gamari2016-11-111-0/+9
|
* rts/linker: Move loadArchive to new source fileBen Gamari2016-11-101-0/+32
| | | | | | | | | | | | | | Test Plan: Validate Reviewers: DemiMarie, austin, simonmar, erikd Reviewed By: DemiMarie Subscribers: Phyx, thomie, hvr Differential Revision: https://phabricator.haskell.org/D2642 GHC Trac Issues: #12388
* linker: Split ELF implementation into separate source fileBen Gamari2016-11-021-0/+12
| | | | | | | | | | | | Test Plan: Validate Reviewers: erikd, austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2650
* linker: Split PEi386 implementation into new source fileBen Gamari2016-11-021-115/+3
| | | | | | | | | | | | Test Plan: Validate Reviewers: erikd, austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2648
* Shuffle declarations into LinkerInternals.hBen Gamari2016-11-021-0/+32
| | | | | | | | | | | | | | Summary: These will be needed across source files shortly. Test Plan: Validate Reviewers: erikd, austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2647
* linker: Move mmapForLinker declaration into LinkerInternals.hBen Gamari2016-11-021-1/+5
| | | | | | | | | | | | Test Plan: Validate Reviewers: simonmar, erikd, austin Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2645
* linker: Shuffle configuration into LinkerInternals.hBen Gamari2016-11-021-0/+24
| | | | | | | | | | | | Test Plan: Validate Reviewers: erikd, austin, simonmar Reviewed By: simonmar Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2644
* Revert "rts/linker: Move loadArchive to new source file"Matthew Pickering2016-10-271-7/+0
| | | | This reverts commit 488a9ed3440fe882ae043ba7f44fed4e84e679ce.
* rts/linker: Move loadArchive to new source fileBen Gamari2016-10-261-0/+7
| | | | | | | | | | | | | | Test Plan: Validate Reviewers: erikd, simonmar, austin, DemiMarie Reviewed By: erikd, simonmar, DemiMarie Subscribers: hvr, thomie Differential Revision: https://phabricator.haskell.org/D2615 GHC Trac Issues: #12388
* Remove unused oc->isImportLib (#12230)Simon Marlow2016-07-011-5/+0
| | | | | | | | | | | | | | | | | | | | Summary: This field is never set, but it was being tested and used to decide whether to resolve an object or not. This caused non-deterministic crashes when using the RTS linker (see #12230). I suspect this is not the correct fix, but putting it up so that Phyx can tell us what the right fix should be. Test Plan: validate Reviewers: austin, Phyx, bgamari, erikd Subscribers: erikd, thomie, ezyang Differential Revision: https://phabricator.haskell.org/D2371 GHC Trac Issues: #12230
* Fix incorrect calculated relocations on Windows x86_64Tamar Christina2016-06-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: See #12031 for analysis, but essentially what happens is: To sum up the issue, the reason this seems to go wrong is because of how we initialize the `.bss` section for Windows in the runtime linker. The first issue is where we calculate the zero space for the section: ``` zspace = stgCallocBytes(1, bss_sz, "ocGetNames_PEi386(anonymous bss)"); sectab_i->PointerToRawData = ((UChar*)zspace) - ((UChar*)(oc->image)); ``` Where ``` UInt32 PointerToRawData; ``` This means we're stuffing a `64-bit` value into a `32-bit` one. Also `zspace` can be larger than `oc->image`. In which case it'll overflow and then get truncated in the cast. The address of a value in the `.bss` section is then calculated as: ``` addr = ((UChar*)(oc->image)) + (sectabent->PointerToRawData + symtab_i->Value); ``` If it does truncate then this calculation won't be correct (which is what is happening). We then later use the value of `addr` as the `S` (Symbol) value for the relocations ``` S = (size_t) lookupSymbol_( (char*)symbol ); ``` Now the majority of the relocations are `R_X86_64_PC32` etc. e.g. They are guaranteed to fit in a `32-bit` value. The `R_X86_64_64` introduced for these pseudo-relocations so they can use the full `48-bit` addressing space isn't as lucky. As for why it sometimes work has to do on whether the value is truncated or not. `PointerToRawData` can't be changed because it's size is fixed by the PE specification. Instead just like with the other platforms, we now use `section` on Windows as well. This gives us a `start` parameter of type `void*` which solves the issue. This refactors the code to use `section.start` and to fix the issues. Test Plan: ./validate and new test added T12031 Reviewers: RyanGlScott, erikd, bgamari, austin, simonmar Reviewed By: simonmar Subscribers: thomie, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D2316 GHC Trac Issues: #12031, #11317
* Refactored SymbolInfo to lower memory usage in RTSTamar Christina2016-06-031-16/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously as part of #11223 a new struct `SymbolInfo` was introduced to keep track it the weak symbol status of a symbol. This structure also kept a copy of the calculated address of the symbol which turns out was useful in ignoring non-weak zero-valued symbols. The information was kept in an array so it means for every symbol two extra bytes were kept even though the vast majority of symbols are non-weak and non-zero valued. This changes the array into a sparse map keeping this information only for the symbols that are weak or zero-valued. This allows for a reduction in the amount of information needed to be kept while giving up a small (negligable) hit in performance as this information now has to be looked up in hashmaps. Test Plan: ./validate on all platforms that use the runtime linker. For unix platforms please ensure `DYNAMIC_GHC_PROGRAMS=NO` is added to your validate file. Reviewers: simonmar, austin, erikd, bgamari Reviewed By: simonmar, bgamari Subscribers: thomie, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D2184 GHC Trac Issues: #11816
* Add Windows import library support to the Runtime LinkerTamar Christina2016-04-171-1/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: Import libraries are files ending in `.dll.a` and `.lib` depending on which compiler creates them (GCC, vs MSVC). Import Libraries are standard `archive` files that contain object files. These object files can have two different formats: 1) The normal COFF Object format for object files (contains all ascii data and very little program code, so do not try to execute.) 2) "short import" format which just contains a symbol name and the dll in which the symbol can be found. Import Libraries are useful for two things: 1) Allowing applications that don't support dynamic linking to link against the import lib (non-short format) which then makes calls into the DLL by loading it at runtime. 2) Allow linking of mutually recursive dlls. if `A.DLL` requires `B.DLL` and vice versa, import libs can be used to break the cycle as they can be created from the expected exports of the DLLs. A side effect of having these two capabilities is that Import libs are often used to hide specific versions of DLLs behind a non-versioned import lib. e.g. GCC_S.a (non-conventional import lib) will point to the correct `libGCC` DLL. With this support Windows Haskell files can now just link to `-lGCC_S` and not have to worry about what the actual name of libGCC is. Also third party libraries such as `icuuc` use import libs to forward to versioned DLLs. e.g. `icuuc.lib` points to `icuuc51.dll` etc. Test Plan: ./validate Two new tests added T11072gcc T11072msvc Two binary files have been added to the test folder because the "short" import library format doesn't seem to be creatable via `dlltool` and requires Microsoft's `lib.exe`. Reviewers: bgamari, RyanGlScott, erikd, goldfire, austin, hvr Reviewed By: RyanGlScott, erikd Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1696 GHC Trac Issues: #11072
* Linker: Fix signedness mismatchBen Gamari2016-04-151-1/+1
| | | | | | | | | | | | | | Test Plan: Validate on OS X Reviewers: erikd, austin, Phyx Reviewed By: austin, Phyx Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D2110 GHC Trac Issues: #11828
* Change runtime linker to perform lazy loading of symbols/sectionsTamar Christina2016-04-111-2/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Runtime Linker is currently eagerly loading all object files on all platforms which do not use the system linker for `GHCi`. The problem with this approach is that it requires all symbols to be found. Even those of functions never used/called. This makes the number of libraries required to link things like `mingwex` quite high. To work around this the `rts` was relying on a trick. It itself was compiled with `MingW64-w`'s `GCC`. So it was already linked against `mingwex`. As such, it re-exported the symbols from itself. While this worked it made it impossible to link against `mingwex` in user libraries. And with this means no `C99` code could ever run in `GHCi` on Windows without having the required symbols re-exported from the rts. Consequently this rules out a large number of packages on Windows. SDL2, HMatrix etc. After talking with @rwbarton I have taken the approach of loading entire object files when a symbol is needed instead of doing the dependency tracking on a per symbol basis. This is a lot less fragile and a lot less complicated to implement. The changes come down to the following steps: 1) modify the linker to and introduce a new state for ObjectCode: `Needed`. A Needed object is one that is required for the linking to succeed. The initial set consists of all Object files passed as arguments to the link. 2) Change `ObjectCode`'s to be indexed but not initialized or resolved. This means we know where we would load the symbols, but haven't actually done so. 3) Mark any `ObjectCode` belonging to `.o` passed as argument as required: ObjectState `NEEDED`. 4) During `Resolve` object calls, mark all `ObjectCode` containing the required symbols as `NEEDED` 5) During `lookupSymbol` lookups, (which is called from `linkExpr` and `linkDecl` in `GHCI.hs`) is the symbol is in a not-yet-loaded `ObjectCode` then load the `ObjectCode` on demand and return the address of the symbol. Otherwise produce an unresolved symbols error as expected. 6) On `unloadObj` we then change the state of the object and remove it's symbols from the `reqSymHash` table so it can be reloaded. This change affects all platforms and OSes which use the runtime linker. It seems there are no real perf tests for `GHCi`, but performance shouldn't be impacted much. We gain a lot of time not loading all `obj` files, and we lose some time in `lookupSymbol` when we're finding sections that have to be loaded. The actual finding itself is O(1) (Assuming the hashtnl is perfect) It also consumes slighly more memory as instead of storing just the address of a symbol I also store some other information, like if the symbol is weak or not. This change will break any packages relying on renamed POSIX functions that were re-named and re-exported by the rts. Any packages following the proper naming for functions as found on MSDN will work fine. Test Plan: ./validate on all platforms which use the Runtime linker. Reviewers: thomie, rwbarton, simonmar, erikd, bgamari, austin, hvr Reviewed By: erikd Subscribers: kgardas, gridaphobe, RyanGlScott, simonmar, rwbarton, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D1805 GHC Trac Issues: #11223
* Linker: #ifdef cleanupErik de Castro Lopo2015-11-041-5/+3
| | | | | | | | | | | | | Test Plan: - Run tests on x86_64/linux and powerpc/linux - Cross compile rts/Linker.c with the i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc Linux to Windows cross-compilers. Reviewers: bgamari, austin, hvr, Phyx Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D1429
* ELF/x86_64: map object file sections separately into the low 2GBSimon Marlow2015-10-151-9/+31
| | | | | | | | | | | | | | | | | | | | | | | On 64-bit ELF we need to link object files into the low 2GB due to the small memory model. Previously we would map the entire object file using MAP_32BIT, but the object file can consist of 75% or more symbols, which only need to be present during linking, so this is wasteful. In our particular application, we're already running out of space here. This patch changes the way we load object files on ELF platforms so that the object is first mapped above the 2GB boundary, parsed, and then the important sections are re-mapped into the low 2GB area. Test Plan: validate (also needs testing on OS X & Windows, preferably 32 & 64 bit) Reviewers: Phyx, trommler, bgamari, austin Subscribers: hsyl20, thomie, bgamari Differential Revision: https://phabricator.haskell.org/D975
* Allow the linker to run concurrently with the GCSimon Marlow2015-01-131-0/+1
|
* Revert "Revert "Make the linker API thread-safe""Simon Marlow2014-12-051-0/+4
| | | | | | | Also includes a fix for the segfaults on Windows caused by the original version of this patch. This reverts commit 4b51194df4090d984f02c12128e868077660fb8b.
* Revert "Make the linker API thread-safe"Simon Peyton Jones2014-12-021-4/+0
| | | | | | | | | | | | | | | | This reverts commit b5e8b3b162b3ff15ae6caf1afc659565365f54a8. I reverted it because one of these two patches 9e6e4796437a7fc23e83605a45db9b2663570123 Add purgeObj() b5e8b3b162b3ff15ae6caf1afc659565365f54a8 Make the linker API thread-safe causes a seg-fault on Windows. The seg-fault happens immediately the linker is invoked, in ghci or in Template Haskell. I believe that it is the "linker API thread-safe" commit that causes the seg-fault; it happens even if the "purgeObj" commit alone is reverted. But since the two patches mess with the same code, to revert the "linker API" patch I had revert both.
* Make the linker API thread-safeSimon Marlow2014-11-281-0/+4
| | | | | | We used to be able to rely on the client to use the API in a single-threaded way, but now that the GC calls into the linker to unload objects this isn't a safe assumption.
* Revert "rts: add Emacs 'Local Variables' to every .c file"Simon Marlow2014-09-291-8/+0
| | | | This reverts commit 39b5c1cbd8950755de400933cecca7b8deb4ffcd.
* rts: add Emacs 'Local Variables' to every .c fileAustin Seipp2014-07-281-0/+8
| | | | | | | | This will hopefully help ensure some basic consistency in the forward by overriding buffer variables. In particular, it sets the wrap length, the offset to 4, and turns off tabs. Signed-off-by: Austin Seipp <austin@well-typed.com>
* Fix linker_unload now that we are running constructors in the linker (#8291)Simon Marlow2013-09-231-0/+13
| | | | | | | See also #5435. Now we have to remember the the StablePtrs that get created by the module initializer so that we can free them again in unloadObj().
* Implement .init/.init_array support for ELF.Edward Z. Yang2013-09-131-0/+1
| | | | Signed-off-by: Edward Z. Yang <ezyang@mit.edu>
View Lorry Controller Status