| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
Use an (Raw)PkgQual datatype instead of `Maybe FastString` to represent
package imports. Factorize the code that renames RawPkgQual into PkgQual
in function `rnPkgQual`. Renaming consists in checking if the FastString
is the magic "this" keyword, the home-unit unit-id or something else.
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At the moment if `-dynamic-too` fails then we rerun the whole pipeline
as if we were just in `-dynamic` mode. I argue this is a misfeature and
we should remove the so-called `DT_Failed` mode.
In what situations do we fall back to `DT_Failed`?
1. If the `dyn_hi` file corresponding to a `hi` file is missing completely.
2. If the interface hash of `dyn_hi` doesn't match the interface hash of `hi`.
What happens in `DT_Failed` mode?
* The whole compiler pipeline is rerun as if the user had just passed `-dynamic`.
* Therefore `dyn_hi/dyn_o` files are used which don't agree with the
`hi/o` files. (As evidenced by `dynamicToo001` test).
* This is very confusing as now a single compiler invocation has
produced further `hi`/`dyn_hi` files which are different to each
other.
Why should we remove it?
* In `--make` mode, which is predominately used `DT_Failed` does not
work (#19782), there can't be users relying on this functionality.
* In `-c` mode, the recovery doesn't fix the root issue, which is the
`dyn_hi` and `hi` files are mismatched. We should instead produce an
error and pass responsibility to the build system using `-c` to ensure
that the prerequisites for `-dynamic-too` (dyn_hi/hi) files are there
before we start compiling.
* It is a misfeature to support use cases like `dynamicToo001` which
allow you to mix different versions of dynamic/non-dynamic interface
files. It's more likely to lead to subtle bugs in your resulting
programs where out-dated build products are used rather than a
deliberate choice.
* In practice, people are usually compiling with `-dynamic-too` rather
than separately with `-dynamic` and `-static`, so the build products
always match and `DT_Failed` is only entered due to compiler bugs (see
!6583)
What should we do instead?
* In `--make` mode, for home packages check during recompilation
checking that `dyn_hi` and `hi` are both present and agree, recompile
the modules if they do not.
* For package modules, when loading the interface check that `dyn_hi`
and `hi` are there and that they agree but fail with an
error message if they are not.
* In `--oneshot` mode, fail with an error message if the right files
aren't already there.
Closes #19782 #20446 #9176 #13616
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ModLocation is the data type which tells you the locations of all the
build products which can affect recompilation. It is now computed in one
place and not modified through the pipeline. Important locations will
now just consult ModLocation rather than construct the dynamic object
path incorrectly.
* Add paths for dynamic object and dynamic interface files to
ModLocation.
* Always use the paths from mod location when looking for where to find
any interface or object file.
* Always use the paths in a ModLocation when deciding where to write an
interface and object file.
* Remove `dynamicOutputFile` and `dynamicOutputHi` functions which
*calculated* (incorrectly) the location of `dyn_o` and `dyn_hi` files.
* Don't set `outputFile_` and so-on in `enableCodeGenWhen`, `-o` and
hence `outputFile_` should not affect the location of object files in
`--make` mode. It is now sufficient to just update the ModLocation with
the temporary paths.
* In `hscGenBackendPipeline` don't recompute the `ModLocation` to
account for `-dynamic-too`, the paths are now accurate from the start
of the run.
* Rename `getLocation` to `mkOneShotModLocation`, as that's the only
place it's used. Increase the locality of the definition by moving it
close to the use-site.
* Load the dynamic interface from ml_dyn_hi_file rather than attempting
to reconstruct it in load_dynamic_too.
* Add a variety of tests to check how -o -dyno etc interact with each
other.
Some other clean-ups
* DeIOify mkHomeModLocation and friends, they are all pure functions.
* Move FinderOpts into GHC.Driver.Config.Finder, next to initFinderOpts.
* Be more precise about whether we mean outputFile or outputFile_: there
were many places where outputFile was used but the result shouldn't have
been affected by `-dyno` (for example the filename of the resulting
executable). In these places dynamicNow would never be set but it's
still more precise to not allow for this possibility.
* Typo fixes suffices -> suffixes in the appropiate places.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If you don't promptly force this field then it ends up retaining a lot
of data structures related to parsing.
For example, the following retaining chain can be observed when using
GHCi.
```
PState 0x4289365ca0 0x4289385d68 0x4289385db0 0x7f81b37a7838 0x7f81b3832fd8 0x4289365cc8 0x4289365cd8 0x4289365cf0 0x4289365cd8 0x4289365d08 0x4289385e48 0x7f81b4e4c290 0x7f818f63f440 0x7f818f63f440 0x7f81925ccd18 0x7f81b4e41230 0x7f818f63f440 0x7f81925ccd18 0x7f818f63f4a8 0x7f81b3832fd8 0x7f81b3832fd8 0x4289365d20 0x7f81b38233b8 0 19 <PState:GHC.Parser.Lexer:_build-ipe/stage1/compiler/build/GHC/Parser/Lexer.hs:3779:46>
_thunk( ) 0x4289384230 0x4289384160 <([LEpaComment], [LEpaComment]):GHC.Parser.Lexer:>
_thunk( ) 0x4289383250 <EpAnnComments:GHC.Parser.Lexer:compiler/GHC/Parser/Lexer.x:2306:19-40>
_thunk( ) 0x4289399850 0x7f818f63f440 0x4289399868 <SrcSpanAnnA:GHC.Parser:_build-ipe/stage1/compiler/build/GHC/Parser.hs:12527:13-30>
L 0x4289397600 0x42893975a8 <GenLocated:GHC.Parser:_build-ipe/stage1/compiler/build/GHC/Parser.hs:12527:32>
0x4289c4e8c8 : 0x4289c4e8b0 <[]:GHC.Parser.Header:compiler/GHC/Parser/Header.hs:104:36-54>
(0x4289c4da70,0x7f818f63f440) <(,):GHC.Parser.Header:compiler/GHC/Parser/Header.hs:104:36-54>
_thunk( ) 0x4289c4d030 <Bool:GHC.Parser.Header:compiler/GHC/Parser/Header.hs:(112,22)-(115,27)>
ExtendedModSummary 0x422e9c8998 0x7f81b617be78 0x422e9c89b0 0x4289c4c0c0 0x7f81925ccd18 0x7f81925ccd18 0x7f81925ccd18 0x7f81925ccd18 0x7f818f63f440 0x4289c4c0d8 0x4289c4c0f0 0x7f81925ccd18 0x422e9c8a20 0x4289c4c108 0x4289c4c730 0x7f818f63f440 <ExtendedModSummary:GHC.Driver.Make:compiler/GHC/Driver/Make.hs:2041:30-38>
ModuleNode 0x4289c4b850 <ModuleGraphNode:GHC.Unit.Module.Graph:compiler/GHC/Unit/Module/Graph.hs:139:14-36>
0x4289c4b590 : 0x4289c4b578 <[]:GHC.Unit.Module.Graph:compiler/GHC/Unit/Module/Graph.hs:139:31-36>
ModuleGraph 0x4289c4b2f8 0x4289c4b310 0x4289c4b340 0x7f818f63f4a0 <ModuleGraph:GHC.Driver.Make:compiler/GHC/Driver/Make.hs:(242,19)-(244,40)>
HscEnv 0x4289d9a4a8 0x4289d9aad0 0x4289d9aae8 0x4217062a88 0x4217060b38 0x4217060b58 0x4217060b68 0x7f81b38a7ce0 0x4217060b78 0x7f818f63f440 0x7f818f63f440 0x4217062af8 0x4289d9ab10 0x7f81b3907b60 0x4217060c00 114 <HscEnv:GHC.Runtime.Eval:compiler/GHC/Runtime/Eval.hs:790:31-44>
```
|
| |
|
|
|
|
| |
* moved deps related code into GHC.Unit.Module.Deps
* refactored Deps module to not export Dependencies constructor to help
maintaining invariants
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch specifies and simplifies the module cycle compilation
in upsweep. How things work are described in the Note [Upsweep]
Note [Upsweep]
~~~~~~~~~~~~~~
Upsweep takes a 'ModuleGraph' as input, computes a build plan and then executes
the plan in order to compile the project.
The first step is computing the build plan from a 'ModuleGraph'.
The output of this step is a `[BuildPlan]`, which is a topologically sorted plan for
how to build all the modules.
```
data BuildPlan = SingleModule ModuleGraphNode -- A simple, single module all alone but *might* have an hs-boot file which isn't part of a cycle
| ResolvedCycle [ModuleGraphNode] -- A resolved cycle, linearised by hs-boot files
| UnresolvedCycle [ModuleGraphNode] -- An actual cycle, which wasn't resolved by hs-boot files
```
The plan is computed in two steps:
Step 1: Topologically sort the module graph without hs-boot files. This returns a [SCC ModuleGraphNode] which contains
cycles.
Step 2: For each cycle, topologically sort the modules in the cycle *with* the relevant hs-boot files. This should
result in an acyclic build plan if the hs-boot files are sufficient to resolve the cycle.
The `[BuildPlan]` is then interpreted by the `interpretBuildPlan` function.
* `SingleModule nodes` are compiled normally by either the upsweep_inst or upsweep_mod functions.
* `ResolvedCycles` need to compiled "together" so that the information which ends up in
the interface files at the end is accurate (and doesn't contain temporary information from
the hs-boot files.)
- During the initial compilation, a `KnotVars` is created which stores an IORef TypeEnv for
each module of the loop. These IORefs are gradually updated as the loop completes and provide
the required laziness to typecheck the module loop.
- At the end of typechecking, all the interface files are typechecked again in
the retypecheck loop. This time, the knot-tying is done by the normal laziness
based tying, so the environment is run without the KnotVars.
* UnresolvedCycles are indicative of a proper cycle, unresolved by hs-boot files
and are reported as an error to the user.
The main trickiness of `interpretBuildPlan` is deciding which version of a dependency
is visible from each module. For modules which are not in a cycle, there is just
one version of a module, so that is always used. For modules in a cycle, there are two versions of
'HomeModInfo'.
1. Internal to loop: The version created whilst compiling the loop by upsweep_mod.
2. External to loop: The knot-tied version created by typecheckLoop.
Whilst compiling a module inside the loop, we need to use the (1). For a module which
is outside of the loop which depends on something from in the loop, the (2) version
is used.
As the plan is interpreted, which version of a HomeModInfo is visible is updated
by updating a map held in a state monad. So after a loop has finished being compiled,
the visible module is the one created by typecheckLoop and the internal version is not
used again.
This plan also ensures the most important invariant to do with module loops:
> If you depend on anything within a module loop, before you can use the dependency,
the whole loop has to finish compiling.
The end result of `interpretBuildPlan` is a `[MakeAction]`, which are pairs
of `IO a` actions and a `MVar (Maybe a)`, somewhere to put the result of running
the action. This list is topologically sorted, so can be run in order to compute
the whole graph.
As well as this `interpretBuildPlan` also outputs an `IO [Maybe (Maybe HomeModInfo)]` which
can be queried at the end to get the result of all modules at the end, with their proper
visibility. For example, if any module in a loop fails then all modules in that loop will
report as failed because the visible node at the end will be the result of retypechecking
those modules together.
Along the way we also fix a number of other bugs in the driver:
* Unify upsweep and parUpsweep.
* Fix #19937 (static points, ghci and -j)
* Adds lots of module loop tests due to Divam.
Also related to #20030
Co-authored-by: Divam Narula <dfordivam@gmail.com>
-------------------------
Metric Decrease:
T10370
-------------------------
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The GHC.Prim module is quite special as there is no interface file,
therefore it doesn't appear in ms_textual_imports, but the ghc-prim
package does appear in the direct package dependencies. This confused
the recompilation checking which couldn't find any modules from ghc-prim
and concluded that the package was no longer a dependency.
The fix is to keep track of whether GHC.Prim is imported separately in
the relevant places.
Fixes #20084
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch comprises of four different but closely related ideas. The
net result is fixing a large number of open issues with the driver
whilst making it simpler to understand.
1. Use the hash of the source file to determine whether the source file
has changed or not. This makes the recompilation checking more robust to
modern build systems which are liable to copy files around changing
their modification times.
2. Remove the concept of a "stable module", a stable module was one
where the object file was older than the source file, and all transitive
dependencies were also stable. Now we don't rely on the modification
time of the source file, the notion of stability is moot.
3. Fix TH/plugin recompilation after the removal of stable modules. The
TH recompilation check used to rely on stable modules. Now there is a
uniform and simple way, we directly track the linkables which were
loaded into the interpreter whilst compiling a module. This is an
over-approximation but more robust wrt package dependencies changing.
4. Fix recompilation checking for dynamic object files. Now we actually
check if the dynamic object file exists when compiling with -dynamic-too
Fixes #19774 #19771 #19758 #17434 #11556 #9121 #8211 #16495 #7277 #16093
|
| |
|
|
|
|
|
|
|
|
| |
In the future, we want `HscEnv` to support multiple home units
at the same time. This means, that there will be 'Target's that do
not belong to the current 'HomeUnit'.
This is an API change without changing behaviour.
Update haddock submodule to incorporate API changes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mode backpack edges
Backpack instantiations need to be typechecked to make sure that the
arguments fit the parameters. `tcRnInstantiateSignature` checks
instantiations with concrete modules, while `tcRnCheckUnit` checks
instantiations with free holes (signatures in the current modules).
Before this change, it worked that `tcRnInstantiateSignature` was called
after typechecking the argument module, see `HscMain.hsc_typecheck`,
while `tcRnCheckUnit` was called in `unsweep'` where-bound in
`GhcMake.upsweep`. `tcRnCheckUnit` was called once per each
instantiation once all the argument sigs were processed. This was done
with simple "to do" and "already done" accumulators in the fold.
`parUpsweep` did not implement the change.
With this change, `tcRnCheckUnit` instead is associated with its own
node in the `ModuleGraph`. Nodes are now:
```haskell
data ModuleGraphNode
-- | Instantiation nodes track the instantiation of other units
-- (backpack dependencies) with the holes (signatures) of the current package.
= InstantiationNode InstantiatedUnit
-- | There is a module summary node for each module, signature, and boot module being built.
| ModuleNode ExtendedModSummary
```
instead of just `ModSummary`; the `InstantiationNode` case is the
instantiation of a unit to be checked. The dependencies of such nodes
are the same "free holes" as was checked with the accumulator before.
Both versions of upsweep on such a node call `tcRnCheckUnit`.
There previously was an `implicitRequirements` function which would
crawl through every non-current-unit module dep to look for all free
holes (signatures) to add as dependencies in `GHC.Driver.Make`. But this
is no good: we shouldn't be looking for transitive anything when
building the graph: the graph should only have immediate edges and the
scheduler takes care that all transitive requirements are met.
So `GHC.Driver.Make` stopped using `implicitRequirements`, and instead
uses a new `implicitRequirementsShallow`, which just returns the
outermost instantiation node (or module name if the immediate dependency
is itself a signature). The signature dependencies are just treated like
any other imported module, but the module ones then go in a list stored
in the `ModuleNode` next to the `ModSummary` as the "extra backpack
dependencies". When `downsweep` creates the mod summaries, it adds this
information too.
------
There is one code quality, and possible correctness thing left: In
addition to `implicitRequirements` there is `findExtraSigImports`, which
says something like "if you are an instantiation argument (you are
substituted or a signature), you need to import its things too". This
is a little non-local so I am not quite sure how to get rid of it in
`GHC.Driver.Make`, but we probably should eventually.
First though, let's try to make a test case that observes that we don't
do this, lest it actually be unneeded. Until then, I'm happy to leave it
as is.
------
Beside the ability to use `-j`, the other major user-visibile side
effect of this change is that that the --make progress log now includes
"Instantiating" messages for these new nodes. Those also are numbered
like module nodes and count towards the total.
------
Fixes #17188
Updates hackage submomdule
Metric Increase:
T12425
T13035
|
|
|
I was working on making DynFlags stateless (#17957), especially by
storing loaded plugins into HscEnv instead of DynFlags. It turned out to
be complicated because HscEnv is in GHC.Driver.Types but LoadedPlugin
isn't: it is in GHC.Driver.Plugins which depends on GHC.Driver.Types. I
didn't feel like introducing yet another hs-boot file to break the loop.
Additionally I remember that while we introduced the module hierarchy
(#13009) we talked about splitting GHC.Driver.Types because it contained
various unrelated types and functions, but we never executed. I didn't
feel like making GHC.Driver.Types bigger with more unrelated Plugins
related types, so finally I bit the bullet and split GHC.Driver.Types.
As a consequence this patch moves a lot of things. I've tried to put
them into appropriate modules but nothing is set in stone.
Several other things moved to avoid loops.
* Removed Binary instances from GHC.Utils.Binary for random compiler
things
* Moved Typeable Binary instances into GHC.Utils.Binary.Typeable: they
import a lot of things that users of GHC.Utils.Binary don't want to
depend on.
* put everything related to Units/Modules under GHC.Unit:
GHC.Unit.Finder, GHC.Unit.Module.{ModGuts,ModIface,Deps,etc.}
* Created several modules under GHC.Types: GHC.Types.Fixity, SourceText,
etc.
* Split GHC.Utils.Error (into GHC.Types.Error)
* Finally removed GHC.Driver.Types
Note that this patch doesn't put loaded plugins into HscEnv. It's left
for another patch.
Bump haddock submodule
|
