Add 'hadrian/' from commit '45f3bff7016a2a0cd9a5455a882ced984655e90b'

git-subtree-dir: hadrian
git-subtree-mainline: 575b35f4cdc18045bccd42d341d6f25d95c0696c
git-subtree-split: 45f3bff7016a2a0cd9a5455a882ced984655e90b

4 files changed, 525 insertions, 0 deletions

diff --git a/hadrian/doc/cross-compile.md b/hadrian/doc/cross-compile.md
new file mode 100644
index 0000000000..724d0e17bd
--- /dev/null
+++ b/hadrian/doc/cross-compile.md
@@ -0,0 +1,59 @@
+## Build a cross-compiling GHC
+
+In this example, our host machine is "Ubuntu 16.04.2 LTS, Linux ubuntu 4.4.0-79-generic 86_64".
+
+We need to download necessary tools, including:
+
+- [LLVM-4.0 source](http://releases.llvm.org/4.0.0/llvm-4.0.0.src.tar.xz), you need to build it yourself. Remember to choose release channel and use gold linker (`cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_LINKER=gold ..`)
+- `sudo apt-get install gcc-arm-linux-gnueabihf` to install the GCC cross-compiler
+- Download and install [Haskell Platform 8.0.2](https://haskell.org/platform/download/8.0.2/haskell-platform-8.0.2-unknown-posix--full-x86_64.tar.gz). Install it according to [instructions here](https://www.haskell.org/platform/linux.html#linux-generic)
+
+After all the dependencies are in place:
+
+- `git clone https://github.com/ghc/ghc`
+- `cd ghc`
+- `git clone https://github.com/snowleopard/hadrian`
+- `git submodule update --init`
+- `./configure --target=arm-linux-gnueabihf`
+- `cd hadrian`
+- Modify `src/UserSettings.hs`, set `stage1Only` to `True`.
+- Build the compiler by e.g. `./build.sh --flavour=quickest --integer-simple -V -j`
+
+After that, you should have built `inplace/bin/ghc-stage1` cross compiler. We will go to the next section to validate this.
+
+**NOTE**: Use of `-c` to configure the target is currently not supported. Please manually run `./configure` like above.
+
+## Test run
+
+Write a simple hello world haskell program:
+
+```haskell
+module Main where
+main = putStrLn "Hello, world!"
+```
+Compile it with cross-compiling GHC: `<ghc-folder>/inplace/bin/ghc-stage1 -static Main`. Note that we created a static version of it which packs together all depending libraries.
+
+- Install QEMU: `sudo apt-get install qemu-system-arm`
+- Download `vmlinuz` (kernel) and `initrd.gz` (initial ramdisk), e.g. from [this mirror](https://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/dists/xenial/main/installer-armhf/current/images/generic-lpae/cdrom/).
+- Add the ARM Linux executable `Main` to the initial ramdisk so we can load it directly into memory. No need for real installation
+  + `gunzip initrd.gz` to get `initrd`
+  + `mkdir tmp2; cd tmp2; sudo cpio -id < ../initrd` to get a file system
+  + `cp /PATH/TO/Main usr/bin`
+  + `find . | cpio --create --format='newc' > /tmp/newinitrd` to pack back the `initrd`
+  + `gzip /tmp/newinitrd`
+  + Move `newinitrd` to where `vmlinuz` is, rename it to `newinitrd.img`
+  + Run the following configured QEMU:
+
+```bash
+#!/bin/sh
+qemu-system-arm \
+    -kernel vmlinuz \
+    -initrd newinitrd.img \
+    -append "root=/dev/vda2 rootfstype=ext4" \
+    -no-reboot \
+    -nographic \
+    -m 1024 \
+    -M virt
+```
+
+This will lead you to a installer interface. But we don't need to do that, so we can save ourself from the hassle of setting up networks etc. We just keep `Go Back`, until see a line `Execute a shell`, and select it. Now you get a shell, go find `/usr/bin/Main` and run it!
diff --git a/hadrian/doc/flavours.md b/hadrian/doc/flavours.md
new file mode 100644
index 0000000000..f276dbb265
--- /dev/null
+++ b/hadrian/doc/flavours.md
@@ -0,0 +1,176 @@
+# Build flavours
+
+Hadrian supports a few predefined _build flavours_, i.e. collections of build
+settings that fully define a GHC build (see `src/Flavour.hs`). Users can add their
+own build flavours if need be, as described
+[here](https://github.com/snowleopard/hadrian/blob/master/doc/user-settings.md#build-flavour).
+
+## Arguments
+
+The following table summarises extra arguments passed to GHC in different build flavours.
+There are four groups of arguments: arguments in `hsDefault` are passed to GHC for all Haskell
+source files, `hsLibrary` arguments are added when compiling libraries, `hsCompiler`
+when compiling the `compiler` library, and `hsGhc` when compiling/linking the GHC program.
+
+<table>
+  <tr>
+    <th rowspan="3">Flavour</th>
+    <th colspan="8">Extra arguments</th>
+  </tr>
+  <tr>
+    <th colspan="2">hsDefault</td>
+    <th colspan="2">hsLibrary</td>
+    <th colspan="2">hsCompiler</td>
+    <th colspan="2">hsGhc</td>
+  </tr>
+  <tr>
+    <th>stage0</td>
+    <th>stage1+</td>
+    <th>stage0</td>
+    <th>stage1+</td>
+    <th>stage0</td>
+    <th>stage1+</td>
+    <th>stage0</td>
+    <th>stage1+</td>
+  </tr>
+  <tr>
+    <th>default<br></td>
+    <td>-O<br>-H64m<br></td>
+    <td>-O2<br>-H64m</td>
+    <td></td>
+    <td></td>
+    <td></td>
+    <td></td>
+    <td></td>
+    <td></td>
+  </tr>
+  <tr>
+    <th>quick</td>
+    <td>-O0<br>-H64m</td>
+    <td>-O0<br>-H64m</td>
+    <td></td>
+    <td>-O</td>
+    <td>-O</td>
+    <td></td>
+    <td>-O</td>
+    <td></td>
+  </tr>
+  <tr>
+    <th>quickest</td>
+    <td>-O0<br>-H64m</td>
+    <td>-O0<br>-H64m</td>
+    <td></td>
+    <td></td>
+    <td>-O</td>
+    <td></td>
+    <td>-O</td>
+    <td></td>
+  </tr>
+  <tr>
+    <th>perf</td>
+    <td>-O<br>-H64m</td>
+    <td>-O<br>-H64m</td>
+    <td></td>
+    <td>-O2</td>
+    <td>-O</td>
+    <td>-O2</td>
+    <td>-O</td>
+    <td>-O2</td>
+  </tr>
+  <tr>
+    <th>prof</td>
+    <td>-O0<br>-H64m</td>
+    <td>-O0<br>-H64m</td>
+    <td></td>
+    <td>-O</td>
+    <td>-O</td>
+    <td>-O</td>
+    <td>-O</td>
+    <td>-O</td>
+  </tr>
+  <tr>
+    <th>devel1</td>
+    <td>-O<br>-H64m</td>
+    <td>-O<br>-H64m</td>
+    <td></td>
+    <td>-dcore-lint</td>
+    <td>-O0<br>-DDEBUG</td>
+    <td></td>
+    <td>-O0<br>-DDEBUG</td>
+    <td></td>
+  </tr>
+  <tr>
+    <th>devel2</td>
+    <td>-O<br>-H64m</td>
+    <td>-O<br>-H64m</td>
+    <td></td>
+    <td>-dcore-lint</td>
+    <td></td>
+    <td>-O0<br>-DDEBUG</td>
+    <td></td>
+    <td>-O0<br>-DDEBUG</td>
+  </tr>
+</table>
+
+## Ways
+
+Libraries and GHC can be built in different _ways_, e.g. with or without profiling
+information. The following table lists ways that are built in different flavours.
+
+<table>
+    <tr>
+        <th rowspan="2">Flavour</th>
+        <th colspan="2">Library ways</th>
+        <th colspan="2">RTS ways</th>
+        <th colspan="2">Profiled GHC</th>
+    </tr>
+    <tr>
+        <th>stage0</th>
+        <th>stage1+</th>
+        <th>stage0</th>
+        <th>stage1+</th>
+        <th>stage0</th>
+        <th>stage1+</th>
+    </tr>
+    <tr>
+    <th>default<br>perf<br>prof<br>devel1<br>devel2</td>
+    <td>vanilla</td>
+    <td>vanilla<br>profiling<br>dynamic</td>
+    <td>logging<br>debug<br>threaded<br>threadedDebug<br>threadedLogging
+        <br>debugDynamic<br>threadedDynamic<br>threadedDebugDynamic
+        <br>loggingDynamic<br>threadedLoggingDynamic
+    </td>
+    <td>
+        logging<br>debug<br>threaded<br>threadedDebug<br>
+        threadedLogging<br>threadedProfiling
+        <br>debugDynamic<br>threadedDynamic<br>threadedDebugDynamic
+        <br>loggingDynamic<br>threadedLoggingDynamic
+    </td>
+    <td>Only in<br>prof<br>flavour</td>
+    <td>Only in<br>prof<br>flavour</td>
+</tr>
+<tr>
+    <th>quick</th>
+    <td>vanilla</td>
+    <td>vanilla<br>dynamic</td>
+    <td>logging<br>debug<br>threaded<br>threadedDebug<br>threadedLogging
+        <br>debugDynamic<br>threadedDynamic<br>threadedDebugDynamic
+        <br>loggingDynamic<br>threadedLoggingDynamic
+    </td>
+    <td>logging<br>debug<br>threaded<br>threadedDebug<br>threadedLogging
+        <br>debugDynamic<br>threadedDynamic<br>threadedDebugDynamic
+        <br>loggingDynamic<br>threadedLoggingDynamic
+    </td>
+    <td>No</td>
+    <td>No</td>
+</tr>
+<tr>
+    <th>quickest</th>
+    <td>vanilla</td>
+    <td>vanilla</td>
+    <td>vanilla<br>threaded</td>
+    <td>vanilla<br>threaded</td>
+    <td>No</td>
+    <td>No</td>
+</tr>
+</table>
diff --git a/hadrian/doc/user-settings.md b/hadrian/doc/user-settings.md
new file mode 100644
index 0000000000..b81c0dc582
--- /dev/null
+++ b/hadrian/doc/user-settings.md
@@ -0,0 +1,220 @@
+# User settings
+
+You can customise Hadrian by copying the file `hadrian/src/UserSettings.hs` to
+`hadrian/UserSettings.hs` and overriding the default build settings (if you don't
+copy the file your changes will be tracked by `git` and you can accidentally commit
+them). Here we document currently supported settings.
+
+## Build flavour
+
+Build _flavour_ is a collection of build settings that fully define a GHC build
+(see `src/Flavour.hs`):
+```haskell
+data Flavour = Flavour {
+    -- | Flavour name, to select this flavour from command line.
+    name :: String,
+    -- | Use these command line arguments.
+    args :: Args,
+    -- | Build these packages.
+    packages :: Stage -> Action [Package],
+    -- | Either 'integerGmp' or 'integerSimple'.
+    integerLibrary :: Action Package,
+    -- | Build libraries these ways.
+    libraryWays :: Ways,
+    -- | Build RTS these ways.
+    rtsWays :: Ways,
+    -- | Build split objects.
+    splitObjects :: Predicate,
+    -- | Build dynamic GHC programs.
+    dynamicGhcPrograms :: Action Bool,
+    -- | Enable GHCi debugger.
+    ghciWithDebugger :: Bool,
+    -- | Build profiled GHC.
+    ghcProfiled :: Bool,
+    -- | Build GHC with debug information.
+    ghcDebugged :: Bool }
+```
+Hadrian provides several built-in flavours (`defaultFlavour`, `quickFlavour`, and a few
+others; see `hadrian/doc/flavours.md`), which can be activated from the command line,
+e.g. by passing `--flavour=quick`. Users can define new build flavours by adding them
+to `userFlavours` list:
+```haskell
+-- | User-defined build flavours. See 'userFlavour' as an example.
+userFlavours :: [Flavour]
+userFlavours = [userFlavour] -- Add more build flavours if need be.
+
+-- | This is an example user-defined build flavour. Feel free to modify it and
+-- use by passing @--flavour=user@ from the command line.
+userFlavour :: Flavour
+userFlavour = defaultFlavour { name = "user" } -- Modify other settings here.
+```
+Now `--flavour=user` will run Hadrian with `userFlavour` settings. In the
+following sections we look at specific fields of the `Flavour` record in
+more detail. Note: `defaultFlavour`, as well as its individual fields such
+as `defaultArgs`, `defaultPackages`, etc. that we use below, are defined in module
+`Settings.Default`.
+
+## Command line arguments
+
+One of the key features of Hadrian is that users can easily modify any build command.
+The build system will detect the change and will rerun all affected build rules during
+the next build, without requiring a full rebuild.
+
+For example, here is how to pass an extra argument `-O0` to all invocations of
+GHC when compiling package `cabal`:
+```haskell
+userFlavour :: Flavour
+userFlavour = defaultFlavour { name = "user", args = defaultArgs <> userArgs }
+
+userArgs :: Args
+userArgs = builder Ghc ? package cabal ? arg "-O0"
+```
+Builders such as `Ghc` are defined in `src/Builder.hs`, and all packages that
+are currently built as part of the GHC are defined in `src/GHC.hs`.
+
+You can combine several custom command line settings using `mconcat`:
+```haskell
+userArgs :: Args
+userArgs = mconcat
+    [ builder Ghc ? package cabal ? arg "-O0"
+    , package rts ? input "//PrimOps.c" ? pure ["-fno-PIC", "-static"] ]
+```
+You can match any combination of the `builder`, `stage`, `package`, `way`, `input`
+and `output` predicates when specifying custom command line arguments. File
+patterns such as `"//Prelude.*"` can be used when matching input and output files,
+where `//` matches an arbitrary number of path components and `*` matches an entire
+path component, excluding any separators.
+
+## Packages
+
+Users can add and remove packages from particular build stages. As an example,
+below we add package `base` to Stage0 and remove package `haskeline` from Stage1:
+```haskell
+userFlavour :: Flavour
+userFlavour = defaultFlavour { name = "user", packages = modifiedPackages }
+
+modifiedPackages :: Stage -> Action [Package]
+modifiedPackages stage = do
+    packages <- defaultPackages stage
+    return $ case stage of
+        Stage0 -> packages ++ [base]
+        Stage1 -> packages \\ [haskeline]
+        _      -> packages
+```
+If you are working on a new GHC package you need to let Hadrian know about it
+by adding it to `userPackages`:
+```haskell
+userPackages :: [Package]
+userPackages = [userPackage]
+
+-- An example package that lives in "libraries/user-package" directory.
+userPackage :: Package
+userPackage = library "user-package"
+```
+You will also need to add `userPackage` to a specific build stage by modifying
+the `packages` setting of the user flavour as otherwise it will not be built.
+
+You can choose which integer library to use when builing GHC using the
+`integerLibrary` setting of the build flavour. Possible values are: `integerGmp`
+(default) and `integerSimple`.
+```haskell
+userFlavour :: Flavour
+userFlavour = defaultFlavour { name = "user", integerLibrary = integerSimple }
+```
+## Build ways
+
+Packages can be built in a number of ways, such as `vanilla`, `profiling` (with
+profiling information enabled), and many others as defined in `src/Way.hs`. You
+can change the default build ways by modifying `libraryWays` and `rtsWays` fields
+of the `Flavour` record as required. As an example, below we remove `profiling`
+from the list of library ways:
+```haskell
+noProfilingFlavour :: Flavour
+noProfilingFlavour = defaultFlavour
+    { name        = "no-profiling"
+    , libraryWays = remove [profiling] defaultLibraryWays
+    , ghcProfiled = False } -- Can't build profiled GHC without profiled libraries
+```
+Note that `rtsWays` is computed from `libraryWays` by default, therefore the above
+change will lead to the removal of `threadedProfiling` way from `rtsWays`. To
+change this behaviour, you can override the default `rtsWays` setting.
+
+Similarly, if we want to completely turn off dynamic linking, we can define a custom
+`Flavour` to this effect:
+``` haskell
+noDynamicFlavour :: Flavour
+noDynamicFlavour = defaultFlavour
+    { name = "no-dynamic"
+    , libraryWays = remove [dynamic] defaultLibraryWays }
+```
+
+## Verbose command lines
+
+By default Hadrian does not print full command lines during the build process
+and instead prints short human readable digests for each executed command. You
+can suppress this behaviour completely or partially using `verboseCommand` setting:
+```haskell
+-- | Set to 'True' to print full command lines during the build process. Note:
+-- this is a 'Predicate', hence you can enable verbose output only for certain
+-- targets, e.g.: @verboseCommand = package ghcPrim@.
+verboseCommand :: Predicate
+verboseCommand = do
+    verbosity <- expr getVerbosity
+    return $ verbosity >= Loud
+```
+For example, to print the full command lines used to compile GHC executables,
+set `verboseCommands` to:
+```haskell
+verboseCommand :: Predicate
+verboseCommand = input "ghc/Main.hs"
+```
+Below are a few other examples:
+```haskell
+-- Print command lines for all Ghc Link invocations:
+verboseCommand = builder (Ghc Link)
+
+-- Print command lines when compiling files in package compiler using Gcc:
+verboseCommand = builder (Gcc Compile) &&^ package compiler
+
+-- Use patterns when matching files:
+verboseCommand = output "//rts/sm/*" &&^ way threaded
+
+-- Print all commands:
+verboseCommand = return True
+```
+
+## Miscellaneous
+
+By setting `stage1Only = True` you can disable building Stage2 GHC and Stage2
+utilities such as `haddock`. Note that all Stage0 and Stage1 libraries will
+still be built.
+
+To change the default behaviour of Hadrian with respect to building split
+objects, override the `splitObjects` setting of the `Flavour` record:
+```haskell
+userFlavour :: Flavour
+userFlavour = defaultFlavour { name = "user", splitObjects = return False }
+```
+
+Hadrian prints various progress info during the build. You can change the colours
+used by default by overriding `buildProgressColour` and `successColour`:
+```haskell
+-- | Set colour for build progress messages (e.g. executing a build command).
+buildProgressColour :: BuildProgressColour
+buildProgressColour = mkBuildProgressColour (Dull Magenta)
+
+-- | Set colour for success messages (e.g. a package is built successfully).
+successColour :: SuccessColour
+successColour = mkSuccessColour (Dull Green)
+```
+
+Your options are `Dull Colour`, `Vivid Colour`, or `Extended Code`. `Dull`
+colours are the ANSI 8-bit colours, `Vivid` correspond to the 16-bit codes that
+end with ";1", and `Extended` let's you enter a manual code for the 256 colour
+set. E.g.
+
+```
+Dull Blue
+Vivid Cyan
+Extended "203"
+```
diff --git a/hadrian/doc/windows.md b/hadrian/doc/windows.md
new file mode 100644
index 0000000000..0ad2086547
--- /dev/null
+++ b/hadrian/doc/windows.md
@@ -0,0 +1,70 @@
+# Building GHC on Windows

+

+[![Windows status](https://img.shields.io/appveyor/ci/snowleopard/hadrian/master.svg?label=Windows)](https://ci.appveyor.com/project/snowleopard/hadrian)

+

+Here is how you can build GHC, from source, on Windows. We assume that `git` and `stack` are installed

+(see [prerequisites](https://github.com/snowleopard/hadrian/blob/master/doc/windows.md#prerequisites)).

+

+```sh

+# Get GHC and Hadrian sources; git core.autocrlf should be set to false (see Prerequisites section)

+git clone --recursive git://git.haskell.org/ghc.git

+cd ghc

+git clone git://github.com/snowleopard/hadrian

+

+# Download and install the bootstrapping GHC and MSYS2

+cd hadrian

+stack setup

+

+# Install utilities required during the GHC build process

+stack exec -- pacman -S autoconf automake-wrapper make patch python tar --noconfirm

+

+# Build Hadrian and dependencies (including GHC dependencies Alex and Happy)

+stack build

+

+# Build GHC

+# Note that the --configure flag is required only for the first build

+stack exec hadrian -- --directory ".." -j --flavour=quickest --configure

+

+# Test GHC

+cd ..

+inplace\bin\ghc-stage2 -e 1+2

+```

+

+The entire process should take about 20 minutes. Note, this will build GHC without

+optimisations. If you need an optimised GHC, drop the `--flavour=quickest` flag from

+the build command line (this will slow down the build to about an hour).

+

+These are currently not the

+[official GHC building instructions](https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows),

+but are much simpler and may also be more robust.

+

+The `stack build` and `stack exec hadrian` commands can be replaced by an invocation

+of Hadrian's Stack-based build script: `build.stack.bat -j --flavour=quickest`. Use this

+script if you plan to work on Hadrian and/or rebuild GHC often.

+

+## Prerequisites

+

+The above works on a clean machine with `git` and `stack` installed (tested with default

+installation settings), which you can get from https://git-scm.com/download/win and

+https://www.stackage.org/stack/windows-x86_64-installer.

+

+Note that `git` should be configured to check out Unix-style line endings. The default behaviour

+of `git` on Windows is to check out Windows-style line endings which can cause issues during the

+build. This can be changed using the following command:

+

+    git config --global core.autocrlf false

+

+If you would like to restore the default behaviour later run:

+

+    git config --global core.autocrlf true

+

+## Testing

+

+These instructions have been tested on a clean Windows 10 machine using the

+[free VirtualBox image](https://dev.windows.com/en-us/microsoft-edge/tools/vms/windows/),

+and are also routinely tested on

+[Hadrian's AppVeyor CI instance](https://ci.appveyor.com/project/snowleopard/hadrian/history).

+

+## Notes

+

+Beware of the [current limitations of Hadrian](https://github.com/snowleopard/hadrian#current-limitations).