Document the bootstrap procedure

 * Remove the now obsolete comments at the beginning of the main Makefile
 * Move and fix documentation of the bootstrap from INSTALL to
   HACKING-bootstrap.adoc
 * Update install instructions

5 files changed, 72 insertions, 37 deletions

diff --git a/BOOTSTRAP.adoc b/BOOTSTRAP.adoc
new file mode 100644
index 0000000000..31a51c1a9a
--- /dev/null
+++ b/BOOTSTRAP.adoc
@@ -0,0 +1,58 @@
+= Bootstrapping the compiler
+
+This file explains how to bootstrap the OCaml compiler, i.e. how to
+update the binaries in the link:boot/[] directory.
+
+A bootstrap is required for example when something changes in the
+runtime system (the magic number of bytecode executables, the format of
+bytecode instructions, the set of available primitives) or when the
+format of .cmi files is modified. In particular, given that the .cmi
+files contain information related to types, modifying the way a type is
+represented will modify the format of .cmi files and thus require a
+bootstrap.
+
+Here is how to perform a change that requires a bootstrap:
+
+1. Make sure you start with a clean source tree (e.g. check with
+   `git status`)
+
+2. Configure your source tree by running:
+
+        ./configure
+
+3. Bring your system to a stable state. Concretely, this means that the
+   boot/ directory should contain a version of ocamlrun and all the
+   \*.cm* files of the standard library. This stable state can be reached
+   by running
+
+        make world
++
+(Actually, running `make coldstart` should be enough but `make world` is
+safer. Similarly, `make world.opt` will also bring you to such a stable
+state but builds more things than actually required.)
+
+4. Now, and only now, edit the sources. Changes here may include adding,
+   removing or renaming a primitive in the runtime, changing the magic
+   number of bytecode executable files, changing the way types are
+   represented or anything else in the format of .cmi files, etc.
+
+5. Run:
+
+        make coreall
++
+This will rebuild byterun/ocamlrun, ocamlc, etc.
+
+6. (optional) The new system can now be tested:
+
+        echo 'let _ = print_string "Hello world!\n"' > foo.ml
+        ./boot/ocamlrun ./ocamlc -I ./stdlib foo.ml
+        ./byterun/ocamlrun a.out
+
+7. We now know the system works and can thus build the new boot/
+   binaries:
+
+        make bootstrap
+
+If you notice that this procedure fails for a given change you are
+trying to implement, please report it so that the procedure can be
+updated to also cope with your change.
diff --git a/Changes b/Changes
index fc5ea0fb65..add196d939 100644
--- a/Changes
+++ b/Changes
@@ -92,6 +92,10 @@ Working version
 - GPR#1797: remove the deprecated Makefile.nt files
   (Sébastien Hinderer, review by Nicolas Ojeda Bar)  
 
+- GPR#1805: fix the bootstrap procedure and its documentation
+  (Sébastien Hinderer, Xavier Leroy and Damien Doligez; review by
+  Gabriel Scherer)
+
 ### Internal/compiler-libs changes:
 
 - GPR#1745: do not generalize the type of every sub-pattern, only of variables
diff --git a/HACKING.adoc b/HACKING.adoc
index 368c592ce9..6dfff8c6e6 100644
--- a/HACKING.adoc
+++ b/HACKING.adoc
@@ -157,13 +157,14 @@ has excellent documentation.
 
 === Complete file listing
 
+  BOOTSTRAP.adoc::        instructions for bootstrapping
   Changes::               what's new with each release
-  configure::             configure script
   CONTRIBUTING.md::       how to contribute to OCaml
   HACKING.adoc::          this file
   INSTALL.adoc::          instructions for installation
   LICENSE::               license and copyright notice
   Makefile::              main Makefile
+  Makefile.common::       common Makefile definitions
   Makefile.tools::        used by manual/ and testsuite/ Makefiles
   README.adoc::           general information on the compiler distribution
   README.win32.adoc::     general information on the Windows ports of OCaml
@@ -175,6 +176,7 @@ has excellent documentation.
   byterun/::              bytecode interpreter and runtime system
   compilerlibs/::         the OCaml compiler as a library
   config/::               configuration files
+  configure::             configure script
   debugger/::             source-level replay debugger
   driver/::               driver code for the compilers
   emacs/::                editing mode and debugger interface for GNU Emacs
@@ -185,6 +187,7 @@ has excellent documentation.
   manual/::               system to generate the manual
   middle_end/::           the flambda optimisation phase
   ocamldoc/::             documentation generator
+  ocamltest/::            test driver
   otherlibs/::            several additional libraries
   parsing/::              syntax analysis -- see link:parsing/HACKING.adoc[]
   stdlib/::               standard library
@@ -223,12 +226,12 @@ installation, the following targets may be of use:
 === Bootstrapping
 
 The OCaml compiler is bootstrapped. This means that
-previously-compiled bytecode versions of the compiler, dependency
-generator and lexer are included in the repository under the
+previously-compiled bytecode versions of the compiler and lexer are
+included in the repository under the
 link:boot/[] directory. These bytecode images are used once the
 bytecode runtime (which is written in C) has been built to compile the
 standard library and then to build a fresh compiler. Details can be
-found in link:INSTALL.adoc#bootstrap[INSTALL.adoc].
+found in link:BOOTSTRAP.adoc[].
 
 === Continuous integration
 
diff --git a/INSTALL.adoc b/INSTALL.adoc
index bf3990fdb9..f48db57e1b 100644
--- a/INSTALL.adoc
+++ b/INSTALL.adoc
@@ -213,25 +213,10 @@ fairly verbose; consider redirecting the output to a file:
         make world > log.world 2>&1     # in sh
         make world >& log.world         # in csh
 
-[[bootstrap]]
-3. (Optional) To be sure everything works well, you can try to bootstrap the
-   system -- that is, to recompile all OCaml sources with the newly created
-   compiler. From the top directory, do:
+3. (Optional) To be sure everything works well, you can run the test suite
+   that comes with the compiler. To do so, do:
 
-        make bootstrap
-+
-or, better:
-
-        make bootstrap > log.bootstrap 2>&1     # in sh
-        make bootstrap >& log.bootstrap         # in csh
-+
-The `make bootstrap` checks that the bytecode programs compiled with the new
-compiler are identical to the bytecode programs compiled with the old compiler.
-If this is the case, you can be pretty sure the system has been correctly
-compiled. Otherwise, this does not necessarily mean something went wrong.  The
-best thing to do is to try a second bootstrapping phase: just do
-`make bootstrap` again.  It will either crash almost immediately, or
-re-re-compile everything correctly and reach the fix-point.
+        make tests
 
 4. You can now install the OCaml system. This will create the following commands
    (in the binary directory selected during autoconfiguration):
diff --git a/Makefile b/Makefile
index 1f6bf33735..4c35fa7296 100644
--- a/Makefile
+++ b/Makefile
@@ -15,21 +15,6 @@
 
 # The main Makefile
 
-# Hard bootstrap how-to:
-# (only necessary if you remove or rename some primitive)
-#
-# make core     [old system -- you were in a stable state]
-# make coreboot [optional -- check state stability]
-# <add new primitives and remove uses of old primitives>
-# make clean && make core
-# if the above fails:
-#     <debug your changes>
-#     make clean && make core
-# make coreboot [intermediate state with both old and new primitives]
-# <remove old primitives>
-# make clean && make runtime && make coreall
-# make coreboot [new system -- now in a stable state]
-
 include config/Makefile
 include Makefile.common