Remove DT_Failed state

At the moment if `-dynamic-too` fails then we rerun the whole pipeline
as if we were just in `-dynamic` mode. I argue this is a misfeature and
we should remove the so-called `DT_Failed` mode.

In what situations do we fall back to `DT_Failed`?

1. If the `dyn_hi` file corresponding to a `hi` file is missing completely.
2. If the interface hash of `dyn_hi` doesn't match the interface hash of `hi`.

What happens in `DT_Failed` mode?

* The whole compiler pipeline is rerun as if the user had just passed `-dynamic`.
* Therefore `dyn_hi/dyn_o` files are used which don't agree with the
  `hi/o` files. (As evidenced by `dynamicToo001` test).
* This is very confusing as now a single compiler invocation has
  produced further `hi`/`dyn_hi` files which are different to each
  other.

Why should we remove it?

* In `--make` mode, which is predominately used `DT_Failed` does not
  work (#19782), there can't be users relying on this functionality.
* In `-c` mode, the recovery doesn't fix the root issue, which is the
  `dyn_hi` and `hi` files are mismatched. We should instead produce an
  error and pass responsibility to the build system using `-c` to ensure
  that the prerequisites for `-dynamic-too` (dyn_hi/hi) files are there
  before we start compiling.
* It is a misfeature to support use cases like `dynamicToo001` which
  allow you to mix different versions of dynamic/non-dynamic interface
  files. It's more likely to lead to subtle bugs in your resulting
  programs where out-dated build products are used rather than a
  deliberate choice.
* In practice, people are usually compiling with `-dynamic-too` rather
  than separately with `-dynamic` and `-static`, so the build products
  always match and `DT_Failed` is only entered due to compiler bugs (see
  !6583)

What should we do instead?

* In `--make` mode, for home packages check during recompilation
  checking that `dyn_hi` and `hi` are both present and agree, recompile
  the modules if they do not.
* For package modules, when loading the interface check that `dyn_hi`
  and `hi` are there and that they agree but fail with an
  error message if they are not.
* In `--oneshot` mode, fail with an error message if the right files
  aren't already there.

Closes #19782 #20446 #9176 #13616

25 files changed, 259 insertions, 16 deletions

diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/Makefile b/testsuite/tests/driver/dynamicToo/dynamicToo001/Makefile
index 30971cf752..ee0d464440 100644
--- a/testsuite/tests/driver/dynamicToo/dynamicToo001/Makefile
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001/Makefile
@@ -4,10 +4,14 @@ include $(TOP)/mk/test.mk
 
 TEST_HC_OPTS_DYN = -dynamic -hisuf dyn_hi -osuf dyn_o
 
+checkExists = [ -f $1 ] || echo $1 missing
+
 .PHONY: dynamicToo001
 # -dynamic-too should notice that the interface files for B don't match,
-# and so compile the dyn way separately. This means that the right value
-# of B.b should get inlined.
+#  so an error is issued.
+#
+#  In the past this used to compile the dyn way separately. This means that the "right" value
+#  of B.b should get inlined. (See #20446 for reasons why this was removed).
 dynamicToo001:
 	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c A.hs
 	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c A.hs
@@ -15,9 +19,33 @@ dynamicToo001:
 	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c B.hs
 	cp B2.hs B.hs
 	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c B.hs
-	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c C.hs -dynamic-too
-	"$(TEST_HC)" $(TEST_HC_OPTS) A.o     B.o     C.o     -o s
-	"$(TEST_HC)" $(TEST_HC_OPTS) A.dyn_o B.dyn_o C.dyn_o -o d
-	./s
-	./d
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c C.hs -dynamic-too || true
+#	"$(TEST_HC)" $(TEST_HC_OPTS) A.o     B.o     C.o     -o s
+#	"$(TEST_HC)" $(TEST_HC_OPTS) A.dyn_o B.dyn_o C.dyn_o -o d
+#	./s
+#	./d
+
+
+# Missing a .dyn_hi file and using --make mode
+.PHONY: dynamicToo001MakeA
+dynamicToo001MakeA:
+	cp B1.hs B.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS) -O C.hs -dynamic-too
+	rm B.dyn_hi
+	# Should recompile
+	"$(TEST_HC)" $(TEST_HC_OPTS) -O C.hs -dynamic-too
+	$(call checkExists,B.dyn_hi)
+
+# Should notice that B.hi and B.dyn_hi are different, and recompile B
+.PHONY: dynamicToo001MakeB
+dynamicToo001MakeB:
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c A.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c A.hs
+	cp B1.hs B.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c B.hs
+	cp B2.hs B.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c B.hs
+	cp B1.hs B.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS) -O  C.hs -dynamic-too
+
 
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stderr b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stderr
new file mode 100644
index 0000000000..349b5f2816
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stderr
@@ -0,0 +1,6 @@
+
+C.hs:5:1: error:
+    Dynamic hash doesn't match for ‘B’
+    Normal interface file from ./B.hi
+    Dynamic interface file from ./B.dyn_hi
+    You probably need to recompile ‘B’
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stdout b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stdout
deleted file mode 100644
index e7aea0735f..0000000000
--- a/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001.stdout
+++ /dev/null
@@ -1,4 +0,0 @@
-'a'
-'b'
-'a'
-'c'
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeA.stdout b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeA.stdout
new file mode 100644
index 0000000000..d80c899cb1
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeA.stdout
@@ -0,0 +1,6 @@
+[1 of 3] Compiling A                ( A.hs, A.o, A.dyn_o )
+[2 of 3] Compiling B                ( B.hs, B.o, B.dyn_o )
+[3 of 3] Compiling Main             ( C.hs, C.o, C.dyn_o )
+Linking C ...
+[2 of 3] Compiling B                ( B.hs, B.o, B.dyn_o ) [Missing dynamic interface file]
+Linking C ...
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeB.stdout b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeB.stdout
new file mode 100644
index 0000000000..56caf28582
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001/dynamicToo001MakeB.stdout
@@ -0,0 +1,3 @@
+[2 of 3] Compiling B                ( B.hs, B.o, B.dyn_o ) [Mismatched dynamic interface file]
+[3 of 3] Compiling Main             ( C.hs, C.o, C.dyn_o )
+Linking C ...
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001/test.T b/testsuite/tests/driver/dynamicToo/dynamicToo001/test.T
index 67b1566b54..2c9636bc89 100644
--- a/testsuite/tests/driver/dynamicToo/dynamicToo001/test.T
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001/test.T
@@ -1,6 +1,7 @@
-
-test('dynamicToo001',
-     [extra_files(['A.hs', 'B.hs', 'B1.hs', 'B2.hs', 'C.hs']),
+opts = [ extra_files(['A.hs', 'B1.hs', 'B2.hs', 'C.hs']),
       when(opsys('mingw32'), expect_broken(7665)), unless(have_vanilla(), skip),
-      unless(have_dynamic(), skip)],
-     makefile_test, [])
+      unless(have_dynamic(), skip) ]
+
+test('dynamicToo001', [opts], makefile_test, [])
+test('dynamicToo001MakeA', [opts], makefile_test, [])
+test('dynamicToo001MakeB', [opts], makefile_test, [])
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/A.hs b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/A.hs
new file mode 100644
index 0000000000..f76166afab
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/A.hs
@@ -0,0 +1,6 @@
+
+module A where
+
+a :: Char
+a = 'a'
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B1.hs-boot b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B1.hs-boot
new file mode 100644
index 0000000000..4cbf619183
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B1.hs-boot
@@ -0,0 +1,5 @@
+
+module B where
+
+b :: Char
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B2.hs-boot b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B2.hs-boot
new file mode 100644
index 0000000000..6f0d8a4aec
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/B2.hs-boot
@@ -0,0 +1,4 @@
+module B where
+
+e :: Char
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/C.hs b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/C.hs
new file mode 100644
index 0000000000..40fb0f7695
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/C.hs
@@ -0,0 +1,9 @@
+
+module Main where
+
+import A
+import {-# SOURCE #-} B
+
+main = do print a
+          print b
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/Makefile b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/Makefile
new file mode 100644
index 0000000000..94ead80abc
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/Makefile
@@ -0,0 +1,19 @@
+TOP=../../../..
+include $(TOP)/mk/boilerplate.mk
+include $(TOP)/mk/test.mk
+
+TEST_HC_OPTS_DYN = -dynamic -hisuf dyn_hi -osuf dyn_o
+
+.PHONY: dynamicToo001boot
+# -dynamic-too should notice that the interface files for B.hs-boot don't match,
+#  and issue an error. This is to check the path calculations are correct for boot files.
+dynamicToo001boot:
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c A.hs
+	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c A.hs
+	cp B1.hs-boot B.hs-boot
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c B.hs-boot
+	cp B2.hs-boot B.hs-boot
+	"$(TEST_HC)" $(TEST_HC_OPTS) $(TEST_HC_OPTS_DYN) -O -c B.hs-boot
+	# This step fails because the hash of B1 and B2 is different
+	"$(TEST_HC)" $(TEST_HC_OPTS)                     -O -c C.hs -dynamic-too || true
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/dynamicToo001boot.stderr b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/dynamicToo001boot.stderr
new file mode 100644
index 0000000000..8b17cac27a
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/dynamicToo001boot.stderr
@@ -0,0 +1,6 @@
+
+C.hs:5:1: error:
+    Dynamic hash doesn't match for ‘B’
+    Normal interface file from ./B.hi-boot
+    Dynamic interface file from ./B.dyn_hi-boot
+    You probably need to recompile ‘B’
diff --git a/testsuite/tests/driver/dynamicToo/dynamicToo001boot/test.T b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/test.T
new file mode 100644
index 0000000000..bca0be654f
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicToo001boot/test.T
@@ -0,0 +1,6 @@
+
+test('dynamicToo001boot',
+     [extra_files(['A.hs', 'B1.hs-boot', 'B2.hs-boot', 'C.hs']),
+      when(opsys('mingw32'), expect_broken(7665)), unless(have_vanilla(), skip),
+      unless(have_dynamic(), skip)],
+     makefile_test, [])
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/Makefile b/testsuite/tests/driver/dynamicToo/dynamicTooMake/Makefile
new file mode 100644
index 0000000000..8e24b5a31c
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/Makefile
@@ -0,0 +1,28 @@
+TOP=../../../..
+include $(TOP)/mk/boilerplate.mk
+include $(TOP)/mk/test.mk
+
+SETUP='$(PWD)/Setup' -v0
+CONFIGURE=$(SETUP) configure $(CABAL_MINIMAL_BUILD) --with-ghc='$(TEST_HC)' --ghc-options='$(TEST_HC_OPTS)' --package-db='$(PWD)/tmp.d' --prefix='$(PWD)/inst' --enable-executable-dynamic
+
+dynamicTooMake: clean
+	'$(GHC_PKG)' init tmp.d
+	'$(TEST_HC)' $(TEST_HC_OPTS) -v0 --make Setup
+	# build p
+	rm -rf p/dist
+	(cd p; $(CONFIGURE) --enable-shared --ipid "p-0.1")
+	(cd p; $(SETUP) build)
+	(cd p; $(SETUP) copy)
+	(cd p; $(SETUP) register)
+	(cd p; echo "q = 0" >> Lib.hs)
+	# build p, but only rebuild hi
+	(cd p; $(CONFIGURE) --disable-shared --ipid "p-0.1")
+	(cd p; $(SETUP) build)
+	(cd p; $(SETUP) copy)
+	(cd p; $(SETUP) register)
+	(cd q; $(CONFIGURE) --disable-shared --ipid "q-0.1")
+	# build q, should be an error as p has mismatched .hi and .dyn_hi files
+	(cd q; $(SETUP) build) || true
+
+clean :
+	$(RM) -r tmp.d inst dist Setup$(exeext)
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/README.md b/testsuite/tests/driver/dynamicToo/dynamicTooMake/README.md
new file mode 100644
index 0000000000..5b9ecd01f1
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/README.md
@@ -0,0 +1,5 @@
+Reproducer for broken implementation of DT_Failed
+
+```
+./run
+```
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/Setup.hs b/testsuite/tests/driver/dynamicToo/dynamicTooMake/Setup.hs
new file mode 100644
index 0000000000..9a994af677
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/Setup.hs
@@ -0,0 +1,2 @@
+import Distribution.Simple
+main = defaultMain
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/dynamicTooMake.stderr b/testsuite/tests/driver/dynamicToo/dynamicTooMake/dynamicTooMake.stderr
new file mode 100644
index 0000000000..23225fc4fc
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/dynamicTooMake.stderr
@@ -0,0 +1,9 @@
+Warning: -rtsopts and -with-rtsopts have no effect with -shared.
+    Call hs_init_ghc() from your main() function to set these options.
+
+app/Main.hs:4:1: error:
+    Dynamic hash doesn't match for ‘Lib’
+    Normal interface file from /run/user/1000/ghctest-hbhb_f3v/test   spaces/testsuite/tests/driver/dynamicToo/dynamicTooMake/dynamicTooMake.run/inst/lib/x86_64-linux-ghc-9.3.20210922/p-0.1/Lib.hi
+    Dynamic interface file from /run/user/1000/ghctest-hbhb_f3v/test   spaces/testsuite/tests/driver/dynamicToo/dynamicTooMake/dynamicTooMake.run/inst/lib/x86_64-linux-ghc-9.3.20210922/p-0.1/Lib.dyn_hi
+    You probably need to recompile ‘Lib’
+make: *** [Makefile:26: dynamicTooMake] Error 1
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/CHANGELOG.md b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/CHANGELOG.md
new file mode 100644
index 0000000000..9ede8b27d4
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/CHANGELOG.md
@@ -0,0 +1,5 @@
+# Revision history for p
+
+## 0.1.0.0 -- YYYY-mm-dd
+
+* First version. Released on an unsuspecting world.
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/Lib.hs b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/Lib.hs
new file mode 100644
index 0000000000..4e718cafbf
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/Lib.hs
@@ -0,0 +1,4 @@
+module Lib where
+
+{-# NOINLINE l #-}
+l = 1
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/p.cabal b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/p.cabal
new file mode 100644
index 0000000000..a0d6c17515
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/p/p.cabal
@@ -0,0 +1,34 @@
+cabal-version:      2.4
+name:               p
+version:            0.1.0.0
+
+-- A short (one-line) description of the package.
+-- synopsis:
+
+-- A longer description of the package.
+-- description:
+
+-- A URL where users can report bugs.
+-- bug-reports:
+
+-- The license under which the package is released.
+-- license:
+author:             Matthew Pickering
+maintainer:         matthewtpickering@gmail.com
+
+-- A copyright notice.
+-- copyright:
+-- category:
+extra-source-files: CHANGELOG.md
+
+library
+    exposed-modules: Lib
+
+    -- Modules included in this executable, other than Main.
+    -- other-modules:
+
+    -- LANGUAGE extensions used by modules in this package.
+    -- other-extensions:
+    build-depends:    base >=4.14.1.0 && <5
+    hs-source-dirs:   .
+    default-language: Haskell2010
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/CHANGELOG.md b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/CHANGELOG.md
new file mode 100644
index 0000000000..62632c5376
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/CHANGELOG.md
@@ -0,0 +1,5 @@
+# Revision history for q
+
+## 0.1.0.0 -- YYYY-mm-dd
+
+* First version. Released on an unsuspecting world.
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/AppLib.hs b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/AppLib.hs
new file mode 100644
index 0000000000..acbae9417c
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/AppLib.hs
@@ -0,0 +1,5 @@
+module AppLib where
+
+a = 10
+
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/Main.hs b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/Main.hs
new file mode 100644
index 0000000000..ea2b9aef96
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/app/Main.hs
@@ -0,0 +1,8 @@
+module Main where
+
+import AppLib
+import Lib
+
+main :: IO ()
+main = print (a + l )
+
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/q.cabal b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/q.cabal
new file mode 100644
index 0000000000..723e1082d5
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/q/q.cabal
@@ -0,0 +1,34 @@
+cabal-version:      2.4
+name:               q
+version:            0.1.0.0
+
+-- A short (one-line) description of the package.
+-- synopsis:
+
+-- A longer description of the package.
+-- description:
+
+-- A URL where users can report bugs.
+-- bug-reports:
+
+-- The license under which the package is released.
+-- license:
+author:             Matthew Pickering
+maintainer:         matthewtpickering@gmail.com
+
+-- A copyright notice.
+-- copyright:
+-- category:
+extra-source-files: CHANGELOG.md
+
+library
+
+    -- Modules included in this executable, other than Main.
+    exposed-modules: AppLib Main
+
+    -- LANGUAGE extensions used by modules in this package.
+    -- other-extensions:
+    ghc-options: -dynamic-too
+    build-depends:    base >=4.14.1.0 && < 5, p
+    hs-source-dirs:   app
+    default-language: Haskell2010
diff --git a/testsuite/tests/driver/dynamicToo/dynamicTooMake/test.T b/testsuite/tests/driver/dynamicToo/dynamicTooMake/test.T
new file mode 100644
index 0000000000..f771282e69
--- /dev/null
+++ b/testsuite/tests/driver/dynamicToo/dynamicTooMake/test.T
@@ -0,0 +1,9 @@
+
+test('dynamicTooMake',
+     [extra_files(['p', 'q', 'Setup.hs']),
+      when(opsys('mingw32'), expect_broken(7665)), unless(have_vanilla(), skip),
+      unless(have_dynamic(), skip),
+      copy_files,
+      grep_errmsg("Dynamic hash")
+      ],
+     makefile_test, [])