Linker/darwin: Properly honour -fno-use-rpaths

The specification is now simple

* On linux, use `-Xlinker -rpath -Xlinker` to set the rpath of the
executable
* On darwin, never use `-Xlinker -rpath -Xlinker`, always inject
the rpath afterwards, see `runInjectRPaths`.
* If `-fno-use-rpaths` is passed then *never* inject anything into the
rpath.

Fixes #20004

(cherry picked from commit 6281a333303a4dbe75a97a87c17c0fbace5268f5)

7 files changed, 74 insertions, 34 deletions

diff --git a/compiler/ghci/Linker.hs b/compiler/ghci/Linker.hs
index df684449fb..7561c0cb08 100644
--- a/compiler/ghci/Linker.hs
+++ b/compiler/ghci/Linker.hs
@@ -914,7 +914,7 @@ dynLoadObjs hsc_env pls@PersistentLinkerState{..} objs = do
                            concatMap (\l -> [ Option ("-l" ++ l) ])
                                      (nub $ snd <$> temp_sos)
                         ++ concatMap (\lp -> Option ("-L" ++ lp)
-                                          : if gopt Opt_RPath dflags
+                                          : if useXLinkerRPath dflags (platformOS platform)
                                             then [ Option "-Xlinker"
                                                  , Option "-rpath"
                                                  , Option "-Xlinker"
@@ -923,7 +923,7 @@ dynLoadObjs hsc_env pls@PersistentLinkerState{..} objs = do
                                      (nub $ fst <$> temp_sos)
                         ++ concatMap
                              (\lp -> Option ("-L" ++ lp)
-                                  : if gopt Opt_RPath dflags
+                                  : if useXLinkerRPath dflags (platformOS platform)
                                     then [ Option "-Xlinker"
                                          , Option "-rpath"
                                          , Option "-Xlinker"
diff --git a/compiler/main/DriverPipeline.hs b/compiler/main/DriverPipeline.hs
index a0be34e8c0..a7663819da 100644
--- a/compiler/main/DriverPipeline.hs
+++ b/compiler/main/DriverPipeline.hs
@@ -1731,7 +1731,7 @@ linkBinary' staticLink dflags o_files dep_packages = do
                                  (l `makeRelativeTo` full_output_fn)
                             else l
                   -- See Note [-Xlinker -rpath vs -Wl,-rpath]
-                  rpath = if gopt Opt_RPath dflags
+                  rpath = if useXLinkerRPath dflags (platformOS platform)
                           then ["-Xlinker", "-rpath", "-Xlinker", libpath]
                           else []
                   -- Solaris 11's linker does not support -rpath-link option. It silently
diff --git a/compiler/main/DynFlags.hs b/compiler/main/DynFlags.hs
index a811698ad9..d222b3b6de 100644
--- a/compiler/main/DynFlags.hs
+++ b/compiler/main/DynFlags.hs
@@ -234,6 +234,7 @@ module DynFlags (
         -- * Linker/compiler information
         LinkerInfo(..),
         CompilerInfo(..),
+        useXLinkerRPath,
 
         -- * File cleanup
         FilesToClean(..), emptyFilesToClean,
@@ -4605,7 +4606,8 @@ defaultFlags settings
       Opt_ProfCountEntries,
       Opt_SharedImplib,
       Opt_SimplPreInlining,
-      Opt_VersionMacros
+      Opt_VersionMacros,
+      Opt_RPath
     ]
 
     ++ [f | (ns,f) <- optLevelFlags, 0 `elem` ns]
@@ -4613,8 +4615,6 @@ defaultFlags settings
 
     ++ default_PIC platform
 
-    ++ default_RPath platform
-
     ++ concatMap (wayGeneralFlags platform) (defaultWays settings)
     ++ validHoleFitDefaults
 
@@ -4670,29 +4670,6 @@ default_PIC platform =
                                           -- information.
     _                        -> []
 
-
--- We usually want to use RPath, except on macOS (OSDarwin).  On recent macOS
--- versions the number of load commands we can embed in a dynamic library is
--- restricted.  Hence since b592bd98ff2 we rely on -dead_strip_dylib to only
--- link the needed dylibs instead of linking the full dependency closure.
---
--- If we split the library linking into injecting -rpath and -l @rpath/...
--- components, we will reduce the number of libraries we link, however we will
--- still inject one -rpath entry for each library, independent of their use.
--- That is, we even inject -rpath values for libraries that we dead_strip in
--- the end. As such we can run afoul of the load command size limit simply
--- by polluting the load commands with RPATH entries.
---
--- Thus, we disable Opt_RPath by default on OSDarwin.  The savvy user can always
--- enable it with -use-rpath if they so wish.
---
--- See Note [Dynamic linking on macOS]
-
-default_RPath :: Platform -> [GeneralFlag]
-default_RPath platform | platformOS platform == OSDarwin = []
-default_RPath _                                          = [Opt_RPath]
-
-
 -- General flags that are switched on/off when other general flags are switched
 -- on
 impliedGFlags :: [(GeneralFlag, TurnOnFlag, GeneralFlag)]
@@ -5924,6 +5901,40 @@ data CompilerInfo
    | UnknownCC
    deriving Eq
 
+
+-- | Should we use `-XLinker -rpath` when linking or not?
+-- See Note [-fno-use-rpaths]
+useXLinkerRPath :: DynFlags -> OS -> Bool
+useXLinkerRPath _ OSDarwin = False -- See Note [Dynamic linking on macOS]
+useXLinkerRPath dflags _ = gopt Opt_RPath dflags
+
+{-
+Note [-fno-use-rpaths]
+~~~~~~~~~~~~~~~~~~~~~~
+
+First read, Note [Dynamic linking on macOS] to understand why on darwin we never
+use `-XLinker -rpath`.
+
+The specification of `Opt_RPath` is as follows:
+
+The default case `-fuse-rpaths`:
+* On darwin, never use `-Xlinker -rpath -Xlinker`, always inject the rpath
+  afterwards, see `runInjectRPaths`. There is no way to use `-Xlinker` on darwin
+  as things stand but it wasn't documented in the user guide before this patch how
+  `-fuse-rpaths` should behave and the fact it was always disabled on darwin.
+* Otherwise, use `-Xlinker -rpath -Xlinker` to set the rpath of the executable,
+  this is the normal way you should set the rpath.
+
+The case of `-fno-use-rpaths`
+* Never inject anything into the rpath.
+
+When this was first implemented, `Opt_RPath` was disabled on darwin, but
+the rpath was still always augmented by `runInjectRPaths`, and there was no way to
+stop this. This was problematic because you couldn't build an executable in CI
+with a clean rpath.
+
+-}
+
 -- -----------------------------------------------------------------------------
 -- RTS hooks
 
diff --git a/compiler/main/SysTools.hs b/compiler/main/SysTools.hs
index bed0251a47..d2b7ba3560 100644
--- a/compiler/main/SysTools.hs
+++ b/compiler/main/SysTools.hs
@@ -248,6 +248,8 @@ linkDynLib dflags0 o_files dep_packages
 
     pkgs <- getPreloadPackagesAnd dflags dep_packages
 
+    let platform = targetPlatform dflags
+        os = platformOS platform
     let pkg_lib_paths = collectLibraryPaths dflags pkgs
     let pkg_lib_path_opts = concatMap get_pkg_lib_path_opts pkg_lib_paths
         get_pkg_lib_path_opts l
@@ -257,7 +259,7 @@ linkDynLib dflags0 o_files dep_packages
            -- Only if we want dynamic libraries
            WayDyn `elem` ways dflags &&
            -- Only use RPath if we explicitly asked for it
-           gopt Opt_RPath dflags
+           useXLinkerRPath dflags os
             = ["-L" ++ l, "-Xlinker", "-rpath", "-Xlinker", l]
               -- See Note [-Xlinker -rpath vs -Wl,-rpath]
          | otherwise = ["-L" ++ l]
@@ -272,9 +274,7 @@ linkDynLib dflags0 o_files dep_packages
     -- not allow undefined symbols.
     -- The RTS library path is still added to the library search path
     -- above in case the RTS is being explicitly linked in (see #3807).
-    let platform = targetPlatform dflags
-        os = platformOS platform
-        pkgs_no_rts = case os of
+    let pkgs_no_rts = case os of
                       OSMinGW32 ->
                           pkgs
                       _ ->
diff --git a/compiler/main/SysTools/Tasks.hs b/compiler/main/SysTools/Tasks.hs
index 9e46327cda..17105ba7a7 100644
--- a/compiler/main/SysTools/Tasks.hs
+++ b/compiler/main/SysTools/Tasks.hs
@@ -261,6 +261,7 @@ figureLlvmVersion dflags = traceToolCommand dflags "llc" $ do
 --
 -- See Note [Dynamic linking on macOS]
 runInjectRPaths :: DynFlags -> [FilePath] -> FilePath -> IO ()
+runInjectRPaths dflags _ _ | not (gopt Opt_RPath dflags) = return ()
 runInjectRPaths dflags lib_paths dylib = do
   info <- lines <$> askOtool dflags Nothing [Option "-L", Option dylib]
   -- filter the output for only the libraries. And then drop the @rpath prefix.
diff --git a/docs/users_guide/expected-undocumented-flags.txt b/docs/users_guide/expected-undocumented-flags.txt
index 837c2a59e2..e9d1673304 100644
--- a/docs/users_guide/expected-undocumented-flags.txt
+++ b/docs/users_guide/expected-undocumented-flags.txt
@@ -124,7 +124,6 @@
 -fstg-lift-lams-rec-args-any
 -fth
 -ftype-function-depth
--fuse-rpaths
 -fversion-macros
 -fvia-c
 -haddock
diff --git a/docs/users_guide/phases.rst b/docs/users_guide/phases.rst
index 8dc55a6873..841e92c058 100644
--- a/docs/users_guide/phases.rst
+++ b/docs/users_guide/phases.rst
@@ -788,6 +788,18 @@ for example).
     Where to find user-supplied libraries… Prepend the directory ⟨dir⟩
     to the library directories path.
 
+.. ghc-flag:: -fuse-rpaths
+    :shortdesc: Set the rpath based on -L flags
+    :type: dynamic
+    :category: linking
+
+    This flag is enabled by default and will set the rpath of the linked
+    object to the library directories of dependent packages.
+
+    When building binaries to distribute it can be useful to pass your own
+    linker options to control the rpath and disable the automatic injection of
+    rpath entries by disabling this flag.
+
 .. ghc-flag:: -framework-path ⟨dir⟩
     :shortdesc: On Darwin/OS X/iOS only, add ⟨dir⟩ to the list of directories
         searched for frameworks. This option corresponds to the ``-F``
@@ -866,6 +878,23 @@ for example).
     libraries at runtime. See :ref:`finding-shared-libs` for a
     description of each mode.
 
+.. ghc-flag:: -flink-rts
+    :shortdesc: Link the runtime when generating a shared or static library
+    :type: dynamic
+    :category: linking
+
+    When linking shared libraries (:ghc-flag:`-shared`) GHC does not
+    automatically link the RTS.  This is to allow choosing the RTS flavour
+    (:ghc-flag:`-threaded`, :ghc-flag:`-eventlog`, etc) when linking an
+    executable.
+    However when the shared library is the intended product it is useful to be
+    able to reverse this default. See :ref:`shared-libraries-c-api` for an
+    usage example.
+
+    When linking a static library (:ghc-flag:`-staticlib`) GHC links the RTS
+    automatically, you can reverse this behaviour by reversing this flag:
+    ``-fno-link-rts``.
+
 .. ghc-flag:: -main-is ⟨thing⟩
     :shortdesc: Set main module and function
     :type: dynamic