[macOS] improved runpath handling

In b592bd98ff25730bbe3c13d6f62a427df8c78e28 we started using
-dead_strip_dylib on macOS when lining dynamic libraries and binaries.
The underlying reason being the Load Command Size Limit in macOS
Sierra (10.14) and later.

GHC will produce @rpath/libHS... dependency entries together with a
corresponding RPATH entry pointing to the location of the libHS...
library. Thus for every library we produce two Load Commands.  One to
specify the dependent library, and one with the path where to find it.
This makes relocating libraries and binaries easier, as we just need to
update the RPATH entry with the install_name_tool. The dynamic linker
will then subsitute each @rpath with the RPATH entries it finds in the
libraries load commands or the environement, when looking up @rpath
relative libraries.

-dead_strip_dylibs intructs the linker to drop unused libraries. This in
turn help us reduce the number of referenced libraries, and subsequently
the size of the load commands.  This however does not remove the RPATH
entries.  Subsequently we can end up (in extreme cases) with only a
single @rpath/libHS... entry, but 100s or more RPATH entries in the Load
Commands.

This patch rectifies this (slighly unorthodox) by passing *no* -rpath
arguments to the linker at link time, but -headerpad 8000.  The
headerpad argument is in hexadecimal and the maxium 32k of the load
command size.  This tells the linker to pad the load command section
enough for us to inject the RPATHs later.  We then proceed to link the
library or binary with -dead_strip_dylibs, and *after* the linking
inspect the library to find the left over (non-dead-stripped)
dependencies (using otool).  We find the corresponding RPATHs for each
@rpath relative dependency, and inject them into the library or binary
using the install_name_tool.  Thus achieving a deadstripped dylib (and
rpaths) build product.

We can not do this in GHC, without starting to reimplement a dynamic
linker as we do not know which symbols and subsequently libraries are
necessary.

Commissioned-by: Mercury Technologies, Inc. (mercury.com)

1 files changed, 12 insertions, 2 deletions

diff --git a/compiler/GHC/SysTools.hs b/compiler/GHC/SysTools.hs
index 57d1757fe9..938f8de110 100644
--- a/compiler/GHC/SysTools.hs
+++ b/compiler/GHC/SysTools.hs
@@ -261,7 +261,10 @@ linkDynLib dflags0 o_files dep_packages
          | ( osElfTarget (platformOS (targetPlatform dflags)) ||
              osMachOTarget (platformOS (targetPlatform dflags)) ) &&
            dynLibLoader dflags == SystemDependent &&
-           WayDyn `Set.member` ways dflags
+           -- Only if we want dynamic libraries
+           WayDyn `Set.member` ways dflags &&
+           -- Only use RPath if we explicitly asked for it
+           gopt Opt_RPath dflags
             = ["-L" ++ l, "-Xlinker", "-rpath", "-Xlinker", l]
               -- See Note [-Xlinker -rpath vs -Wl,-rpath]
          | otherwise = ["-L" ++ l]
@@ -386,8 +389,15 @@ linkDynLib dflags0 o_files dep_packages
                  ++ map Option pkg_lib_path_opts
                  ++ map Option pkg_link_opts
                  ++ map Option pkg_framework_opts
-                 ++ [ Option "-Wl,-dead_strip_dylibs" ]
+                 -- dead_strip_dylibs, will remove unused dylibs, and thus save
+                 -- space in the load commands. The -headerpad is necessary so
+                 -- that we can inject more @rpath's later for the leftover
+                 -- libraries in the runInjectRpaths phase below.
+                 --
+                 -- See Note [Dynamic linking on macOS]
+                 ++ [ Option "-Wl,-dead_strip_dylibs", Option "-Wl,-headerpad,8000" ]
               )
+            runInjectRPaths dflags pkg_lib_paths output_fn
         _ -> do
             -------------------------------------------------------------------
             -- Making a DSO