rts/posix: Use less aggressive backoff schedule for heap reservation sizing

When we allocate the heap on POSIX platforms we generally just ask for a
1TB chunk of address space and call it a day. However, if the user has
set a ulimit then this request will fail. In this case we would
previously try successively smaller allocation requests, reducing the
request size by a factor of two each time.

However, this means that GHC will significantly allocate a significantly
smaller heap than the available physical memory size in some
circumstances.  Imagine, for instance, a machine with 512 GB of physical
memory but a ulimit of 511 GB: we would be limited to a 256 GB heap.

We now use a less aggressive back-off policy, reducing by one-eighth the
last allocation size each try.

Thanks to luispedro for the suggested approach.

Test Plan: Validate

Reviewers: simonmar, erikd

Subscribers: rwbarton, thomie

GHC Trac Issues: #14492

Differential Revision: https://phabricator.haskell.org/D4215

1 files changed, 8 insertions, 3 deletions

diff --git a/rts/posix/OSMem.c b/rts/posix/OSMem.c
index 1df18abf6e..9ae9a4bd36 100644
--- a/rts/posix/OSMem.c
+++ b/rts/posix/OSMem.c
@@ -514,9 +514,14 @@ void *osReserveHeapMemory(void *startAddressPtr, W_ *len)
         if (at == NULL) {
             // This means that mmap failed which we take to mean that we asked
             // for too much memory. This can happen due to POSIX resource
-            // limits. In this case we reduce our allocation request by a factor
-            // of two and try again.
-            *len /= 2;
+            // limits. In this case we reduce our allocation request by a
+            // fraction of the current size and try again.
+            //
+            // Note that the previously would instead decrease the request size
+            // by a factor of two; however, this meant that significant amounts
+            // of memory will be wasted (e.g. imagine a machine with 512GB of
+            // physical memory but a 511GB ulimit). See #14492.
+            *len -= *len / 8;
         } else if ((W_)at >= minimumAddress) {
             // Success! We were given a block of memory starting above the 8 GB
             // mark, which is what we were looking for.