Fix segmentation fault when .prof file not writeable

There are two ways to do retainer profiling. Quoting from the user's guide:
  1. `+RTS -hr` "Breaks down the graph by retainer set"
  2. `+RTS -hr<cc> -h<x>`, where `-h<x>` is one of normal heap profiling
     break-down options (e.g. `-hc`), and `-hr<cc> means "Restrict the
     profile to closures with retainer sets containing cost-centre
     stacks with one of the specified cost centres at the top."

Retainer profiling writes to a .hp file, like the other heap profiling
options, but also to a .prof file. Therefore, when the .prof file is not
writeable for whatever reason, retainer profiling should be turned off
completely.

This worked ok when running the program with `+RTS -hr` (option 1), but a
segfault would occur when using `+RTS -hr<cc> -h<x>`, with `x!=r` (option 2).

This commit fixes that.

Reviewed by: bgamari

Differential Revision: https://phabricator.haskell.org/D1849

GHC Trac Issues: #11489

1 files changed, 5 insertions, 8 deletions

diff --git a/rts/Profiling.c b/rts/Profiling.c
index 2c2981a02f..c67b0817df 100644
--- a/rts/Profiling.c
+++ b/rts/Profiling.c
@@ -254,9 +254,7 @@ initProfilingLogFile(void)
     }
 #endif
 
-    if (RtsFlags.CcFlags.doCostCentres == 0 &&
-        RtsFlags.ProfFlags.doHeapProfile != HEAP_BY_RETAINER &&
-        RtsFlags.ProfFlags.retainerSelector == NULL)
+    if (RtsFlags.CcFlags.doCostCentres == 0 && !doingRetainerProfiling())
     {
         /* No need for the <prog>.prof file */
         prof_filename = NULL;
@@ -272,11 +270,11 @@ initProfilingLogFile(void)
         if ((prof_file = fopen(prof_filename, "w")) == NULL) {
             debugBelch("Can't open profiling report file %s\n", prof_filename);
             RtsFlags.CcFlags.doCostCentres = 0;
-            // The following line was added by Sung; retainer/LDV profiling may need
-            // two output files, i.e., <program>.prof/hp.
-            if (RtsFlags.ProfFlags.doHeapProfile == HEAP_BY_RETAINER)
+            // Retainer profiling (`-hr` or `-hr<cc> -h<x>`) writes to
+            // both <program>.hp as <program>.prof.
+            if (doingRetainerProfiling()) {
                 RtsFlags.ProfFlags.doHeapProfile = 0;
-            return;
+            }
         }
     }
 
@@ -290,7 +288,6 @@ initProfilingLogFile(void)
             debugBelch("Can't open profiling report file %s\n",
                     hp_filename);
             RtsFlags.ProfFlags.doHeapProfile = 0;
-            return;
         }
     }
 }