Overhaul of infrastructure for profiling, coverage (HPC) and breakpoints

User visible changes
====================

Profilng
--------

Flags renamed (the old ones are still accepted for now):

  OLD            NEW
  ---------      ------------
  -auto-all      -fprof-auto
  -auto          -fprof-exported
  -caf-all       -fprof-cafs

New flags:

  -fprof-auto              Annotates all bindings (not just top-level
                           ones) with SCCs

  -fprof-top               Annotates just top-level bindings with SCCs

  -fprof-exported          Annotates just exported bindings with SCCs

  -fprof-no-count-entries  Do not maintain entry counts when profiling
                           (can make profiled code go faster; useful with
                           heap profiling where entry counts are not used)

Cost-centre stacks have a new semantics, which should in most cases
result in more useful and intuitive profiles.  If you find this not to
be the case, please let me know.  This is the area where I have been
experimenting most, and the current solution is probably not the
final version, however it does address all the outstanding bugs and
seems to be better than GHC 7.2.

Stack traces
------------

+RTS -xc now gives more information.  If the exception originates from
a CAF (as is common, because GHC tends to lift exceptions out to the
top-level), then the RTS walks up the stack and reports the stack in
the enclosing update frame(s).

Result: +RTS -xc is much more useful now - but you still have to
compile for profiling to get it.  I've played around a little with
adding 'head []' to GHC itself, and +RTS -xc does pinpoint the problem
quite accurately.

I plan to add more facilities for stack tracing (e.g. in GHCi) in the
future.

Coverage (HPC)
--------------

 * derived instances are now coloured yellow if they weren't used
 * likewise record field names
 * entry counts are more accurate (hpc --fun-entry-count)
 * tab width is now correct (markup was previously off in source with
   tabs)

Internal changes
================

In Core, the Note constructor has been replaced by

        Tick (Tickish b) (Expr b)

which is used to represent all the kinds of source annotation we
support: profiling SCCs, HPC ticks, and GHCi breakpoints.

Depending on the properties of the Tickish, different transformations
apply to Tick.  See CoreUtils.mkTick for details.

Tickets
=======

This commit closes the following tickets, test cases to follow:

  - Close #2552: not a bug, but the behaviour is now more intuitive
    (test is T2552)

  - Close #680 (test is T680)

  - Close #1531 (test is result001)

  - Close #949 (test is T949)

  - Close #2466: test case has bitrotted (doesn't compile against current
    version of vector-space package)

2 files changed, 97 insertions, 94 deletions

diff --git a/includes/rts/prof/CCS.h b/includes/rts/prof/CCS.h
index b210991017..4692d166b5 100644
--- a/includes/rts/prof/CCS.h
+++ b/includes/rts/prof/CCS.h
@@ -21,6 +21,7 @@ int rts_isProfiled(void);
  * Data Structures 
  * ---------------------------------------------------------------------------*/  
 /*
+ * Note [struct alignment]
  * NB. be careful to avoid unwanted padding between fields, by
  * putting the 8-byte fields on an 8-byte boundary.  Padding can
  * vary between C compilers, and we don't take into account any
@@ -29,35 +30,49 @@ int rts_isProfiled(void);
  */
 
 typedef struct _CostCentre {
-  StgInt ccID;
+    StgInt ccID;              // Unique Id, allocated by the RTS
 
-  char * label;
-  char * module;
- 
-  /* used for accumulating costs at the end of the run... */
-  StgWord   time_ticks;
-  StgWord64 mem_alloc;      /* align 8 (see above) */
+    char * label;
+    char * module;
+
+    // used for accumulating costs at the end of the run...
+    StgWord   time_ticks;
+    StgWord64 mem_alloc;      // align 8 (Note [struct alignment])
 
-  StgInt    is_caf;
+    StgInt is_caf;            // non-zero for a CAF cost centre
 
-  struct _CostCentre *link;
+    struct _CostCentre *link;
 } CostCentre;
 
 typedef struct _CostCentreStack {
-  StgInt ccsID;
+    StgInt ccsID;               // unique ID, allocated by the RTS
+
+    CostCentre *cc;             // Cost centre at the top of the stack
+
+    struct _CostCentreStack *prevStack;   // parent
+    struct _IndexTable      *indexTable;  // children
+    struct _CostCentreStack *root;        // root of stack
+    StgWord    depth;           // number of items in the stack
 
-  CostCentre *cc;
-  struct _CostCentreStack *prevStack;
-  struct _IndexTable *indexTable;
+    StgWord64  scc_count;       // Count of times this CCS is entered
+                                // align 8 (Note [struct alignment])
 
-  StgWord64  scc_count;       /* align 8 (see above) */
-  StgWord    selected;
-  StgWord    time_ticks;
-  StgWord64  mem_alloc;       /* align 8 (see above) */
-  StgWord64  inherited_alloc; /* align 8 (see above) */
-  StgWord    inherited_ticks;
+    StgWord    selected;        // is this CCS shown in the heap
+                                // profile? (zero if excluded via -hc
+                                // -hm etc.)
 
-  CostCentre *root;
+    StgWord    time_ticks;      // number of time ticks accumulated by
+                                // this CCS
+
+    StgWord64  mem_alloc;       // mem allocated by this CCS
+                                // align 8 (Note [struct alignment])
+
+    StgWord64  inherited_alloc; // sum of mem_alloc over all children
+                                // (calculated at the end)
+                                // align 8 (Note [struct alignment])
+
+    StgWord    inherited_ticks; // sum of time_ticks over all children
+                                // (calculated at the end)
 } CostCentreStack;
 
 
@@ -74,21 +89,24 @@ typedef struct _CostCentreStack {
 #define EMPTY_STACK NULL
 #define EMPTY_TABLE NULL
 
-/* Constants used to set sumbsumed flag on CostCentres */
-
+/* Constants used to set is_caf flag on CostCentres */
 #define CC_IS_CAF      'c'            /* 'c'  => *is* a CAF cc           */
-#define CC_IS_BORING   'B'            /* 'B'  => *not* a CAF/sub cc      */
-
+#define CC_NOT_CAF     0
 
 /* -----------------------------------------------------------------------------
  * Data Structures
  * ---------------------------------------------------------------------------*/
 
+// IndexTable is the list of children of a CCS. (Alternatively it is a
+// cache of the results of pushing onto a CCS, so that the second and
+// subsequent times we push a certain CC on a CCS we get the same
+// result).
+
 typedef struct _IndexTable {
-  CostCentre *cc;
-  CostCentreStack *ccs;
-  struct _IndexTable *next;
-  unsigned int back_edge;
+    CostCentre *cc;
+    CostCentreStack *ccs;
+    struct _IndexTable *next;
+    unsigned int back_edge;
 } IndexTable;
 
      
@@ -101,48 +119,44 @@ extern CostCentreStack * RTS_VAR(CCCS);	        /* current CCS */
 #if IN_STG_CODE
 
 extern StgWord CC_MAIN[];	
-extern StgWord CCS_MAIN[];      /* Top CCS */
+extern StgWord CCS_MAIN[];      // Top CCS
 
 extern StgWord CC_SYSTEM[];	
-extern StgWord CCS_SYSTEM[];    /* RTS costs */
+extern StgWord CCS_SYSTEM[];    // RTS costs
 
 extern StgWord CC_GC[];
-extern StgWord CCS_GC[];	 /* Garbage collector costs */
-
-extern StgWord CC_SUBSUMED[];	
-extern StgWord CCS_SUBSUMED[];   /* Costs are subsumed by caller */
+extern StgWord CCS_GC[];         // Garbage collector costs
 
 extern StgWord CC_OVERHEAD[];
-extern StgWord CCS_OVERHEAD[];   /* Profiling overhead */
+extern StgWord CCS_OVERHEAD[];   // Profiling overhead
 
 extern StgWord CC_DONT_CARE[];
-extern StgWord CCS_DONT_CARE[];  /* shouldn't ever get set */
+extern StgWord CCS_DONT_CARE[];  // CCS attached to static constructors
 
 #else
 
 extern CostCentre      CC_MAIN[];	
-extern CostCentreStack CCS_MAIN[];      /* Top CCS */
+extern CostCentreStack CCS_MAIN[];      // Top CCS
 
 extern CostCentre      CC_SYSTEM[];	
-extern CostCentreStack CCS_SYSTEM[];    /* RTS costs */
+extern CostCentreStack CCS_SYSTEM[];    // RTS costs
 
 extern CostCentre      CC_GC[];
-extern CostCentreStack CCS_GC[];	 /* Garbage collector costs */
-
-extern CostCentre      CC_SUBSUMED[];	
-extern CostCentreStack CCS_SUBSUMED[];   /* Costs are subsumed by caller */
+extern CostCentreStack CCS_GC[];         // Garbage collector costs
 
 extern CostCentre      CC_OVERHEAD[];
-extern CostCentreStack CCS_OVERHEAD[];   /* Profiling overhead */
+extern CostCentreStack CCS_OVERHEAD[];   // Profiling overhead
 
 extern CostCentre      CC_DONT_CARE[];
-extern CostCentreStack CCS_DONT_CARE[];  /* shouldn't ever get set */
+extern CostCentreStack CCS_DONT_CARE[];  // shouldn't ever get set
+
+extern CostCentre      CC_PINNED[];
+extern CostCentreStack CCS_PINNED[];     // pinned memory
 
 #endif /* IN_STG_CODE */
 
-extern unsigned int RTS_VAR(CC_ID);	/* global ids */
+extern unsigned int RTS_VAR(CC_ID);     // global ids
 extern unsigned int RTS_VAR(CCS_ID);
-extern unsigned int RTS_VAR(HP_ID);
 
 extern unsigned int RTS_VAR(era);
 
@@ -150,37 +164,23 @@ extern unsigned int RTS_VAR(era);
  * Functions 
  * ---------------------------------------------------------------------------*/
 
-void EnterFunCCS ( CostCentreStack *ccsfn );
-CostCentreStack *PushCostCentre ( CostCentreStack *, CostCentre * );
-CostCentreStack *AppendCCS ( CostCentreStack *ccs1, CostCentreStack *ccs2 );
-
-extern unsigned int RTS_VAR(entering_PAP);
+CostCentreStack * pushCostCentre (CostCentreStack *, CostCentre *);
+void              enterFunCCS    (CostCentreStack *);
 
 /* -----------------------------------------------------------------------------
- * Registering CCs
- 
- Cost centres are registered at startup by calling a registering
- routine in each module. Each module registers its cost centres and
- calls the registering routine for all imported modules. The RTS calls
- the registering routine for the module Main. This registering must be
- done before initialisation since the evaluation required for
- initialisation may use the cost centres.
- 
- As the code for each module uses tail calls we use an auxiliary stack
- (in the heap) to record imported modules still to be registered. At
- the bottom of the stack is NULL which indicates that
- @miniInterpretEnd@ should be resumed.
+   Registering CCs and CCSs
  
- @START_REGISTER@ and @END_REGISTER@ are special macros used to
- delimit the function. @END_REGISTER@ pops the next registering
- routine off the stack and jumps to it. @REGISTER_CC@ registers a cost
- centre. @REGISTER_IMPORT@ pushes a modules registering routine onto
- the register stack.
+   Registering a CC or CCS consists of
+     - assigning it a unique ID
+     - linking it onto the list of registered CCs/CCSs
 
+   Cost centres are registered at startup by a C constructor function
+   generated by the compiler in the _stub.c file for each module.  The
+   macros below are invoked by that C code to register CCs and CCSs.
  -------------------------------------------------------------------------- */
 
-extern CostCentre * RTS_VAR(CC_LIST);               /* registered CC list */
-extern CostCentreStack * RTS_VAR(CCS_LIST);         /* registered CCS list */
+extern CostCentre * RTS_VAR(CC_LIST);               // registered CC list
+extern CostCentreStack * RTS_VAR(CCS_LIST);         // registered CCS list
 
 #define REGISTER_CC(cc)					\
 	do {						\
@@ -202,29 +202,31 @@ extern CostCentreStack * RTS_VAR(CCS_LIST);         /* registered CCS list */
  * Declaring Cost Centres & Cost Centre Stacks.
  * -------------------------------------------------------------------------- */
 
-# define CC_DECLARE(cc_ident,name,module,caf,is_local)		\
-     is_local CostCentre cc_ident[1]				\
-	= {{ 0,							\
-	     name,						\
-	     module,						\
-             0,							\
-	     0,							\
-	     caf,						\
-	     0 }};
-
-# define CCS_DECLARE(ccs_ident,cc_ident,is_local)		\
-     is_local CostCentreStack ccs_ident[1]			\
-       = {{ ccsID 		: 0,				\
-	    cc 			: cc_ident,			\
-	    prevStack 		: NULL,				\
-	    indexTable 		: NULL,				\
-            selected            : 0,				\
-	    scc_count 		: 0,				\
-	    time_ticks 		: 0,				\
-	    mem_alloc 		: 0,				\
-	    inherited_ticks 	: 0,				\
-	    inherited_alloc	: 0,				\
-	    root 		: 0,				\
+# define CC_DECLARE(cc_ident,name,mod,caf,is_local)     \
+     is_local CostCentre cc_ident[1]                    \
+       = {{ ccID       : 0,                             \
+            label      : name,                          \
+            module     : mod,                           \
+            time_ticks : 0,                             \
+            mem_alloc  : 0,                             \
+            link       : 0,                             \
+            is_caf     : caf                            \
+         }};
+
+# define CCS_DECLARE(ccs_ident,cc_ident,is_local)       \
+     is_local CostCentreStack ccs_ident[1]              \
+       = {{ ccsID 		: 0,                    \
+	    cc 			: cc_ident,             \
+	    prevStack 		: NULL,                 \
+	    indexTable 		: NULL,                 \
+            root                : NULL,                 \
+            depth               : 0,                    \
+            selected            : 0,                    \
+	    scc_count 		: 0,                    \
+	    time_ticks 		: 0,                    \
+	    mem_alloc 		: 0,                    \
+	    inherited_ticks 	: 0,                    \
+            inherited_alloc     : 0                     \
        }};
 
 /* -----------------------------------------------------------------------------
diff --git a/includes/stg/MiscClosures.h b/includes/stg/MiscClosures.h
index ed0bf655e1..45dc8369c7 100644
--- a/includes/stg/MiscClosures.h
+++ b/includes/stg/MiscClosures.h
@@ -62,6 +62,7 @@ RTS_RET(stg_unmaskAsyncExceptionszh_ret);
 RTS_RET(stg_maskUninterruptiblezh_ret);
 RTS_RET(stg_maskAsyncExceptionszh_ret);
 RTS_RET(stg_stack_underflow_frame);
+RTS_RET(stg_restore_cccs);
 
 // RTS_FUN(stg_interp_constr_entry);
 //