path: root/docs/users_guide/profiling.xml

<?xml version="1.0" encoding="iso-8859-1"?>
<chapter id="profiling">
  <title>Profiling</title>
  <indexterm><primary>profiling</primary>
  </indexterm>
  <indexterm><primary>cost-centre profiling</primary></indexterm>

  <para>GHC comes with a time and space profiling system, so that you
  can answer questions like "why is my program so slow?", or "why is
  my program using so much memory?".</para>

  <para>Profiling a program is a three-step process:</para>

  <orderedlist>
    <listitem>
      <para>Re-compile your program for profiling with the
      <literal>-prof</literal> option, and probably one of the options
      for adding automatic annotations:
      <literal>-fprof-auto</literal> is the most common<footnote><para><option>-fprof-auto</option> was known as <option>-auto-all</option><indexterm><primary><literal>-auto-all</literal></primary>
      </indexterm> prior to GHC 7.4.1.</para></footnote>.
      <indexterm><primary><literal>-fprof-auto</literal></primary>
      </indexterm></para>

      <para>If you are using external packages with
      <literal>cabal</literal>, you may need to reinstall these
      packages with profiling support; typically this is done with
      <literal>cabal install -p <replaceable>package</replaceable>
      --reinstall</literal>.</para>
    </listitem>

    <listitem>
      <para>Having compiled the program for profiling, you now need to
      run it to generate the profile.  For example, a simple time
      profile can be generated by running the program with
      <option>+RTS
      -p</option><indexterm><primary><option>-p</option></primary><secondary>RTS
      option</secondary></indexterm>, which generates a file named
      <literal><replaceable>prog</replaceable>.prof</literal> where
      <replaceable>prog</replaceable> is the name of your program
      (without the <literal>.exe</literal> extension, if you are on
      Windows).</para>

      <para>There are many different kinds of profile that can be
      generated, selected by different RTS options.  We will be
      describing the various kinds of profile throughout the rest of
      this chapter.  Some profiles require further processing using
      additional tools after running the program.</para>
    </listitem>

    <listitem>
      <para>Examine the generated profiling information, use the
      information to optimise your program, and repeat as
      necessary.</para>
    </listitem>

  </orderedlist>

  <sect1 id="cost-centres">
    <title>Cost centres and cost-centre stacks</title>

    <para>GHC's profiling system assigns <firstterm>costs</firstterm>
    to <firstterm>cost centres</firstterm>.  A cost is simply the time
    or space (memory) required to evaluate an expression.  Cost centres are
    program annotations around expressions; all costs incurred by the
    annotated expression are assigned to the enclosing cost centre.
    Furthermore, GHC will remember the stack of enclosing cost centres
    for any given expression at run-time and generate a call-tree of
    cost attributions.</para>

    <para>Let's take a look at an example:</para>

    <programlisting>
main = print (fib 30)
fib n = if n &lt; 2 then 1 else fib (n-1) + fib (n-2)
</programlisting>

    <para>Compile and run this program as follows:</para>

    <screen>
$ ghc -prof -fprof-auto -rtsopts Main.hs
$ ./Main +RTS -p
121393
$
</screen>

    <para>When a GHC-compiled program is run with the
    <option>-p</option> RTS option, it generates a file called
    <filename><replaceable>prog</replaceable>.prof</filename>.  In this case, the file
    will contain something like this:</para>

<screen>
        Wed Oct 12 16:14 2011 Time and Allocation Profiling Report  (Final)

           Main +RTS -p -RTS

        total time  =        0.68 secs   (34 ticks @ 20 ms)
        total alloc = 204,677,844 bytes  (excludes profiling overheads)

COST CENTRE MODULE  %time %alloc

fib         Main    100.0  100.0


                                                      individual     inherited
COST CENTRE MODULE                  no.     entries  %time %alloc   %time %alloc

MAIN        MAIN                    102           0    0.0    0.0   100.0  100.0
 CAF        GHC.IO.Handle.FD        128           0    0.0    0.0     0.0    0.0
 CAF        GHC.IO.Encoding.Iconv   120           0    0.0    0.0     0.0    0.0
 CAF        GHC.Conc.Signal         110           0    0.0    0.0     0.0    0.0
 CAF        Main                    108           0    0.0    0.0   100.0  100.0
  main      Main                    204           1    0.0    0.0   100.0  100.0
   fib      Main                    205     2692537  100.0  100.0   100.0  100.0
</screen>

    <para>The first part of the file gives the program name and
    options, and the total time and total memory allocation measured
    during the run of the program (note that the total memory
    allocation figure isn't the same as the amount of
    <emphasis>live</emphasis> memory needed by the program at any one
    time; the latter can be determined using heap profiling, which we
    will describe later in <xref linkend="prof-heap" />).</para>

    <para>The second part of the file is a break-down by cost centre
    of the most costly functions in the program.  In this case, there
    was only one significant function in the program, namely
    <function>fib</function>, and it was responsible for 100&percnt;
    of both the time and allocation costs of the program.</para>

    <para>The third and final section of the file gives a profile
    break-down by cost-centre stack.  This is roughly a call-tree
    profile of the program.  In the example above, it is clear that
    the costly call to <function>fib</function> came from
    <function>main</function>.</para>

    <para>The time and allocation incurred by a given part of the
    program is displayed in two ways: &ldquo;individual&rdquo;, which
    are the costs incurred by the code covered by this cost centre
    stack alone, and &ldquo;inherited&rdquo;, which includes the costs
    incurred by all the children of this node.</para>

    <para>The usefulness of cost-centre stacks is better demonstrated
    by  modifying the example slightly:</para>

    <programlisting>
main = print (f 30 + g 30)
  where
    f n  = fib n
    g n  = fib (n `div` 2)

fib n = if n &lt; 2 then 1 else fib (n-1) + fib (n-2)
</programlisting>

    <para>Compile and run this program as before, and take a look at
    the new profiling results:</para>

<screen>
COST CENTRE MODULE                  no.     entries  %time %alloc   %time %alloc

MAIN        MAIN                    102           0    0.0    0.0   100.0  100.0
 CAF        GHC.IO.Handle.FD        128           0    0.0    0.0     0.0    0.0
 CAF        GHC.IO.Encoding.Iconv   120           0    0.0    0.0     0.0    0.0
 CAF        GHC.Conc.Signal         110           0    0.0    0.0     0.0    0.0
 CAF        Main                    108           0    0.0    0.0   100.0  100.0
  main      Main                    204           1    0.0    0.0   100.0  100.0
   main.g   Main                    207           1    0.0    0.0     0.0    0.1
    fib     Main                    208        1973    0.0    0.1     0.0    0.1
   main.f   Main                    205           1    0.0    0.0   100.0   99.9
    fib     Main                    206     2692537  100.0   99.9   100.0   99.9
</screen>

    <para>Now although we had two calls to <function>fib</function> in
    the program, it is immediately clear that it was the call from
    <function>f</function> which took all the time.  The functions
    <literal>f</literal> and <literal>g</literal> which are defined in
    the <literal>where</literal> clause in <literal>main</literal> are
    given their own cost centres, <literal>main.f</literal> and
    <literal>main.g</literal> respectively.</para>

    <para>The actual meaning of the various columns in the output is:</para>

    <variablelist>
      <varlistentry>
	<term>entries</term>
	<listitem>
	  <para>The number of times this particular point in the call
	  tree was entered.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>individual &percnt;time</term>
	<listitem>
	  <para>The percentage of the total run time of the program
	  spent at this point in the call tree.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>individual &percnt;alloc</term>
	<listitem>
	  <para>The percentage of the total memory allocations
	  (excluding profiling overheads) of the program made by this
	  call.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>inherited &percnt;time</term>
	<listitem>
	  <para>The percentage of the total run time of the program
	  spent below this point in the call tree.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>inherited &percnt;alloc</term>
	<listitem>
	  <para>The percentage of the total memory allocations
	  (excluding profiling overheads) of the program made by this
	  call and all of its sub-calls.</para>
	</listitem>
      </varlistentry>
    </variablelist>

    <para>In addition you can use the <option>-P</option> RTS option
    <indexterm><primary><option>-P</option></primary></indexterm> to
    get the following additional information:</para>

    <variablelist>
      <varlistentry>
	<term><literal>ticks</literal></term>
	<listitem>
	  <para>The raw number of time &ldquo;ticks&rdquo; which were
          attributed to this cost-centre; from this, we get the
          <literal>&percnt;time</literal> figure mentioned
          above.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><literal>bytes</literal></term>
	<listitem>
	  <para>Number of bytes allocated in the heap while in this
          cost-centre; again, this is the raw number from which we get
          the <literal>&percnt;alloc</literal> figure mentioned
          above.</para>
	</listitem>
      </varlistentry>
    </variablelist>

    <para>What about recursive functions, and mutually recursive
    groups of functions?  Where are the costs attributed?  Well,
    although GHC does keep information about which groups of functions
    called each other recursively, this information isn't displayed in
    the basic time and allocation profile, instead the call-graph is
    flattened into a tree as follows: a call to a function that occurs
    elsewhere on the current stack does not push another entry on the
    stack, instead the costs for this call are aggregated into the
    caller<footnote><para>Note that this policy has changed slightly
    in GHC 7.4.1 relative to earlier versions, and may yet change
    further, feedback is welcome.</para></footnote>.</para>

    <sect2 id="scc-pragma"><title>Inserting cost centres by hand</title>

      <para>Cost centres are just program annotations.  When you say
      <option>-fprof-auto</option> to the compiler, it automatically
      inserts a cost centre annotation around every binding not marked
      INLINE in your program, but you are entirely free to add cost
      centre annotations yourself.</para>

      <para>The syntax of a cost centre annotation is</para>

<programlisting>
     {-# SCC "name" #-} &lt;expression&gt;
</programlisting>

      <para>where <literal>"name"</literal> is an arbitrary string,
      that will become the name of your cost centre as it appears
      in the profiling output, and
      <literal>&lt;expression&gt;</literal> is any Haskell
      expression.  An <literal>SCC</literal> annotation extends as
      far to the right as possible when parsing. (SCC stands for "Set
      Cost Centre").  The double quotes can be omitted
      if <literal>name</literal> is a Haskell identifier, for example:</para>

<programlisting>
     {-# SCC my_function #-} &lt;expression&gt;
</programlisting>

      <para>Here is an example of a program with a couple of SCCs:</para>

<programlisting>
main :: IO ()
main = do let xs = [1..1000000]
          let ys = [1..2000000]
          print $ {-# SCC last_xs #-} last xs
          print $ {-# SCC last_init_xs #-} last $ init xs
          print $ {-# SCC last_ys #-} last ys
          print $ {-# SCC last_init_ys #-}last $ init ys
</programlisting>

      <para>which gives this profile when run:</para>

<screen>
COST CENTRE     MODULE                  no.     entries  %time %alloc   %time %alloc

MAIN            MAIN                    102           0    0.0    0.0   100.0  100.0
 CAF            GHC.IO.Handle.FD        130           0    0.0    0.0     0.0    0.0
 CAF            GHC.IO.Encoding.Iconv   122           0    0.0    0.0     0.0    0.0
 CAF            GHC.Conc.Signal         111           0    0.0    0.0     0.0    0.0
 CAF            Main                    108           0    0.0    0.0   100.0  100.0
  main          Main                    204           1    0.0    0.0   100.0  100.0
   last_init_ys Main                    210           1   25.0   27.4    25.0   27.4
   main.ys      Main                    209           1   25.0   39.2    25.0   39.2
   last_ys      Main                    208           1   12.5    0.0    12.5    0.0
   last_init_xs Main                    207           1   12.5   13.7    12.5   13.7
   main.xs      Main                    206           1   18.8   19.6    18.8   19.6
   last_xs      Main                    205           1    6.2    0.0     6.2    0.0
</screen>

    </sect2>

    <sect2 id="prof-rules">
      <title>Rules for attributing costs</title>

      <para>While running a program with profiling turned on, GHC
      maintains a cost-centre stack behind the scenes, and attributes
      any costs (memory allocation and time) to whatever the current
      cost-centre stack is at the time the cost is incurred.</para>

      <para>The mechanism is simple: whenever the program evaluates an
      expression with an SCC annotation, <literal>{-# SCC c -#}
      E</literal>, the cost centre <literal>c</literal> is pushed on
      the current stack, and the entry count for this stack is
      incremented by one.  The stack also sometimes has to be saved
      and restored; in particular when the program creates a
      <firstterm>thunk</firstterm> (a lazy suspension), the current
      cost-centre stack is stored in the thunk, and restored when the
      thunk is evaluated.  In this way, the cost-centre stack is
      independent of the actual evaluation order used by GHC at
      runtime.</para>

      <para>At a function call, GHC takes the stack stored in the
      function being called (which for a top-level function will be
      empty), and <emphasis>appends</emphasis> it to the current
      stack, ignoring any prefix that is identical to a prefix of the
      current stack.</para>

      <para>We mentioned earlier that lazy computations, i.e. thunks,
      capture the current stack when they are created, and restore
      this stack when they are evaluated.  What about top-level
      thunks?  They are "created" when the program is compiled, so
      what stack should we give them?  The technical name for a
      top-level thunk is a CAF ("Constant Applicative Form").  GHC
      assigns every CAF in a module a stack consisting of the single
      cost centre <literal>M.CAF</literal>, where <literal>M</literal>
      is the name of the module.  It is also possible to give each CAF
      a different stack, using the option
      <option>-fprof-cafs</option><indexterm><primary><option>-fprof-cafs</option></primary></indexterm>.
      This is especially useful when compiling with
      <option>-ffull-laziness</option> (as is default with
      <option>-O</option> and higher), as constants in function bodies
      will be lifted to the top-level and become CAFs.  You will probably
      need to consult the Core (<option>-ddump-simpl</option>) in order
      to determine what these CAFs correspond to.</para>
    </sect2>
  </sect1>

  <sect1 id="prof-compiler-options">
    <title>Compiler options for profiling</title>

    <indexterm><primary>profiling</primary><secondary>options</secondary></indexterm>
    <indexterm><primary>options</primary><secondary>for profiling</secondary></indexterm>

    <variablelist>
      <varlistentry>
	<term>
          <option>-prof</option>:
          <indexterm><primary><option>-prof</option></primary></indexterm>
        </term>
	<listitem>
	  <para>To make use of the profiling system
          <emphasis>all</emphasis> modules must be compiled and linked
          with the <option>-prof</option> option. Any
          <literal>SCC</literal> annotations you've put in your source
          will spring to life.</para>

	  <para>Without a <option>-prof</option> option, your
          <literal>SCC</literal>s are ignored; so you can compile
          <literal>SCC</literal>-laden code without changing
          it.</para>
	</listitem>
      </varlistentry>
    </variablelist>

    <para>There are a few other profiling-related compilation options.
    Use them <emphasis>in addition to</emphasis>
    <option>-prof</option>.  These do not have to be used consistently
    for all modules in a program.</para>

    <variablelist>
      <varlistentry>
	<term>
          <option>-fprof-auto</option>:
          <indexterm><primary><option>-fprof-auto</option></primary></indexterm>
        </term>
	<listitem>
          <para><emphasis>All</emphasis> bindings not marked INLINE,
          whether exported or not, top level or nested, will be given
          automatic <literal>SCC</literal> annotations.  Functions
          marked INLINE must be given a cost centre manually.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fprof-auto-top</option>:
          <indexterm><primary><option>-fprof-auto-top</option></primary></indexterm>
          <indexterm><primary>cost centres</primary><secondary>automatically inserting</secondary></indexterm>
        </term>
	<listitem>
	  <para>GHC will automatically add <literal>SCC</literal>
	  annotations for all top-level bindings not marked INLINE. If
	  you want a cost centre on an INLINE function, you have to
	  add it manually.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fprof-auto-exported</option>:
          <indexterm><primary><option>-fprof-auto-top</option></primary></indexterm>
          <indexterm><primary>cost centres</primary><secondary>automatically inserting</secondary></indexterm>
        </term>
	<listitem>
	  <para>GHC will automatically add <literal>SCC</literal>
          annotations for all exported functions not marked
	  INLINE. If you want a cost centre on an INLINE function, you
	  have to add it manually.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fprof-auto-calls</option>:
          <indexterm><primary><option>-fprof-auto-calls</option></primary></indexterm>
        </term>
	<listitem>
          <para>Adds an automatic <literal>SCC</literal> annotation to
            all <emphasis>call sites</emphasis>.  This is particularly
            useful when using profiling for the purposes of generating
            stack traces; see the
            function <literal>traceStack</literal> in the
            module <literal>Debug.Trace</literal>, or
            the <literal>-xc</literal> RTS flag
            (<xref linkend="rts-options-debugging" />) for more
            details.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fprof-cafs</option>:
          <indexterm><primary><option>-fprof-cafs</option></primary></indexterm>
        </term>
	<listitem>
	  <para>The costs of all CAFs in a module are usually
	  attributed to one &ldquo;big&rdquo; CAF cost-centre. With
	  this option, all CAFs get their own cost-centre.  An
	  &ldquo;if all else fails&rdquo; option&hellip;</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fno-prof-auto</option>:
          <indexterm><primary><option>-no-fprof-auto</option></primary></indexterm>
        </term>
	<listitem>
          <para>Disables any previous <option>-fprof-auto</option>,
          <option>-fprof-auto-top</option>, or
          <option>-fprof-auto-exported</option> options.
          </para>
        </listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fno-prof-cafs</option>:
          <indexterm><primary><option>-fno-prof-cafs</option></primary></indexterm>
        </term>
	<listitem>
          <para>Disables any previous <option>-fprof-cafs</option> option.
          </para>
        </listitem>
      </varlistentry>

      <varlistentry>
	<term>
          <option>-fno-prof-count-entries</option>:
          <indexterm><primary><option>-fno-prof-count-entries</option></primary></indexterm>
        </term>
	<listitem>
          <para>Tells GHC not to collect information about how often
          functions are entered at runtime (the "entries" column of
          the time profile), for this module.  This tends to make the
          profiled code run faster, and hence closer to the speed of
          the unprofiled code, because GHC is able to optimise more
          aggressively if it doesn't have to maintain correct entry
          counts.  This option can be useful if you aren't interested
          in the entry counts (for example, if you only intend to do
          heap profiling).
          </para>
        </listitem>
      </varlistentry>
    </variablelist>

  </sect1>

  <sect1 id="prof-time-options">
    <title>Time and allocation profiling</title>

    <para>To generate a time and allocation profile, give one of the
    following RTS options to the compiled program when you run it (RTS
    options should be enclosed between <literal>+RTS...-RTS</literal>
    as usual):</para>

    <variablelist>
      <varlistentry>
	<term>
          <option>-p</option> or <option>-P</option> or <option>-pa</option>:
          <indexterm><primary><option>-p</option></primary></indexterm>
          <indexterm><primary><option>-P</option></primary></indexterm>
          <indexterm><primary><option>-pa</option></primary></indexterm>
          <indexterm><primary>time profile</primary></indexterm>
        </term>
	<listitem>
	  <para>The <option>-p</option> option produces a standard
          <emphasis>time profile</emphasis> report.  It is written
          into the file
          <filename><replaceable>program</replaceable>.prof</filename>.</para>

	  <para>The <option>-P</option> option produces a more
          detailed report containing the actual time and allocation
          data as well.  (Not used much.)</para>

          <para>The <option>-pa</option> option produces the most detailed
          report containing all cost centres in addition to the actual time
          and allocation data.</para>
	</listitem>
      </varlistentry>

     <varlistentry>
       <term><option>-V<replaceable>secs</replaceable></option>
       <indexterm><primary><option>-V</option></primary><secondary>RTS
       option</secondary></indexterm></term>
       <listitem>
         <para>Sets the interval that the RTS clock ticks at, which is
         also the sampling interval of the time and allocation profile.
         The default is 0.02&nbsp;seconds.</para>
       </listitem>
     </varlistentry>

      <varlistentry>
	<term>
          <option>-xc</option>
          <indexterm><primary><option>-xc</option></primary><secondary>RTS option</secondary></indexterm>
        </term>
	<listitem>
          <para>This option causes the runtime to print out the
          current cost-centre stack whenever an exception is raised.
          This can be particularly useful for debugging the location
          of exceptions, such as the notorious <literal>Prelude.head:
          empty list</literal> error.  See <xref
          linkend="rts-options-debugging"/>.</para>
        </listitem>
      </varlistentry>
    </variablelist>

  </sect1>

  <sect1 id="prof-heap">
    <title>Profiling memory usage</title>

    <para>In addition to profiling the time and allocation behaviour
    of your program, you can also generate a graph of its memory usage
    over time.  This is useful for detecting the causes of
    <firstterm>space leaks</firstterm>, when your program holds on to
    more memory at run-time that it needs to.  Space leaks lead to
    slower execution due to heavy garbage collector activity, and may
    even cause the program to run out of memory altogether.</para>

    <para>To generate a heap profile from your program:</para>

    <orderedlist>
      <listitem>
	<para>Compile the program for profiling (<xref
	linkend="prof-compiler-options"/>).</para>
      </listitem>
      <listitem>
	<para>Run it with one of the heap profiling options described
        below (eg. <option>-h</option> for a basic producer profile).
	This generates the file
	<filename><replaceable>prog</replaceable>.hp</filename>.</para>
      </listitem>
      <listitem>
	<para>Run <command>hp2ps</command> to produce a Postscript
	file,
	<filename><replaceable>prog</replaceable>.ps</filename>.  The
	<command>hp2ps</command> utility is described in detail in
	<xref linkend="hp2ps"/>.</para>
      </listitem>
      <listitem>
	<para>Display the heap profile using a postscript viewer such
	as <application>Ghostview</application>, or print it out on a
	Postscript-capable printer.</para>
      </listitem>
    </orderedlist>

    <para>For example, here is a heap profile produced for the program given above in <xref linkend="scc-pragma" />:</para>

      <!--
           contentwidth/contentheight don't appear to have any effect
           other than making the PS file generation work, rather than
           falling over.  The result seems to be broken PS on the page
           with the image. -->
      <imagedata fileref="prof_scc" contentwidth="645px"
      contentdepth="428px"/>

    <para>You might also want to take a look
      at <ulink url="http://www.haskell.org/haskellwiki/Hp2any">hp2any</ulink>,
      a more advanced suite of tools (not distributed with GHC) for
      displaying heap profiles.</para>

    <sect2 id="rts-options-heap-prof">
      <title>RTS options for heap profiling</title>

      <para>There are several different kinds of heap profile that can
      be generated.  All the different profile types yield a graph of
      live heap against time, but they differ in how the live heap is
      broken down into bands.  The following RTS options select which
      break-down to use:</para>

      <variablelist>
	<varlistentry>
	  <term>
            <option>-hc</option>
            <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
            <para>(can be shortened to <option>-h</option>). Breaks down the graph by the cost-centre stack which
	    produced the data.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hm</option>
            <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Break down the live heap by the module containing
	    the code which produced the data.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hd</option>
            <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Breaks down the graph by <firstterm>closure
	    description</firstterm>.  For actual data, the description
	    is just the constructor name, for other closures it is a
	    compiler-generated string identifying the closure.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hy</option>
            <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Breaks down the graph by
	    <firstterm>type</firstterm>.  For closures which have
	    function type or unknown/polymorphic type, the string will
	    represent an approximation to the actual type.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hr</option>
            <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Break down the graph by <firstterm>retainer
	    set</firstterm>.  Retainer profiling is described in more
	    detail below (<xref linkend="retainer-prof"/>).</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hb</option>
            <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Break down the graph by
	    <firstterm>biography</firstterm>.  Biographical profiling
	    is described in more detail below (<xref
	    linkend="biography-prof"/>).</para>
	  </listitem>
	</varlistentry>
      </variablelist>

      <para>In addition, the profile can be restricted to heap data
      which satisfies certain criteria - for example, you might want
      to display a profile by type but only for data produced by a
      certain module, or a profile by retainer for a certain type of
      data.  Restrictions are specified as follows:</para>

      <variablelist>
	<varlistentry>
	  <term>
            <option>-hc</option><replaceable>name</replaceable>,...
            <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures produced by
	    cost-centre stacks with one of the specified cost centres
	    at the top.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hC</option><replaceable>name</replaceable>,...
            <indexterm><primary><option>-hC</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures produced by
	    cost-centre stacks with one of the specified cost centres
	    anywhere in the stack.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hm</option><replaceable>module</replaceable>,...
            <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures produced by the
	    specified modules.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hd</option><replaceable>desc</replaceable>,...
            <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures with the specified
	    description strings.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hy</option><replaceable>type</replaceable>,...
            <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures with the specified
	    types.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hr</option><replaceable>cc</replaceable>,...
            <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures with retainer sets
	    containing cost-centre stacks with one of the specified
	    cost centres at the top.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-hb</option><replaceable>bio</replaceable>,...
            <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Restrict the profile to closures with one of the
	    specified biographies, where
	    <replaceable>bio</replaceable> is one of
	    <literal>lag</literal>, <literal>drag</literal>,
	    <literal>void</literal>, or <literal>use</literal>.</para>
	  </listitem>
	</varlistentry>
      </variablelist>

      <para>For example, the following options will generate a
      retainer profile restricted to <literal>Branch</literal> and
      <literal>Leaf</literal> constructors:</para>

<screen>
<replaceable>prog</replaceable> +RTS -hr -hdBranch,Leaf
</screen>

      <para>There can only be one "break-down" option
      (eg. <option>-hr</option> in the example above), but there is no
      limit on the number of further restrictions that may be applied.
      All the options may be combined, with one exception: GHC doesn't
      currently support mixing the <option>-hr</option> and
      <option>-hb</option> options.</para>

      <para>There are three more options which relate to heap
      profiling:</para>

      <variablelist>
	<varlistentry>
	  <term>
            <option>-i<replaceable>secs</replaceable></option>:
            <indexterm><primary><option>-i</option></primary></indexterm>
          </term>
	  <listitem>
	    <para>Set the profiling (sampling) interval to
            <replaceable>secs</replaceable> seconds (the default is
            0.1&nbsp;second).  Fractions are allowed: for example
            <option>-i0.2</option> will get 5 samples per second.
            This only affects heap profiling; time profiles are always
            sampled with the frequency of the RTS clock. See
            <xref linkend="prof-time-options"/> for changing that.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-xt</option>
            <indexterm><primary><option>-xt</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>Include the memory occupied by threads in a heap
	    profile.  Each thread takes up a small area for its thread
	    state in addition to the space allocated for its stack
	    (stacks normally start small and then grow as
	    necessary).</para>

	    <para>This includes the main thread, so using
	    <option>-xt</option> is a good way to see how much stack
	    space the program is using.</para>

	    <para>Memory occupied by threads and their stacks is
	    labelled as &ldquo;TSO&rdquo; and &ldquo;STACK&rdquo;
	    respectively when displaying the profile by closure
	    description or type description.</para>
	  </listitem>
	</varlistentry>

	<varlistentry>
	  <term>
            <option>-L<replaceable>num</replaceable></option>
            <indexterm><primary><option>-L</option></primary><secondary>RTS option</secondary></indexterm>
          </term>
	  <listitem>
	    <para>
          Sets the maximum length of a cost-centre stack name in a
          heap profile. Defaults to 25.
	    </para>
	  </listitem>
	</varlistentry>
      </variablelist>

    </sect2>

    <sect2 id="retainer-prof">
      <title>Retainer Profiling</title>

      <para>Retainer profiling is designed to help answer questions
      like <quote>why is this data being retained?</quote>.  We start
      by defining what we mean by a retainer:</para>

      <blockquote>
        <para>A retainer is either the system stack, an unevaluated
        closure (thunk), or an explicitly mutable object.</para>
      </blockquote>

      <para>In particular, constructors are <emphasis>not</emphasis>
      retainers.</para>

      <para>An object B retains object A if (i) B is a retainer object and
     (ii) object A can be reached by recursively following pointers
     starting from object B, but not meeting any other retainer
     objects on the way. Each live object is retained by one or more
     retainer objects, collectively called its retainer set, or its
      <firstterm>retainer set</firstterm>, or its
      <firstterm>retainers</firstterm>.</para>

      <para>When retainer profiling is requested by giving the program
      the <option>-hr</option> option, a graph is generated which is
      broken down by retainer set.  A retainer set is displayed as a
      set of cost-centre stacks; because this is usually too large to
      fit on the profile graph, each retainer set is numbered and
      shown abbreviated on the graph along with its number, and the
      full list of retainer sets is dumped into the file
      <filename><replaceable>prog</replaceable>.prof</filename>.</para>

      <para>Retainer profiling requires multiple passes over the live
      heap in order to discover the full retainer set for each
      object, which can be quite slow.  So we set a limit on the
      maximum size of a retainer set, where all retainer sets larger
      than the maximum retainer set size are replaced by the special
      set <literal>MANY</literal>.  The maximum set size defaults to 8
      and can be altered with the <option>-R</option> RTS
      option:</para>

      <variablelist>
	<varlistentry>
	  <term><option>-R</option><replaceable>size</replaceable></term>
	  <listitem>
	    <para>Restrict the number of elements in a retainer set to
	    <replaceable>size</replaceable> (default 8).</para>
	  </listitem>
	</varlistentry>
      </variablelist>

      <sect3>
	<title>Hints for using retainer profiling</title>

	<para>The definition of retainers is designed to reflect a
        common cause of space leaks: a large structure is retained by
        an unevaluated computation, and will be released once the
        computation is forced.  A good example is looking up a value in
        a finite map, where unless the lookup is forced in a timely
        manner the unevaluated lookup will cause the whole mapping to
        be retained.  These kind of space leaks can often be
        eliminated by forcing the relevant computations to be
        performed eagerly, using <literal>seq</literal> or strictness
        annotations on data constructor fields.</para>

	<para>Often a particular data structure is being retained by a
        chain of unevaluated closures, only the nearest of which will
        be reported by retainer profiling - for example A retains B, B
        retains C, and C retains a large structure.  There might be a
        large number of Bs but only a single A, so A is really the one
        we're interested in eliminating.  However, retainer profiling
        will in this case report B as the retainer of the large
        structure.  To move further up the chain of retainers, we can
        ask for another retainer profile but this time restrict the
        profile to B objects, so we get a profile of the retainers of
        B:</para>

<screen>
<replaceable>prog</replaceable> +RTS -hr -hcB
</screen>

	<para>This trick isn't foolproof, because there might be other
        B closures in the heap which aren't the retainers we are
        interested in, but we've found this to be a useful technique
        in most cases.</para>
      </sect3>
    </sect2>

    <sect2 id="biography-prof">
      <title>Biographical Profiling</title>

      <para>A typical heap object may be in one of the following four
      states at each point in its lifetime:</para>

      <itemizedlist>
	<listitem>
	  <para>The <firstterm>lag</firstterm> stage, which is the
	  time between creation and the first use of the
	  object,</para>
	</listitem>
	<listitem>
	  <para>the <firstterm>use</firstterm> stage, which lasts from
	  the first use until the last use of the object, and</para>
	</listitem>
	<listitem>
	  <para>The <firstterm>drag</firstterm> stage, which lasts
	  from the final use until the last reference to the object
	  is dropped.</para>
	</listitem>
	<listitem>
	  <para>An object which is never used is said to be in the
	  <firstterm>void</firstterm> state for its whole
	  lifetime.</para>
	</listitem>
      </itemizedlist>

      <para>A biographical heap profile displays the portion of the
      live heap in each of the four states listed above.  Usually the
      most interesting states are the void and drag states: live heap
      in these states is more likely to be wasted space than heap in
      the lag or use states.</para>

      <para>It is also possible to break down the heap in one or more
      of these states by a different criteria, by restricting a
      profile by biography.  For example, to show the portion of the
      heap in the drag or void state by producer: </para>

<screen>
<replaceable>prog</replaceable> +RTS -hc -hbdrag,void
</screen>

      <para>Once you know the producer or the type of the heap in the
      drag or void states, the next step is usually to find the
      retainer(s):</para>

<screen>
<replaceable>prog</replaceable> +RTS -hr -hc<replaceable>cc</replaceable>...
</screen>

      <para>NOTE: this two stage process is required because GHC
      cannot currently profile using both biographical and retainer
      information simultaneously.</para>
    </sect2>

    <sect2 id="mem-residency">
      <title>Actual memory residency</title>

      <para>How does the heap residency reported by the heap profiler relate to
	the actual memory residency of your program when you run it?  You might
	see a large discrepancy between the residency reported by the heap
	profiler, and the residency reported by tools on your system
	(eg. <literal>ps</literal> or <literal>top</literal> on Unix, or the
	Task Manager on Windows).  There are several reasons for this:</para>

      <itemizedlist>
	<listitem>
	  <para>There is an overhead of profiling itself, which is subtracted
	    from the residency figures by the profiler.  This overhead goes
	    away when compiling without profiling support, of course.  The
	    space overhead is currently 2 extra
	    words per heap object, which probably results in
	    about a 30% overhead.</para>
	</listitem>

	<listitem>
	  <para>Garbage collection requires more memory than the actual
	    residency.  The factor depends on the kind of garbage collection
	    algorithm in use:  a major GC in the standard
	    generation copying collector will usually require 3L bytes of
	    memory, where L is the amount of live data.  This is because by
	    default (see the <option>+RTS -F</option> option) we allow the old
	    generation to grow to twice its size (2L) before collecting it, and
	    we require additionally L bytes to copy the live data into.  When
	    using compacting collection (see the <option>+RTS -c</option>
	    option), this is reduced to 2L, and can further be reduced by
	    tweaking the <option>-F</option> option.  Also add the size of the
	    allocation area (currently a fixed 512Kb).</para>
	</listitem>

	<listitem>
	  <para>The stack isn't counted in the heap profile by default.  See the
    <option>+RTS -xt</option> option.</para>
	</listitem>

	<listitem>
	  <para>The program text itself, the C stack, any non-heap data (eg. data
	    allocated by foreign libraries, and data allocated by the RTS), and
	    <literal>mmap()</literal>'d memory are not counted in the heap profile.</para>
	</listitem>
      </itemizedlist>
    </sect2>

  </sect1>

  <sect1 id="hp2ps">
    <title><command>hp2ps</command>--heap profile to PostScript</title>

    <indexterm><primary><command>hp2ps</command></primary></indexterm>
    <indexterm><primary>heap profiles</primary></indexterm>
    <indexterm><primary>postscript, from heap profiles</primary></indexterm>
    <indexterm><primary><option>-h&lt;break-down&gt;</option></primary></indexterm>

    <para>Usage:</para>

<screen>
hp2ps [flags] [&lt;file&gt;[.hp]]
</screen>

    <para>The program
    <command>hp2ps</command><indexterm><primary>hp2ps
    program</primary></indexterm> converts a heap profile as produced
    by the <option>-h&lt;break-down&gt;</option> runtime option into a
    PostScript graph of the heap profile. By convention, the file to
    be processed by <command>hp2ps</command> has a
    <filename>.hp</filename> extension. The PostScript output is
    written to <filename>&lt;file&gt;@.ps</filename>. If
    <filename>&lt;file&gt;</filename> is omitted entirely, then the
    program behaves as a filter.</para>

    <para><command>hp2ps</command> is distributed in
    <filename>ghc/utils/hp2ps</filename> in a GHC source
    distribution. It was originally developed by Dave Wakeling as part
    of the HBC/LML heap profiler.</para>

    <para>The flags are:</para>

    <variablelist>

      <varlistentry>
	<term><option>-d</option></term>
	<listitem>
	  <para>In order to make graphs more readable,
          <command>hp2ps</command> sorts the shaded bands for each
          identifier. The default sort ordering is for the bands with
          the largest area to be stacked on top of the smaller ones.
          The <option>-d</option> option causes rougher bands (those
          representing series of values with the largest standard
          deviations) to be stacked on top of smoother ones.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-b</option></term>
	<listitem>
	  <para>Normally, <command>hp2ps</command> puts the title of
          the graph in a small box at the top of the page. However, if
          the JOB string is too long to fit in a small box (more than
          35 characters), then <command>hp2ps</command> will choose to
          use a big box instead.  The <option>-b</option> option
          forces <command>hp2ps</command> to use a big box.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-e&lt;float&gt;[in&verbar;mm&verbar;pt]</option></term>
	<listitem>
	  <para>Generate encapsulated PostScript suitable for
          inclusion in LaTeX documents.  Usually, the PostScript graph
          is drawn in landscape mode in an area 9 inches wide by 6
          inches high, and <command>hp2ps</command> arranges for this
          area to be approximately centred on a sheet of a4 paper.
          This format is convenient of studying the graph in detail,
          but it is unsuitable for inclusion in LaTeX documents.  The
          <option>-e</option> option causes the graph to be drawn in
          portrait mode, with float specifying the width in inches,
          millimetres or points (the default).  The resulting
          PostScript file conforms to the Encapsulated PostScript
          (EPS) convention, and it can be included in a LaTeX document
          using Rokicki's dvi-to-PostScript converter
          <command>dvips</command>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-g</option></term>
	<listitem>
	  <para>Create output suitable for the <command>gs</command>
          PostScript previewer (or similar). In this case the graph is
          printed in portrait mode without scaling. The output is
          unsuitable for a laser printer.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-l</option></term>
	<listitem>
	  <para>Normally a profile is limited to 20 bands with
          additional identifiers being grouped into an
          <literal>OTHER</literal> band. The <option>-l</option> flag
          removes this 20 band and limit, producing as many bands as
          necessary. No key is produced as it won't fit!. It is useful
          for creation time profiles with many bands.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-m&lt;int&gt;</option></term>
	<listitem>
	  <para>Normally a profile is limited to 20 bands with
          additional identifiers being grouped into an
          <literal>OTHER</literal> band. The <option>-m</option> flag
          specifies an alternative band limit (the maximum is
          20).</para>

	  <para><option>-m0</option> requests the band limit to be
          removed. As many bands as necessary are produced. However no
          key is produced as it won't fit! It is useful for displaying
          creation time profiles with many bands.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-p</option></term>
	<listitem>
	  <para>Use previous parameters. By default, the PostScript
          graph is automatically scaled both horizontally and
          vertically so that it fills the page.  However, when
          preparing a series of graphs for use in a presentation, it
          is often useful to draw a new graph using the same scale,
          shading and ordering as a previous one. The
          <option>-p</option> flag causes the graph to be drawn using
          the parameters determined by a previous run of
          <command>hp2ps</command> on <filename>file</filename>. These
          are extracted from <filename>file@.aux</filename>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-s</option></term>
	<listitem>
	  <para>Use a small box for the title.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-t&lt;float&gt;</option></term>
	<listitem>
	  <para>Normally trace elements which sum to a total of less
          than 1&percnt; of the profile are removed from the
          profile. The <option>-t</option> option allows this
          percentage to be modified (maximum 5&percnt;).</para>

	  <para><option>-t0</option> requests no trace elements to be
          removed from the profile, ensuring that all the data will be
          displayed.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-c</option></term>
	<listitem>
	  <para>Generate colour output.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-y</option></term>
	<listitem>
	  <para>Ignore marks.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><option>-?</option></term>
	<listitem>
	  <para>Print out usage information.</para>
	</listitem>
      </varlistentry>
    </variablelist>


    <sect2 id="manipulating-hp">
      <title>Manipulating the hp file</title>

<para>(Notes kindly offered by Jan-Willem Maessen.)</para>

<para>
The <filename>FOO.hp</filename> file produced when you ask for the
heap profile of a program <filename>FOO</filename> is a text file with a particularly
simple structure. Here's a representative example, with much of the
actual data omitted:
<screen>
JOB "FOO -hC"
DATE "Thu Dec 26 18:17 2002"
SAMPLE_UNIT "seconds"
VALUE_UNIT "bytes"
BEGIN_SAMPLE 0.00
END_SAMPLE 0.00
BEGIN_SAMPLE 15.07
  ... sample data ...
END_SAMPLE 15.07
BEGIN_SAMPLE 30.23
  ... sample data ...
END_SAMPLE 30.23
... etc.
BEGIN_SAMPLE 11695.47
END_SAMPLE 11695.47
</screen>
The first four lines (<literal>JOB</literal>, <literal>DATE</literal>, <literal>SAMPLE_UNIT</literal>, <literal>VALUE_UNIT</literal>) form a
header.  Each block of lines starting with <literal>BEGIN_SAMPLE</literal> and ending
with <literal>END_SAMPLE</literal> forms a single sample (you can think of this as a
vertical slice of your heap profile).  The hp2ps utility should accept
any input with a properly-formatted header followed by a series of
*complete* samples.
</para>
</sect2>

    <sect2>
      <title>Zooming in on regions of your profile</title>

<para>
You can look at particular regions of your profile simply by loading a
copy of the <filename>.hp</filename> file into a text editor and deleting the unwanted
samples.  The resulting <filename>.hp</filename> file can be run through <command>hp2ps</command> and viewed
or printed.
</para>
</sect2>

    <sect2>
      <title>Viewing the heap profile of a running program</title>

<para>
The <filename>.hp</filename> file is generated incrementally as your
program runs.  In principle, running <command>hp2ps</command> on the incomplete file
should produce a snapshot of your program's heap usage.  However, the
last sample in the file may be incomplete, causing <command>hp2ps</command> to fail.  If
you are using a machine with UNIX utilities installed, it's not too
hard to work around this problem (though the resulting command line
looks rather Byzantine):
<screen>
  head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
    | hp2ps > FOO.ps
</screen>

The command <command>fgrep -n END_SAMPLE FOO.hp</command> finds the
end of every complete sample in <filename>FOO.hp</filename>, and labels each sample with
its ending line number.  We then select the line number of the last
complete sample using <command>tail</command> and <command>cut</command>.  This is used as a
parameter to <command>head</command>; the result is as if we deleted the final
incomplete sample from <filename>FOO.hp</filename>.  This results in a properly-formatted
.hp file which we feed directly to <command>hp2ps</command>.
</para>
</sect2>
    <sect2>
      <title>Viewing a heap profile in real time</title>

<para>
The <command>gv</command> and <command>ghostview</command> programs
have a "watch file" option can be used to view an up-to-date heap
profile of your program as it runs.  Simply generate an incremental
heap profile as described in the previous section.  Run <command>gv</command> on your
profile:
<screen>
  gv -watch -seascape FOO.ps
</screen>
If you forget the <literal>-watch</literal> flag you can still select
"Watch file" from the "State" menu.  Now each time you generate a new
profile <filename>FOO.ps</filename> the view will update automatically.
</para>

<para>
This can all be encapsulated in a little script:
<screen>
  #!/bin/sh
  head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
    | hp2ps > FOO.ps
  gv -watch -seascape FOO.ps &amp;
  while [ 1 ] ; do
    sleep 10 # We generate a new profile every 10 seconds.
    head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
      | hp2ps > FOO.ps
  done
</screen>
Occasionally <command>gv</command> will choke as it tries to read an incomplete copy of
<filename>FOO.ps</filename> (because <command>hp2ps</command> is still running as an update
occurs).  A slightly more complicated script works around this
problem, by using the fact that sending a SIGHUP to gv will cause it
to re-read its input file:
<screen>
  #!/bin/sh
  head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
    | hp2ps > FOO.ps
  gv FOO.ps &amp;
  gvpsnum=$!
  while [ 1 ] ; do
    sleep 10
    head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
      | hp2ps > FOO.ps
    kill -HUP $gvpsnum
  done
</screen>
</para>
</sect2>
  </sect1>

  <sect1 id="prof-threaded">
    <title>Profiling Parallel and Concurrent Programs</title>

    <para>Combining <option>-threaded</option>
      and <option>-prof</option> is perfectly fine, and indeed it is
      possible to profile a program running on multiple processors
      with the <option>+RTS -N</option> option.<footnote>This feature
      was added in GHC 7.4.1.</footnote>
    </para>

    <para>
      Some caveats apply, however.  In the current implementation, a
      profiled program is likely to scale much less well than the
      unprofiled program, because the profiling implementation uses
      some shared data structures which require locking in the runtime
      system.  Furthermore, the memory allocation statistics collected
      by the profiled program are stored in shared memory
      but <emphasis>not</emphasis> locked (for speed), which means
      that these figures might be inaccurate for parallel programs.
    </para>

    <para>
      We strongly recommend that you
      use <option>-fno-prof-count-entries</option> when compiling a
      program to be profiled on multiple cores, because the entry
      counts are also stored in shared memory, and continuously
      updating them on multiple cores is extremely slow.
    </para>

    <para>
      We also recommend
      using <ulink url="http://www.haskell.org/haskellwiki/ThreadScope">ThreadScope</ulink>
      for profiling parallel programs; it offers a GUI for visualising
      parallel execution, and is complementary to the time and space
      profiling features provided with GHC.
    </para>
  </sect1>

  <sect1 id="hpc">
    <title>Observing Code Coverage</title>
    <indexterm><primary>code coverage</primary></indexterm>
    <indexterm><primary>Haskell Program Coverage</primary></indexterm>
    <indexterm><primary>hpc</primary></indexterm>

    <para>
      Code coverage tools allow a programmer to determine what parts
      of their code have been actually executed, and which parts have
      never actually been invoked.  GHC has an option for generating
      instrumented code that records code coverage as part of the
      Haskell Program Coverage (HPC) toolkit, which is included with
      GHC. HPC tools can be used to render the generated code coverage
      information into human understandable format.  </para>

    <para>
      Correctly instrumented code provides coverage information of two
      kinds: source coverage and boolean-control coverage. Source
      coverage is the extent to which every part of the program was
      used, measured at three different levels: declarations (both
      top-level and local), alternatives (among several equations or
      case branches) and expressions (at every level).  Boolean
      coverage is the extent to which each of the values True and
      False is obtained in every syntactic boolean context (ie. guard,
      condition, qualifier).  </para>

    <para>
      HPC displays both kinds of information in two primary ways:
      textual reports with summary statistics (<literal>hpc report</literal>) and sources
      with color mark-up (<literal>hpc markup</literal>).  For boolean coverage, there
      are four possible outcomes for each guard, condition or
      qualifier: both True and False values occur; only True; only
      False; never evaluated. In hpc-markup output, highlighting with
      a yellow background indicates a part of the program that was
      never evaluated; a green background indicates an always-True
      expression and a red background indicates an always-False one.
    </para>

   <sect2><title>A small example: Reciprocation</title>

    <para>
     For an example we have a program, called <filename>Recip.hs</filename>, which computes exact decimal
     representations of reciprocals, with recurring parts indicated in
     brackets.
    </para>
<programlisting>
reciprocal :: Int -> (String, Int)
reciprocal n | n > 1 = ('0' : '.' : digits, recur)
             | otherwise = error
	          "attempting to compute reciprocal of number &lt;= 1"
  where
  (digits, recur) = divide n 1 []
divide :: Int -> Int -> [Int] -> (String, Int)
divide n c cs | c `elem` cs = ([], position c cs)
              | r == 0      = (show q, 0)
              | r /= 0      = (show q ++ digits, recur)
  where
  (q, r) = (c*10) `quotRem` n
  (digits, recur) = divide n r (c:cs)

position :: Int -> [Int] -> Int
position n (x:xs) | n==x      = 1
                  | otherwise = 1 + position n xs

showRecip :: Int -> String
showRecip n =
  "1/" ++ show n ++ " = " ++
  if r==0 then d else take p d ++ "(" ++ drop p d ++ ")"
  where
  p = length d - r
  (d, r) = reciprocal n

main = do
  number &lt;- readLn
  putStrLn (showRecip number)
  main
</programlisting>

    <para>HPC instrumentation is enabled with the -fhpc flag:
    </para>

<screen>
$ ghc -fhpc Recip.hs
</screen>
    <para>GHC creates a subdirectory <filename>.hpc</filename> in the
    current directory, and puts HPC index (<filename>.mix</filename>)
    files in there, one for each module compiled.  You don't need to
    worry about these files: they contain information needed by the
    <literal>hpc</literal> tool to generate the coverage data for
    compiled modules after the program is run.</para>
<screen>
$ ./Recip
1/3
= 0.(3)
</screen>
    <para>Running the program generates a file with the
    <literal>.tix</literal> suffix, in this case
    <filename>Recip.tix</filename>, which contains the coverage data
    for this run of the program.  The program may be run multiple
    times (e.g. with different test data), and the coverage data from
    the separate runs is accumulated in the <filename>.tix</filename>
    file.  To reset the coverage data and start again, just remove the
    <filename>.tix</filename> file.
    </para>

    <para>Having run the program, we can generate a textual summary of
    coverage:</para>
<screen>
$ hpc report Recip
 80% expressions used (81/101)
 12% boolean coverage (1/8)
      14% guards (1/7), 3 always True,
                        1 always False,
                        2 unevaluated
       0% 'if' conditions (0/1), 1 always False
     100% qualifiers (0/0)
 55% alternatives used (5/9)
100% local declarations used (9/9)
100% top-level declarations used (5/5)
</screen>
    <para>We can also generate a marked-up version of the source.</para>
<screen>
$ hpc markup Recip
writing Recip.hs.html
</screen>
    <para>
		This generates one file per Haskell module, and 4 index files,
		hpc_index.html, hpc_index_alt.html, hpc_index_exp.html,
		hpc_index_fun.html.
	</para>
     </sect2>

     <sect2><title>Options for instrumenting code for coverage</title>

     <variablelist>
       <varlistentry>
         <term><option>-fhpc</option></term>
         <indexterm><primary><option>-fhpc</option></primary></indexterm>
        <listitem>
          <para>Enable code coverage for the current module or modules
          being compiled.</para>

          <para>Modules compiled with this option can be freely mixed
          with modules compiled without it; indeed, most libraries
          will typically be compiled without <option>-fhpc</option>.
          When the program is run, coverage data will only be
          generated for those modules that were compiled with
          <option>-fhpc</option>, and the <literal>hpc</literal> tool
          will only show information about those modules.
          </para>
	</listitem>
      </varlistentry>
     </variablelist>

     </sect2>

     <sect2><title>The hpc toolkit</title>

      <para>The hpc command has several sub-commands:</para>
<screen>
$ hpc
Usage: hpc COMMAND ...

Commands:
  help        Display help for hpc or a single command
Reporting Coverage:
  report      Output textual report about program coverage
  markup      Markup Haskell source with program coverage
Processing Coverage files:
  sum         Sum multiple .tix files in a single .tix file
  combine     Combine two .tix files in a single .tix file
  map         Map a function over a single .tix file
Coverage Overlays:
  overlay     Generate a .tix file from an overlay file
  draft       Generate draft overlay that provides 100% coverage
Others:
  show        Show .tix file in readable, verbose format
  version     Display version for hpc
</screen>

     <para>In general, these options act on a
     <filename>.tix</filename> file after an instrumented binary has
     generated it.
	</para>

	<para>
		The hpc tool assumes you are in the top-level directory of
                the location where you built your application, and the <filename>.tix</filename>
		file is in the same top-level directory. You can use the
                flag <option>--srcdir</option> to use <literal>hpc</literal> for any other directory, and use
                <option>--srcdir</option> multiple times to analyse programs compiled from
		difference locations, as is typical for packages.
	</para>

	<para>
	We now explain in more details the major modes of hpc.
     </para>

       <sect3><title>hpc report</title>
                <para><literal>hpc report</literal> gives a textual report of coverage. By default,
			all modules and packages are considered in generating report,
			unless include or exclude are used. The report is a summary
                        unless the <option>--per-module</option> flag is used. The <option>--xml-output</option> option
			allows for tools to use hpc to glean coverage.
		</para>
<screen>
$ hpc help report
Usage: hpc report [OPTION] .. &lt;TIX_FILE&gt; [&lt;MODULE&gt; [&lt;MODULE&gt; ..]]

Options:

    --per-module                  show module level detail
    --decl-list                   show unused decls
    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --srcdir=DIR                  path to source directory of .hs files
                                  multi-use of srcdir possible
    --hpcdir=DIR                  append sub-directory that contains .mix files
                                  default .hpc [rarely used]
    --reset-hpcdirs               empty the list of hpcdir's
                                  [rarely used]
    --xml-output                  show output in XML
</screen>
       </sect3>
       <sect3><title>hpc markup</title>
                <para><literal>hpc markup</literal> marks up source files into colored html.
		</para>
<screen>
$ hpc help markup
Usage: hpc markup [OPTION] .. &lt;TIX_FILE&gt; [&lt;MODULE&gt; [&lt;MODULE&gt; ..]]

Options:

    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --srcdir=DIR                  path to source directory of .hs files
                                  multi-use of srcdir possible
    --hpcdir=DIR                  append sub-directory that contains .mix files
                                  default .hpc [rarely used]
    --reset-hpcdirs               empty the list of hpcdir's
                                  [rarely used]
    --fun-entry-count             show top-level function entry counts
    --highlight-covered           highlight covered code, rather that code gaps
    --destdir=DIR                 path to write output to
</screen>

       </sect3>
       <sect3><title>hpc sum</title>
                <para><literal>hpc sum</literal> adds together any number of <filename>.tix</filename> files into a single
                <filename>.tix</filename> file. <literal>hpc sum</literal> does not change the original <filename>.tix</filename> file; it generates a new <filename>.tix</filename> file.
		</para>
<screen>
$ hpc help sum
Usage: hpc sum [OPTION] .. &lt;TIX_FILE&gt; [&lt;TIX_FILE&gt; [&lt;TIX_FILE&gt; ..]]
Sum multiple .tix files in a single .tix file

Options:

    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --output=FILE                 output FILE
    --union                       use the union of the module namespace (default is intersection)
</screen>
       </sect3>
       <sect3><title>hpc combine</title>
                <para><literal>hpc combine</literal> is the swiss army knife of <literal>hpc</literal>. It can be
                 used to take the difference between <filename>.tix</filename> files, to subtract one
		<filename>.tix</filename> file from another, or to add two <filename>.tix</filename> files. hpc combine does not
		change the original <filename>.tix</filename> file; it generates a new <filename>.tix</filename> file.
		</para>
<screen>
$ hpc help combine
Usage: hpc combine [OPTION] .. &lt;TIX_FILE&gt; &lt;TIX_FILE&gt;
Combine two .tix files in a single .tix file

Options:

    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --output=FILE                 output FILE
    --function=FUNCTION           combine .tix files with join function, default = ADD
                                  FUNCTION = ADD | DIFF | SUB
    --union                       use the union of the module namespace (default is intersection)
</screen>
       </sect3>
       <sect3><title>hpc map</title>
		<para>hpc map inverts or zeros a <filename>.tix</filename> file. hpc map does not
		change the original <filename>.tix</filename> file; it generates a new <filename>.tix</filename> file.
		</para>
<screen>
$ hpc help map
Usage: hpc map [OPTION] .. &lt;TIX_FILE&gt;
Map a function over a single .tix file

Options:

    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --output=FILE                 output FILE
    --function=FUNCTION           apply function to .tix files, default = ID
                                  FUNCTION = ID | INV | ZERO
    --union                       use the union of the module namespace (default is intersection)
</screen>
       </sect3>
       <sect3><title>hpc overlay and hpc draft</title>
		<para>
			Overlays are an experimental feature of HPC, a textual description
			of coverage. hpc draft is used to generate a draft overlay from a .tix file,
			and hpc overlay generates a .tix files from an overlay.
		</para>
<screen>
% hpc help overlay
Usage: hpc overlay [OPTION] .. &lt;OVERLAY_FILE&gt; [&lt;OVERLAY_FILE&gt; [...]]

Options:

    --srcdir=DIR   path to source directory of .hs files
                   multi-use of srcdir possible
    --hpcdir=DIR                  append sub-directory that contains .mix files
                                  default .hpc [rarely used]
    --reset-hpcdirs               empty the list of hpcdir's
                                  [rarely used]
    --output=FILE  output FILE
% hpc help draft
Usage: hpc draft [OPTION] .. &lt;TIX_FILE&gt;

Options:

    --exclude=[PACKAGE:][MODULE]  exclude MODULE and/or PACKAGE
    --include=[PACKAGE:][MODULE]  include MODULE and/or PACKAGE
    --srcdir=DIR                  path to source directory of .hs files
                                  multi-use of srcdir possible
    --hpcdir=DIR                  append sub-directory that contains .mix files
                                  default .hpc [rarely used]
    --reset-hpcdirs               empty the list of hpcdir's
                                  [rarely used]
    --output=FILE                 output FILE
</screen>
      </sect3>
     </sect2>
     <sect2><title>Caveats and Shortcomings of Haskell Program Coverage</title>
	  <para>
		HPC does not attempt to lock the <filename>.tix</filename> file, so multiple concurrently running
		binaries in the same directory will exhibit a race condition. There is no way
		to change the name of the <filename>.tix</filename> file generated, apart from renaming the binary.
		HPC does not work with GHCi.
  	  </para>
    </sect2>
  </sect1>

  <sect1 id="ticky-ticky">
    <title>Using &ldquo;ticky-ticky&rdquo; profiling (for implementors)</title>
    <indexterm><primary>ticky-ticky profiling</primary></indexterm>

    <para>Because ticky-ticky profiling requires a certain familiarity
    with GHC internals, we have moved the documentation to the
    wiki. Take a look at its <ulink
    url="http://ghc.haskell.org/trac/ghc/wiki/Commentary/Profiling">overview
    of the profiling options</ulink>, which includeds a link to the
    ticky-ticky profiling page.</para>
  </sect1>

</chapter>

<!-- Emacs stuff:
     ;;; Local Variables: ***
     ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") ***
     ;;; End: ***
 -->