diff options
Diffstat (limited to 'docs')
113 files changed, 49935 insertions, 1 deletions
diff --git a/docs/Makefile b/docs/Makefile index a9140dd843..b384a934c6 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,6 +1,6 @@ TOP = .. include $(TOP)/mk/boilerplate.mk -SUBDIRS = docbook-cheat-sheet +SUBDIRS = building docbook-cheat-sheet ext-core storage-mgt users_guide include $(TOP)/mk/target.mk diff --git a/docs/building/Makefile b/docs/building/Makefile new file mode 100644 index 0000000000..fb9cce6ff5 --- /dev/null +++ b/docs/building/Makefile @@ -0,0 +1,7 @@ +TOP = ../.. +include $(TOP)/mk/boilerplate.mk + +XML_DOC = building +INSTALL_XML_DOC = building + +include $(TOP)/mk/target.mk diff --git a/docs/building/building.xml b/docs/building/building.xml new file mode 100644 index 0000000000..e1967e9fec --- /dev/null +++ b/docs/building/building.xml @@ -0,0 +1,4279 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" + "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [ + <!ENTITY hacking SYSTEM "../../HACKING"> +]> + +<article id="building-guide"> + +<articleinfo> + +<title>Building and developing GHC</title> +<author><othername>The GHC Team</othername></author> +<address><email>glasgow-haskell-{users,bugs}@haskell.org</email></address> + + <abstract> + <para>This Guide is primarily aimed at those who want to build and/or + hack on GHC. It describes how to get started with building GHC on your + machine, and how to tweak the settings to get the kind of build you + want. It also describes the inner workings of the build system, so you + can extend it, modify it, and use it to build your code.</para> + + <para>The bulk of this guide applies to building on Unix + systems; see <xref linkend="winbuild"/> for Windows notes.</para> + </abstract> + +</articleinfo> + + + <sect1 id="sec-getting"> + <title>Getting the sources</title> + + <para>You can get your hands on the GHC sources in two ways:</para> + + <variablelist> + + <varlistentry> + <term><indexterm><primary>Source + distributions</primary></indexterm>Source distributions</term> + <listitem> + <para>You have a supported platform, but (a) you like + the warm fuzzy feeling of compiling things yourself; + (b) you want to build something ``extra”—e.g., a + set of libraries with strictness-analysis turned off; or + (c) you want to hack on GHC yourself.</para> + + <para>A source distribution contains complete sources for + GHC. Not only that, but the more awkward + machine-independent steps are done for you. For example, if + you don't have + <command>happy</command><indexterm><primary>happy</primary></indexterm> + you'll find it convenient that the source distribution + contains the result of running <command>happy</command> on + the parser specifications. If you don't want to alter the + parser then this saves you having to find and install + <command>happy</command>. You will still need a working + version of GHC (version 5.x or later) on your machine in + order to compile (most of) the sources, however.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>The darcs repository.<indexterm><primary>darcs repository</primary></indexterm></term> + <listitem> + <para>We make releases infrequently. If you want more + up-to-the minute (but less tested) source code then you need + to get access to our darcs repository.</para> + + <para>Information on accessing the darcs repository is on + the wiki: <ulink + url="http://hackage.haskell.org/trac/ghc/wiki/GhcDarcs" + />.</para> + + <para>The repository holds source code only. It holds no + mechanically generated files at all. So if you check out a + source tree from darcs you will need to install every utility + so that you can build all the derived files from + scratch.</para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + <sect1 id="sec-build-checks"> + <title>Things to check before you start</title> + + <para>Here's a list of things to check before you get + started.</para> + + <orderedlist> + + <listitem><para><indexterm><primary>Disk space needed</primary></indexterm>Disk + space needed: from about 100Mb for a basic GHC + build, up to probably 500Mb for a GHC build with everything + included (libraries built several different ways, + etc.).</para> + </listitem> + + <listitem> + <para>Use an appropriate machine / operating system. <xref + linkend="sec-port-info"/> lists the supported platforms; if + yours isn't amongst these then you can try porting GHC (see + <xref linkend="sec-porting-ghc"/>).</para> + </listitem> + + <listitem> + <para>Be sure that the “pre-supposed” utilities are + installed. <xref linkend="sec-pre-supposed"/> + elaborates.</para> + </listitem> + + <listitem> + <para>If you have any problem when building or installing the + Glasgow tools, please check the “known pitfalls” (<xref + linkend="sec-build-pitfalls"/>). Also check the FAQ for the + version you're building, which is part of the User's Guide and + available on the <ulink url="http://www.haskell.org/ghc/" >GHC web + site</ulink>.</para> + + <indexterm><primary>bugs</primary><secondary>known</secondary></indexterm> + + <para>If you feel there is still some shortcoming in our + procedure or instructions, please report it.</para> + + <para>For GHC, please see the <ulink + url="http://www.haskell.org/ghc/docs/latest/set/bug-reporting.html">bug-reporting + section of the GHC Users' Guide</ulink>, to maximise the + usefulness of your report.</para> + + <indexterm><primary>bugs</primary><secondary>seporting</secondary></indexterm> + <para>If in doubt, please send a message to + <email>glasgow-haskell-bugs@haskell.org</email>. + <indexterm><primary>bugs</primary><secondary>mailing + list</secondary></indexterm></para> + </listitem> + </orderedlist> + </sect1> + + <sect1 id="sec-port-info"> + <title>What machines GHC runs on</title> + +<indexterm><primary>ports</primary><secondary>GHC</secondary></indexterm> +<indexterm><primary>GHC</primary><secondary>ports</secondary></indexterm> +<indexterm><primary>platforms</primary><secondary>supported</secondary></indexterm> + + <para>A “platform” is a + architecture/manufacturer/operating-system combination, such as + <literal>sparc-sun-solaris2</literal>. Other common ones are + <literal>alpha-dec-osf2</literal>, + <literal>hppa1.1-hp-hpux9</literal>, + <literal>i386-unknown-linux</literal>, + <literal>i386-unknown-solaris2</literal>, + <literal>i386-unknown-freebsd</literal>, + <literal>i386-unknown-cygwin32</literal>, + <literal>m68k-sun-sunos4</literal>, + <literal>mips-sgi-irix5</literal>, + <literal>sparc-sun-sunos4</literal>, + <literal>sparc-sun-solaris2</literal>, + <literal>powerpc-ibm-aix</literal>.</para> + + <para>Some libraries may only work on a limited number of + platforms; for example, a sockets library is of no use unless the + operating system supports the underlying BSDisms.</para> + + <indexterm><primary>fully-supported platforms</primary></indexterm> + <indexterm><primary>native-code generator</primary></indexterm> + <indexterm><primary>registerised ports</primary></indexterm> + <indexterm><primary>unregisterised ports</primary></indexterm> + + <para>The GHC hierarchy of Porting Goodness: (a) Best is a + native-code generator; (b) next best is a + “registerised” port; (c) the bare minimum is an + “unregisterised” port. + (“Unregisterised” is so terrible that we won't say + more about it).</para> + + <para>Here's everything that's known about GHC ports. We + identify platforms by their “canonical” + CPU/Manufacturer/OS triple.</para> + + <variablelist> + <varlistentry> + <term>alpha-dec-{osf,linux,freebsd,openbsd,netbsd}: + <indexterm><primary>alpha-dec-osf</primary></indexterm> + <indexterm><primary>alpha-dec-linux</primary></indexterm> + <indexterm><primary>alpha-dec-freebsd</primary></indexterm> + <indexterm><primary>alpha-dec-openbsd</primary></indexterm> + <indexterm><primary>alpha-dec-netbsd</primary></indexterm> + </term> + <listitem> + <para>The OSF port is currently working (as of GHC version + 5.02.1) and well supported. The native code generator is + currently non-working. Other operating systems will + require some minor porting.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>sparc-sun-sunos4 + <indexterm><primary>sparc-sun-sunos4</primary></indexterm> + </term> + <listitem> + <para>Probably works with minor tweaks, hasn't been tested + for a while.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>sparc-sun-solaris2 + <indexterm><primary>sparc-sun-solaris2</primary></indexterm> + </term> + <listitem> + <para>Fully supported (at least for Solaris 2.7 and 2.6), + including native-code generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>sparc-unknown-openbsd + <indexterm><primary>sparc-unknown-openbsd</primary></indexterm> + </term> + <listitem> + <para>Supported, including native-code generator. The + same should also be true of NetBSD</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>hppa1.1-hp-hpux (HP-PA boxes running HPUX 9.x) + <indexterm><primary>hppa1.1-hp-hpux</primary></indexterm> + </term> + <listitem> + <para>A registerised port is available for version 4.08, + but GHC hasn't been built on that platform since (as far + as we know). No native-code generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>i386-unknown-linux (PCs running Linux, ELF binary format) + <indexterm><primary>i386-*-linux</primary></indexterm> + </term> + <listitem> + <para>GHC works registerised and has a native code + generator. You <emphasis>must</emphasis> have GCC 2.7.x + or later. NOTE about <literal>glibc</literal> versions: + GHC binaries built on a system running <literal>glibc + 2.0</literal> won't work on a system running + <literal>glibc 2.1</literal>, and vice versa. In general, + don't expect compatibility between + <literal>glibc</literal> versions, even if the shared + library version hasn't changed.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>i386-unknown-freebsd (PCs running FreeBSD 2.2 or higher) + <indexterm><primary>i386-unknown-freebsd</primary></indexterm> + </term> + <listitem> + <para>GHC works registerised. Pre-built packages are + available in the native package format, so if you just + need binaries you're better off just installing the + package (it might even be on your installation + CD!).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>i386-unknown-openbsd (PCs running OpenBSD) + <indexterm><primary>i386-unknown-openbsd</primary></indexterm> + </term> + <listitem> + <para>Supported, with native code generator. Packages are + available through the ports system in the native package + format.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>i386-unknown-netbsd (PCs running NetBSD) + <indexterm><primary>i386-unknown-netbsd</primary></indexterm> + </term> + <listitem> + <para>Will require some minor porting effort, but should + work registerised.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>i386-unknown-mingw32 (PCs running Windows) + <indexterm><primary>i386-unknown-mingw32</primary></indexterm> + </term> + <listitem> + <para>Fully supported under Win9x, WinNT, Win2k, and + WinXP. Includes a native code generator. Building from + source requires a recent <ulink + url="http://www.cygwin.com/">Cygwin</ulink> distribution + to be installed.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>ia64-unknown-linux + <indexterm><primary>ia64-unknown-linux</primary></indexterm> + </term> + <listitem> + <para>Supported, except there is no native code + generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>x86_64-unknown-linux + <indexterm><primary>x86_64-unknown-linux</primary></indexterm> + </term> +<term>amd64-unknown-openbsd + <indexterm><primary>amd64-unknown-linux</primary></indexterm> + </term> + <listitem> + <para>Fully supported, with a native code generator and GHCi.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>mips-sgi-irix5 + <indexterm><primary>mips-sgi-irix[5-6]</primary></indexterm> + </term> + <listitem> + <para>Port has worked in the past, but hasn't been tested + for some time (and will certainly have rotted in various + ways). As usual, we don't have access to machines and + there hasn't been an overwhelming demand for this port, + but feel free to get in touch.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>mips64-sgi-irix6 + <indexterm><primary>mips-sgi-irix6</primary></indexterm> + </term> + <listitem> + <para>GHC currently works unregisterised.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>powerpc-ibm-aix + <indexterm><primary>powerpc-ibm-aix</primary></indexterm> + </term> + <listitem> + <para>Port currently doesn't work, needs some minimal + porting effort. As usual, we don't have access to + machines and there hasn't been an overwhelming demand for + this port, but feel free to get in touch.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>powerpc-apple-darwin + <indexterm><primary>powerpc-apple-darwin</primary></indexterm> + </term> + <listitem> + <para>Supported registerised. Native code generator is + almost working.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>powerpc-apple-linux + <indexterm><primary>powerpc-apple-linux</primary></indexterm> + </term> + <listitem> + <para>Not supported (yet).</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Various other systems have had GHC ported to them in the + distant past, including various Motorola 68k boxes. The 68k + support still remains, but porting to one of these systems will + certainly be a non-trivial task.</para> + </sect1> + + <sect1 id="sec-pre-supposed"> + <title>Installing pre-supposed utilities</title> + + <indexterm><primary>pre-supposed utilities</primary></indexterm> + <indexterm><primary>utilities, pre-supposed</primary></indexterm> + + <para>Here are the gory details about some utility programs you + may need; <command>perl</command>, <command>gcc</command> and + <command>happy</command> are the only important + ones. (PVM<indexterm><primary>PVM</primary></indexterm> is + important if you're going for Parallel Haskell.) The + <command>configure</command><indexterm><primary>configure</primary></indexterm> + script will tell you if you are missing something.</para> + + <variablelist> + + <varlistentry> + <term>GHC + <indexterm><primary>pre-supposed: GHC</primary></indexterm> + <indexterm><primary>GHC, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>GHC is required to build many of the tools, including + GHC itself. If you need to port GHC to your platform + because there isn't a binary distribution of GHC available, + then see <xref linkend="sec-porting-ghc"/>.</para> + + <para>Which version of GHC you need will depend on the + packages you intend to build. GHC itself will normally + build using one of several older versions of itself - check + the announcement or release notes for details.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Perl + <indexterm><primary>pre-supposed: Perl</primary></indexterm> + <indexterm><primary>Perl, pre-supposed</primary></indexterm> + </term> + <listitem> + <para><emphasis>You have to have Perl to proceed!</emphasis> + Perl version 5 at least is required. GHC has been known to + tickle bugs in Perl, so if you find that Perl crashes when + running GHC try updating (or downgrading) your Perl + installation. Versions of Perl before 5.6 have been known to have + various bugs tickled by GHC, so the configure script + will look for version 5.6 or later.</para> + + <para>For Win32 platforms, you should use the binary + supplied in the InstallShield (copy it to + <filename>/bin</filename>). The Cygwin-supplied Perl seems + not to work.</para> + + <para>Perl should be put somewhere so that it can be invoked + by the <literal>#!</literal> script-invoking + mechanism. The full pathname may need to be less than 32 + characters long on some systems.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GNU C (<command>gcc</command>) + <indexterm><primary>pre-supposed: GCC (GNU C compiler)</primary></indexterm> + <indexterm><primary>GCC (GNU C compiler), pre-supposed</primary></indexterm> + </term> + <listitem> + <para>Most GCC versions should work with the most recent GHC + sources. Expect trouble if you use a recent GCC with + an older GHC, though (trouble in the form of mis-compiled code, + link errors, and errors from the <literal>ghc-asm</literal> + script).</para> + + <para>If your GCC dies with “internal error” on + some GHC source file, please let us know, so we can report + it and get things improved. (Exception: on x86 + boxes—you may need to fiddle with GHC's + <option>-monly-N-regs</option> option; see the User's + Guide)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GNU Make + <indexterm><primary>make</primary><secondary>GNU</secondary></indexterm> + </term> + <listitem> + <para>The fptools build system makes heavy use of features + specific to GNU <command>make</command>, so you must have + this installed in order to build any of the fptools + suite.</para> + + <para>NB. it has been reported that version 3.79 no longer + works to build GHC, and 3.80 is required.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><ulink url="http://www.haskell.org/happy">Happy</ulink> + <indexterm><primary>Happy</primary></indexterm> + </term> + <listitem> + <para>Happy is a parser generator tool for Haskell, and is + used to generate GHC's parsers.</para> + + <para>If you start from a source tarball of GHC (i.e. not a darcs + checkout), then you don't need Happy, because we supply the + pre-processed versions of the Happy parsers. If you intend to + modify the compiler and/or you're using a darcs checkout, then you + need Happy.</para> + + <para>Happy version 1.15 is currently required to build GHC.</para> + + <para>Happy is written in + Haskell, and is a project in the CVS repository + (<literal>fptools/happy</literal>). It can be built from + source, but bear in mind that you'll need GHC installed in + order to build it. To avoid the chicken/egg problem, + install a binary distribution of either Happy or GHC to get + started. Happy distributions are available from <ulink url="http://www.haskell.org/happy/">Happy's Web + Page</ulink>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Alex + <indexterm><primary>Alex</primary></indexterm> + </term> + <listitem> + <para>Alex is a lexical-analyser generator for Haskell, + which GHC uses to generate its lexer.</para> + + <para>Like Happy, you don't need Alex if you're building GHC from a + source tarball, but you do need it if you're modifying GHC and/or + building a darcs checkout.</para> + + <para>Alex is + written in Haskell and is a project in the darcs repository. + Alex distributions are available from <ulink url="http://www.haskell.org/alex/">Alex's Web + Page</ulink>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>autoconf + <indexterm><primary>pre-supposed: autoconf</primary></indexterm> + <indexterm><primary>autoconf, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>GNU autoconf is needed if you intend to build from the + darcs sources, it is <emphasis>not</emphasis> needed if you + just intend to build a standard source distribution.</para> + + <para>Version 2.52 or later of the autoconf package is required. + NB. version 2.13 will no longer work, as of GHC version + 6.1.</para> + + <para><command>autoreconf</command> (from the autoconf package) + recursively builds <command>configure</command> scripts from + the corresponding <filename>configure.ac</filename> and + <filename>aclocal.m4</filename> files. If you modify one of + the latter files, you'll need <command>autoreconf</command> to + rebuild the corresponding <filename>configure</filename>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><command>sed</command> + <indexterm><primary>pre-supposed: sed</primary></indexterm> + <indexterm><primary>sed, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>You need a working <command>sed</command> if you are + going to build from sources. The build-configuration stuff + needs it. GNU sed version 2.0.4 is no good! It has a bug + in it that is tickled by the build-configuration. 2.0.5 is + OK. Others are probably OK too (assuming we don't create too + elaborate configure scripts.)</para> + </listitem> + </varlistentry> + </variablelist> + + <para>One <literal>fptools</literal> project is worth a quick note + at this point, because it is useful for all the others: + <literal>glafp-utils</literal> contains several utilities which + aren't particularly Glasgow-ish, but Occasionally Indispensable. + Like <command>lndir</command> for creating symbolic link + trees.</para> + + <sect2 id="pre-supposed-gph-tools"> + <title>Tools for building parallel GHC (GPH)</title> + + <variablelist> + <varlistentry> + <term>PVM version 3: + <indexterm><primary>pre-supposed: PVM3 (Parallel Virtual Machine)</primary></indexterm> + <indexterm><primary>PVM3 (Parallel Virtual Machine), pre-supposed</primary></indexterm> + </term> + <listitem> + <para>PVM is the Parallel Virtual Machine on which + Parallel Haskell programs run. (You only need this if you + plan to run Parallel Haskell. Concurrent Haskell, which + runs concurrent threads on a uniprocessor doesn't need + it.) Underneath PVM, you can have (for example) a network + of workstations (slow) or a multiprocessor box + (faster).</para> + + <para>The current version of PVM is 3.3.11; we use 3.3.7. + It is readily available on the net; I think I got it from + <literal>research.att.com</literal>, in + <filename>netlib</filename>.</para> + + <para>A PVM installation is slightly quirky, but easy to + do. Just follow the <filename>Readme</filename> + instructions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><command>bash</command>: + <indexterm><primary>bash, presupposed (Parallel Haskell only)</primary></indexterm> + </term> + <listitem> + <para>Sadly, the <command>gr2ps</command> script, used to + convert “parallelism profiles” to PostScript, + is written in Bash (GNU's Bourne Again shell). This bug + will be fixed (someday).</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="pre-supposed-other-tools"> + <title>Other useful tools</title> + + <variablelist> + <varlistentry> + <term>Flex + <indexterm><primary>pre-supposed: flex</primary></indexterm> + <indexterm><primary>flex, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>This is a quite-a-bit-better-than-Lex lexer. Used + to build a couple of utilities in + <literal>glafp-utils</literal>. Depending on your + operating system, the supplied <command>lex</command> may + or may not work; you should get the GNU version.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>More tools are required if you want to format the documentation + that comes with GHC and other fptools projects. See <xref + linkend="building-docs"/>.</para> + </sect2> + </sect1> + + <sect1 id="sec-building-from-source"> + <title>Building from source</title> + + <indexterm><primary>Building from source</primary></indexterm> + <indexterm><primary>Source, building from</primary></indexterm> + + <para>“I just want to build it!”</para> + + <para>No problem. This recipe should build and install a working GHC with + all the default settings. (unless you're + on Windows, in which case go to <xref linkend="winbuild" />).</para> + +<screen>$ autoreconf<footnote><para>not necessary if you started from a source tarball</para> + </footnote> +$ ./configure +$ make +$ make install</screen> + + <para>For GHC, this will do a 2-stage bootstrap build of the + compiler, with profiling libraries, and install the + results in the default location (under <filename>/usr/local</filename> on + Unix, for example).</para> + + <para>The <literal>configure</literal> script is a standard GNU + <literal>autoconf</literal> script, and accepts the usual options for + changing install locations and the like. Run + <literal>./configure --help</literal> for a list of options.</para> + + <para>If you want to do anything at all non-standard, or you + want to do some development, read on...</para> + </sect1> + + <sect1 id="quick-start"> + <title>Quick start for GHC developers</title> + + <para>This section is a copy of the file + <literal>ghc/HACKING</literal> from the GHC source tree. It describes + how to get started with setting up your build tree for developing GHC + or its libraries, and how to start building.</para> + +<screen> +&hacking; + </screen> + </sect1> + + <sect1 id="sec-working-with-the-build-system"> + <title>Working with the build system</title> + + <para>This rest of this guide is intended for duffers like me, who + aren't really interested in Makefiles and systems configurations, + but who need a mental model of the interlocking pieces so that + they can make them work, extend them consistently when adding new + software, and lay hands on them gently when they don't + work.</para> + + <sect2 id="sec-source-tree"> + <title>Your source tree</title> + + <para>The source code is held in your <emphasis>source + tree</emphasis>. The root directory of your source tree + <emphasis>must</emphasis> contain the following directories and + files:</para> + + <itemizedlist> + <listitem> + <para><filename>Makefile</filename>: the root + Makefile.</para> + </listitem> + + <listitem> + <para><filename>mk/</filename>: the directory that contains + the main Makefile code, shared by all the + <literal>fptools</literal> software.</para> + </listitem> + + <listitem> + <para><filename>configure.ac</filename>, + <filename>config.sub</filename>, + <filename>config.guess</filename>: these files support the + configuration process.</para> + </listitem> + + <listitem> + <para><filename>install-sh</filename>.</para> + </listitem> + </itemizedlist> + + <para>All the other directories are individual + <emphasis>projects</emphasis> of the <literal>fptools</literal> + system—for example, the Glasgow Haskell Compiler + (<literal>ghc</literal>), the Happy parser generator + (<literal>happy</literal>), the <literal>nofib</literal> + benchmark suite, and so on. You can have zero or more of these. + Needless to say, some of them are needed to build others.</para> + + <para>The important thing to remember is that even if you want + only one project (<literal>happy</literal>, say), you must have + a source tree whose root directory contains + <filename>Makefile</filename>, <filename>mk/</filename>, + <filename>configure.ac</filename>, and the project(s) you want + (<filename>happy/</filename> in this case). You cannot get by + with just the <filename>happy/</filename> directory.</para> + </sect2> + + <sect2> + <title>Build trees</title> + <indexterm><primary>build trees</primary></indexterm> + <indexterm><primary>link trees, for building</primary></indexterm> + + <para>If you just want to build the software once on a single + platform, then your source tree can also be your build tree, and + you can skip the rest of this section.</para> + + <para>We often want to build multiple versions of our software + for different architectures, or with different options + (e.g. profiling). It's very desirable to share a single copy of + the source code among all these builds.</para> + + <para>So for every source tree we have zero or more + <emphasis>build trees</emphasis>. Each build tree is initially + an exact copy of the source tree, except that each file is a + symbolic link to the source file, rather than being a copy of + the source file. There are “standard” Unix + utilities that make such copies, so standard that they go by + different names: + <command>lndir</command><indexterm><primary>lndir</primary></indexterm>, + <command>mkshadowdir</command><indexterm><primary>mkshadowdir</primary></indexterm> + are two (If you don't have either, the source distribution + includes sources for the X11 + <command>lndir</command>—check out + <filename>fptools/glafp-utils/lndir</filename>). See <xref + linkend="sec-storysofar"/> for a typical invocation.</para> + + <para>The build tree does not need to be anywhere near the + source tree in the file system. Indeed, one advantage of + separating the build tree from the source is that the build tree + can be placed in a non-backed-up partition, saving your systems + support people from backing up untold megabytes of + easily-regenerated, and rapidly-changing, gubbins. The golden + rule is that (with a single exception—<xref + linkend="sec-build-config"/>) <emphasis>absolutely everything in + the build tree is either a symbolic link to the source tree, or + else is mechanically generated</emphasis>. It should be + perfectly OK for your build tree to vanish overnight; an hour or + two compiling and you're on the road again.</para> + + <para>You need to be a bit careful, though, that any new files + you create (if you do any development work) are in the source + tree, not a build tree!</para> + + <para>Remember, that the source files in the build tree are + <emphasis>symbolic links</emphasis> to the files in the source + tree. (The build tree soon accumulates lots of built files like + <filename>Foo.o</filename>, as well.) You can + <emphasis>delete</emphasis> a source file from the build tree + without affecting the source tree (though it's an odd thing to + do). On the other hand, if you <emphasis>edit</emphasis> a + source file from the build tree, you'll edit the source-tree + file directly. (You can set up Emacs so that if you edit a + source file from the build tree, Emacs will silently create an + edited copy of the source file in the build tree, leaving the + source file unchanged; but the danger is that you think you've + edited the source file whereas actually all you've done is edit + the build-tree copy. More commonly you do want to edit the + source file.)</para> + + <para>Like the source tree, the top level of your build tree + must be (a linked copy of) the root directory of the + <literal>fptools</literal> suite. Inside Makefiles, the root of + your build tree is called + <constant>$(FPTOOLS_TOP)</constant><indexterm><primary>FPTOOLS_TOP</primary></indexterm>. + In the rest of this document path names are relative to + <constant>$(FPTOOLS_TOP)</constant> unless + otherwise stated. For example, the file + <filename>ghc/mk/target.mk</filename> is actually + <filename>$(FPTOOLS_TOP)/ghc/mk/target.mk</filename>.</para> + </sect2> + + <sect2 id="sec-build-config"> + <title>Getting the build you want</title> + + <para>When you build <literal>fptools</literal> you will be + compiling code on a particular <emphasis>host + platform</emphasis>, to run on a particular <emphasis>target + platform</emphasis> (usually the same as the host + platform)<indexterm><primary>platform</primary></indexterm>. + The difficulty is that there are minor differences between + different platforms; minor, but enough that the code needs to be + a bit different for each. There are some big differences too: + for a different architecture we need to build GHC with a + different native-code generator.</para> + + <para>There are also knobs you can turn to control how the + <literal>fptools</literal> software is built. For example, you + might want to build GHC optimised (so that it runs fast) or + unoptimised (so that you can compile it fast after you've + modified it. Or, you might want to compile it with debugging on + (so that extra consistency-checking code gets included) or off. + And so on.</para> + + <para>All of this stuff is called the + <emphasis>configuration</emphasis> of your build. You set the + configuration using a three-step process.</para> + + <variablelist> + <varlistentry> + <term>Step 1: get ready for configuration.</term> + <listitem> + <para>NOTE: if you're starting from a source distribution, + rather than darcs sources, you can skip this step.</para> + + <para>Change directory to + <constant>$(FPTOOLS_TOP)</constant> and + issue the command</para> +<screen>$ autoreconf</screen> + <indexterm><primary>autoreconf</primary></indexterm> + <para>(with no arguments). This GNU program (recursively) converts + <filename>$(FPTOOLS_TOP)/configure.ac</filename> and + <filename>$(FPTOOLS_TOP)/aclocal.m4</filename> + to a shell script called + <filename>$(FPTOOLS_TOP)/configure</filename>. + If <command>autoreconf</command> bleats that it can't write the file <filename>configure</filename>, + then delete the latter and try again. Note that you must use <command>autoreconf</command>, + and not the old <command>autoconf</command>! If you erroneously use the latter, you'll get + a message like "No rule to make target 'mk/config.h.in'". + </para> + + <para>Some projects, including GHC, have their own configure script. + <command>autoreconf</command> takes care of that, too, so all you have + to do is calling <command>autoreconf</command> in the top-level directory + <filename>$(FPTOOLS_TOP)</filename>.</para> + + <para>These steps are completely platform-independent; they just mean + that the human-written files (<filename>configure.ac</filename> and + <filename>aclocal.m4</filename>) can be short, although the resulting + files (the <command>configure</command> shell scripts and the C header + template <filename>mk/config.h.in</filename>) are long.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Step 2: system configuration.</term> + <listitem> + <para>Runs the newly-created <command>configure</command> + script, thus:</para> + +<screen>$ ./configure <optional><parameter>args</parameter></optional></screen> + + <para><command>configure</command>'s mission is to scurry + round your computer working out what architecture it has, + what operating system, whether it has the + <function>vfork</function> system call, where + <command>tar</command> is kept, whether + <command>gcc</command> is available, where various obscure + <literal>#include</literal> files are, whether it's a + leap year, and what the systems manager had for lunch. It + communicates these snippets of information in two + ways:</para> + + <itemizedlist> + <listitem> + + <para>It translates + <filename>mk/config.mk.in</filename><indexterm><primary>config.mk.in</primary></indexterm> + to + <filename>mk/config.mk</filename><indexterm><primary>config.mk</primary></indexterm>, + substituting for things between + “<literal>@</literal>” brackets. So, + “<literal>@HaveGcc@</literal>” will be + replaced by “<literal>YES</literal>” or + “<literal>NO</literal>” depending on what + <command>configure</command> finds. + <filename>mk/config.mk</filename> is included by every + Makefile (directly or indirectly), so the + configuration information is thereby communicated to + all Makefiles.</para> + </listitem> + + <listitem> + <para> It translates + <filename>mk/config.h.in</filename><indexterm><primary>config.h.in</primary></indexterm> + to + <filename>mk/config.h</filename><indexterm><primary>config.h</primary></indexterm>. + The latter is <literal>#include</literal>d by + various C programs, which can thereby make use of + configuration information.</para> + </listitem> + </itemizedlist> + + <para><command>configure</command> takes some optional + arguments. Use <literal>./configure --help</literal> to + get a list of the available arguments. Here are some of + the ones you might need:</para> + + <variablelist> + <varlistentry> + <term><literal>--with-ghc=<parameter>path</parameter></literal> + <indexterm><primary><literal>--with-ghc</literal></primary></indexterm> + </term> + <listitem> + <para>Specifies the path to an installed GHC which + you would like to use. This compiler will be used + for compiling GHC-specific code (eg. GHC itself). + This option <emphasis>cannot</emphasis> be specified + using <filename>build.mk</filename> (see later), + because <command>configure</command> needs to + auto-detect the version of GHC you're using. The + default is to look for a compiler named + <literal>ghc</literal> in your path.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>--with-hc=<parameter>path</parameter></literal> + <indexterm><primary><literal>--with-hc</literal></primary></indexterm> + </term> + <listitem> + <para>Specifies the path to any installed Haskell + compiler. This compiler will be used for compiling + generic Haskell code. The default is to use + <literal>ghc</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>--with-gcc=<parameter>path</parameter></literal> + <indexterm><primary><literal>--with-gcc</literal></primary></indexterm> + </term> + <listitem> + <para>Specifies the path to the installed GCC. This + compiler will be used to compile all C files, + <emphasis>except</emphasis> any generated by the + installed Haskell compiler, which will have its own + idea of which C compiler (if any) to use. The + default is to use <literal>gcc</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term>Step 3: build configuration.</term> + <listitem> + <para>Next, you say how this build of + <literal>fptools</literal> is to differ from the standard + defaults by creating a new file + <filename>mk/build.mk</filename><indexterm><primary>build.mk</primary></indexterm> + <emphasis>in the build tree</emphasis>. This file is the + one and only file you edit in the build tree, precisely + because it says how this build differs from the source. + (Just in case your build tree does die, you might want to + keep a private directory of <filename>build.mk</filename> + files, and use a symbolic link in each build tree to point + to the appropriate one.) So + <filename>mk/build.mk</filename> never exists in the + source tree—you create one in each build tree from + the template. We'll discuss what to put in it + shortly.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>And that's it for configuration. Simple, eh?</para> + + <para>What do you put in your build-specific configuration file + <filename>mk/build.mk</filename>? <emphasis>For almost all + purposes all you will do is put make variable definitions that + override those in</emphasis> + <filename>mk/config.mk.in</filename>. The whole point of + <filename>mk/config.mk.in</filename>—and its derived + counterpart <filename>mk/config.mk</filename>—is to define + the build configuration. It is heavily commented, as you will + see if you look at it. So generally, what you do is look at + <filename>mk/config.mk.in</filename>, and add definitions in + <filename>mk/build.mk</filename> that override any of the + <filename>config.mk</filename> definitions that you want to + change. (The override occurs because the main boilerplate file, + <filename>mk/boilerplate.mk</filename><indexterm><primary>boilerplate.mk</primary></indexterm>, + includes <filename>build.mk</filename> after + <filename>config.mk</filename>.)</para> + + <para>For your convenience, there's a file called <filename>build.mk.sample</filename> + that can serve as a starting point for your <filename>build.mk</filename>.</para> + + <para>For example, <filename>config.mk.in</filename> contains + the definition:</para> + +<programlisting>GhcHcOpts=-O -Rghc-timing</programlisting> + + <para>The accompanying comment explains that this is the list of + flags passed to GHC when building GHC itself. For doing + development, it is wise to add <literal>-DDEBUG</literal>, to + enable debugging code. So you would add the following to + <filename>build.mk</filename>:</para> + + <para>or, if you prefer,</para> + +<programlisting>GhcHcOpts += -DDEBUG</programlisting> + + <para>GNU <command>make</command> allows existing definitions to + have new text appended using the “<literal>+=</literal>” + operator, which is quite a convenient feature.)</para> + + <para>If you want to remove the <literal>-O</literal> as well (a + good idea when developing, because the turn-around cycle gets a + lot quicker), you can just override + <literal>GhcLibHcOpts</literal> altogether:</para> + +<programlisting>GhcHcOpts=-DDEBUG -Rghc-timing</programlisting> + + <para>When reading <filename>config.mk.in</filename>, remember + that anything between “@...@” signs is going to be substituted + by <command>configure</command> later. You + <emphasis>can</emphasis> override the resulting definition if + you want, but you need to be a bit surer what you are doing. + For example, there's a line that says:</para> + +<programlisting>TAR = @TarCmd@</programlisting> + + <para>This defines the Make variables <constant>TAR</constant> + to the pathname for a <command>tar</command> that + <command>configure</command> finds somewhere. If you have your + own pet <command>tar</command> you want to use instead, that's + fine. Just add this line to <filename>mk/build.mk</filename>:</para> + +<programlisting>TAR = mytar</programlisting> + + <para>You do not <emphasis>have</emphasis> to have a + <filename>mk/build.mk</filename> file at all; if you don't, + you'll get all the default settings from + <filename>mk/config.mk.in</filename>.</para> + + <para>You can also use <filename>build.mk</filename> to override + anything that <command>configure</command> got wrong. One place + where this happens often is with the definition of + <constant>FPTOOLS_TOP_ABS</constant>: this + variable is supposed to be the canonical path to the top of your + source tree, but if your system uses an automounter then the + correct directory is hard to find automatically. If you find + that <command>configure</command> has got it wrong, just put the + correct definition in <filename>build.mk</filename>.</para> + + </sect2> + + <sect2 id="sec-storysofar"> + <title>The story so far</title> + + <para>Let's summarise the steps you need to carry to get + yourself a fully-configured build tree from scratch.</para> + + <orderedlist> + <listitem> + <para> Get your source tree from somewhere (darcs repository + or source distribution). Say you call the root directory + <filename>myfptools</filename> (it does not have to be + called <filename>fptools</filename>). Make sure that you + have the essential files (see <xref + linkend="sec-source-tree"/>).</para> + </listitem> + + <listitem> + + <para>(Optional) Use <command>lndir</command> or + <command>mkshadowdir</command> to create a build tree.</para> + +<screen>$ cd myfptools +$ mkshadowdir . /scratch/joe-bloggs/myfptools-sun4</screen> + + <para>(N.B. <command>mkshadowdir</command>'s first argument + is taken relative to its second.) You probably want to give + the build tree a name that suggests its main defining + characteristic (in your mind at least), in case you later + add others.</para> + </listitem> + + <listitem> + <para>Change directory to the build tree. Everything is + going to happen there now.</para> + +<screen>$ cd /scratch/joe-bloggs/myfptools-sun4</screen> + + </listitem> + + <listitem> + <para>Prepare for system configuration:</para> + +<screen>$ autoreconf</screen> + + <para>(You can skip this step if you are starting from a + source distribution, and you already have + <filename>configure</filename> and + <filename>mk/config.h.in</filename>.)</para> + </listitem> + + <listitem> + <para>Do system configuration:</para> + +<screen>$ ./configure</screen> + + <para>Don't forget to check whether you need to add any + arguments to <literal>configure</literal>; for example, a + common requirement is to specify which GHC to use with + <option>--with-ghc=<replaceable>ghc</replaceable></option>.</para> + </listitem> + + <listitem> + <para>Create the file <filename>mk/build.mk</filename>, + adding definitions for your desired configuration + options.</para> + +<screen>$ emacs mk/build.mk</screen> + </listitem> + </orderedlist> + + <para>You can make subsequent changes to + <filename>mk/build.mk</filename> as often as you like. You do + not have to run any further configuration programs to make these + changes take effect. In theory you should, however, say + <command>gmake clean</command>, <command>gmake all</command>, + because configuration option changes could affect + anything—but in practice you are likely to know what's + affected.</para> + </sect2> + + <sect2> + <title>Making things</title> + + <para>At this point you have made yourself a fully-configured + build tree, so you are ready to start building real + things.</para> + + <para>The first thing you need to know is that <emphasis>you + must use GNU <command>make</command>, usually called + <command>gmake</command>, not standard Unix + <command>make</command></emphasis>. If you use standard Unix + <command>make</command> you will get all sorts of error messages + (but no damage) because the <literal>fptools</literal> + <command>Makefiles</command> use GNU <command>make</command>'s + facilities extensively.</para> + + <para>To just build the whole thing, <command>cd</command> to + the top of your <literal>fptools</literal> tree and type + <command>gmake</command>. This will prepare the tree and build + the various projects in the correct order.</para> + </sect2> + + <sect2 id="sec-bootstrapping"> + <title>Bootstrapping GHC</title> + + <para>GHC requires a 2-stage bootstrap in order to provide + full functionality, including GHCi. By a 2-stage bootstrap, we + mean that the compiler is built once using the installed GHC, + and then again using the compiler built in the first stage. You + can also build a stage 3 compiler, but this normally isn't + necessary except to verify that the stage 2 compiler is working + properly.</para> + + <para>Note that when doing a bootstrap, the stage 1 compiler + must be built, followed by the runtime system and libraries, and + then the stage 2 compiler. The correct ordering is implemented + by the top-level fptools <filename>Makefile</filename>, so if + you want everything to work automatically it's best to start + <command>make</command> from the top of the tree. When building + GHC, the top-level fptools <filename>Makefile</filename> is set + up to do a 2-stage bootstrap by default (when you say + <command>make</command>). Some other targets it supports + are:</para> + + <variablelist> + <varlistentry> + <term>stage1</term> + <listitem> + <para>Build everything as normal, including the stage 1 + compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>stage2</term> + <listitem> + <para>Build the stage 2 compiler only.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>stage3</term> + <listitem> + <para>Build the stage 3 compiler only.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>bootstrap</term> <term>bootstrap2</term> + <listitem> + <para>Build stage 1 followed by stage 2.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>bootstrap3</term> + <listitem> + <para>Build stages 1, 2 and 3.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>install</term> + <listitem> + <para>Install everything, including the compiler built in + stage 2. To override the stage, say <literal>make install + stage=<replaceable>n</replaceable></literal> where + <replaceable>n</replaceable> is the stage to install.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>The top-level <filename>Makefile</filename> also arranges + to do the appropriate <literal>make boot</literal> steps (see + below) before actually building anything.</para> + + <para>The <literal>stage1</literal>, <literal>stage2</literal> + and <literal>stage3</literal> targets also work in the + <literal>ghc/compiler</literal> directory, but don't forget that + each stage requires its own <literal>make boot</literal> step: + for example, you must do</para> + + <screen>$ make boot stage=2</screen> + + <para>before <literal>make stage2</literal> in + <literal>ghc/compiler</literal>.</para> + </sect2> + + <sect2 id="sec-standard-targets"> + <title>Standard Targets</title> + <indexterm><primary>targets, standard makefile</primary></indexterm> + <indexterm><primary>makefile targets</primary></indexterm> + + <para>In any directory you should be able to make the following:</para> + + <variablelist> + <varlistentry> + <term><literal>boot</literal></term> + <listitem> + <para>does the one-off preparation required to get ready + for the real work. Notably, it does <command>gmake + depend</command> in all directories that contain programs. + It also builds the necessary tools for compilation to + proceed.</para> + + <para>Invoking the <literal>boot</literal> target + explicitly is not normally necessary. From the top-level + <literal>fptools</literal> directory, invoking + <literal>gmake</literal> causes <literal>gmake boot + all</literal> to be invoked in each of the project + subdirectories, in the order specified by + <literal>$(AllTargets)</literal> in + <literal>config.mk</literal>.</para> + + <para>If you're working in a subdirectory somewhere and + need to update the dependencies, <literal>gmake + boot</literal> is a good way to do it.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>all</literal></term> + <listitem> + <para>makes all the final target(s) for this Makefile. + Depending on which directory you are in a “final + target” may be an executable program, a library + archive, a shell script, or a Postscript file. Typing + <command>gmake</command> alone is generally the same as + typing <command>gmake all</command>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>install</literal></term> + <listitem> + <para>installs the things built by <literal>all</literal> + (except for the documentation). Where does it install + them? That is specified by + <filename>mk/config.mk.in</filename>; you can override it + in <filename>mk/build.mk</filename>, or by running + <command>configure</command> with command-line arguments + like <literal>--bindir=/home/simonpj/bin</literal>; see + <literal>./configure --help</literal> for the full + details.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>install-docs</literal></term> + <listitem> + <para>installs the documentation. Otherwise behaves just + like <literal>install</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>uninstall</literal></term> + <listitem> + <para>reverses the effect of + <literal>install</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>clean</literal></term> + <listitem> + <para>Delete all files from the current directory that are + normally created by building the program. Don't delete + the files that record the configuration, or files + generated by <command>gmake boot</command>. Also preserve + files that could be made by building, but normally aren't + because the distribution comes with them.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>distclean</literal></term> + <listitem> + <para>Delete all files from the current directory that are + created by configuring or building the program. If you + have unpacked the source and built the program without + creating any other files, <literal>make + distclean</literal> should leave only the files that were + in the distribution.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>mostlyclean</literal></term> + <listitem> + <para>Like <literal>clean</literal>, but may refrain from + deleting a few files that people normally don't want to + recompile.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>maintainer-clean</literal></term> + <listitem> + <para>Delete everything from the current directory that + can be reconstructed with this Makefile. This typically + includes everything deleted by + <literal>distclean</literal>, plus more: C source files + produced by Bison, tags tables, Info files, and so + on.</para> + + <para>One exception, however: <literal>make + maintainer-clean</literal> should not delete + <filename>configure</filename> even if + <filename>configure</filename> can be remade using a rule + in the <filename>Makefile</filename>. More generally, + <literal>make maintainer-clean</literal> should not delete + anything that needs to exist in order to run + <filename>configure</filename> and then begin to build the + program.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>check</literal></term> + <listitem> + <para>run the test suite.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>All of these standard targets automatically recurse into + sub-directories. Certain other standard targets do not:</para> + + <variablelist> + <varlistentry> + <term><literal>configure</literal></term> + <listitem> + <para>is only available in the root directory + <constant>$(FPTOOLS_TOP)</constant>; it has + been discussed in <xref + linkend="sec-build-config"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>depend</literal></term> + <listitem> + <para>make a <filename>.depend</filename> file in each + directory that needs it. This <filename>.depend</filename> + file contains mechanically-generated dependency + information; for example, suppose a directory contains a + Haskell source module <filename>Foo.lhs</filename> which + imports another module <literal>Baz</literal>. Then the + generated <filename>.depend</filename> file will contain + the dependency:</para> + +<programlisting>Foo.o : Baz.hi</programlisting> + + <para>which says that the object file + <filename>Foo.o</filename> depends on the interface file + <filename>Baz.hi</filename> generated by compiling module + <literal>Baz</literal>. The <filename>.depend</filename> + file is automatically included by every Makefile.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>binary-dist</literal></term> + <listitem> + <para>make a binary distribution. This is the target we + use to build the binary distributions of GHC and + Happy.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>dist</literal></term> + <listitem> + <para>make a source distribution. Note that this target + does “make distclean” as part of its work; + don't use it if you want to keep what you've built.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Most <filename>Makefile</filename>s have targets other + than these. You can discover them by looking in the + <filename>Makefile</filename> itself.</para> + </sect2> + + <sect2> + <title>Using a project from the build tree</title> + + <para>If you want to build GHC (say) and just use it direct from + the build tree without doing <literal>make install</literal> + first, you can run the in-place driver script: + <filename>ghc/compiler/ghc-inplace</filename>.</para> + + <para> Do <emphasis>NOT</emphasis> use + <filename>ghc/compiler/ghc</filename>, or + <filename>ghc/compiler/ghc-6.xx</filename>, as these are the + scripts intended for installation, and contain hard-wired paths + to the installed libraries, rather than the libraries in the + build tree.</para> + + <para>Happy can similarly be run from the build tree, using + <filename>happy/src/happy-inplace</filename>, and similarly for + Alex and Haddock.</para> + </sect2> + + <sect2> + <title>Fast Making</title> + + <indexterm><primary>fastmake</primary></indexterm> + <indexterm><primary>dependencies, omitting</primary></indexterm> + <indexterm><primary>FAST, makefile variable</primary></indexterm> + + <para>Sometimes the dependencies get in the way: if you've made + a small change to one file, and you're absolutely sure that it + won't affect anything else, but you know that + <command>make</command> is going to rebuild everything anyway, + the following hack may be useful:</para> + +<screen>$ gmake FAST=YES</screen> + + <para>This tells the make system to ignore dependencies and just + build what you tell it to. In other words, it's equivalent to + temporarily removing the <filename>.depend</filename> file in + the current directory (where <command>mkdependHS</command> and + friends store their dependency information).</para> + + <para>A bit of history: GHC used to come with a + <command>fastmake</command> script that did the above job, but + GNU make provides the features we need to do it without + resorting to a script. Also, we've found that fastmaking is + less useful since the advent of GHC's recompilation checker (see + the User's Guide section on "Separate Compilation").</para> + </sect2> + </sect1> + + <sect1 id="sec-makefile-arch"> + <title>The <filename>Makefile</filename> architecture</title> + <indexterm><primary>makefile architecture</primary></indexterm> + + <para><command>make</command> is great if everything + works—you type <command>gmake install</command> and lo! the + right things get compiled and installed in the right places. Our + goal is to make this happen often, but somehow it often doesn't; + instead some weird error message eventually emerges from the + bowels of a directory you didn't know existed.</para> + + <para>The purpose of this section is to give you a road-map to + help you figure out what is going right and what is going + wrong.</para> + + <sect2> + <title>Debugging</title> + + <para>Debugging <filename>Makefile</filename>s is something of a + black art, but here's a couple of tricks that we find + particularly useful. The following command allows you to see + the contents of any make variable in the context of the current + <filename>Makefile</filename>:</para> + +<screen>$ make show VALUE=HS_SRCS</screen> + + <para>where you can replace <literal>HS_SRCS</literal> with the + name of any variable you wish to see the value of.</para> + + <para>GNU make has a <option>-d</option> option which generates + a dump of the decision procedure used to arrive at a conclusion + about which files should be recompiled. Sometimes useful for + tracking down problems with superfluous or missing + recompilations.</para> + </sect2> + + <sect2> + <title>A small project</title> + + <para>To get started, let us look at the + <filename>Makefile</filename> for an imaginary small + <literal>fptools</literal> project, <literal>small</literal>. + Each project in <literal>fptools</literal> has its own directory + in <constant>FPTOOLS_TOP</constant>, so the + <literal>small</literal> project will have its own directory + <constant>FPOOLS_TOP/small/</constant>. Inside the + <filename>small/</filename> directory there will be a + <filename>Makefile</filename>, looking something like + this:</para> + +<indexterm><primary>Makefile, minimal</primary></indexterm> + +<programlisting># Makefile for fptools project "small" + +TOP = .. +include $(TOP)/mk/boilerplate.mk + +SRCS = $(wildcard *.lhs) $(wildcard *.c) +HS_PROG = small + +include $(TOP)/target.mk</programlisting> + + <para>this <filename>Makefile</filename> has three + sections:</para> + + <orderedlist> + <listitem> + <para>The first section includes +<footnote> +<para> +One of the most important +features of GNU <command>make</command> that we use is the ability for a <filename>Makefile</filename> to +include another named file, very like <command>cpp</command>'s <literal>#include</literal> +directive. +</para> +</footnote> + + a file of “boilerplate” code from the level + above (which in this case will be + <filename>FPTOOLS_TOP/mk/boilerplate.mk</filename><indexterm><primary>boilerplate.mk</primary></indexterm>). + As its name suggests, <filename>boilerplate.mk</filename> + consists of a large quantity of standard + <filename>Makefile</filename> code. We discuss this + boilerplate in more detail in <xref linkend="sec-boiler"/>. + <indexterm><primary>include, directive in + Makefiles</primary></indexterm> <indexterm><primary>Makefile + inclusion</primary></indexterm></para> + + <para>Before the <literal>include</literal> statement, you + must define the <command>make</command> variable + <constant>TOP</constant><indexterm><primary>TOP</primary></indexterm> + to be the directory containing the <filename>mk</filename> + directory in which the <filename>boilerplate.mk</filename> + file is. It is <emphasis>not</emphasis> OK to simply say</para> + +<programlisting>include ../mk/boilerplate.mk # NO NO NO</programlisting> + + + <para>Why? Because the <filename>boilerplate.mk</filename> + file needs to know where it is, so that it can, in turn, + <literal>include</literal> other files. (Unfortunately, + when an <literal>include</literal>d file does an + <literal>include</literal>, the filename is treated relative + to the directory in which <command>gmake</command> is being + run, not the directory in which the + <literal>include</literal>d sits.) In general, + <emphasis>every file <filename>foo.mk</filename> assumes + that + <filename>$(TOP)/mk/foo.mk</filename> + refers to itself.</emphasis> It is up to the + <filename>Makefile</filename> doing the + <literal>include</literal> to ensure this is the case.</para> + + <para>Files intended for inclusion in other + <filename>Makefile</filename>s are written to have the + following property: <emphasis>after + <filename>foo.mk</filename> is <literal>include</literal>d, + it leaves <constant>TOP</constant> containing the same value + as it had just before the <literal>include</literal> + statement</emphasis>. In our example, this invariant + guarantees that the <literal>include</literal> for + <filename>target.mk</filename> will look in the same + directory as that for <filename>boilerplate.mk</filename>.</para> + </listitem> + + <listitem> + <para> The second section defines the following standard + <command>make</command> variables: + <constant>SRCS</constant><indexterm><primary>SRCS</primary></indexterm> + (the source files from which is to be built), and + <constant>HS_PROG</constant><indexterm><primary>HS_PROG</primary></indexterm> + (the executable binary to be built). We will discuss in + more detail what the “standard variables” are, + and how they affect what happens, in <xref + linkend="sec-targets"/>.</para> + + <para>The definition for <constant>SRCS</constant> uses the + useful GNU <command>make</command> construct + <literal>$(wildcard $pat$)</literal><indexterm><primary>wildcard</primary></indexterm>, + which expands to a list of all the files matching the + pattern <literal>pat</literal> in the current directory. In + this example, <constant>SRCS</constant> is set to the list + of all the <filename>.lhs</filename> and + <filename>.c</filename> files in the directory. (Let's + suppose there is one of each, <filename>Foo.lhs</filename> + and <filename>Baz.c</filename>.)</para> + </listitem> + + <listitem> + <para>The last section includes a second file of standard + code, called + <filename>target.mk</filename><indexterm><primary>target.mk</primary></indexterm>. + It contains the rules that tell <command>gmake</command> how + to make the standard targets (<xref + linkend="sec-standard-targets"/>). Why, you ask, can't this + standard code be part of + <filename>boilerplate.mk</filename>? Good question. We + discuss the reason later, in <xref + linkend="sec-boiler-arch"/>.</para> + + <para>You do not <emphasis>have</emphasis> to + <literal>include</literal> the + <filename>target.mk</filename> file. Instead, you can write + rules of your own for all the standard targets. Usually, + though, you will find quite a big payoff from using the + canned rules in <filename>target.mk</filename>; the price + tag is that you have to understand what canned rules get + enabled, and what they do (<xref + linkend="sec-targets"/>).</para> + </listitem> + </orderedlist> + + <para>In our example <filename>Makefile</filename>, most of the + work is done by the two <literal>include</literal>d files. When + you say <command>gmake all</command>, the following things + happen:</para> + + <itemizedlist> + <listitem> + <para><command>gmake</command> figures out that the object + files are <filename>Foo.o</filename> and + <filename>Baz.o</filename>.</para> + </listitem> + + <listitem> + <para>It uses a boilerplate pattern rule to compile + <filename>Foo.lhs</filename> to <filename>Foo.o</filename> + using a Haskell compiler. (Which one? That is set in the + build configuration.)</para> + </listitem> + + <listitem> + <para>It uses another standard pattern rule to compile + <filename>Baz.c</filename> to <filename>Baz.o</filename>, + using a C compiler. (Ditto.)</para> + </listitem> + + <listitem> + <para>It links the resulting <filename>.o</filename> files + together to make <literal>small</literal>, using the Haskell + compiler to do the link step. (Why not use + <command>ld</command>? Because the Haskell compiler knows + what standard libraries to link in. How did + <command>gmake</command> know to use the Haskell compiler to + do the link, rather than the C compiler? Because we set the + variable <constant>HS_PROG</constant> rather than + <constant>C_PROG</constant>.)</para> + </listitem> + </itemizedlist> + + <para>All <filename>Makefile</filename>s should follow the above + three-section format.</para> + </sect2> + + <sect2> + <title>A larger project</title> + + <para>Larger projects are usually structured into a number of + sub-directories, each of which has its own + <filename>Makefile</filename>. (In very large projects, this + sub-structure might be iterated recursively, though that is + rare.) To give you the idea, here's part of the directory + structure for the (rather large) GHC project:</para> + +<programlisting>$(FPTOOLS_TOP)/ghc/ + Makefile + mk/ + boilerplate.mk + rules.mk + docs/ + Makefile + ...source files for documentation... + driver/ + Makefile + ...source files for driver... + compiler/ + Makefile + parser/...source files for parser... + renamer/...source files for renamer... + ...etc...</programlisting> + + <para>The sub-directories <filename>docs</filename>, + <filename>driver</filename>, <filename>compiler</filename>, and + so on, each contains a sub-component of GHC, and each has its + own <filename>Makefile</filename>. There must also be a + <filename>Makefile</filename> in + <filename>$(FPTOOLS_TOP)/ghc</filename>. + It does most of its work by recursively invoking + <command>gmake</command> on the <filename>Makefile</filename>s + in the sub-directories. We say that + <filename>ghc/Makefile</filename> is a <emphasis>non-leaf + <filename>Makefile</filename></emphasis>, because it does little + except organise its children, while the + <filename>Makefile</filename>s in the sub-directories are all + <emphasis>leaf <filename>Makefile</filename>s</emphasis>. (In + principle the sub-directories might themselves contain a + non-leaf <filename>Makefile</filename> and several + sub-sub-directories, but that does not happen in GHC.)</para> + + <para>The <filename>Makefile</filename> in + <filename>ghc/compiler</filename> is considered a leaf + <filename>Makefile</filename> even though the + <filename>ghc/compiler</filename> has sub-directories, because + these sub-directories do not themselves have + <filename>Makefile</filename>s in them. They are just used to + structure the collection of modules that make up GHC, but all + are managed by the single <filename>Makefile</filename> in + <filename>ghc/compiler</filename>.</para> + + <para>You will notice that <filename>ghc/</filename> also + contains a directory <filename>ghc/mk/</filename>. It contains + GHC-specific <filename>Makefile</filename> boilerplate code. + More precisely:</para> + + <itemizedlist> + <listitem> + <para><filename>ghc/mk/boilerplate.mk</filename> is included + at the top of <filename>ghc/Makefile</filename>, and of all + the leaf <filename>Makefile</filename>s in the + sub-directories. It in turn <literal>include</literal>s the + main boilerplate file + <filename>mk/boilerplate.mk</filename>.</para> + </listitem> + + <listitem> + <para><filename>ghc/mk/target.mk</filename> is + <literal>include</literal>d at the bottom of + <filename>ghc/Makefile</filename>, and of all the leaf + <filename>Makefile</filename>s in the sub-directories. It + in turn <literal>include</literal>s the file + <filename>mk/target.mk</filename>.</para> + </listitem> + </itemizedlist> + + <para>So these two files are the place to look for GHC-wide + customisation of the standard boilerplate.</para> + </sect2> + + <sect2 id="sec-boiler-arch"> + <title>Boilerplate architecture</title> + <indexterm><primary>boilerplate architecture</primary></indexterm> + + <para>Every <filename>Makefile</filename> includes a + <filename>boilerplate.mk</filename><indexterm><primary>boilerplate.mk</primary></indexterm> + file at the top, and + <filename>target.mk</filename><indexterm><primary>target.mk</primary></indexterm> + file at the bottom. In this section we discuss what is in these + files, and why there have to be two of them. In general:</para> + + <itemizedlist> + <listitem> + <para><filename>boilerplate.mk</filename> consists of:</para> + + <itemizedlist> + <listitem> + <para><emphasis>Definitions of millions of + <command>make</command> variables</emphasis> that + collectively specify the build configuration. Examples: + <constant>HC_OPTS</constant><indexterm><primary>HC_OPTS</primary></indexterm>, + the options to feed to the Haskell compiler; + <constant>NoFibSubDirs</constant><indexterm><primary>NoFibSubDirs</primary></indexterm>, + the sub-directories to enable within the + <literal>nofib</literal> project; + <constant>GhcWithHc</constant><indexterm><primary>GhcWithHc</primary></indexterm>, + the name of the Haskell compiler to use when compiling + GHC in the <literal>ghc</literal> project.</para> + </listitem> + + <listitem> + <para><emphasis>Standard pattern rules</emphasis> that + tell <command>gmake</command> how to construct one file + from another.</para> + </listitem> + </itemizedlist> + + <para><filename>boilerplate.mk</filename> needs to be + <literal>include</literal>d at the <emphasis>top</emphasis> + of each <filename>Makefile</filename>, so that the user can + replace the boilerplate definitions or pattern rules by + simply giving a new definition or pattern rule in the + <filename>Makefile</filename>. <command>gmake</command> + simply takes the last definition as the definitive one.</para> + + <para>Instead of <emphasis>replacing</emphasis> boilerplate + definitions, it is also quite common to + <emphasis>augment</emphasis> them. For example, a + <filename>Makefile</filename> might say:</para> + +<programlisting>SRC_HC_OPTS += -O</programlisting> + + <para>thereby adding “<option>-O</option>” to + the end of + <constant>SRC_HC_OPTS</constant><indexterm><primary>SRC_HC_OPTS</primary></indexterm>.</para> + </listitem> + + <listitem> + <para><filename>target.mk</filename> contains + <command>make</command> rules for the standard targets + described in <xref linkend="sec-standard-targets"/>. These + rules are selectively included, depending on the setting of + certain <command>make</command> variables. These variables + are usually set in the middle section of the + <filename>Makefile</filename> between the two + <literal>include</literal>s.</para> + + <para><filename>target.mk</filename> must be included at the + end (rather than being part of + <filename>boilerplate.mk</filename>) for several tiresome + reasons:</para> + + <itemizedlist> + <listitem> + + <para><command>gmake</command> commits target and + dependency lists earlier than it should. For example, + <filename>target.mk</filename> has a rule that looks + like this:</para> + +<programlisting>$(HS_PROG) : $(OBJS) + $(HC) $(LD_OPTS) $< -o $@</programlisting> + + <para>If this rule was in + <filename>boilerplate.mk</filename> then + <constant>$(HS_PROG)</constant><indexterm><primary>HS_PROG</primary></indexterm> + and + <constant>$(OBJS)</constant><indexterm><primary>OBJS</primary></indexterm> + would not have their final values at the moment + <command>gmake</command> encountered the rule. Alas, + <command>gmake</command> takes a snapshot of their + current values, and wires that snapshot into the rule. + (In contrast, the commands executed when the rule + “fires” are only substituted at the moment + of firing.) So, the rule must follow the definitions + given in the <filename>Makefile</filename> itself.</para> + </listitem> + + <listitem> + <para>Unlike pattern rules, ordinary rules cannot be + overriden or replaced by subsequent rules for the same + target (at least, not without an error message). + Including ordinary rules in + <filename>boilerplate.mk</filename> would prevent the + user from writing rules for specific targets in specific + cases.</para> + </listitem> + + <listitem> + <para>There are a couple of other reasons I've + forgotten, but it doesn't matter too much.</para> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> + </sect2> + + <sect2 id="sec-boiler"> + <title>The main <filename>mk/boilerplate.mk</filename> file</title> + <indexterm><primary>boilerplate.mk</primary></indexterm> + + <para>If you look at + <filename>$(FPTOOLS_TOP)/mk/boilerplate.mk</filename> + you will find that it consists of the following sections, each + held in a separate file:</para> + + <variablelist> + <varlistentry> + <term><filename>config.mk</filename> + <indexterm><primary>config.mk</primary></indexterm> + </term> + <listitem> + <para>is the build configuration file we discussed at + length in <xref linkend="sec-build-config"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>paths.mk</filename> + <indexterm><primary>paths.mk</primary></indexterm> + </term> + <listitem> + <para>defines <command>make</command> variables for + pathnames and file lists. This file contains code for + automatically compiling lists of source files and deriving + lists of object files from those. The results can be + overriden in the <filename>Makefile</filename>, but in + most cases the automatic setup should do the right + thing.</para> + + <para>The following variables may be set in the + <filename>Makefile</filename> to affect how the automatic + source file search is done:</para> + + <variablelist> + <varlistentry> + <term><literal>ALL_DIRS</literal> + <indexterm><primary><literal>ALL_DIRS</literal></primary></indexterm> + </term> + <listitem> + <para>Set to a list of directories to search in + addition to the current directory for source + files.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>EXCLUDED_SRCS</literal> + <indexterm><primary><literal>EXCLUDED_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>Set to a list of source files (relative to the + current directory) to omit from the automatic + search. The source searching machinery is clever + enough to know that if you exclude a source file + from which other sources are derived, then the + derived sources should also be excluded. For + example, if you set <literal>EXCLUDED_SRCS</literal> + to include <filename>Foo.y</filename>, then + <filename>Foo.hs</filename> will also be + excluded.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>EXTRA_SRCS</literal> + <indexterm><primary><literal>EXTRA_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>Set to a list of extra source files (perhaps + in directories not listed in + <literal>ALL_DIRS</literal>) that should be + considered.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>The results of the automatic source file search are + placed in the following make variables:</para> + + <variablelist> + <varlistentry> + <term><literal>SRCS</literal> + <indexterm><primary><literal>SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>All source files found, sorted and without + duplicates, including those which might not exist + yet but will be derived from other existing sources. + <literal>SRCS</literal> <emphasis>can</emphasis> be + overriden if necessary, in which case the variables + below will follow suit.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HS_SRCS</literal> + <indexterm><primary><literal>HS_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>all Haskell source files in the current + directory, including those derived from other source + files (eg. Happy sources also give rise to Haskell + sources).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HS_OBJS</literal> + <indexterm><primary><literal>HS_OBJS</literal></primary></indexterm> + </term> + <listitem> + <para>Object files derived from + <literal>HS_SRCS</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HS_IFACES</literal> + <indexterm><primary><literal>HS_IFACES</literal></primary></indexterm> + </term> + <listitem> + <para>Interface files (<literal>.hi</literal> files) + derived from <literal>HS_SRCS</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>C_SRCS</literal> + <indexterm><primary><literal>C_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>All C source files found.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>C_OBJS</literal> + <indexterm><primary><literal>C_OBJS</literal></primary></indexterm> + </term> + <listitem> + <para>Object files derived from + <literal>C_SRCS</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>SCRIPT_SRCS</literal> + <indexterm><primary><literal>SCRIPT_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>All script source files found + (<literal>.lprl</literal> files).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>SCRIPT_OBJS</literal> + <indexterm><primary><literal>SCRIPT_OBJS</literal></primary></indexterm> + </term> + <listitem> + <para><quote>object</quote> files derived from + <literal>SCRIPT_SRCS</literal> + (<literal>.prl</literal> files).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HSC_SRCS</literal> + <indexterm><primary><literal>HSC_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>All <literal>hsc2hs</literal> source files + (<literal>.hsc</literal> files).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>HAPPY_SRCS</literal> + <indexterm><primary><literal>HAPPY_SRCS</literal></primary></indexterm> + </term> + <listitem> + <para>All <literal>happy</literal> source files + (<literal>.y</literal> or <literal>.hy</literal> files).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>OBJS</literal> + <indexterm><primary>OBJS</primary></indexterm> + </term> + <listitem> + <para>the concatenation of + <literal>$(HS_OBJS)</literal>, + <literal>$(C_OBJS)</literal>, and + <literal>$(SCRIPT_OBJS)</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Any or all of these definitions can easily be + overriden by giving new definitions in your + <filename>Makefile</filename>.</para> + + <para>What, exactly, does <filename>paths.mk</filename> + consider a <quote>source file</quote> to be? It's based + on the file's suffix (e.g. <filename>.hs</filename>, + <filename>.lhs</filename>, <filename>.c</filename>, + <filename>.hy</filename>, etc), but this is the kind of + detail that changes, so rather than enumerate the source + suffices here the best thing to do is to look in + <filename>paths.mk</filename>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>opts.mk</filename> + <indexterm><primary>opts.mk</primary></indexterm> + </term> + <listitem> + <para>defines <command>make</command> variables for option + strings to pass to each program. For example, it defines + <constant>HC_OPTS</constant><indexterm><primary>HC_OPTS</primary></indexterm>, + the option strings to pass to the Haskell compiler. See + <xref linkend="sec-suffix"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>suffix.mk</filename> + <indexterm><primary>suffix.mk</primary></indexterm> + </term> + <listitem> + <para>defines standard pattern rules—see <xref + linkend="sec-suffix"/>.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Any of the variables and pattern rules defined by the + boilerplate file can easily be overridden in any particular + <filename>Makefile</filename>, because the boilerplate + <literal>include</literal> comes first. Definitions after this + <literal>include</literal> directive simply override the default + ones in <filename>boilerplate.mk</filename>.</para> + </sect2> + + <sect2 id="sec-platforms"> + <title>Platform settings</title> + <indexterm><primary>Platform settings</primary> + </indexterm> + + <para>There are three platforms of interest when building GHC:</para> + + <variablelist> + <varlistentry> + <term>The <emphasis>build</emphasis> platform</term> + <listitem> + <para>The platform on which we are doing this build.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>The <emphasis>host</emphasis> platform</term> + <listitem> + <para>The platform on which these binaries will run.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>The <emphasis>target</emphasis> platform</term> + <listitem> + <para>The platform for which this compiler will generate code.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>These platforms are set when running the + <literal>configure</literal> script, using the + <option>--build</option>, <option>--host</option>, and + <option>--target</option> options. The <filename>mk/config.mk</filename> + file defines several symbols related to the platform settings (see + <filename>mk/config.mk</filename> for details).</para> + + <para>We don't currently support build & host being different, because + the build process creates binaries that are both run during the build, + and also installed.</para> + + <para>If host and target are different, then we are building a + cross-compiler. For GHC, this means a compiler + which will generate intermediate .hc files to port to the target + architecture for bootstrapping. The libraries and stage 2 compiler + will be built as HC files for the target system (see <xref + linkend="sec-porting-ghc" /> for details.</para> + + <para>More details on when to use BUILD, HOST or TARGET can be found in + the comments in <filename>config.mk</filename>.</para> + </sect2> + + <sect2 id="sec-suffix"> + <title>Pattern rules and options</title> + <indexterm><primary>Pattern rules</primary></indexterm> + + <para>The file + <filename>suffix.mk</filename><indexterm><primary>suffix.mk</primary></indexterm> + defines standard <emphasis>pattern rules</emphasis> that say how + to build one kind of file from another, for example, how to + build a <filename>.o</filename> file from a + <filename>.c</filename> file. (GNU <command>make</command>'s + <emphasis>pattern rules</emphasis> are more powerful and easier + to use than Unix <command>make</command>'s <emphasis>suffix + rules</emphasis>.)</para> + + <para>Almost all the rules look something like this:</para> + +<programlisting>%.o : %.c + $(RM) $@ + $(CC) $(CC_OPTS) -c $< -o $@</programlisting> + + <para>Here's how to understand the rule. It says that + <emphasis>something</emphasis><filename>.o</filename> (say + <filename>Foo.o</filename>) can be built from + <emphasis>something</emphasis><filename>.c</filename> + (<filename>Foo.c</filename>), by invoking the C compiler (path + name held in <constant>$(CC)</constant>), passing to it + the options <constant>$(CC_OPTS)</constant> and + the rule's dependent file of the rule + <literal>$<</literal> (<filename>Foo.c</filename> in + this case), and putting the result in the rule's target + <literal>$@</literal> (<filename>Foo.o</filename> in this + case).</para> + + <para>Every program is held in a <command>make</command> + variable defined in <filename>mk/config.mk</filename>—look + in <filename>mk/config.mk</filename> for the complete list. One + important one is the Haskell compiler, which is called + <constant>$(HC)</constant>.</para> + + <para>Every program's options are are held in a + <command>make</command> variables called + <constant><prog>_OPTS</constant>. the + <constant><prog>_OPTS</constant> variables are + defined in <filename>mk/opts.mk</filename>. Almost all of them + are defined like this:</para> + +<programlisting>CC_OPTS = \ + $(SRC_CC_OPTS) $(WAY$(_way)_CC_OPTS) $($*_CC_OPTS) $(EXTRA_CC_OPTS)</programlisting> + + <para>The four variables from which + <constant>CC_OPTS</constant> is built have the following + meaning:</para> + + <variablelist> + <varlistentry> + <term><constant>SRC_CC_OPTS</constant><indexterm><primary>SRC_CC_OPTS</primary></indexterm>:</term> + <listitem> + <para>options passed to all C compilations.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>WAY_<way>_CC_OPTS</constant>:</term> + <listitem> + <para>options passed to C compilations for way + <literal><way></literal>. For example, + <constant>WAY_mp_CC_OPTS</constant> + gives options to pass to the C compiler when compiling way + <literal>mp</literal>. The variable + <constant>WAY_CC_OPTS</constant> holds + options to pass to the C compiler when compiling the + standard way. (<xref linkend="sec-ways"/> dicusses + multi-way compilation.)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant><module>_CC_OPTS</constant>:</term> + <listitem> + <para>options to pass to the C compiler that are specific + to module <literal><module></literal>. For example, + <constant>SMap_CC_OPTS</constant> gives the + specific options to pass to the C compiler when compiling + <filename>SMap.c</filename>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>EXTRA_CC_OPTS</constant><indexterm><primary>EXTRA_CC_OPTS</primary></indexterm>:</term> + <listitem> + <para>extra options to pass to all C compilations. This + is intended for command line use, thus:</para> + +<screen>$ gmake libHS.a EXTRA_CC_OPTS="-v"</screen> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="sec-targets"> + <title>The main <filename>mk/target.mk</filename> file</title> + <indexterm><primary>target.mk</primary></indexterm> + + <para><filename>target.mk</filename> contains canned rules for + all the standard targets described in <xref + linkend="sec-standard-targets"/>. It is complicated by the fact + that you don't want all of these rules to be active in every + <filename>Makefile</filename>. Rather than have a plethora of + tiny files which you can include selectively, there is a single + file, <filename>target.mk</filename>, which selectively includes + rules based on whether you have defined certain variables in + your <filename>Makefile</filename>. This section explains what + rules you get, what variables control them, and what the rules + do. Hopefully, you will also get enough of an idea of what is + supposed to happen that you can read and understand any weird + special cases yourself.</para> + + <variablelist> + <varlistentry> + <term><constant>HS_PROG</constant><indexterm><primary>HS_PROG</primary></indexterm>.</term> + <listitem> + <para>If <constant>HS_PROG</constant> is defined, + you get rules with the following targets:</para> + + <variablelist> + <varlistentry> + <term><filename>HS_PROG</filename><indexterm><primary>HS_PROG</primary></indexterm></term> + <listitem> + <para>itself. This rule links + <constant>$(OBJS)</constant> with the Haskell + runtime system to get an executable called + <constant>$(HS_PROG)</constant>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>install</literal><indexterm><primary>install</primary></indexterm></term> + <listitem> + <para>installs + <constant>$(HS_PROG)</constant> in + <constant>$(bindir)</constant>.</para> + </listitem> + </varlistentry> + </variablelist> + + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>C_PROG</constant><indexterm><primary>C_PROG</primary></indexterm></term> + <listitem> + <para>is similar to <constant>HS_PROG</constant>, + except that the link step links + <constant>$(C_OBJS)</constant> with the C + runtime system.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>LIBRARY</constant><indexterm><primary>LIBRARY</primary></indexterm></term> + <listitem> + <para>is similar to <constant>HS_PROG</constant>, + except that it links + <constant>$(LIB_OBJS)</constant> to make the + library archive <constant>$(LIBRARY)</constant>, + and <literal>install</literal> installs it in + <constant>$(libdir)</constant>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>LIB_DATA</constant><indexterm><primary>LIB_DATA</primary></indexterm></term> + <listitem> + <para>…</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>LIB_EXEC</constant><indexterm><primary>LIB_EXEC</primary></indexterm></term> + <listitem> + <para>…</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><constant>HS_SRCS</constant><indexterm><primary>HS_SRCS</primary></indexterm>, <constant>C_SRCS</constant><indexterm><primary>C_SRCS</primary></indexterm>.</term> + <listitem> + <para>If <constant>HS_SRCS</constant> is defined + and non-empty, a rule for the target + <literal>depend</literal> is included, which generates + dependency information for Haskell programs. Similarly + for <constant>C_SRCS</constant>.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>All of these rules are “double-colon” rules, + thus</para> + +<programlisting>install :: $(HS_PROG) + ...how to install it...</programlisting> + + <para>GNU <command>make</command> treats double-colon rules as + separate entities. If there are several double-colon rules for + the same target it takes each in turn and fires it if its + dependencies say to do so. This means that you can, for + example, define both <constant>HS_PROG</constant> and + <constant>LIBRARY</constant>, which will generate two rules for + <literal>install</literal>. When you type <command>gmake + install</command> both rules will be fired, and both the program + and the library will be installed, just as you wanted.</para> + </sect2> + + <sect2 id="sec-subdirs"> + <title>Recursion</title> + <indexterm><primary>recursion, in makefiles</primary></indexterm> + <indexterm><primary>Makefile, recursing into subdirectories</primary></indexterm> + + <para>In leaf <filename>Makefile</filename>s the variable + <constant>SUBDIRS</constant><indexterm><primary>SUBDIRS</primary></indexterm> + is undefined. In non-leaf <filename>Makefile</filename>s, + <constant>SUBDIRS</constant> is set to the list of + sub-directories that contain subordinate + <filename>Makefile</filename>s. <emphasis>It is up to you to + set <constant>SUBDIRS</constant> in the + <filename>Makefile</filename>.</emphasis> There is no automation + here—<constant>SUBDIRS</constant> is too important to + automate.</para> + + <para>When <constant>SUBDIRS</constant> is defined, + <filename>target.mk</filename> includes a rather neat rule for + the standard targets (<xref linkend="sec-standard-targets"/> that + simply invokes <command>make</command> recursively in each of + the sub-directories.</para> + + <para><emphasis>These recursive invocations are guaranteed to + occur in the order in which the list of directories is specified + in <constant>SUBDIRS</constant>. </emphasis>This guarantee can + be important. For example, when you say <command>gmake + boot</command> it can be important that the recursive invocation + of <command>make boot</command> is done in one sub-directory + (the include files, say) before another (the source files). + Generally, put the most independent sub-directory first, and the + most dependent last.</para> + </sect2> + + <sect2 id="sec-ways"> + <title>Way management</title> + <indexterm><primary>way management</primary></indexterm> + + <para>We sometimes want to build essentially the same system in + several different “ways”. For example, we want to build GHC's + <literal>Prelude</literal> libraries with and without profiling, + so that there is an appropriately-built library archive to link + with when the user compiles his program. It would be possible + to have a completely separate build tree for each such “way”, + but it would be horribly bureaucratic, especially since often + only parts of the build tree need to be constructed in multiple + ways.</para> + + <para>Instead, the + <filename>target.mk</filename><indexterm><primary>target.mk</primary></indexterm> + contains some clever magic to allow you to build several + versions of a system; and to control locally how many versions + are built and how they differ. This section explains the + magic.</para> + + <para>The files for a particular way are distinguished by + munging the suffix. The <quote>normal way</quote> is always + built, and its files have the standard suffices + <filename>.o</filename>, <filename>.hi</filename>, and so on. + In addition, you can build one or more extra ways, each + distinguished by a <emphasis>way tag</emphasis>. The object + files and interface files for one of these extra ways are + distinguished by their suffix. For example, way + <literal>mp</literal> has files + <filename>.mp_o</filename> and + <filename>.mp_hi</filename>. Library archives have their + way tag the other side of the dot, for boring reasons; thus, + <filename>libHS_mp.a</filename>.</para> + + <para>A <command>make</command> variable called + <constant>way</constant> holds the current way tag. + <emphasis><constant>way</constant> is only ever set on the + command line of <command>gmake</command></emphasis> (usually in + a recursive invocation of <command>gmake</command> by the + system). It is never set inside a + <filename>Makefile</filename>. So it is a global constant for + any one invocation of <command>gmake</command>. Two other + <command>make</command> variables, + <constant>way_</constant> and + <constant>_way</constant> are immediately derived from + <constant>$(way)</constant> and never altered. If + <constant>way</constant> is not set, then neither are + <constant>way_</constant> and + <constant>_way</constant>, and the invocation of + <command>make</command> will build the <quote>normal + way</quote>. If <constant>way</constant> is set, then the other + two variables are set in sympathy. For example, if + <constant>$(way)</constant> is “<literal>mp</literal>”, + then <constant>way_</constant> is set to + “<literal>mp_</literal>” and + <constant>_way</constant> is set to + “<literal>_mp</literal>”. These three variables are + then used when constructing file names.</para> + + <para>So how does <command>make</command> ever get recursively + invoked with <constant>way</constant> set? There are two ways + in which this happens:</para> + + <itemizedlist> + <listitem> + <para>For some (but not all) of the standard targets, when + in a leaf sub-directory, <command>make</command> is + recursively invoked for each way tag in + <constant>$(WAYS)</constant>. You set + <constant>WAYS</constant> in the + <filename>Makefile</filename> to the list of way tags you + want these targets built for. The mechanism here is very + much like the recursive invocation of + <command>make</command> in sub-directories (<xref + linkend="sec-subdirs"/>). It is up to you to set + <constant>WAYS</constant> in your + <filename>Makefile</filename>; this is how you control what + ways will get built.</para> + </listitem> + + <listitem> + <para>For a useful collection of targets (such as + <filename>libHS_mp.a</filename>, + <filename>Foo.mp_o</filename>) there is a rule which + recursively invokes <command>make</command> to make the + specified target, setting the <constant>way</constant> + variable. So if you say <command>gmake + Foo.mp_o</command> you should see a recursive + invocation <command>gmake Foo.mp_o way=mp</command>, + and <emphasis>in this recursive invocation the pattern rule + for compiling a Haskell file into a <filename>.o</filename> + file will match</emphasis>. The key pattern rules (in + <filename>suffix.mk</filename>) look like this: + +<programlisting>%.$(way_)o : %.lhs + $(HC) $(HC_OPTS) $< -o $@</programlisting> + + Neat, eh?</para> + </listitem> + + <listitem> + <para>You can invoke <command>make</command> with a + particular <literal>way</literal> setting yourself, in order + to build files related to a particular + <literal>way</literal> in the current directory. eg. + +<screen>$ make way=p</screen> + + will build files for the profiling way only in the current + directory. </para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>When the canned rule isn't right</title> + + <para>Sometimes the canned rule just doesn't do the right thing. + For example, in the <literal>nofib</literal> suite we want the + link step to print out timing information. The thing to do here + is <emphasis>not</emphasis> to define + <constant>HS_PROG</constant> or + <constant>C_PROG</constant>, and instead define a special + purpose rule in your own <filename>Makefile</filename>. By + using different variable names you will avoid the canned rules + being included, and conflicting with yours.</para> + </sect2> + </sect1> + + <sect1 id="building-docs"> + <title>Building the documentation</title> + + <sect2 id="pre-supposed-doc-tools"> + <title>Tools for building the Documentation</title> + + <para>The following additional tools are required if you want to + format the documentation that comes with the + <literal>fptools</literal> projects:</para> + + <variablelist> + <varlistentry> + <term>DocBook + <indexterm><primary>pre-supposed: DocBook</primary></indexterm> + <indexterm><primary>DocBook, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>Much of our documentation is written in DocBook XML, instructions + on installing and configuring the DocBook tools are below.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>TeX + <indexterm><primary>pre-supposed: TeX</primary></indexterm> + <indexterm><primary>TeX, pre-supposed</primary></indexterm> + </term> + <listitem> + <para>A decent TeX distribution is required if you want to + produce printable documentation. We recomment teTeX, + which includes just about everything you need.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Haddock + <indexterm><primary>Haddock</primary></indexterm> + </term> + <listitem> + <para>Haddock is a Haskell documentation tool that we use + for automatically generating documentation from the + library source code. It is an <literal>fptools</literal> + project in itself. To build documentation for the + libraries (<literal>fptools/libraries</literal>) you + should check out and build Haddock in + <literal>fptools/haddock</literal>. Haddock requires GHC + to build.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2> + <title>Installing the DocBook tools</title> + + <sect3> + <title>Installing the DocBook tools on Linux</title> + + <para>If you're on a recent RedHat (7.0+) or SuSE (8.1+) system, + you probably have working DocBook tools already installed. The + configure script should detect your setup and you're away.</para> + + <para>If you don't have DocBook tools installed, and you are + using a system that can handle RPM packages, you can use <ulink + url="http://rpmfind.net/">Rpmfind.net</ulink> to find suitable + packages for your system. Search for the packages + <literal>docbook-dtd</literal>, + <literal>docbook-xsl-stylesheets</literal>, + <literal>libxslt</literal>, + <literal>libxml2</literal>, + <literal>fop</literal>, + <literal>xmltex</literal>, and + <literal>dvips</literal>.</para> + </sect3> + + <sect3> + <title>Installing DocBook on FreeBSD</title> + + <para>On FreeBSD systems, the easiest way to get DocBook up + and running is to install it from the ports tree or a + pre-compiled package (packages are available from your local + FreeBSD mirror site).</para> + + <para>To use the ports tree, do this: +<screen>$ cd /usr/ports/textproc/docproj +$ make install</screen> + This installs the FreeBSD documentation project tools, which + includes everything needed to format the GHC + documentation.</para> + </sect3> + + <sect3> + <title>Installing from binaries on Windows</title> + + <para>Probably the fastest route to a working DocBook environment on + Windows is to install <ulink url="http://www.cygwin.com/">Cygwin</ulink> + with the complete <literal>Doc</literal> category. If you are using + <ulink url="http://www.mingw.org/">MinGW</ulink> for compilation, you + have to help <command>configure</command> a little bit: Set the + environment variables <envar>XmllintCmd</envar> and + <envar>XsltprocCmd</envar> to the paths of the Cygwin executables + <command>xmllint</command> and <command>xsltproc</command>, + respectively, and set <envar>fp_cv_dir_docbook_xsl</envar> to the path + of the directory where the XSL stylesheets are installed, + e.g. <filename>c:/cygwin/usr/share/docbook-xsl</filename>. + </para> + + <para>If you want to build HTML Help, you have to install the + <ulink url="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/htmlhelp/html/hworiHTMLHelpStartPage.asp">HTML Help SDK</ulink>, + too, and make sure that <command>hhc</command> is in your <envar>PATH</envar>.</para> + </sect3> + + </sect2> + + <sect2> + <title>Configuring the DocBook tools</title> + + <para>Once the DocBook tools are installed, the configure script + will detect them and set up the build system accordingly. If you + have a system that isn't supported, let us know, and we'll try + to help.</para> + </sect2> + + <sect2> + <title>Building the documentation</title> + + <para>To build documentation in a certain format, you can + say, for example,</para> + +<screen>$ make html</screen> + + <para>to build HTML documentation below the current directory. + The available formats are: <literal>dvi</literal>, + <literal>ps</literal>, <literal>pdf</literal>, + <literal>html</literal>, and <literal>rtf</literal>. Note that + not all documentation can be built in all of these formats: HTML + documentation is generally supported everywhere, and DocBook + documentation might support the other formats (depending on what + other tools you have installed).</para> + + <para>All of these targets are recursive; that is, saying + <literal>make html</literal> will make HTML docs for all the + documents recursively below the current directory.</para> + + <para>Because there are many different formats that the DocBook + documentation can be generated in, you have to select which ones + you want by setting the <literal>XMLDocWays</literal> variable + to a list of them. For example, in + <filename>build.mk</filename> you might have a line:</para> + +<screen>XMLDocWays = html ps</screen> + + <para>This will cause the documentation to be built in the requested + formats as part of the main build (the default is not to build + any documentation at all).</para> + </sect2> + + <sect2> + <title>Installing the documentation</title> + + <para>To install the documentation, use:</para> + +<screen>$ make install-docs</screen> + + <para>This will install the documentation into + <literal>$(datadir)</literal> (which defaults to + <literal>$(prefix)/share</literal>). The exception is HTML + documentation, which goes into + <literal>$(datadir)/html</literal>, to keep things tidy.</para> + + <para>Note that unless you set <literal>$(XMLDocWays)</literal> + to a list of formats, the <literal>install-docs</literal> target + won't do anything for DocBook XML documentation.</para> + </sect2> + + </sect1> + + + <sect1 id="sec-porting-ghc"> + <title>Porting GHC</title> + + <para>This section describes how to port GHC to a currenly + unsupported platform. There are two distinct + possibilities:</para> + + <itemizedlist> + <listitem> + <para>The hardware architecture for your system is already + supported by GHC, but you're running an OS that isn't + supported (or perhaps has been supported in the past, but + currently isn't). This is the easiest type of porting job, + but it still requires some careful bootstrapping. Proceed to + <xref linkend="sec-booting-from-hc"/>.</para> + </listitem> + + <listitem> + <para>Your system's hardware architecture isn't supported by + GHC. This will be a more difficult port (though by comparison + perhaps not as difficult as porting gcc). Proceed to <xref + linkend="unregisterised-porting"/>.</para> + </listitem> + </itemizedlist> + + <sect2 id="sec-booting-from-hc"> + <title>Booting/porting from C (<filename>.hc</filename>) files</title> + + <indexterm><primary>building GHC from .hc files</primary></indexterm> + <indexterm><primary>booting GHC from .hc files</primary></indexterm> + <indexterm><primary>porting GHC</primary></indexterm> + + <para>Bootstrapping GHC on a system without GHC already + installed is achieved by taking the intermediate C files (known + as HC files) from another GHC compilation, compiling them using gcc to + get a working GHC.</para> + + <para><emphasis>NOTE: GHC versions 5.xx were hard to bootstrap + from C. We recommend using GHC 6.0.1 or + later.</emphasis></para> + + <para>HC files are platform-dependent, so you have to get a set + that were generated on <emphasis>the same platform</emphasis>. There + may be some supplied on the GHC download page, otherwise you'll have to + compile some up yourself, or start from + <emphasis>unregisterised</emphasis> HC files - see <xref + linkend="unregisterised-porting"/>.</para> + + <para>The following steps should result in a working GHC build + with full libraries:</para> + + <itemizedlist> + <listitem> + <para>Unpack the HC files on top of a fresh source tree + (make sure the source tree version matches the version of + the HC files <emphasis>exactly</emphasis>!). This will + place matching <filename>.hc</filename> files next to the + corresponding Haskell source (<filename>.hs</filename> or + <filename>.lhs</filename>) in the compiler subdirectory + <filename>ghc/compiler</filename> and in the libraries + (subdirectories of + <literal>libraries</literal>).</para> + </listitem> + + <listitem> + <para>The actual build process is fully automated by the + <filename>hc-build</filename> script located in the + <filename>distrib</filename> directory. If you eventually + want to install GHC into the directory + <replaceable>dir</replaceable>, the following + command will execute the whole build process (it won't + install yet):</para> + +<screen>$ distrib/hc-build --prefix=<replaceable>dir</replaceable></screen> +<indexterm><primary>--hc-build</primary></indexterm> + + <para>By default, the installation directory is + <filename>/usr/local</filename>. If that is what you want, + you may omit the argument to <filename>hc-build</filename>. + Generally, any option given to <filename>hc-build</filename> + is passed through to the configuration script + <filename>configure</filename>. If + <filename>hc-build</filename> successfully completes the + build process, you can install the resulting system, as + normal, with</para> + +<screen>$ make install</screen> + </listitem> + </itemizedlist> + </sect2> + + <sect2 id="unregisterised-porting"> + <title>Porting GHC to a new architecture</title> + + <para>The first step in porting to a new architecture is to get + an <firstterm>unregisterised</firstterm> build working. An + unregisterised build is one that compiles via vanilla C only. + By contrast, a registerised build uses the following + architecture-specific hacks for speed:</para> + + <itemizedlist> + <listitem> + <para>Global register variables: certain abstract machine + <quote>registers</quote> are mapped to real machine + registers, depending on how many machine registers are + available (see + <filename>ghc/includes/MachRegs.h</filename>).</para> + </listitem> + + <listitem> + <para>Assembly-mangling: when compiling via C, we feed the + assembly generated by gcc though a Perl script known as the + <firstterm>mangler</firstterm> (see + <filename>ghc/driver/mangler/ghc-asm.lprl</filename>). The + mangler rearranges the assembly to support tail-calls and + various other optimisations.</para> + </listitem> + </itemizedlist> + + <para>In an unregisterised build, neither of these hacks are + used — the idea is that the C code generated by the + compiler should compile using gcc only. The lack of these + optimisations costs about a factor of two in performance, but + since unregisterised compilation is usually just a step on the + way to a full registerised port, we don't mind too much.</para> + + <para>Notes on GHC portability in general: we've tried to stick + to writing portable code in most parts of the system, so it + should compile on any POSIXish system with gcc, but in our + experience most systems differ from the standards in one way or + another. Deal with any problems as they arise - if you get + stuck, ask the experts on + <email>glasgow-haskell-users@haskell.org</email>.</para> + + <para>Lots of useful information about the innards of GHC is + available in the <ulink + url="http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/">GHC + Commentary</ulink>, which might be helpful if you run into some + code which needs tweaking for your system.</para> + + <sect3> + <title>Cross-compiling to produce an unregisterised GHC</title> + + <para>NOTE! These instructions apply to GHC 6.4 and (hopefully) + later. If you need instructions for an earlier version of GHC, try + to get hold of the version of this document that was current at the + time. It should be available from the appropriate download page on + the <ulink + url="http://www.haskell.org/ghc/">GHC homepage</ulink>.</para> + + <para>In this section, we explain how to bootstrap GHC on a + new platform, using unregisterised intermediate C files. We + haven't put a great deal of effort into automating this + process, for two reasons: it is done very rarely, and the + process usually requires human intervention to cope with minor + porting issues anyway.</para> + + <para>The following step-by-step instructions should result in + a fully working, albeit unregisterised, GHC. Firstly, you + need a machine that already has a working GHC (we'll call this + the <firstterm>host</firstterm> machine), in order to + cross-compile the intermediate C files that we will use to + bootstrap the compiler on the <firstterm>target</firstterm> + machine.</para> + + <itemizedlist> + <listitem> + <para>On the target machine:</para> + + <itemizedlist> + <listitem> + <para>Unpack a source tree (preferably a released + version). We will call the path to the root of this + tree <replaceable>T</replaceable>.</para> + </listitem> + + <listitem> +<screen>$ cd <replaceable>T</replaceable> +$ ./configure --enable-hc-boot --enable-hc-boot-unregisterised</screen> + + <para>You might need to update + <filename>configure.in</filename> to recognise the new + architecture, and re-generate + <filename>configure</filename> with + <literal>autoreconf</literal>.</para> + </listitem> + + <listitem> +<screen>$ cd <replaceable>T</replaceable>/ghc/includes +$ make</screen> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para>On the host machine:</para> + + <itemizedlist> + <listitem> + <para>Unpack a source tree (same released version). Call + this directory <replaceable>H</replaceable>.</para> + </listitem> + + <listitem> +<screen>$ cd <replaceable>H</replaceable> +$ ./configure</screen> + </listitem> + + <listitem> + <para>Create + <filename><replaceable>H</replaceable>/mk/build.mk</filename>, + with the following contents:</para> + +<programlisting>GhcUnregisterised = YES +GhcLibHcOpts = -O -fvia-C -keep-hc-files +GhcRtsHcOpts = -keep-hc-files +GhcLibWays = +SplitObjs = NO +GhcWithNativeCodeGen = NO +GhcWithInterpreter = NO +GhcStage1HcOpts = -O +GhcStage2HcOpts = -O -fvia-C -keep-hc-files +SRC_HC_OPTS += -H32m +GhcBootLibs = YES</programlisting> + </listitem> + + <listitem> + <para>Edit + <filename><replaceable>H</replaceable>/mk/config.mk</filename>:</para> + <itemizedlist> + <listitem> + <para>change <literal>TARGETPLATFORM</literal> + appropriately, and set the variables involving + <literal>TARGET</literal> to the correct values for + the target platform. This step is necessary because + currently <literal>configure</literal> doesn't cope + with specifying different values for the + <literal>--host</literal> and + <literal>--target</literal> flags.</para> + </listitem> + <listitem> + <para>copy <literal>LeadingUnderscore</literal> + setting from target.</para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para>Copy + <filename><replaceable>T</replaceable>/ghc/includes/ghcautoconf.h</filename>, <filename><replaceable>T</replaceable>/ghc/includes/DerivedConstants.h</filename>, and <filename><replaceable>T</replaceable>/ghc/includes/GHCConstants.h</filename> + to + <filename><replaceable>H</replaceable>/ghc/includes</filename>. + Note that we are building on the host machine, using the + target machine's configuration files. This + is so that the intermediate C files generated here will + be suitable for compiling on the target system.</para> + </listitem> + + <listitem> + <para>Touch the generated configuration files, just to make + sure they don't get replaced during the build:</para> +<screen>$ cd <filename><replaceable>H</replaceable></filename>/ghc/includes +$ touch ghcautoconf.h DerivedConstants.h GHCConstants.h mkDerivedConstants.c +$ touch mkDerivedConstantsHdr mkDerivedConstants.o mkGHCConstants mkGHCConstants.o</screen> + + <para>Note: it has been reported that these files still get + overwritten during the next stage. We have installed a fix + for this in GHC 6.4.2, but if you are building a version + before that you need to watch out for these files getting + overwritte by the <literal>Makefile</literal> in + <literal>ghc/includes</literal>. If your system supports + it, you might be able to prevent it by making them + immutable:</para> +<screen>$ chflags uchg ghc/includes/{ghcautoconf.h,DerivedConstants.h,GHCConstants.h}</screen> + </listitem> + + <listitem> + <para>Now build the compiler:</para> +<screen>$ cd <replaceable>H</replaceable>/glafp-utils && make boot && make +$ cd <replaceable>H</replaceable>/ghc && make boot && make</screen> + <para>Don't worry if the build falls over in the RTS, we + don't need the RTS yet.</para> + </listitem> + + <listitem> +<screen>$ cd <replaceable>H</replaceable>/libraries +$ make boot && make</screen> + </listitem> + + <listitem> +<screen>$ cd <replaceable>H</replaceable>/ghc/compiler +$ make boot stage=2 && make stage=2</screen> + </listitem> + + <listitem> +<screen>$ cd <replaceable>H</replaceable>/ghc/lib/compat +$ make clean +$ rm .depend +$ make boot UseStage1=YES +$ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files' +$ cd <replaceable>H</replaceable>/ghc/utils +$ make clean +$ make -k UseStage1=YES EXTRA_HC_OPTS='-O -fvia-C -keep-hc-files'</screen> + </listitem> + + <listitem> +<screen>$ cd <replaceable>H</replaceable> +$ make hc-file-bundle Project=Ghc</screen> + </listitem> + + <listitem> + <para>copy + <filename><replaceable>H</replaceable>/*-hc.tar.gz</filename> + to <filename><replaceable>T</replaceable>/..</filename>.</para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para>On the target machine:</para> + + <para>At this stage we simply need to bootstrap a compiler + from the intermediate C files we generated above. The + process of bootstrapping from C files is automated by the + script in <literal>distrib/hc-build</literal>, and is + described in <xref linkend="sec-booting-from-hc"/>.</para> + +<screen>$ ./distrib/hc-build --enable-hc-boot-unregisterised</screen> + + <para>However, since this is a bootstrap on a new machine, + the automated process might not run to completion the + first time. For that reason, you might want to treat the + <literal>hc-build</literal> script as a list of + instructions to follow, rather than as a fully automated + script. This way you'll be able to restart the process + part-way through if you need to fix anything on the + way.</para> + + <para>Don't bother with running + <literal>make install</literal> in the newly + bootstrapped tree; just use the compiler in that tree to + build a fresh compiler from scratch, this time without + booting from C files. Before doing this, you might want + to check that the bootstrapped compiler is generating + working binaries:</para> + +<screen>$ cat >hello.hs +main = putStrLn "Hello World!\n" +^D +$ <replaceable>T</replaceable>/ghc/compiler/ghc-inplace hello.hs -o hello +$ ./hello +Hello World!</screen> + + <para>Once you have the unregisterised compiler up and + running, you can use it to start a registerised port. The + following sections describe the various parts of the + system that will need architecture-specific tweaks in + order to get a registerised build going.</para> + + </listitem> + </itemizedlist> + </sect3> + + <sect3> + <title>Porting the RTS</title> + + <para>The following files need architecture-specific code for a + registerised build:</para> + + <variablelist> + <varlistentry> + <term><filename>ghc/includes/MachRegs.h</filename> + <indexterm><primary><filename>MachRegs.h</filename></primary></indexterm> + </term> + <listitem> + <para>Defines the STG-register to machine-register + mapping. You need to know your platform's C calling + convention, and which registers are generally available + for mapping to global register variables. There are + plenty of useful comments in this file.</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>ghc/includes/TailCalls.h</filename> + <indexterm><primary><filename>TailCalls.h</filename></primary></indexterm> + </term> + <listitem> + <para>Macros that cooperate with the mangler (see <xref + linkend="sec-mangler"/>) to make proper tail-calls + work.</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>ghc/rts/Adjustor.c</filename> + <indexterm><primary><filename>Adjustor.c</filename></primary></indexterm> + </term> + <listitem> + <para>Support for + <literal>foreign import "wrapper"</literal> + (aka + <literal>foreign export dynamic</literal>). + Not essential for getting GHC bootstrapped, so this file + can be deferred until later if necessary.</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>ghc/rts/StgCRun.c</filename> + <indexterm><primary><filename>StgCRun.c</filename></primary></indexterm> + </term> + <listitem> + <para>The little assembly layer between the C world and + the Haskell world. See the comments and code for the + other architectures in this file for pointers.</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>ghc/rts/MBlock.h</filename> + <indexterm><primary><filename>MBlock.h</filename></primary></indexterm> + </term> + <term><filename>ghc/rts/MBlock.c</filename> + <indexterm><primary><filename>MBlock.c</filename></primary></indexterm> + </term> + <listitem> + <para>These files are really OS-specific rather than + architecture-specific. In <filename>MBlock.h</filename> + is specified the absolute location at which the RTS + should try to allocate memory on your platform (try to + find an area which doesn't conflict with code or dynamic + libraries). In <filename>Mblock.c</filename> you might + need to tweak the call to <literal>mmap()</literal> for + your OS.</para> + </listitem> + </varlistentry> + </variablelist> + </sect3> + + <sect3 id="sec-mangler"> + <title>The mangler</title> + + <para>The mangler is an evil Perl-script + (<filename>ghc/driver/mangler/ghc-asm.lprl</filename>) that + rearranges the assembly code output from gcc to do two main + things:</para> + + <itemizedlist> + <listitem> + <para>Remove function prologues and epilogues, and all + movement of the C stack pointer. This is to support + tail-calls: every code block in Haskell code ends in an + explicit jump, so we don't want the C-stack overflowing + while we're jumping around between code blocks.</para> + </listitem> + <listitem> + <para>Move the <firstterm>info table</firstterm> for a + closure next to the entry code for that closure. In + unregisterised code, info tables contain a pointer to the + entry code, but in registerised compilation we arrange + that the info table is shoved right up against the entry + code, and addressed backwards from the entry code pointer + (this saves a word in the info table and an extra + indirection when jumping to the closure entry + code).</para> + </listitem> + </itemizedlist> + + <para>The mangler is abstracted to a certain extent over some + architecture-specific things such as the particular assembler + directives used to herald symbols. Take a look at the + definitions for other architectures and use these as a + starting point.</para> + </sect3> + + <sect3> + <title>The splitter</title> + + <para>The splitter is another evil Perl script + (<filename>ghc/driver/split/ghc-split.lprl</filename>). It + cooperates with the mangler to support object splitting. + Object splitting is what happens when the + <option>-split-objs</option> option is passed to GHC: the + object file is split into many smaller objects. This feature + is used when building libraries, so that a program statically + linked against the library will pull in less of the + library.</para> + + <para>The splitter has some platform-specific stuff; take a + look and tweak it for your system.</para> + </sect3> + + <sect3> + <title>The native code generator</title> + + <para>The native code generator isn't essential to getting a + registerised build going, but it's a desirable thing to have + because it can cut compilation times in half. The native code + generator is described in some detail in the <ulink + url="http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/">GHC + commentary</ulink>.</para> + </sect3> + + <sect3> + <title>GHCi</title> + + <para>To support GHCi, you need to port the dynamic linker + (<filename>fptools/ghc/rts/Linker.c</filename>). The linker + currently supports the ELF and PEi386 object file formats - if + your platform uses one of these then things will be + significantly easier. The majority of Unix platforms use the + ELF format these days. Even so, there are some + machine-specific parts of the ELF linker: for example, the + code for resolving particular relocation types is + machine-specific, so some porting of this code to your + architecture will probaly be necessary.</para> + + <para>If your system uses a different object file format, then + you have to write a linker — good luck!</para> + </sect3> + </sect2> + + </sect1> + +<sect1 id="sec-build-pitfalls"> +<title>Known pitfalls in building Glasgow Haskell + +<indexterm><primary>problems, building</primary></indexterm> +<indexterm><primary>pitfalls, in building</primary></indexterm> +<indexterm><primary>building pitfalls</primary></indexterm></title> + +<para> +WARNINGS about pitfalls and known “problems”: +</para> + +<para> + +<orderedlist> +<listitem> + +<para> +One difficulty that comes up from time to time is running out of space +in <literal>TMPDIR</literal>. (It is impossible for the configuration stuff to +compensate for the vagaries of different sysadmin approaches to temp +space.) +<indexterm><primary>tmp, running out of space in</primary></indexterm> + +The quickest way around it is <command>setenv TMPDIR /usr/tmp</command><indexterm><primary>TMPDIR</primary></indexterm> or +even <command>setenv TMPDIR .</command> (or the equivalent incantation with your shell +of choice). + +The best way around it is to say + +<programlisting>export TMPDIR=<dir></programlisting> + +in your <filename>build.mk</filename> file. +Then GHC and the other <literal>fptools</literal> programs will use the appropriate directory +in all cases. + + +</para> +</listitem> +<listitem> + +<para> +In compiling some support-code bits, e.g., in <filename>ghc/rts/gmp</filename> and even +in <filename>ghc/lib</filename>, you may get a few C-compiler warnings. We think these +are OK. + +</para> +</listitem> +<listitem> + +<para> +When compiling via C, you'll sometimes get “warning: assignment from +incompatible pointer type” out of GCC. Harmless. + +</para> +</listitem> +<listitem> + +<para> +Similarly, <command>ar</command>chiving warning messages like the following are not +a problem: + +<screen>ar: filename GlaIOMonad__1_2s.o truncated to GlaIOMonad_ +ar: filename GlaIOMonad__2_2s.o truncated to GlaIOMonad_ +...</screen> + + +</para> +</listitem> +<listitem> + +<para> + In compiling the compiler proper (in <filename>compiler/</filename>), you <emphasis>may</emphasis> +get an “Out of heap space” error message. These can vary with the +vagaries of different systems, it seems. The solution is simple: + + +<itemizedlist> +<listitem> + +<para> + If you're compiling with GHC 4.00 or later, then the +<emphasis>maximum</emphasis> heap size must have been reached. This +is somewhat unlikely, since the maximum is set to 64M by default. +Anyway, you can raise it with the +<option>-optCrts-M<size></option> flag (add this flag to +<constant><module>_HC_OPTS</constant> +<command>make</command> variable in the appropriate +<filename>Makefile</filename>). + +</para> +</listitem> +<listitem> + +<para> + For GHC < 4.00, add a suitable <option>-H</option> flag to the <filename>Makefile</filename>, as +above. + +</para> +</listitem> + +</itemizedlist> + + +and try again: <command>gmake</command>. (see <xref linkend="sec-suffix"/> for information about +<constant><module>_HC_OPTS</constant>.) + +Alternatively, just cut to the chase: + +<screen>$ cd ghc/compiler +$ make EXTRA_HC_OPTS=-optCrts-M128M</screen> + + +</para> +</listitem> +<listitem> + +<para> +If you try to compile some Haskell, and you get errors from GCC about +lots of things from <filename>/usr/include/math.h</filename>, then your GCC was +mis-installed. <command>fixincludes</command> wasn't run when it should've been. + +As <command>fixincludes</command> is now automagically run as part of GCC installation, +this bug also suggests that you have an old GCC. + + +</para> +</listitem> +<listitem> + +<para> +You <emphasis>may</emphasis> need to re-<command>ranlib</command><indexterm><primary>ranlib</primary></indexterm> your libraries (on Sun4s). + + +<screen>$ cd $(libdir)/ghc-x.xx/sparc-sun-sunos4 +$ foreach i ( `find . -name '*.a' -print` ) # or other-shell equiv... +? ranlib $i +? # or, on some machines: ar s $i +? end</screen> + + +We'd be interested to know if this is still necessary. + + +</para> +</listitem> +<listitem> + +<para> +GHC's sources go through <command>cpp</command> before being compiled, and <command>cpp</command> varies +a bit from one Unix to another. One particular gotcha is macro calls +like this: + + +<programlisting>SLIT("Hello, world")</programlisting> + + +Some <command>cpp</command>s treat the comma inside the string as separating two macro +arguments, so you get + + +<screen>:731: macro `SLIT' used with too many (2) args</screen> + + +Alas, <command>cpp</command> doesn't tell you the offending file! + +Workaround: don't put weird things in string args to <command>cpp</command> macros. +</para> +</listitem> + +</orderedlist> + +</para> + +</sect1> + + +<sect1 id="platforms"><title>Platforms, scripts, and file names</title> +<para> +GHC is designed both to be built, and to run, on both Unix and Windows. This flexibility +gives rise to a good deal of brain-bending detail, which we have tried to collect in this chapter. +</para> + +<sect2 id="cygwin-and-mingw"><title>Windows platforms: Cygwin, MSYS, and MinGW</title> + +<para> The build system is built around Unix-y makefiles. Because it's not native, +the Windows situation for building GHC is particularly confusing. This section +tries to clarify, and to establish terminology.</para> + +<sect3 id="ghc-mingw"><title>MinGW</title> + +<para> <ulink url="http://www.mingw.org">MinGW (Minimalist GNU for Windows)</ulink> +is a collection of header +files and import libraries that allow one to use <command>gcc</command> and produce +native Win32 programs that do not rely on any third-party DLLs. The +current set of tools include GNU Compiler Collection (<command>gcc</command>), GNU Binary +Utilities (Binutils), GNU debugger (Gdb), GNU make, and a assorted +other utilities. +</para> + +<para> The down-side of MinGW is that the MinGW libraries do not support anything like the full +Posix interface. +</para> +</sect3> + +<sect3 id="ghc-cygwin"><title>Cygwin and MSYS</title> + +<para>You can't use the MinGW to <emphasis>build</emphasis> GHC, because MinGW doesn't have a shell, +or the standard Unix commands such as <command>mv</command>, <command>rm</command>, +<command>ls</command>, nor build-system stuff such as <command>make</command> and <command>darcs</command>. +For that, there are two choices: <ulink url="http://www.cygwin.com">Cygwin</ulink> +and <ulink url="http://www.mingw.org/msys.shtml">MSYS</ulink>: + +<itemizedlist> +<listitem><para> +Cygwin comes with compilation tools (<command>gcc</command>, <command>ld</command> and so on), which +compile code that has access to all of Posix. The price is that the executables must be +dynamically linked with the Cygwin DLL, so that <emphasis>you cannot run a Cywin-compiled program on a machine +that doesn't have Cygwin</emphasis>. Worse, Cygwin is a moving target. The name of the main DLL, <literal>cygwin1.dll</literal> +does not change, but the implementation certainly does. Even the interfaces to functions +it exports seem to change occasionally. </para> +</listitem> + +<listitem><para> +MSYS is a fork of the Cygwin tree, so they +are fundamentally similar. However, MSYS is by design much smaller and simpler. Access to the file system goes +through fewer layers, so MSYS is quite a bit faster too. +</para> + +<para>Furthermore, MSYS provides no compilation tools; it relies instead on the MinGW tools. These +compile binaries that run with no DLL support, on any Win32 system. +However, MSYS does come with all the make-system tools, such as <command>make</command>, <command>autoconf</command>, +<command>darcs</command>, <command>ssh</command> etc. To get these, you have to download the +MsysDTK (Developer Tool Kit) package, as well as the base MSYS package. +</para> +<para>MSYS does have a DLL, but it's only used by MSYS commands (<command>sh</command>, <command>rm</command>, +<command>ssh</command> and so on), +not by programs compiled under MSYS. +</para></listitem> + +</itemizedlist> + +</para> +</sect3> + +<sect3><title>Targeting MinGW</title> + +<para>We want GHC to compile programs that work on any Win32 system. Hence: +<itemizedlist> +<listitem><para> +GHC does invoke a C compiler, assembler, linker and so on, but we ensure that it only +invokes the MinGW tools, not the Cygwin ones. That means that the programs GHC compiles +will work on any system, but it also means that the programs GHC compiles do not have access +to all of Posix. In particular, they cannot import the (Haskell) Posix +library; they have to do +their input output using standard Haskell I/O libraries, or native Win32 bindings.</para> +<para> We will call a GHC that targets MinGW in this way <emphasis>GHC-mingw</emphasis>.</para> +</listitem> + +<listitem><para> +To make the GHC distribution self-contained, the GHC distribution includes the MinGW <command>gcc</command>, +<command>as</command>, <command>ld</command>, and a bunch of input/output libraries. +</para></listitem> +</itemizedlist> +So <emphasis>GHC targets MinGW</emphasis>, not Cygwin. +It is in principle possible to build a version of GHC, <emphasis>GHC-cygwin</emphasis>, +that targets Cygwin instead. The up-side of GHC-cygwin is +that Haskell programs compiled by GHC-cygwin can import the (Haskell) Posix library. +<emphasis>We do not support GHC-cygwin, however; it is beyond our resources.</emphasis> +</para> + +<para>While GHC <emphasis>targets</emphasis> MinGW, that says nothing about +how GHC is <emphasis>built</emphasis>. We use both MSYS and Cygwin as build environments for +GHC; both work fine, though MSYS is rather lighter weight.</para> + +<para>In your build tree, you build a compiler called <command>ghc-inplace</command>. It +uses the <command>gcc</command> that you specify using the +<option>--with-gcc</option> flag when you run +<command>configure</command> (see below). +The makefiles are careful to use <command>ghc-inplace</command> (not <command>gcc</command>) +to compile any C files, so that it will in turn invoke the correct <command>gcc</command> rather that +whatever one happens to be in your path. However, the makefiles do use whatever <command>ld</command> +and <command>ar</command> happen to be in your path. This is a bit naughty, but (a) they are only +used to glom together .o files into a bigger .o file, or a .a file, +so they don't ever get libraries (which would be bogus; they might be the wrong libraries), and (b) +Cygwin and MinGW use the same .o file format. So its ok. +</para> +</sect3> + +<sect3><title> File names </title> + +<para>Cygwin, MSYS, and the underlying Windows file system all understand file paths of form <literal>c:/tmp/foo</literal>. +However: +<itemizedlist> +<listitem><para> +MSYS programs understand <filename>/bin</filename>, <filename>/usr/bin</filename>, and map Windows's lettered drives as +<filename>/c/tmp/foo</filename> etc. The exact mount table is given in the doc subdirectory of the MSYS distribution. +</para> +<para> When it invokes a command, the MSYS shell sees whether the invoked binary lives in the MSYS <filename>/bin</filename> +directory. If so, it just invokes it. If not, it assumes the program is no an MSYS program, and walks over the command-line +arguments changing MSYS paths into native-compatible paths. It does this inside sub-arguments and inside quotes. For example, +if you invoke +<programlisting>foogle -B/c/tmp/baz</programlisting> +the MSYS shell will actually call <literal>foogle</literal> with argument <literal>-Bc:/tmp/baz</literal>. +</para></listitem> + +<listitem><para> +Cygwin programs have a more complicated mount table, and map the lettered drives as <filename>/cygdrive/c/tmp/foo</filename>. +</para> +<para>The Cygwin shell does no argument processing when invoking non-Cygwin programs. +</para></listitem> +</itemizedlist> +</para> +</sect3> + +<sect3><title>Crippled <command>ld</command></title> + +<para> +It turns out that on both Cygwin and MSYS, the <command>ld</command> has a +limit of 32kbytes on its command line. Especially when using split object +files, the make system can emit calls to <command>ld</command> with thousands +of files on it. Then you may see something like this: +<programlisting> +(cd Graphics/Rendering/OpenGL/GL/QueryUtils_split && /mingw/bin/ld -r -x -o ../QueryUtils.o *.o) +/bin/sh: /mingw/bin/ld: Invalid argument +</programlisting> +The solution is either to switch off object file splitting (set +<option>SplitObjs</option> to <literal>NO</literal> in your +<filename>build.mk</filename>), +or to make the module smaller. +</para> +</sect3> + +<sect3><title>Host System vs Target System</title> + +<para> +In the source code you'll find various ifdefs looking like: +<programlisting>#ifdef mingw32_HOST_OS + ...blah blah... +#endif</programlisting> +and +<programlisting>#ifdef mingw32_TARGET_OS + ...blah blah... +#endif</programlisting> +These macros are set by the configure script (via the file config.h). +Which is which? The criterion is this. In the ifdefs in GHC's source code: +<itemizedlist> + <listitem> + <para>The "host" system is the one on which GHC itself will be run.</para> + </listitem> + <listitem> + <para>The "target" system is the one for which the program compiled by GHC will be run.</para> + </listitem> +</itemizedlist> +For a stage-2 compiler, in which GHCi is available, the "host" and "target" systems must be the same. +So then it doesn't really matter whether you use the HOST_OS or TARGET_OS cpp macros. + +</para> +</sect3> + +</sect2> + +<sect2><title>Wrapper scripts</title> + +<para> +Many programs, including GHC itself and hsc2hs, need to find associated binaries and libraries. +For <emphasis>installed</emphasis> programs, the strategy depends on the platform. We'll use +GHC itself as an example: +<itemizedlist> + <listitem> <para> + On Unix, the command <command>ghc</command> is a shell script, generated by adding installation + paths to the front of the source file <filename>ghc.sh</filename>, + that invokes the real binary, passing "-B<emphasis>path</emphasis>" as an argument to tell <command>ghc</command> + where to find its supporting files. + </para> </listitem> + + <listitem> <para> + On vanilla Windows, it turns out to be much harder to make reliable script to be run by the + native Windows shell <command>cmd</command> (e.g. limits on the length + of the command line). So instead we invoke the GHC binary directly, with no -B flag. + GHC uses the Windows <literal>getExecDir</literal> function to find where the executable is, + and from that figures out where the supporting files are. + </para> </listitem> +</itemizedlist> +(You can find the layout of GHC's supporting files in the + section "Layout of installed files" of Section 2 of the GHC user guide.) +</para> +<para> +Things work differently for <emphasis>in-place</emphasis> execution, where you want to +execute a program that has just been built in a build tree. The difference is that the +layout of the supporting files is different. +In this case, whether on Windows or Unix, we always use a shell script. This works OK +on Windows because the script is executed by MSYS or Cygwin, which don't have the +shortcomings of the native Windows <command>cmd</command> shell. +</para> + +</sect2> + +</sect1> + +<sect1 id="winbuild"><title>Instructions for building under Windows</title> + +<para> +This section gives detailed instructions for how to build +GHC from source on your Windows machine. Similar instructions for +installing and running GHC may be found in the user guide. In general, +Win95/Win98 behave the same, and WinNT/Win2k behave the same. +</para> +<para> +Make sure you read the preceding section on platforms (<xref linkend="platforms"/>) +before reading section. +You don't need Cygwin or MSYS to <emphasis>use</emphasis> GHC, +but you do need one or the other to <emphasis>build</emphasis> GHC.</para> + + +<sect2 id="msys-install"><title>Installing and configuring MSYS</title> + +<para> +MSYS is a lightweight alternative to Cygwin. +You don't need MSYS to <emphasis>use</emphasis> GHC, +but you do need it or Cygwin to <emphasis>build</emphasis> GHC. +Here's how to install MSYS. +<itemizedlist> +<listitem><para> +Go to <ulink url="http://www.mingw.org/download.shtml">http://www.mingw.org/download.shtml</ulink> and +download the following (of course, the version numbers will differ): +<itemizedlist> + <listitem><para>The main MSYS package (binary is sufficient): <literal>MSYS-1.0.9.exe</literal> + </para></listitem> + <listitem><para>The MSYS developer's toolkit (binary is sufficient): <literal>msysDTK-1.0.1.exe</literal>. + This provides <command>make</command>, <command>autoconf</command>, + <command>ssh</command> and probably more besides. + </para></listitem> +</itemizedlist> +Run both executables (in the order given above) to install them. I put them in <literal>c:/msys</literal> +</para></listitem> + +<listitem><para> +Set the following environment variables +<itemizedlist> + <listitem><para><literal>PATH</literal>: add <literal>c:/msys/1.0/bin</literal> and + <literal>c:/msys/1.0/local/bin</literal> + to your path. (Of course, the version number may differ.) + MSYS mounts the former as both <literal>/bin</literal> and + <literal>/usr/bin</literal> and the latter as <literal>/usr/local/bin</literal>. + </para></listitem> + + <listitem><para><literal>HOME</literal>: set to your home directory (e.g. <literal>c:/userid</literal>). + This is where, among other things, <command>ssh</command> will look for your <literal>.ssh</literal> directory. + </para></listitem> + + <listitem><para><literal>SHELL</literal>: set to <literal>c:/msys/1.0/bin/sh.exe</literal> + </para></listitem> + + <listitem><para><literal>CVS_RSH</literal>: set to <literal>c:/msys/1.0/bin/ssh.exe</literal>. Only necessary if + you are using CVS. + </para></listitem> + + <listitem><para><literal>MAKE_MODE</literal>: set to <literal>UNIX</literal>. (I'm not certain this is necessary for MSYS.) + </para></listitem> + +</itemizedlist> +</para></listitem> + +<listitem><para> +Check that the <literal>CYGWIN</literal> environment variable is <emphasis>not</emphasis> set. It's a bad bug +that MSYS is affected by this, but if you have CYGWIN set to "ntsec ntea", which is right for Cygwin, it +causes the MSYS <command>ssh</command> to bogusly fail complaining that your <filename>.ssh/identity</filename> +file has too-liberal permissinos. +</para></listitem> + +</itemizedlist> +</para> +<para>Here are some points to bear in mind when using MSYS: +<itemizedlist> +<listitem> <para> MSYS does some kind of special magic to binaries stored in +<filename>/bin</filename> and <filename>/usr/bin</filename>, which are by default both mapped +to <filename>c:/msys/1.0/bin</filename> (assuming you installed MSYS in <filename>c:/msys</filename>). +Do not put any other binaries (such as GHC or Alex) in this directory or its sub-directories: +they fail in mysterious ways. However, it's fine to put other binaries in <filename>/usr/local/bin</filename>, +which maps to <filename>c:/msys/1.0/local/bin</filename>.</para></listitem> + +<listitem> <para> MSYS seems to implement symbolic links by copying, so sharing is lost. +</para></listitem> + +<listitem> <para> +Win32 has a <command>find</command> command which is not the same as MSYS's find. +You will probably discover that the Win32 <command>find</command> appears in your <constant>PATH</constant> +before the MSYS one, because it's in the <emphasis>system</emphasis> <constant>PATH</constant> +environment variable, whereas you have probably modified the <emphasis>user</emphasis> <constant>PATH</constant> +variable. You can always invoke <command>find</command> with an absolute path, or rename it. +</para></listitem> + +<listitem> <para> +MSYS comes with <command>bzip</command>, and MSYS's <command>tar</command>'s <literal>-j</literal> +will bunzip an archive (e.g. <literal>tar xvjf foo.tar.bz2</literal>). Useful when you get a +bzip'd dump.</para></listitem> + +</itemizedlist> +</para> +</sect2> + +<sect2 id="install-cygwin"><title>Installing and configuring Cygwin</title> + +<para> Install Cygwin from <ulink url="http://www.cygwin.com/">http://www.cygwin.com/</ulink>. +The installation process is straightforward; we install it in +<filename>c:/cygwin</filename>.</para> +<para> +You must install enough Cygwin <emphasis>packages</emphasis> to support +building GHC. If you miss out any of these, strange things will happen to you. There are two ways to do this: +<itemizedlist> +<listitem><para>The direct, but laborious way is to +select all of the following packages in the installation dialogue: + <command>cvs</command>, + <command>openssh</command>, + <command>autoconf</command>, + <command>binutils</command> (includes ld and (I think) ar), + <command>gcc</command>, + <command>flex</command>, + <command>make</command>. +To see thse packages, +click on the "View" button in the "Select Packages" +stage of Cygwin's installation dialogue, until the view says "Full". The default view, which is +"Category" isn't very helpful, and the "View" button is rather unobtrousive. +</para> +</listitem> + +<listitem><para>The clever way is to point the Cygwin installer at the +<command>ghc-depends</command> package, which is kept at <ulink +url="http://haskell.org/ghc/cygwin">http://haskell.org/ghc/cygwin</ulink>. +When the Cygwin installer asks you to "Choose a Download Site", choose one of +the +offered mirror sites; and then type "http://haskell.org/ghc/cygwin" into the +"User URL" box and click "Add"; now two sites are selected. (The Cygwin +installer remembers this for next time.) +Click "Next".</para> +<para>In the "Select Packages" dialogue box that follows, click the "+" sign by +"Devel", scroll down to the end of the "Devel" packages, and choose +<command>ghc-depends</command>. +The package <command>ghc-depends</command> will not actually install anything itself, +but forces additional packages to be added by the Cygwin installer. +</para> +</listitem> +</itemizedlist> +</para> + +<para> Now set the following user environment variables: +<itemizedlist> + +<listitem><para> Add <filename>c:/cygwin/bin</filename> and <filename>c:/cygwin/usr/bin</filename> to your +<constant>PATH</constant></para></listitem> + +<listitem> +<para> +Set <constant>MAKE_MODE</constant> to <literal>UNIX</literal>. If you +don't do this you get very weird messages when you type +<command>make</command>, such as: +<screen>/c: /c: No such file or directory</screen> +</para> +</listitem> + +<listitem><para> Set <constant>SHELL</constant> to +<filename>c:/cygwin/bin/bash</filename>. When you invoke a shell in Emacs, this +<constant>SHELL</constant> is what you get. +</para></listitem> + +<listitem><para> Set <constant>HOME</constant> to point to your +home directory. This is where, for example, +<command>bash</command> will look for your <filename>.bashrc</filename> +file. Ditto <command>emacs</command> looking for <filename>.emacsrc</filename> +</para></listitem> +</itemizedlist> +</para> + +<para>Here are some things to be aware of when using Cygwin: +<itemizedlist> +<listitem> <para>Cygwin doesn't deal well with filenames that include +spaces. "<filename>Program Files</filename>" and "<filename>Local files</filename>" are +common gotchas. +</para></listitem> + +<listitem> <para> Cygwin implements a symbolic link as a text file with some +magical text in it. So other programs that don't use Cygwin's +I/O libraries won't recognise such files as symlinks. +In particular, programs compiled by GHC are meant to be runnable +without having Cygwin, so they don't use the Cygwin library, so +they don't recognise symlinks. +</para></listitem> + +<listitem> <para> +See the notes in <xref linkend="msys-install"/> about <command>find</command> and <command>bzip</command>, +which apply to Cygwin too. +</para></listitem> + +<listitem> +<para> +Some script files used in the make system start with "<command>#!/bin/perl</command>", +(and similarly for <command>sh</command>). Notice the hardwired path! +So you need to ensure that your <filename>/bin</filename> directory has at least +<command>sh</command>, <command>perl</command>, and <command>cat</command> in it. +All these come in Cygwin's <filename>bin</filename> directory, which you probably have +installed as <filename>c:/cygwin/bin</filename>. By default Cygwin mounts "<filename>/</filename>" as +<filename>c:/cygwin</filename>, so if you just take the defaults it'll all work ok. +(You can discover where your Cygwin +root directory <filename>/</filename> is by typing <command>mount</command>.) +Provided <filename>/bin</filename> points to the Cygwin <filename>bin</filename> +directory, there's no need to copy anything. If not, copy these binaries from the <filename>cygwin/bin</filename> +directory (after fixing the <filename>sh.exe</filename> stuff mentioned in the previous bullet). +</para> +</listitem> + +<listitem> +<para> +By default, cygwin provides the command shell <filename>ash</filename> +as <filename>sh.exe</filename>. It seems to be fine now, but in the past we +saw build-system problems that turned out to be due to bugs in <filename>ash</filename> +(to do with quoting and length of command lines). On the other hand <filename>bash</filename> seems +to be rock solid. +If this happens to you (which it shouldn't), in <filename>cygwin/bin</filename> +remove the supplied <filename>sh.exe</filename> (or rename it as <filename>ash.exe</filename>), +and copy <filename>bash.exe</filename> to <filename>sh.exe</filename>. +You'll need to do this in Windows Explorer or the Windows <command>cmd</command> shell, because +you can't rename a running program! +</para> +</listitem> +</itemizedlist> +</para> + +</sect2> + + +<sect2 id="configure-ssh"><title>Configuring SSH</title> + +<para><command>ssh</command> comes with both Cygwin and MSYS. +(Cygwin note: you need to ask for package <command>openssh</command> (not ssh) +in the Cygwin list of packages; or use the <command>ghc-depends</command> +package -- see <xref linkend="install-cygwin"/>.)</para> + +<para>There are several strange things about <command>ssh</command> on Windows that you need to know. +<itemizedlist> +<listitem> +<para> + The programs <command>ssh-keygen1</command>, <command>ssh1</command>, and <command>cvs</command>, + seem to lock up <command>bash</command> entirely if they try to get user input (e.g. if + they ask for a password). To solve this, start up <filename>cmd.exe</filename> + and run it as follows: +<screen>c:\tmp> set CYGWIN32=tty +c:\tmp> c:/user/local/bin/ssh-keygen1</screen> </para> +</listitem> + +<listitem><para> (Cygwin-only problem, I think.) +<command>ssh</command> needs to access your directory <filename>.ssh</filename>, in your home directory. +To determine your home directory <command>ssh</command> first looks in +<filename>c:/cygwin/etc/passwd</filename> (or wherever you have Cygwin installed). If there's an entry +there with your userid, it'll use that entry to determine your home directory, <emphasis>ignoring +the setting of the environment variable $HOME</emphasis>. If the home directory is +bogus, <command>ssh</command> fails horribly. The best way to see what is going on is to say +<screen>ssh -v cvs.haskell.org</screen> +which makes <command>ssh</command> print out information about its activity. +</para> +<para> You can fix this problem, either by correcting the home-directory field in +<filename>c:/cygwin/etc/passwd</filename>, or by simply deleting the entire entry for your userid. If +you do that, <command>ssh</command> uses the $HOME environment variable instead. +</para> + +</listitem> + +<listitem> + <para>To protect your + <literal>.ssh</literal> from access by anyone else, + right-click your <literal>.ssh</literal> directory, and + select <literal>Properties</literal>. If you are not on + the access control list, add yourself, and give yourself + full permissions (the second panel). Remove everyone else + from the access control list. Don't leave them there but + deny them access, because 'they' may be a list that + includes you!</para> +</listitem> + +<listitem> + <para>In fact <command>ssh</command> 3.6.1 now seems to <emphasis>require</emphasis> + you to have Unix permissions 600 (read/write for owner only) + on the <literal>.ssh/identity</literal> file, else it + bombs out. For your local C drive, it seems that <literal>chmod 600 identity</literal> works, + but on Windows NT/XP, it doesn't work on a network drive (exact dteails obscure). + The solution seems to be to set the $CYGWIN environment + variable to "<literal>ntsec neta</literal>". The $CYGWIN environment variable is discussed + in <ulink url="http://cygwin.com/cygwin-ug-net/using-cygwinenv.html">the Cygwin User's Guide</ulink>, + and there are more details in <ulink url="http://cygwin.com/faq/faq_4.html#SEC44">the Cygwin FAQ</ulink>. + </para> +</listitem> +</itemizedlist> +</para> +</sect2> + +<sect2><title>Other things you need to install</title> + +<para>You have to install the following other things to build GHC, listed below.</para> + +<para>On Windows you often install executables in directories with spaces, such as +"<filename>Program Files</filename>". However, the <literal>make</literal> system for fptools doesn't +deal with this situation (it'd have to do more quoting of binaries), so you are strongly advised +to put binaries for all tools in places with no spaces in their path. +On both MSYS and Cygwin, it's perfectly OK to install such programs in the standard Unixy places, +<filename>/usr/local/bin</filename> and <filename>/usr/local/lib</filename>. But it doesn't matter, +provided they are in your path. +<itemizedlist> +<listitem> +<para> +Install an executable GHC, from <ulink url="http://www.haskell.org/ghc">http://www.haskell.org/ghc</ulink>. +This is what you will use to compile GHC. Add it in your +<constant>PATH</constant>: the installer tells you the path element +you need to add upon completion. +</para> +</listitem> + +<listitem> +<para> +Install an executable Happy, from <ulink url="http://www.haskell.org/happy">http://www.haskell.org/happy</ulink>. +Happy is a parser generator used to compile the Haskell grammar. Under MSYS or Cygwin you can easily +build it from the source distribution using +<screen>$ ./configure +$ make +$ make install</screen> +This should install it in <filename>/usr/local/bin</filename> (which maps to <filename>c:/msys/1.0/local/bin</filename> +on MSYS). +Make sure the installation directory is in your +<constant>PATH</constant>. +</para> +</listitem> + + <listitem> + <para>Install an executable Alex. This can be done by building from the + source distribution in the same way as Happy. Sources are + available from <ulink + url="http://www.haskell.org/alex">http://www.haskell.org/alex</ulink>.</para> + </listitem> + +<listitem> +<para>GHC uses the <emphasis>mingw</emphasis> C compiler to +generate code, so you have to install that (see <xref linkend="cygwin-and-mingw"/>). +Just pick up a mingw bundle at +<ulink url="http://www.mingw.org/">http://www.mingw.org/</ulink>. +We install it in <filename>c:/mingw</filename>. +</para> + +<para><emphasis>On MSYS</emphasis>, add <literal>c:/mingw/bin</literal> to your PATH. MSYS does not provide <command>gcc</command>, +<command>ld</command>, <command>ar</command>, and so on, because it just uses the MinGW ones. So you need them +in your path. +</para> + +<para><emphasis>On Cygwin, do not</emphasis> add any of the <emphasis>mingw</emphasis> binaries to your path. +They are only going to get used by explicit access (via the --with-gcc flag you +give to <command>configure</command> later). If you do add them to your path +you are likely to get into a mess because their names overlap with Cygwin +binaries. +On the other hand, you <emphasis>do</emphasis> need <command>ld</command>, <command>ar</command> +(and perhaps one or two other things) in your path. The Cygwin ones are fine, +but you must have them; hence needing the Cygwin binutils package. +</para> +</listitem> + + +<listitem> +<para>We use <command>emacs</command> a lot, so we install that too. +When you are in <filename>fptools/ghc/compiler</filename>, you can use +"<literal>make tags</literal>" to make a TAGS file for emacs. That uses the utility +<filename>fptools/ghc/utils/hasktags/hasktags</filename>, so you need to make that first. +The most convenient way to do this is by going <literal>make boot</literal> in <filename>fptools/ghc</filename>. +The <literal>make tags</literal> command also uses <command>etags</command>, which comes with <command>emacs</command>, +so you will need to add <filename>emacs/bin</filename> to your <literal>PATH</literal>. +</para> +</listitem> + + <listitem> + <para>You might want to install GLUT in your MSYS/Cygwin + installation, otherwise the GLUT package will not be built with + GHC.</para> + </listitem> + +<listitem> +<para> Finally, check out a copy of GHC sources from +the darcs repository, following the instructions at <ulink url="http://hackage.haskell.org/trac/ghc/wiki/GhcDarcs" />.</para> +</listitem> +</itemizedlist> +</para> +</sect2> + +<sect2><title>Building GHC</title> + +<para>OK! +Now go read the documentation above on building from source (<xref linkend="sec-building-from-source"/>); +the bullets below only tell +you about Windows-specific wrinkles.</para> +<itemizedlist> +<listitem> +<para> +If you used <command>autoconf</command> instead of <command>autoreconf</command>, +you'll get an error when you run <filename>./configure</filename>: +<screen> +...lots of stuff... +creating mk/config.h +mk/config.h is unchanged +configuring in ghc +running /bin/sh ./configure --cache-file=.././config.cache --srcdir=. +./configure: ./configure: No such file or directory +configure: error: ./configure failed for ghc</screen> +</para> +</listitem> + +<listitem> <para><command>autoreconf</command> seems to create the file <filename>configure</filename> +read-only. So if you need to run autoreconf again (which I sometimes do for safety's sake), +you get +<screen>/usr/bin/autoconf: cannot create configure: permission denied</screen> +Solution: delete <filename>configure</filename> first. +</para></listitem> + +<listitem> + <para> + After <command>autoreconf</command> run <command>./configure</command> in + <filename>fptools/</filename> thus: + +<screen>$ ./configure --host=i386-unknown-mingw32 --with-gcc=c:/mingw/bin/gcc</screen> +This is the point at which you specify that you are building GHC-mingw +(see <xref linkend="ghc-mingw"/>). </para> + +<para> Both these options are important! It's possible to get into +trouble using the wrong C compiler!</para> +<para> +Furthermore, it's <emphasis>very important</emphasis> that you specify a +full MinGW path for <command>gcc</command>, not a Cygwin path, because GHC (which +uses this path to invoke <command>gcc</command>) is a MinGW program and won't +understand a Cygwin path. For example, if you +say <literal>--with-gcc=/mingw/bin/gcc</literal>, it'll be interpreted as +<filename>/cygdrive/c/mingw/bin/gcc</filename>, and GHC will fail the first +time it tries to invoke it. Worse, the failure comes with +no error message whatsoever. GHC simply fails silently when first invoked, +typically leaving you with this: +<screen>make[4]: Leaving directory `/cygdrive/e/fptools-stage1/ghc/rts/gmp' +../../ghc/compiler/ghc-inplace -optc-mno-cygwin -optc-O + -optc-Wall -optc-W -optc-Wstrict-prototypes -optc-Wmissing-prototypes + -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return + -optc-Wbad-function-cast -optc-Wcast-align -optc-I../includes + -optc-I. -optc-Iparallel -optc-DCOMPILING_RTS + -optc-fomit-frame-pointer -O2 -static + -package-name rts -O -dcore-lint -c Adjustor.c -o Adjustor.o +make[2]: *** [Adjustor.o] Error 1 +make[1]: *** [all] Error 1 +make[1]: Leaving directory `/cygdrive/e/fptools-stage1/ghc' +make: *** [all] Error 1</screen> +Be warned! +</para> + +<para> +If you want to build GHC-cygwin (<xref linkend="ghc-cygwin"/>) +you'll have to do something more like: +<screen>$ ./configure --with-gcc=...the Cygwin gcc...</screen> +</para> +</listitem> + +<listitem><para> +If you are paranoid, delete <filename>config.cache</filename> if it exists. +This file occasionally remembers out-of-date configuration information, which +can be really confusing. +</para> +</listitem> + +<listitem><para> You almost certainly want to set +<programlisting>SplitObjs = NO</programlisting> +in your <filename>build.mk</filename> configuration file (see <xref linkend="sec-build-config"/>). +This tells the build system not to split each library into a myriad of little object files, one +for each function. Doing so reduces binary sizes for statically-linked binaries, but on Windows +it dramatically increases the time taken to build the libraries in the first place. +</para> +</listitem> + +<listitem><para> Do not attempt to build the documentation. +It needs all kinds of wierd Jade stuff that we haven't worked out for +Win32.</para></listitem> +</itemizedlist> +</sect2> + + +<sect2><title>A Windows build log using Cygwin</title> + +<para>Here is a complete, from-scratch, log of all you need to build GHC using +Cygwin, kindly provided by Claus Reinke. It does not discuss alternative +choices, but it gives a single path that works.</para> +<programlisting>- Install some editor (vim, emacs, whatever) + +- Install cygwin (http://www.cygwin.com) + ; i used 1.5.16-1, installed in c:\cygwin + - run 'setup.exe' + Choose a Download Source: + select 'download from internet'; + Select Root Install Directory: + root dir: c:\cygwin; + install for: all users; + default file type: unix + Select Local Package Directory + choose a spare temporary home + Select Your Internet Connection + Use IE5 settings + Choose a Download Site + Choose your preferred main mirror and + Add 'http://www.haskell.org/ghc/cygwin' + Select Packages + In addition to 'Base' (default install), + select 'Devel->ghc-depends' + +- Install mingw (http://www.mingw.org/) + ; i used MinGW-3.1.0-1.exe + ; installed in c:\mingw + - you probably want to add GLUT + ; (http://www.xmission.com/~nate/glut.html) + ; i used glut-3.7.3-mingw32.tar + +- Get recent binary snapshot of ghc-6.4.1 for mingw + ; (http://www.haskell.org/ghc/dist/stable/dist/) + - unpack in c:/ghc + - add C:\ghc\ghc-6.4.1\bin to %PATH% + (Start->Control Panel->System->Advanced->Environment Variables) + +- Get darcs version of ghc + ; also, subscribe to cvs-all@haskell.org, or follow the mailing list + ; archive, in case you checkout a version with problems + ; http://www.haskell.org//pipermail/cvs-all/ + - mkdir c:/fptools; cd c:/fptools + ; (or whereever you want your darcs tree to be) + - darcs get http://darcs.haskell.org/ghc + - cd ghc + - chmod +x darcs-all + - ./darcs-all get + +- Build ghc, using cygwin and mingw, targetting mingw + - export PATH=/cygdrive/c/ghc/ghc-6.4.1:$PATH + ; for haddock, alex, happy (*) + - export PATH=/cygdrive/c/mingw/bin:$PATH + ; without, we pick up some cygwin tools at best! + - cd c:/fptools/fptools + ; (if you aren't there already) + - autoreconf + - ./configure --host=i386-unknown-mingw32 --with-gcc=C:/Mingw/bin/gcc.exe + ; we use cygwin, but build for windows + - cp mk/build.mk.sample mk/build.mk + - in mk/build.mk: + add line: SplitObjs = NO + (MSYS seems slow when there are zillions of object files) + uncomment line: BuildFlavour = perf + (or BuildFlavour = devel, if you are doing development) + add line: BIN_DIST=1 + - make 2>&1 | tee make.log + ; always useful to have a log around + +- Package up binary distribution + - make binary-dist Project=Ghc 2>&1 | tee make-bin-dist.log + ; always useful to have a log around + - cd ghc-6.5 + - chmod +x ../distrib/prep-bin-dist-mingw + ; if you're happy with the script's contents (*) + - ../distrib/prep-bin-dist-mingw + ; then tar up, unpack where wanted, and enjoy</programlisting> +</sect2> +</sect1> + +<index/> + +</article> diff --git a/docs/comm/exts/ndp.html b/docs/comm/exts/ndp.html new file mode 100644 index 0000000000..0c94c3960b --- /dev/null +++ b/docs/comm/exts/ndp.html @@ -0,0 +1,360 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Parallel Arrays</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Parallel Arrays</h1> + <p> + This section describes an experimental extension by high-performance + arrays, which comprises special syntax for array types and array + comprehensions, a set of optimising program transformations, and a set + of special purpose libraries. The extension is currently only partially + implemented, but the development will be tracked here. + <p> + Parallel arrays originally got their name from the aim to provide an + architecture-independent programming model for a range of parallel + computers. However, since experiments showed that the approach is also + worthwhile for sequential array code, the emphasis has shifted to their + parallel evaluation semantics: As soon as any element in a parallel + array is demanded, all the other elements are evaluated, too. This + makes parallel arrays more strict than <a + href="http://haskell.org/onlinelibrary/array.html">standard Haskell 98 + arrays</a>, but also opens the door for a loop-based implementation + strategy that leads to significantly more efficient code. + <p> + The programming model as well as the use of the <em>flattening + transformation</em>, which is central to the approach, has its origin in + the programming language <a + href="http://www.cs.cmu.edu/~scandal/nesl.html">Nesl</a>. + + <h2>More Sugar: Special Syntax for Array Comprehensions</h2> + <p> + The option <code>-fparr</code>, which is a dynamic hsc option that can + be reversed with <code>-fno-parr</code>, enables special syntax for + parallel arrays, which is not essential to using parallel arrays, but + makes for significantly more concise programs. The switch works by + making the lexical analyser (located in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/Lex.lhs"><code>Lex.lhs</code></a>) + recognise the tokens <code>[:</code> and <code>:]</code>. Given that + the additional productions in the parser (located in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/Parser.y"><code>Parser.y</code></a>) + cannot be triggered without the lexer generating the necessary tokens, + there is no need to alter the behaviour of the parser. + <p> + The following additional syntax is accepted (the non-terminals are those + from the <a href="http://haskell.org/onlinereport/">Haskell 98 language + definition</a>): + <p> + <blockquote><pre> +atype -> '[:' type ':] (parallel array type) + +aexp -> '[:' exp1 ',' ... ',' expk ':]' (explicit array, k >= 0) + | '[:' exp1 [',' exp2] '..' exp3 ':]' (arithmetic array sequence) + | '[:' exp '|' quals1 '|' ... '|' qualsn ':]' (array comprehension, n >= 1) + +quals -> qual1 ',' ... ',' qualn (qualifier list, n >= 1) + +apat -> '[:' pat1 ',' ... ',' patk ':]' (array pattern, k >= 0) +</pre> + </blockquote> + <p> + Moreover, the extended comprehension syntax that allows for <em>parallel + qualifiers</em> (i.e., qualifiers separated by "<code>|</code>") is also + supported in list comprehensions. In general, the similarity to the + special syntax for list is obvious. The two main differences are that + (a) arithmetic array sequences are always finite and (b) + <code>[::]</code> is not treated as a constructor in expressions and + patterns, but rather as a special case of the explicit array syntax. + The former is a simple consequence of the parallel evaluation semantics + of parallel arrays and the latter is due to arrays not being a + constructor-based data type. + <p> + As a naming convention, types and functions that are concerned with + parallel arrays usually contain the string <code>parr</code> or + <code>PArr</code> (often as a prefix), and where corresponding types or + functions for handling lists exist, the name is identical, except for + containing the substring <code>parr</code> instead of <code>list</code> + (possibly in caps). + <p> + The following implications are worth noting explicitly: + <ul> + <li>As the value and pattern <code>[::]</code> is an empty explicit + parallel array (i.e., something of the form <code>ExplicitPArr ty + []</code> in the AST). This is in contrast to lists, which use the + nil-constructor instead. In the case of parallel arrays, using a + constructor would be rather awkward, as it is not a constructor-based + type. (This becomes rather clear in the desugarer.) + <li>As a consequence, array patterns have the general form <code>[:p1, + p2, ..., pn:]</code>, where <code>n</code> >= 0. Thus, two array + patterns overlap iff they have the same length -- an important property + for the pattern matching compiler. + </ul> + + <h2>Prelude Support for Parallel Arrays</h2> + <p> + The Prelude module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelPArr.lhs"><code>PrelPArr</code></a> + defines the standard operations and their types on parallel arrays and + provides a basic implementation based on boxed arrays. The interface of + <code>PrelPArr</code> is oriented by H98's <code>PrelList</code>, but + leaving out all functions that require infinite structures and adding + frequently needed array operations, such as permutations. Parallel + arrays are quite unqiue in that they use an entirely different + representation as soon as the flattening transformation is activated, + which is described further below. In particular, <code>PrelPArr</code> + defines the type <code>[::]</code> and operations to create, process, + and inspect parallel arrays. The type as well as the names of some of + the operations are also hardwired in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/TysWiredIn.lhs"><code>TysWiredIn</code></a> + (see the definition of <code>parrTyCon</code> in this module) and <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/PrelNames.lhs"><code>PrelNames</code></a>. + This is again very much like the case of lists, where the type is + defined in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelBase.lhs"><code>PrelBase</code></a> + and similarly wired in; however, for lists the entirely + constructor-based definition is exposed to user programs, which is not + the case for parallel arrays. + + <h2>Desugaring Parallel Arrays</h2> + <p> + The parallel array extension requires the desugarer to replace all + occurrences of (1) explicit parallel arrays, (2) array patterns, and (3) + array comprehensions by a suitable combination of invocations of + operations defined in <code>PrelPArr</code>. + + <h4>Explicit Parallel Arrays</h4> + <p> + These are by far the simplest to remove. We simply replace every + occurrence of <code>[:<i>e<sub>1</sub></i>, ..., + <i>e<sub>n</sub></i>:]</code> by + <blockquote> + <code> + toP [<i>e<sub>1</sub></i>, ..., <i>e<sub>n</sub></i>] + </code> + </blockquote> + <p> + i.e., we build a list of the array elements, which is, then, converted + into a parallel array. + + <h4>Parallel Array Patterns</h4> + <p> + Array patterns are much more tricky as element positions may contain + further patterns and the <a + href="../the-beast/desugar.html#patmat">pattern matching compiler</a> + requires us to flatten all those out. But before we turn to the gory + details, here first the basic idea: A flat array pattern matches exactly + iff it's length corresponds to the length of the matched array. Hence, + if we have a set of flat array patterns matching an array value + <code>a</code>, it suffices to generate a Core <code>case</code> + expression that scrutinises <code>lengthP a</code> and has one + alternative for every length of array occuring in one of the patterns. + Moreover, there needs to be a default case catching all other array + lengths. In each alternative, array indexing (i.e., the functions + <code>!:</code>) is used to bind array elements to the corresponding + pattern variables. This sounds easy enough and is essentially what the + parallel array equation of the function <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsUtils.lhs"><code>DsUtils</code></a><code>.mkCoAlgCaseMatchResult</code> + does. + <p> + Unfortunately, however, the pattern matching compiler expects that it + can turn (almost) any pattern into variable patterns, literals, or + constructor applications by way of the functions <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/Match.lhs"><code>Match</code></a><code>.tidy1</code>. + And to make matters worse, some weird machinery in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/Check.lhs"><code>Check</code></a> + insists on being able to reverse the process (essentially to pretty + print patterns in warnings for incomplete or overlapping patterns). + <p> + The solution to this is an (unlimited) set of <em>fake</em> constructors + for parallel arrays, courtesy of <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/TysWiredIn.lhs"><code>TysWiredIn</code></a><code>.parrFakeCon</code>. + In other words, any pattern of the form <code>[:<i>p<sub>1</sub></i>, + ..., <i>p<sub>n</sub></i>:]</code> is transformed into + <blockquote> + <code> + MkPArray<i>n</i> <i>p<sub>1</sub></i> ... <i>p<sub>n</sub></i> + </code> + </blockquote> + <p> + by <code>Match.tidy1</code>, then, run through the rest of the pattern + matching compiler, and finally, picked up by + <code>DsUtils.mkCoAlgCaseMatchResult</code>, which converts it into a + <code>case</code> expression as outlined above. + <p> + As an example consider the source expression + <blockquote><pre> +case v of + [:x1:] -> e1 + [:x2, x3, x4:] -> e2 + _ -> e3</pre> + </blockquote> + <p> + <code>Match.tidy1</code> converts it into a form that is equivalent to + <blockquote><pre> +case v of { + MkPArr1 x1 -> e1; + MkPArr2 x2 x3 x4 -> e2; + _ -> e3; +}</pre> + </blockquote> + <p> + which <code>DsUtils.mkCoAlgCaseMatchResult</code> turns into the + following Core code: + <blockquote><pre> + case lengthP v of + Int# i# -> + case i# of l { + DFT -> e3 + 1 -> let x1 = v!:0 in e1 + 3 -> let x2 = v!:0; x2 = v!:1; x3 = v!:2 in e2 + }</pre> + </blockquote> + + <h4>Parallel Array Comprehensions</h4> + <p> + The most challenging construct of the three are array comprehensions. + In principle, it would be possible to transform them in essentially the + same way as list comprehensions, but this would lead to abysmally slow + code as desugaring of list comprehensions generates code that is + optimised for sequential, constructor-based structures. In contrast, + array comprehensions need to be transformed into code that solely relies + on collective operations and avoids the creation of many small + intermediate arrays. + <p> + The transformation is implemented by the function <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsListComp.lhs"><code>DsListComp</code></a><code>.dsPArrComp</code>. + In the following, we denote this transformation function by the form + <code><<<i>e</i>>> pa ea</code>, where <code><i>e</i></code> + is the comprehension to be compiled and the arguments <code>pa</code> + and <code>ea</code> denote a pattern and the currently processed array + expression, respectively. The invariant constraining these two + arguments is that all elements in the array produced by <code>ea</code> + will <em>successfully</em> match against <code>pa</code>. + <p> + Given a source-level comprehensions <code>[:e | qss:]</code>, we compile + it with <code><<[:e | qss:]>> () [:():]</code> using the + rules + <blockquote><pre> +<<[:e' | :]>> pa ea = mapP (\pa -> e') ea +<<[:e' | b , qs:]>> pa ea = <<[:e' | qs:]>> pa (filterP (\pa -> b) ea) +<<[:e' | p <- e, qs:]>> pa ea = + let ef = filterP (\x -> case x of {p -> True; _ -> False}) e + in + <<[:e' | qs:]>> (pa, p) (crossP ea ef) +<<[:e' | let ds, qs:]>> pa ea = + <<[:e' | qs:]>> (pa, (x_1, ..., x_n)) + (mapP (\v@pa -> (v, let ds in (x_1, ..., x_n))) ea) +where + {x_1, ..., x_n} = DV (ds) -- Defined Variables +<<[:e' | qs | qss:]>> pa ea = + <<[:e' | qss:]>> (pa, (x_1, ..., x_n)) + (zipP ea <<[:(x_1, ..., x_n) | qs:]>>) +where + {x_1, ..., x_n} = DV (qs)</pre> + </blockquote> + <p> + We assume the denotation of <code>crossP</code> to be given by + <blockquote><pre> +crossP :: [:a:] -> [:b:] -> [:(a, b):] +crossP a1 a2 = let + len1 = lengthP a1 + len2 = lengthP a2 + x1 = concatP $ mapP (replicateP len2) a1 + x2 = concatP $ replicateP len1 a2 + in + zipP x1 x2</pre> + </blockquote> + <p> + For a more efficient implementation of <code>crossP</code>, see + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelPArr.lhs"><code>PrelPArr</code></a>. + <p> + Moreover, the following optimisations are important: + <ul> + <li>In the <code>p <- e</code> rule, if <code>pa == ()</code>, drop + it and simplify the <code>crossP ea e</code> to <code>e</code>. + <li>We assume that fusion will optimise sequences of array processing + combinators. + <li>FIXME: Do we want to have the following function? + <blockquote><pre> +mapFilterP :: (a -> Maybe b) -> [:a:] -> [:b:]</pre> + </blockquote> + <p> + Even with fusion <code>(mapP (\p -> e) . filterP (\p -> + b))</code> may still result in redundant pattern matching + operations. (Let's wait with this until we have seen what the + Simplifier does to the generated code.) + </ul> + + <h2>Doing Away With Nested Arrays: The Flattening Transformation</h2> + <p> + On the quest towards an entirely unboxed representation of parallel + arrays, the flattening transformation is the essential ingredient. GHC + uses a <a + href="http://www.cse.unsw.edu.au/~chak/papers/CK00.html">substantially + improved version</a> of the transformation whose original form was + described by Blelloch & Sabot. The flattening transformation + replaces values of type <code>[:a:]</code> as well as functions + operating on these values by alternative, more efficient data structures + and functions. + <p> + The flattening machinery is activated by the option + <code>-fflatten</code>, which is a static hsc option. It consists of + two steps: function vectorisation and array specialisation. Only the + first of those is implemented so far. If selected, the transformation + is applied to a module in Core form immediately after the <a + href="../the-beast/desugar.html">desugarer,</a> before the <a + href="../the-beast/simplifier.html">Mighty Simplifier</a> gets to do its + job. After vectorisation, the Core program can be dumped using the + option <code>-ddump-vect</code>. The is a good reason for us to perform + flattening immediately after the desugarer. In <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/HscMain.lhs"><code>HscMain</code></a><code>.hscRecomp</code> + the so-called <em>persistent compiler state</em> is maintained, which + contains all the information about imported interface files needed to + lookup the details of imported names (e.g., during renaming and type + checking). However, all this information is zapped before the + simplifier is invoked (supposedly to reduce the space-consumption of + GHC). As flattening has to get at all kinds of identifiers from Prelude + modules, we need to do it before the relevant information in the + persistent compiler state is gone. + + <p> + As flattening generally requires all libraries to be compiled for + flattening (just like profiling does), there is a <em>compiler way</em> + <code>"ndp"</code>, which can be selected using the way option code + <code>-ndp</code>. This option will automagically select + <code>-fparr</code> and <code>-fflatten</code>. + + <h4><code>FlattenMonad</code></h4> + <p> + The module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/ndpFlatten/FlattenMonad.lhs"><code>FlattenMonad</code></a> + implements the monad <code>Flatten</code> that is used during + vectorisation to keep track of various sets of bound variables and a + variable substitution map; moreover, it provides a supply of new uniques + and allows us to look up names in the persistent compiler state (i.e., + imported identifiers). + <p> + In order to be able to look up imported identifiers in the persistent + compiler state, it is important that these identifies are included into + the free variable lists computed by the renamer. More precisely, all + names needed by flattening are included in the names produced by + <code>RnEnv.getImplicitModuleFVs</code>. To avoid putting + flattening-dependent lists of names into the renamer, the module + <code>FlattenInfo</code> exports <code>namesNeededForFlattening</code>. + + [FIXME: It might be worthwhile to document in the non-Flattening part of + the Commentary that the persistent compiler state is zapped after + desugaring and how the free variables determined by the renamer imply + which names are imported.] + + <p><small> +<!-- hhmts start --> +Last modified: Tue Feb 12 01:44:21 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/exts/th.html b/docs/comm/exts/th.html new file mode 100644 index 0000000000..dbb168aa0e --- /dev/null +++ b/docs/comm/exts/th.html @@ -0,0 +1,197 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Template Haskell</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Template Haskell</h1> + <p> + The Template Haskell (TH) extension to GHC adds a meta-programming + facility in which all meta-level code is executed at compile time. The + design of this extension is detailed in "Template Meta-programming for + Haskell", Tim Sheard and Simon Peyton Jones, <a + href="http://portal.acm.org/toc.cfm?id=581690&type=proceeding&coll=portal&dl=ACM&part=series&WantType=proceedings&idx=unknown&title=unknown">ACM + SIGPLAN 2002 Haskell Workshop,</a> 2002. However, some of the details + changed after the paper was published. + </p> + + <h2>Meta Sugar</h2> + <p> + The extra syntax of TH (quasi-quote brackets, splices, and reification) + is handled in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsMeta.hs"><code>DsMeta</code></a>. + In particular, the function <code>dsBracket</code> desugars the four + types of quasi-quote brackets (<code>[|...|]</code>, + <code>[p|...|]</code>, <code>[d|...|]</code>, and <code>[t|...|]</code>) + and <code>dsReify</code> desugars the three types of reification + operations (<code>reifyType</code>, <code>reifyDecl</code>, and + <code>reifyFixity</code>). + </p> + + <h3>Desugaring of Quasi-Quote Brackets</h3> + <p> + A term in quasi-quote brackets needs to be translated into Core code + that, when executed, yields a <em>representation</em> of that term in + the form of the abstract syntax trees defined in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/template-haskell/Language/Haskell/TH/Syntax.hs"><code>Language.Haskell.TH.Syntax</code></a>. + Within <code>DsMeta</code>, this is achieved by four functions + corresponding to the four types of quasi-quote brackets: + <code>repE</code> (for <code>[|...|]</code>), <code>repP</code> (for + <code>[p|...|]</code>), <code>repTy</code> (for <code>[t|...|]</code>), + and <code>repTopDs</code> (for <code>[d|...|]</code>). All four of + these functions receive as an argument the GHC-internal Haskell AST of + the syntactic form that they quote (i.e., arguments of type <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsExpr.lhs"><code>HsExpr</code></a><code>.HsExpr + Name</code>, <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsPat.lhs"><code>HsPat</code></a><code>.HsPat Name</code>, + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsTypes.lhs"><code>HsType</code></a><code>.HsType + Name</code>, and <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsDecls.lhs"><code>HsDecls</code></a><code>.HsGroup + Name</code>, respectively). + </p> + <p> + To increase the static type safety in <code>DsMeta</code>, the functions + constructing representations do not just return plain values of type <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/coreSyn/CoreSyn.lhs"><code>CoreSyn</code></a> + <code>.CoreExpr</code>; instead, <code>DsMeta</code> introduces a + parametrised type <code>Core</code> whose dummy type parameter indicates + the source-level type of the value computed by the corresponding Core + expression. All construction of Core fragments in <code>DsMeta</code> + is performed by smart constructors whose type signatures use the dummy + type parameter to constrain the contexts in which they are applicable. + For example, a function that builds a Core expression that evaluates to + a TH type representation, which has type + <code>Language.Haskell.TH.Syntax.Type</code>, would return a value of + type + </p> + <blockquote> + <pre> +Core Language.Haskell.TH.Syntax.Type</pre> + </blockquote> + + <h3>Desugaring of Reification Operators</h3> + <p> + The TH paper introduces four reification operators: + <code>reifyType</code>, <code>reifyDecl</code>, + <code>reifyFixity</code>, and <code>reifyLocn</code>. Of these, + currently (= 9 Nov 2002), only the former two are implemented. + </p> + <p> + The operator <code>reifyType</code> receives the name of a function or + data constructor as its argument and yields a representation of this + entity's type in the form of a value of type + <code>TH.Syntax.Type</code>. Similarly, <code>reifyDecl</code> receives + the name of a type and yields a representation of the type's declaration + as a value of type <code>TH.Syntax.Decl</code>. The name of the reified + entity is mapped to the GHC-internal representation of the entity by + using the function <code>lookupOcc</code> on the name. + </p> + + <h3>Representing Binding Forms</h3> + <p> + Care needs to be taken when constructing TH representations of Haskell + terms that include binding forms, such as lambda abstractions or let + bindings. To avoid name clashes, fresh names need to be generated for + all defined identifiers. This is achieved via the routine + <code>DsMeta.mkGenSym</code>, which, given a <code>Name</code>, produces + a <code>Name</code> / <code>Id</code> pair (of type + <code>GenSymBind</code>) that associates the given <code>Name</code> + with a Core identifier that at runtime will be bound to a string that + contains the fresh name. Notice the two-level nature of this + arrangement. It is necessary, as the Core code that constructs the + Haskell term representation may be executed multiple types at runtime + and it must be ensured that different names are generated in each run. + </p> + <p> + Such fresh bindings need to be entered into the meta environment (of + type <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsMonad.lhs"><code>DsMonad</code></a><code>.DsMetaEnv</code>), + which is part of the state (of type <code>DsMonad.DsEnv</code>) + maintained in the desugarer monad (of type <code>DsMonad.DsM</code>). + This is done using the function <code>DsMeta.addBinds</code>, which + extends the current environment by a list of <code>GenSymBind</code>s + and executes a subcomputation in this extended environment. Names can + be looked up in the meta environment by way of the functions + <code>DsMeta.lookupOcc</code> and <code>DsMeta.lookupBinder</code>; more + details about the difference between these two functions can be found in + the next subsection. + </p> + <p> + NB: <code>DsMeta</code> uses <code>mkGenSym</code> only when + representing terms that may be embedded into a context where names can + be shadowed. For example, a lambda abstraction embedded into an + expression can potentially shadow names defined in the context it is + being embedded into. In contrast, this can never be the case for + top-level declarations, such as data type declarations; hence, the type + variables that a parametric data type declaration abstracts over are not + being gensym'ed. As a result, variables in defining positions are + handled differently depending on the syntactic construct in which they + appear. + </p> + + <h3>Binders Versus Occurences</h3> + <p> + Name lookups in the meta environment of the desugarer use two functions + with slightly different behaviour, namely <code>DsMeta.lookupOcc</code> + and <code>lookupBinder</code>. The module <code>DsMeta</code> contains + the following explanation as to the difference of these functions: + </p> + <blockquote> + <pre> +When we desugar [d| data T = MkT |] +we want to get + Data "T" [] [Con "MkT" []] [] +and *not* + Data "Foo:T" [] [Con "Foo:MkT" []] [] +That is, the new data decl should fit into whatever new module it is +asked to fit in. We do *not* clone, though; no need for this: + Data "T79" .... + +But if we see this: + data T = MkT + foo = reifyDecl T + +then we must desugar to + foo = Data "Foo:T" [] [Con "Foo:MkT" []] [] + +So in repTopDs we bring the binders into scope with mkGenSyms and addBinds, +but in dsReify we do not. And we use lookupOcc, rather than lookupBinder +in repTyClD and repC.</pre> + </blockquote> + <p> + This implies that <code>lookupOcc</code>, when it does not find the name + in the meta environment, uses the function <code>DsMeta.globalVar</code> + to construct the <em>original name</em> of the entity (cf. the TH paper + for more details regarding original names). This name uniquely + identifies the entity in the whole program and is in scope + <em>independent</em> of whether the user name of the same entity is in + scope or not (i.e., it may be defined in a different module without + being explicitly imported) and has the form <module>:<name>. + <strong>NB:</strong> Incidentally, the current implementation of this + mechanisms facilitates breaking any abstraction barrier. + </p> + + <h3>Known-key Names for Template Haskell</h3> + <p> + During the construction of representations, the desugarer needs to use a + large number of functions defined in the library + <code>Language.Haskell.TH.Syntax</code>. The names of these functions + need to be made available to the compiler in the way outlined <a + href="../the-beast/prelude.html">Primitives and the Prelude.</a> + Unfortunately, any change to <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/PrelNames.lhs"><code>PrelNames</code></a> + triggers a significant amount of recompilation. Hence, the names needed + for TH are defined in <code>DsMeta</code> instead (at the end of the + module). All library functions needed by TH are contained in the name + set <code>DsMeta.templateHaskellNames</code>. + </p> + + <p><small> +<!-- hhmts start --> +Last modified: Wed Nov 13 18:01:48 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/feedback.html b/docs/comm/feedback.html new file mode 100644 index 0000000000..1da8b10f29 --- /dev/null +++ b/docs/comm/feedback.html @@ -0,0 +1,34 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Feedback</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>Feedback</h1> + <p> + <a href="mailto:chak@cse.unsw.edu.au">I</a> welcome any feedback on the + material and in particular would appreciated comments on which parts of + the document are incomprehensible or miss explanation -- e.g., due to + the use of GHC speak that is explained nowhere (words like infotable or + so). Moreover, I would be interested to know which areas of GHC you + would like to see covered here. + <p> + For the moment is probably best if feedback is directed to + <p> + <blockquote> + <a + href="mailto:chak@cse.unsw.edu.au"><code>chak@cse.unsw.edu.au</code></a> + </blockquote> + <p> + However, if there is sufficient interest, we might consider setting up a + mailing list. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 8 00:11:42 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/genesis/genesis.html b/docs/comm/genesis/genesis.html new file mode 100644 index 0000000000..30b16fec46 --- /dev/null +++ b/docs/comm/genesis/genesis.html @@ -0,0 +1,82 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Outline of the Genesis</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Outline of the Genesis</h1> + <p> + Building GHC happens in two stages: First you have to prepare the tree + with <code>make boot</code>; and second, you build the compiler and + associated libraries with <code>make all</code>. The <code>boot</code> + stage builds some tools used during the main build process, generates + parsers and other pre-computed source, and finally computes dependency + information. There is considerable detail on the build process in GHC's + <a + href="http://haskell.cs.yale.edu/ghc/docs/latest/building/building-guide.html">Building Guide.</a> + + <h4>Debugging the Beast</h4> + <p> + If you are hacking the compiler or like to play with unstable + development versions, chances are that the compiler someday just crashes + on you. Then, it is a good idea to load the <code>core</code> into + <code>gdb</code> as usual, but unfortunately there is usually not too + much useful information. + <p> + The next step, then, is somewhat tedious. You should build a compiler + producing programs with a runtime system that has debugging turned on + and use that to build the crashing compiler. There are many sanity + checks in the RTS, which may detect inconsistency before they lead to a + crash and you may include more debugging information, which helps + <code>gdb.</code> For a RTS with debugging turned on, add the following + to <code>build.mk</code> (see also the comment in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/mk/config.mk.in"><code>config.mk.in</code></a> that you find when searching for + <code>GhcRtsHcOpts</code>): +<blockquote><pre> +GhcRtsHcOpts+=-optc-DDEBUG +GhcRtsCcOpts+=-g +EXTRA_LD_OPTS=-lbfd -liberty</pre></blockquote> + <p> + Then go into <code>fptools/ghc/rts</code> and <code>make clean boot && + make all</code>. With the resulting runtime system, you have to re-link + the compiler. Go into <code>fptools/ghc/compiler</code>, delete the + file <code>hsc</code> (up to version 4.08) or + <code>ghc-<version></code>, and execute <code>make all</code>. + <p> + The <code>EXTRA_LD_OPTS</code> are necessary as some of the debugging + code uses the BFD library, which in turn requires <code>liberty</code>. + I would also recommend (in 4.11 and from 5.0 upwards) adding these linker + options to the files <code>package.conf</code> and + <code>package.conf.inplace</code> in the directory + <code>fptools/ghc/driver/</code> to the <code>extra_ld_opts</code> entry + of the package <code>RTS</code>. Otherwise, you have to supply them + whenever you compile and link a program with a compiler that uses the + debugging RTS for the programs it produces. + <p> + To run GHC up to version 4.08 in <code>gdb</code>, first invoke the + compiler as usual, but pass it the option <code>-v</code>. This will + show you the exact invocation of the compiler proper <code>hsc</code>. + Run <code>hsc</code> with these options in <code>gdb</code>. The + development version 4.11 and stable releases from 5.0 on do no longer + use the Perl driver; so, you can run them directly with gdb. + <p> + <strong>Debugging a compiler during building from HC files.</strong> + If you are boot strapping the compiler on new platform from HC files and + it crashes somewhere during the build (e.g., when compiling the + libraries), do as explained above, but you may have to re-configure the + build system with <code>--enable-hc-boot</code> before re-making the + code in <code>fptools/ghc/driver/</code>. + If you do this with a compiler up to version 4.08, run the build process + with <code>make EXTRA_HC_OPTS=-v</code> to get the exact arguments with + which you have to invoke <code>hsc</code> in <code>gdb</code>. + + <p><small> +<!-- hhmts start --> +Last modified: Sun Apr 24 22:16:30 CEST 2005 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/genesis/makefiles.html b/docs/comm/genesis/makefiles.html new file mode 100644 index 0000000000..957a82eb85 --- /dev/null +++ b/docs/comm/genesis/makefiles.html @@ -0,0 +1,51 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Mindboggling Makefiles</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Mindboggling Makefiles</h1> + <p> + The size and structure of GHC's makefiles makes it quite easy to scream + out loud - in pain - during the process of tracking down problems in the + make system or when attempting to alter it. GHC's <a + href="http://haskell.cs.yale.edu/ghc/docs/latest/building/building-guide.html">Building + Guide</a> has valuable information on <a + href="http://haskell.cs.yale.edu/ghc/docs/latest/building/sec-makefile-arch.html">the + makefile architecture.</a> + + <h4>A maze of twisty little passages, all alike</h4> + <p> + The <code>fptools/</code> toplevel and the various project directories + contain not only a <code>Makefile</code> each, but there are + subdirectories of name <code>mk/</code> at various levels that contain + rules, targets, and so on specific to a project - or, in the case of the + toplevel, the default rules for the whole system. Each <code>mk/</code> + directory contains a file <code>boilerplate.mk</code> that ties the + various other makefiles together. Files called <code>target.mk</code>, + <code>paths.mk</code>, and <code>suffix.mk</code> contain make targets, + definitions of variables containing paths, and suffix rules, + respectively. + <p> + One particularly nasty trick used in this hierarchy of makefiles is the + way in which the variable <code>$(TOP)</code> is used. AFAIK, + <code>$(TOP)</code> always points to a directory containing an + <code>mk/</code> subdirectory; however, it not necessarily points to the + toplevel <code>fptools/</code> directory. For example, within the GHC + subtree, <code>$(TOP)</code> points to <code>fptools/ghc/</code>. + However, some of the makefiles in <code>fptools/ghc/mk/</code> will then + <em>temporarily</em> redefine <code>$(TOP)</code> to point a level + higher (i.e., to <code>fptools/</code>) while they are including the + toplevel boilerplate. After that <code>$(TOP)</code> is redefined to + whatever value it had before including makefiles from higher up in the + hierarchy. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 22 16:46:33 GMT Daylight Time 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/genesis/modules.html b/docs/comm/genesis/modules.html new file mode 100644 index 0000000000..de59cce6d3 --- /dev/null +++ b/docs/comm/genesis/modules.html @@ -0,0 +1,164 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Marvellous Module Structure of GHC </title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Marvellous Module Structure of GHC </h1> + <p> + +GHC is built out of about 245 Haskell modules. It can be quite tricky +to figure out what the module dependency graph looks like. It can be +important, too, because loops in the module dependency graph need to +be broken carefully using <tt>.hi-boot</tt> interface files. +<p> +This section of the commentary documents the subtlest part of +the module dependency graph, namely the part near the bottom. +<ul> +<li> The list is given in compilation order: that is, +module near the top are more primitive, and are compiled earlier. +<li> Each module is listed together with its most critical +dependencies in parentheses; that is, the dependencies that prevent it being +compiled earlier. +<li> Modules in the same bullet don't depend on each other. +<li> Loops are documented by a dependency such as "<tt>loop Type.Type</tt>". +This means tha the module imports <tt>Type.Type</tt>, but module <tt>Type</tt> +has not yet been compiled, so the import comes from <tt>Type.hi-boot</tt>. +</ul> + +Compilation order is as follows: +<ul> +<li> +<strong>First comes a layer of modules that have few interdependencies, +and which implement very basic data types</strong>: +<tt> <ul> +<li> Util +<li> OccName +<li> Pretty +<li> Outputable +<li> StringBuffer +<li> ListSetOps +<li> Maybes +<li> etc +</ul> </tt> + +<p> +<li> <strong> Now comes the main subtle layer, involving types, classes, type constructors +identifiers, expressions, rules, and their operations.</strong> + +<tt> +<ul> +<li> Name <br> PrimRep +<p><li> + PrelNames +<p><li> + Var (Name, loop IdInfo.IdInfo, + loop Type.Type, loop Type.Kind) +<p><li> + VarEnv, VarSet, ThinAir +<p><li> + Class (loop TyCon.TyCon, loop Type.Type) +<p><li> + TyCon (loop Type.Type, loop Type.Kind, loop DataCon.DataCon, loop Generics.GenInfo) +<p><li> + TypeRep (loop DataCon.DataCon, loop Subst.substTyWith) +<p><li> + Type (loop PprType.pprType, loop Subst.substTyWith) +<p><li> + FieldLabel(Type) <br> + TysPrim(Type) <br> +<p><li> + Literal (TysPrim, PprType) <br> + DataCon (loop PprType, loop Subst.substTyWith, FieldLabel.FieldLabel) +<p><li> + TysWiredIn (loop MkId.mkDataConIds) +<p><li> + TcType( lots of TysWiredIn stuff) +<p><li> + PprType( lots of TcType stuff ) +<p><li> + PrimOp (PprType, TysWiredIn) +<p><li> + CoreSyn [does not import Id] +<p><li> + IdInfo (CoreSyn.Unfolding, CoreSyn.CoreRules) +<p><li> + Id (lots from IdInfo) +<p><li> + CoreFVs <br> + PprCore +<p><li> + CoreUtils (PprCore.pprCoreExpr, CoreFVs.exprFreeVars, + CoreSyn.isEvaldUnfolding CoreSyn.maybeUnfoldingTemplate) +<p><li> + CoreLint( CoreUtils ) <br> + OccurAnal (CoreUtils.exprIsTrivial) <br> + CoreTidy (CoreUtils.exprArity ) <br> +<p><li> + CoreUnfold (OccurAnal.occurAnalyseGlobalExpr) +<p><li> + Subst (CoreUnfold.Unfolding, CoreFVs) <br> + Generics (CoreUnfold.mkTopUnfolding) <br> + Rules (CoreUnfold.Unfolding, PprCore.pprTidyIdRules) +<p><li> + MkId (CoreUnfold.mkUnfolding, Subst, Rules.addRule) +<p><li> + PrelInfo (MkId) <br> + HscTypes( Rules.RuleBase ) +</ul></tt> + +<p><li> <strong>That is the end of the infrastructure. Now we get the + main layer of mdoules that perform useful work.</strong> + +<tt><ul> +<p><li> + CoreTidy (HscTypes.PersistentCompilerState) +</ul></tt> +</ul> + +HsSyn stuff +<ul> +<li> HsPat.hs-boot +<li> HsExpr.hs-boot (loop HsPat.LPat) +<li> HsTypes (loop HsExpr.HsSplice) +<li> HsBinds (HsTypes.LHsType, loop HsPat.LPat, HsExpr.pprFunBind and others) + HsLit (HsTypes.SyntaxName) +<li> HsPat (HsBinds, HsLit) + HsDecls (HsBinds) +<li> HsExpr (HsDecls, HsPat) +</ul> + + + +<h2>Library stuff: base package</h2> + +<ul> +<li> GHC.Base +<li> Data.Tuple (GHC.Base), GHC.Ptr (GHC.Base) +<li> GHC.Enum (Data.Tuple) +<li> GHC.Show (GHC.Enum) +<li> GHC.Num (GHC.Show) +<li> GHC.ST (GHC.Num), GHC.Real (GHC.Num) +<li> GHC.Arr (GHC.ST) GHC.STRef (GHC.ST) +<li> GHC.IOBase (GHC.Arr) +<li> Data.Bits (GHC.Real) +<li> Data.HashTable (Data.Bits, Control.Monad) +<li> Data.Typeable (GHC.IOBase, Data.HashTable) +<li> GHC.Weak (Data.Typeable, GHC.IOBase) +</ul> + + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 22 16:46:33 GMT Daylight Time 2001 +<!-- hhmts end --> + </small> + </body> +</html> + + + + + diff --git a/docs/comm/index.html b/docs/comm/index.html new file mode 100644 index 0000000000..5ccd5f0ca9 --- /dev/null +++ b/docs/comm/index.html @@ -0,0 +1,121 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Beast Explained</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The Glasgow Haskell Compiler (GHC) Commentary [v0.17]</h1> + <p> + <!-- Contributors: Whoever makes substantial additions or changes to the + document, please add your name and keep the order alphabetic. Moreover, + please bump the version number for any substantial modification that you + check into CVS. + --> + <strong>Manuel M. T. Chakravarty</strong><br> + <strong>Sigbjorn Finne</strong><br> + <strong>Simon Marlow</strong><br> + <strong>Simon Peyton Jones</strong><br> + <strong>Julian Seward</strong><br> + <strong>Reuben Thomas</strong><br> + <br> + <p> + This document started as a collection of notes describing what <a + href="mailto:chak@cse.unsw.edu.au">I</a> learnt when poking around in + the <a href="http://haskell.org/ghc/">GHC</a> sources. During the + <i>Haskell Implementers Workshop</i> in January 2001, it was decided to + put the commentary into + <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/">GHC's CVS + repository</a> + to allow the whole developer community to add their wizardly insight to + the document. + <p> + <strong>The document is still far from being complete - help it + grow!</strong> + + <h2>Before the Show Begins</h2> + <p> + <ul> + <li><a href="feedback.html">Feedback</a> + <li><a href="others.html">Other Sources of Wisdom</a> + </ul> + + <h2>Genesis</h2> + <p> + <ul> + <li><a href="genesis/genesis.html">Outline of the Genesis</a> + <li><a href="genesis/makefiles.html">Mindboggling Makefiles</a> + <li><a href="genesis/modules.html">GHC's Marvellous Module Structure</a> + </ul> + + <h2>The Beast Dissected</h2> + <p> + <ul> + <li><a href="the-beast/coding-style.html">Coding style used in + the compiler</a> + <li><a href="the-beast/driver.html">The Glorious Driver</a> + <li><a href="the-beast/prelude.html">Primitives and the Prelude</a> + <li><a href="the-beast/syntax.html">Just Syntax</a> + <li><a href="the-beast/basicTypes.html">The Basics</a> + <li><a href="the-beast/modules.html">Modules, ModuleNames and + Packages</a> + <li><a href="the-beast/names.html">The truth about names: Names and OccNames</a> + <li><a href="the-beast/vars.html">The Real Story about Variables, Ids, + TyVars, and the like</a> + <li><a href="the-beast/data-types.html">Data types and constructors</a> + <li><a href="the-beast/renamer.html">The Glorious Renamer</a> + <li><a href="the-beast/types.html">Hybrid Types</a> + <li><a href="the-beast/typecheck.html">Checking Types</a> + <li><a href="the-beast/desugar.html">Sugar Free: From Haskell To Core</a> + <li><a href="the-beast/simplifier.html">The Mighty Simplifier</a> + <li><a href="the-beast/mangler.html">The Evil Mangler</a> + <li><a href="the-beast/alien.html">Alien Functions</a> + <li><a href="the-beast/stg.html">You Got Control: The STG-language</a> + <li><a href="the-beast/ncg.html">The Native Code Generator</a> + <li><a href="the-beast/ghci.html">GHCi</a> + <li><a href="the-beast/fexport.html">Implementation of + <code>foreign export</code></a> + <li><a href="the-beast/main.html">Compiling and running the Main module</code></a> + </ul> + + <h2>RTS & Libraries</h2> + <p> + <ul> + <li><a href="rts-libs/coding-style.html">Coding Style Guidelines</a> + <li><a href="rts-libs/stgc.html">Spineless Tagless C</a> + <li><a href="rts-libs/primitives.html">Primitives</a> + <li><a href="rts-libs/prelfound.html">Prelude Foundations</a> + <li><a href="rts-libs/prelude.html">Cunning Prelude Code</a> + <li><a href="rts-libs/foreignptr.html">On why we have <tt>ForeignPtr</tt></a> + <li><a href="rts-libs/non-blocking.html">Non-blocking I/O for Win32</a> + <li><a href="rts-libs/multi-thread.html">Supporting multi-threaded interoperation</a> + </ul> + + <h2>Extensions, or Making a Complicated System More Complicated</h2> + <p> + <ul> + <li><a href="exts/th.html">Template Haskell</a> + <li><a href="exts/ndp.html">Parallel Arrays</a> + </ul> + + <h2>The Source</h2> + <p> + The online master copy of the Commentary is at + <blockquote> + <a href="http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/">http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/</a> + </blockquote> + <p> + This online version is updated + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/docs/comm/">from + CVS</a> + daily. + + <p><small> +<!-- hhmts start --> +Last modified: Thu May 12 19:03:42 EST 2005 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/others.html b/docs/comm/others.html new file mode 100644 index 0000000000..52d87e9419 --- /dev/null +++ b/docs/comm/others.html @@ -0,0 +1,60 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Other Sources of Wisdom</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>Other Sources of Wisdom</h1> + <p> + Believe it or not, but there are other people besides you who are + masochistic enough to study the innards of the beast. Some of the have + been kind (or cruel?) enough to share their insights with us. Here is a + probably incomplete list: + <p> + <ul> + + <li>The <a + href="http://www.cee.hw.ac.uk/~dsg/gph/docs/StgSurvival.ps.gz">STG + Survival Sheet</a> has -- according to its header -- been written by + `a poor wee soul',<sup><a href="#footnote1">1</a></sup> which + probably has been pushed into the torments of madness by the very + act of contemplating the inner workings of the STG runtime system. + This document discusses GHC's runtime system with a focus on + support for parallel processing (aka GUM). + + <li>Instructions on <a + href="http://www-users.cs.york.ac.uk/~olaf/PUBLICATIONS/extendGHC.html">Adding + an Optimisation Pass to the Glasgow Haskell Compiler</a> + have been compiled by <a + href="http://www-users.cs.york.ac.uk/~olaf/">Olaf Chitil</a>. + Unfortunately, this document is already a little aged. + + <li><a href="http://www.cs.pdx.edu/~apt/">Andrew Tolmach</a> has defined + <a href="http://www.haskell.org/ghc/docs/papers/core.ps.gz">an external + representation of + GHC's <em>Core</em> language</a> and also implemented a GHC pass + that emits the intermediate form into <code>.hcr</code> files. The + option <code>-fext-core</code> triggers GHC to emit Core code after + optimisation; in addition, <code>-fno-code</code> is often used to + stop compilation after Core has been emitted. + + <!-- Add references to other background texts listed on the GHC docu + page + --> + + </ul> + + <p><hr><p> + <sup><a name="footnote1">1</a></sup>Usually reliable sources have it that + the poor soul in question is no one less than GUM hardcore hacker <a + href="http://www.cee.hw.ac.uk/~hwloidl/">Hans-Wolfgang Loidl</a>. + + <p><small> +<!-- hhmts start --> +Last modified: Tue Nov 13 10:56:57 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/coding-style.html b/docs/comm/rts-libs/coding-style.html new file mode 100644 index 0000000000..58f5b4f9bb --- /dev/null +++ b/docs/comm/rts-libs/coding-style.html @@ -0,0 +1,516 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Style Guidelines for RTS C code</title> + </head> + +<body> +<H1>The GHC Commentary - Style Guidelines for RTS C code</h1> + +<h2>Comments</h2> + +<p>These coding style guidelines are mainly intended for use in +<tt>ghc/rts</tt> and <tt>ghc/includes</tt>. + +<p>NB These are just suggestions. They're not set in stone. Some of +them are probably misguided. If you disagree with them, feel free to +modify this document (and make your commit message reasonably +informative) or mail someone (eg. <a +href="glasgow-haskell-users@haskell.org">The GHC mailing list</a>) + +<h2>References</h2> + +If you haven't read them already, you might like to check the following. +Where they conflict with our suggestions, they're probably right. + +<ul> + +<li> +The C99 standard. One reasonable reference is <a +href="http://home.tiscalinet.ch/t_wolf/tw/c/c9x_changes.html">here</a>. + +<p><li> +Writing Solid Code, Microsoft Press. (Highly recommended. Possibly +the only Microsoft Press book that's worth reading.) + +<p><li> +Autoconf documentation. +See also <a href="http://peti.gmd.de/autoconf-archive/">The autoconf macro archive</a> and +<a href="http://www.cyclic.com/cyclic-pages/autoconf.html">Cyclic Software's description</a> + +<p><li> <a +href="http://www.cs.umd.edu/users/cml/cstyle/indhill-cstyle.html">Indian +Hill C Style and Coding Standards</a>. + +<p><li> +<a href="http://www.cs.umd.edu/users/cml/cstyle/">A list of C programming style links</a> + +<p><li> +<a href="http://www.lysator.liu.se/c/c-www.html">A very large list of C programming links</a> + +<p><li> +<a href="http://www.geek-girl.com/unix.html">A list of Unix programming links</a> + +</ul> + + +<h2>Portability issues</h2> + +<ul> +<p><li> We try to stick to C99 where possible. We use the following +C99 features relative to C89, some of which were previously GCC +extensions (possibly with different syntax): + +<ul> +<p><li>Variable length arrays as the last field of a struct. GCC has +a similar extension, but the syntax is slightly different: in GCC you +would declare the array as <tt>arr[0]</tt>, whereas in C99 it is +declared as <tt>arr[]</tt>. + +<p><li>Inline annotations on functions (see later) + +<p><li>Labeled elements in initialisers. Again, GCC has a slightly +different syntax from C99 here, and we stick with the GCC syntax until +GCC implements the C99 proposal. + +<p><li>C++-style comments. These are part of the C99 standard, and we +prefer to use them whenever possible. +</ul> + +<p>In addition we use ANSI-C-style function declarations and +prototypes exclusively. Every function should have a prototype; +static function prototypes may be placed near the top of the file in +which they are declared, and external prototypes are usually placed in +a header file with the same basename as the source file (although there +are exceptions to this rule, particularly when several source files +together implement a subsystem which is described by a single external +header file). + +<p><li>We use the following GCC extensions, but surround them with +<tt>#ifdef __GNUC__</tt>: + +<ul> +<p><li>Function attributes (mostly just <code>no_return</code> and +<code>unused</code>) +<p><li>Inline assembly. +</ul> + +<p><li> +char can be signed or unsigned - always say which you mean + +<p><li>Our POSIX policy: try to write code that only uses POSIX (IEEE +Std 1003.1) interfaces and APIs. We used to define +<code>POSIX_SOURCE</code> by default, but found that this caused more +problems than it solved, so now we require any code that is +POSIX-compliant to explicitly say so by having <code>#include +"PosixSource.h"</code> at the top. Try to do this whenever possible. + +<p><li> Some architectures have memory alignment constraints. Others +don't have any constraints but go faster if you align things. These +macros (from <tt>ghcconfig.h</tt>) tell you which alignment to use + +<pre> + /* minimum alignment of unsigned int */ + #define ALIGNMENT_UNSIGNED_INT 4 + + /* minimum alignment of long */ + #define ALIGNMENT_LONG 4 + + /* minimum alignment of float */ + #define ALIGNMENT_FLOAT 4 + + /* minimum alignment of double */ + #define ALIGNMENT_DOUBLE 4 +</pre> + +<p><li> Use <tt>StgInt</tt>, <tt>StgWord</tt> and <tt>StgPtr</tt> when +reading/writing ints and ptrs to the stack or heap. Note that, by +definition, <tt>StgInt</tt>, <tt>StgWord</tt> and <tt>StgPtr</tt> are +the same size and have the same alignment constraints even if +<code>sizeof(int) != sizeof(ptr)</code> on that platform. + +<p><li> Use <tt>StgInt8</tt>, <tt>StgInt16</tt>, etc when you need a +certain minimum number of bits in a type. Use <tt>int</tt> and +<tt>nat</tt> when there's no particular constraint. ANSI C only +guarantees that ints are at least 16 bits but within GHC we assume +they are 32 bits. + +<p><li> Use <tt>StgFloat</tt> and <tt>StgDouble</tt> for floating +point values which will go on/have come from the stack or heap. Note +that <tt>StgDouble</tt> may occupy more than one <tt>StgWord</tt>, but +it will always be a whole number multiple. + +<p> +Use <code>PK_FLT(addr)</code>, <code>PK_DBL(addr)</code> to read +<tt>StgFloat</tt> and <tt>StgDouble</tt> values from the stack/heap, +and <code>ASSIGN_FLT(val,addr)</code> / +<code>ASSIGN_DBL(val,addr)</code> to assign StgFloat/StgDouble values +to heap/stack locations. These macros take care of alignment +restrictions. + +<p> +Heap/Stack locations are always <tt>StgWord</tt> aligned; the +alignment requirements of an <tt>StgDouble</tt> may be more than that +of <tt>StgWord</tt>, but we don't pad misaligned <tt>StgDoubles</tt> +because doing so would be too much hassle (see <code>PK_DBL</code> & +co above). + +<p><li> +Avoid conditional code like this: + +<pre> + #ifdef solaris_host_OS + // do something solaris specific + #endif +</pre> + +Instead, add an appropriate test to the configure.ac script and use +the result of that test instead. + +<pre> + #ifdef HAVE_BSD_H + // use a BSD library + #endif +</pre> + +<p>The problem is that things change from one version of an OS to another +- things get added, things get deleted, things get broken, some things +are optional extras. Using "feature tests" instead of "system tests" +makes things a lot less brittle. Things also tend to get documented +better. + +</ul> + +<h2>Debugging/robustness tricks</h2> + + +Anyone who has tried to debug a garbage collector or code generator +will tell you: "If a program is going to crash, it should crash as +soon, as noisily and as often as possible." There's nothing worse +than trying to find a bug which only shows up when running GHC on +itself and doesn't manifest itself until 10 seconds after the actual +cause of the problem. + +<p>We put all our debugging code inside <tt>#ifdef DEBUG</tt>. The +general policy is we don't ship code with debugging checks and +assertions in it, but we do run with those checks in place when +developing and testing. Anything inside <tt>#ifdef DEBUG</tt> should +not slow down the code by more than a factor of 2. + +<p>We also have more expensive "sanity checking" code for hardcore +debugging - this can slow down the code by a large factor, but is only +enabled on demand by a command-line flag. General sanity checking in +the RTS is currently enabled with the <tt>-DS</tt> RTS flag. + +<p>There are a number of RTS flags which control debugging output and +sanity checking in various parts of the system when <tt>DEBUG</tt> is +defined. For example, to get the scheduler to be verbose about what +it is doing, you would say <tt>+RTS -Ds -RTS</tt>. See +<tt>includes/RtsFlags.h</tt> and <tt>rts/RtsFlags.c</tt> for the full +set of debugging flags. To check one of these flags in the code, +write: + +<pre> + IF_DEBUG(gc, fprintf(stderr, "...")); +</pre> + +would check the <tt>gc</tt> flag before generating the output (and the +code is removed altogether if <tt>DEBUG</tt> is not defined). + +<p>All debugging output should go to <tt>stderr</tt>. + +<p> +Particular guidelines for writing robust code: + +<ul> +<p><li> +Use assertions. Use lots of assertions. If you write a comment +that says "takes a +ve number" add an assertion. If you're casting +an int to a nat, add an assertion. If you're casting an int to a char, +add an assertion. We use the <tt>ASSERT</tt> macro for writing +assertions; it goes away when <tt>DEBUG</tt> is not defined. + +<p><li> +Write special debugging code to check the integrity of your data structures. +(Most of the runtime checking code is in <tt>rts/Sanity.c</tt>) +Add extra assertions which call this code at the start and end of any +code that operates on your data structures. + +<p><li> +When you find a hard-to-spot bug, try to think of some assertions, +sanity checks or whatever that would have made the bug easier to find. + +<p><li> +When defining an enumeration, it's a good idea not to use 0 for normal +values. Instead, make 0 raise an internal error. The idea here is to +make it easier to detect pointer-related errors on the assumption that +random pointers are more likely to point to a 0 than to anything else. + +<pre> +typedef enum + { i_INTERNAL_ERROR /* Instruction 0 raises an internal error */ + , i_PANIC /* irrefutable pattern match failed! */ + , i_ERROR /* user level error */ + + ... +</pre> + +<p><li> Use <tt>#warning</tt> or <tt>#error</tt> whenever you write a +piece of incomplete/broken code. + +<p><li> When testing, try to make infrequent things happen often. + For example, make a context switch/gc/etc happen every time a + context switch/gc/etc can happen. The system will run like a + pig but it'll catch a lot of bugs. + +</ul> + +<h2>Syntactic details</h2> + +<ul> +<p><li><b>Important:</b> Put "redundant" braces or parens in your code. +Omitting braces and parens leads to very hard to spot bugs - +especially if you use macros (and you might have noticed that GHC does +this a lot!) + +<p> +In particular: +<ul> +<p><li> +Put braces round the body of for loops, while loops, if statements, etc. +even if they "aren't needed" because it's really hard to find the resulting +bug if you mess up. Indent them any way you like but put them in there! +</ul> + +<p><li> +When defining a macro, always put parens round args - just in case. +For example, write: +<pre> + #define add(x,y) ((x)+(y)) +</pre> +instead of +<pre> + #define add(x,y) x+y +</pre> + +<p><li> Don't declare and initialize variables at the same time. +Separating the declaration and initialization takes more lines, but +make the code clearer. + +<p><li> +Use inline functions instead of macros if possible - they're a lot +less tricky to get right and don't suffer from the usual problems +of side effects, evaluation order, multiple evaluation, etc. + +<ul> +<p><li>Inline functions get the naming issue right. E.g. they + can have local variables which (in an expression context) + macros can't. + +<p><li> Inline functions have call-by-value semantics whereas macros + are call-by-name. You can be bitten by duplicated computation + if you aren't careful. + +<p><li> You can use inline functions from inside gdb if you compile with + -O0 or -fkeep-inline-functions. If you use macros, you'd better + know what they expand to. +</ul> + +However, note that macros can serve as both l-values and r-values and +can be "polymorphic" as these examples show: +<pre> + // you can use this as an l-value or an l-value + #define PROF_INFO(cl) (((StgClosure*)(cl))->header.profInfo) + + // polymorphic case + // but note that min(min(1,2),3) does 3 comparisions instead of 2!! + #define min(x,y) (((x)<=(y)) ? (x) : (y)) +</pre> + +<p><li> +Inline functions should be "static inline" because: +<ul> +<p><li> +gcc will delete static inlines if not used or theyre always inlined. + +<p><li> + if they're externed, we could get conflicts between 2 copies of the + same function if, for some reason, gcc is unable to delete them. + If they're static, we still get multiple copies but at least they don't conflict. +</ul> + +OTOH, the gcc manual says this +so maybe we should use extern inline? + +<pre> + When a function is both inline and `static', if all calls to the +function are integrated into the caller, and the function's address is +never used, then the function's own assembler code is never referenced. +In this case, GNU CC does not actually output assembler code for the +function, unless you specify the option `-fkeep-inline-functions'. +Some calls cannot be integrated for various reasons (in particular, +calls that precede the function's definition cannot be integrated, and +neither can recursive calls within the definition). If there is a +nonintegrated call, then the function is compiled to assembler code as +usual. The function must also be compiled as usual if the program +refers to its address, because that can't be inlined. + + When an inline function is not `static', then the compiler must +assume that there may be calls from other source files; since a global +symbol can be defined only once in any program, the function must not +be defined in the other source files, so the calls therein cannot be +integrated. Therefore, a non-`static' inline function is always +compiled on its own in the usual fashion. + + If you specify both `inline' and `extern' in the function +definition, then the definition is used only for inlining. In no case +is the function compiled on its own, not even if you refer to its +address explicitly. Such an address becomes an external reference, as +if you had only declared the function, and had not defined it. + + This combination of `inline' and `extern' has almost the effect of a +macro. The way to use it is to put a function definition in a header +file with these keywords, and put another copy of the definition +(lacking `inline' and `extern') in a library file. The definition in +the header file will cause most calls to the function to be inlined. +If any uses of the function remain, they will refer to the single copy +in the library. +</pre> + +<p><li> +Don't define macros that expand to a list of statements. +You could just use braces as in: + +<pre> + #define ASSIGN_CC_ID(ccID) \ + { \ + ccID = CC_ID; \ + CC_ID++; \ + } +</pre> + +(but it's usually better to use an inline function instead - see above). + +<p><li> +Don't even write macros that expand to 0 statements - they can mess you +up as well. Use the doNothing macro instead. +<pre> + #define doNothing() do { } while (0) +</pre> + +<p><li> +This code +<pre> +int* p, q; +</pre> +looks like it declares two pointers but, in fact, only p is a pointer. +It's safer to write this: +<pre> +int* p; +int* q; +</pre> +You could also write this: +<pre> +int *p, *q; +</pre> +but it is preferrable to split the declarations. + +<p><li> +Try to use ANSI C's enum feature when defining lists of constants of +the same type. Among other benefits, you'll notice that gdb uses the +name instead of its (usually inscrutable) number when printing values +with enum types and gdb will let you use the name in expressions you +type. + +<p> +Examples: +<pre> + typedef enum { /* N.B. Used as indexes into arrays */ + NO_HEAP_PROFILING, + HEAP_BY_CC, + HEAP_BY_MOD, + HEAP_BY_GRP, + HEAP_BY_DESCR, + HEAP_BY_TYPE, + HEAP_BY_TIME + } ProfilingFlags; +</pre> +instead of +<pre> + # define NO_HEAP_PROFILING 0 /* N.B. Used as indexes into arrays */ + # define HEAP_BY_CC 1 + # define HEAP_BY_MOD 2 + # define HEAP_BY_GRP 3 + # define HEAP_BY_DESCR 4 + # define HEAP_BY_TYPE 5 + # define HEAP_BY_TIME 6 +</pre> +and +<pre> + typedef enum { + CCchar = 'C', + MODchar = 'M', + GRPchar = 'G', + DESCRchar = 'D', + TYPEchar = 'Y', + TIMEchar = 'T' + } ProfilingTag; +</pre> +instead of +<pre> + # define CCchar 'C' + # define MODchar 'M' + # define GRPchar 'G' + # define DESCRchar 'D' + # define TYPEchar 'Y' + # define TIMEchar 'T' +</pre> + +<p><li> Please keep to 80 columns: the line has to be drawn somewhere, +and by keeping it to 80 columns we can ensure that code looks OK on +everyone's screen. Long lines are hard to read, and a sign that the +code needs to be restructured anyway. + +<p><li> When commenting out large chunks of code, use <code>#ifdef 0 +... #endif</code> rather than <code>/* ... */</code> because C doesn't +have nested comments. + +<p><li>When declaring a typedef for a struct, give the struct a name +as well, so that other headers can forward-reference the struct name +and it becomes possible to have opaque pointers to the struct. Our +convention is to name the struct the same as the typedef, but add a +leading underscore. For example: + +<pre> + typedef struct _Foo { + ... + } Foo; +</pre> + +<p><li>Do not use <tt>!</tt> instead of explicit comparison against +<tt>NULL</tt> or <tt>'\0'</tt>; the latter is much clearer. + +<p><li> We don't care too much about your indentation style but, if +you're modifying a function, please try to use the same style as the +rest of the function (or file). If you're writing new code, a +tab width of 4 is preferred. + +</ul> + +<h2>CVS issues</h2> + +<ul> +<p><li> +Don't be tempted to reindent or reorganise large chunks of code - it +generates large diffs in which it's hard to see whether anything else +was changed. +<p> +If you must reindent or reorganise, don't include any functional +changes that commit and give advance warning that you're about to do +it in case anyone else is changing that file. +</ul> + + +</body> +</html> diff --git a/docs/comm/rts-libs/foreignptr.html b/docs/comm/rts-libs/foreignptr.html new file mode 100644 index 0000000000..febe9fe422 --- /dev/null +++ b/docs/comm/rts-libs/foreignptr.html @@ -0,0 +1,68 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - why we have <tt>ForeignPtr</tt></title> + </head> + + <body BGCOLOR="FFFFFF"> + + <h1>On why we have <tt>ForeignPtr</tt></h1> + + <p>Unfortunately it isn't possible to add a finalizer to a normal + <tt>Ptr a</tt>. We already have a generic finalization mechanism: + see the Weak module in package lang. But the only reliable way to + use finalizers is to attach one to an atomic heap object - that + way the compiler's optimiser can't interfere with the lifetime of + the object. + + <p>The <tt>Ptr</tt> type is really just a boxed address - it's + defined like + + <pre> +data Ptr a = Ptr Addr# +</pre> + + <p>where <tt>Addr#</tt> is an unboxed native address (just a 32- + or 64- bit word). Putting a finalizer on a <tt>Ptr</tt> is + dangerous, because the compiler's optimiser might remove the box + altogether. + + <p><tt>ForeignPtr</tt> is defined like this + + <pre> +data ForeignPtr a = ForeignPtr ForeignObj# +</pre> + + <p>where <tt>ForeignObj#</tt> is a *boxed* address, it corresponds + to a real heap object. The heap object is primitive from the + point of view of the compiler - it can't be optimised away. So it + works to attach a finalizer to the <tt>ForeignObj#</tt> (but not + to the <tt>ForeignPtr</tt>!). + + <p>There are several primitive objects to which we can attach + finalizers: <tt>MVar#</tt>, <tt>MutVar#</tt>, <tt>ByteArray#</tt>, + etc. We have special functions for some of these: eg. + <tt>MVar.addMVarFinalizer</tt>. + + <p>So a nicer interface might be something like + +<pre> +class Finalizable a where + addFinalizer :: a -> IO () -> IO () + +instance Finalizable (ForeignPtr a) where ... +instance Finalizable (MVar a) where ... +</pre> + + <p>So you might ask why we don't just get rid of <tt>Ptr</tt> and + rename <tt>ForeignPtr</tt> to <tt>Ptr</tt>. The reason for that + is just efficiency, I think. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Sep 26 09:49:37 BST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/multi-thread.html b/docs/comm/rts-libs/multi-thread.html new file mode 100644 index 0000000000..67a544be85 --- /dev/null +++ b/docs/comm/rts-libs/multi-thread.html @@ -0,0 +1,445 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> +<head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> +<title>The GHC Commentary - Supporting multi-threaded interoperation</title> +</head> +<body> +<h1>The GHC Commentary - Supporting multi-threaded interoperation</h1> +<em> +<p> +Authors: sof@galois.com, simonmar@microsoft.com<br> +Date: April 2002 +</p> +</em> +<p> +This document presents the implementation of an extension to +Concurrent Haskell that provides two enhancements: +</p> +<ul> +<li>A Concurrent Haskell thread may call an external (e.g., C) +function in a manner that's transparent to the execution/evaluation of +other Haskell threads. Section <a href="#callout">Calling out"</a> covers this. +</li> +<li> +OS threads may safely call Haskell functions concurrently. Section +<a href="#callin">"Calling in"</a> covers this. +</li> +</ul> + +<!---- *************************************** -----> +<h2 id="callout">The problem: foreign calls that block</h2> +<p> +When a Concurrent Haskell(CH) thread calls a 'foreign import'ed +function, the runtime system(RTS) has to handle this in a manner +transparent to other CH threads. That is, they shouldn't be blocked +from making progress while the CH thread executes the external +call. Presently, all threads will block. +</p> +<p> +Clearly, we have to rely on OS-level threads in order to support this +kind of concurrency. The implementation described here defines the +(abstract) OS threads interface that the RTS assumes. The implementation +currently provides two instances of this interface, one for POSIX +threads (pthreads) and one for the Win32 threads. +</p> + +<!---- *************************************** -----> +<h3>Multi-threading the RTS</h3> + +<p> +A simple and efficient way to implement non-blocking foreign calls is like this: +<ul> +<li> Invariant: only one OS thread is allowed to +execute code inside of the GHC runtime system. [There are alternate +designs, but I won't go into details on their pros and cons here.] +We'll call the OS thread that is currently running Haskell threads +the <em>Current Haskell Worker Thread</em>. +<p> +The Current Haskell Worker Thread repeatedly grabs a Haskell thread, executes it until its +time-slice expires or it blocks on an MVar, then grabs another, and executes +that, and so on. +</p> +<li> +<p> +When the Current Haskell Worker comes to execute a potentially blocking 'foreign +import', it leaves the RTS and ceases being the Current Haskell Worker, but before doing so it makes certain that +another OS worker thread is available to become the Current Haskell Worker. +Consequently, even if the external call blocks, the new Current Haskell Worker +continues execution of the other Concurrent Haskell threads. +When the external call eventually completes, the Concurrent Haskell +thread that made the call is passed the result and made runnable +again. +</p> +<p> +<li> +A pool of OS threads are constantly trying to become the Current Haskell Worker. +Only one succeeds at any moment. If the pool becomes empty, the RTS creates more workers. +<p><li> +The OS worker threads are regarded as interchangeable. A given Haskell thread +may, during its lifetime, be executed entirely by one OS worker thread, or by more than one. +There's just no way to tell. + +<p><li>If a foreign program wants to call a Haskell function, there is always a thread switch involved. +The foreign program uses thread-safe mechanisms to create a Haskell thread and make it runnable; and +the current Haskell Worker Thread exectutes it. See Section <a href="#callin">Calling in</a>. +</ul> +<p> +The rest of this section describes the mechanics of implementing all +this. There's two parts to it, one that describes how a native (OS) thread +leaves the RTS to service the external call, the other how the same +thread handles returning the result of the external call back to the +Haskell thread. +</p> + +<!---- *************************************** -----> +<h3>Making the external call</h3> + +<p> +Presently, GHC handles 'safe' C calls by effectively emitting the +following code sequence: +</p> + +<pre> + ...save thread state... + t = suspendThread(); + r = foo(arg1,...,argn); + resumeThread(t); + ...restore thread state... + return r; +</pre> + +<p> +After having squirreled away the state of a Haskell thread, +<tt>Schedule.c:suspendThread()</tt> is called which puts the current +thread on a list [<tt>Schedule.c:suspended_ccalling_threads</tt>] +containing threads that are currently blocked waiting for external calls +to complete (this is done for the purposes of finding roots when +garbage collecting). +</p> + +<p> +In addition to putting the Haskell thread on +<tt>suspended_ccalling_threads</tt>, <tt>suspendThread()</tt> now also +does the following: +</p> +<ul> +<li>Instructs the <em>Task Manager</em> to make sure that there's a +another native thread waiting in the wings to take over the execution +of Haskell threads. This might entail creating a new +<em>worker thread</em> or re-using one that's currently waiting for +more work to do. The <a href="#taskman">Task Manager</a> section +presents the functionality provided by this subsystem. +</li> + +<li>Releases its capability to execute within the RTS. By doing +so, another worker thread will become unblocked and start executing +code within the RTS. See the <a href="#capability">Capability</a> +section for details. +</li> + +<li><tt>suspendThread()</tt> returns a token which is used to +identify the Haskell thread that was added to +<tt>suspended_ccalling_threads</tt>. This is done so that once the +external call has completed, we know what Haskell thread to pull off +the <tt>suspended_ccalling_threads</tt> list. +</li> +</ul> + +<p> +Upon return from <tt>suspendThread()</tt>, the OS thread is free of +its RTS executing responsibility, and can now invoke the external +call. Meanwhile, the other worker thread that have now gained access +to the RTS will continue executing Concurrent Haskell code. Concurrent +'stuff' is happening! +</p> + +<!---- *************************************** -----> +<h3>Returning the external result</h3> + +<p> +When the native thread eventually returns from the external call, +the result needs to be communicated back to the Haskell thread that +issued the external call. The following steps takes care of this: +</p> + +<ul> +<li>The returning OS thread calls <tt>Schedule.c:resumeThread()</tt>, +passing along the token referring to the Haskell thread that made the +call we're returning from. +</li> + +<li> +The OS thread then tries to grab hold of a <em>returning worker +capability</em>, via <tt>Capability.c:grabReturnCapability()</tt>. +Until granted, the thread blocks waiting for RTS permissions. Clearly we +don't want the thread to be blocked longer than it has to, so whenever +a thread that is executing within the RTS enters the Scheduler (which +is quite often, e.g., when a Haskell thread context switch is made), +it checks to see whether it can give up its RTS capability to a +returning worker, which is done by calling +<tt>Capability.c:yieldToReturningWorker()</tt>. +</li> + +<li> +If a returning worker is waiting (the code in <tt>Capability.c</tt> +keeps a counter of the number of returning workers that are currently +blocked waiting), it is woken up and the given the RTS execution +priviledges/capabilities of the worker thread that gave up its. +</li> + +<li> +The thread that gave up its capability then tries to re-acquire +the capability to execute RTS code; this is done by calling +<tt>Capability.c:waitForWorkCapability()</tt>. +</li> + +<li> +The returning worker that was woken up will continue execution in +<tt>resumeThread()</tt>, removing its associated Haskell thread +from the <tt>suspended_ccalling_threads</tt> list and start evaluating +that thread, passing it the result of the external call. +</li> +</ul> + +<!---- *************************************** -----> +<h3 id="rts-exec">RTS execution</h3> + +<p> +If a worker thread inside the RTS runs out of runnable Haskell +threads, it goes to sleep waiting for the external calls to complete. +It does this by calling <tt>waitForWorkCapability</tt> +</p> + +<p> +The availability of new runnable Haskell threads is signalled when: +</p> + +<ul> +<li>When an external call is set up in <tt>suspendThread()</tt>.</li> +<li>When a new Haskell thread is created (e.g., whenever +<tt>Concurrent.forkIO</tt> is called from within Haskell); this is +signalled in <tt>Schedule.c:scheduleThread_()</tt>. +</li> +<li>Whenever a Haskell thread is removed from a 'blocking queue' +attached to an MVar (only?). +</li> +</ul> + +<!---- *************************************** -----> +<h2 id="callin">Calling in</h2> + +Providing robust support for having multiple OS threads calling into +Haskell is not as involved as its dual. + +<ul> +<li>The OS thread issues the call to a Haskell function by going via +the <em>Rts API</em> (as specificed in <tt>RtsAPI.h</tt>). +<li>Making the function application requires the construction of a +closure on the heap. This is done in a thread-safe manner by having +the OS thread lock a designated block of memory (the 'Rts API' block, +which is part of the GC's root set) for the short period of time it +takes to construct the application. +<li>The OS thread then creates a new Haskell thread to execute the +function application, which (eventually) boils down to calling +<tt>Schedule.c:createThread()</tt> +<li> +Evaluation is kicked off by calling <tt>Schedule.c:scheduleExtThread()</tt>, +which asks the Task Manager to possibly create a new worker (OS) +thread to execute the Haskell thread. +<li> +After the OS thread has done this, it blocks waiting for the +Haskell thread to complete the evaluation of the Haskell function. +<p> +The reason why a separate worker thread is made to evaluate the Haskell +function and not the OS thread that made the call-in via the +Rts API, is that we want that OS thread to return as soon as possible. +We wouldn't be able to guarantee that if the OS thread entered the +RTS to (initially) just execute its function application, as the +Scheduler may side-track it and also ask it to evaluate other Haskell threads. +</li> +</ul> + +<p> +<strong>Note:</strong> As of 20020413, the implementation of the RTS API +only serializes access to the allocator between multiple OS threads wanting +to call into Haskell (via the RTS API.) It does not coordinate this access +to the allocator with that of the OS worker thread that's currently executing +within the RTS. This weakness/bug is scheduled to be tackled as part of an +overhaul/reworking of the RTS API itself. + + +<!---- *************************************** -----> +<h2>Subsystems introduced/modified</h2> + +<p> +These threads extensions affect the Scheduler portions of the runtime +system. To make it more manageable to work with, the changes +introduced a couple of new RTS 'sub-systems'. This section presents +the functionality and API of these sub-systems. +</p> + +<!---- *************************************** -----> +<h3 id="#capability">Capabilities</h3> + +<p> +A Capability represent the token required to execute STG code, +and all the state an OS thread/task needs to run Haskell code: +its STG registers, a pointer to its TSO, a nursery etc. During +STG execution, a pointer to the capabilitity is kept in a +register (BaseReg). +</p> +<p> +Only in an SMP build will there be multiple capabilities, for +the threaded RTS and other non-threaded builds, there is only +one global capability, namely <tt>MainCapability</tt>. + +<p> +The Capability API is as follows: +<pre> +/* Capability.h */ +extern void initCapabilities(void); + +extern void grabReturnCapability(Mutex* pMutex, Capability** pCap); +extern void waitForWorkCapability(Mutex* pMutex, Capability** pCap, rtsBool runnable); +extern void releaseCapability(Capability* cap); + +extern void yieldToReturningWorker(Mutex* pMutex, Capability* cap); + +extern void grabCapability(Capability** cap); +</pre> + +<ul> +<li><tt>initCapabilities()</tt> initialises the subsystem. + +<li><tt>grabReturnCapability()</tt> is called by worker threads +returning from an external call. It blocks them waiting to gain +permissions to do so. + +<li><tt>waitForWorkCapability()</tt> is called by worker threads +already inside the RTS, but without any work to do. It blocks them +waiting for there to new work to become available. + +<li><tt>releaseCapability()</tt> hands back a capability. If a +'returning worker' is waiting, it is signalled that a capability +has become available. If not, <tt>releaseCapability()</tt> tries +to signal worker threads that are blocked waiting inside +<tt>waitForWorkCapability()</tt> that new work might now be +available. + +<li><tt>yieldToReturningWorker()</tt> is called by the worker thread +that's currently inside the Scheduler. It checks whether there are other +worker threads waiting to return from making an external call. If so, +they're given preference and a capability is transferred between worker +threads. One of the waiting 'returning worker' threads is signalled and made +runnable, with the other, yielding, worker blocking to re-acquire +a capability. +</ul> + +<p> +The condition variables used to implement the synchronisation between +worker consumers and providers are local to the Capability +implementation. See source for details and comments. +</p> + +<!---- *************************************** -----> +<h3 id="taskman">The Task Manager</h3> + +<p> +The Task Manager API is responsible for managing the creation of +OS worker RTS threads. When a Haskell thread wants to make an +external call, the Task Manager is asked to possibly create a +new worker thread to take over the RTS-executing capability of +the worker thread that's exiting the RTS to execute the external call. + +<p> +The Capability subsystem keeps track of idle worker threads, so +making an informed decision about whether or not to create a new OS +worker thread is easy work for the task manager. The Task manager +provides the following API: +</p> + +<pre> +/* Task.h */ +extern void startTaskManager ( nat maxTasks, void (*taskStart)(void) ); +extern void stopTaskManager ( void ); + +extern void startTask ( void (*taskStart)(void) ); +</pre> + +<ul> +<li><tt>startTaskManager()</tt> and <tt>stopTaskManager()</tt> starts +up and shuts down the subsystem. When starting up, you have the option +to limit the overall number of worker threads that can be +created. An unbounded (modulo OS thread constraints) number of threads +is created if you pass '0'. +<li><tt>startTask()</tt> is called when a worker thread calls +<tt>suspendThread()</tt> to service an external call, asking another +worker thread to take over its RTS-executing capability. It is also +called when an external OS thread invokes a Haskell function via the +<em>Rts API</em>. +</ul> + +<!---- *************************************** -----> +<h3>Native threads API</h3> + +To hide OS details, the following API is used by the task manager and +the scheduler to interact with an OS' threads API: + +<pre> +/* OSThreads.h */ +typedef <em>..OS specific..</em> Mutex; +extern void initMutex ( Mutex* pMut ); +extern void grabMutex ( Mutex* pMut ); +extern void releaseMutex ( Mutex* pMut ); + +typedef <em>..OS specific..</em> Condition; +extern void initCondition ( Condition* pCond ); +extern void closeCondition ( Condition* pCond ); +extern rtsBool broadcastCondition ( Condition* pCond ); +extern rtsBool signalCondition ( Condition* pCond ); +extern rtsBool waitCondition ( Condition* pCond, + Mutex* pMut ); + +extern OSThreadId osThreadId ( void ); +extern void shutdownThread ( void ); +extern void yieldThread ( void ); +extern int createOSThread ( OSThreadId* tid, + void (*startProc)(void) ); +</pre> + + + +<!---- *************************************** -----> +<h2>User-level interface</h2> + +To signal that you want an external call to be serviced by a separate +OS thread, you have to add the attribute <tt>threadsafe</tt> to +a foreign import declaration, i.e., + +<pre> +foreign import "bigComp" threadsafe largeComputation :: Int -> IO () +</pre> + +<p> +The distinction between 'safe' and thread-safe C calls is made +so that we may call external functions that aren't re-entrant but may +cause a GC to occur. +<p> +The <tt>threadsafe</tt> attribute subsumes <tt>safe</tt>. +</p> + +<!---- *************************************** -----> +<h2>Building the GHC RTS</h2> + +The multi-threaded extension isn't currently enabled by default. To +have it built, you need to run the <tt>fptools</tt> configure script +with the extra option <tt>--enable-threaded-rts</tt> turned on, and +then proceed to build the compiler as per normal. + +<hr> +<small> +<!-- hhmts start --> Last modified: Wed Apr 10 14:21:57 Pacific Daylight Time 2002 <!-- hhmts end --> +</small> +</body> </html> + diff --git a/docs/comm/rts-libs/non-blocking.html b/docs/comm/rts-libs/non-blocking.html new file mode 100644 index 0000000000..627bde8d88 --- /dev/null +++ b/docs/comm/rts-libs/non-blocking.html @@ -0,0 +1,133 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Non-blocking I/O on Win32</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Non-blocking I/O on Win32</h1> + <p> + +This note discusses the implementation of non-blocking I/O on +Win32 platforms. It is not implemented yet (Apr 2002), but it seems worth +capturing the ideas. Thanks to Sigbjorn for writing them. + +<h2> Background</h2> + +GHC has provided non-blocking I/O support for Concurrent Haskell +threads on platforms that provide 'UNIX-style' non-blocking I/O for +quite a while. That is, platforms that let you alter the property of a +file descriptor to instead of having a thread block performing an I/O +operation that cannot be immediately satisfied, the operation returns +back a special error code (EWOULDBLOCK.) When that happens, the CH +thread that made the blocking I/O request is put into a blocked-on-IO +state (see Foreign.C.Error.throwErrnoIfRetryMayBlock). The RTS will +in a timely fashion check to see whether I/O is again possible +(via a call to select()), and if it is, unblock the thread & have it +re-try the I/O operation. The result is that other Concurrent Haskell +threads won't be affected, but can continue operating while a thread +is blocked on I/O. +<p> +Non-blocking I/O hasn't been supported by GHC on Win32 platforms, for +the simple reason that it doesn't provide the OS facilities described +above. + +<h2>Win32 non-blocking I/O, attempt 1</h2> + +Win32 does provide something select()-like, namely the +WaitForMultipleObjects() API. It takes an array of kernel object +handles plus a timeout interval, and waits for either one (or all) of +them to become 'signalled'. A handle representing an open file (for +reading) becomes signalled once there is input available. +<p> +So, it is possible to observe that I/O is possible using this +function, but not whether there's "enough" to satisfy the I/O request. +So, if we were to mimic select() usage with WaitForMultipleObjects(), +we'd correctly avoid blocking initially, but a thread may very well +block waiting for their I/O requests to be satisified once the file +handle has become signalled. [There is a fix for this -- only read +and write one byte at a the time -- but I'm not advocating that.] + + +<h2>Win32 non-blocking I/O, attempt 2</h2> + +Asynchronous I/O on Win32 is supported via 'overlapped I/O'; that is, +asynchronous read and write requests can be made via the ReadFile() / +WriteFile () APIs, specifying position and length of the operation. +If the I/O requests cannot be handled right away, the APIs won't +block, but return immediately (and report ERROR_IO_PENDING as their +status code.) +<p> +The completion of the request can be reported in a number of ways: +<ul> + <li> synchronously, by blocking inside Read/WriteFile(). (this is the + non-overlapped case, really.) +<p> + + <li> as part of the overlapped I/O request, pass a HANDLE to an event + object. The I/O system will signal this event once the request + completed, which a waiting thread will then be able to see. +<p> + + <li> by supplying a pointer to a completion routine, which will be + called as an Asynchronous Procedure Call (APC) whenever a thread + calls a select bunch of 'alertable' APIs. +<p> + + <li> by associating the file handle with an I/O completion port. Once + the request completes, the thread servicing the I/O completion + port will be notified. +</ul> +The use of I/O completion port looks the most interesting to GHC, +as it provides a central point where all I/O requests are reported. +<p> +Note: asynchronous I/O is only fully supported by OSes based on +the NT codebase, i.e., Win9x don't permit async I/O on files and +pipes. However, Win9x does support async socket operations, and +I'm currently guessing here, console I/O. In my view, it would +be acceptable to provide non-blocking I/O support for NT-based +OSes only. +<p> +Here's the design I currently have in mind: +<ul> +<li> Upon startup, an RTS helper thread whose only purpose is to service + an I/O completion port, is created. +<p> +<li> All files are opened in 'overlapping' mode, and associated + with an I/O completion port. +<p> +<li> Overlapped I/O requests are used to implement read() and write(). +<p> +<li> If the request cannot be satisified without blocking, the Haskell + thread is put on the blocked-on-I/O thread list & a re-schedule + is made. +<p> +<li> When the completion of a request is signalled via the I/O completion + port, the RTS helper thread will move the associated Haskell thread + from the blocked list onto the runnable list. (Clearly, care + is required here to have another OS thread mutate internal Scheduler + data structures.) + +<p> +<li> In the event all Concurrent Haskell threads are blocked waiting on + I/O, the main RTS thread blocks waiting on an event synchronisation + object, which the helper thread will signal whenever it makes + a Haskell thread runnable. + +</ul> + +I might do the communication between the RTS helper thread and the +main RTS thread differently though: rather than have the RTS helper +thread manipluate thread queues itself, thus requiring careful +locking, just have it change a bit on the relevant TSO, which the main +RTS thread can check at regular intervals (in some analog of +awaitEvent(), for example). + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 8 19:30:18 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/prelfound.html b/docs/comm/rts-libs/prelfound.html new file mode 100644 index 0000000000..25407eed43 --- /dev/null +++ b/docs/comm/rts-libs/prelfound.html @@ -0,0 +1,57 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Prelude Foundations</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Prelude Foundations</h1> + <p> + The standard Haskell Prelude as well as GHC's Prelude extensions are + constructed from GHC's <a href="primitives.html">primitives</a> in a + couple of layers. + + <h4><code>PrelBase.lhs</code></h4> + <p> + Some most elementary Prelude definitions are collected in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelBase.lhs"><code>PrelBase.lhs</code></a>. + In particular, it defines the boxed versions of Haskell primitive types + - for example, <code>Int</code> is defined as + <blockquote><pre> +data Int = I# Int#</pre> + </blockquote> + <p> + Saying that a boxed integer <code>Int</code> is formed by applying the + data constructor <code>I#</code> to an <em>unboxed</em> integer of type + <code>Int#</code>. Unboxed types are hardcoded in the compiler and + exported together with the <a href="primitives.html">primitive + operations</a> understood by GHC. + <p> + <code>PrelBase.lhs</code> similarly defines basic types, such as, + boolean values + <blockquote><pre> +data Bool = False | True deriving (Eq, Ord)</pre> + </blockquote> + <p> + the unit type + <blockquote><pre> +data () = ()</pre> + </blockquote> + <p> + and lists + <blockquote><pre> +data [] a = [] | a : [a]</pre> + </blockquote> + <p> + It also contains instance delarations for these types. In addition, + <code>PrelBase.lhs</code> contains some <a href="prelude.html">tricky + machinery</a> for efficient list handling. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 8 19:30:18 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/prelude.html b/docs/comm/rts-libs/prelude.html new file mode 100644 index 0000000000..4ad6c20338 --- /dev/null +++ b/docs/comm/rts-libs/prelude.html @@ -0,0 +1,121 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Cunning Prelude Code</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Cunning Prelude Code</h1> + <p> + GHC's uses a many optimsations and GHC specific techniques (unboxed + values, RULES pragmas, and so on) to make the heavily used Prelude code + as fast as possible. + + <hr> + <h4>Par, seq, and lazy</h4> + + In GHC.Conc you will dinf +<blockquote><pre> + pseq a b = a `seq` lazy b +</pre></blockquote> + What's this "lazy" thing. Well, <tt>pseq</tt> is a <tt>seq</tt> for a parallel setting. + We really mean "evaluate a, then b". But if the strictness analyser sees that pseq is strict + in b, then b might be evaluated <em>before</em> a, which is all wrong. +<p> +Solution: wrap the 'b' in a call to <tt>GHC.Base.lazy</tt>. This function is just the identity function, +except that it's put into the built-in environment in MkId.lhs. That is, the MkId.lhs defn over-rides the +inlining and strictness information that comes in from GHC.Base.hi. And that makes <tt>lazy</tt> look +lazy, and have no inlining. So the strictness analyser gets no traction. +<p> +In the worker/wrapper phase, after strictness analysis, <tt>lazy</tt> is "manually" inlined (see WorkWrap.lhs), +so we get all the efficiency back. +<p> +This supersedes an earlier scheme involving an even grosser hack in which par# and seq# returned an +Int#. Now there is no seq# operator at all. + + + <hr> + <h4>fold/build</h4> + <p> + There is a lot of magic in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelBase.lhs"><code>PrelBase.lhs</code></a> - + among other things, the <a + href="http://haskell.cs.yale.edu/ghc/docs/latest/set/rewrite-rules.html">RULES + pragmas</a> implementing the <a + href="http://research.microsoft.com/Users/simonpj/Papers/deforestation-short-cut.ps.Z">fold/build</a> + optimisation. The code for <code>map</code> is + a good example for how it all works. In the prelude code for version + 5.03 it reads as follows: + <blockquote><pre> +map :: (a -> b) -> [a] -> [b] +map _ [] = [] +map f (x:xs) = f x : map f xs + +-- Note eta expanded +mapFB :: (elt -> lst -> lst) -> (a -> elt) -> a -> lst -> lst +{-# INLINE [0] mapFB #-} +mapFB c f x ys = c (f x) ys + +{-# RULES +"map" [~1] forall f xs. map f xs = build (\c n -> foldr (mapFB c f) n xs) +"mapList" [1] forall f. foldr (mapFB (:) f) [] = map f +"mapFB" forall c f g. mapFB (mapFB c f) g = mapFB c (f.g) + #-}</pre> + </blockquote> + <p> + Up to (but not including) phase 1, we use the <code>"map"</code> rule to + rewrite all saturated applications of <code>map</code> with its + build/fold form, hoping for fusion to happen. In phase 1 and 0, we + switch off that rule, inline build, and switch on the + <code>"mapList"</code> rule, which rewrites the foldr/mapFB thing back + into plain map. + <p> + It's important that these two rules aren't both active at once + (along with build's unfolding) else we'd get an infinite loop + in the rules. Hence the activation control using explicit phase numbers. + <p> + The "mapFB" rule optimises compositions of map. + <p> + The mechanism as described above is new in 5.03 since January 2002, + where the <code>[~</code><i>N</i><code>]</code> syntax for phase number + annotations at rules was introduced. Before that the whole arrangement + was more complicated, as the corresponding prelude code for version + 4.08.1 shows: + <blockquote><pre> +map :: (a -> b) -> [a] -> [b] +map = mapList + +-- Note eta expanded +mapFB :: (elt -> lst -> lst) -> (a -> elt) -> a -> lst -> lst +mapFB c f x ys = c (f x) ys + +mapList :: (a -> b) -> [a] -> [b] +mapList _ [] = [] +mapList f (x:xs) = f x : mapList f xs + +{-# RULES +"map" forall f xs. map f xs = build (\c n -> foldr (mapFB c f) n xs) +"mapFB" forall c f g. mapFB (mapFB c f) g = mapFB c (f.g) +"mapList" forall f. foldr (mapFB (:) f) [] = mapList f + #-}</pre> + </blockquote> + <p> + This code is structured as it is, because the "map" rule first + <em>breaks</em> the map <em>open,</em> which exposes it to the various + foldr/build rules, and if no foldr/build rule matches, the "mapList" + rule <em>closes</em> it again in a later phase of optimisation - after + build was inlined. As a consequence, the whole thing depends a bit on + the timing of the various optimsations (the map might be closed again + before any of the foldr/build rules fires). To make the timing + deterministic, <code>build</code> gets a <code>{-# INLINE 2 build + #-}</code> pragma, which delays <code>build</code>'s inlining, and thus, + the closing of the map. [NB: Phase numbering was forward at that time.] + + <p><small> +<!-- hhmts start --> +Last modified: Mon Feb 11 20:00:49 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/primitives.html b/docs/comm/rts-libs/primitives.html new file mode 100644 index 0000000000..28abc79426 --- /dev/null +++ b/docs/comm/rts-libs/primitives.html @@ -0,0 +1,70 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Primitives</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Primitives</h1> + <p> + Most user-level Haskell types and functions provided by GHC (in + particular those from the Prelude and GHC's Prelude extensions) are + internally constructed from even more elementary types and functions. + Most notably, GHC understands a notion of <em>unboxed types,</em> which + are the Haskell representation of primitive bit-level integer, float, + etc. types (as opposed to their boxed, heap allocated counterparts) - + cf. <a + href="http://research.microsoft.com/Users/simonpj/Papers/unboxed-values.ps.Z">"Unboxed + Values as First Class Citizens."</a> + + <h4>The Ultimate Source of Primitives</h4> + <p> + The hardwired types of GHC are brought into scope by the module + <code>PrelGHC</code>. This modules only exists in the form of a + handwritten interface file <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/lib/std/PrelGHC.hi-boot"><code>PrelGHC.hi-boot</code>,</a> + which lists the type and function names, as well as instance + declarations. The actually types of these names as well as their + implementation is hardwired into GHC. Note that the names in this file + are z-encoded, and in particular, identifiers ending on <code>zh</code> + denote user-level identifiers ending in a hash mark (<code>#</code>), + which is used to flag unboxed values or functions operating on unboxed + values. For example, we have <code>Char#</code>, <code>ord#</code>, and + so on. + + <h4>The New Primitive Definition Scheme</h4> + <p> + As of (about) the development version 4.11, the types and various + properties of primitive operations are defined in the file <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/primops.txt.pp"><code>primops.txt.pp</code></a>. + (Personally, I don't think that the <code>.txt</code> suffix is really + appropriate, as the file is used for automatic code generation; the + recent addition of <code>.pp</code> means that the file is now mangled + by cpp.) + <p> + The utility <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/utils/genprimopcode/"><code>genprimopcode</code></a> + generates a series of Haskell files from <code>primops.txt</code>, which + encode the types and various properties of the primitive operations as + compiler internal data structures. These Haskell files are not complete + modules, but program fragments, which are included into compiler modules + during the GHC build process. The generated include files can be found + in the directory <code>fptools/ghc/compiler/</code> and carry names + matching the pattern <code>primop-*.hs-incl</code>. They are generate + during the execution of the <code>boot</code> target in the + <code>fptools/ghc/</code> directory. This scheme significantly + simplifies the maintenance of primitive operations. + <p> + As of development version 5.02, the <code>primops.txt</code> file also allows the + recording of documentation about intended semantics of the primitives. This can + be extracted into a latex document (or rather, into latex document fragments) + via an appropriate switch to <code>genprimopcode</code>. In particular, see <code>primops.txt</code> + for full details of how GHC is configured to cope with different machine word sizes. + <p><small> +<!-- hhmts start --> +Last modified: Mon Nov 26 18:03:16 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/stgc.html b/docs/comm/rts-libs/stgc.html new file mode 100644 index 0000000000..196ec9150d --- /dev/null +++ b/docs/comm/rts-libs/stgc.html @@ -0,0 +1,45 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Spineless Tagless C</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Spineless Tagless C</h1> + <p> + The C code generated by GHC doesn't use higher-level features of C to be + able to control as precisely as possible what code is generated. + Moreover, it uses special features of gcc (such as, first class labels) + to produce more efficient code. + <p> + STG C makes ample use of C's macro language to define idioms, which also + reduces the size of the generated C code (thus, reducing I/O times). + These macros are defined in the C headers located in GHC's <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/includes/"><code>includes</code></a> + directory. + + <h4><code>TailCalls.h</code></h4> + <p> + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/includes/TailCalls.h"><code>TailCalls.h</code></a> + defines how tail calls are implemented - and in particular - optimised + in GHC generated code. The default case, for an architecture for which + GHC is not optimised, is to use the mini interpreter described in the <a + href="http://research.microsoft.com/copyright/accept.asp?path=/users/simonpj/papers/spineless-tagless-gmachine.ps.gz&pub=34">STG paper.</a> + <p> + For supported architectures, various tricks are used to generate + assembler implementing proper tail calls. On i386, gcc's first class + labels are used to directly jump to a function pointer. Furthermore, + markers of the form <code>--- BEGIN ---</code> and <code>--- END + ---</code> are added to the assembly right after the function prologue + and before the epilogue. These markers are used by <a + href="../the-beast/mangler.html">the Evil Mangler.</a> + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 8 19:28:29 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/rts-libs/threaded-rts.html b/docs/comm/rts-libs/threaded-rts.html new file mode 100644 index 0000000000..499aeec767 --- /dev/null +++ b/docs/comm/rts-libs/threaded-rts.html @@ -0,0 +1,126 @@ +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Multi-threaded runtime, and multiprocessor execution</title> + </head> + + <body> + <h1>The GHC Commentary - The Multi-threaded runtime, and multiprocessor execution</h1> + + <p>This section of the commentary explains the structure of the runtime system + when used in threaded or SMP mode.</p> + + <p>The <em>threaded</em> version of the runtime supports + bound threads and non-blocking foreign calls, and an overview of its + design can be found in the paper <a + href="http://www.haskell.org/~simonmar/papers/conc-ffi.pdf">Extending + the Haskell Foreign Function Interface with Concurrency</a>. To + compile the runtime with threaded support, add the line + +<pre>GhcRTSWays += thr</pre> + + to <tt>mk/build.mk</tt>. When building C code in the runtime for the threaded way, + the symbol <tt>THREADED_RTS</tt> is defined (this is arranged by the + build system when building for way <tt>thr</tt>, see + <tt>mk/config.mk</tt>). To build a Haskell program + with the threaded runtime, pass the flag <tt>-threaded</tt> to GHC (this + can be used in conjunction with <tt>-prof</tt>, and possibly + <tt>-debug</tt> and others depending on which versions of the RTS have + been built.</p> + + <p>The <em>SMP</em> version runtime supports the same facilities as the + threaded version, and in addition supports execution of Haskell code by + multiple simultaneous OS threads. For SMP support, both the runtime and + the libraries must be built a special way: add the lines + + <pre> +GhcRTSWays += thr +GhcLibWays += s</pre> + + to <tt>mk/build.mk</tt>. To build Haskell code for + SMP execution, use the flag <tt>-smp</tt> to GHC (this can be used in + conjunction with <tt>-debug</tt>, but no other way-flags at this time). + When building C code in the runtime for SMP + support, the symbol <tt>SMP</tt> is defined (this is arranged by the + compiler when the <tt>-smp</tt> flag is given, see + <tt>ghc/compiler/main/StaticFlags.hs</tt>).</p> + + <p>When building the runtime in either the threaded or SMP ways, the symbol + <tt>RTS_SUPPORTS_THREADS</tt> will be defined (see <tt>Rts.h</tt>).</p> + + <h2>Overall design</h2> + + <p>The system is based around the notion of a <tt>Capability</tt>. A + <tt>Capability</tt> is an object that represents both the permission to + execute some Haskell code, and the state required to do so. In order + to execute some Haskell code, a thread must therefore hold a + <tt>Capability</tt>. The available pool of capabilities is managed by + the <tt>Capability</tt> API, described below.</p> + + <p>In the threaded runtime, there is only a single <tt>Capabililty</tt> in the + system, indicating that only a single thread can be executing Haskell + code at any one time. In the SMP runtime, there can be an arbitrary + number of capabilities selectable at runtime with the <tt>+RTS -N<em>n</em></tt> + flag; in practice the number is best chosen to be the same as the number of + processors on the host machine.</p> + + <p>There are a number of OS threads running code in the runtime. We call + these <em>tasks</em> to avoid confusion with Haskell <em>threads</em>. + Tasks are managed by the <tt>Task</tt> subsystem, which is mainly + concerned with keeping track of statistics such as how much time each + task spends executing Haskell code, and also keeping track of how many + tasks are around when we want to shut down the runtime.</p> + + <p>Some tasks are created by the runtime itself, and some may be here + as a result of a call to Haskell from foreign code (we + call this an in-call). The + runtime can support any number of concurrent foreign in-calls, but the + number of these calls that will actually run Haskell code in parallel is + determined by the number of available capabilities. Each in-call creates + a <em>bound thread</em>, as described in the FFI/Concurrency paper (cited + above).</p> + + <p>In the future we may want to bind a <tt>Capability</tt> to a particular + processor, so that we can support a notion of affinity - avoiding + accidental migration of work from one CPU to another, so that we can make + best use of a CPU's local cache. For now, the design ignores this + issue.</p> + + <h2>The <tt>OSThreads</tt> interface</h2> + + <p>This interface is merely an abstraction layer over the OS-specific APIs + for managing threads. It has two main implementations: Win32 and + POSIX.</p> + + <p>This is the entirety of the interface:</p> + +<pre> +/* Various abstract types */ +typedef Mutex; +typedef Condition; +typedef OSThreadId; + +extern OSThreadId osThreadId ( void ); +extern void shutdownThread ( void ); +extern void yieldThread ( void ); +extern int createOSThread ( OSThreadId* tid, + void (*startProc)(void) ); + +extern void initCondition ( Condition* pCond ); +extern void closeCondition ( Condition* pCond ); +extern rtsBool broadcastCondition ( Condition* pCond ); +extern rtsBool signalCondition ( Condition* pCond ); +extern rtsBool waitCondition ( Condition* pCond, + Mutex* pMut ); + +extern void initMutex ( Mutex* pMut ); + </pre> + + <h2>The Task interface</h2> + + <h2>The Capability interface</h2> + + <h2>Multiprocessor Haskell Execution</h2> + + </body> +</html> diff --git a/docs/comm/the-beast/alien.html b/docs/comm/the-beast/alien.html new file mode 100644 index 0000000000..3d4776ebc9 --- /dev/null +++ b/docs/comm/the-beast/alien.html @@ -0,0 +1,56 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Alien Functions</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Alien Functions</h1> + <p> + GHC implements experimental (by now it is actually quite well tested) + support for access to foreign functions and generally the interaction + between Haskell code and code written in other languages. Code + generation in this context can get quite tricky. This section attempts + to cast some light on this aspect of the compiler. + + <h4>FFI Stub Files</h4> + <p> + For each Haskell module that contains a <code>foreign export + dynamic</code> declaration, GHC generates a <code>_stub.c</code> file + that needs to be linked with any program that imports the Haskell + module. When asked about it <a + href="mailto:simonmar@microsoft.com">Simon Marlow</a> justified the + existence of these files as follows: + <blockquote> + The stub files contain the helper function which invokes the Haskell + code when called from C. + <p> + Each time the foreign export dynamic is invoked to create a new + callback function, a small piece of code has to be dynamically + generated (by code in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/rts/Adjustor.c"><code>Adjustor.c</code></a>). It is the address of this dynamically generated bit of + code that is returned as the <code>Addr</code> (or <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/hslibs/lang/Ptr.lhs"><code>Ptr</code></a>). + When called from C, the dynamically generated code must somehow invoke + the Haskell function which was originally passed to the + f.e.d. function -- it does this by invoking the helper function, + passing it a <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/hslibs/lang/StablePtr.lhs"><code>StablePtr</code></a> + to the Haskell function. It's split this way for two reasons: the + same helper function can be used each time the f.e.d. function is + called, and to keep the amount of dynamically generated code to a + minimum. + </blockquote> + <p> + The stub code is generated by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsForeign.lhs"><code>DSForeign</code></a><code>.fexportEntry</code>. + + + <p><small> +<!-- hhmts start --> +Last modified: Fri Aug 10 11:47:41 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/basicTypes.html b/docs/comm/the-beast/basicTypes.html new file mode 100644 index 0000000000..ca56d6b6a8 --- /dev/null +++ b/docs/comm/the-beast/basicTypes.html @@ -0,0 +1,132 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Basics</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Basics</h1> + <p> + The directory <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/"><code>fptools/ghc/compiler/basicTypes/</code></a> + contains modules that define some of the essential types definition for + the compiler - such as, identifiers, variables, modules, and unique + names. Some of those are discussed in the following. See elsewhere for more + detailed information on: + <ul> + <li> <a href="vars.html"><code>Var</code>s, <code>Id</code>s, and <code>TyVar</code>s</a> + <li> <a href="renamer.html"><code>OccName</code>s, <code>RdrName</code>s, and <code>Names</code>s</a> + </ul> + + <h2>Elementary Types</h2> + + <h4><code>Id</code>s</h4> + <p> + An <code>Id</code> (defined in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/Id.lhs"><code>Id.lhs</code></a> + essentially records information about value and data constructor + identifiers -- to be precise, in the case of data constructors, two + <code>Id</code>s are used to represent the worker and wrapper functions + for the data constructor, respectively. The information maintained in + the <code>Id</code> abstraction includes among other items strictness, + occurrence, specialisation, and unfolding information. + <p> + Due to the way <code>Id</code>s are used for data constructors, + all <code>Id</code>s are represented as variables, which contain a + <code>varInfo</code> field of abstract type <code><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/IdInfo.lhs">IdInfo</a>.IdInfo</code>. + This is where the information about <code>Id</code>s is really stored. + The following is a (currently, partial) list of the various items in an + <code>IdInfo</code>: + <p> + <dl> + <dt><a name="occInfo">Occurence information</a> + <dd>The <code>OccInfo</code> data type is defined in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/BasicTypes.lhs"><code>BasicTypes.lhs</code></a>. + Apart from the trivial <code>NoOccInfo</code>, it distinguishes + between variables that do not occur at all (<code>IAmDead</code>), + occur just once (<code>OneOcc</code>), or a <a + href="simplifier.html#loopBreaker">loop breakers</a> + (<code>IAmALoopBreaker</code>). + </dl> + + <h2>Sets, Finite Maps, and Environments</h2> + <p> + Sets of variables, or more generally names, which are needed throughout + the compiler, are provided by the modules <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/VarSet.lhs"><code>VarSet.lhs</code></a> + and <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/NameSet.lhs"><code>NameSet.lhs</code></a>, + respectively. Moreover, frequently maps from variables (or names) to + other data is needed. For example, a substitution is represented by a + finite map from variable names to expressions. Jobs like this are + solved by means of variable and name environments implemented by the + modules <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/VarEnv.lhs"><code>VarEnv.lhs</code></a> + and <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/NameEnv.lhs"><code>NameEnv.lhs</code></a>. + + <h4>The Module <code>VarSet</code></h4> + <p> + The Module <code>VarSet</code> provides the types <code>VarSet</code>, + <code>IdSet</code>, and <code>TyVarSet</code>, which are synonyms in the + current implementation, as <code>Var</code>, <code>Id</code>, and + <code>TyVar</code> are synonyms. The module provides all the operations + that one would expect including the creating of sets from individual + variables and lists of variables, union and intersection operations, + element checks, deletion, filter, fold, and map functions. + <p> + The implementation is based on <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/utils/UniqSet.lhs"><code>UniqSet</code></a>s, + which in turn are simply <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/utils/UniqFM.lhs"><code>UniqFM</code></a>s + (i.e., finite maps with uniques as keys) that map each unique to the + variable that it represents. + + <h4>The Module <code>NameSet</code></h4> + <p> + The Module <code>NameSet</code> provides the same functionality as + <code>VarSet</code> only for <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/Name.lhs"><code>Name</code></a>s. + As for the difference between <code>Name</code>s and <code>Var</code>s, + a <code>Var</code> is built from a <code>Name</code> plus additional + information (mostly importantly type information). + + <h4>The Module <code>VarEnv</code></h4> + <p> + The module <code>VarEnv</code> provides the types <code>VarEnv</code>, + <code>IdEnv</code>, and <code>TyVarEnv</code>, which are again + synonyms. The provided base functionality is similar to + <code>VarSet</code> with the main difference that a type <code>VarEnv + T</code> associates a value of type <code>T</code> with each variable in + the environment, thus effectively implementing a finite map from + variables to values of type <code>T</code>. + <p> + The implementation of <code>VarEnv</code> is also by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/utils/UniqFM.lhs"><code>UniqFM</code></a>, + which entails the slightly surprising implication that it is + <em>not</em> possible to retrieve the domain of a variable environment. + In other words, there is no function corresponding to + <code>VarSet.varSetElems :: VarSet -> [Var]</code> in + <code>VarEnv</code>. This is because the <code>UniqFM</code> used to + implement <code>VarEnv</code> stores only the unique corresponding to a + variable in the environment, but not the entire variable (and there is + no mapping from uniques to variables). + <p> + In addition to plain variable environments, the module also contains + special substitution environments - the type <code>SubstEnv</code> - + that associates variables with a special purpose type + <code>SubstResult</code>. + + <h4>The Module <code>NameEnv</code></h4> + <p> + The type <code>NameEnv.NameEnv</code> is like <code>VarEnv</code> only + for <code>Name</code>s. + + <p><hr><small> +<!-- hhmts start --> +Last modified: Tue Jan 8 18:29:52 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/coding-style.html b/docs/comm/the-beast/coding-style.html new file mode 100644 index 0000000000..41347c6902 --- /dev/null +++ b/docs/comm/the-beast/coding-style.html @@ -0,0 +1,230 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Coding Style Guidelines</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Coding Style Guidelines</h1> + + <p>This is a rough description of some of the coding practices and + style that we use for Haskell code inside <tt>ghc/compiler</tt>. + + <p>The general rule is to stick to the same coding style as is + already used in the file you're editing. If you must make + stylistic changes, commit them separately from functional changes, + so that someone looking back through the change logs can easily + distinguish them. + + <h2>To literate or not to literate?</h2> + + <p>In GHC we use a mixture of literate (<tt>.lhs</tt>) and + non-literate (<tt>.hs</tt>) source. I (Simon M.) prefer to use + non-literate style, because I think the + <tt>\begin{code}..\end{code}</tt> clutter up the source too much, + and I like to use Haddock-style comments (we haven't tried + processing the whole of GHC with Haddock yet, though). + + <h2>To CPP or not to CPP?</h2> + + <p>We pass all the compiler sources through CPP. The + <tt>-cpp</tt> flag is always added by the build system. + + <p>The following CPP symbols are used throughout the compiler: + + <dl> + <dt><tt>DEBUG</tt></dt> + + <dd>Used to enables extra checks and debugging output in the + compiler. The <tt>ASSERT</tt> macro (see <tt>HsVersions.h</tt>) + provides assertions which disappear when <tt>DEBUG</tt> is not + defined. + + <p>All debugging output should be placed inside <tt>#ifdef + DEBUG</tt>; we generally use this to provide warnings about + strange cases and things that might warrant investigation. When + <tt>DEBUG</tt> is off, the compiler should normally be silent + unless something goes wrong (exception when the verbosity level + is greater than zero). + + <p>A good rule of thumb is that <tt>DEBUG</tt> shouldn't add + more than about 10-20% to the compilation time. This is the case + at the moment. If it gets too expensive, we won't use it. For + more expensive runtime checks, consider adding a flag - see for + example <tt>-dcore-lint</tt>. + </dd> + + <dt><tt>GHCI</tt></dt> + + <dd>Enables GHCi support, including the byte code generator and + interactive user interface. This isn't the default, because the + compiler needs to be bootstrapped with itself in order for GHCi + to work properly. The reason is that the byte-code compiler and + linker are quite closely tied to the runtime system, so it is + essential that GHCi is linked with the most up-to-date RTS. + Another reason is that the representation of certain datatypes + must be consistent between GHCi and its libraries, and if these + were inconsistent then disaster could follow. + </dd> + + </dl> + + <h2>Platform tests</h2> + + <p>There are three platforms of interest to GHC: + + <ul> + <li>The <b>Build</b> platform. This is the platform on which we + are building GHC.</li> + <li>The <b>Host</b> platform. This is the platform on which we + are going to run this GHC binary, and associated tools.</li> + <li>The <b>Target</b> platform. This is the platform for which + this GHC binary will generate code.</li> + </ul> + + <p>At the moment, there is very limited support for having + different values for buil, host, and target. In particular:</p> + + <ul> + <li>The build platform is currently always the same as the host + platform. The build process needs to use some of the tools in + the source tree, for example <tt>ghc-pkg</tt> and + <tt>hsc2hs</tt>.</li> + + <li>If the target platform differs from the host platform, then + this is generally for the purpose of building <tt>.hc</tt> files + from Haskell source for porting GHC to the target platform. + Full cross-compilation isn't supported (yet).</li> + </ul> + + <p>In the compiler's source code, you may make use of the + following CPP symbols:</p> + + <ul> + <li><em>xxx</em><tt>_TARGET_ARCH</tt></li> + <li><em>xxx</em><tt>_TARGET_VENDOR</tt></li> + <li><em>xxx</em><tt>_TARGET_OS</tt></li> + <li><em>xxx</em><tt>_HOST_ARCH</tt></li> + <li><em>xxx</em><tt>_HOST_VENDOR</tt></li> + <li><em>xxx</em><tt>_HOST_OS</tt></li> + </ul> + + <p>where <em>xxx</em> is the appropriate value: + eg. <tt>i386_TARGET_ARCH</tt>. + + <h2>Compiler versions</h2> + + <p>GHC must be compilable by every major version of GHC from 5.02 + onwards, and itself. It isn't necessary for it to be compilable + by every intermediate development version (that includes last + week's CVS sources). + + <p>To maintain compatibility, use <tt>HsVersions.h</tt> (see + below) where possible, and try to avoid using <tt>#ifdef</tt> in + the source itself. + + <h2>The source file</h2> + + <p>We now describe a typical source file, annotating stylistic + choices as we go. + +<pre> +{-# OPTIONS ... #-} +</pre> + + <p>An <tt>OPTIONS</tt> pragma is optional, but if present it + should go right at the top of the file. Things you might want to + put in <tt>OPTIONS</tt> include: + + <ul> + <li><tt>-#include</tt> options to bring into scope prototypes + for FFI declarations</li> + <li><tt>-fvia-C</tt> if you know that + this module won't compile with the native code generator. + </ul> + + <p>Don't bother putting <tt>-cpp</tt> or <tt>-fglasgow-exts</tt> + in the <tt>OPTIONS</tt> pragma; these are already added to the + command line by the build system. + + +<pre> +module Foo ( + T(..), + foo, -- :: T -> T + ) where +</pre> + + <p>We usually (99% of the time) include an export list. The only + exceptions are perhaps where the export list would list absolutely + everything in the module, and even then sometimes we do it anyway. + + <p>It's helpful to give type signatures inside comments in the + export list, but hard to keep them consistent, so we don't always + do that. + +<pre> +#include "HsVersions.h" +</pre> + + <p><tt>HsVersions.h</tt> is a CPP header file containing a number + of macros that help smooth out the differences between compiler + versions. It defines, for example, macros for library module + names which have moved between versions. Take a look. + +<pre> +-- friends +import SimplMonad + +-- GHC +import CoreSyn +import Id ( idName, idType ) +import BasicTypes + +-- libraries +import DATA_IOREF ( newIORef, readIORef ) + +-- std +import List ( partition ) +import Maybe ( fromJust ) +</pre> + + <p>List imports in the following order: + + <ul> + <li>Local to this subsystem (or directory) first</li> + + <li>Compiler imports, generally ordered from specific to generic + (ie. modules from <tt>utils/</tt> and <tt>basicTypes/</tt> + usually come last)</li> + + <li>Library imports</li> + + <li>Standard Haskell 98 imports last</li> + </ul> + + <p>Import library modules from the <tt>base</tt> and + <tt>haskell98</tt> packages only. Use <tt>#defines</tt> in + <tt>HsVersions.h</tt> when the modules names differ between + versions of GHC (eg. <tt>DATA_IOREF</tt> in the example above). + For code inside <tt>#ifdef GHCI</tt>, don't need to worry about GHC + versioning (because we are bootstrapped). + + <p>We usually use import specs to give an explicit list of the + entities imported from a module. The main reason for doing this is + so that you can search the file for an entity and find which module + it comes from. However, huge import lists can be a pain to + maintain, so we often omit the import specs when they start to get + long (actually I start omitting them when they don't fit on one + line --Simon M.). Tip: use GHC's <tt>-fwarn-unused-imports</tt> + flag so that you get notified when an import isn't being used any + more. + + <p>If the module can be compiled multiple ways (eg. GHCI + vs. non-GHCI), make sure the imports are properly <tt>#ifdefed</tt> + too, so as to avoid spurious unused import warnings. + + <p><em>ToDo: finish this</em> + </body> +</html> diff --git a/docs/comm/the-beast/data-types.html b/docs/comm/the-beast/data-types.html new file mode 100644 index 0000000000..fef4852d4d --- /dev/null +++ b/docs/comm/the-beast/data-types.html @@ -0,0 +1,242 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Data types and data constructors</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Data types and data constructors</h1> + <p> + +This chapter was thoroughly changed Feb 2003. + +<h2>Data types</h2> + +Consider the following data type declaration: + +<pre> + data T a = MkT !(a,a) !(T a) | Nil + + f x = case x of + MkT p q -> MkT p (q+1) + Nil -> Nil +</pre> +The user's source program mentions only the constructors <tt>MkT</tt> +and <tt>Nil</tt>. However, these constructors actually <em>do</em> something +in addition to building a data value. For a start, <tt>MkT</tt> evaluates +its arguments. Secondly, with the flag <tt>-funbox-strict-fields</tt> GHC +will flatten (or unbox) the strict fields. So we may imagine that there's the +<em>source</em> constructor <tt>MkT</tt> and the <em>representation</em> constructor +<tt>MkT</tt>, and things start to get pretty confusing. +<p> +GHC now generates three unique <tt>Name</tt>s for each data constructor: +<pre> + ---- OccName ------ + String Name space Used for + --------------------------------------------------------------------------- + The "source data con" MkT DataName The DataCon itself + The "worker data con" MkT VarName Its worker Id + aka "representation data con" + The "wrapper data con" $WMkT VarName Its wrapper Id (optional) +</pre> +Recall that each occurrence name (OccName) is a pair of a string and a +name space (see <a href="names.html">The truth about names</a>), and +two OccNames are considered the same only if both components match. +That is what distinguishes the name of the name of the DataCon from +the name of its worker Id. To keep things unambiguous, in what +follows we'll write "MkT{d}" for the source data con, and "MkT{v}" for +the worker Id. (Indeed, when you dump stuff with "-ddumpXXX", if you +also add "-dppr-debug" you'll get stuff like "Foo {- d rMv -}". The +"d" part is the name space; the "rMv" is the unique key.) +<p> +Each of these three names gets a distinct unique key in GHC's name cache. + +<h2>The life cycle of a data type</h2> + +Suppose the Haskell source looks like this: +<pre> + data T a = MkT !(a,a) !Int | Nil + + f x = case x of + Nil -> Nil + MkT p q -> MkT p (q+1) +</pre> +When the parser reads it in, it decides which name space each lexeme comes +from, thus: +<pre> + data T a = MkT{d} !(a,a) !Int | Nil{d} + + f x = case x of + Nil{d} -> Nil{d} + MkT{d} p q -> MkT{d} p (q+1) +</pre> +Notice that in the Haskell source <em>all data contructors are named via the "source data con" MkT{d}</em>, +whether in pattern matching or in expressions. +<p> +In the translated source produced by the type checker (-ddump-tc), the program looks like this: +<pre> + f x = case x of + Nil{d} -> Nil{v} + MkT{d} p q -> $WMkT p (q+1) + +</pre> +Notice that the type checker replaces the occurrence of MkT by the <em>wrapper</em>, but +the occurrence of Nil by the <em>worker</em>. Reason: Nil doesn't have a wrapper because there is +nothing to do in the wrapper (this is the vastly common case). +<p> +Though they are not printed out by "-ddump-tc", behind the scenes, there are +also the following: the data type declaration and the wrapper function for MkT. +<pre> + data T a = MkT{d} a a Int# | Nil{d} + + $WMkT :: (a,a) -> T a -> T a + $WMkT p t = case p of + (a,b) -> seq t (MkT{v} a b t) +</pre> +Here, the <em>wrapper</em> <tt>$WMkT</tt> evaluates and takes apart the argument <tt>p</tt>, +evaluates the argument <tt>t</tt>, and builds a three-field data value +with the <em>worker</em> constructor <tt>MkT{v}</tt>. (There are more notes below +about the unboxing of strict fields.) The worker $WMkT is called an <em>implicit binding</em>, +because it's introduced implicitly by the data type declaration (record selectors +are also implicit bindings, for example). Implicit bindings are injected into the code +just before emitting code or External Core. +<p> +After desugaring into Core (-ddump-ds), the definition of <tt>f</tt> looks like this: +<pre> + f x = case x of + Nil{d} -> Nil{v} + MkT{d} a b r -> let { p = (a,b); q = I# r } in + $WMkT p (q+1) +</pre> +Notice the way that pattern matching has been desugared to take account of the fact +that the "real" data constructor MkT has three fields. +<p> +By the time the simplifier has had a go at it, <tt>f</tt> will be transformed to: +<pre> + f x = case x of + Nil{d} -> Nil{v} + MkT{d} a b r -> MkT{v} a b (r +# 1#) +</pre> +Which is highly cool. + + +<h2> The constructor wrapper functions </h2> + +The wrapper functions are automatically generated by GHC, and are +really emitted into the result code (albeit only after CorePre; see +<tt>CorePrep.mkImplicitBinds</tt>). +The wrapper functions are inlined very +vigorously, so you will not see many occurrences of the wrapper +functions in an optimised program, but you may see some. For example, +if your Haskell source has +<pre> + map MkT xs +</pre> +then <tt>$WMkT</tt> will not be inlined (because it is not applied to anything). +That is why we generate real top-level bindings for the wrapper functions, +and generate code for them. + + +<h2> The constructor worker functions </h2> + +Saturated applications of the constructor worker function MkT{v} are +treated specially by the code generator; they really do allocation. +However, we do want a single, shared, top-level definition for +top-level nullary constructors (like True and False). Furthermore, +what if the code generator encounters a non-saturated application of a +worker? E.g. <tt>(map Just xs)</tt>. We could declare that to be an +error (CorePrep should saturate them). But instead we currently +generate a top-level defintion for each constructor worker, whether +nullary or not. It takes the form: +<pre> + MkT{v} = \ p q r -> MkT{v} p q r +</pre> +This is a real hack. The occurrence on the RHS is saturated, so the code generator (both the +one that generates abstract C and the byte-code generator) treats it as a special case and +allocates a MkT; it does not make a recursive call! So now there's a top-level curried +version of the worker which is available to anyone who wants it. +<p> +This strange defintion is not emitted into External Core. Indeed, you might argue that +we should instead pass the list of <tt>TyCon</tt>s to the code generator and have it +generate magic bindings directly. As it stands, it's a real hack: see the code in +CorePrep.mkImplicitBinds. + + +<h2> External Core </h2> + +When emitting External Core, we should see this for our running example: + +<pre> + data T a = MkT a a Int# | Nil{d} + + $WMkT :: (a,a) -> T a -> T a + $WMkT p t = case p of + (a,b) -> seq t (MkT a b t) + + f x = case x of + Nil -> Nil + MkT a b r -> MkT a b (r +# 1#) +</pre> +Notice that it makes perfect sense as a program all by itself. Constructors +look like constructors (albeit not identical to the original Haskell ones). +<p> +When reading in External Core, the parser is careful to read it back in just +as it was before it was spat out, namely: +<pre> + data T a = MkT{d} a a Int# | Nil{d} + + $WMkT :: (a,a) -> T a -> T a + $WMkT p t = case p of + (a,b) -> seq t (MkT{v} a b t) + + f x = case x of + Nil{d} -> Nil{v} + MkT{d} a b r -> MkT{v} a b (r +# 1#) +</pre> + + +<h2> Unboxing strict fields </h2> + +If GHC unboxes strict fields (as in the first argument of <tt>MkT</tt> above), +it also transforms +source-language case expressions. Suppose you write this in your Haskell source: +<pre> + case e of + MkT p t -> ..p..t.. +</pre> +GHC will desugar this to the following Core code: +<pre> + case e of + MkT a b t -> let p = (a,b) in ..p..t.. +</pre> +The local let-binding reboxes the pair because it may be mentioned in +the case alternative. This may well be a bad idea, which is why +<tt>-funbox-strict-fields</tt> is an experimental feature. +<p> +It's essential that when importing a type <tt>T</tt> defined in some +external module <tt>M</tt>, GHC knows what representation was used for +that type, and that in turn depends on whether module <tt>M</tt> was +compiled with <tt>-funbox-strict-fields</tt>. So when writing an +interface file, GHC therefore records with each data type whether its +strict fields (if any) should be unboxed. + +<h2> Labels and info tables </h2> + +<em>Quick rough notes: SLPJ March 2003</em>. +<p> +Every data constructor <tt>C</tt>has two info tables: +<ul> +<li> The static info table (label <tt>C_static_info</tt>), used for statically-allocated constructors. + +<li> The dynamic info table (label <tt>C_con_info</tt>), used for dynamically-allocated constructors. +</ul> +Statically-allocated constructors are not moved by the garbage collector, and therefore have a different closure +type from dynamically-allocated constructors; hence they need +a distinct info table. +Both info tables share the same entry code, but since the entry code is phyiscally juxtaposed with the +info table, it must be duplicated (<tt>C_static_entry</tt> and <tt>C_con_entry</tt> respectively). + + </body> +</html> + diff --git a/docs/comm/the-beast/desugar.html b/docs/comm/the-beast/desugar.html new file mode 100644 index 0000000000..a66740259b --- /dev/null +++ b/docs/comm/the-beast/desugar.html @@ -0,0 +1,156 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Sugar Free: From Haskell To Core</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Sugar Free: From Haskell To Core</h1> + <p> + Up until after type checking, GHC keeps the source program in an + abstract representation of Haskell source without removing any of the + syntactic sugar (such as, list comprehensions) that could easily be + represented by more primitive Haskell. This complicates part of the + front-end considerably as the abstract syntax of Haskell (as exported by + the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsSyn.lhs"><code>HsSyn</code></a>) + is much more complex than a simplified representation close to, say, the + <a href="http://haskell.org/onlinereport/intro.html#sect1.2">Haskell + Kernel</a> would be. However, having a representation that is as close + as possible to the surface syntax simplifies the generation of clear + error messages. As GHC (quite in contrast to "conventional" compilers) + prints code fragments as part of error messages, the choice of + representation is especially important. + <p> + Nonetheless, as soon as the input has passed all static checks, it is + transformed into GHC's principal intermediate language that goes by the + name of <em>Core</em> and whose representation is exported by the + module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/coreSyn/CoreSyn.lhs"><code>CoreSyn</code></a>. + All following compiler phases, except code generation operate on Core. + Due to Andrew Tolmach's effort, there is also an <a + href="http://www.haskell.org/ghc/docs/papers/core.ps.gz">external + representation for Core.</a> + <p> + The conversion of the compiled module from <code>HsSyn</code> into that + of <code>CoreSyn</code> is performed by a phase called the + <em>desugarer</em>, which is located in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/"><code>fptools/ghc/compiler/deSugar/</code></a>. + It's operation is detailed in the following. + </p> + + <h2>Auxilliary Functions</h2> + <p> + The modules <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsMonad.lhs"><code>DsMonad</code></a> + defines the desugarer monad (of type <code>DsM</code>) which maintains + the environment needed for desugaring. In particular, it encapsulates a + unique supply for generating new variables, a map to lookup standard + names (such as functions from the prelude), a source location for error + messages, and a pool to collect warning messages generated during + desugaring. Initialisation of the environment happens in the function <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/Desugar.lhs"><code>Desugar</code></a><code>.desugar</code>, + which is also the main entry point into the desugarer. + <p> + The generation of Core code often involves the use of standard functions + for which proper identifiers (i.e., values of type <code>Id</code> that + actually refer to the definition in the right Prelude) need to be + obtained. This is supported by the function + <code>DsMonad.dsLookupGlobalValue :: Name -> DsM Id</code>. + + <h2><a name="patmat">Pattern Matching</a></h2> + <p> + Nested pattern matching with guards and everything is translated into + the simple, flat case expressions of Core by the following modules: + <dl> + <dt><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/Match.lhs"><code>Match</code></a>: + <dd>This modules contains the main pattern-matching compiler in the form + of a function called <code>match</code>. There is some documentation + as to how <code>match</code> works contained in the module itself. + Generally, the implemented algorithm is similar to the one described + in Phil Wadler's Chapter ? of Simon Peyton Jones' <em>The + Implementation of Functional Programming Languages</em>. + <code>Match</code> exports a couple of functions with not really + intuitive names. In particular, it exports <code>match</code>, + <code>matchWrapper</code>, <code>matchExport</code>, and + <code>matchSimply</code>. The function <code>match</code>, which is + the main work horse, is only used by the other matching modules. The + function <code>matchExport</code> - despite it's name - is merely used + internally in <code>Match</code> and handles warning messages (see + below for more details). The actual interface to the outside is + <code>matchWrapper</code>, which converts the output of the type + checker into the form needed by the pattern matching compiler (i.e., a + list of <code>EquationInfo</code>). Similar in function to + <code>matchWrapper</code> is <code>matchSimply</code>, which provides + an interface for the case where a single expression is to be matched + against a single pattern (as, for example, is the case in bindings in + a <code>do</code> expression). + <dt><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/MatchCon.lhs"><code>MatchCon</code></a>: + <dd>This module generates code for a set of alternative constructor + patterns that belong to a single type by means of the routine + <code>matchConFamily</code>. More precisely, the routine gets a set + of equations where the left-most pattern of each equation is a + constructor pattern with a head symbol from the same type as that of + all the other equations. A Core case expression is generated that + distinguihes between all these constructors. The routine is clever + enough to generate a sparse case expression and to add a catch-all + default case only when needed (i.e., if the case expression isn't + exhaustive already). There is also an explanation at the start of the + modules. + <dt><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/MatchLit.lhs"><code>MatchLit</code></a>: + <dd>Generates code for a set of alternative literal patterns by means of + the routine <code>matchLiterals</code>. The principle is similar to + that of <code>matchConFamily</code>, but all left-most patterns are + literals of the same type. + <dt><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/DsUtils.lhs"><code>DsUtils</code></a>: + <dd>This module provides a set of auxilliary definitions as well as the + data types <code>EquationInfo</code> and <code>MatchResult</code> that + form the input and output, respectively, of the pattern matching + compiler. + <dt><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/deSugar/Check.lhs"><code>Check</code></a>: + <dd>This module does not really contribute the compiling pattern + matching, but it inspects sets of equations to find whether there are + any overlapping patterns or non-exhaustive pattern sets. This task is + implemented by the function <code>check</code>, which returns a list of + patterns that are part of a non-exhaustive case distinction as well as a + set of equation labels that can be reached during execution of the code; + thus, the remaining equations are shadowed due to overlapping patterns. + The function <code>check</code> is invoked and its result converted into + suitable warning messages by the function <code>Match.matchExport</code> + (which is a wrapper for <code>Match.match</code>). + </dl> + <p> + The central function <code>match</code>, given a set of equations, + proceeds in a number of steps: + <ol> + <li>It starts by desugaring the left-most pattern of each equation using + the function <code>tidy1</code> (indirectly via + <code>tidyEqnInfo</code>). During this process, non-elementary + pattern (e.g., those using explicit list syntax <code>[x, y, ..., + z]</code>) are converted to a standard constructor pattern and also + irrefutable pattern are removed. + <li>Then, a process called <em>unmixing</em> clusters the equations into + blocks (without re-ordering them), such that the left-most pattern of + all equations in a block are either all variables, all literals, or + all constructors. + <li>Each block is, then, compiled by <code>matchUnmixedEqns</code>, + which forwards the handling of literal pattern blocks to + <code>MatchLit.matchLiterals</code>, of constructor pattern blocks to + <code>MatchCon.matchConFamily</code>, and hands variable pattern + blocks back to <code>match</code>. + </ol> + + <p><hr><small> +<!-- hhmts start --> +Last modified: Mon Feb 11 22:35:25 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/driver.html b/docs/comm/the-beast/driver.html new file mode 100644 index 0000000000..fbf65e33e7 --- /dev/null +++ b/docs/comm/the-beast/driver.html @@ -0,0 +1,179 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Glorious Driver</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Glorious Driver</h1> + <p> + The Glorious Driver (GD) is the part of GHC that orchestrates the + interaction of all the other pieces that make up GHC. It supersedes the + <em>Evil Driver (ED),</em> which was a Perl script that served the same + purpose and was in use until version 4.08.1 of GHC. Simon Marlow + eventually slayed the ED and instated the GD. The GD is usually called + the <em>Compilation Manager</em> these days. + </p> + <p> + The GD has been substantially extended for GHCi, i.e., the interactive + variant of GHC that integrates the compiler with a (meta-circular) + interpreter since version 5.00. Most of the driver is located in the + directory + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/"><code>fptools/ghc/compiler/main/</code></a>. + </p> + + <h2>Command Line Options</h2> + <p> + GHC's many flavours of command line options make the code interpreting + them rather involved. The following provides a brief overview of the + processing of these options. Since the addition of the interactive + front-end to GHC, there are two kinds of options: <em>static + options</em> and <em>dynamic options.</em> The former can only be set + when the system is invoked, whereas the latter can be altered in the + course of an interactive session. A brief explanation on the difference + between these options and related matters is at the start of the module + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/CmdLineOpts.lhs"><code>CmdLineOpts</code></a>. + The same module defines the enumeration <code>DynFlag</code>, which + contains all dynamic flags. Moreover, there is the labelled record + <code>DynFlags</code> that collects all the flag-related information + that is passed by the compilation manager to the compiler proper, + <code>hsc</code>, whenever a compilation is triggered. If you like to + find out whether an option is static, use the predicate + <code>isStaticHscFlag</code> in the same module. + <p> + The second module that contains a lot of code related to the management + of flags is <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/DriverFlags.hs"><code>DriverFlags.hs</code></a>. + In particular, the module contains two association lists that map the + textual representation of the various flags to a data structure that + tells the driver how to parse the flag (e.g., whether it has any + arguments) and provides its internal representation. All static flags + are contained in <code>static_flags</code>. A whole range of + <code>-f</code> flags can be negated by adding a <code>-f-no-</code> + prefix. These flags are contained in the association list + <code>fFlags</code>. + <p> + The driver uses a nasty hack based on <code>IORef</code>s that permits + the rest of the compiler to access static flags as CAFs; i.e., there is + a family of toplevel variable definitions in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/CmdLineOpts.lhs"><code>CmdLineOpts</code></a>, + below the literate section heading <i>Static options</i>, each of which + contains the value of one static option. This is essentially realised + via global variables (in the sense of C-style, updatable, global + variables) defined via an evil pre-processor macro named + <code>GLOBAL_VAR</code>, which is defined in a particularly ugly corner + of GHC, namely the C header file + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/HsVersions.h"><code>HsVersions.h</code></a>. + + <h2>What Happens When</h2> + <p> + Inside the Haskell compiler proper (<code>hsc</code>), a whole series of + stages (``passes'') are executed in order to transform your Haskell program + into C or native code. This process is orchestrated by + <code>main/HscMain.hscMain</code> and its relative + <code>hscReComp</code>. The latter directly invokes, in order, + the parser, the renamer, the typechecker, the desugarer, the + simplifier (Core2Core), the CoreTidy pass, the CorePrep pass, + conversion to STG (CoreToStg), the interface generator + (MkFinalIface), the code generator, and code output. The + simplifier is the most complex of these, and is made up of many + sub-passes. These are controlled by <code>buildCoreToDo</code>, + as described below. + + <h2>Scheduling Optimisations Phases</h2> + <p> + GHC has a large variety of optimisations at its disposal, many of which + have subtle interdependencies. The overall plan for program + optimisation is fixed in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/DriverState.hs"><code>DriverState.hs</code></a>. + First of all, there is the variable <code>hsc_minusNoO_flags</code> that + determines the <code>-f</code> options that you get without + <code>-O</code> (aka optimisation level 0) as well as + <code>hsc_minusO_flags</code> and <code>hsc_minusO2_flags</code> for + <code>-O</code> and <code>-O2</code>. + <p> + However, most of the strategic decisions about optimisations on the + intermediate language Core are encoded in the value produced by + <code>buildCoreToDo</code>, which is a list with elements of type + <code>CoreToDo</code>. Each element of this list specifies one step in + the sequence of core optimisations executed by the <a + href="simplifier.html">Mighty Simplifier</a>. The type + <code>CoreToDo</code> is defined in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/CmdLineOpts.lhs"><code>CmdLineOpts.lhs</code></a>. + The actual execution of the optimisation plan produced by + <code>buildCoreToDo</code> is performed by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/SimplCore.lhs"><code>SimpleCore</code></a><code>.doCorePasses</code>. + Core optimisation plans consist of a number of simplification phases + (currently, three for optimisation levels of 1 or higher) with + decreasing phase numbers (the lowest, corresponding to the last phase, + namely 0). Before and after these phases, optimisations such as + specialisation, let floating, worker/wrapper, and so on are executed. + The sequence of phases is such that the synergistic effect of the phases + is maximised -- however, this is a fairly fragile arrangement. + <p> + There is a similar construction for optimisations on STG level stored in + the variable <code>buildStgToDo :: [StgToDo]</code>. However, this is a + lot less complex than the arrangement for Core optimisations. + + <h2>Linking the <code>RTS</code> and <code>libHSstd</code></h2> + <p> + Since the RTS and HSstd refer to each other, there is a Cunning + Hack to avoid putting them each on the command-line twice or + thrice (aside: try asking for `plaice and chips thrice' in a + fish and chip shop; bet you only get two lots). The hack involves + adding + the symbols that the RTS needs from libHSstd, such as + <code>PrelWeak_runFinalizzerBatch_closure</code> and + <code>__stginit_Prelude</code>, to the link line with the + <code>-u</code> flag. The standard library appears before the + RTS on the link line, and these options cause the corresponding + symbols to be picked up even so the linked might not have seen them + being used as the RTS appears later on the link line. As a result, + when the RTS is also scanned, these symbols are already resolved. This + avoids the linker having to read the standard library and RTS + multiple times. + </p> + <p> + This does, however, leads to a complication. Normal Haskell + programs do not have a <code>main()</code> function, so this is + supplied by the RTS (in the file + <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/rts/Main.c"><code>Main.c</code></a>). + It calls <code>startupHaskell</code>, which + itself calls <code>__stginit_PrelMain</code>, which is therefore, + since it occurs in the standard library, one of the symbols + passed to the linker using the <code>-u</code> option. This is fine + for standalone Haskell programs, but as soon as the Haskell code is only + used as part of a program implemented in a foreign language, the + <code>main()</code> function of that foreign language should be used + instead of that of the Haskell runtime. In this case, the previously + described arrangement unfortunately fails as + <code>__stginit_PrelMain</code> had better not be linked in, + because it tries to call <code>__stginit_Main</code>, which won't + exist. In other words, the RTS's <code>main()</code> refers to + <code>__stginit_PrelMain</code> which in turn refers to + <code>__stginit_Main</code>. Although the RTS's <code>main()</code> + might not be linked in if the program provides its own, the driver + will normally force <code>__stginit_PrelMain</code> to be linked in anyway, + using <code>-u</code>, because it's a back-reference from the + RTS to HSstd. This case is coped with by the <code>-no-hs-main</code> + flag, which suppresses passing the corresonding <code>-u</code> option + to the linker -- although in some versions of the compiler (e.g., 5.00.2) + it didn't work. In addition, the driver generally places the C program + providing the <code>main()</code> that we want to use before the RTS + on the link line. Therefore, the RTS's main is never used and + without the <code>-u</code> the label <code>__stginit_PrelMain</code> + will not be linked. + </p> + + <p><small> +<!-- hhmts start --> +Last modified: Tue Feb 19 11:09:00 UTC 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/fexport.html b/docs/comm/the-beast/fexport.html new file mode 100644 index 0000000000..956043bafb --- /dev/null +++ b/docs/comm/the-beast/fexport.html @@ -0,0 +1,231 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - foreign export</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - foreign export</h1> + + The implementation scheme for foreign export, as of 27 Feb 02, is + as follows. There are four cases, of which the first two are easy. + <p> + <b>(1) static export of an IO-typed function from some module <code>MMM</code></b> + <p> + <code>foreign export foo :: Int -> Int -> IO Int</code> + <p> + For this we generate no Haskell code. However, a C stub is + generated, and it looks like this: + <p> + <pre> +extern StgClosure* MMM_foo_closure; + +HsInt foo (HsInt a1, HsInt a2) +{ + SchedulerStatus rc; + HaskellObj ret; + rc = rts_evalIO( + rts_apply(rts_apply(MMM_foo_closure,rts_mkInt(a1)), + rts_mkInt(a2) + ), + &ret + ); + rts_checkSchedStatus("foo",rc); + return(rts_getInt(ret)); +} +</pre> + <p> + This does the obvious thing: builds in the heap the expression + <code>(foo a1 a2)</code>, calls <code>rts_evalIO</code> to run it, + and uses <code>rts_getInt</code> to fish out the result. + + <p> + <b>(2) static export of a non-IO-typed function from some module <code>MMM</code></b> + <p> + <code>foreign export foo :: Int -> Int -> Int</code> + <p> + This is identical to case (1), with the sole difference that the + stub calls <code>rts_eval</code> rather than + <code>rts_evalIO</code>. + <p> + + <b>(3) dynamic export of an IO-typed function from some module <code>MMM</code></b> + <p> + <code>foreign export mkCallback :: (Int -> Int -> IO Int) -> IO (FunPtr a)</code> + <p> + Dynamic exports are a whole lot more complicated than their static + counterparts. + <p> + First of all, we get some Haskell code, which, when given a + function <code>callMe :: (Int -> Int -> IO Int)</code> to be made + C-callable, IO-returns a <code>FunPtr a</code>, which is the + address of the resulting C-callable code. This address can now be + handed out to the C-world, and callers to it will get routed + through to <code>callMe</code>. + <p> + The generated Haskell function looks like this: + <p> +<pre> +mkCallback f + = do sp <- mkStablePtr f + r <- ccall "createAdjustorThunk" sp (&"run_mkCallback") + return r +</pre> + <p> + <code>createAdjustorThunk</code> is a gruesome, + architecture-specific function in the RTS. It takes a stable + pointer to the Haskell function to be run, and the address of the + associated C wrapper, and returns a piece of machine code, + which, when called from the outside (C) world, eventually calls + through to <code>f</code>. + <p> + This machine code fragment is called the "Adjustor Thunk" (don't + ask me why). What it does is simply to call onwards to the C + helper + function <code>run_mkCallback</code>, passing all the args given + to it but also conveying <code>sp</code>, which is a stable + pointer + to the Haskell function to run. So: + <p> +<pre> +createAdjustorThunk ( StablePtr sp, CCodeAddress addr_of_helper_C_fn ) +{ + create malloc'd piece of machine code "mc", behaving thusly: + + mc ( args_to_mc ) + { + jump to addr_of_helper_C_fn, passing sp as an additional + argument + } +</pre> + <p> + This is a horrible hack, because there is no portable way, even at + the machine code level, to function which adds one argument and + then transfers onwards to another C function. On x86s args are + pushed R to L onto the stack, so we can just push <code>sp</code>, + fiddle around with return addresses, and jump onwards to the + helper C function. However, on architectures which use register + windows and/or pass args extensively in registers (Sparc, Alpha, + MIPS, IA64), this scheme borders on the unviable. GHC has a + limited <code>createAdjustorThunk</code> implementation for Sparc + and Alpha, which handles only the cases where all args, including + the extra one, fit in registers. + <p> + Anyway: the other lump of code generated as a result of a + f-x-dynamic declaration is the C helper stub. This is basically + the same as in the static case, except that it only ever gets + called from the adjustor thunk, and therefore must accept + as an extra argument, a stable pointer to the Haskell function + to run, naturally enough, as this is not known until run-time. + It then dereferences the stable pointer and does the call in + the same way as the f-x-static case: +<pre> +HsInt Main_d1kv ( StgStablePtr the_stableptr, + void* original_return_addr, + HsInt a1, HsInt a2 ) +{ + SchedulerStatus rc; + HaskellObj ret; + rc = rts_evalIO( + rts_apply(rts_apply((StgClosure*)deRefStablePtr(the_stableptr), + rts_mkInt(a1) + ), + rts_mkInt(a2) + ), + &ret + ); + rts_checkSchedStatus("Main_d1kv",rc); + return(rts_getInt(ret)); +} +</pre> + <p> + Note how this function has a purely made-up name + <code>Main_d1kv</code>, since unlike the f-x-static case, this + function is never called from user code, only from the adjustor + thunk. + <p> + Note also how the function takes a bogus parameter + <code>original_return_addr</code>, which is part of this extra-arg + hack. The usual scheme is to leave the original caller's return + address in place and merely push the stable pointer above that, + hence the spare parameter. + <p> + Finally, there is some extra trickery, detailed in + <code>ghc/rts/Adjustor.c</code>, to get round the following + problem: the adjustor thunk lives in mallocville. It is + quite possible that the Haskell code will actually + call <code>free()</code> on the adjustor thunk used to get to it + -- because otherwise there is no way to reclaim the space used + by the adjustor thunk. That's all very well, but it means that + the C helper cannot return to the adjustor thunk in the obvious + way, since we've already given it back using <code>free()</code>. + So we leave, on the C stack, the address of whoever called the + adjustor thunk, and before calling the helper, mess with the stack + such that when the helper returns, it returns directly to the + adjustor thunk's caller. + <p> + That's how the <code>stdcall</code> convention works. If the + adjustor thunk has been called using the <code>ccall</code> + convention, we return indirectly, via a statically-allocated + yet-another-magic-piece-of-code, which takes care of removing the + extra argument that the adjustor thunk pushed onto the stack. + This is needed because in <code>ccall</code>-world, it is the + caller who removes args after the call, and the original caller of + the adjustor thunk has no way to know about the extra arg pushed + by the adjustor thunk. + <p> + You didn't really want to know all this stuff, did you? + <p> + + + + <b>(4) dynamic export of an non-IO-typed function from some module <code>MMM</code></b> + <p> + <code>foreign export mkCallback :: (Int -> Int -> Int) -> IO (FunPtr a)</code> + <p> + (4) relates to (3) as (2) relates to (1), that is, it's identical, + except the C stub uses <code>rts_eval</code> instead of + <code>rts_evalIO</code>. + <p> + + + <h2>Some perspective on f-x-dynamic</h2> + + The only really horrible problem with f-x-dynamic is how the + adjustor thunk should pass to the C helper the stable pointer to + use. Ideally we would like this to be conveyed via some invisible + side channel, since then the adjustor thunk could simply jump + directly to the C helper, with no non-portable stack fiddling. + <p> + Unfortunately there is no obvious candidate for the invisible + side-channel. We've chosen to pass it on the stack, with the + bad consequences detailed above. Another possibility would be to + park it in a global variable, but this is non-reentrant and + non-(OS-)thread-safe. A third idea is to put it into a callee-saves + register, but that has problems too: the C helper may not use that + register and therefore we will have trashed any value placed there + by the caller; and there is no C-level portable way to read from + the register inside the C helper. + <p> + In short, we can't think of a really satisfactory solution. I'd + vote for introducing some kind of OS-thread-local-state and passing + it in there, but that introduces complications of its own. + <p> + <b>OS-thread-safety</b> is of concern in the C stubs, whilst + building up the expressions to run. These need to have exclusive + access to the heap whilst allocating in it. Also, there needs to + be some guarantee that no GC will happen in between the + <code>deRefStablePtr</code> call and when <code>rts_eval[IO]</code> + starts running. At the moment there are no guarantees for + either property. This needs to be sorted out before the + implementation can be regarded as fully safe to use. + +<p><small> + +<!-- hhmts start --> +Last modified: Weds 27 Feb 02 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/ghci.html b/docs/comm/the-beast/ghci.html new file mode 100644 index 0000000000..b893acdeb4 --- /dev/null +++ b/docs/comm/the-beast/ghci.html @@ -0,0 +1,407 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - GHCi</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - GHCi</h1> + + This isn't a coherent description of how GHCi works, sorry. What + it is (currently) is a dumping ground for various bits of info + pertaining to GHCi, which ought to be recorded somewhere. + + <h2>Debugging the interpreter</h2> + + The usual symptom is that some expression / program crashes when + running on the interpreter (commonly), or gets wierd results + (rarely). Unfortunately, finding out what the problem really is + has proven to be extremely difficult. In retrospect it may be + argued a design flaw that GHC's implementation of the STG + execution mechanism provides only the weakest of support for + automated internal consistency checks. This makes it hard to + debug. + <p> + Execution failures in the interactive system can be due to + problems with the bytecode interpreter, problems with the bytecode + generator, or problems elsewhere. From the bugs seen so far, + the bytecode generator is often the culprit, with the interpreter + usually being correct. + <p> + Here are some tips for tracking down interactive nonsense: + <ul> + <li>Find the smallest source fragment which causes the problem. + <p> + <li>Using an RTS compiled with <code>-DDEBUG</code> (nb, that + means the RTS from the previous stage!), run with <code>+RTS + -D2</code> to get a listing in great detail from the + interpreter. Note that the listing is so voluminous that + this is impractical unless you have been diligent in + the previous step. + <p> + <li>At least in principle, using the trace and a bit of GDB + poking around at the time of death, you can figure out what + the problem is. In practice you quickly get depressed at + the hopelessness of ever making sense of the mass of + details. Well, I do, anyway. + <p> + <li><code>+RTS -D2</code> tries hard to print useful + descriptions of what's on the stack, and often succeeds. + However, it has no way to map addresses to names in + code/data loaded by our runtime linker. So the C function + <code>ghci_enquire</code> is provided. Given an address, it + searches the loaded symbol tables for symbols close to that + address. You can run it from inside GDB: + <pre> + (gdb) p ghci_enquire ( 0x50a406f0 ) + 0x50a406f0 + -48 == `PrelBase_Czh_con_info' + 0x50a406f0 + -12 == `PrelBase_Izh_static_info' + 0x50a406f0 + -48 == `PrelBase_Czh_con_entry' + 0x50a406f0 + -24 == `PrelBase_Izh_con_info' + 0x50a406f0 + 16 == `PrelBase_ZC_con_entry' + 0x50a406f0 + 0 == `PrelBase_ZMZN_static_entry' + 0x50a406f0 + -36 == `PrelBase_Czh_static_entry' + 0x50a406f0 + -24 == `PrelBase_Izh_con_entry' + 0x50a406f0 + 64 == `PrelBase_EQ_static_info' + 0x50a406f0 + 0 == `PrelBase_ZMZN_static_info' + 0x50a406f0 + 48 == `PrelBase_LT_static_entry' + $1 = void + </pre> + In this case the enquired-about address is + <code>PrelBase_ZMZN_static_entry</code>. If no symbols are + close to the given addr, nothing is printed. Not a great + mechanism, but better than nothing. + <p> + <li>We have had various problems in the past due to the bytecode + generator (<code>compiler/ghci/ByteCodeGen.lhs</code>) being + confused about the true set of free variables of an + expression. The compilation scheme for <code>let</code>s + applies the BCO for the RHS of the let to its free + variables, so if the free-var annotation is wrong or + misleading, you end up with code which has wrong stack + offsets, which is usually fatal. + <p> + <li>The baseline behaviour of the interpreter is to interpret + BCOs, and hand all other closures back to the scheduler for + evaluation. However, this causes a huge number of expensive + context switches, so the interpreter knows how to enter the + most common non-BCO closure types by itself. + <p> + These optimisations complicate the interpreter. + If you think you have an interpreter problem, re-enable the + define <code>REFERENCE_INTERPRETER</code> in + <code>ghc/rts/Interpreter.c</code>. All optimisations are + thereby disabled, giving the baseline + I-only-know-how-to-enter-BCOs behaviour. + <p> + <li>Following the traces is often problematic because execution + hops back and forth between the interpreter, which is + traced, and compiled code, which you can't see. + Particularly annoying is when the stack looks OK in the + interpreter, then compiled code runs for a while, and later + we arrive back in the interpreter, with the stack corrupted, + and usually in a completely different place from where we + left off. + <p> + If this is biting you baaaad, it may be worth copying + sources for the compiled functions causing the problem, into + your interpreted module, in the hope that you stay in the + interpreter more of the time. Of course this doesn't work + very well if you've defined + <code>REFERENCE_INTERPRETER</code> in + <code>ghc/rts/Interpreter.c</code>. + <p> + <li>There are various commented-out pieces of code in + <code>Interpreter.c</code> which can be used to get the + stack sanity-checked after every entry, and even after after + every bytecode instruction executed. Note that some + bytecodes (<code>PUSH_UBX</code>) leave the stack in + an unwalkable state, so the <code>do_print_stack</code> + local variable is used to suppress the stack walk after + them. + </ul> + + + <h2>Useful stuff to know about the interpreter</h2> + + The code generation scheme is straightforward (naive, in fact). + <code>-ddump-bcos</code> prints each BCO along with the Core it + was generated from, which is very handy. + <ul> + <li>Simple lets are compiled in-line. For the general case, let + v = E in ..., E is compiled into a new BCO which takes as + args its free variables, and v is bound to AP(the new BCO, + free vars of E). + <p> + <li><code>case</code>s as usual, become: push the return + continuation, enter the scrutinee. There is some magic to + make all combinations of compiled/interpreted calls and + returns work, described below. In the interpreted case, all + case alts are compiled into a single big return BCO, which + commences with instructions implementing a switch tree. + </ul> + <p> + <b>ARGCHECK magic</b> + <p> + You may find ARGCHECK instructions at the start of BCOs which + don't appear to need them; case continuations in particular. + These play an important role: they force objects which should + evaluated to BCOs to actually be BCOs. + <p> + Typically, there may be an application node somewhere in the heap. + This is a thunk which when leant on turns into a BCO for a return + continuation. The thunk may get entered with an update frame on + top of the stack. This is legitimate since from one viewpoint + this is an AP which simply reduces to a data object, so does not + have functional type. However, once the AP turns itself into a + BCO (so to speak) we cannot simply enter the BCO, because that + expects to see args on top of the stack, not an update frame. + Therefore any BCO which expects something on the stack above an + update frame, even non-function BCOs, start with an ARGCHECK. In + this case it fails, the update is done, the update frame is + removed, and the BCO re-entered. Subsequent entries of the BCO of + course go unhindered. + <p> + The optimised (<code>#undef REFERENCE_INTERPRETER</code>) handles + this case specially, so that a trip through the scheduler is + avoided. When reading traces from <code>+RTS -D2 -RTS</code>, you + may see BCOs which appear to execute their initial ARGCHECK insn + twice. The first time it fails; the interpreter does the update + immediately and re-enters with no further comment. + <p> + This is all a bit ugly, and, as SimonM correctly points out, it + would have been cleaner to make BCOs unpointed (unthunkable) + objects, so that a pointer to something <code>:: BCO#</code> + really points directly at a BCO. + <p> + <b>Stack management</b> + <p> + There isn't any attempt to stub the stack, minimise its growth, or + generally remove unused pointers ahead of time. This is really + due to lazyness on my part, although it does have the minor + advantage that doing something cleverer would almost certainly + increase the number of bytecodes that would have to be executed. + Of course we SLIDE out redundant stuff, to get the stack back to + the sequel depth, before returning a HNF, but that's all. As + usual this is probably a cause of major space leaks. + <p> + <b>Building constructors</b> + <p> + Constructors are built on the stack and then dumped into the heap + with a single PACK instruction, which simply copies the top N + words of the stack verbatim into the heap, adds an info table, and zaps N + words from the stack. The constructor args are pushed onto the + stack one at a time. One upshot of this is that unboxed values + get pushed untaggedly onto the stack (via PUSH_UBX), because that's how they + will be in the heap. That in turn means that the stack is not + always walkable at arbitrary points in BCO execution, although + naturally it is whenever GC might occur. + <p> + Function closures created by the interpreter use the AP-node + (tagged) format, so although their fields are similarly + constructed on the stack, there is never a stack walkability + problem. + <p> + <b>Unpacking constructors</b> + <p> + At the start of a case continuation, the returned constructor is + unpacked onto the stack, which means that unboxed fields have to + be tagged. Rather than burdening all such continuations with a + complex, general mechanism, I split it into two. The + allegedly-common all-pointers case uses a single UNPACK insn + to fish out all fields with no further ado. The slow case uses a + sequence of more complex UPK_TAG insns, one for each field (I + think). This seemed like a good compromise to me. + <p> + <b>Perspective</b> + <p> + I designed the bytecode mechanism with the experience of both STG + hugs and Classic Hugs in mind. The latter has an small + set of bytecodes, a small interpreter loop, and runs amazingly + fast considering the cruddy code it has to interpret. The former + had a large interpretative loop with many different opcodes, + including multiple minor variants of the same thing, which + made it difficult to optimise and maintain, yet it performed more + or less comparably with Classic Hugs. + <p> + My design aims were therefore to minimise the interpreter's + complexity whilst maximising performance. This means reducing the + number of opcodes implemented, whilst reducing the number of insns + despatched. In particular there are only two opcodes, PUSH_UBX + and UPK_TAG, which deal with tags. STG Hugs had dozens of opcodes + for dealing with tagged data. In cases where the common + all-pointers case is significantly simpler (UNPACK) I deal with it + specially. Finally, the number of insns executed is reduced a + little by merging multiple pushes, giving PUSH_LL and PUSH_LLL. + These opcode pairings were determined by using the opcode-pair + frequency profiling stuff which is ifdef-d out in + <code>Interpreter.c</code>. These significantly improve + performance without having much effect on the uglyness or + complexity of the interpreter. + <p> + Overall, the interpreter design is something which turned out + well, and I was pleased with it. Unfortunately I cannot say the + same of the bytecode generator. + + <h2><code>case</code> returns between interpreted and compiled code</h2> + + Variants of the following scheme have been drifting around in GHC + RTS documentation for several years. Since what follows is + actually what is implemented, I guess it supersedes all other + documentation. Beware; the following may make your brain melt. + In all the pictures below, the stack grows downwards. + <p> + <b>Returning to interpreted code</b>. + <p> + Interpreted returns employ a set of polymorphic return infotables. + Each element in the set corresponds to one of the possible return + registers (R1, D1, F1) that compiled code will place the returned + value in. In fact this is a bit misleading, since R1 can be used + to return either a pointer or an int, and we need to distinguish + these cases. So, supposing the set of return registers is {R1p, + R1n, D1, F1}, there would be four corresponding infotables, + <code>stg_ctoi_ret_R1p_info</code>, etc. In the pictures below we + call them <code>stg_ctoi_ret_REP_info</code>. + <p> + These return itbls are polymorphic, meaning that all 8 vectored + return codes and the direct return code are identical. + <p> + Before the scrutinee is entered, the stack is arranged like this: + <pre> + | | + +--------+ + | BCO | -------> the return contination BCO + +--------+ + | itbl * | -------> stg_ctoi_ret_REP_info, with all 9 codes as follows: + +--------+ + BCO* bco = Sp[1]; + push R1/F1/D1 depending on REP + push bco + yield to sched + </pre> + On entry, the interpreted contination BCO expects the stack to look + like this: + <pre> + | | + +--------+ + | BCO | -------> the return contination BCO + +--------+ + | itbl * | -------> ret_REP_ctoi_info, with all 9 codes as follows: + +--------+ + : VALUE : (the returned value, shown with : since it may occupy + +--------+ multiple stack words) + </pre> + A machine code return will park the returned value in R1/F1/D1, + and enter the itbl on the top of the stack. Since it's our magic + itbl, this pushes the returned value onto the stack, which is + where the interpreter expects to find it. It then pushes the BCO + (again) and yields. The scheduler removes the BCO from the top, + and enters it, so that the continuation is interpreted with the + stack as shown above. + <p> + An interpreted return will create the value to return at the top + of the stack. It then examines the return itbl, which must be + immediately underneath the return value, to see if it is one of + the magic <code>stg_ctoi_ret_REP_info</code> set. Since this is so, + it knows it is returning to an interpreted contination. It + therefore simply enters the BCO which it assumes it immediately + underneath the itbl on the stack. + + <p> + <b>Returning to compiled code</b>. + <p> + Before the scrutinee is entered, the stack is arranged like this: + <pre> + ptr to vec code 8 ------> return vector code 8 + | | .... + +--------+ ptr to vec code 1 ------> return vector code 1 + | itbl * | -- Itbl end + +--------+ \ .... + \ Itbl start + ----> direct return code + </pre> + The scrutinee value is then entered. + The case continuation(s) expect the stack to look the same, with + the returned HNF in a suitable return register, R1, D1, F1 etc. + <p> + A machine code return knows whether it is doing a vectored or + direct return, and, if the former, which vector element it is. + So, for a direct return we jump to <code>Sp[0]</code>, and for a + vectored return, jump to <code>((CodePtr*)(Sp[0]))[ - ITBL_LENGTH + - vector number ]</code>. This is (of course) the scheme that + compiled code has been using all along. + <p> + An interpreted return will, as described just above, have examined + the itbl immediately beneath the return value it has just pushed, + and found it not to be one of the <code>ret_REP_ctoi_info</code> set, + so it knows this must be a return to machine code. It needs to + pop the return value, currently on the stack, into R1/F1/D1, and + jump through the info table. Unfortunately the first part cannot + be accomplished directly since we are not in Haskellised-C world. + <p> + We therefore employ a second family of magic infotables, indexed, + like the first, on the return representation, and therefore with + names of the form <code>stg_itoc_ret_REP_info</code>. (Note: + <code>itoc</code>; the previous bunch were <code>ctoi</code>). + This is pushed onto the stack (note, tagged values have their tag + zapped), giving: + <pre> + | | + +--------+ + | itbl * | -------> arbitrary machine code return itbl + +--------+ + : VALUE : (the returned value, possibly multiple words) + +--------+ + | itbl * | -------> stg_itoc_ret_REP_info, with code: + +--------+ + pop myself (stg_itoc_ret_REP_info) off the stack + pop return value into R1/D1/F1 + do standard machine code return to itbl at t.o.s. + </pre> + We then return to the scheduler, asking it to enter the itbl at + t.o.s. When entered, <code>stg_itoc_ret_REP_info</code> removes + itself from the stack, pops the return value into the relevant + return register, and returns to the itbl to which we were trying + to return in the first place. + <p> + Amazingly enough, this stuff all actually works! Well, mostly ... + <p> + <b>Unboxed tuples: a Right Royal Spanner In The Works</b> + <p> + The above scheme depends crucially on having magic infotables + <code>stg_{itoc,ctoi}_ret_REP_info</code> for each return + representation <code>REP</code>. It unfortunately fails miserably + in the face of unboxed tuple returns, because the set of required + tables would be infinite; this despite the fact that for any given + unboxed tuple return type, the scheme could be made to work fine. + <p> + This is a serious problem, because it prevents interpreted + code from doing <code>IO</code>-typed returns, since <code>IO + t</code> is implemented as <code>(# t, RealWorld# #)</code> or + thereabouts. This restriction in turn rules out FFI stuff in the + interpreter. Not good. + <p> + Although we have no way to make general unboxed tuples work, we + can at least make <code>IO</code>-types work using the following + ultra-kludgey observation: <code>RealWorld#</code> doesn't really + exist and so has zero size, in compiled code. In turn this means + that a type of the form <code>(# t, RealWorld# #)</code> has the + same representation as plain <code>t</code> does. So the bytecode + generator, whilst rejecting code with general unboxed tuple + returns, recognises and accepts this special case. Which means + that <code>IO</code>-typed stuff works in the interpreter. Just. + <p> + If anyone asks, I will claim I was out of radio contact, on a + 6-month walking holiday to the south pole, at the time this was + ... er ... dreamt up. + + +<p><small> + +<!-- hhmts start --> +Last modified: Thursday February 7 15:33:49 GMT 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/main.html b/docs/comm/the-beast/main.html new file mode 100644 index 0000000000..332ffaa501 --- /dev/null +++ b/docs/comm/the-beast/main.html @@ -0,0 +1,35 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Compiling and running the Main module</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>Compiling and running the Main module</h1> + +GHC allows you to determine which module contains the "main" function, and +what that function is called, via the <code>-fmain-is</code> flag. The trouble is +that the runtime system is fixed, so what symbol should it link to? +<p> +The current solution is this. Suppose the main function is <code>Foo.run</code>. +<ul> +<li> +Then, when compiling module <code>Foo</code>, GHC adds an extra definition: +<pre> + :Main.main = runIO Foo.run +</pre> +Now the RTS can invoke <code>:Main.main</code> to start the program. (This extra +definition is inserted in TcRnDriver.checkMain.) +<p><li> +Before starting the program, though, the RTS also initialises the module tree +by calling <code>init_:Main</code>, so when compiling the main module (Foo in this case), +as well as generating <code>init_Foo</code> as usual, GHC also generates +<pre> + init_zcMain() { init_Foo; } +</pre> +This extra initialisation code is generated in CodeGen.mkModuleInit. +</ul> + + </body> +</html> diff --git a/docs/comm/the-beast/mangler.html b/docs/comm/the-beast/mangler.html new file mode 100644 index 0000000000..1ad80f0d5c --- /dev/null +++ b/docs/comm/the-beast/mangler.html @@ -0,0 +1,79 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Evil Mangler</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Evil Mangler</h1> + <p> + The Evil Mangler (EM) is a Perl script invoked by the <a + href="driver.html">Glorious Driver</a> after the C compiler (gcc) has + translated the GHC-produced C code into assembly. Consequently, it is + only of interest if <code>-fvia-C</code> is in effect (either explicitly + or implicitly). + + <h4>Its purpose</h4> + <p> + The EM reads the assembly produced by gcc and re-arranges code blocks as + well as nukes instructions that it considers <em>non-essential.</em> It + derives it evilness from its utterly ad hoc, machine, compiler, and + whatnot dependent design and implementation. More precisely, the EM + performs the following tasks: + <ul> + <li>The code executed when a closure is entered is moved adjacent to + that closure's infotable. Moreover, the order of the info table + entries is reversed. Also, SRT pointers are removed from closures that + don't need them (non-FUN, RET and THUNK ones). + <li>Function prologue and epilogue code is removed. (GHC generated code + manages its own stack and uses the system stack only for return + addresses and during calls to C code.) + <li>Certain code patterns are replaced by simpler code (eg, loads of + fast entry points followed by indirect jumps are replaced by direct + jumps to the fast entry point). + </ul> + + <h4>Implementation</h4> + <p> + The EM is located in the Perl script <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/driver/mangler/ghc-asm.lprl"><code>ghc-asm.lprl</code></a>. + The script reads the <code>.s</code> file and chops it up into + <em>chunks</em> (that's how they are actually called in the script) that + roughly correspond to basic blocks. Each chunk is annotated with an + educated guess about what kind of code it contains (e.g., infotable, + fast entry point, slow entry point, etc.). The annotations also contain + the symbol introducing the chunk of assembly and whether that chunk has + already been processed or not. + <p> + The parsing of the input into chunks as well as recognising assembly + instructions that are to be removed or altered is based on a large + number of Perl regular expressions sprinkled over the whole code. These + expressions are rather fragile as they heavily rely on the structure of + the generated code - in fact, they even rely on the right amount of + white space and thus on the formatting of the assembly. + <p> + Afterwards, the chunks are reordered, some of them purged, and some + stripped of some useless instructions. Moreover, some instructions are + manipulated (eg, loads of fast entry points followed by indirect jumps + are replaced by direct jumps to the fast entry point). + <p> + The EM knows which part of the code belongs to function prologues and + epilogues as <a href="../rts-libs/stgc.html">STG C</a> adds tags of the + form <code>--- BEGIN ---</code> and <code>--- END ---</code> the + assembler just before and after the code proper of a function starts. + It adds these tags using gcc's <code>__asm__</code> feature. + <p> + <strong>Update:</strong> Gcc 2.96 upwards performs more aggressive basic + block re-ordering and dead code elimination. This seems to make the + whole <code>--- END ---</code> tag business redundant -- in fact, if + proper code is generated, no <code>--- END ---</code> tags survive gcc + optimiser. + + <p><small> +<!-- hhmts start --> +Last modified: Sun Feb 17 17:55:47 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/modules.html b/docs/comm/the-beast/modules.html new file mode 100644 index 0000000000..a6655a68a7 --- /dev/null +++ b/docs/comm/the-beast/modules.html @@ -0,0 +1,80 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Modules, ModuleNames and Packages</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>Modules, ModuleNames and Packages</h1> + + <p>This section describes the datatypes <code>ModuleName</code> + <code>Module</code> and <code>PackageName</code> all available + from the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/Module.lhs"><code>Module</code></a>.<p> + + <h2>Packages</h2> + + <p>A package is a collection of (zero or more) Haskell modules, + together with some information about external libraries, extra C + compiler options, and other things that this collection of modules + requires. When using DLLs on windows (or shared libraries on a + Unix system; currently unsupported), a package can consist of only + a single shared library of Haskell code; the reason for this is + described below. + + <p>Packages are further described in the User's Guide <a + href="http://www.haskell.org/ghc/docs/latest/packages.html">here</a>. + + <h2>The ModuleName type</h2> + + <p>At the bottom of the hierarchy is a <code>ModuleName</code>, + which, as its name suggests, is simply the name of a module. It + is represented as a Z-encoded FastString, and is an instance of + <code>Uniquable</code> so we can build <code>FiniteMap</code>s + with <code>ModuleName</code>s as the keys. + + <p>A <code>ModuleName</code> can be built from a + <code>String</code>, using the <code>mkModuleName</code> function. + + <h2>The Module type</h2> + + <p>For a given module, the compiler also needs to know whether the + module is in the <em>home package</em>, or in another package. + This distinction is important for two reasons: + + <ul> + <li><p>When generating code to call a function in another package, + the compiler might have to generate a cross-DLL call, which is + different from an intra-DLL call (hence the restriction that the + code in a package can only reside in a single DLL). + + <li><p>We avoid putting version information in an interface file + for entities defined in another package, on the grounds that other + packages are generally "stable". This also helps keep the size of + interface files down. + </ul> + + <p>The <code>Module</code> type contains a <code>ModuleName</code> + and a <code>PackageInfo</code> field. The + <code>PackageInfo</code> indicates whether the given + <code>Module</code> comes from the current package or from another + package. + + <p>To get the actual package in which a given module resides, you + have to read the interface file for that module, which contains + the package name (actually the value of the + <code>-package-name</code> flag when that module was built). This + information is currently unused inside the compiler, but we might + make use of it in the future, especially with the advent of + hierarchical modules, to allow the compiler to automatically + figure out which packages a program should be linked with, and + thus avoid the need to specify <code>-package</code> options on + the command line. + + <p><code>Module</code>s are also instances of + <code>Uniquable</code>, and indeed the unique of a + <code>Module</code> is the same as the unique of the underlying + <code>ModuleName</code>. + </body> +</html> diff --git a/docs/comm/the-beast/names.html b/docs/comm/the-beast/names.html new file mode 100644 index 0000000000..061fae3ebf --- /dev/null +++ b/docs/comm/the-beast/names.html @@ -0,0 +1,169 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The truth about names: OccNames, and Names</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The truth about names: OccNames, and Names</h1> + <p> + Every entity (type constructor, class, identifier, type variable) has a + <code>Name</code>. The <code>Name</code> type is pervasive in GHC, and + is defined in <code>basicTypes/Name.lhs</code>. Here is what a Name + looks like, though it is private to the Name module. + </p> + <blockquote> + <pre> +data Name = Name { + n_sort :: NameSort, -- What sort of name it is + n_occ :: !OccName, -- Its occurrence name + n_uniq :: Unique, -- Its identity + n_loc :: !SrcLoc -- Definition site + }</pre> + </blockquote> + <ul> + <li> The <code>n_sort</code> field says what sort of name this is: see + <a href="#sort">NameSort below</a>. + <li> The <code>n_occ</code> field gives the "occurrence name" of the + Name; see + <a href="#occname">OccName below</a>. + <li> The <code>n_uniq</code> field allows fast tests for equality of + Names. + <li> The <code>n_loc</code> field gives some indication of where the + name was bound. + </ul> + + <h2><a name="sort">The <code>NameSort</code> of a <code>Name</code></a></h2> + <p> + There are four flavours of <code>Name</code>: + </p> + <blockquote> + <pre> +data NameSort + = External Module (Maybe Name) + -- (Just parent) => this Name is a subordinate name of 'parent' + -- e.g. data constructor of a data type, method of a class + -- Nothing => not a subordinate + + | WiredIn Module (Maybe Name) TyThing BuiltInSyntax + -- A variant of External, for wired-in things + + | Internal -- A user-defined Id or TyVar + -- defined in the module being compiled + + | System -- A system-defined Id or TyVar. Typically the + -- OccName is very uninformative (like 's')</pre> + </blockquote> + <ul> + <li>Here are the sorts of Name an entity can have: + <ul> + <li> Class, TyCon: External. + <li> Id: External, Internal, or System. + <li> TyVar: Internal, or System. + </ul> + </li> + <li>An <code>External</code> name has a globally-unique + (module name, occurrence name) pair, namely the + <em>original name</em> of the entity, + describing where the thing was originally defined. So for example, + if we have + <blockquote> + <pre> +module M where + f = e1 + g = e2 + +module A where + import qualified M as Q + import M + a = Q.f + g</pre> + </blockquote> + <p> + then the RdrNames for "a", "Q.f" and "g" get replaced (by the + Renamer) by the Names "A.a", "M.f", and "M.g" respectively. + </p> + </li> + <li>An <code>InternalName</code> + has only an occurrence name. Distinct InternalNames may have the same + occurrence name; use the Unique to distinguish them. + </li> + <li>An <code>ExternalName</code> has a unique that never changes. It + is never cloned. This is important, because the simplifier invents + new names pretty freely, but we don't want to lose the connnection + with the type environment (constructed earlier). An + <code>InternalName</code> name can be cloned freely. + </li> + <li><strong>Before CoreTidy</strong>: the Ids that were defined at top + level in the original source program get <code>ExternalNames</code>, + whereas extra top-level bindings generated (say) by the type checker + get <code>InternalNames</code>. q This distinction is occasionally + useful for filtering diagnostic output; e.g. for -ddump-types. + </li> + <li><strong>After CoreTidy</strong>: An Id with an + <code>ExternalName</code> will generate symbols that + appear as external symbols in the object file. An Id with an + <code>InternalName</code> cannot be referenced from outside the + module, and so generates a local symbol in the object file. The + CoreTidy pass makes the decision about which names should be External + and which Internal. + </li> + <li>A <code>System</code> name is for the most part the same as an + <code>Internal</code>. Indeed, the differences are purely cosmetic: + <ul> + <li>Internal names usually come from some name the + user wrote, whereas a System name has an OccName like "a", or "t". + Usually there are masses of System names with the same OccName but + different uniques, whereas typically there are only a handful of + distince Internal names with the same OccName. + </li> + <li>Another difference is that when unifying the type checker tries + to unify away type variables with System names, leaving ones with + Internal names (to improve error messages). + </li> + </ul> + </li> + </ul> + + <h2><a name="occname">Occurrence names: <code>OccName</code></a></h2> + <p> + An <code>OccName</code> is more-or-less just a string, like "foo" or + "Tree", giving the (unqualified) name of an entity. + </p> + <p> + Well, not quite just a string, because in Haskell a name like "C" could + mean a type constructor or data constructor, depending on context. So + GHC defines a type <tt>OccName</tt> (defined in + <tt>basicTypes/OccName.lhs</tt>) that is a pair of a <tt>FastString</tt> + and a <tt>NameSpace</tt> indicating which name space the name is drawn + from: + <blockquote> + <pre> +data OccName = OccName NameSpace EncodedFS</pre> + </blockquote> + <p> + The <tt>EncodedFS</tt> is a synonym for <tt>FastString</tt> indicating + that the string is Z-encoded. (Details in <tt>OccName.lhs</tt>.) + Z-encoding encodes funny characters like '%' and '$' into alphabetic + characters, like "zp" and "zd", so that they can be used in object-file + symbol tables without confusing linkers and suchlike. + </p> + <p> + The name spaces are: + </p> + <ul> + <li> <tt>VarName</tt>: ordinary variables</li> + <li> <tt>TvName</tt>: type variables</li> + <li> <tt>DataName</tt>: data constructors</li> + <li> <tt>TcClsName</tt>: type constructors and classes (in Haskell they + share a name space) </li> + </ul> + + <small> +<!-- hhmts start --> +Last modified: Wed May 4 14:57:55 EST 2005 +<!-- hhmts end --> + </small> + </body> +</html> + diff --git a/docs/comm/the-beast/ncg.html b/docs/comm/the-beast/ncg.html new file mode 100644 index 0000000000..5810a35212 --- /dev/null +++ b/docs/comm/the-beast/ncg.html @@ -0,0 +1,749 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Native Code Generator</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Native Code Generator</h1> + <p> + On some platforms (currently x86 and PowerPC, with bitrotted + support for Sparc and Alpha), GHC can generate assembly code + directly, without having to go via C. This can sometimes almost + halve compilation time, and avoids the fragility and + horribleness of the <a href="mangler.html">mangler</a>. The NCG + is enabled by default for + non-optimising compilation on supported platforms. For most programs + it generates code which runs only 1-3% slower + (depending on platform and type of code) than that + created by gcc on x86s, so it is well worth using even with + optimised compilation. FP-intensive x86 programs see a bigger + slowdown, and all Sparc code runs about 5% slower due to + us not filling branch delay slots. + <p> + The NCG has always been something of a second-class citizen + inside GHC, an unloved child, rather. This means that its + integration into the compiler as a whole is rather clumsy, which + brings some problems described below. That apart, the NCG + proper is fairly cleanly designed, as target-independent as it + reasonably can be, and so should not be difficult to retarget. + <p> + <b>NOTE!</b> The native code generator was largely rewritten as part + of the C-- backend changes, around May 2004. Unfortunately the + rest of this document still refers to the old version, and was written + with relation to the CVS head as of end-Jan 2002. Some of it is relevant, + some of it isn't. + + <h2>Overview</h2> + The top-level code generator fn is + <p> + <code>absCtoNat :: AbstractC -> UniqSM (SDoc, Pretty.Doc)</code> + <p> + The returned <code>SDoc</code> is for debugging, so is empty unless + you specify <code>-ddump-stix</code>. The <code>Pretty.Doc</code> + bit is the final assembly code. Translation involves three main + phases, the first and third of which are target-independent. + <ul> + <li><b>Translation into the <code>Stix</code> representation.</b> Stix + is a simple tree-like RTL-style language, in which you can + mention: + <p> + <ul> + <li>An infinite number of temporary, virtual registers. + <li>The STG "magic" registers (<code>MagicId</code>), such as + the heap and stack pointers. + <li>Literals and low-level machine ops (<code>MachOp</code>). + <li>Simple address computations. + <li>Reads and writes of: memory, virtual regs, and various STG + regs. + <li>Labels and <code>if ... goto ...</code> style control-flow. + </ul> + <p> + Stix has two main associated types: + <p> + <ul> + <li><code>StixStmt</code> -- trees executed for their side + effects: assignments, control transfers, and auxiliary junk + such as segment changes and literal data. + <li><code>StixExpr</code> -- trees which denote a value. + </ul> + <p> + Translation into Stix is almost completely + target-independent. Needed dependencies are knowledge of + word size and endianness, used when generating code to do + deal with half-word fields in info tables. This could be + abstracted out easily enough. Also, the Stix translation + needs to know which <code>MagicId</code>s map to registers + on the given target, and which are stored in offsets from + <code>BaseReg</code>. + <p> + After initial Stix generation, the trees are cleaned up with + constant-folding and a little copy-propagation ("Stix + inlining", as the code misleadingly calls it). We take + the opportunity to translate <code>MagicId</code>s which are + stored in memory on the given target, into suitable memory + references. Those which are stored in registers are left + alone. There is also a half-hearted attempt to lift literal + strings to the top level in cases where nested strings have + been observed to give incorrect code in the past. + <p> + Primitive machine-level operations will already be phrased in + terms of <code>MachOp</code>s in the presented Abstract C, and + these are passed through unchanged. We comment only that the + <code>MachOp</code>s have been chosen so as to be easy to + implement on all targets, and their meaning is intended to be + unambiguous, and the same on all targets, regardless of word + size or endianness. + <p> + <b>A note on <code>MagicId</code>s.</b> + Those which are assigned to + registers on the current target are left unmodified. Those + which are not are stored in memory as offsets from + <code>BaseReg</code> (which is assumed to permanently have the + value <code>(&MainCapability.r)</code>), so the constant folder + calculates the offsets and inserts suitable loads/stores. One + complication is that not all archs have <code>BaseReg</code> + itself in a register, so for those (sparc), we instead + generate the address as an offset from the static symbol + <code>MainCapability</code>, since the register table lives in + there. + <p> + Finally, <code>BaseReg</code> does occasionally itself get + mentioned in Stix expression trees, and in this case what is + denoted is precisely <code>(&MainCapability.r)</code>, not, as + in all other cases, the value of memory at some offset from + the start of the register table. Since what it denotes is an + r-value and not an l-value, assigning <code>BaseReg</code> is + meaningless, so the machinery checks to ensure this never + happens. All these details are taken into account by the + constant folder. + <p> + <li><b>Instruction selection.</b> This is the only majorly + target-specific phase. It turns Stix statements and + expressions into sequences of <code>Instr</code>, a data + type which is different for each architecture. + <code>Instr</code>, unsurprisingly, has various supporting + types, such as <code>Reg</code>, <code>Operand</code>, + <code>Imm</code>, etc. The generated instructions may refer + to specific machine registers, or to arbitrary virtual + registers, either those created within the instruction + selector, or those mentioned in the Stix passed to it. + <p> + The instruction selectors live in <code>MachCode.lhs</code>. + The core functions, for each target, are: + <p> + <code> + getAmode :: StixExpr -> NatM Amode + <br>getRegister :: StixExpr -> NatM Register + <br>assignMem_IntCode :: PrimRep -> StixExpr -> StixExpr -> NatM InstrBlock + <br>assignReg_IntCode :: PrimRep -> StixReg -> StixExpr -> NatM InstrBlock + </code> + <p> + The insn selectors use the "maximal munch" algorithm. The + bizarrely-misnamed <code>getRegister</code> translates + expressions. A simplified version of its type is: + <p> + <code>getRegister :: StixExpr -> NatM (OrdList Instr, Reg)</code> + <p> + That is: it (monadically) turns a <code>StixExpr</code> into a + sequence of instructions, and a register, with the meaning + that after executing the (possibly empty) sequence of + instructions, the (possibly virtual) register will + hold the resulting value. The real situation is complicated + by the presence of fixed registers, and is detailed below. + <p> + Maximal munch is a greedy algorithm and is known not to give + globally optimal code sequences, but it is good enough, and + fast and simple. Early incarnations of the NCG used something + more sophisticated, but that is long gone now. + <p> + Similarly, <code>getAmode</code> translates a value, intended + to denote an address, into a sequence of insns leading up to + a (processor-specific) addressing mode. This stuff could be + done using the general <code>getRegister</code> selector, but + would necessarily generate poorer code, because the calculated + address would be forced into a register, which might be + unnecessary if it could partially or wholly be calculated + using an addressing mode. + <p> + Finally, <code>assignMem_IntCode</code> and + <code>assignReg_IntCode</code> create instruction sequences to + calculate a value and store it in the given register, or at + the given address. Because these guys translate a statement, + not a value, they just return a sequence of insns and no + associated register. Floating-point and 64-bit integer + assignments have analogous selectors. + <p> + Apart from the complexities of fixed vs floating registers, + discussed below, the instruction selector is as simple + as it can be. It looks long and scary but detailed + examination reveals it to be fairly straightforward. + <p> + <li><b>Register allocation.</b> The register allocator, + <code>AsmRegAlloc.lhs</code> takes sequences of + <code>Instr</code>s which mention a mixture of real and + virtual registers, and returns a modified sequence referring + only to real ones. It is gloriously and entirely + target-independent. Well, not exactly true. Instead it + regards <code>Instr</code> (instructions) and <code>Reg</code> + (virtual and real registers) as abstract types, to which it has + the following interface: + <p> + <code> + insnFuture :: Instr -> InsnFuture + <br>regUsage :: Instr -> RegUsage + <br>patchRegs :: Instr -> (Reg -> Reg) -> Instr + </code> + <p> + <code>insnFuture</code> is used to (re)construct the graph of + all possible control transfers between the insns to be + allocated. <code>regUsage</code> returns the sets of registers + read and written by an instruction. And + <code>patchRegs</code> is used to apply the allocator's final + decision on virtual-to-real reg mapping to an instruction. + <p> + Clearly these 3 fns have to be written anew for each + architecture. They are defined in + <code>RegAllocInfo.lhs</code>. Think twice, no, thrice, + before modifying them: making false claims about insn + behaviour will lead to hard-to-find register allocation + errors. + <p> + <code>AsmRegAlloc.lhs</code> contains detailed comments about + how the allocator works. Here is a summary. The head honcho + <p> + <code>allocUsingTheseRegs :: [Instr] -> [Reg] -> (Bool, [Instr])</code> + <p> + takes a list of instructions and a list of real registers + available for allocation, and maps as many of the virtual regs + in the input into real ones as it can. The returned + <code>Bool</code> indicates whether or not it was + successful. If so, that's the end of it. If not, the caller + of <code>allocUsingTheseRegs</code> will attempt spilling. + More of that later. What <code>allocUsingTheseRegs</code> + does is: + <p> + <ul> + <li>Implicitly number each instruction by its position in the + input list. + <p> + <li>Using <code>insnFuture</code>, create the set of all flow + edges -- possible control transfers -- within this set of + insns. + <p> + <li>Using <code>regUsage</code> and iterating around the flow + graph from the previous step, calculate, for each virtual + register, the set of flow edges on which it is live. + <p> + <li>Make a real-register committment map, which gives the set + of edges for which each real register is committed (in + use). These sets are initially empty. For each virtual + register, attempt to find a real register whose current + committment does not intersect that of the virtual + register -- ie, is uncommitted on all edges that the + virtual reg is live. If successful, this means the vreg + can be assigned to the realreg, so add the vreg's set to + the realreg's committment. + <p> + <li>If all the vregs were assigned to a realreg, use + <code>patchInstr</code> to apply the mapping to the insns themselves. + </ul> + <p> + <b>Spilling</b> + <p> + If <code>allocUsingTheseRegs</code> fails, a baroque + mechanism comes into play. We now know that much simpler + schemes are available to do the same thing and give better + results. + Anyways: + <p> + The logic above <code>allocUsingTheseRegs</code>, in + <code>doGeneralAlloc</code> and <code>runRegAllocate</code>, + observe that allocation has failed with some set R of real + registers. So they apply <code>runRegAllocate</code> a second + time to the code, but remove (typically) two registers from R + before doing so. This naturally fails too, but returns a + partially-allocated sequence. <code>doGeneralAlloc</code> + then inserts spill code into the sequence, and finally re-runs + <code>allocUsingTheseRegs</code>, but supplying the original, + unadulterated R. This is guaranteed to succeed since the two + registers previously removed from R are sufficient to allocate + all the spill/restore instructions added. + <p> + Because x86 is very short of registers, and in the worst case + needs three removed from R, a softly-softly approach is used. + <code>doGeneralAlloc</code> first tries with zero regs removed + from R, then if that fails one, then two, etc. This means + <code>allocUsingTheseRegs</code> may get run several times + before a successful arrangement is arrived at. + <code>findReservedRegs</code> cooks up the sets of spill + registers to try with. + <p> + The resulting machinery is complicated and the generated spill + code is appalling. The saving grace is that spills are very + rare so it doesn't matter much. I did not invent this -- I inherited it. + <p> + <b>Dealing with common cases fast</b> + <p> + The entire reg-alloc mechanism described so far is general and + correct, but expensive overkill for many simple code blocks. + So to begin with we use + <code>doSimpleAlloc</code>, which attempts to do something + simple. It exploits the observation that if the total number + of virtual registers does not exceed the number of real ones + available, we can simply dole out a new realreg each time we + see mention of a new vreg, with no regard for control flow. + <code>doSimpleAlloc</code> therefore attempts this in a + single pass over the code. It gives up if it runs out of real + regs or sees any condition which renders the above observation + invalid (fixed reg uses, for example). + <p> + This clever hack handles the majority of code blocks quickly. + It was copied from the previous reg-allocator (the + Mattson/Partain/Marlow/Gill one). + </ul> + +<p> +<h2>Complications, observations, and possible improvements</h2> + +<h3>Real vs virtual registers in the instruction selectors</h3> + +The instruction selectors for expression trees, namely +<code>getRegister</code>, are complicated by the fact that some +expressions can only be computed into a specific register, whereas +the majority can be computed into any register. We take x86 as an +example, but the problem applies to all archs. +<p> +Terminology: <em>rreg</em> means real register, a real machine +register. <em>vreg</em> means one of an infinite set of virtual +registers. The type <code>Reg</code> is the sum of <em>rreg</em> and +<em>vreg</em>. The instruction selector generates sequences with +unconstrained use of vregs, leaving the register allocator to map them +all into rregs. +<p> +Now, where was I ? Oh yes. We return to the type of +<code>getRegister</code>, which despite its name, selects instructions +to compute the value of an expression tree. +<pre> + getRegister :: StixExpr -> NatM Register + + data Register + = Fixed PrimRep Reg InstrBlock + | Any PrimRep (Reg -> InstrBlock) + + type InstrBlock -- sequence of instructions +</pre> +At first this looks eminently reasonable (apart from the stupid +name). <code>getRegister</code>, and nobody else, knows whether or +not a given expression has to be computed into a fixed rreg or can be +computed into any rreg or vreg. In the first case, it returns +<code>Fixed</code> and indicates which rreg the result is in. In the +second case it defers committing to any specific target register by +returning a function from <code>Reg</code> to <code>InstrBlock</code>, +and the caller can specify the target reg as it sees fit. +<p> +Unfortunately, that forces <code>getRegister</code>'s callers (usually +itself) to use a clumsy and confusing idiom in the common case where +they do not care what register the result winds up in. The reason is +that although a value might be computed into a fixed rreg, we are +forbidden (on pain of segmentation fault :) from subsequently +modifying the fixed reg. This and other rules are record in "Rules of +the game" inside <code>MachCode.lhs</code>. +<p> +Why can't fixed registers be modified post-hoc? Consider a simple +expression like <code>Hp+1</code>. Since the heap pointer +<code>Hp</code> is definitely in a fixed register, call it R, +<code>getRegister</code> on subterm <code>Hp</code> will simply return +<code>Fixed</code> with an empty sequence and R. But we can't just +emit an increment instruction for R, because that trashes +<code>Hp</code>; instead we first have to copy it into a fresh vreg +and increment that. +<p> +With all that in mind, consider now writing a <code>getRegister</code> +clause for terms of the form <code>(1 + E)</code>. Contrived, yes, +but illustrates the matter. First we do +<code>getRegister</code> on E. Now we are forced to examine what +comes back. +<pre> + getRegister (OnePlus e) + = getRegister e `thenNat` \ e_result -> + case e_result of + Fixed e_code e_fixed + -> returnNat (Any IntRep (\dst -> e_code ++ [MOV e_fixed dst, INC dst])) + Any e_any + -> Any (\dst -> e_any dst ++ [INC dst]) +</pre> +This seems unreasonably cumbersome, yet the instruction selector is +full of such idioms. A good example of the complexities induced by +this scheme is shown by <code>trivialCode</code> for x86 in +<code>MachCode.lhs</code>. This deals with general integer dyadic +operations on x86 and has numerous cases. It was difficult to get +right. +<p> +An alternative suggestion is to simplify the type of +<code>getRegister</code> to this: +<pre> + getRegister :: StixExpr -> NatM (InstrBloc, VReg) + type VReg = .... a vreg ... +</pre> +and then we could safely write +<pre> + getRegister (OnePlus e) + = getRegister e `thenNat` \ (e_code, e_vreg) -> + returnNat (e_code ++ [INC e_vreg], e_vreg) +</pre> +which is about as straightforward as you could hope for. +Unfortunately, it requires <code>getRegister</code> to insert moves of +values which naturally compute into an rreg, into a vreg. Consider: +<pre> + 1 + ccall some-C-fn +</pre> +On x86 the ccall result is returned in rreg <code>%eax</code>. The +resulting sequence, prior to register allocation, would be: +<pre> + # push args + call some-C-fn + # move %esp to nuke args + movl %eax, %vreg + incl %vreg +</pre> +If, as is likely, <code>%eax</code> is not held live beyond this point +for any other purpose, the move into a fresh register is pointless; +we'd have been better off leaving the value in <code>%eax</code> as +long as possible. +<p> +The simplified <code>getRegister</code> story is attractive. It would +clean up the instruction selectors significantly and make it simpler +to write new ones. The only drawback is that it generates redundant +register moves. I suggest that eliminating these should be the job +of the register allocator. Indeed: +<ul> +<li>There has been some work on this already ("Iterated register + coalescing" ?), so this isn't a new idea. +<p> +<li>You could argue that the existing scheme inappropriately blurs the + boundary between the instruction selector and the register + allocator. The instruction selector should .. well .. just + select instructions, without having to futz around worrying about + what kind of registers subtrees get generated into. Register + allocation should be <em>entirely</em> the domain of the register + allocator, with the proviso that it should endeavour to allocate + registers so as to minimise the number of non-redundant reg-reg + moves in the final output. +</ul> + + +<h3>Selecting insns for 64-bit values/loads/stores on 32-bit platforms</h3> + +Note that this stuff doesn't apply on 64-bit archs, since the +<code>getRegister</code> mechanism applies there. + +The relevant functions are: +<pre> + assignMem_I64Code :: StixExpr -> StixExpr -> NatM InstrBlock + assignReg_I64Code :: StixReg -> StixExpr -> NatM InstrBlock + iselExpr64 :: StixExpr -> NatM ChildCode64 + + data ChildCode64 -- a.k.a "Register64" + = ChildCode64 + InstrBlock -- code + VRegUnique -- unique for the lower 32-bit temporary +</pre> +<code>iselExpr64</code> is the 64-bit, plausibly-named analogue of +<code>getRegister</code>, and <code>ChildCode64</code> is the analogue +of <code>Register</code>. The aim here was to generate working 64 +bit code as simply as possible. To this end, I used the +simplified <code>getRegister</code> scheme described above, in which +<code>iselExpr64</code>generates its results into two vregs which +can always safely be modified afterwards. +<p> +Virtual registers are, unsurprisingly, distinguished by their +<code>Unique</code>s. There is a small difficulty in how to +know what the vreg for the upper 32 bits of a value is, given the vreg +for the lower 32 bits. The simple solution adopted is to say that +any low-32 vreg may also have a hi-32 counterpart which shares the +same unique, but is otherwise regarded as a separate entity. +<code>getHiVRegFromLo</code> gets one from the other. +<pre> + data VRegUnique + = VRegUniqueLo Unique -- lower part of a split quantity + | VRegUniqueHi Unique -- upper part thereof +</pre> +Apart from that, 64-bit code generation is really simple. The sparc +and x86 versions are almost copy-n-pastes of each other, with minor +adjustments for endianness. The generated code isn't wonderful but +is certainly acceptable, and it works. + + + +<h3>Shortcomings and inefficiencies in the register allocator</h3> + +<h4>Redundant reconstruction of the control flow graph</h4> + +The allocator goes to considerable computational expense to construct +all the flow edges in the group of instructions it's allocating for, +by using the <code>insnFuture</code> function in the +<code>Instr</code> pseudo-abstract type. +<p> +This is really silly, because all that information is present at the +abstract C stage, but is thrown away in the translation to Stix. +So a good thing to do is to modify that translation to +produce a directed graph of Stix straight-line code blocks, +and to preserve that structure through the insn selector, so the +allocator can see it. +<p> +This would eliminate the fragile, hacky, arch-specific +<code>insnFuture</code> mechanism, and probably make the whole +compiler run measurably faster. Register allocation is a fair chunk +of the time of non-optimising compilation (10% or more), and +reconstructing the flow graph is an expensive part of reg-alloc. +It would probably accelerate the vreg liveness computation too. + +<h4>Really ridiculous method for doing spilling</h4> + +This is a more ambitious suggestion, but ... reg-alloc should be +reimplemented, using the scheme described in "Quality and speed in +linear-scan register allocation." (Traub?) For straight-line code +blocks, this gives an elegant one-pass algorithm for assigning +registers and creating the minimal necessary spill code, without the +need for reserving spill registers ahead of time. +<p> +I tried it in Rigr, replacing the previous spiller which used the +current GHC scheme described above, and it cut the number of spill +loads and stores by a factor of eight. Not to mention being simpler, +easier to understand and very fast. +<p> +The Traub paper also describes how to extend their method to multiple +basic blocks, which will be needed for GHC. It comes down to +reconciling multiple vreg-to-rreg mappings at points where control +flow merges. + +<h4>Redundant-move support for revised instruction selector suggestion</h4> + +As mentioned above, simplifying the instruction selector will require +the register allocator to try and allocate source and destination +vregs to the same rreg in reg-reg moves, so as to make as many as +possible go away. Without that, the revised insn selector would +generate worse code than at present. I know this stuff has been done +but know nothing about it. The Linear-scan reg-alloc paper mentioned +above does indeed mention a bit about it in the context of single +basic blocks, but I don't know if that's sufficient. + + + +<h3>x86 arcana that you should know about</h3> + +The main difficulty with x86 is that many instructions have fixed +register constraints, which can occasionally make reg-alloc fail +completely. And the FPU doesn't have the flat register model which +the reg-alloc abstraction (implicitly) assumes. +<p> +Our strategy is: do a good job for the common small subset, that is +integer loads, stores, address calculations, basic ALU ops (+, -, +and, or, xor), and jumps. That covers the vast majority of +executed insns. And indeed we do do a good job, with a loss of +less than 2% compared with gcc. +<p> +Initially we tried to handle integer instructions with awkward +register constraints (mul, div, shifts by non-constant amounts) via +various jigglings of the spiller et al. This never worked robustly, +and putting platform-specific tweaks in the generic infrastructure is +a big No-No. (Not quite true; shifts by a non-constant amount are +still done by a giant kludge, and should be moved into this new +framework.) +<p> +Fortunately, all such insns are rare. So the current scheme is to +pretend that they don't have any such constraints. This fiction is +carried all the way through the register allocator. When the insn +finally comes to be printed, we emit a sequence which copies the +operands through memory (<code>%esp</code>-relative), satisfying the +constraints of the real instruction. This localises the gruesomeness +to just one place. Here, for example, is the code generated for +integer divison of <code>%esi</code> by <code>%ecx</code>: +<pre> + # BEGIN IQUOT %ecx, %esi + pushl $0 + pushl %eax + pushl %edx + pushl %ecx + movl %esi,% eax + cltd + idivl 0(%esp) + movl %eax, 12(%esp) + popl %edx + popl %edx + popl %eax + popl %esi + # END IQUOT %ecx, %esi +</pre> +This is not quite as appalling as it seems, if you consider that the +division itself typically takes 16+ cycles, whereas the rest of the +insns probably go through in about 1 cycle each. +<p> +This trick is taken to extremes for FP operations. +<p> +All notions of the x86 FP stack and its insns have been removed. +Instead, we pretend, to the instruction selector and register +allocator, that x86 has six floating point registers, +<code>%fake0</code> .. <code>%fake5</code>, which can be used in the +usual flat manner. We further claim that x86 has floating point +instructions very similar to SPARC and Alpha, that is, a simple +3-operand register-register arrangement. Code generation and register +allocation proceed on this basis. +<p> +When we come to print out the final assembly, our convenient fiction +is converted to dismal reality. Each fake instruction is +independently converted to a series of real x86 instructions. +<code>%fake0</code> .. <code>%fake5</code> are mapped to +<code>%st(0)</code> .. <code>%st(5)</code>. To do reg-reg arithmetic +operations, the two operands are pushed onto the top of the FP stack, +the operation done, and the result copied back into the relevant +register. When one of the operands is also the destination, we emit a +slightly less scummy translation. There are only six +<code>%fake</code> registers because 2 are needed for the translation, +and x86 has 8 in total. +<p> +The translation is inefficient but is simple and it works. A cleverer +translation would handle a sequence of insns, simulating the FP stack +contents, would not impose a fixed mapping from <code>%fake</code> to +<code>%st</code> regs, and hopefully could avoid most of the redundant +reg-reg moves of the current translation. +<p> +There are, however, two unforeseen bad side effects: +<ul> +<li>This doesn't work properly, because it doesn't observe the normal + conventions for x86 FP code generation. It turns out that each of + the 8 elements in the x86 FP register stack has a tag bit which + indicates whether or not that register is notionally in use or + not. If you do a FPU operation which happens to read a + tagged-as-empty register, you get an x87 FPU (stack invalid) + exception, which is normally handled by the FPU without passing it + to the OS: the program keeps going, but the resulting FP values + are garbage. The OS can ask for the FPU to pass it FP + stack-invalid exceptions, but it usually doesn't. + <p> + Anyways: inside NCG created x86 FP code this all works fine. + However, the NCG's fiction of a flat register set does not operate + the x87 register stack in the required stack-like way. When + control returns to a gcc-generated world, the stack tag bits soon + cause stack exceptions, and thus garbage results. + <p> + The only fix I could think of -- and it is horrible -- is to clear + all the tag bits just before the next STG-level entry, in chunks + of code which use FP insns. <code>i386_insert_ffrees</code> + inserts the relevant <code>ffree</code> insns into such code + blocks. It depends critically on <code>is_G_instr</code> to + detect such blocks. +<p> +<li>It's very difficult to read the generated assembly and + reason about it when debugging, because there's so much clutter. + We print the fake insns as comments in the output, and that helps + a bit. +</ul> + + + +<h3>Generating code for ccalls</h3> + +For reasons I don't really understand, the instruction selectors for +generating calls to C (<code>genCCall</code>) have proven surprisingly +difficult to get right, and soaked up a lot of debugging time. As a +result, I have once again opted for schemes which are simple and not +too difficult to argue as correct, even if they don't generate +excellent code. +<p> +The sparc ccall generator in particular forces all arguments into +temporary virtual registers before moving them to the final +out-registers (<code>%o0</code> .. <code>%o5</code>). This creates +some unnecessary reg-reg moves. The reason is explained in a +comment in the code. + + +<h3>Duplicate implementation for many STG macros</h3> + +This has been discussed at length already. It has caused a couple of +nasty bugs due to subtle untracked divergence in the macro +translations. The macro-expander really should be pushed up into the +Abstract C phase, so the problem can't happen. +<p> +Doing so would have the added benefit that the NCG could be used to +compile more "ways" -- well, at least the 'p' profiling way. + + +<h3>How to debug the NCG without losing your sanity/hair/cool</h3> + +Last, but definitely not least ... +<p> +The usual syndrome is that some program, when compiled via C, works, +but not when compiled via the NCG. Usually the problem is fairly +simple to fix, once you find the specific code block which has been +mistranslated. But the latter can be nearly impossible, since most +modules generate at least hundreds and often thousands of them. +<p> +My solution: cheat. +<p> +Because the via-C and native routes diverge only late in the day, +it is not difficult to construct a 1-1 correspondence between basic +blocks on the two routes. So, if the program works via C but not on +the NCG, do the following: +<ul> +<li>Recompile <code>AsmCodeGen.lhs</code> in the afflicted compiler + with <code>-DDEBUG_NCG</code>, so that it inserts + <code>___ncg_debug_marker</code>s + into the assembly it emits. +<p> +<li>Using a binary search on modules, find the module which is causing + the problem. +<p> +<li>Compile that module to assembly code, with identical flags, twice, + once via C and once via NCG. + Call the outputs <code>ModuleName.s-gcc</code> and + <code>ModuleName.s-nat</code>. Check that the latter does indeed have + <code>___ncg_debug_marker</code>s in it; otherwise the next steps fail. +<p> +<li>Build (with a working compiler) the program + <code>fptools/ghc/utils/debugNCG/diff_gcc_nat</code>. +<p> +<li>Run: <code>diff_gcc_nat ModuleName.s</code>. This will + construct the 1-1 correspondence, and emits on stdout + a cppable assembly output. Place this in a file -- I always + call it <code>synth.S</code>. Note, the capital S is important; + otherwise it won't get cpp'd. You can feed this file directly to + ghc and it will automatically get cpp'd; you don't have to do so + yourself. +<p> +<li>By messing with the <code>#define</code>s at the top of + <code>synth.S</code>, do a binary search to find the incorrect + block. Keep a careful record of where you are in the search; it + is easy to get confused. Remember also that multiple blocks may + be wrong, which also confuses matters. Finally, I usually start + off by re-checking that I can build the executable with all the + <code>#define</code>s set to 0 and then all to 1. This ensures + you won't get halfway through the search and then get stuck due to + some snafu with gcc-specific literals. Usually I set + <code>UNMATCHED_GCC</code> to 1 all the time, and this bit should + contain only literal data. + <code>UNMATCHED_NAT</code> should be empty. +</ul> +<p> +<code>diff_gcc_nat</code> was known to work correctly last time I used +it, in December 01, for both x86 and sparc. If it doesn't work, due +to changes in assembly syntax, or whatever, make it work. The +investment is well worth it. Searching for the incorrect block(s) any +other way is a total time waster. + + + +</ul> + + + + + <p><small> +<!-- hhmts start --> +Last modified: Fri Feb 1 16:14:11 GMT 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/optimistic.html b/docs/comm/the-beast/optimistic.html new file mode 100644 index 0000000000..4d158022e8 --- /dev/null +++ b/docs/comm/the-beast/optimistic.html @@ -0,0 +1,65 @@ +<h2> Architectural stuff </h2> + +New fields in the TSO: +<ul> +<li> New global speculation-depth register; always counts the number of specuation frames +on the stack; incremented when +starting speculation, decremented when finishing. +<li> Profiling stuff +</ul> + + +<h2> Speculation frames </h2> + +The info table for a speculation frame points to the static spec-depth configuration +for that speculation point. (Points to, because the config is mutable, and the info +table has to be adjacent to the (immutable) code.) + + + +<h2> Abortion</h2> + +Abortion is modelled by a special asynchronous exception ThreadAbort. + +<ul> +<li> In the scheduler, if a thread returns with ThreadBlocked, and non-zero SpecDepth, send it +an asynchronous exception. + +<li> In the implementation of the <tt>catch#</tt> primop, raise an asynchonous exception if +SpecDepth is nonzero. + +<li> Timeout, administered by scheduler. Current story: abort if a speculation frame lasts from +one minor GC to the next. We detect this by seeing if there's a profiling frame on the stack --- a +profiling frame is added at a minor GC in place of a speculation frame (see Online Profiling). +</ul> + + +When tearing frames off the stack, we start a new chunk at every speculation frame, as well as every +update frame. We proceed down to the deepest speculation frame. +<p> +The <tt>AP_STACK</tt> closure built for a speculation frame must be careful <em>not</em> to enter the +next <tt>AP_STACK</tt> closure up, because that would re-enter a possible loop. +<p> +Delivering an asynch exception to a thread that is speculating. Invariant: there can be no catch frames +inside speculation (we abort in <tt>catch#</tt> when speculating. So the asynch exception just +tears off frames in the standard way until it gets to a catch frame, just as it would usually do. +<p> +Abortion can punish one or more of the speculation frames by decrementing their static config variables. + +<h3>Synchronous exceptions</h3> + +Synchronous exceptions are treated similarly as before. The stack is discarded up to an update frame; the +thunk to be updated is overwritten with "raise x", and the process continues. Until a catch frame. +<p> +When we find a spec frame, we allocate a "raise x" object, and resume execution with the return address +in the spec frame. In that way the spec frame is like a catch frame; it stops the unwinding process. +<p> +It's essential that every hard failure is caught, else speculation is unsafe. In particular, divide by zero +is hard to catch using OS support, so we test explicitly in library code. You can shoot yourself in the foot +by writing <tt>x `div#` 0</tt>, side-stepping the test. + + +<h3> Online profiling </h3> + +Sampling can be more frequent than minor GC (by jiggling the end-of-block code) but cannot +be less frequent, because GC doesn't expect to see profiling frames.
\ No newline at end of file diff --git a/docs/comm/the-beast/prelude.html b/docs/comm/the-beast/prelude.html new file mode 100644 index 0000000000..64b607def5 --- /dev/null +++ b/docs/comm/the-beast/prelude.html @@ -0,0 +1,207 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Primitives and the Prelude</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Primitives and the Prelude</h1> + <p> + One of the trickiest aspects of GHC is the delicate interplay + between what knowledge is baked into the compiler, and what + knowledge it gets by reading the interface files of library + modules. In general, the less that is baked in, the better. +<p> + Most of what the compiler has to have wired in about primitives and + prelude definitions is in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/"><code>fptools/ghc/compiler/prelude/</code></a>. + </p> + +GHC recognises these main classes of baked-in-ness: +<dl> +<dt><strong>Primitive types.</strong> +<dd>Primitive types cannot be defined in Haskell, and are utterly baked into the compiler. +They are notionally defined in the fictional module <tt>GHC.Prim</tt>. The <tt>TyCon</tt>s for these types are all defined +in module <tt>TysPrim</tt>; for example, +<pre> + intPrimTyCon :: TyCon + intPrimTyCon = .... +</pre> +Examples: +<tt>Int#, Float#, Addr#, State#</tt>. +<p> +<dt><strong>Wired-in types.</strong> +<dd>Wired-in types can be defined in Haskell, and indeed are (many are defined in </tt>GHC.Base</tt>). +However, it's very convenient for GHC to be able to use the type constructor for (say) <tt>Int</tt> +without looking it up in any environment. So module <tt>TysWiredIn</tt> contains many definitions +like this one: +<pre> + intTyCon :: TyCon + intTyCon = .... + + intDataCon :: DataCon + intDataCon = .... +</pre> +However, since a <tt>TyCon</tt> value contains the entire type definition inside it, it follows +that the complete definition of <tt>Int</tt> is thereby baked into the compiler. +<p> +Nevertheless, the library module <tt>GHC.Base</tt> still contains a definition for <tt>Int</tt> +just so that its info table etc get generated somewhere. Chaos will result if the wired-in definition +in <tt>TysWiredIn</tt> differs from that in <tt>GHC.Base</tt>. +<p> +The rule is that only very simple types should be wired in (for example, <tt>Ratio</tt> is not, +and <tt>IO</tt> is certainly not). No class is wired in: classes are just too complicated. +<p> +Examples: <tt>Int</tt>, <tt>Float</tt>, <tt>List</tt>, tuples. + +<p> +<dt><strong>Known-key things.</strong> +<dd>GHC knows of the existence of many, many other types, classes and values. <em>But all it knows is +their <tt>Name</tt>.</em> Remember, a <tt>Name</tt> includes a unique key that identifies the +thing, plus its defining module and occurrence name +(see <a href="names.html">The truth about Names</a>). Knowing a <tt>Name</tt>, therefore, GHC can +run off to the interface file for the module and find out everything else it might need. +<p> +Most of these known-key names are defined in module <tt>PrelNames</tt>; a further swathe concerning +Template Haskell are defined in <tt>DsMeta</tt>. The allocation of unique keys is done manually; +chaotic things happen if you make a mistake here, which is why they are all together. +</dl> + +All the <tt>Name</tt>s from all the above categories are used to initialise the global name cache, +which maps (module,occurrence-name) pairs to the globally-unique <tt>Name</tt> for that +thing. (See <tt>HscMain.initOrigNames</tt>.) + +<p> +The next sections elaborate these three classes a bit. + + + <h2>Primitives (module <tt>TysPrim</tt>)</h2> + <p> + Some types and functions have to be hardwired into the compiler as they + are atomic; all other code is essentially built around this primitive + functionality. This includes basic arithmetic types, such as integers, + and their elementary operations as well as pointer types. Primitive + types and functions often receive special treatment in the code + generator, which means that these entities have to be explicitly + represented in the compiler. Moreover, many of these types receive some + explicit treatment in the runtime system, and so, there is some further + information about <a href="../rts-libs/primitives.html">primitives in + the RTS section</a> of this document. + <p> + The module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/TysPrim.lhs"><code>TysPrim</code></a> + exports a list of all primitive type constructors as <code>primTyCons :: + [TyCon]</code>. All of these type constructors (of type + <code>TyCon</code>) are also exported as <code>intPrimTyCon</code>, + <code>stablePtrPrimTyCon</code>, and so on. In addition, for each + nullary type constructor the corresponding type (of type + <code>Type</code>) is also exported; for example, we have + <code>intPrimTy :: Type</code>. For all other type constructors, a + function is exported that constructs the type obtained by applying the + type constructors to an argument type (of type <code>Type</code>); for + example, we have <code>mkStablePtrPrimTy :: Type -> Type</code>. + <p> + As it is inconvenient to identify type that receive a special treatment + by the code generator by looking at their name, the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/PrimRep.lhs"><code>PrimRep</code></a> + exports a data type <code>PrimRep</code>, which lists all + machine-manipulable implementation types. The module also exports a set + of query functions on <code>PrimRep</code> that define properties, such + as a type's byte size or whether a primitive type is a pointer type. + Moreover, the function <code>TysPrim.primRepTyCon :: PrimRep -> + TyCon</code> converts <code>PrimRep</code> values into the corresponding + type constructor. + + <h2>Wired in types (module <tt>TysWiredIn</tt>)</h2> + <p> + In addition to entities that are primitive, as the compiler has to treat + them specially in the backend, there is a set of types, functions, + etc. that the Haskell language definition flags as essential to the + language by placing them into the special module <code>Prelude</code> + that is implicitly imported into each Haskell module. For some of these + entities it suffices to define them (by standard Haskell definitions) in + a <code>Prelude</code> module and ensuring that this module is treated + specially by being always imported . + <p> + However, there is a set of entities (such as, for example, the list type + and the corresponding data constructors) that have an inbetween status: + They are not truly primitive (lists, for example, can easily be defined + by a <code>data</code> declaration), but the compiler has to have extra + knowledge about them, as they are associated with some particular + features of the language (in the case of lists, there is special syntax, + such as list comprehensions, associated with the type). Another + example, for a special kind of entity are type classes that can be used + in a <code>deriving</code> clause. All types that are not-primitive, + but about which the compiler nonetheless has to have some extra + knowledge are defined in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/TysWiredIn.lhs"><code>TysWiredIn</code></a>. + <p> + All wired in type constructors are contained in <code>wiredInTyCons :: + [TyCon]</code>. In addition to that list, <code>TysWiredIn</code> + exports variables bound to representations of all listed type + constructors and their data constructors. So, for example, we have + <code>listTyCon</code> together with <code>nilDataCon</cons> and + </code>consDataCon</code>. There are also convenience functions, such + as <code>mkListTy</code> and <code>mkTupleTy</code>, which construct + compound types. + <p> + + <h2>Known-key names (module <tt>PrelNames</tt>)</h2> + + All names of types, functions, etc. known to the compiler are defined in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/PrelNames.lhs"><code>PrelNames</code></a>. + This includes the names of types and functions exported from + <code>TysWiredIn</code>, but also others. In particular, this module + also fixes the names of all prelude modules; i.e., of the modules whose + name starts with <code>Prel</code>, which GHC's library uses to bring + some structure into the quite large number of <code>Prelude</code> + definitions. + <p> + <code>PrelNames.knownKeyNames :: [Name]</code> contains all names known + to the compiler, but the elements of the list are also exported + individually as variables, such as <code>floatTyConName</code> (having + the lexeme <code>Float</code>) and <code>floatDataConName</code> (having + the lexeme <code>F#</code>). For each of these names, + <code>PrelNames</code> derfines a unique key with a definition, such as + <p> +<blockquote><pre> +floatPrimTyConKey = mkPreludeTyConUnique 11</pre> +</blockquote> + <p> + that is, all unique keys for known prelude names are hardcoded into + <code>PrelNames</code> (and uniqueness has to be manually ensured in + that module). To simplify matching the types of important groups of + type constructors, <code>PrelNames</code> also exports lists, such as + <code>numericTyKeys</code> (keys of all numeric types), that contain the + unique keys of all names in that group. In addition, derivable type + classes and their structure is defined by + <code>derivableClassKeys</code> and related definitions. + <p> + In addition to names that have unique keys, <code>PrelNames</code> also + defines a set of names without uniqueness information. These names end + on the suffix <code>_RDR</code> and are of type <code>RdrName</code> (an + example, is <code>times_RDR</code>, which represents the lexeme + <code>*</code>). The names are used in locations where they pass + through the renamer anyway (e.g., special constructors encountered by + the parser, such as [], and code generated from deriving clauses), which + will take care of adding uniqueness information. + <p> + +<h2>Gathering it all together (module <tt>PrelInfo</tt>)</h2> + The module + <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/prelude/PrelInfo.lhs"><code>PrelInfo</code></a> + in some sense ties all the above together and provides a reasonably + restricted interface to these definition to the rest of the compiler. + However, from what I have seen, this doesn't quite work out and the + earlier mentioned modules are directly imported in many places. + + <p><small> +<!-- hhmts start --> +Last modified: Tue Dec 11 17:54:07 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/renamer.html b/docs/comm/the-beast/renamer.html new file mode 100644 index 0000000000..828b569bb9 --- /dev/null +++ b/docs/comm/the-beast/renamer.html @@ -0,0 +1,249 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Glorious Renamer</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Glorious Renamer</h1> + <p> + The <em>renamer</em> sits between the parser and the typechecker. + However, its operation is quite tightly interwoven with the + typechecker. This is partially due to support for Template Haskell, + where spliced code has to be renamed and type checked. In particular, + top-level splices lead to multiple rounds of renaming and type + checking. + </p> + <p> + The main externally used functions of the renamer are provided by the + module <code>rename/RnSource.lhs</code>. In particular, we have + </p> + <blockquote> + <pre> +rnSrcDecls :: HsGroup RdrName -> RnM (TcGblEnv, HsGroup Name) +rnTyClDecls :: [LTyClDecl RdrName] -> RnM [LTyClDecl Name] +rnSplice :: HsSplice RdrName -> RnM (HsSplice Name, FreeVars)</pre> + </blockquote> + <p> + All of which execute in the renamer monad <code>RnM</code>. The first + function, <code>rnSrcDecls</code> renames a binding group; the second, + <code>rnTyClDecls</code> renames a list of (toplevel) type and class + declarations; and the third, <code>rnSplice</code> renames a Template + Haskell splice. As the types indicate, the main task of the renamer is + to convert converts all the <tt>RdrNames</tt> to <a + href="names.html"><tt>Names</tt></a>, which includes a number of + well-formedness checks (no duplicate declarations, all names are in + scope, and so on). In addition, the renamer performs other, not + strictly name-related, well-formedness checks, which includes checking + that the appropriate flags have been supplied whenever language + extensions are used in the source. + </p> + + <h2>RdrNames</h2> + <p> + A <tt>RdrName.RdrName</tt> is pretty much just a string (for an + unqualified name like "<tt>f</tt>") or a pair of strings (for a + qualified name like "<tt>M.f</tt>"): + </p> + <blockquote> + <pre> +data RdrName + = Unqual OccName + -- Used for ordinary, unqualified occurrences + + | Qual Module OccName + -- A qualified name written by the user in + -- *source* code. The module isn't necessarily + -- the module where the thing is defined; + -- just the one from which it is imported + + | Orig Module OccName + -- An original name; the module is the *defining* module. + -- This is used when GHC generates code that will be fed + -- into the renamer (e.g. from deriving clauses), but where + -- we want to say "Use Prelude.map dammit". + + | Exact Name + -- We know exactly the Name. This is used + -- (a) when the parser parses built-in syntax like "[]" + -- and "(,)", but wants a RdrName from it + -- (b) when converting names to the RdrNames in IfaceTypes + -- Here an Exact RdrName always contains an External Name + -- (Internal Names are converted to simple Unquals) + -- (c) by Template Haskell, when TH has generated a unique name</pre> + </blockquote> + <p> + The OccName type is described in <a href="names.html#occname">The + truth about names</a>. + </p> + + <h2>The Renamer Monad</h2> + <p> + Due to the tight integration of the renamer with the typechecker, both + use the same monad in recent versions of GHC. So, we have + </p> + <blockquote> + <pre> +type RnM a = TcRn a -- Historical +type TcM a = TcRn a -- Historical</pre> + </blockquote> + <p> + with the combined monad defined as + </p> + <blockquote> + <pre> +type TcRn a = TcRnIf TcGblEnv TcLclEnv a +type TcRnIf a b c = IOEnv (Env a b) c + +data Env gbl lcl -- Changes as we move into an expression + = Env { + env_top :: HscEnv, -- Top-level stuff that never changes + -- Includes all info about imported things + + env_us :: TcRef UniqSupply, -- Unique supply for local varibles + + env_gbl :: gbl, -- Info about things defined at the top level + -- of the module being compiled + + env_lcl :: lcl -- Nested stuff; changes as we go into + -- an expression + }</pre> + </blockquote> + <p> + the details of the global environment type <code>TcGblEnv</code> and + local environment type <code>TcLclEnv</code> are also defined in the + module <code>typecheck/TcRnTypes.lhs</code>. The monad + <code>IOEnv</code> is defined in <code>utils/IOEnv.hs</code> and extends + the vanilla <code>IO</code> monad with an additional state parameter + <code>env</code> that is treated as in a reader monad. (Side effecting + operations, such as updating the unique supply, are done with + <code>TcRef</code>s, which are simply a synonym for <code>IORef</code>s.) + </p> + + <h2>Name Space Management</h2> + <p> + As anticipated by the variants <code>Orig</code> and <code>Exact</code> + of <code>RdrName</code> some names should not change during renaming, + whereas others need to be turned into unique names. In this context, + the two functions <code>RnEnv.newTopSrcBinder</code> and + <code>RnEnv.newLocals</code> are important: + </p> + <blockquote> + <pre> +newTopSrcBinder :: Module -> Maybe Name -> Located RdrName -> RnM Name +newLocalsRn :: [Located RdrName] -> RnM [Name]</pre> + </blockquote> + <p> + The two functions introduces new toplevel and new local names, + respectively, where the first two arguments to + <code>newTopSrcBinder</code> determine the currently compiled module and + the parent construct of the newly defined name. Both functions create + new names only for <code>RdrName</code>s that are neither exact nor + original. + </p> + + <h3>Introduction of Toplevel Names: Global RdrName Environment</h3> + <p> + A global <code>RdrName</code> environment + <code>RdrName.GlobalRdrEnv</code> is a map from <code>OccName</code>s to + lists of qualified names. More precisely, the latter are + <code>Name</code>s with an associated <code>Provenance</code>: + </p> + <blockquote> + <pre> +data Provenance + = LocalDef -- Defined locally + Module + + | Imported -- Imported + [ImportSpec] -- INVARIANT: non-empty + Bool -- True iff the thing was named *explicitly* + -- in *any* of the import specs rather than being + -- imported as part of a group; + -- e.g. + -- import B + -- import C( T(..) ) + -- Here, everything imported by B, and the constructors of T + -- are not named explicitly; only T is named explicitly. + -- This info is used when warning of unused names.</pre> + </blockquote> + <p> + The part of the global <code>RdrName</code> environment for a module + that contains the local definitions is created by the function + <code>RnNames.importsFromLocalDecls</code>, which also computes a data + structure recording all imported declarations in the form of a value of + type <code>TcRnTypes.ImportAvails</code>. + </p> + <p> + The function <code>importsFromLocalDecls</code>, in turn, makes use of + <code>RnNames.getLocalDeclBinders :: Module -> HsGroup RdrName -> RnM + [AvailInfo]</code> to extract all declared names from a binding group, + where <code>HscTypes.AvailInfo</code> is essentially a collection of + <code>Name</code>s; i.e., <code>getLocalDeclBinders</code>, on the fly, + generates <code>Name</code>s from the <code>RdrName</code>s of all + top-level binders of the module represented by the <code>HsGroup + RdrName</code> argument. + </p> + <p> + It is important to note that all this happens before the renamer + actually descends into the toplevel bindings of a module. In other + words, before <code>TcRnDriver.rnTopSrcDecls</code> performs the + renaming of a module by way of <code>RnSource.rnSrcDecls</code>, it uses + <code>importsFromLocalDecls</code> to set up the global + <code>RdrName</code> environment, which contains <code>Name</code>s for + all imported <em>and</em> all locally defined toplevel binders. Hence, + when the helpers of <code>rnSrcDecls</code> come across the + <em>defining</em> occurences of a toplevel <code>RdrName</code>, they + don't rename it by generating a new name, but they simply look up its + name in the global <code>RdrName</code> environment. + </p> + + <h2>Rebindable syntax</h2> + <p> + In Haskell when one writes "3" one gets "fromInteger 3", where + "fromInteger" comes from the Prelude (regardless of whether the + Prelude is in scope). If you want to completely redefine numbers, + that becomes inconvenient. So GHC lets you say + "-fno-implicit-prelude"; in that case, the "fromInteger" comes from + whatever is in scope. (This is documented in the User Guide.) + </p> + <p> + This feature is implemented as follows (I always forget). + <ul> + <li>Names that are implicitly bound by the Prelude, are marked by the + type <code>HsExpr.SyntaxExpr</code>. Moreover, the association list + <code>HsExpr.SyntaxTable</code> is set up by the renamer to map + rebindable names to the value they are bound to. + </li> + <li>Currently, five constructs related to numerals + (<code>HsExpr.NegApp</code>, <code>HsPat.NPat</code>, + <code>HsPat.NPlusKPat</code>, <code>HsLit.HsIntegral</code>, and + <code>HsLit.HsFractional</code>) and + two constructs related to code>do</code> expressions + (<code>HsExpr.BindStmt</code> and + <code>HsExpr.ExprStmt</code>) have rebindable syntax. + </li> + <li> When the parser builds these constructs, it puts in the + built-in Prelude Name (e.g. PrelNum.fromInteger). + </li> + <li> When the renamer encounters these constructs, it calls + <tt>RnEnv.lookupSyntaxName</tt>. + This checks for <tt>-fno-implicit-prelude</tt>; if not, it just + returns the same Name; otherwise it takes the occurrence name of the + Name, turns it into an unqualified RdrName, and looks it up in the + environment. The returned name is plugged back into the construct. + </li> + <li> The typechecker uses the Name to generate the appropriate typing + constraints. + </li> + </ul> + + <p><small> +<!-- hhmts start --> +Last modified: Wed May 4 17:16:15 EST 2005 +<!-- hhmts end --> + </small> + </body> +</html> + diff --git a/docs/comm/the-beast/simplifier.html b/docs/comm/the-beast/simplifier.html new file mode 100644 index 0000000000..40cf7cf892 --- /dev/null +++ b/docs/comm/the-beast/simplifier.html @@ -0,0 +1,86 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Mighty Simplifier</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Mighty Simplifier</h1> + <p> + Most of the optimising program transformations applied by GHC are + performed on an intermediate language called <em>Core,</em> which + essentially is a compiler-friendly formulation of rank-2 polymorphic + lambda terms defined in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/coreSyn/CoreSyn.lhs/"><code>CoreSyn.lhs</code>.</a> + The transformation engine optimising Core programs is called the + <em>Simplifier</em> and composed from a couple of modules located in the + directory <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/"><code>fptools/ghc/compiler/simplCore/</code>.</a> + The main engine of the simplifier is contained in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/Simplify.lhs"><code>Simplify.lhs</code>.</a> + and its driver is the routine <code>core2core</code> in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/SimplCore.lhs"><code>SimplCore.lhs</code>.</a> + <p> + The program that the simplifier has produced after applying its various + optimisations can be obtained by passing the option + <code>-ddump-simpl</code> to GHC. Moreover, the various intermediate + stages of the optimisation process is printed when passing + <code>-dverbose-core2core</code>. + + <h4><a name="loopBreaker">Recursive Definitions</a></h4> + <p> + The simplification process has to take special care when handling + recursive binding groups; otherwise, the compiler might loop. + Therefore, the routine <code>reOrderRec</code> in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/OccurAnal.lhs"><code>OccurAnal.lhs</code></a> + computes a set of <em>loop breakers</em> - a set of definitions that + together cut any possible loop in the binding group. It marks the + identifiers bound by these definitions as loop breakers by enriching + their <a href="basicTypes.html#occInfo">occurence information.</a> Loop + breakers will <em>never</em> be inlined by the simplifier; thus, + guaranteeing termination of the simplification procedure. (This is not + entirely accurate -- see <a href="#rules">rewrite rules</a> below.) + + The processes finding loop breakers works as follows: First, the + strongly connected components (SCC) of the graph representing all + function dependencies is computed. Then, each SCC is inspected in turn. + If it contains only a single binding (self-recursive function), this is + the loop breaker. In case of multiple recursive bindings, the function + attempts to select bindings where the decision not to inline them does + cause the least harm - in the sense of inhibiting optimisations in the + code. This is achieved by considering each binding in turn and awarding + a <em>score</em> between 0 and 4, where a lower score means that the + function is less useful for inlining - and thus, a better loop breaker. + The evaluation of bingings is performed by the function + <code>score</code> locally defined in <code>OccurAnal</code>. + + Note that, because core programs represent function definitions as + <em>one</em> binding choosing between the possibly many equations in the + source program with a <code>case</code> construct, a loop breaker cannot + inline any of its possibly many alternatives (not even the non-recursive + alternatives). + + <h4><a name="rules">Rewrite Rules</a></h4> + <p> + The application of rewrite rules is controlled in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/simplCore/Simplify.lhs"><code>Simplify.lhs</code></a> + by the function <code>completeCall</code>. This function first checks + whether it should inline the function applied at the currently inspected + call site, then simplifies the arguments, and finally, checks whether + any rewrite rule can be applied (and also whether there is a matching + specialised version of the applied function). The actual check for rule + application is performed by the function <code><a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/specialise/Rules.lhs">Rules</a>.lookupRule</code>. + <p> + It should be note that the application of rewrite rules is not subject + to the loop breaker check - i.e., rules of loop breakers will be applied + regardless of whether this may cause the simplifier to diverge. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Aug 8 19:25:33 EST 2001 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/stg.html b/docs/comm/the-beast/stg.html new file mode 100644 index 0000000000..4581da7d1f --- /dev/null +++ b/docs/comm/the-beast/stg.html @@ -0,0 +1,164 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - You Got Control: The STG-language</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - You Got Control: The STG-language</h1> + <p> + GHC contains two completely independent backends: the byte code + generator and the machine code generator. The decision over which of + the two is invoked is made in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/HscMain.lhs"><code>HscMain</code></a><code>.hscCodeGen</code>. + The machine code generator proceeds itself in a number of phases: First, + the <a href="desugar.html">Core</a> intermediate language is translated + into <em>STG-language</em>; second, STG-language is transformed into a + GHC-internal variant of <a href="http://www.cminusminus.org/">C--</a>; + and thirdly, this is either emitted as concrete C--, converted to GNU C, + or translated to native code (by the <a href="ncg.html">native code + generator</a> which targets IA32, Sparc, and PowerPC [as of March '5]). + </p> + <p> + In the following, we will have a look at the first step of machine code + generation, namely the translation steps involving the STG-language. + Details about the underlying abstract machine, the <em>Spineless Tagless + G-machine</em>, are in <a + href="http://research.microsoft.com/copyright/accept.asp?path=/users/simonpj/papers/spineless-tagless-gmachine.ps.gz&pub=34">Implementing + lazy functional languages on stock hardware: the Spineless Tagless + G-machine</a>, SL Peyton Jones, Journal of Functional Programming 2(2), + Apr 1992, pp127-202. (Some details have changed since the publication of + this article, but it still gives a good introduction to the main + concepts.) + </p> + + <h2>The STG Language</h2> + <p> + The AST of the STG-language and the generation of STG code from Core is + both located in the <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/"><code>stgSyn/</code></a> + directory; in the modules <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/StgSyn.lhs"><code>StgSyn</code></a> + and <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/CoreToStg.lhs"><code>CoreToStg</code></a>, + respectively. + </p> + <p> + Conceptually, the STG-language is a lambda calculus (including data + constructors and case expressions) whose syntax is restricted to make + all control flow explicit. As such, it can be regarded as a variant of + <em>administrative normal form (ANF).</em> (C.f., <a + href="http://doi.acm.org/10.1145/173262.155113">The essence of compiling + with continuations.</a> Cormac Flanagan, Amr Sabry, Bruce F. Duba, and + Matthias Felleisen. <em>ACM SIGPLAN Conference on Programming Language + Design and Implementation,</em> ACM Press, 1993.) Each syntactic from + has a precise operational interpretation, in addition to the + denotational interpretation inherited from the lambda calculus. The + concrete representation of the STG language inside GHC also includes + auxiliary attributes, such as <em>static reference tables (SRTs),</em> + which determine the top-level bindings referenced by each let binding + and case expression. + </p> + <p> + As usual in ANF, arguments to functions etc. are restricted to atoms + (i.e., constants or variables), which implies that all sub-expressions + are explicitly named and evaluation order is explicit. Specific to the + STG language is that all let bindings correspond to closure allocation + (thunks, function closures, and data constructors) and that case + expressions encode both computation and case selection. There are two + flavours of case expressions scrutinising boxed and unboxed values, + respectively. The former perform function calls including demanding the + evaluation of thunks, whereas the latter execute primitive operations + (such as arithmetic on fixed size integers and floating-point numbers). + </p> + <p> + The representation of STG language defined in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/StgSyn.lhs"><code>StgSyn</code></a> + abstracts over both binders and occurences of variables. The type names + involved in this generic definition all carry the prefix + <code>Gen</code> (such as in <code>GenStgBinding</code>). Instances of + these generic definitions, where both binders and occurences are of type + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/basicTypes/Id.lhs"><code>Id</code></a><code>.Id</code> + are defined as type synonyms and use type names that drop the + <code>Gen</code> prefix (i.e., becoming plain <code>StgBinding</code>). + Complete programs in STG form are represented by values of type + <code>[StgBinding]</code>. + </p> + + <h2>From Core to STG</h2> + <p> + Although, the actual translation from Core AST into STG AST is performed + by the function <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/CoreToStg.lhs"><code>CoreToStg</code></a><code>.coreToStg</code> + (or <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/CoreToStg.lhs"><code>CoreToStg</code></a><code>.coreExprToStg</code> + for individual expressions), the translation crucial depends on <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/coreSyn/CorePrep.lhs"><code>CorePrep</code></a><code>.corePrepPgm</code> + (resp. <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/coreSyn/CorePrep.lhs"><code>CorePrep</code></a><code>.corePrepExpr</code>), + which prepares Core code for code generation (for both byte code and + machine code generation). <code>CorePrep</code> saturates primitive and + constructor applications, turns the code into A-normal form, renames all + identifiers into globally unique names, generates bindings for + constructor workers, constructor wrappers, and record selectors plus + some further cleanup. + </p> + <p> + In other words, after Core code is prepared for code generation it is + structurally already in the form required by the STG language. The main + work performed by the actual transformation from Core to STG, as + performed by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/CoreToStg.lhs"><code>CoreToStg</code></a><code>.coreToStg</code>, + is to compute the live and free variables as well as live CAFs (constant + applicative forms) at each let binding and case alternative. In + subsequent phases, the live CAF information is used to compute SRTs. + The live variable information is used to determine which stack slots + need to be zapped (to avoid space leaks) and the free variable + information is need to construct closures. Moreover, hints for + optimised code generation are computed, such as whether a closure needs + to be updated after is has been evaluated. + </p> + + <h2>STG Passes</h2> + <p> + These days little actual work is performed on programs in STG form; in + particular, the code is not further optimised. All serious optimisation + (except low-level optimisations which are performed during native code + generation) has already been done on Core. The main task of <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/stgSyn/CoreToStg.lhs"><code>CoreToStg</code></a><code>.stg2stg</code> + is to compute SRTs from the live CAF information determined during STG + generation. Other than that, <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/profiling/SCCfinal.lhs"><code>SCCfinal</code></a><code>.stgMassageForProfiling</code> + is executed when compiling for profiling and information may be dumped + for debugging purposes. + </p> + + <h2>Towards C--</h2> + <p> + GHC's internal form of C-- is defined in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/cmm/Cmm.hs"><code>Cmm</code></a>. + The definition is generic in that it abstracts over the type of static + data and of the contents of basic blocks (i.e., over the concrete + representation of constant data and instructions). These generic + definitions have names carrying the prefix <code>Gen</code> (such as + <code>GenCmm</code>). The same module also instantiates the generic + form to a concrete form where data is represented by + <code>CmmStatic</code> and instructions are represented by + <code>CmmStmt</code> (giving us, e.g., <code>Cmm</code> from + <code>GenCmm</code>). The concrete form more or less follows the + external <a href="http://www.cminusminus.org/">C--</a> language. + </p> + <p> + Programs in STG form are translated to <code>Cmm</code> by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/codeGen/CodeGen.lhs"><code>CodeGen</code></a><code>.codeGen</code> + </p> + + <p><hr><small> +<!-- hhmts start --> +Last modified: Sat Mar 5 22:55:25 EST 2005 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/syntax.html b/docs/comm/the-beast/syntax.html new file mode 100644 index 0000000000..be5bbefa17 --- /dev/null +++ b/docs/comm/the-beast/syntax.html @@ -0,0 +1,99 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Just Syntax</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Just Syntax</h1> + <p> + The lexical and syntactic analyser for Haskell programs are located in + <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/"><code>fptools/ghc/compiler/parser/</code></a>. + </p> + + <h2>The Lexer</h2> + <p> + The lexer is a rather tedious piece of Haskell code contained in the + module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/Lex.lhs"><code>Lex</code></a>. + Its complexity partially stems from covering, in addition to Haskell 98, + also the whole range of GHC language extensions plus its ability to + analyse interface files in addition to normal Haskell source. The lexer + defines a parser monad <code>P a</code>, where <code>a</code> is the + type of the result expected from a successful parse. More precisely, a + result of type +<blockquote><pre> +data ParseResult a = POk PState a + | PFailed Message</pre> +</blockquote> + <p> + is produced with <code>Message</code> being from <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/ErrUtils.lhs"><code>ErrUtils</code></a> + (and currently is simply a synonym for <code>SDoc</code>). + <p> + The record type <code>PState</code> contains information such as the + current source location, buffer state, contexts for layout processing, + and whether Glasgow extensions are accepted (either due to + <code>-fglasgow-exts</code> or due to reading an interface file). Most + of the fields of <code>PState</code> store unboxed values; in fact, even + the flag indicating whether Glasgow extensions are enabled is + represented by an unboxed integer instead of by a <code>Bool</code>. My + (= chak's) guess is that this is to avoid having to perform a + <code>case</code> on a boxed value in the inner loop of the lexer. + <p> + The same lexer is used by the Haskell source parser, the Haskell + interface parser, and the package configuration parser. + + <h2>The Haskell Source Parser</h2> + <p> + The parser for Haskell source files is defined in the form of a parser + specification for the parser generator <a + href="http://haskell.org/happy/">Happy</a> in the file <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/Parser.y"><code>Parser.y</code></a>. + The parser exports three entry points for parsing entire modules + (<code>parseModule</code>, individual statements + (<code>parseStmt</code>), and individual identifiers + (<code>parseIdentifier</code>), respectively. The last two are needed + for GHCi. All three require a parser state (of type + <code>PState</code>) and are invoked from <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/HscMain.lhs"><code>HscMain</code></a>. + <p> + Parsing of Haskell is a rather involved process. The most challenging + features are probably the treatment of layout and expressions that + contain infix operators. The latter may be user-defined and so are not + easily captured in a static syntax specification. Infix operators may + also appear in the right hand sides of value definitions, and so, GHC's + parser treats those in the same way as expressions. In other words, as + general expressions are a syntactic superset of expressions - ok, they + <em>nearly</em> are - the parser simply attempts to parse a general + expression in such positions. Afterwards, the generated parse tree is + inspected to ensure that the accepted phrase indeed forms a legal + pattern. This and similar checks are performed by the routines from <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/parser/ParseUtil.lhs"><code>ParseUtil</code></a>. In + some cases, these routines do, in addition to checking for + wellformedness, also transform the parse tree, such that it fits into + the syntactic context in which it has been parsed; in fact, this happens + for patterns, which are transformed from a representation of type + <code>RdrNameHsExpr</code> into a representation of type + <code>RdrNamePat</code>. + + <h2>The Haskell Interface Parser</h2> + <p> + The parser for interface files is also generated by Happy from <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/rename/ParseIface.y"><code>ParseIface.y</code></a>. + It's main routine <code>parseIface</code> is invoked from <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/rename/RnHiFiles.lhs"><code>RnHiFiles</code></a><code>.readIface</code>. + + <h2>The Package Configuration Parser</h2> + <p> + The parser for configuration files is by far the smallest of the three + and defined in <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/ParsePkgConf.y"><code>ParsePkgConf.y</code></a>. + It exports <code>loadPackageConfig</code>, which is used by <a href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/main/DriverState.hs"><code>DriverState</code></a><code>.readPackageConf</code>. + + <p><small> +<!-- hhmts start --> +Last modified: Wed Jan 16 00:30:14 EST 2002 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/typecheck.html b/docs/comm/the-beast/typecheck.html new file mode 100644 index 0000000000..8d22784b8a --- /dev/null +++ b/docs/comm/the-beast/typecheck.html @@ -0,0 +1,316 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Checking Types</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Checking Types</h1> + <p> + Probably the most important phase in the frontend is the type checker, + which is located at <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/"><code>fptools/ghc/compiler/typecheck/</code>.</a> + GHC type checks programs in their original Haskell form before the + desugarer converts them into Core code. This complicates the type + checker as it has to handle the much more verbose Haskell AST, but it + improves error messages, as those message are based on the same + structure that the user sees. + </p> + <p> + GHC defines the abstract syntax of Haskell programs in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/hsSyn/HsSyn.lhs"><code>HsSyn</code></a> + using a structure that abstracts over the concrete representation of + bound occurences of identifiers and patterns. The module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcHsSyn.lhs"><code>TcHsSyn</code></a> + defines a number of helper function required by the type checker. Note + that the type <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcRnTypes.lhs"><code>TcRnTypes</code></a>.<code>TcId</code> + used to represent identifiers in some signatures during type checking + is, in fact, nothing but a synonym for a <a href="vars.html">plain + <code>Id</code>.</a> + </p> + <p> + It is also noteworthy, that the representations of types changes during + type checking from <code>HsType</code> to <code>TypeRep.Type</code>. + The latter is a <a href="types.html">hybrid type representation</a> that + is used to type Core, but still contains sufficient information to + recover source types. In particular, the type checker maintains and + compares types in their <code>Type</code> form. + </p> + + <h2>The Overall Flow of Things</h2> + + <h4>Entry Points Into the Type Checker</h4> + <p> + The interface of the type checker (and <a + href="renamer.html">renamer</a>) to the rest of the compiler is provided + by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcRnDriver.lhs"><code>TcRnDriver</code></a>. + Entire modules are processed by calling <code>tcRnModule</code> and GHCi + uses <code>tcRnStmt</code>, <code>tcRnExpr</code>, and + <code>tcRnType</code> to typecheck statements and expressions, and to + kind check types, respectively. Moreover, <code>tcRnExtCore</code> is + provided to typecheck external Core code. Moreover, + <code>tcTopSrcDecls</code> is used by Template Haskell - more + specifically by <code>TcSplice.tc_bracket</code> + - to type check the contents of declaration brackets. + </p> + + <h4>Renaming and Type Checking a Module</h4> + <p> + The function <code>tcRnModule</code> controls the complete static + analysis of a Haskell module. It sets up the combined renamer and type + checker monad, resolves all import statements, initiates the actual + renaming and type checking process, and finally, wraps off by processing + the export list. + </p> + <p> + The actual type checking and renaming process is initiated via + <code>TcRnDriver.tcRnSrcDecls</code>, which uses a helper called + <code>tc_rn_src_decls</code> to implement the iterative renaming and + type checking process required by <a href="../exts/th.html">Template + Haskell</a>. However, before it invokes <code>tc_rn_src_decls</code>, + it takes care of hi-boot files; afterwards, it simplifies type + constraints and zonking (see below regarding the later). + </p> + <p> + The function <code>tc_rn_src_decls</code> partitions static analysis of + a whole module into multiple rounds, where the initial round is followed + by an additional one for each toplevel splice. It collects all + declarations up to the next splice into an <code>HsDecl.HsGroup</code> + to rename and type check that <em>declaration group</em> by calling + <code>TcRnDriver.tcRnGroup</code>. Afterwards, it executes the + splice (if there are any left) and proceeds to the next group, which + includes the declarations produced by the splice. + </p> + <p> + The function <code>tcRnGroup</code>, finally, gets down to invoke the + actual renaming and type checking via + <code>TcRnDriver.rnTopSrcDecls</code> and + <code>TcRnDriver.tcTopSrcDecls</code>, respectively. The renamer, apart + from renaming, computes the global type checking environment, of type + <code>TcRnTypes.TcGblEnv</code>, which is stored in the type checking + monad before type checking commences. + </p> + + <h2>Type Checking a Declaration Group</h2> + <p> + The type checking of a declaration group, performed by + <code>tcTopSrcDecls</code> starts by processing of the type and class + declarations of the current module, using the function + <code>TcTyClsDecls.tcTyAndClassDecls</code>. This is followed by a + first round over instance declarations using + <code>TcInstDcls.tcInstDecls1</code>, which in particular generates all + additional bindings due to the deriving process. Then come foreign + import declarations (<code>TcForeign.tcForeignImports</code>) and + default declarations (<code>TcDefaults.tcDefaults</code>). + </p> + <p> + Now, finally, toplevel value declarations (including derived ones) are + type checked using <code>TcBinds.tcTopBinds</code>. Afterwards, + <code>TcInstDcls.tcInstDecls2</code> traverses instances for the second + time. Type checking concludes with processing foreign exports + (<code>TcForeign.tcForeignExports</code>) and rewrite rules + (<code>TcRules.tcRules</code>). Finally, the global environment is + extended with the new bindings. + </p> + + <h2>Type checking Type and Class Declarations</h2> + <p> + Type and class declarations are type checked in a couple of phases that + contain recursive dependencies - aka <em>knots.</em> The first knot + encompasses almost the whole type checking of these declarations and + forms the main piece of <code>TcTyClsDecls.tcTyAndClassDecls</code>. + </p> + <p> + Inside this big knot, the first main operation is kind checking, which + again involves a knot. It is implemented by <code>kcTyClDecls</code>, + which performs kind checking of potentially recursively-dependent type + and class declarations using kind variables for initially unknown kinds. + During processing the individual declarations some of these variables + will be instantiated depending on the context; the rest gets by default + kind <code>*</code> (during <em>zonking</em> of the kind signatures). + Type synonyms are treated specially in this process, because they can + have an unboxed type, but they cannot be recursive. Hence, their kinds + are inferred in dependency order. Moreover, in contrast to class + declarations and other type declarations, synonyms are not entered into + the global environment as a global <code>TyThing</code>. + (<code>TypeRep.TyThing</code> is a sum type that combines the various + flavours of typish entities, such that they can be stuck into type + environments and similar.) + </p> + + <h2>More Details</h2> + + <h4>Types Variables and Zonking</h4> + <p> + During type checking type variables are represented by mutable variables + - cf. the <a href="vars.html#TyVar">variable story.</a> Consequently, + unification can instantiate type variables by updating those mutable + variables. This process of instantiation is (for reasons that elude me) + called <a + href="http://www.dictionary.com/cgi-bin/dict.pl?term=zonk&db=*">zonking</a> + in GHC's sources. The zonking routines for the various forms of Haskell + constructs are responsible for most of the code in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcHsSyn.lhs"><code>TcHsSyn</code>,</a> + whereas the routines that actually operate on mutable types are defined + in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcMType.lhs"><code>TcMType</code></a>; + this includes the zonking of type variables and type terms, routines to + create mutable structures and update them as well as routines that check + constraints, such as that type variables in function signatures have not + been instantiated during type checking. The actual type unification + routine is <code>uTys</code> in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcUnify.lhs"><code>TcUnify</code></a>. + </p> + <p> + All type variables that may be instantiated (those in signatures + may not), but haven't been instantiated during type checking, are zonked + to <code>()</code>, so that after type checking all mutable variables + have been eliminated. + </p> + + <h4>Type Representation</h4> + <p> + The representation of types is fixed in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcRep.lhs"><code>TcRep</code></a> + and exported as the data type <code>Type</code>. As explained in <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcType.lhs"><code>TcType</code></a>, + GHC supports rank-N types, but, in the type checker, maintains the + restriction that type variables cannot be instantiated to quantified + types (i.e., the type system is predicative). The type checker floats + universal quantifiers outside and maintains types in prenex form. + (However, quantifiers can, of course, not float out of negative + positions.) Overall, we have + </p> + <blockquote> + <pre> +sigma -> forall tyvars. phi +phi -> theta => rho +rho -> sigma -> rho + | tau +tau -> tyvar + | tycon tau_1 .. tau_n + | tau_1 tau_2 + | tau_1 -> tau_2</pre> + </blockquote> + <p> + where <code>sigma</code> is in prenex form; i.e., there is never a + forall to the right of an arrow in a <code>phi</code> type. Moreover, a + type of the form <code>tau</code> never contains a quantifier (which + includes arguments to type constructors). + </p> + <p> + Of particular interest are the variants <code>SourceTy</code> and + <code>NoteTy</code> of <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TypeRep.lhs"><code>TypeRep</code></a>.<code>Type</code>. + The constructor <code>SourceTy :: SourceType -> Type</code> represents a + type constraint; that is, a predicate over types represented by a + dictionary. The type checker treats a <code>SourceTy</code> as opaque, + but during the translation to core it will be expanded into its concrete + representation (i.e., a dictionary type) by the function <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/types/Type.lhs"><code>Type</code></a>.<code>sourceTypeRep</code>. + Note that newtypes are not covered by <code>SourceType</code>s anymore, + even if some comments in GHC still suggest this. Instead, all newtype + applications are initially represented as a <code>NewTcApp</code>, until + they are eliminated by calls to <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/types/Type.lhs"><code>Type</code></a>.<code>newTypeRep</code>. + </p> + <p> + The <code>NoteTy</code> constructor is used to add non-essential + information to a type term. Such information has the type + <code>TypeRep.TyNote</code> and is either the set of free type variables + of the annotated expression or the unexpanded version of a type synonym. + Free variables sets are cached as notes to save the overhead of + repeatedly computing the same set for a given term. Unexpanded type + synonyms are useful for generating comprehensible error messages, but + have no influence on the process of type checking. + </p> + + <h4>Type Checking Environment</h4> + <p> + During type checking, GHC maintains a <em>type environment</em> whose + type definitions are fixed in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcRnTypes.lhs"><code>TcRnTypes</code></a> with the operations defined in +<a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcEnv.lhs"><code>TcEnv</code></a>. + Among other things, the environment contains all imported and local + instances as well as a list of <em>global</em> entities (imported and + local types and classes together with imported identifiers) and + <em>local</em> entities (locally defined identifiers). This environment + is threaded through the type checking monad, whose support functions + including initialisation can be found in the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcRnMonad.lhs"><code>TcRnMonad</code>.</a> + + <h4>Expressions</h4> + <p> + Expressions are type checked by <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/TcExpr.lhs"><code>TcExpr</code>.</a> + <p> + Usage occurences of identifiers are processed by the function + <code>tcId</code> whose main purpose is to <a href="#inst">instantiate + overloaded identifiers.</a> It essentially calls + <code>TcInst.instOverloadedFun</code> once for each universally + quantified set of type constraints. It should be noted that overloaded + identifiers are replaced by new names that are first defined in the LIE + (Local Instance Environment?) and later promoted into top-level + bindings. + + <h4><a name="inst">Handling of Dictionaries and Method Instances</a></h4> + <p> + GHC implements overloading using so-called <em>dictionaries.</em> A + dictionary is a tuple of functions -- one function for each method in + the class of which the dictionary implements an instance. During type + checking, GHC replaces each type constraint of a function with one + additional argument. At runtime, the extended function gets passed a + matching class dictionary by way of these additional arguments. + Whenever the function needs to call a method of such a class, it simply + extracts it from the dictionary. + <p> + This sounds simple enough; however, the actual implementation is a bit + more tricky as it wants to keep track of all the instances at which + overloaded functions are used in a module. This information is useful + to optimise the code. The implementation is the module <a + href="http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/ghc/compiler/typecheck/Inst.lhs"><code>Inst.lhs</code>.</a> + <p> + The function <code>instOverloadedFun</code> is invoked for each + overloaded usage occurence of an identifier, where overloaded means that + the type of the idendifier contains a non-trivial type constraint. It + proceeds in two steps: (1) Allocation of a method instance + (<code>newMethodWithGivenTy</code>) and (2) instantiation of functional + dependencies. The former implies allocating a new unique identifier, + which replaces the original (overloaded) identifier at the currently + type-checked usage occurrence. + <p> + The new identifier (after being threaded through the LIE) eventually + will be bound by a top-level binding whose rhs contains a partial + application of the original overloaded identifier. This papp applies + the overloaded function to the dictionaries needed for the current + instance. In GHC lingo, this is called a <em>method.</em> Before + becoming a top-level binding, the method is first represented as a value + of type <code>Inst.Inst</code>, which makes it easy to fold multiple + instances of the same identifier at the same types into one global + definition. (And probably other things, too, which I haven't + investigated yet.) + + <p> + <strong>Note:</strong> As of 13 January 2001 (wrt. to the code in the + CVS HEAD), the above mechanism interferes badly with RULES pragmas + defined over overloaded functions. During instantiation, a new name is + created for an overloaded function partially applied to the dictionaries + needed in a usage position of that function. As the rewrite rule, + however, mentions the original overloaded name, it won't fire anymore + -- unless later phases remove the intermediate definition again. The + latest CVS version of GHC has an option + <code>-fno-method-sharing</code>, which avoids sharing instantiation + stubs. This is usually/often/sometimes sufficient to make the rules + fire again. + + <p><small> +<!-- hhmts start --> +Last modified: Thu May 12 22:52:46 EST 2005 +<!-- hhmts end --> + </small> + </body> +</html> diff --git a/docs/comm/the-beast/types.html b/docs/comm/the-beast/types.html new file mode 100644 index 0000000000..383b71f054 --- /dev/null +++ b/docs/comm/the-beast/types.html @@ -0,0 +1,179 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - Hybrid Types</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - Hybrid Types</h1> + <p> + GHC essentially supports two type systems: (1) the <em>source type + system</em> (which is a heavily extended version of the type system of + Haskell 98) and (2) the <em>Core type system,</em> which is the type system + used by the intermediate language (see also <a + href="desugar.html">Sugar Free: From Haskell To Core</a>). + </p> + <p> + During parsing and renaming, type information is represented in a form + that is very close to Haskell's concrete syntax; it is defined by + <code>HsTypes.HsType</code>. In addition, type, class, and instance + declarations are maintained in their source form as defined in the + module <code>HsDecl</code>. The situation changes during type checking, + where types are translated into a second representation, which is + defined in the module <code>types/TypeRep.lhs</code>, as type + <code>Type</code>. This second representation is peculiar in that it is + a hybrid between the source representation of types and the Core + representation of types. Using functions, such as + <code>Type.coreView</code> and <code>Type.deepCoreView</code>, a value + of type <code>Type</code> exhibits its Core representation. On the + other hand, pretty printing a <code>Type</code> with + <code>TypeRep.pprType</code> yields the type's source representation. + </p> + <p> + In fact, the <a href="typecheck.html">type checker</a> maintains type + environments based on <code>Type</code>, but needs to perform type + checking on source-level types. As a result, we have functions + <code>Type.tcEqType</code> and <code>Type.tcCmpType</code>, which + compare types based on their source representation, as well as the + function <code>coreEqType</code>, which compares them based on their + core representation. The latter is needed during type checking of Core + (as performed by the functions in the module + <code>coreSyn/CoreLint.lhs</code>). + </p> + + <h2>Type Synonyms</h2> + <p> + Type synonyms in Haskell are essentially a form of macro definitions on + the type level. For example, when the type checker compares two type + terms, synonyms are always compared in their expanded form. However, to + produce good error messages, we like to avoid expanding type synonyms + during pretty printing. Hence, <code>Type</code> has a variant + <code>NoteTy TyNote Type</code>, where + </p> + <blockquote> + <pre> +data TyNote + = FTVNote TyVarSet -- The free type variables of the noted expression + + | SynNote Type -- Used for type synonyms + -- The Type is always a TyConApp, and is the un-expanded form. + -- The type to which the note is attached is the expanded form.</pre> + </blockquote> + <p> + In other words, a <code>NoteTy</code> represents the expanded form of a + type synonym together with a note stating its source form. + </p> + + <h3>Creating Representation Types of Synonyms</h3> + <p> + During translation from <code>HsType</code> to <code>Type</code> the + function <code>Type.mkSynTy</code> is used to construct representations + of applications of type synonyms. It creates a <code>NoteTy</code> node + if the synonym is applied to a sufficient number of arguments; + otherwise, it builds a simple <code>TyConApp</code> and leaves it to + <code>TcMType.checkValidType</code> to pick up invalid unsaturated + synonym applications. While creating a <code>NoteTy</code>, + <code>mkSynTy</code> also expands the synonym by substituting the type + arguments for the parameters of the synonym definition, using + <code>Type.substTyWith</code>. + </p> + <p> + The function <code>mkSynTy</code> is used indirectly via + <code>mkGenTyConApp</code>, <code>mkAppTy</code>, and + <code>mkAppTy</code>, which construct type representations involving + type applications. The function <code>mkSynTy</code> is also used + directly during type checking interface files; this is for tedious + reasons to do with forall hoisting - see the comment at + <code>TcIface.mkIfTcApp</code>. + </p> + + <h2>Newtypes</h2> + <p> + Data types declared by a <code>newtype</code> declarations constitute new + type constructors---i.e., they are not just type macros, but introduce + new type names. However, provided that a newtype is not recursive, we + still want to implement it by its representation type. GHC realises this + by providing two flavours of type equality: (1) <code>tcEqType</code> is + source-level type equality, which compares newtypes and + <code>PredType</code>s by name, and (2) <code>coreEqType</code> compares + them structurally (by using <code>deepCoreView</code> to expand the + representation before comparing). The function + <code>deepCoreView</code> (via <code>coreView</code>) invokes + <code>expandNewTcApp</code> for every type constructor application + (<code>TyConApp</code>) to determine whether we are looking at a newtype + application that needs to be expanded to its representation type. + </p> + + <h2>Predicates</h2> + <p> + The dictionary translation of type classes, translates each predicate in + a type context of a type signature into an additional argument, which + carries a dictionary with the functions overloaded by the corresponding + class. The <code>Type</code> data type has a special variant + <code>PredTy PredType</code> for predicates, where + </p> + <blockquote> + <pre> +data PredType + = ClassP Class [Type] -- Class predicate + | IParam (IPName Name) Type -- Implicit parameter</pre> + </blockquote> + <p> + These types need to be handled as source type during type checking, but + turn into their representations when inspected through + <code>coreView</code>. The representation is determined by + <code>Type.predTypeRep</code>. + </p> + + <h2>Representation of Type Constructors</h2> + <p> + Type constructor applications are represented in <code>Type</code> by + the variant <code>TyConApp :: TyCon -> [Type] -> Type</code>. The first + argument to <code>TyConApp</code>, namely <code>TyCon.TyCon</code>, + distinguishes between function type constructors (variant + <code>FunTyCon</code>) and algebraic type constructors (variant + <code>AlgTyCon</code>), which arise from data and newtype declarations. + The variant <code>AlgTyCon</code> contains all the information available + from the data/newtype declaration as well as derived information, such + as the <code>Unique</code> and argument variance information. This + includes a field <code>algTcRhs :: AlgTyConRhs</code>, where + <code>AlgTyConRhs</code> distinguishes three kinds of algebraic data + type declarations: (1) declarations that have been exported abstractly, + (2) <code>data</code> declarations, and (3) <code>newtype</code> + declarations. The last two both include their original right hand side; + in addition, the third variant also caches the "ultimate" representation + type, which is the right hand side after expanding all type synonyms and + non-recursive newtypes. + </p> + <p> + Both data and newtype declarations refer to their data constructors + represented as <code>DataCon.DataCon</code>, which include all details + of their signature (as derived from the original declaration) as well + information for code generation, such as their tag value. + </p> + + <h2>Representation of Classes and Instances</h2> + <p> + Class declarations turn into values of type <code>Class.Class</code>. + They represent methods as the <code>Id</code>s of the dictionary + selector functions. Similar selector functions are available for + superclass dictionaries. + </p> + <p> + Instance declarations turn into values of type + <code>InstEnv.Instance</code>, which in interface files are represented + as <code>IfaceSyn.IfaceInst</code>. Moreover, the type + <code>InstEnv.InstEnv</code>, which is a synonym for <code>UniqFM + ClsInstEnv</code>, provides a mapping of classes to their + instances---<code>ClsInstEnv</code> is essentially a list of instance + declarations. + </p> + + <p><small> +<!-- hhmts start --> +Last modified: Sun Jun 19 13:07:22 EST 2005 +<!-- hhmts end --> + </small></p> + </body> +</html> diff --git a/docs/comm/the-beast/vars.html b/docs/comm/the-beast/vars.html new file mode 100644 index 0000000000..9bbd310c60 --- /dev/null +++ b/docs/comm/the-beast/vars.html @@ -0,0 +1,235 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> + <head> + <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=ISO-8859-1"> + <title>The GHC Commentary - The Real Story about Variables, Ids, TyVars, and the like</title> + </head> + + <body BGCOLOR="FFFFFF"> + <h1>The GHC Commentary - The Real Story about Variables, Ids, TyVars, and the like</h1> + <p> + + +<h2>Variables</h2> + +The <code>Var</code> type, defined in <code>basicTypes/Var.lhs</code>, +represents variables, both term variables and type variables: +<pre> + data Var + = Var { + varName :: Name, + realUnique :: FastInt, + varType :: Type, + varDetails :: VarDetails, + varInfo :: IdInfo + } +</pre> +<ul> +<li> The <code>varName</code> field contains the identity of the variable: +its unique number, and its print-name. See "<a href="names.html">The truth about names</a>". + +<p><li> The <code>realUnique</code> field caches the unique number in the +<code>varName</code> field, just to make comparison of <code>Var</code>s a little faster. + +<p><li> The <code>varType</code> field gives the type of a term variable, or the kind of a +type variable. (Types and kinds are both represented by a <code>Type</code>.) + +<p><li> The <code>varDetails</code> field distinguishes term variables from type variables, +and makes some further distinctions (see below). + +<p><li> For term variables (only) the <code>varInfo</code> field contains lots of useful +information: strictness, unfolding, etc. However, this information is all optional; +you can always throw away the <code>IdInfo</code>. In contrast, you can't safely throw away +the <code>VarDetails</code> of a <code>Var</code> +</ul> +<p> +It's often fantastically convenient to have term variables and type variables +share a single data type. For example, +<pre> + exprFreeVars :: CoreExpr -> VarSet +</pre> +If there were two types, we'd need to return two sets. Simiarly, big lambdas and +little lambdas use the same constructor in Core, which is extremely convenient. +<p> +We define a couple of type synonyms: +<pre> + type Id = Var -- Term variables + type TyVar = Var -- Type variables +</pre> +just to help us document the occasions when we are expecting only term variables, +or only type variables. + + +<h2> The <code>VarDetails</code> field </h2> + +The <code>VarDetails</code> field tells what kind of variable this is: +<pre> +data VarDetails + = LocalId -- Used for locally-defined Ids (see NOTE below) + LocalIdDetails + + | GlobalId -- Used for imported Ids, dict selectors etc + GlobalIdDetails + + | TyVar + | MutTyVar (IORef (Maybe Type)) -- Used during unification; + TyVarDetails +</pre> + +<a name="TyVar"> +<h2>Type variables (<code>TyVar</code>)</h2> +</a> +<p> +The <code>TyVar</code> case is self-explanatory. The <code>MutTyVar</code> +case is used only during type checking. Then a type variable can be unified, +using an imperative update, with a type, and that is what the +<code>IORef</code> is for. The <code>TcType.TyVarDetails</code> field records +the sort of type variable we are dealing with. It is defined as +<pre> +data TyVarDetails = SigTv | ClsTv | InstTv | VanillaTv +</pre> +<code>SigTv</code> marks type variables that were introduced when +instantiating a type signature prior to matching it against the inferred type +of a definition. The variants <code>ClsTv</code> and <code>InstTv</code> mark +scoped type variables introduced by class and instance heads, respectively. +These first three sorts of type variables are skolem variables (tested by the +predicate <code>isSkolemTyVar</code>); i.e., they must <em>not</em> be +instantiated. All other type variables are marked as <code>VanillaTv</code>. +<p> +For a long time I tried to keep mutable Vars statically type-distinct +from immutable Vars, but I've finally given up. It's just too painful. +After type checking there are no MutTyVars left, but there's no static check +of that fact. + +<h2>Term variables (<code>Id</code>)</h2> + +A term variable (of type <code>Id</code>) is represented either by a +<code>LocalId</code> or a <code>GlobalId</code>: +<p> +A <code>GlobalId</code> is +<ul> +<li> Always bound at top-level. +<li> Always has a <code>GlobalName</code>, and hence has + a <code>Unique</code> that is globally unique across the whole + GHC invocation (a single invocation may compile multiple modules). +<li> Has <code>IdInfo</code> that is absolutely fixed, forever. +</ul> + +<p> +A <code>LocalId</code> is: +<ul> +<li> Always bound in the module being compiled: +<ul> +<li> <em>either</em> bound within an expression (lambda, case, local let(rec)) +<li> <em>or</em> defined at top level in the module being compiled. +</ul> +<li> Has IdInfo that changes as the simpifier bashes repeatedly on it. +</ul> +<p> +The key thing about <code>LocalId</code>s is that the free-variable finder +typically treats them as candidate free variables. That is, it ignores +<code>GlobalId</code>s such as imported constants, data contructors, etc. +<p> +An important invariant is this: <em>All the bindings in the module +being compiled (whether top level or not) are <code>LocalId</code>s +until the CoreTidy phase.</em> In the CoreTidy phase, all +externally-visible top-level bindings are made into GlobalIds. This +is the point when a <code>LocalId</code> becomes "frozen" and becomes +a fixed, immutable <code>GlobalId</code>. +<p> +(A binding is <em>"externally-visible"</em> if it is exported, or +mentioned in the unfolding of an externally-visible Id. An +externally-visible Id may not have an unfolding, either because it is +too big, or because it is the loop-breaker of a recursive group.) + +<h3>Global Ids and implicit Ids</h3> + +<code>GlobalId</code>s are further categorised by their <code>GlobalIdDetails</code>. +This type is defined in <code>basicTypes/IdInfo</code>, because it mentions other +structured types like <code>DataCon</code>. Unfortunately it is *used* in <code>Var.lhs</code> +so there's a <code>hi-boot</code> knot to get it there. Anyway, here's the declaration: +<pre> +data GlobalIdDetails + = NotGlobalId -- Used as a convenient extra return value + -- from globalIdDetails + + | VanillaGlobal -- Imported from elsewhere + + | PrimOpId PrimOp -- The Id for a primitive operator + | FCallId ForeignCall -- The Id for a foreign call + + -- These next ones are all "implicit Ids" + | RecordSelId FieldLabel -- The Id for a record selector + | DataConId DataCon -- The Id for a data constructor *worker* + | DataConWrapId DataCon -- The Id for a data constructor *wrapper* + -- [the only reasons we need to know is so that + -- a) we can suppress printing a definition in the interface file + -- b) when typechecking a pattern we can get from the + -- Id back to the data con] +</pre> +The <code>GlobalIdDetails</code> allows us to go from the <code>Id</code> for +a record selector, say, to its field name; or the <code>Id</code> for a primitive +operator to the <code>PrimOp</code> itself. +<p> +Certain <code>GlobalId</code>s are called <em>"implicit"</em> Ids. An implicit +Id is derived by implication from some other declaration. So a record selector is +derived from its data type declaration, for example. An implicit Ids is always +a <code>GlobalId</code>. For most of the compilation, the implicit Ids are just +that: implicit. If you do -ddump-simpl you won't see their definition. (That's +why it's true to say that until CoreTidy all Ids in this compilation unit are +LocalIds.) But at CorePrep, a binding is added for each implicit Id defined in +this module, so that the code generator will generate code for the (curried) function. +<p> +Implicit Ids carry their unfolding inside them, of course, so they may well have +been inlined much earlier; but we generate the curried top-level defn just in +case its ever needed. + +<h3>LocalIds</h3> + +The <code>LocalIdDetails</code> gives more info about a <code>LocalId</code>: +<pre> +data LocalIdDetails + = NotExported -- Not exported + | Exported -- Exported + | SpecPragma -- Not exported, but not to be discarded either + -- It's unclean that this is so deeply built in +</pre> +From this we can tell whether the <code>LocalId</code> is exported, and that +tells us whether we can drop an unused binding as dead code. +<p> +The <code>SpecPragma</code> thing is a HACK. Suppose you write a SPECIALIZE pragma: +<pre> + foo :: Num a => a -> a + {-# SPECIALIZE foo :: Int -> Int #-} + foo = ... +</pre> +The type checker generates a dummy call to <code>foo</code> at the right types: +<pre> + $dummy = foo Int dNumInt +</pre> +The Id <code>$dummy</code> is marked <code>SpecPragma</code>. Its role is to hang +onto that call to <code>foo</code> so that the specialiser can see it, but there +are no calls to <code>$dummy</code>. +The simplifier is careful not to discard <code>SpecPragma</code> Ids, so that it +reaches the specialiser. The specialiser processes the right hand side of a <code>SpecPragma</code> Id +to find calls to overloaded functions, <em>and then discards the <code>SpecPragma</code> Id</em>. +So <code>SpecPragma</code> behaves a like <code>Exported</code>, at least until the specialiser. + + +<h3> ExternalNames and InternalNames </h3> + +Notice that whether an Id is a <code>LocalId</code> or <code>GlobalId</code> is +not the same as whether the Id has an <code>ExternaName</code> or an <code>InternalName</code> +(see "<a href="names.html#sort">The truth about Names</a>"): +<ul> +<li> Every <code>GlobalId</code> has an <code>ExternalName</code>. +<li> A <code>LocalId</code> might have either kind of <code>Name</code>. +</ul> + +<!-- hhmts start --> +Last modified: Fri Sep 12 15:17:18 BST 2003 +<!-- hhmts end --> + </small> + </body> +</html> + diff --git a/docs/ext-core/Makefile b/docs/ext-core/Makefile new file mode 100644 index 0000000000..8c32a7bb25 --- /dev/null +++ b/docs/ext-core/Makefile @@ -0,0 +1,42 @@ +# General makefile for Latex stuff + +dvi: core.dvi +ps: core.ps + +core.dvi: core.tex prims.tex + latex core.tex + latex core.tex + +../../compiler/prelude/primops.txt: ../../compiler/prelude/primops.txt.pp + (cd ../../compiler/prelude; gcc -E -I../../includes -traditional -x c primops.txt.pp | /bin/sed -e '/^#/d' > primops.txt) + +prims.tex: ../../compiler/prelude/primops.txt + ../../utils/genprimopcode/genprimopcode --make-latex-doc < ../../compiler/prelude/primops.txt > prims.tex + + +######## General rules +.SUFFIXES: +.PRECIOUS: %.tex %.ps %.bbl + + +%.ps: %.dvi + dvips -f < $< > $@ + +clean: + $(RM) *.aux *.log + +distclean: clean + $(RM) prims.tex *.dvi *.ps *.bbl *.blg *.gz + +maintainer-clean: distclean + +# dummy targets +all: +boot: +install: +install-docs: +html: +chm: +HxS: + +# End of file diff --git a/docs/ext-core/a4wide.sty b/docs/ext-core/a4wide.sty new file mode 100644 index 0000000000..9f651505d7 --- /dev/null +++ b/docs/ext-core/a4wide.sty @@ -0,0 +1,39 @@ +%NAME: a4wide.sty +% "moretext" document style option. +% Jean-Francois Lamy, July 86 +% +% Redefines the margins so that they are more in line with +% what we are used to see. +% +% [Minimally modified for LaTeX2e, Alexander Holt, August 1994] + +\NeedsTeXFormat{LaTeX2e} +\ProvidesPackage{a4wide}[1994/08/30] +\RequirePackage{a4} + +\ifcase \@ptsize + % mods for 10 pt + \oddsidemargin 0.15 in % Left margin on odd-numbered pages. + \evensidemargin 0.35 in % Left margin on even-numbered pages. + \marginparwidth 1 in % Width of marginal notes. + \oddsidemargin 0.25 in % Note that \oddsidemargin = \evensidemargin + \evensidemargin 0.25 in + \marginparwidth 0.75 in + \textwidth 5.875 in % Width of text line. +\or % mods for 11 pt + \oddsidemargin 0.1 in % Left margin on odd-numbered pages. + \evensidemargin 0.15 in % Left margin on even-numbered pages. + \marginparwidth 1 in % Width of marginal notes. + \oddsidemargin 0.125 in % Note that \oddsidemargin = \evensidemargin + \evensidemargin 0.125 in + \marginparwidth 0.75 in + \textwidth 6.125 in % Width of text line. +\or % mods for 12 pt + \oddsidemargin -10 pt % Left margin on odd-numbered pages. + \evensidemargin 10 pt % Left margin on even-numbered pages. + \marginparwidth 1 in % Width of marginal notes. + \oddsidemargin 0 in % Note that \oddsidemargin = \evensidemargin + \evensidemargin 0 in + \marginparwidth 0.75 in + \textwidth 6.375 true in % Width of text line. +\fi diff --git a/docs/ext-core/code.sty b/docs/ext-core/code.sty new file mode 100644 index 0000000000..3b62685057 --- /dev/null +++ b/docs/ext-core/code.sty @@ -0,0 +1,83 @@ + +% I have enclosed code.sty, which achieves 99% of what you want without +% the need for a separate preprocessor. At the start of your document +% you write "\makeatactive". From then on, inline code is written as @\x +% -> x_1 & y@. The only difference with what you are used to, is that +% instead of +% +% @ +% foo :: Int -> Int +% foo = \n -> n+1 +% @ +% +% you have to write +% +% \begin{code} +% foo :: Int -> Int +% foo = \n -> n+1 +% \end{code} +% +% and that you cannot use @ in \section{} and \caption{}. For the paper that occured twice, in which case I had to replace @...@ b y \texttt{...}. +% +% +% code.sty --- nice verbatim mode for code + +\def\icode{% + \relax\ifmmode\hbox\else\leavevmode\null\fi + \bgroup + %\begingroup + \@noligs + \verbatim@font + \verb@eol@error + \let\do\@makeother \dospecials + \@vobeyspaces + \frenchspacing + \@icode} +\def\@icode#1{% + \catcode`#1\active + \lccode`\~`#1% + \lowercase{\let~\icode@egroup}} +\def\icode@egroup{% + %\endgroup} + \egroup} + +% The \makeatactive command: +% makes @ active, in such a way that @...@ behaves as \icode@...@: +{ +\catcode`@=\active +\gdef\makeatactive{ + \catcode`@=\active \def@{\icode@} + % Since @ becomes active, it has to be taken care of in verbatim-modes: + \let\olddospecials\dospecials \def\dospecials{\do\@\olddospecials}} +} +% \gdef\makeatother{\g@remfrom@specials{\@}\@makeother\@} +\gdef\makeatother{\@makeother\@} + +\newcommand\codetabwidth{42pt} +{\catcode`\^^I=\active% +\gdef\@vobeytab{\catcode`\^^I\active\let^^I\@xobeytab}} +\def\@xobeytab{\leavevmode\penalty10000\hskip\codetabwidth} + +\begingroup \catcode `|=0 \catcode `[= 1 +\catcode`]=2 \catcode `\{=12 \catcode `\}=12 +\catcode`\\=12 |gdef|@xcode#1\end{code}[#1|end[code]] +|endgroup +\def\@code{\trivlist \item\relax + \if@minipage\else\vskip\parskip\fi + \leftskip\@totalleftmargin\rightskip\z@skip + \parindent\z@\parfillskip\@flushglue\parskip\z@skip + \@@par + \@tempswafalse + \def\par{% + \if@tempswa + \leavevmode \null \@@par\penalty\interlinepenalty + \else + \@tempswatrue + \ifhmode\@@par\penalty\interlinepenalty\fi + \fi}% + \obeylines \verbatim@font \@noligs + \let\do\@makeother \dospecials + \everypar \expandafter{\the\everypar \unpenalty}% +} +\def\code{\@code \frenchspacing\@vobeytab\@vobeyspaces \@xcode} +\def\endcode{\if@newlist \leavevmode\fi\endtrivlist} diff --git a/docs/ext-core/core.tex b/docs/ext-core/core.tex new file mode 100644 index 0000000000..266d857c46 --- /dev/null +++ b/docs/ext-core/core.tex @@ -0,0 +1,926 @@ +\documentclass[10pt]{article} +\usepackage{a4wide} +\usepackage{code} + + +\sloppy +\setlength{\parskip}{0.5\baselineskip plus 0.2\baselineskip minus 0.1\baselineskip} +\setlength{\parsep}{\parskip} +\setlength{\topsep}{0cm} +\setlength{\parindent}{0cm} +%\oddsidemargin -0.5 in +%\evensidemargin -0.5 in +%\textwidth 7.375 in + +\newcommand{\derives}{\mbox{$\rightarrow$}} +\newcommand{\orderives}{\mbox{$\mid$}} +\newcommand{\many}[1]{\{ {#1} \}} +\newcommand{\oneormore}[1]{\{ {#1} \}$^{+}$} +\newcommand{\optional}[1]{[ {#1} ]} + +\newcommand{\at}{\texttt{@}} +\newcommand{\att}{@} +\newcommand{\lam}{\texttt{\char`\\}} + +\newcommand{\workingnote}[1]% + {\begin{quote} + \framebox{\parbox{.8 \linewidth} + {\textbf{\textsl{Working note:}} \textsl{#1}}} + \end{quote}} + +\begin{document} + +\title{An External Representation for the GHC Core Language (DRAFT for GHC5.02)} +\author{Andrew Tolmach ({\tt apt@cs.pdx.edu})\\and The GHC Team} + +\maketitle +\makeatactive + +\abstract{ +This document provides a precise definition for the GHC Core language, +so that it can be used to communicate between GHC and new stand-alone +compilation tools such as back-ends or optimizers. +The definition includes a formal grammar and an informal semantics. +An executable typechecker and interpreter (in Haskell), +which formally embody the static and dynamic semantics, +are available separately. + +Note: This is a draft document, which attempts to describe GHC's current +behavior as precisely as possible. Working notes scattered throughout indicate +areas where further work is needed. Constructive comments are very welcome, +both on the presentation, and on ways in which GHC could be improved in order +to simplify the Core story. +} + +\section{Introduction} + +The Glasgow Haskell Compiler (GHC) uses an intermediate language, called +``Core,'' as its internal program representation during +several key stages of compiling. +Core resembles a subset of Haskell, but with explicit type annotations +in the style of the polymorphic lambda calculus (F$_\omega$). +GHC's front end translates full Haskell 98 (plus some extensions) into +well-typed Core, which is then repeatedly rewritten by the GHC optimizer. +Ultimately, GHC translates Core into STG-machine code and then into +C or native code. The rationale for the design of Core and its use are discussed +in existing papers~\cite{ghc-inliner,comp-by-trans-scp}, although the (two different) +idealized versions of Core described therein differ in significant ways +from the actual Core language in current GHC. + +Researchers interested in writing just {\it part} of a Haskell compiler, +such as a new back-end or a new optimizer pass, might like to make +use of GHC to provide the other parts of the compiler. For example, they +might like to use GHC's front end to parse, desugar, and type-check source Haskell, +then feeding the resulting code to their own back-end tool. +Currently, they can only do this by linking their code into the +GHC executable, which is an arduous process (and essentially requires +the new code to be written in Haskell). It would be much easier for +external developers if GHC could be made to produce Core files in +an agreed-upon external format. To allow the widest range of interoperability, +the external format should be text-based; pragmatically, it should +also be human-readable. (It may ultimately be desirable to use a +standard interchange base format such as ASDL or XML.) + +In the past, Core has had no rigorously defined external representation, although +by setting certain compiler flags, one can get a (rather ad-hoc) textual +representation to be printed at various points in the compilation process; +this is usually done to help debug the compiler. To make Core fully useable +a bi-directional communication format, it will be necssary to + +\begin{enumerate} +\item define precisely the external format of Core; + +\item modify GHC to produce external Core files, if so requested, at one or more +useful points in the compilation sequence -- e.g., just before optimization, +or just after; + +\item modify GHC to accept external Core files in place of Haskell +source files, again at one or more useful points. + +\end{enumerate} + +The first two facilities will let one couple GHC's front-end (parser, +type-checker, etc.), and optionally its optimizer, with new back-end tools. +Adding the last facility will let one implement new Core-to-Core +transformations in an external tool and integrate them into GHC. It will also +allow new front-ends to generate Core that can be fed into GHC's optimizer or +back end; however, because there are many (undocumented) +idiosynracies in the way GHC produces Core from source Haskell, it will be hard +for an external tool to produce Core that can be integrated with GHC-produced core +(e.g., for the Prelude), and we don't aim to support this. + +This document addresses the first requirement, a formal Core definition, +by proposing a formal grammar for an external representation of Core +(Section~\ref{sec:external}, and +an informal semantics (Section~\ref{sec:informal}. + +Beginning in GHC5.02, external Core (post-optimization) adhering to this definition +can be generated using the compiler flag @-fext-core@. + +Formal static and dynamic semantics in the form of an executable typechecker and interpreter +are available separately in the GHC source tree under @fptools/ghc/utils/ext-core@. + +\section{External Grammar of Core} +\label{sec:external} + +In designing the external grammar, we have tried to strike a balance among +a number of competing goals, including easy parseability by machines, +easy readability by humans, and adequate structural simplicity to +allow straightforward presentations of the semantics. This has inevitably +led to certain compromise. In particular: + +\begin{itemize} +\item In order to avoid explosion of parentheses, various standard precedences +and short-cuts are supported for expressions, types, and kinds; this led to the introduction +of multiple non-terminals for each of these syntactic categories, which +makes the concrete grammar longer and more complex than the underlying abstract syntax. + +\item On the other hand, the grammar has been kept simpler by avoiding special syntax for +tuple types and terms; tuples (both boxed and unboxed) are treated +as ordinary constructors. + +\item All type abstractions and applications are given in full, even though +some of them (e.g., for tuples) could be reconstructed; this permits Core to +be parsed without the necessity of performing any type reconstruction. + +\item The syntax of identifiers is heavily restricted (essentially to just +alphanumerics); this again makes Core easier to parse but harder to read. +\end{itemize} + +\workingnote{These choices are certainly debatable. In particular, keeping +type applications on tuples and case arms considerably increases the size of core files and +makes them less human-readable, though it allows a Core parser to be simpler.} + +We use the following notational conventions for syntax: + +\begin{tabular}{ll} +{\it [ pat ]} & optional \\ +{\it \{ pat \}} & zero or more repetitions \\ +{\it \{ pat \}$^{+}$} & one or more repetitions \\ +{\it pat$_1$ \orderives\ pat$_2$} & choice \\ +@fibonacci@ & terminal syntax in typewriter font \\ +\end{tabular} + +{\it +\begin{tabular}{lrclr} +{\rm Module} & module & \derives & + \multicolumn{2}{l}{@\%module@ mident \many{tdef @;@} \many{\optional{@\%local@} vdefg @;@}} \\ +\\ +{\rm Type defn.} & tdef & \derives & @%data@ qtycon \many{tbind} @=@ @{@ cdef \many{@;@ cdef} @}@ & {\rm algebraic type}\\ + & & \orderives & @%newtype@ qtycon \many{tbind} \optional{@=@ ty} & {\rm newtype} \\ +\\ +{\rm Constr. defn.} & cdef & \derives & qdcon \many{@\at@ tbind} \many{aty} \\ +\\ +{\rm Value defn.} & vdefg & \derives & @%rec@ @{@ vdef \many{@;@ vdef} @}@ & {\rm recursive} \\ + & & \orderives & vdef & {\rm non-recursive} \\ + & vdef & \derives & qvar @::@ ty @=@ exp & \\ +\\ +{\rm Atomic expr.} & aexp & \derives & qvar & {\rm variable} \\ + & & \orderives & qdcon & {\rm data constructor}\\ + & & \orderives & lit & {\rm literal} \\ + & & \orderives & @(@ exp @)@ & {\rm nested expr.}\\ +\\ +{\rm Expression} & exp & \derives & aexp & {\rm atomic expresion}\\ + & & \orderives & aexp \oneormore{arg} & {\rm application}\\ + & & \orderives & @\@ \oneormore{binder} @->@ exp & {\rm abstraction}\\ + & & \orderives & @%let@ vdefg @%in@ exp & {\rm local definition}\\ + & & \orderives & @%case@ exp @%of@ vbind @{@ alt \many{@;@ alt} @}@ & {\rm case expression}\\ + & & \orderives & @%coerce@ aty exp & {\rm type coercion}\\ + & & \orderives & @%note@ @"@ \many{char} @"@ exp & {\rm expression note}\\ + & & \orderives & @%external@ @"@ \many{char} @"@ aty & {\rm external reference}\\ +\\ +{\rm Argument} & arg & \derives & \at\ aty & {\rm type argument}\\ + & & \orderives & aexp & {\rm value argument} \\ +\\ +{\rm Case alt.} & alt & \derives & qdcon \many {@\at@ tbind} \many{vbind} @->@ exp &{\rm constructor alternative}\\ + & & \orderives & lit @->@ exp & {\rm literal alternative} \\ + & & \orderives & @%_@ @->@ exp & {\rm default alternative} \\ +\\ +{\rm Binder} & binder & \derives & \at\ tbind & {\rm type binder}\\ + & & \orderives & vbind & {\rm value binder}\\ +\\ +{\rm Type binder} & tbind & \derives & tyvar & {\rm implicitly of kind @*@} \\ + & & \orderives & @(@ tyvar @::@ kind @)@ & {\rm explicitly kinded} \\ +\\ +{\rm Value binder} & vbind & \derives & @(@ var @::@ ty @)@ \\ +\\ +{\rm Literal} & lit & \derives & @(@ [@-@] \oneormore{digit} @::@ ty @)@ & {\rm integer} \\ + & & \orderives & @(@ [@-@] \oneormore{digit} @.@ \oneormore{digit} @::@ ty @)@ & {\rm rational} \\ + & & \orderives & @(@ @'@ char @'@ @::@ ty @)@ & {\rm character} \\ + & & \orderives & @(@ @"@ \many{char} @"@ @::@ ty @)@ & {\rm string} \\ +\\ +{\rm Character} & char & \derives & \multicolumn{2}{l}{any ASCII character in range 0x20-0x7E except 0x22,0x27,0x5c}\\ + & & \orderives & @\x@ hex hex & {\rm ASCII code escape sequence} \\ + & hex & \derives & @0@ \orderives \ldots \orderives @9@ \orderives @a@ \orderives \ldots \orderives @f@ \\ +\end{tabular} + +\begin{tabular}{lrclr} +{\rm Atomic type} & aty & \derives & tyvar & {\rm type variable} \\ + & & \orderives & qtycon & {\rm type constructor}\\ + & & \orderives & @(@ ty @)@ & {\rm nested type}\\ +\\ +{\rm Basic type} & bty & \derives & aty & {\rm atomic type}\\ + & & \orderives & bty aty & {\rm type application}\\ +\\ +{\rm Type} & ty & \derives & bty & {\rm basic type}\\ + & & \orderives & @%forall@ \oneormore{tbind} @.@ ty & {\rm type abstraction}\\ + & & \orderives & bty @->@ ty & {\rm arrow type construction} \\ +\\ +{\rm Atomic kind} & akind & \derives & @*@ & {\rm lifted kind}\\ + & & \orderives & @#@ & {\rm unlifted kind}\\ + & & \orderives & @?@ & {\rm open kind}\\ + & & \orderives & @(@ kind @)@& {\rm nested kind}\\ +\\ +{\rm Kind} & kind & \derives & akind & {\rm atomic kind}\\ + & & \orderives & akind @->@ kind & {\rm arrow kind} \\ +\\ +{\rm Identifier} & mident & \derives &uname & {\rm module} \\ + & tycon & \derives & uname & {\rm type constr.} \\ + & qtycon & \derives & mident @.@ tycon & {\rm qualified type constr.} \\ + & tyvar & \derives & lname & {\rm type variable} \\ + & dcon & \derives & uname & {\rm data constr.} \\ + & qdcon & \derives & mident @.@ dcon & {\rm qualified data constr.} \\ + & var & \derives & lname & {\rm variable} \\ + & qvar & \derives & [ mident @.@ ] var & {\rm optionally qualified variable} \\ +\\ +{\rm Name} & lname & \derives & lower \many{namechar} \\ + & uname & \derives & upper \many{namechar} & \\ + & namechar & \derives & lower \orderives\ upper \orderives\ digit \orderives\ @'@ \\ + & lower & \derives & @a@ \orderives\ @b@ \orderives\ \ldots \orderives\ @z@ \orderives\ @_@ \\ + & upper & \derives & @A@ \orderives\ @B@ \orderives\ \ldots \orderives\ @Z@ \\ + & digit & \derives & @0@ \orderives\ @1@ \orderives\ \ldots \orderives\ @9@ \\ +\\ +\end{tabular} +} + +\workingnote{Should add some provision for comments.} + +\section{Informal Semantics} +\label{sec:informal} + +Core resembles a explicitly-typed polymorphic lambda calculus (F$_\omega$), with the addition +of local @let@ bindings, algebraic type definitions, constructors, and @case@ expressions, +and primitive types, literals and operators. +It is hoped that this makes it easy to obtain an informal understanding of Core programs +without elaborate description. This section therefore concentrates on the less obvious points. + +\subsection{Program Organization and Modules} + +Core programs are organized into {\em modules}, corresponding directly to source-level Haskell modules. +Each module has a identifying name {\it mident}. + +Each module may contain the following kinds of top-level declarations: +\begin{itemize} +\item Algebraic data type declarations, each defining a type constructor and one or more data constructors; +\item Newtype declarations, corresponding to Haskell @newtype@ declarations, each defining a type constructor; and +\item Value declarations, defining the types and values of top-level variables. +\end{itemize} +No type constructor, data constructor, or top-level value may be declared more than once within a given module. +All the type declarations are (potentially) mutually recursive. Value declarations must be +in dependency order, with explicit grouping of mutually recursive declarations. + +Identifiers defined in top-level declarations may be {\it external} or {\it internal}. +External identifiers can be referenced from any other module in +the program, using conventional dot notation (e.g., @PrelBase.Bool@, @PrelBase.True@). +Internal identifiers are visible only within the defining module. +All type and data constructors are external, and are always defined and referenced using +fully qualified names (with dots). A top-level value is external if it is defined and referenced +using a fully qualified name with a dot (e.g., @MyModule.foo = ...@); otherwise, it is internal +(e.g., @bar = ...@). +Note that the notion of external identifier does not necessarily coincide with that of ``exported'' +identifier in a Haskell source module: all constructors are external, even if not exported, and +non-exported values may be external if they are referenced from potentially in-lineable exported values. +Core modules have no explicit import or export lists. +Modules may be mutually recursive. + +\workingnote{But in the presence of inter-module recursion, is there much point in +keeping track of recursive groups within modules? Options: (1) don't worry about it; +(2) put all declarations in module (indeed whole program) into one huge recursive pot; +(3) abandon general module recursion, and introduce some kind of import declaration to define the +types (only) of things from external modules that currently introduce module recursion.} + +There is also an implicitly-defined module @PrelGHC@, which exports the ``built-in'' types and values +that must be provided by any implementation of Core (including GHC). Details of this +module are in Section~\ref{sec:prims}. + +A Core {\em program} is a collection of distinctly-named modules that includes a module +called @Main@ having an exported value called @main@ of type @PrelIOBase.IO a@ (for some type @a@). + +Many modules of interest derive from library modules, such as @PrelBase@, which implement parts of +the Haskell basis library. In principle, these modules have no special status. In practice, the +requirement on the type of @Main.main@ implies that every program will contain a large subset of +the Prelude library modules. + +\subsection{Namespaces} + +There are five distinct name spaces: +\begin{enumerate} +\item module identifiers (@mident@), +\item type constructors (@tycon@), +\item type variables (@tyvar@), +\item data constructors (@dcon@), +\item term variables (@var@). +\end{enumerate} +Spaces (1), (2+3), and (4+5) can be distinguished from each other by context. +To distinguish (2) from (3) and (4) from (5), we require that +(both sorts of) constructors begin with an upper-case character +and that (both sorts of) variables begin with a lower-case character (or @_@). +Primitive types and operators are not syntactically distinguished. + +A given variable (type or term) may have multiple (local) definitions within a module. +However, definitions never ``shadow'' one another; that is, the scope of the definition +of a given variable never contains a redefinition of the same variable. The only exception +to this is that (necessarily closed) types labelling @%external@ expressions may contain +@tyvar@ bindings that shadow outer bindings. + +Core generated by GHC makes heavy use of encoded names, in which the characters @Z@ and @z@ are +used to introduce escape sequences for non-alphabetic characters such as dollar sign @$@ (@zd@), +hash @#@ (@zh@), plus @+@ (@zp@), etc. This is the same encoding used in @.hi@ files and in the +back-end of GHC itself, except that we sometimes change an initial @z@ to @Z@, or vice-versa, +in order to maintain case distinctions. + +\subsection{Types and Kinds} + +In Core, all type abstractions and applications are explicit. This make it easy to +typecheck any (closed) fragment. An full executable typechecker is available separately. + +Types are described by type expressions, which +are built from named type constructors and type variables +using type application and universal quantification. +Each type constructor has a fixed arity $\geq 0$. +Because it is so widely used, there is +special infix syntax for the fully-applied function type constructor (@->@). +(The prefix identifier for this constructor is @PrelGHC.ZLzmzgZR@; this should +only appear in unapplied or partially applied form.) +There are also a number of other primitive type constructors (e.g., @Intzh@) that +are predefined in the @PrelGHC@ module, but have no special syntax. +Additional type constructors are +introduced by @%data@ and @%newtype@ declarations, as described below. +Type constructors are distinguished solely by name. + +As described in the Haskell definition, it is necessary to distinguish +well-formed type-expressions by classifying them into different {\it kinds}. +In particular, Core explicitly records the kind of every bound type variable. +Base kinds (@*@,@#@, and @?@) represent actual types, i.e., those that can be assigned +to term variables; all the nullary type constructors have one of these kinds. +Non-nullary type constructors have higher kinds of the form $k_1 @->@ k_2$, +where $k_1$ and $k_2$ are kinds. For example, the function type constructor +@->@ has kind @* -> (* -> *)@. Since Haskell allows abstracting over type +constructors, it is possible for type variables to have higher kinds; however, +it is much more common for them to have kind @*@, so this is the default if +the kind is omitted in a type binder. + +The three base kinds distinguish the {\it liftedness} of the types they classify: +@*@ represents lifted types; @#@ represents unlifted types; and @?@ represents +``open'' types, which may be either lifted or unlifted. Of these, only @*@ ever +appears in Core code generated from user code; the other two are needed to describe +certain types in primitive (or otherwise specially-generated) code. +Semantically, a type is lifted if and only if it has bottom as an element. +Operationally, lifted types may be represented by closures; hence, any unboxed +value is necessarily unlifted. +In particular, no top-level identifier (except in @PrelGHC@) has a type of kind @#@ or @?@. +Currently, all the primitive types are unlifted +(including a few boxed primitive types such as @ByteArrayzh@). +The ideas behind the use of unboxed and unlifted types are described in ~\cite{pj:unboxed}. + +There is no mechanism for defining type synonyms (corresponding to +Haskell @type@ declarations). +Type equivalence is just syntactic equivalence on type expressions +(of base kinds) modulo: + +\begin{itemize} +\item alpha-renaming of variables bound in @%forall@ types; +\item the identity $a$ @->@ $b$ $\equiv$ @PrelGHC.ZLzmzgZR@ $a$ $b$ +\item the substitution of representation types for {\it fully applied} instances of newtypes +(see Section~\ref{sec:newtypes}). +\end{itemize} + +\subsection{Algebraic data types} + +Each @data@ declaration introduces a new type constructor and a set of one or +more data constructors, normally corresponding directly to a source Haskell @data@ declaration. +For example, the source declaration +\begin{code} +data Bintree a = + Fork (Bintree a) (Bintree a) +| Leaf a +\end{code} +might induce the following Core declaration +\begin{code} +%data Bintree a = { + Fork (Bintree a) (Bintree a); + Leaf a)} +\end{code} +which introduces the unary type constructor @Bintree@ of kind @*->*@ and two data constructors with types +\begin{code} +Fork :: %forall a . Bintree a -> Bintree a -> Bintree a +Leaf :: %forall a . a -> Bintree a +\end{code} +We define the {\it arity} of each data constructor to be the number of value arguments it takes; +e.g. @Fork@ has arity 2 and @Leaf@ has arity 1. + +For a less conventional example illustrating the possibility of higher-order kinds, the Haskell source declaration +\begin{code} +data A f a = MkA (f a) +\end{code} +might induce the core declaration +\begin{code} +%data A (f::*->*) (a::*) = { MkA (f a) } +\end{code} +which introduces the constructor +\begin{code} +MkA :: %forall (f::*->*) (a::*) . (f a) -> (A f) a +\end{code} + + +GHC (like some other Haskell implementations) supports an extension to Haskell98 +for existential types such as +\begin{code} +data T = forall a . MkT a (a -> Bool) +\end{code} +This is represented by the Core declaration +\begin{code} +%data T = {MkT @a a (a -> Bool)} +\end{code} +which introduces the nullary type constructor @T@ and the data constructor +\begin{code} +MkT :: %forall a . a -> (a -> Bool) -> T +\end{code} +In general, existentially quantified variables appear as extra univerally +quantified variables in the data contructor types. +An example of how to construct and deconstruct values of type @T@ is shown in +Section~\ref{sec:exprs}. + +\subsection{Newtypes} +\label{sec:newtypes} + + +Each Core @%newtype@ declaration introduces a new type constructor and (usually) an associated +representation type, corresponding to a source Haskell @newtype@ +declaration. However, unlike in source Haskell, no data constructors are introduced. +In fact, newtypes seldom appear in value types +in Core programs, because GHC usually replaces them with their representation type. +For example, the Haskell fragment +\begin{code} +newtype U = MkU Bool +u = MkU True +v = case u of + MkU b -> not b +\end{code} +might induce the Core fragment +\begin{code} +%newtype U = Bool; +u :: Bool = True; +v :: Bool = + %let b :: Bool = u + %in not b; +\end{code} +The main purpose of including @%newtype@ declarations in Core is to permit checking of +type expressions in which partially-applied newtype constructors are used to instantiate higher-kinded +type variables. For example: +\begin{code} +newtype W a = MkW (Bool -> a) +data S k = MkS (k Bool) +a :: S W = MkS (MkW(\x -> not x)) +\end{code} +might generate this Core: +\begin{code} +%newtype W a = Bool -> a; +%data S (k::(*->*)) = MkS (k Bool); +a :: S W = MkS @ W (\(x::Bool) -> not x) +\end{code} +The type application @(S W)@ cannot be checked without a definition for @W@. + +Very rarely, source @newtype@ declarations may be (directly or indirectly) recursive. In such +cases, it is not possible to subsitute the representation type for the new type; +in fact, the representation type is omitted from the corresponding Core @%newtype@ declaration. +Elements of the new +type can only be created or examined by first explicitly coercing them from/to +the representation type, using a @%coerce@ expression. For example, the silly +Haskell fragment +\begin{code} +newtype U = MkU (U -> Bool) +u = MkU (\x -> True) +v = case u of + MkU f -> f u +\end{code} +might induce the Core fragment +\begin{code} +%newtype U; +u :: U = %coerce U (\ (x::U) -> True); +v :: Bool = + %let f :: U -> Bool = %coerce (U -> Bool) u + %in f u; +\end{code} + +\workingnote{The treatment of newtypes is still very unattractive: acres of explanation for +very rare phenomena.} + +\subsection{Expression Forms} +\label{sec:exprs} + +Variables and data constructors are straightforward. + +Literal ({\it lit}) expressions consist of a literal value, in one of four different formats, +and a (primitive) type annotation. Only certain combinations of format and type +are permitted; see Section~\ref{sec:prims}. The character and string formats can describe only +8-bit ASCII characters. Moreover, because strings are interpreted as C-style null-terminated +strings, they should not contain embedded nulls. + +Both value applications and type applications are made explicit, and similarly +for value and type abstractions. To tell them apart, type arguments in applications +and formal type arguments in abstractions are preceded by an \at\ symbol. (In abstractions, +the \at\ plays essentially the same role as the more usual $\Lambda$ symbol.) +For example, the Haskell source declaration +\begin{code} +f x = Leaf (Leaf x) +\end{code} +might induce the Core declaration +\begin{code} +f :: %forall a . a -> BinTree (BinTree a) = + \ @a (x::a) -> Leaf @(Bintree a) (Leaf @a x) +\end{code} + +Value applications may be of user-defined functions, data constructors, or primitives. +None of these sorts of applications are necessarily saturated (although previously published variants +of Core did require the latter two sorts to be). + +Note that the arguments of type applications are not always of kind @*@. For example, +given our previous definition of type @A@: +\begin{code} +data A f a = MkA (f a) +\end{code} +the source code +\begin{code} +MkA (Leaf True) +\end{code} +becomes +\begin{code} +(MkA @Bintree @Bool) (Leaf @Bool True) +\end{code} + +Local bindings, of a single variable or of a set of mutually recursive variables, +are represented by @%let@ expressions in the usual way. + +By far the most complicated expression form is @%case@. +@%case@ expressions are permitted over values of any type, although they will normally +be algebraic or primitive types (with literal values). +Evaluating a @%case@ forces the evaluation of the expression being +tested (the ``scrutinee''). The value of the scrutinee is bound to the variable +following the @%of@ keyword, which is in scope in all alternatives; +this is useful when the scrutinee is a non-atomic +expression (see next example). + +In an algebraic @%case@, all the case alternatives must be +labeled with distinct data constructors from the algebraic type, followed by +any existential type variable bindings (see below), and +typed term variable bindings corresponding to the data constructor's +arguments. The number of variables must match the data constructor's arity. + +For example, the following Haskell source expression +\begin{code} +case g x of + Fork l r -> Fork r l + t@(Leaf v) -> Fork t t +\end{code} +might induce the Core expression +\begin{code} +%case g x %of (t::Bintree a) + Fork (l::Bintree a) (r::Bintree a) -> + Fork @a r l + Leaf (v::a) -> + Fork @a t t +\end{code} + +When performing a @%case@ over a value of an existentially-quantified algebraic +type, the alternative must include extra local type bindings +for the existentially-quantified variables. For example, given +\begin{code} +data T = forall a . MkT a (a -> Bool) +\end{code} +the source +\begin{code} +case x of + MkT w g -> g w +\end{code} +becomes +\begin{code} +%case x %of (x'::T) + MkT @b (w::b) (g::b->Bool) -> g w +\end{code} + +In a @%case@ over literal alternatives, +all the case alternatives must be distinct literals of the same primitive type. + +The list of alternatives may begin with a +default alternative labeled with an underscore (@%_@), which will be chosen if +none of the other alternative match. The default is optional except for a case +over a primitive type, or when there are no other alternatives. +If the case is over neither an +algebraic type nor a primitive type, the default alternative is the {\it only} +one that can appear. +For algebraic cases, the set of alternatives +need not be exhaustive, even if no default is given; if alternatives are missing, +this implies that GHC has deduced that they cannot occur. + +The @%coerce@ expression is primarily used in conjunction with manipulation of +newtypes, as described in Section~\ref{sec:newtypes}. +However, @%coerce@ is sometimes used for +other purposes, e.g. to coerce the return type of a function (such as @error@) +that is guaranteed never to return. By their natures, uses of @%coerce@ cannot +be independently justified, and must be taken on faith by a type-checker for Core. + +A @%note@ expression is used to carry arbitrary internal information of interest to +GHC. The information must be encoded as a string. Expression notes currently generated by GHC +include the inlining pragma (@InlineMe@) and cost-center labels for profiling. + +A @%external@ expression denotes an external identifier, which has +the indicated type (always expressed in terms of Haskell primitive types). +\workingnote{The present syntax is sufficient for describing C functions and labels. +Interfacing to other languages may require additional information or a different interpretation +of the name string.} + + +\subsection{Expression Evaluation} + +The dynamic semantics of Core are defined on the type-erasure of the program; +ie. we ignore all type abstractions and applications. The denotational semantics +the resulting type-free program are just the conventional ones for a call-by-name +language, in which expressions are only evaluated on demand. +But Core is intended to be a call-by-{\it{need}} language, in which +expressions are only evaluated {\it once}. To express the sharing behavior +of call-by-need, we give an operational model in the style of Launchbury. +This section describes the model informally; a more formal semantics is +separately available in the form of an executable interpreter. + +To simplify the semantics, we consider only ``well-behaved'' Core programs in which +constructor and primitive applications are fully saturated, and in which +non-trivial expresssions of unlifted kind (@#@) appear only as scrutinees +in @%case@ expressions. Any program can easily be put into this form; +a separately available executable preprocessor illustrates how. +In the remainder of this section, we use ``Core'' to mean ``well-behaved'' Core. + +Evaluating a Core expression means reducing it to {\it weak-head normal form (WHNF)}, +i.e., a primitive value, lambda abstraction, or fully-applied data constructor. +Evaluation of a program is evaluation of the expression @Main.main@. + +To make sure that expression evaluation is shared, we +make use of a {\it heap}, which can contain +\begin{itemize} +\item {\em Thunks} representing suspended (i.e., as yet unevaluated) expressions. + +\item {\em WHNF}s representing the result of evaluating such thunks. Computations over +primitive types are never suspended, so these results are always closures (representing +lambda abstractions) or data constructions. +\end{itemize} +Thunks are allocated when it +is necessary to suspend a computation whose result may be shared. +This occurs when evaluating three different kinds of expressions: +\begin{itemize} +\item Value definitions at top-level or within a local @let@ expression. +Here, the defining expressions are suspended and the defined names +are bound to heap pointers to the suspensions. + +\item User function applications. Here, the actual argument expression is +suspended and the formal argument is bound to a heap pointer to the suspension. + +\item Constructor applications. Here, the actual argument expression is +suspended and a heap pointer to the suspension is embedded in the constructed value. +\end{itemize} + +As computation proceeds, copies of the heap pointer propagate. +When the computation is eventually forced, the heap entry is overwritten with the resulting +WHNF, so all copies of the pointer now point to this WHNF. Forcing occurs +only in the context of +\begin{itemize} +\item evaluating the operator expression of an application; + +\item evaluating the ``scrutinee'' of a @case@ expression; or + +\item evaluating an argument to a primitive or external function application +\end{itemize} + +Ultimately, if there are no remaining pointers to the heap entry (whether suspended or evaluated), +the entry can be garbage-collected; this is assumed to happen implicitly. + +With the exception of functions, arrays, and mutable variables, the intention is that values of all primitive types +should be held {\it unboxed}, i.e., not heap-allocated. This causes no problems for laziness because all +primitive types are {\it unlifted}. Unboxed tuple types are not heap-allocated either. + +Certain primitives and @%external@ functions cause side-effects to state threads or to the real world. +Where the ordering of these side-effects matters, Core already forces this order +by means of data dependencies on the psuedo-values representing the threads. + +The @raisezh@ and @handlezh@ primitives requires special support in an implementation, such as a handler stack; +again, real-world threading guarantees that they will execute in the correct order. + +\section{Primitive Module} +\label{sec:prims} + +This section describes the contents and informal semantics of the primitive module @PrimGHC@. +Nearly all the primitives are required in order to cover GHC's implementation of the Haskell98 +standard prelude; the only operators that can be completely omitted are those supporting the byte-code interpreter, +parallelism, and foreign objects. Some of the concurrency primitives are needed, but can be +given degenerate implementations if it desired to target a purely sequential backend; see Section~\ref{sec:sequential}. + +In addition to these primitives, a large number of C library functions are required to implement +the full standard Prelude, particularly to handle I/O and arithmetic on less usual types. +% We list these separately in section~\ref{sec:ccalls}. + +\subsection{Types} + +\begin{tabular}{|l|l|l|} +\hline +Type & Kind & Description \\ +\hline +@ZLzmzgZR@ & @* -> * -> *@ & functions (@->@) \\ +@Z1H@ & @? -> #@ & unboxed 1-tuple \\ +@Z2H@ & @? -> ? -> #@ & unboxed 2-tuple \\ +\ldots & \ldots & \ldots \\ +@Z100H@ & @? -> ? -> ? -> ... -> ? -> #@ & unboxed 100-tuple \\ +@Addrzh@ & @#@ & machine address (pointer) \\ +@Charzh@ & @#@ & unicode character (31 bits) \\ +@Doublezh@ & @#@ & double-precision float \\ +@Floatzh@ & @#@ & float \\ +@Intzh@ & @#@ & int (30+ bits) \\ +@Int32zh@ & @#@ & int (32 bits) \\ +@Int64zh@ & @#@ & int (64 bits) \\ +@Wordzh@ & @#@ & unsigned word (30+ bits) \\ +@Word32zh@ & @#@ & unsigned word (32 bits) \\ +@Word64zh@ & @#@ & unsigned word (64 bits) \\ +@RealWorld@ & @*@ & pseudo-type for real world state \\ +@Statezh@ & @* -> #@ & mutable state \\ +@Arrayzh@ & @* -> #@ & immutable arrays \\ +@ByteArrayzh@ & @#@ & immutable byte arrays \\ +@MutableArrayzh@ & @* -> * -> #@ & mutable arrays \\ +@MutableByteArrayzh@ & @* -> #@ & mutable byte arrays \\ +@MutVarzh@ & @* -> * -> #@ & mutable variables \\ +@MVarzh@ & @* -> * -> #@ & synchronized mutable variables \\ +@Weakzh@ & @* -> #@ & weak pointers \\ +@StablePtrzh@ & @* -> #@ & stable pointers \\ +@ForeignObjzh@ & @#@ & foreign object \\ +@ThreadIdzh@ & @#@ & thread id \\ +@ZCTCCallable@ & @? -> *@ & dictionaries for CCallable pseudo-class \\ +@ZCTCReturnable@ & @? -> *@ & dictionaries for CReturnable pseudo-class \\ +\hline +\end{tabular} + +In addition, the types @PrelBase.Bool@ and @PrelBase.Unit@, which are non-primitive +and are defined as ordinary algebraic types in module @PrelBase@, are used in +the types of some operators in @PrelGHC@. + +The unboxed tuple types are quite special: they hold sets of values in an unlifted +context, i.e., to be manipulated directly rather than being stored in the heap. They can only +appear in limited contexts in programs; in particular, they cannot be bound by a +lambda abstraction or case alternative pattern. Note that they can hold either lifted +or unlifted values. The limitation to 100-tuples is an arbitrary one set by GHC. + +The type of arbitrary precision integers (@Integer@) is not primitive; it is made +up of an ordinary primitive integer (@Intzh@) and a byte array (@ByteArrzh@). +The components of an @Integer@ are passed to primitive operators as two separate +arguments and returned as an unboxed pair. + +The @Statezh@ type constructor takes a dummy type argument that is used only +to distinguish different state {\it threads}~\cite{Launchbury94}. +The @RealWorld@ type is used only as an argument to @Statezh@, and represents +the thread of real-world state; it contains just the single value @realWorldzh@. +The mutable data types @MutableArrayzh@,@MutableByteArrayzh@,@MutVarzh@ +take an initial type argument of the form @(Statezh@ $t$@)@ for some thread $t$. +The synchronized mutable variable type constructor @MVarzh@ always takes an argument of type +@Statezh RealWorld@. + +@Weakzh@ is the type of weak pointers. + +@StablePtrzh@ is the type of stable pointers, which are guaranteed not to move +during garbage collections; these are useful in connection with foreign functions. + +@ForeignPtrzh@ is the type of foreign pointers. + +The dictionary types @ZCTCCallable@ and @ZCTCReturnable@ are just placeholders +which can be represented by a void type; +any code they appear in should be unreachable. + +\subsubsection{Non-concurrent Back End} +\label{sec:sequential} + +The Haskell98 standard prelude doesn't include any concurrency support, but GHC's +implementation of it relies on the existence of some concurrency primitives. However, +it never actually forks multiple threads. Hence, the concurrency primitives can +be given degenerate implementations that will work in a non-concurrent setting, +as follows: +\begin{itemize} +\item @ThreadIdzh@ can be represented +by a singleton type, whose (unique) value is returned by @myThreadIdzh@. + +\item @forkzh@ can just die with an ``unimplemented'' message. + +\item @killThreadzh@ and @yieldzh@ can also just die ``unimplemented'' since +in a one-thread world, the only thread a thread can kill is itself, and +if a thread yields the program hangs. + +\item @MVarzh a@ can be represented by @MutVarzh (Maybe a)@; +where a concurrent implementation would block, the sequential implementation can +just die with a suitable message (since no other thread exists to unblock it). + +\item @waitReadzh@ and @waitWritezh@ can be implemented using a @select@ with no timeout. +\end{itemize} + +\subsection{Literals} + +Only the following combination of literal forms and types are permitted: + +\begin{tabular}{|l|l|l|} +\hline +Literal form & Type & Description \\ +\hline +integer & @Intzh@ & Int \\ +% & @Int32zh@ & Int32 \\ +% & @Int64zh@ & Int64 \\ + & @Wordzh@ & Word \\ +% & @Word32zh@ & Word32 \\ +% & @Word64zh@ & Word64 \\ + & @Addrzh@ & Address \\ + & @Charzh@ & Unicode character code \\ +rational & @Floatzh@ & Float \\ + & @Doublezh@ & Double \\ +character & @Charzh@ & Unicode character specified by ASCII character\\ +string & @Addrzh@ & Address of specified C-format string \\ +\hline +\end{tabular} + +\subsection{Data Constructors} + +The only primitive data constructors are for unboxed tuples: + +\begin{tabular}{|l|l|l|} +\hline +Constructor & Type & Description \\ +\hline +@ZdwZ1H@ & @%forall (a::?).a -> Z1H a@ & unboxed 1-tuple \\ +@ZdwZ2H@ & @%forall (a1::?) (a2::?).a1 -> a2 -> Z2H a1 a2@ & unboxed 2-tuple \\ +\ldots & \ldots & \ldots \\ +@ZdwZ100H@ & @%forall (a1::?) (a2::?)... (a100::?) .@ & \\ +& \ \ \ @a1 -> a2 -> ... -> a100 -> Z100H a1 a2 ... a100@ & unboxed 100-tuple \\ +\hline +\end{tabular} + +\subsection{Values} + +Operators are (roughly) divided into collections according to the primary +type on which they operate. + +\workingnote{How do primitives fail, e.g., on division by zero or +attempting an invalid narrowing coercion?} + +\workingnote{The following primop descriptions are automatically generated. +The exact set of primops and their types presented here +depends on the underlying word size at the time of generation; these +were done for 32 bit words. This is a bit stupid. +More importantly, the word size has a big impact on just what gets produced +in a Core file, but this isn't documented anywhere in the file itself. +Perhaps there should be a global flag in the file?} + +\newcommand{\primoptions}[7]{{#1} {#2} {#3} {#4} {#5}} + +\newcommand{\primopsection}[2]{\subsubsection{#1}{#2}\vspace*{0.1in}} +\newcommand{\primopdefaults}[1]{Unless otherwise noted, each primop has the following default characteristics: {#1}} + +\newcommand{\primopdesc}[8]{ +\par\noindent{\texttt{{{#3} :: {#6}}}} +\\{#7} {#8}\\} + +\input{prims.tex} + +\subsubsection{RealWorld} + +There is just one value of type @RealWorld@, namely @realWorldzh@. It is used +only for dependency threading of side-effecting operations. + +\begin{thebibliography}{} + +\bibitem[Launchbury and {Peyton~Jones}, 1994]{Launchbury94} +Launchbury, J. and {Peyton~Jones}, S. (1994). +\newblock Lazy functional state threads. +\newblock Technical report FP-94-05, Department of Computing Science, + University of Glasgow. + +\bibitem[{Peyton~Jones} and Launchbury, 1991]{pj:unboxed} +{Peyton~Jones}, S. and Launchbury, J. (1991). +\newblock Unboxed values as first class citizens. +\newblock In {\em ACM Conference on Functional Programming and Computer + Architecture (FPCA'91)}, pages 636--666, Boston. ACM. + +\bibitem[{Peyton~Jones} and Marlow, 1999]{ghc-inliner} +{Peyton~Jones}, S. and Marlow, S. (1999). +\newblock Secrets of the {Glasgow Haskell Compiler} inliner. +\newblock In {\em Workshop on Implementing Declarative Languages}, Paris, + France. + +\bibitem[Peyton~Jones and Santos, 1998]{comp-by-trans-scp} +Peyton~Jones, S. and Santos, A. (1998). +\newblock A transformation-based optimiser for {Haskell}. +\newblock {\em Science of Computer Programming}, 32(1-3):3--47. + +\end{thebibliography} + +\end{document} diff --git a/docs/ghci/ghci.tex b/docs/ghci/ghci.tex new file mode 100644 index 0000000000..c4638a6719 --- /dev/null +++ b/docs/ghci/ghci.tex @@ -0,0 +1,1598 @@ +% +% (c) The OBFUSCATION-THROUGH-GRATUITOUS-PREPROCESSOR-ABUSE Project, +% Glasgow University, 1990-2000 +% + +% \documentstyle[preprint]{acmconf} +\documentclass[11pt]{article} +\oddsidemargin 0.1 in % Note that \oddsidemargin = \evensidemargin +\evensidemargin 0.1 in +\marginparwidth 0.85in % Narrow margins require narrower marginal notes +\marginparsep 0 in +\sloppy + +%\usepackage{epsfig} +\usepackage{shortvrb} +\MakeShortVerb{\@} + +%\newcommand{\note}[1]{{\em Note: #1}} +\newcommand{\note}[1]{{{\bf Note:}\sl #1}} +\newcommand{\ToDo}[1]{{{\bf ToDo:}\sl #1}} +\newcommand{\Arg}[1]{\mbox{${\tt arg}_{#1}$}} +\newcommand{\bottom}{\perp} + +\newcommand{\secref}[1]{Section~\ref{sec:#1}} +\newcommand{\figref}[1]{Figure~\ref{fig:#1}} +\newcommand{\Section}[2]{\section{#1}\label{sec:#2}} +\newcommand{\Subsection}[2]{\subsection{#1}\label{sec:#2}} +\newcommand{\Subsubsection}[2]{\subsubsection{#1}\label{sec:#2}} + +% DIMENSION OF TEXT: +\textheight 8.5 in +\textwidth 6.25 in + +\topmargin 0 in +\headheight 0 in +\headsep .25 in + + +\setlength{\parskip}{0.15cm} +\setlength{\parsep}{0.15cm} +\setlength{\topsep}{0cm} % Reduces space before and after verbatim, + % which is implemented using trivlist +\setlength{\parindent}{0cm} + +\renewcommand{\textfraction}{0.2} +\renewcommand{\floatpagefraction}{0.7} + +\begin{document} + +\title{The GHCi Draft Design, round 2} +\author{MSR Cambridge Haskell Crew \\ + Microsoft Research Ltd., Cambridge} + +\maketitle + +%%%\tableofcontents +%%%\newpage + +%%-----------------------------------------------------------------%% +\section{Details} + +\subsection{Outline of the design} +\label{sec:details-intro} + +The design falls into three major parts: +\begin{itemize} +\item The compilation manager (CM), which coordinates the + system and supplies a HEP-like interface to clients. +\item The module compiler (@compile@), which translates individual + modules to interpretable or machine code. +\item The linker (@link@), + which maintains the executable image in interpreted mode. +\end{itemize} + +There are also three auxiliary parts: the finder, which locates +source, object and interface files, the summariser, which quickly +finds dependency information for modules, and the static info +(compiler flags and package details), which is unchanged over the +course of a session. + +This section continues with an overview of the session-lifetime data +structures. Then follows the finder (section~\ref{sec:finder}), +summariser (section~\ref{sec:summariser}), +static info (section~\ref{sec:staticinfo}), +and finally the three big sections +(\ref{sec:manager},~\ref{sec:compiler},~\ref{sec:linker}) +on the compilation manager, compiler and linker respectively. + +\subsubsection*{Some terminology} + +Lifetimes: the phrase {\bf session lifetime} covers a complete run of +GHCI, encompassing multiple recompilation runs. {\bf Module lifetime} +is a lot shorter, being that of data needed to translate a single +module, but then discarded, for example Core, AbstractC, Stix trees. + +Data structures with module lifetime are well documented and understood. +This document is mostly concerned with session-lifetime data. +Most of these structures are ``owned'' by CM, since that's +the only major component of GHCI which deals with session-lifetime +issues. + +Modules and packages: {\bf home} refers to modules in this package, +precisely the ones tracked and updated by the compilation manager. +{\bf Package} refers to all other packages, which are assumed static. + +\subsubsection*{A summary of all session-lifetime data structures} + +These structures have session lifetime but not necessarily global +visibility. Subsequent sections elaborate who can see what. +\begin{itemize} +\item {\bf Home Symbol Table (HST)} (owner: CM) holds the post-renaming + environments created by compiling each home module. +\item {\bf Home Interface Table (HIT)} (owner: CM) holds in-memory + representations of the interface file created by compiling + each home module. +\item {\bf Unlinked Images (UI)} (owner: CM) are executable but as-yet + unlinked translations of home modules only. +\item {\bf Module Graph (MG)} (owner: CM) is the current module graph. +\item {\bf Static Info (SI)} (owner: CM) is the package configuration + information (PCI) and compiler flags (FLAGS). +\item {\bf Persistent Compiler State (PCS)} (owner: @compile@) + is @compile@'s private cache of information about package + modules. +\item {\bf Persistent Linker State (PLS)} (owner: @link@) is + @link@'s private information concerning the the current + state of the (in-memory) executable image. +\end{itemize} + + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{The finder (\mbox{\tt type Finder})} +\label{sec:finder} + +@Path@ could be an indication of a location in a filesystem, or it +could be some more generic kind of resource identifier, a URL for +example. +\begin{verbatim} + data Path = ... +\end{verbatim} + +And some names. @Module@s are now used as primary keys for various +maps, so they are given a @Unique@. +\begin{verbatim} + type ModName = String -- a module name + type PkgName = String -- a package name + type Module = -- contains ModName and a Unique, at least +\end{verbatim} + +A @ModLocation@ says where a module is, what it's called and in what +form it is. +\begin{verbatim} + data ModLocation = SourceOnly Module Path -- .hs + | ObjectCode Module Path Path -- .o, .hi + | InPackage Module PkgName + -- examine PCI to determine package Path +\end{verbatim} + +The module finder generates @ModLocation@s from @ModName@s. We expect +it will assume packages to be static, but we want to be able to track +changes in home modules during the session. Specifically, we want to +be able to notice that a module's object and interface have been +updated, presumably by a compile run outside of the GHCI session. +Hence the two-stage type: +\begin{verbatim} + type Finder = ModName -> IO ModLocation + newFinder :: PCI -> IO Finder +\end{verbatim} +@newFinder@ examines the package information right at the start, but +returns an @IO@-typed function which can inspect home module changes +later in the session. + + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{The summariser (\mbox{\tt summarise})} +\label{sec:summariser} + +A @ModSummary@ records the minimum information needed to establish the +module graph and determine whose source has changed. @ModSummary@s +can be created quickly. +\begin{verbatim} + data ModSummary = ModSummary + ModLocation -- location and kind + (Maybe (String, Fingerprint)) + -- source and fingerprint if .hs + (Maybe [ModName]) -- imports if .hs or .hi + + type Fingerprint = ... -- file timestamp, or source checksum? + + summarise :: ModLocation -> IO ModSummary +\end{verbatim} + +The summary contains the location and source text, and the location +contains the name. We would like to remove the assumption that +sources live on disk, but I'm not sure this is good enough yet. + +\ToDo{Should @ModSummary@ contain source text for interface files too?} +\ToDo{Also say that @ModIFace@ contains its module's @ModSummary@ (why?).} + + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{Static information (SI)} +\label{sec:staticinfo} + +PCI, the package configuration information, is a list of @PkgInfo@, +each containing at least the following: +\begin{verbatim} + data PkgInfo + = PkgInfo PkgName -- my name + Path -- path to my base location + [PkgName] -- who I depend on + [ModName] -- modules I supply + [Unlinked] -- paths to my object files + + type PCI = [PkgInfo] +\end{verbatim} +The @Path@s in it, including those in the @Unlinked@s, are set up +when GHCI starts. + +FLAGS is a bunch of compiler options. We haven't figured out yet how +to partition them into those for the whole session vs those for +specific source files, so currently the best we can do is: +\begin{verbatim} + data FLAGS = ... +\end{verbatim} + +The static information (SI) is the both of these: +\begin{verbatim} + data SI = SI PCI + FLAGS +\end{verbatim} + + + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{The Compilation Manager (CM)} +\label{sec:manager} + +\subsubsection{Data structures owned by CM} + +CM maintains two maps (HST, HIT) and a set (UI). It's important to +realise that CM only knows about the map/set-ness, and has no idea +what a @ModDetails@, @ModIFace@ or @Linkable@ is. Only @compile@ and +@link@ know that, and CM passes these types around without +inspecting them. + +\begin{itemize} +\item + {\bf Home Symbol Table (HST)} @:: FiniteMap Module ModDetails@ + + The @ModDetails@ (a couple of layers down) contain tycons, classes, + instances, etc, collectively known as ``entities''. Referrals from + other modules to these entities is direct, with no intervening + indirections of any kind; conversely, these entities refer directly + to other entities, regardless of module boundaries. HST only holds + information for home modules; the corresponding wired-up details + for package (non-home) modules are created on demand in the package + symbol table (PST) inside the persistent compiler's state (PCS). + + CM maintains the HST, which is passed to, but not modified by, + @compile@. If compilation of a module is successful, @compile@ + returns the resulting @ModDetails@ (inside the @CompResult@) which + CM then adds to HST. + + CM throws away arbitrarily large parts of HST at the start of a + rebuild, and uses @compile@ to incrementally reconstruct it. + +\item + {\bf Home Interface Table (HIT)} @:: FiniteMap Module ModIFace@ + + (Completely private to CM; nobody else sees this). + + Compilation of a module always creates a @ModIFace@, which contains + the unlinked symbol table entries. CM maintains this @FiniteMap@ + @ModName@ @ModIFace@, with session lifetime. CM never throws away + @ModIFace@s, but it does update them, by passing old ones to + @compile@ if they exist, and getting new ones back. + + CM acquires @ModuleIFace@s from @compile@, which it only applies + to modules in the home package. As a result, HIT only contains + @ModuleIFace@s for modules in the home package. Those from other + packages reside in the package interface table (PIT) which is a + component of PCS. + +\item + {\bf Unlinked Images (UI)} @:: Set Linkable@ + + The @Linkable@s in UI represent executable but as-yet unlinked + module translations. A @Linkable@ can contain the name of an + object, archive or DLL file. In interactive mode, it may also be + the STG trees derived from translating a module. So @compile@ + returns a @Linkable@ from each successful run, namely that of + translating the module at hand. + + At link-time, CM supplies @Linkable@s for the upwards closure of + all packages which have changed, to @link@. It also examines the + @ModSummary@s for all home modules, and by examining their imports + and the SI.PCI (package configuration info) it can determine the + @Linkable@s from all required imported packages too. + + @Linkable@s and @ModIFace@s have a close relationship. Each + translated module has a corresponding @Linkable@ somewhere. + However, there may be @Linkable@s with no corresponding modules + (the RTS, for example). Conversely, multiple modules may share a + single @Linkable@ -- as is the case for any module from a + multi-module package. For these reasons it seems appropriate to + keep the two concepts distinct. @Linkable@s also provide + information about the sequence in which individual package + components should be linked, and that isn't the business of any + specific module to know. + + CM passes @compile@ a module's old @ModIFace@, if it has one, in + the hope that the module won't need recompiling. If so, @compile@ + can just return the new @ModDetails@ created from it, and CM will + re-use the old @ModIFace@. If the module {\em is} recompiled (or + scheduled to be loaded from disk), @compile@ returns both the + new @ModIFace@ and new @Linkable@. + +\item + {\bf Module Graph (MG)} @:: known-only-to-CM@ + + Records, for CM's purposes, the current module graph, + up-to-dateness and summaries. More details when I get to them. + Only contains home modules. +\end{itemize} +Probably all this stuff is rolled together into the Persistent CM +State (PCMS): +\begin{verbatim} + data PCMS = PCMS HST HIT UI MG + emptyPCMS :: IO PCMS +\end{verbatim} + +\subsubsection{What CM implements} +It pretty much implements the HEP interface. First, though, define a +containing structure for the state of the entire CM system and its +subsystems @compile@ and @link@: +\begin{verbatim} + data CmState + = CmState PCMS -- CM's stuff + PCS -- compile's stuff + PLS -- link's stuff + SI -- the static info, never changes + Finder -- the finder +\end{verbatim} + +The @CmState@ is threaded through the HEP interface. In reality +this might be done using @IORef@s, but for clarity: +\begin{verbatim} + type ModHandle = ... (opaque to CM/HEP clients) ... + type HValue = ... (opaque to CM/HEP clients) ... + + cmInit :: FLAGS + -> [PkgInfo] + -> IO CmState + + cmLoadModule :: CmState + -> ModName + -> IO (CmState, Either [SDoc] ModHandle) + + cmGetExpr :: ModHandle + -> CmState + -> String -> IO (CmState, Either [SDoc] HValue) + + cmRunExpr :: HValue -> IO () -- don't need CmState here +\end{verbatim} +Almost all the huff and puff in this document pertains to @cmLoadModule@. + + +\subsubsection{Implementing \mbox{\tt cmInit}} +@cmInit@ creates an empty @CmState@ using @emptyPCMS@, @emptyPCS@, +@emptyPLS@, making SI from the supplied flags and package info, and +by supplying the package info the @newFinder@. + + +\subsubsection{Implementing \mbox{\tt cmLoadModule}} + +\begin{enumerate} +\item {\bf Downsweep:} using @finder@ and @summarise@, chase from + the given module to + establish the new home module graph (MG). Do not chase into + package modules. +\item Remove from HIT, HST, UI any modules in the old MG which are + not in the new one. The old MG is then replaced by the new one. +\item Topologically sort MG to generate a bottom-to-top traversal + order, giving a worklist. +\item {\bf Upsweep:} call @compile@ on each module in the worklist in + turn, passing it + the ``correct'' HST, PCS, the old @ModIFace@ if + available, and the summary. ``Correct'' HST in the sense that + HST contains only the modules in the this module's downward + closure, so that @compile@ can construct the correct instance + and rule environments simply as the union of those in + the module's downward closure. + + If @compile@ doesn't return a new interface/linkable pair, + compilation wasn't necessary. Either way, update HST with + the new @ModDetails@, and UI and HIT respectively if a + compilation {\em did} occur. + + Keep going until the root module is successfully done, or + compilation fails. + +\item If the previous step terminated because compilation failed, + define the successful set as those modules in successfully + completed SCCs, i.e. all @Linkable@s returned by @compile@ excluding + those from modules in any cycle which includes the module which failed. + Remove from HST, HIT, UI and MG all modules mentioned in MG which + are not in the successful set. Call @link@ with the successful + set, + which should succeed. The net effect is to back off to a point + in which those modules which are still aboard are correctly + compiled and linked. + + If the previous step terminated successfully, + call @link@ passing it the @Linkable@s in the upward closure of + all those modules for which @compile@ produced a new @Linkable@. +\end{enumerate} +As a small optimisation, do this: +\begin{enumerate} +\item[3a.] Remove from the worklist any module M where M's source + hasn't changed and neither has the source of any module in M's + downward closure. This has the effect of not starting the upsweep + right at the bottom of the graph when that's not needed. + Source-change checking can be done quickly by CM by comparing + summaries of modules in MG against corresponding + summaries from the old MG. +\end{enumerate} + + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{The compiler (\mbox{\tt compile})} +\label{sec:compiler} + +\subsubsection{Data structures owned by \mbox{\tt compile}} + +{\bf Persistent Compiler State (PCS)} @:: known-only-to-compile@ + +This contains info about foreign packages only, acting as a cache, +which is private to @compile@. The cache never becomes out of +date. There are three parts to it: + + \begin{itemize} + \item + {\bf Package Interface Table (PIT)} @:: FiniteMap Module ModIFace@ + + @compile@ reads interfaces from modules in foreign packages, and + caches them in the PIT. Subsequent imports of the same module get + them directly out of the PIT, avoiding slow lexing/parsing phases. + Because foreign packages are assumed never to become out of date, + all contents of PIT remain valid forever. @compile@ of course + tries to find package interfaces in PIT in preference to reading + them from files. + + Both successful and failed runs of @compile@ can add arbitrary + numbers of new interfaces to the PIT. The failed runs don't matter + because we assume that packages are static, so the data cached even + by a failed run is valid forever (ie for the rest of the session). + + \item + {\bf Package Symbol Table (PST)} @:: FiniteMap Module ModDetails@ + + Adding an package interface to PIT doesn't make it directly usable + to @compile@, because it first needs to be wired (renamed + + typechecked) into the sphagetti of the HST. On the other hand, + most modules only use a few entities from any imported interface, + so wiring-in the interface at PIT-entry time might be a big time + waster. Also, wiring in an interface could mean reading other + interfaces, and we don't want to do that unnecessarily. + + The PST avoids these problems by allowing incremental wiring-in to + happen. Pieces of foreign interfaces are copied out of the holding + pen (HP), renamed, typechecked, and placed in the PST, but only as + @compile@ discovers it needs them. In the process of incremental + renaming/typechecking, @compile@ may need to read more package + interfaces, which are added to the PIT and hence to + HP.~\ToDo{How? When?} + + CM passes the PST to @compile@ and is returned an updated version + on both success and failure. + + \item + {\bf Holding Pen (HP)} @:: HoldingPen@ + + HP holds parsed but not-yet renamed-or-typechecked fragments of + package interfaces. As typechecking of other modules progresses, + fragments are removed (``slurped'') from HP, renamed and + typechecked, and placed in PCS.PST (see above). Slurping a + fragment may require new interfaces to be read into HP. The hope + is, though, that many fragments will never get slurped, reducing + the total number of interfaces read (as compared to eager slurping). + + \end{itemize} + + PCS is opaque to CM; only @compile@ knows what's in it, and how to + update it. Because packages are assumed static, PCS never becomes + out of date. So CM only needs to be able to create an empty PCS, + with @emptyPCS@, and thence just passes it through @compile@ with + no further ado. + + In return, @compile@ must promise not to store in PCS any + information pertaining to the home modules. If it did so, CM would + need to have a way to remove this information prior to commencing a + rebuild, which conflicts with PCS's opaqueness to CM. + + + + +\subsubsection{What {\tt compile} does} +@compile@ is necessarily somewhat complex. We've decided to do away +with private global variables -- they make the design specification +less clear, although the implementation might use them. Without +further ado: +\begin{verbatim} + compile :: SI -- obvious + -> Finder -- to find modules + -> ModSummary -- summary, including source + -> Maybe ModIFace + -- former summary, if avail + -> HST -- for home module ModDetails + -> PCS -- IN: the persistent compiler state + + -> IO CompResult + + data CompResult + = CompOK ModDetails -- new details (== HST additions) + (Maybe (ModIFace, Linkable)) + -- summary and code; Nothing => compilation + -- not needed (old summary and code are still valid) + PCS -- updated PCS + [SDoc] -- warnings + + | CompErrs PCS -- updated PCS + [SDoc] -- warnings and errors + + data PCS + = MkPCS PIT -- package interfaces + PST -- post slurping global symtab contribs + HoldingPen -- pre slurping interface bits and pieces + + emptyPCS :: IO PCS -- since CM has no other way to make one +\end{verbatim} +Although @compile@ is passed three of the global structures (FLAGS, +HST and PCS), it only modifies PCS. The rest are modified by CM as it +sees fit, from the stuff returned in the @CompResult@. + +@compile@ is allowed to return an updated PCS even if compilation +errors occur, since the information in it pertains only to foreign +packages and is assumed to be always-correct. + +What @compile@ does: \ToDo{A bit vague ... needs refining. How does + @finder@ come into the game?} +\begin{itemize} +\item Figure out if this module needs recompilation. + \begin{itemize} + \item If there's no old @ModIFace@, it does. Else: + \item Compare the @ModSummary@ supplied with that in the + old @ModIFace@. If the source has changed, recompilation + is needed. Else: + \item Compare the usage version numbers in the old @ModIFace@ with + those in the imported @ModIFace@s. All needed interfaces + for this should be in either HIT or PIT. If any version + numbers differ, recompilation is needed. + \item Otherwise it isn't needed. + \end{itemize} + +\item + If recompilation is not needed, create a new @ModDetails@ from the + old @ModIFace@, looking up information in HST and PCS.PST as + necessary. Return the new details, a @Nothing@ denoting + compilation was not needed, the PCS \ToDo{I don't think the PCS + should be updated, but who knows?}, and an empty warning list. + +\item + Otherwise, compilation is needed. + + If the module is only available in object+interface form, read the + interface, make up details, create a linkable pointing at the + object code. \ToDo{Does this involve reading any more interfaces? Does + it involve updating PST?} + + Otherwise, translate from source, then create and return: an + details, interface, linkable, updated PST, and warnings. + + When looking for a new interface, search HST, then PCS.PIT, and only + then read from disk. In which case add the new interface(s) to + PCS.PIT. + + \ToDo{If compiling a module with a boot-interface file, check the + boot interface against the inferred interface.} +\end{itemize} + + +\subsubsection{Contents of \mbox{\tt ModDetails}, + \mbox{\tt ModIFace} and \mbox{\tt HoldingPen}} +Only @compile@ can see inside these three types -- they are opaque to +everyone else. @ModDetails@ holds the post-renaming, +post-typechecking environment created by compiling a module. + +\begin{verbatim} + data ModDetails + = ModDetails { + moduleExports :: Avails + moduleEnv :: GlobalRdrEnv -- == FM RdrName [Name] + typeEnv :: FM Name TyThing -- TyThing is in TcEnv.lhs + instEnv :: InstEnv + fixityEnv :: FM Name Fixity + ruleEnv :: FM Id [Rule] + } +\end{verbatim} + +@ModIFace@ is nearly the same as @ParsedIFace@ from @RnMonad.lhs@: +\begin{verbatim} + type ModIFace = ParsedIFace -- not really, but ... + data ParsedIface + = ParsedIface { + pi_mod :: Module, -- Complete with package info + pi_vers :: Version, -- Module version number + pi_orphan :: WhetherHasOrphans, -- Whether this module has orphans + pi_usages :: [ImportVersion OccName], -- Usages + pi_exports :: [ExportItem], -- Exports + pi_insts :: [RdrNameInstDecl], -- Local instance declarations + pi_decls :: [(Version, RdrNameHsDecl)], -- Local definitions + pi_fixity :: (Version, [RdrNameFixitySig]), -- Local fixity declarations, + -- with their version + pi_rules :: (Version, [RdrNameRuleDecl]), -- Rules, with their version + pi_deprecs :: [RdrNameDeprecation] -- Deprecations + } +\end{verbatim} + +@HoldingPen@ is a cleaned-up version of that found in @RnMonad.lhs@, +retaining just the 3 pieces actually comprising the holding pen: +\begin{verbatim} + data HoldingPen + = HoldingPen { + iDecls :: DeclsMap, -- A single, global map of Names to decls + + iInsts :: IfaceInsts, + -- The as-yet un-slurped instance decls; this bag is depleted when we + -- slurp an instance decl so that we don't slurp the same one twice. + -- Each is 'gated' by the names that must be available before + -- this instance decl is needed. + + iRules :: IfaceRules + -- Similar to instance decls, only for rules + } +\end{verbatim} + +%%-- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --%% +\subsection{The linker (\mbox{\tt link})} +\label{sec:linker} + +\subsubsection{Data structures owned by the linker} + +In the same way that @compile@ has a persistent compiler state (PCS), +the linker has a persistent (session-lifetime) state, PLS, the +Linker's Persistent State. In batch mode PLS is entirely irrelevant, +because there is only a single link step, and can be a unit value +ignored by everybody. In interactive mode PLS is composed of the +following three parts: + +\begin{itemize} +\item +\textbf{The Source Symbol Table (SST)}@ :: FiniteMap RdrName HValue@ + The source symbol table is used when linking interpreted code. + Unlinked interpreted code consists of an STG tree where + the leaves are @RdrNames@. The linker's job is to resolve these to + actual addresses (the alternative is to resolve these lazily when + the code is run, but this requires passing the full symbol table + through the interpreter and the repeated lookups will probably be + expensive). + + The source symbol table therefore maps @RdrName@s to @HValue@s, for + every @RdrName@ that currently \emph{has} an @HValue@, including all + exported functions from object code modules that are currently + linked in. Linking therefore turns a @StgTree RdrName@ into an + @StgTree HValue@. + + It is important that we can prune this symbol table by throwing away + the mappings for an entire module, whenever we recompile/relink a + given module. The representation is therefore probably a two-level + mapping, from module names, to function/constructor names, to + @HValue@s. + +\item \textbf{The Object Symbol Table (OST)}@ :: FiniteMap String Addr@ + This is a lower level symbol table, mapping symbol names in object + modules to their addresses in memory. It is used only when + resolving the external references in an object module, and contains + only entries that are defined in object modules. + + Why have two symbol tables? Well, there is a clear distinction + between the two: the source symbol table maps Haskell symbols to + Haskell values, and the object symbol table maps object symbols to + addresses. There is some overlap, in that Haskell symbols certainly + have addresses, and we could look up a Haskell symbol's address by + manufacturing the right object symbol and looking that up in the + object symbol table, but this is likely to be slow and would force + us to extend the object symbol table with all the symbols + ``exported'' by interpreted code. Doing it this way enables us to + decouple the object management subsystem from the rest of the linker + with a minimal interface; something like + + \begin{verbatim} + loadObject :: Unlinked -> IO Object + unloadModule :: Unlinked -> IO () + lookupSymbol :: String -> IO Addr + \end{verbatim} + + Rather unfortunately we need @lookupSymbol@ in order to populate the + source symbol table when linking in a new compiled module. Our + object management subsystem is currently written in C, so decoupling + this interface as much as possible is highly desirable. + +\item + {\bf Linked Image (LI)} @:: no-explicit-representation@ + + LI isn't explicitly represented in the system, but we record it + here for completeness anyway. LI is the current set of + linked-together module, package and other library fragments + constituting the current executable mass. LI comprises: + \begin{itemize} + \item Machine code (@.o@, @.a@, @.DLL@ file images) in memory. + These are loaded from disk when needed, and stored in + @malloc@ville. To simplify storage management, they are + never freed or reused, since this creates serious + complications for storage management. When no longer needed, + they are simply abandoned. New linkings of the same object + code produces new copies in memory. We hope this not to be + too much of a space leak. + \item STG trees, which live in the GHCI heap and are managed by the + storage manager in the usual way. They are held alive (are + reachable) via the @HValue@s in the OST. Such @HValue@s are + applications of the interpreter function to the trees + themselves. Linking a tree comprises travelling over the + tree, replacing all the @Id@s with pointers directly to the + relevant @_closure@ labels, as determined by searching the + OST. Once the leaves are linked, trees are wrapped with the + interpreter function. The resulting @HValue@s then behave + indistinguishably from compiled versions of the same code. + \end{itemize} + Because object code is outside the heap and never deallocated, + whilst interpreted code is held alive via the HST, there's no need + to have a data structure which ``is'' the linked image. + + For batch compilation, LI doesn't exist because OST doesn't exist, + and because @link@ doesn't load code into memory, instead just + invokes the system linker. + + \ToDo{Do we need to say anything about CAFs and SRTs? Probably ...} +\end{itemize} +As with PCS, CM has no way to create an initial PLS, so we supply +@emptyPLS@ for that purpose. + +\subsubsection{The linker's interface} + +In practice, the PLS might be hidden in the I/O monad rather +than passed around explicitly. (The same might be true for PCS). +Anyway: + +\begin{verbatim} + data PLS -- as described above; opaque to everybody except the linker + + link :: PCI -> ??? -> [[Linkable]] -> PLS -> IO LinkResult + + data LinkResult = LinkOK PLS + | LinkErrs PLS [SDoc] + + emptyPLS :: IO PLS -- since CM has no other way to make one +\end{verbatim} + +CM uses @link@ as follows: + +After repeatedly using @compile@ to compile all modules which are +out-of-date, the @link@ is invoked. The @[[Linkable]]@ argument to +@link@ represents the list of (recursive groups of) home modules which +have been newly compiled, along with @Linkable@s for each of +the packages in use (the compilation manager knows which external +packages are referenced by the home package). The order of the list +is important: it is sorted in such a way that linking any prefix of +the list will result in an image with no unresolved references. Note +that for batch linking there may be further restrictions; for example +it may not be possible to link recursive groups containing libraries. + +@link@ does the following: + +\begin{itemize} + \item + In batch mode, do nothing. In interactive mode, + examine the supplied @[[Linkable]]@ to determine which home + module @Unlinked@s are new. Remove precisely these @Linkable@s + from PLS. (In fact we really need to remove their upwards + transitive closure, but I think it is an invariant that CM will + supply an upwards transitive closure of new modules). + See below for descriptions of @Linkable@ and @Unlinked@. + + \item + Batch system: invoke the external linker to link everything in one go. + Interactive: bind the @Unlinked@s for the newly compiled modules, + plus those for any newly required packages, into PLS. + + Note that it is the linker's responsibility to remember which + objects and packages have already been linked. By comparing this + with the @Linkable@s supplied to @link@, it can determine which + of the linkables in LI are out of date +\end{itemize} + +If linking in of a group should fail for some reason, @link@ should +not modify its PLS at all. In other words, linking each group +is atomic; it either succeeds or fails. + +\subsubsection*{\mbox{\tt Unlinked} and \mbox{\tt Linkable}} + +Two important types: @Unlinked@ and @Linkable@. The latter is a +higher-level representation involving multiple of the former. +An @Unlinked@ is a reference to unlinked executable code, something +a linker could take as input: + +\begin{verbatim} + data Unlinked = DotO Path + | DotA Path + | DotDLL Path + | Trees [StgTree RdrName] +\end{verbatim} + +The first three describe the location of a file (presumably) +containing the code to link. @Trees@, which only exists in +interactive mode, gives a list of @StgTrees@, in which the unresolved +references are @RdrNames@ -- hence it's non-linkedness. Once linked, +those @RdrNames@ are replaced with pointers to the machine code +implementing them. + +A @Linkable@ gathers together several @Unlinked@s and associates them +with either a module or package: + +\begin{verbatim} + data Linkable = LM Module [Unlinked] -- a module + | LP PkgName -- a package +\end{verbatim} + +The order of the @Unlinked@s in the list is important, as +they are linked in left-to-right order. The @Unlinked@ objects for a +particular package can be obtained from the package configuration (see +Section \ref{sec:staticinfo}). + +\ToDo{When adding @Addr@s from an object module to SST, we need to + somehow find out the @RdrName@s of the symbols exported by that + module. + So we'd need to pass in the @ModDetails@ or @ModIFace@ or some such?} + + + +%%-----------------------------------------------------------------%% +\section{Background ideas} +\subsubsection*{Out of date, but correct in spirit} + +\subsection{Restructuring the system} + +At the moment @hsc@ compiles one source module into C or assembly. +This functionality is pushed inside a function called @compile@, +introduced shortly. The main new chunk of code is CM, the compilation manager, +which supervises multiple runs of @compile@ so as to create up-to-date +translations of a whole bunch of modules, as quickly as possible. +CM also employs some minor helper functions, @finder@, @summarise@ and +@link@, to do its work. + +Our intent is to allow CM to be used as the basis either of a +multi-module, batch mode compilation system, or to supply an +interactive environment similar to that of Hugs. +Only minor modifications to the behaviour of @compile@ and @link@ +are needed to give these different behaviours. + +CM and @compile@, and, for interactive use, an interpreter, are the +main code components. The most important data structure is the global +symbol table; much design effort has been expended thereupon. + + +\subsection{How the global symbol table is implemented} + +The top level symbol table is a @FiniteMap@ @ModuleName@ +@ModuleDetails@. @ModuleDetails@ contains essentially the environment +created by compiling a module. CM manages this finite map, adding and +deleting module entries as required. + +The @ModuleDetails@ for a module @M@ contains descriptions of all +tycons, classes, instances, values, unfoldings, etc (henceforth +referred to as ``entities''), available from @M@. These are just +trees in the GHCI heap. References from other modules to these +entities is direct -- when you have a @TyCon@ in your hand, you really +have a pointer directly to the @TyCon@ structure in the defining module, +rather than some kind of index into a global symbol table. So there +is a global symbol table, but it has a distributed (sphagetti-like?) +nature. + +This gives fast and convenient access to tycon, class, instance, +etc, information. But because there are no levels of indirection, +there's a problem when we replace @M@ with an updated version of @M@. +We then need to find all references to entities in the old @M@'s +sphagetti, and replace them with pointers to the new @M@'s sphagetti. +This problem motivates a large part of the design. + + + +\subsection{Implementing incremental recompilation -- simple version} +Given the following module graph +\begin{verbatim} + D + / \ + / \ + B C + \ / + \ / + A +\end{verbatim} +(@D@ imports @B@ and @C@, @B@ imports @A@, @C@ imports @A@) the aim is to do the +least possible amount of compilation to bring @D@ back up to date. The +simplest scheme we can think of is: +\begin{itemize} +\item {\bf Downsweep}: + starting with @D@, re-establish what the current module graph is + (it might have changed since last time). This means getting a + @ModuleSummary@ of @D@. The summary can be quickly generated, + contains @D@'s import lists, and gives some way of knowing whether + @D@'s source has changed since the last time it was summarised. + + Transitively follow summaries from @D@, thereby establishing the + module graph. +\item + Remove from the global symbol table (the @FiniteMap@ @ModuleName@ + @ModuleDetails@) the upwards closure of all modules in this package + which are out-of-date with respect to their previous versions. Also + remove all modules no longer reachable from @D@. +\item {\bf Upsweep}: + Starting at the lowest point in the still-in-date module graph, + start compiling upwards, towards @D@. At each module, call + @compile@, passing it a @FiniteMap@ @ModuleName@ @ModuleDetails@, + and getting a new @ModuleDetails@ for the module, which is added to + the map. + + When compiling a module, the compiler must be able to know which + entries in the map are for modules in its strict downwards closure, + and which aren't, so that it can manufacture the instance + environment correctly (as union of instances in its downwards + closure). +\item + Once @D@ has been compiled, invoke some kind of linking phase + if batch compilation. For interactive use, can either do it all + at the end, or as you go along. +\end{itemize} +In this simple world, recompilation visits the upwards closure of +all changed modules. That means when a module @M@ is recompiled, +we can be sure no-one has any references to entities in the old @M@, +because modules importing @M@ will have already been removed from the +top-level finite map in the second step above. + +The upshot is that we don't need to worry about updating links to @M@ in +the global symbol table -- there shouldn't be any to update. +\ToDo{What about mutually recursive modules?} + +CM will happily chase through module interfaces in other packages in +the downsweep. But it will only process modules in this package +during the upsweep. So it assumes that modules in other packages +never become out of date. This is a design decision -- we could have +decided otherwise. + +In fact we go further, and require other packages to be compiled, +i.e. to consist of a collection of interface files, and one or more +source files. CM will never apply @compile@ to a foreign package +module, so there's no way a package can be built on the fly from source. + +We require @compile@ to cache foreign package interfaces it reads, so +that subsequent uses don't have to re-read them. The cache never +becomes out of date, since we've assumed that the source of foreign +packages doesn't change during the course of a session (run of GHCI). +As well as caching interfaces, @compile@ must cache, in some sense, +the linkable code for modules. In batch compilation this might simply +mean remembering the names of object files to link, whereas in +interactive mode @compile@ probably needs to load object code into +memory in preparation for in-memory linking. + +Important signatures for this simple scheme are: +\begin{verbatim} + finder :: ModuleName -> ModLocation + + summarise :: ModLocation -> IO ModSummary + + compile :: ModSummary + -> FM ModName ModDetails + -> IO CompileResult + + data CompileResult = CompOK ModDetails + | CompErr [ErrMsg] + + link :: [ModLocation] -> [PackageLocation] -> IO Bool -- linked ok? +\end{verbatim} + + +\subsection{Implementing incremental recompilation -- clever version} + +So far, our upsweep, which is the computationally expensive bit, +recompiles a module if either its source is out of date, or it +imports a module which has been recompiled. Sometimes we know +we can do better than this: +\begin{verbatim} + module B where module A + import A ( f ) {-# NOINLINE f #-} + ... f ... f x = x + 42 +\end{verbatim} +If the definition of @f@ is changed to @f x = x + 43@, the simple +upsweep would recompile @B@ unnecessarily. We would like to detect +this situation and avoid propagating recompilation all the way to the +top. There are two parts to this: detecting when a module doesn't +need recompilation, and managing inter-module references in the +global symbol table. + +\subsubsection*{Detecting when a module doesn't need recompilation} + +To do this, we introduce a new concept: the @ModuleIFace@. This is +effectively an in-memory interface file. References to entities in +other modules are done via strings, rather than being pointers +directly to those entities. Recall that, by comparison, +@ModuleDetails@ do contain pointers directly to the entities they +refer to. So a @ModuleIFace@ is not part of the global symbol table. + +As before, compiling a module produces a @ModuleDetails@ (inside the +@CompileResult@), but it also produces a @ModuleIFace@. The latter +records, amongst things, the version numbers of all imported entities +needed for the compilation of that module. @compile@ optionally also +takes the old @ModuleIFace@ as input during compilation: +\begin{verbatim} + data CompileResult = CompOK ModDetails ModIFace + | CompErr [ErrMsg] + + compile :: ModSummary + -> FM ModName ModDetails + -> Maybe ModuleIFace + -> IO CompileResult +\end{verbatim} +Now, if the @ModuleSummary@ indicates this module's source hasn't +changed, we only need to recompile it if something it depends on has +changed. @compile@ can detect this by inspecting the imported entity +version numbers in the module's old @ModuleIFace@, and comparing them +with the version numbers from the entities in the modules being +imported. If they are all the same, nothing it depends on has +changed, so there's no point in recompiling. + +\subsubsection*{Managing inter-module references in the global symbol table} + +In the above example with @A@, @B@ and @f@, the specified change to @f@ would +require @A@ but not @B@ to be recompiled. That generates a new +@ModuleDetails@ for @A@. Problem is, if we leave @B@'s @ModuleDetails@ +unchanged, they continue to refer (directly) to the @f@ in @A@'s old +@ModuleDetails@. This is not good, especially if equality between +entities is implemented using pointer equality. + +One solution is to throw away @B@'s @ModuleDetails@ and recompile @B@. +But this is precisely what we're trying to avoid, as it's expensive. +Instead, a cheaper mechanism achieves the same thing: recreate @B@'s +details directly from the old @ModuleIFace@. The @ModuleIFace@ will +(textually) mention @f@; @compile@ can then find a pointer to the +up-to-date global symbol table entry for @f@, and place that pointer +in @B@'s @ModuleDetails@. The @ModuleDetails@ are, therefore, +regenerated just by a quick lookup pass over the module's former +@ModuleIFace@. All this applies, of course, only when @compile@ has +concluded it doesn't need to recompile @B@. + +Now @compile@'s signature becomes a little clearer. @compile@ has to +recompile the module, generating a fresh @ModuleDetails@ and +@ModuleIFace@, if any of the following hold: +\begin{itemize} +\item + The old @ModuleIFace@ wasn't supplied, for some reason (perhaps + we've never compiled this module before?) +\item + The module's source has changed. +\item + The module's source hasn't changed, but inspection of @ModuleIFaces@ + for this and its imports indicates that an imported entity has + changed. +\end{itemize} +If none of those are true, we're in luck: quickly knock up a new +@ModuleDetails@ from the old @ModuleIFace@, and return them both. + +As a result, the upsweep still visits all modules in the upwards +closure of those whose sources have changed. However, at some point +we hopefully make a transition from generating new @ModuleDetails@ the +expensive way (recompilation) to a cheap way (recycling old +@ModuleIFaces@). Either way, all modules still get new +@ModuleDetails@, so the global symbol table is correctly +reconstructed. + + +\subsection{How linking works, roughly} + +When @compile@ translates a module, it produces a @ModuleDetails@, +@ModuleIFace@ and a @Linkable@. The @Linkable@ contains the +translated but un-linked code for the module. And when @compile@ +ventures into an interface in package it hasn't seen so far, it +copies the package's object code into memory, producing one or more +@Linkable@s. CM keeps track of these linkables. + +Once all modules have been @compile@d, CM invokes @link@, supplying +the all the @Linkable@s it knows about. If @compile@ had also been +linking incrementally as it went along, @link@ doesn't have to do +anything. On the other hand, @compile@ could choose not to be +incremental, and leave @link@ to do all the work. + +@Linkable@s are opaque to CM. For batch compilation, a @Linkable@ +can record just the name of an object file, DLL, archive, or whatever, +in which case the CM's call to @link@ supplies exactly the set of +file names to be linked. @link@ can pass these verbatim to the +standard system linker. + + + + +%%-----------------------------------------------------------------%% +\section{Ancient stuff} +\subsubsection*{Should be selectively merged into ``Background ideas''} + +\subsection{Overall} +Top level structure is: +\begin{itemize} +\item The Compilation Manager (CM) calculates and maintains module + dependencies, and knows how create up-to-date object or bytecode + for a given module. In doing so it may need to recompile + arbitrary other modules, based on its knowledge of the module + dependencies. +\item On top of the CM are the ``user-level'' services. We envisage + both a HEP-like interface, for interactive use, and an + @hmake@ style batch compiler facility. +\item The CM only deals with inter-module issues. It knows nothing + about how to recompile an individual module, nor where the compiled + result for a module lives, nor how to tell if + a module is up to date, nor how to find the dependencies of a module. + Instead, these services are supplied abstractly to CM via a + @Compiler@ record. To a first approximation, a @Compiler@ + contains + the same functionality as @hsc@ has had until now -- the ability to + translate a single Haskell module to C/assembly/object/bytecode. + + Different clients of CM (HEP vs @hmake@) may supply different + @Compiler@s, since they need slightly different behaviours. + Specifically, HEP needs a @Compiler@ which creates bytecode + in memory, and knows how to link it, whereas @hmake@ wants + the traditional behaviour of emitting assembly code to disk, + and making no attempt at linkage. +\end{itemize} + +\subsection{Open questions} +\begin{itemize} +\item + Error reporting from @open@ and @compile@. +\item + Instance environment management +\item + We probably need to make interface files say what + packages they depend on (so that we can figure out + which packages to load/link). +\item + CM is parameterised both by the client uses and the @Compiler@ + supplied. But it doesn't make sense to have a HEP-style client + attached to a @hmake@-style @Compiler@. So, really, the + parameterising entity should contain both aspects, not just the + current @Compiler@ contents. +\end{itemize} + +\subsection{Assumptions} + +\begin{itemize} +\item Packages other than the "current" one are assumed to be + already compiled. +\item + The "current" package is usually "MAIN", + but we can set it with a command-line flag. + One invocation of ghci has only one "current" package. +\item + Packages are not mutually recursive +\item + All the object code for a package P is in libP.a or libP.dll +\end{itemize} + +\subsection{Stuff we need to be able to do} +\begin{itemize} +\item Create the environment in which a module has been translated, + so that interactive queries can be satisfied as if ``in'' that + module. +\end{itemize} + +%%-----------------------------------------------------------------%% +\section{The Compilation Manager} + +CM (@compilationManager@) is a functor, thus: +\begin{verbatim} +compilationManager :: Compiler -> IO HEP -- IO so that it can create + -- global vars (IORefs) + +data HEP = HEP { + load :: ModuleName -> IO (), + compileString :: ModuleName -> String -> IO HValue, + .... + } + +newCompiler :: IO Compiler -- ??? this is a peer of compilationManager? + +run :: HValue -> IO () -- Run an HValue of type IO () + -- In HEP? +\end{verbatim} + +@load@ is the central action of CM: its job is to bring a module and +all its descendents into an executable state, by doing the following: +\begin{enumerate} +\item + Use @summarise@ to descend the module hierarchy, starting from the + nominated root, creating @ModuleSummary@s, and + building a map @ModuleName@ @->@ @ModuleSummary@. @summarise@ + expects to be passed absolute paths to files. Use @finder@ to + convert module names to file paths. +\item + Topologically sort the map, + using dependency info in the @ModuleSummary@s. +\item + Clean up the symbol table by deleting the upward closure of + changed modules. +\item + Working bottom to top, call @compile@ on the upward closure of + all modules whose source has changed. A module's source has + changed when @sourceHasChanged@ indicates there is a difference + between old and new summaries for the module. Update the running + @FiniteMap@ @ModuleName@ @ModuleDetails@ with the new details + for this module. Ditto for the running + @FiniteMap@ @ModuleName@ @ModuleIFace@. +\item + Call @compileDone@ to signify that we've reached the top, so + that the batch system can now link. +\end{enumerate} + + +%%-----------------------------------------------------------------%% +\section{A compiler} + +Most of the system's complexity is hidden inside the functions +supplied in the @Compiler@ record: +\begin{verbatim} +data Compiler = Compiler { + + finder :: PackageConf -> [Path] -> IO (ModuleName -> ModuleLocation) + + summarise :: ModuleLocation -> IO ModuleSummary + + compile :: ModuleSummary + -> Maybe ModuleIFace + -> FiniteMap ModuleName ModuleDetails + -> IO CompileResult + + compileDone :: IO () + compileStarting :: IO () -- still needed? I don't think so. + } + +type ModuleName = String (or some such) +type Path = String -- an absolute file name +\end{verbatim} + +\subsection{The module \mbox{\tt finder}} +The @finder@, given a package configuration file and a list of +directories to look in, will map module names to @ModuleLocation@s, +in which the @Path@s are filenames, probably with an absolute path +to them. +\begin{verbatim} +data ModuleLocation = SourceOnly Path -- .hs + | ObjectCode Path Path -- .o & .hi + | InPackage Path -- .hi +\end{verbatim} +@SourceOnly@ and @ObjectCode@ are unremarkable. For sanity, +we require that a module's object and interface be in the same +directory. @InPackage@ indicates that the module is in a +different package. + +@Module@ values -- perhaps all @Name@ish things -- contain the name of +their package. That's so that +\begin{itemize} +\item Correct code can be generated for in-DLL vs out-of-DLL refs. +\item We don't have version number dependencies for symbols + imported from different packages. +\end{itemize} + +Somehow or other, it will be possible to know all the packages +required, so that the for the linker can load them. +We could detect package dependencies by recording them in the +@compile@r's @ModuleIFace@ cache, and with that and the +package config info, figure out the complete set of packages +to link. Or look at the command line args on startup. + +\ToDo{Need some way to tell incremental linkers about packages, + since in general we'll need to load and link them before + linking any modules in the current package.} + + +\subsection{The module \mbox{\tt summarise}r} +Given a filename of a module (\ToDo{presumably source or iface}), +create a summary of it. A @ModuleSummary@ should contain only enough +information for CM to construct an up-to-date picture of the +dependency graph. Rather than expose CM to details of timestamps, +etc, @summarise@ merely provides an up-to-date summary of any module. +CM can extract the list of dependencies from a @ModuleSummary@, but +other than that has no idea what's inside it. +\begin{verbatim} +data ModuleSummary = ... (abstract) ... + +depsFromSummary :: ModuleSummary -> [ModuleName] -- module names imported +sourceHasChanged :: ModuleSummary -> ModuleSummary -> Bool +\end{verbatim} +@summarise@ is intended to be fast -- a @stat@ of the source or +interface to see if it has changed, and, if so, a quick semi-parse to +determine the new imports. + +\subsection{The module \mbox{\tt compile}r} +@compile@ traffics in @ModuleIFace@s and @ModuleDetails@. + +A @ModuleIFace@ is an in-memory representation of the contents of an +interface file, including version numbers, unfoldings and pragmas, and +the linkable code for the module. @ModuleIFace@s are un-renamed, +using @HsSym@/@RdrNames@ rather than (globally distinct) @Names@. + +@ModuleDetails@, by contrast, is an in-memory representation of the +static environment created by compiling a module. It is phrased in +terms of post-renaming @Names@, @TyCon@s, etc, so it's basically a +renamed-to-global-uniqueness rendition of a @ModuleIFace@. + +In an interactive session, we'll want to be able to evaluate +expressions as if they had been compiled in the scope of some +specified module. This means that the @ModuleDetails@ must contain +the type of everything defined in the module, rather than just the +types of exported stuff. As a consequence, @ModuleIFace@ must also +contain the type of everything, because it should always be possible +to generate a module's @ModuleDetails@ from its @ModuleIFace@. + +CM maintains two mappings, one from @ModuleName@s to @ModuleIFace@s, +the other from @ModuleName@s to @ModuleDetail@s. It passes the former +to each call of @compile@. This is used to supply information about +modules compiled prior to this one (lower down in the graph). The +returned @CompileResult@ supplies a new @ModuleDetails@ for the module +if compilation succeeded, and CM adds this to the mapping. The +@CompileResult@ also supplies a new @ModuleIFace@, which is either the +same as that supplied to @compile@, if @compile@ decided not to +retranslate the module, or is the result of a fresh translation (from +source). So these mappings are an explicitly-passed-around part of +the global system state. + +@compile@ may also {\em optionally} also accumulate @ModuleIFace@s for +modules in different packages -- that is, interfaces which we read, +but never attempt to recompile source for. Such interfaces, being +from foreign packages, never change, so @compile@ can accumulate them +in perpetuity in a private global variable. Indeed, a major motivator +of this design is to facilitate this caching of interface files, +reading of which is a serious bottleneck for the current compiler. + +When CM restarts compilation down at the bottom of the module graph, +it first needs to throw away all \ToDo{all?} @ModuleDetails@ in the +upward closure of the out-of-date modules. So @ModuleDetails@ don't +persist across recompilations. But @ModuleIFace@s do, since they +are conceptually equivalent to interface files. + + +\subsubsection*{What @compile@ returns} +@compile@ returns a @CompileResult@ to CM. +Note that the @compile@'s foreign-package interface cache can +become augmented even as a result of reading interfaces for a +compilation attempt which ultimately fails, although it will not be +augmented with a new @ModuleIFace@ for the failed module. +\begin{verbatim} +-- CompileResult is not abstract to the Compilation Manager +data CompileResult + = CompOK ModuleIFace + ModuleDetails -- compiled ok, here are new details + -- and new iface + + | CompErr [SDoc] -- compilation gave errors + + | NoChange -- no change required, meaning: + -- exports, unfoldings, strictness, etc, + -- unchanged, and executable code unchanged +\end{verbatim} + + + +\subsubsection*{Re-establishing local-to-global name mappings} +Consider +\begin{verbatim} +module Upper where module Lower ( f ) where +import Lower ( f ) f = ... +g = ... f ... +\end{verbatim} +When @Lower@ is first compiled, @f@ is allocated a @Unique@ +(presumably inside an @Id@ or @Name@?). When @Upper@ is then +compiled, its reference to @f@ is attached directly to the +@Id@ created when compiling @Lower@. + +If the definition of @f@ is now changed, but not the type, +unfolding, strictness, or any other thing which affects the way +it should be called, we will have to recompile @Lower@, but not +@Upper@. This creates a problem -- @g@ will then refer to the +the old @Id@ for @f@, not the new one. This may or may not +matter, but it seems safer to ensure that all @Unique@-based +references into child modules are always up to date. + +So @compile@ recreates the @ModuleDetails@ for @Upper@ from +the @ModuleIFace@ of @Upper@ and the @ModuleDetails@ of @Lower@. + +The rule is: if a module is up to date with respect to its +source, but a child @C@ has changed, then either: +\begin{itemize} +\item On examination of the version numbers in @C@'s + interface/@ModuleIFace@ that we used last time, we discover that + an @Id@/@TyCon@/class/instance we depend on has changed. So + we need to retranslate the module from its source, generating + a new @ModuleIFace@ and @ModuleDetails@. +\item Or: there's nothing in @C@'s interface that we depend on. + So we quickly recreate a new @ModuleDetails@ from the existing + @ModuleIFace@, creating fresh links to the new @Unique@-world + entities in @C@'s new @ModuleDetails@. +\end{itemize} + +Upshot: we need to redo @compile@ on all modules all the way up, +rather than just the ones that need retranslation. However, we hope +that most modules won't need retranslation -- just regeneration of the +@ModuleDetails@ from the @ModuleIFace@. In effect, the @ModuleIFace@ +is a quickly-compilable representation of the module's contents, just +enough to create the @ModuleDetails@. + +\ToDo{Is there anything in @ModuleDetails@ which can't be + recreated from @ModuleIFace@ ?} + +So the @ModuleIFace@s persist across calls to @HEP.load@, whereas +@ModuleDetails@ are reconstructed on every compilation pass. This +means that @ModuleIFace@s have the same lifetime as the byte/object +code, and so should somehow contain their code. + +The behind-the-scenes @ModuleIFace@ cache has some kind of holding-pen +arrangement, to lazify the copying-out of stuff from it, and thus to +minimise redundant interface reading. \ToDo{Burble burble. More +details.}. + +When CM starts working back up the module graph with @compile@, it +needs to remove from the travelling @FiniteMap@ @ModuleName@ +@ModuleDetails@ the details for all modules in the upward closure of +the compilation start points. However, since we're going to visit +precisely those modules and no others on the way back up, we might as +well just zap them the old @ModuleDetails@ incrementally. This does +mean that the @FiniteMap@ @ModuleName@ @ModuleDetails@ will be +inconsistent until we reach the top. + +In interactive mode, each @compile@ call on a module for which no +object code is available, or for which it is out of date wrt source, +emit bytecode into memory, update the resulting @ModuleIFace@ with the +address of the bytecode image, and link the image. + +In batch mode, emit assembly or object code onto disk. Record +somewhere \ToDo{where?} that this object file needs to go into the +final link. + +When we reach the top, @compileDone@ is called, to signify that batch +linking can now proceed, if need be. + +Modules in other packages never get a @ModuleIFace@ or @ModuleDetails@ +entry in CM's maps -- those maps are only for modules in this package. +As previously mentioned, @compile@ may optionally cache @ModuleIFace@s +for foreign package modules. When reading such an interface, we don't +need to read the version info for individual symbols, since foreign +packages are assumed static. + +\subsubsection*{What's in a \mbox{\tt ModuleIFace}?} + +Current interface file contents? + + +\subsubsection*{What's in a \mbox{\tt ModuleDetails}?} + +There is no global symbol table @:: Name -> ???@. To look up a +@Name@, first extract the @ModuleName@ from it, look that up in +the passed-in @FiniteMap@ @ModuleName@ @ModuleDetails@, +and finally look in the relevant @Env@. + +\ToDo{Do we still have the @HoldingPen@, or is it now composed from +per-module bits too?} +\begin{verbatim} +data ModuleDetails = ModuleDetails { + + moduleExports :: what it exports (Names) + -- roughly a subset of the .hi file contents + + moduleEnv :: RdrName -> Name + -- maps top-level entities in this module to + -- globally distinct (Uniq-ified) Names + + moduleDefs :: Bag Name -- All the things in the global symbol table + -- defined by this module + + package :: Package -- what package am I in? + + lastCompile :: Date -- of last compilation + + instEnv :: InstEnv -- local inst env + typeEnv :: Name -> TyThing -- local tycon env? + } + +-- A (globally unique) symbol table entry. Note that Ids contain +-- unfoldings. +data TyThing = AClass Class + | ATyCon TyCon + | AnId Id +\end{verbatim} +What's the stuff in @ModuleDetails@ used for? +\begin{itemize} +\item @moduleExports@ so that the stuff which is visible from outside + the module can be calculated. +\item @moduleEnv@: \ToDo{umm err} +\item @moduleDefs@: one reason we want this is so that we can nuke the + global symbol table contribs from this module when it leaves the + system. \ToDo{except ... we don't have a global symbol table any + more.} +\item @package@: we will need to chase arbitrarily deep into the + interfaces of other packages. Of course we don't want to + recompile those, but as we've read their interfaces, we may + as well cache that info. So @package@ indicates whether this + module is in the default package, or, if not, which it is in. + + Also, when we come to linking, we'll need to know which + packages are demanded, so we know to load their objects. + +\item @lastCompile@: When the module was last compiled. If the + source is older than that, then a recompilation can only be + required if children have changed. +\item @typeEnv@: obvious?? +\item @instEnv@: the instances contributed by this module only. The + Report allegedly says that when a module is translated, the + available + instance env is all the instances in the downward closure of + itself in the module graph. + + We choose to use this simple representation -- each module + holds just its own instances -- and do the naive thing when + creating an inst env for compilation with. If this turns out + to be a performance problem we'll revisit the design. +\end{itemize} + + + +%%-----------------------------------------------------------------%% +\section{Misc text looking for a home} + +\subsection*{Linking} + +\ToDo{All this linking stuff is now bogus.} + +There's an abstract @LinkState@, which is threaded through the linkery +bits. CM can call @addpkgs@ to notify the linker of packages +required, and it can call @addmods@ to announce modules which need to +be linked. Finally, CM calls @endlink@, after which an executable +image should be ready. The linker may link incrementally, during each +call of @addpkgs@ and @addmods@, or it can just store up names and do +all the linking when @endlink@ is called. + +In order that incremental linking is possible, CM should specify +packages and module groups in dependency order, ie, from the bottom up. + +\subsection*{In-memory linking of bytecode} +When being HEP-like, @compile@ will translate sources to bytecodes +in memory, with all the bytecode for a module as a contiguous lump +outside the heap. It needs to communicate the addresses of these +lumps to the linker. The linker also needs to know whether a +given module is available as in-memory bytecode, or whether it +needs to load machine code from a file. + +I guess @LinkState@ needs to map module names to base addresses +of their loaded images, + the nature of the image, + whether or not +the image has been linked. + +\subsection*{On disk linking of object code, to give an executable} +The @LinkState@ in this case is just a list of module and package +names, which @addpkgs@ and @addmods@ add to. The final @endlink@ +call can invoke the system linker. + +\subsection{Finding out about packages, dependencies, and auxiliary + objects} + +Ask the @packages.conf@ file that lives with the driver at the mo. + +\ToDo{policy about upward closure?} + + + +\ToDo{record story about how in memory linking is done.} + +\ToDo{linker start/stop/initialisation/persistence. Need to + say more about @LinkState@.} + + +\end{document} + + diff --git a/docs/rts/closure.ps b/docs/rts/closure.ps new file mode 100644 index 0000000000..241bf9b404 --- /dev/null +++ b/docs/rts/closure.ps @@ -0,0 +1,129 @@ +%! +%%Title: closure.fig +%%Creator: fig2dev +%%CreationDate: Wed May 28 08:22:23 1997 +%%For: sigbjorn@lassi (Sigbjorn Finne,,,) +%%Pages: 0 +%%BoundingBox: 0 0 259 171 +%%EndComments +/$F2psDict 32 dict def +$F2psDict begin + $F2psDict /mtrx matrix put + + /DrawEllipse { + /endangle exch def + /startangle exch def + /yrad exch def + /xrad exch def + /y exch def + /x exch def + /savematrix mtrx currentmatrix def + x y translate xrad yrad scale 0 0 1 startangle endangle arc + savematrix setmatrix + } def newpath 0 0 0 0 0 1 DrawEllipse stroke + + end + /$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def + /$F2psEnd {$F2psEnteredState restore end} def + %%EndProlog + +$F2psBegin +1 setlinecap 1 setlinejoin +-18 18 translate +0.000000 171.000000 translate 0.900 -0.900 scale +1.000 setlinewidth +% Ellipse +newpath 57 47 3 3 0 360 DrawEllipse gsave 0.000 setgray fill grestore stroke +% Polyline +newpath 57 48 moveto 57 92 lineto 88 92 lineto stroke +newpath 80.000 90.000 moveto 88.000 92.000 lineto 80.000 94.000 lineto stroke +% Polyline +newpath 184 31 moveto 184 57 lineto stroke +% Polyline +newpath 260 31 moveto 298 31 lineto 298 57 lineto 260 57 lineto stroke + [1 3.000000] 0 setdash +% Polyline +newpath 209 31 moveto 260 31 lineto stroke + [] 0 setdash + [1 3.000000] 0 setdash +% Polyline +newpath 209 57 moveto 260 57 lineto stroke + [] 0 setdash +% Polyline +newpath 158 57 moveto 209 57 lineto stroke +% Polyline +newpath 158 31 moveto 209 31 lineto stroke + [1 3.000000] 0 setdash +% Polyline +newpath 107 57 moveto 158 57 lineto stroke + [] 0 setdash + [1 3.000000] 0 setdash +% Polyline +newpath 107 31 moveto 158 31 lineto stroke + [] 0 setdash +% Polyline +newpath 107 31 moveto 31 31 lineto 31 57 lineto 107 57 lineto stroke +% Polyline +newpath 95 31 moveto 95 57 lineto stroke +% Polyline +newpath 19 19 moveto 307 19 lineto 307 209 lineto 19 209 lineto closepath stroke +% Polyline +newpath 91 98 moveto 156 98 lineto stroke +% Polyline +newpath 91 113 moveto 156 113 lineto stroke +% Polyline +newpath 92 129 moveto 156 129 lineto stroke +% Polyline +newpath 124 105 moveto 206 105 lineto stroke +newpath 198.000 103.000 moveto 206.000 105.000 lineto 198.000 107.000 lineto stroke +% Polyline +newpath 91 82 moveto 155 82 lineto 155 147 lineto 91 147 lineto closepath stroke +% Polyline +newpath 124 88 moveto 206 88 lineto stroke +newpath 198.000 86.000 moveto 206.000 88.000 lineto 198.000 90.000 lineto stroke +% Polyline +newpath 282 167 moveto 282 112 lineto 211 112 lineto 211 167 lineto closepath stroke +% Polyline +newpath 125 138 moveto 125 188 lineto 153 188 lineto stroke +newpath 145.000 186.000 moveto 153.000 188.000 lineto 145.000 190.000 lineto stroke +/Times-Roman findfont 8.000 scalefont setfont +107 77 moveto +1 -1 scale +(Info table) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +104 48 moveto +1 -1 scale +(Pointer words) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +209 48 moveto +1 -1 scale +(Non-pointer words) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +37 41 moveto +1 -1 scale +(Info pointer) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +99 124 moveto +1 -1 scale +(Constructor tag) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +215 154 moveto +1 -1 scale +(Size and shape info) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +232 163 moveto +1 -1 scale +(for GC) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +156 191 moveto +1 -1 scale +(Update code) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +213 108 moveto +1 -1 scale +(Representation table) gsave 0.000 rotate show grestore 1 -1 scale +/Times-Roman findfont 8.000 scalefont setfont +213 91 moveto +1 -1 scale +(Entry code) gsave 0.000 rotate show grestore 1 -1 scale +$F2psEnd diff --git a/docs/rts/closure.tex b/docs/rts/closure.tex new file mode 100644 index 0000000000..572a8516cf --- /dev/null +++ b/docs/rts/closure.tex @@ -0,0 +1,7 @@ +\makebox[3.597in][l]{ + \vbox to 2.375in{ + \vfill + \special{psfile=closure.ps} + } + \vspace{-\baselineskip} +} diff --git a/docs/rts/hugs_ret.pstex b/docs/rts/hugs_ret.pstex new file mode 100644 index 0000000000..9a7ed98456 --- /dev/null +++ b/docs/rts/hugs_ret.pstex @@ -0,0 +1,145 @@ +%!PS-Adobe-2.0 EPSF-2.0 +%%Title: /tmp/xfig-fig007314 +%%Creator: fig2dev Version 3.1 Patchlevel 2 +%%CreationDate: Wed Oct 15 13:06:42 1997 +%%For: simonm@solander.dcs.gla.ac.uk (Simon Marlow,SM,,,,OCT99, ) +%%Orientation: Portrait +%%BoundingBox: 0 0 204 214 +%%Pages: 0 +%%BeginSetup +%%IncludeFeature: *PageSize Letter +%%EndSetup +%Magnification: 0.80 +%%EndComments +/$F2psDict 200 dict def +$F2psDict begin +$F2psDict /mtrx matrix put +/col-1 {0 setgray} bind def +/col0 {0.000 0.000 0.000 srgb} bind def +/col1 {0.000 0.000 1.000 srgb} bind def +/col2 {0.000 1.000 0.000 srgb} bind def +/col3 {0.000 1.000 1.000 srgb} bind def +/col4 {1.000 0.000 0.000 srgb} bind def +/col5 {1.000 0.000 1.000 srgb} bind def +/col6 {1.000 1.000 0.000 srgb} bind def +/col7 {1.000 1.000 1.000 srgb} bind def +/col8 {0.000 0.000 0.560 srgb} bind def +/col9 {0.000 0.000 0.690 srgb} bind def +/col10 {0.000 0.000 0.820 srgb} bind def +/col11 {0.530 0.810 1.000 srgb} bind def +/col12 {0.000 0.560 0.000 srgb} bind def +/col13 {0.000 0.690 0.000 srgb} bind def +/col14 {0.000 0.820 0.000 srgb} bind def +/col15 {0.000 0.560 0.560 srgb} bind def +/col16 {0.000 0.690 0.690 srgb} bind def +/col17 {0.000 0.820 0.820 srgb} bind def +/col18 {0.560 0.000 0.000 srgb} bind def +/col19 {0.690 0.000 0.000 srgb} bind def +/col20 {0.820 0.000 0.000 srgb} bind def +/col21 {0.560 0.000 0.560 srgb} bind def +/col22 {0.690 0.000 0.690 srgb} bind def +/col23 {0.820 0.000 0.820 srgb} bind def +/col24 {0.500 0.190 0.000 srgb} bind def +/col25 {0.630 0.250 0.000 srgb} bind def +/col26 {0.750 0.380 0.000 srgb} bind def +/col27 {1.000 0.500 0.500 srgb} bind def +/col28 {1.000 0.630 0.630 srgb} bind def +/col29 {1.000 0.750 0.750 srgb} bind def +/col30 {1.000 0.880 0.880 srgb} bind def +/col31 {1.000 0.840 0.000 srgb} bind def + +end +save +-42.0 271.0 translate +1 -1 scale + +/cp {closepath} bind def +/ef {eofill} bind def +/gr {grestore} bind def +/gs {gsave} bind def +/sa {save} bind def +/rs {restore} bind def +/l {lineto} bind def +/m {moveto} bind def +/rm {rmoveto} bind def +/n {newpath} bind def +/s {stroke} bind def +/sh {show} bind def +/slc {setlinecap} bind def +/slj {setlinejoin} bind def +/slw {setlinewidth} bind def +/srgb {setrgbcolor} bind def +/rot {rotate} bind def +/sc {scale} bind def +/sd {setdash} bind def +/ff {findfont} bind def +/sf {setfont} bind def +/scf {scalefont} bind def +/sw {stringwidth} bind def +/tr {translate} bind def +/tnt {dup dup currentrgbcolor + 4 -2 roll dup 1 exch sub 3 -1 roll mul add + 4 -2 roll dup 1 exch sub 3 -1 roll mul add + 4 -2 roll dup 1 exch sub 3 -1 roll mul add srgb} + bind def +/shd {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul + 4 -2 roll mul srgb} bind def +/$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def +/$F2psEnd {$F2psEnteredState restore end} def +%%EndProlog + +$F2psBegin +10 setmiterlimit +n 0 792 m 0 0 l 612 0 l 612 792 l cp clip + 0.04800 0.04800 sc +/Helvetica ff 180.00 scf sf +3405 3885 m +gs 1 -1 sc (Info) col-1 sh gr +7.500 slw +% Polyline +n 900 3000 m 2100 3000 l gs col-1 s gr +% Polyline +n 900 2700 m 2100 2700 l gs col-1 s gr +% Polyline +gs clippath +3003 4545 m 3123 4575 l 3003 4605 l 3165 4605 l 3165 4545 l cp clip +n 1425 3150 m 2550 3150 l 2550 4575 l 3150 4575 l gs col-1 s gr gr + +% arrowhead +n 3003 4545 m 3123 4575 l 3003 4605 l col-1 s +% Polyline + [15 50.0] 50.0 sd +n 3150 4575 m 3150 5625 l gs col-1 s gr [] 0 sd +% Polyline +n 3150 4575 m 3975 4575 l 3975 3600 l 3150 3600 l cp gs col-1 s gr +% Polyline + [15 50.0] 50.0 sd +n 3975 4575 m 3975 5625 l gs col-1 s gr [] 0 sd +% Polyline +gs clippath +3003 2820 m 3123 2850 l 3003 2880 l 3165 2880 l 3165 2820 l cp clip +n 1425 2850 m 3150 2850 l gs col-1 s gr gr + +% arrowhead +n 3003 2820 m 3123 2850 l 3003 2880 l col-1 s +% Polyline +n 3150 2700 m 4500 2700 l 4500 3075 l 3150 3075 l cp gs col-1 s gr +/Helvetica ff 180.00 scf sf +3585 2955 m +gs 1 -1 sc (BCO) col-1 sh gr +/Helvetica ff 180.00 scf sf +1170 1530 m +gs 1 -1 sc (Stack) col-1 sh gr +/Helvetica ff 180.00 scf sf +3300 4125 m +gs 1 -1 sc (Table) col-1 sh gr +/Helvetica ff 180.00 scf sf +3315 5070 m +gs 1 -1 sc (Code) col-1 sh gr +/Helvetica ff 180.00 scf sf +4140 4650 m +gs 1 -1 sc (HUGS_RET) col-1 sh gr +% Polyline +n 900 1200 m 900 3300 l 2100 3300 l 2100 1200 l gs col-1 s gr +$F2psEnd +rs diff --git a/docs/rts/hugs_ret.pstex_t b/docs/rts/hugs_ret.pstex_t new file mode 100644 index 0000000000..3b844da3f0 --- /dev/null +++ b/docs/rts/hugs_ret.pstex_t @@ -0,0 +1,13 @@ +\begin{picture}(0,0)% +\epsfig{file=hugs_ret.pstex}% +\end{picture}% +\setlength{\unitlength}{0.00066700in}% +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +\begin{picture}(3624,4449)(889,-4798) +\end{picture} diff --git a/docs/rts/hugs_ret2.pstex b/docs/rts/hugs_ret2.pstex new file mode 100644 index 0000000000..74d081c40c --- /dev/null +++ b/docs/rts/hugs_ret2.pstex @@ -0,0 +1,130 @@ +%!PS-Adobe-2.0 EPSF-2.0 +%%Title: /tmp/xfig-fig007314 +%%Creator: fig2dev Version 3.1 Patchlevel 2 +%%CreationDate: Wed Oct 15 13:18:31 1997 +%%For: simonm@solander.dcs.gla.ac.uk (Simon Marlow,SM,,,,OCT99, ) +%%Orientation: Portrait +%%BoundingBox: 0 0 185 139 +%%Pages: 0 +%%BeginSetup +%%IncludeFeature: *PageSize Letter +%%EndSetup +%Magnification: 0.80 +%%EndComments +/$F2psDict 200 dict def +$F2psDict begin +$F2psDict /mtrx matrix put +/col-1 {0 setgray} bind def +/col0 {0.000 0.000 0.000 srgb} bind def +/col1 {0.000 0.000 1.000 srgb} bind def +/col2 {0.000 1.000 0.000 srgb} bind def +/col3 {0.000 1.000 1.000 srgb} bind def +/col4 {1.000 0.000 0.000 srgb} bind def +/col5 {1.000 0.000 1.000 srgb} bind def +/col6 {1.000 1.000 0.000 srgb} bind def +/col7 {1.000 1.000 1.000 srgb} bind def +/col8 {0.000 0.000 0.560 srgb} bind def +/col9 {0.000 0.000 0.690 srgb} bind def +/col10 {0.000 0.000 0.820 srgb} bind def +/col11 {0.530 0.810 1.000 srgb} bind def +/col12 {0.000 0.560 0.000 srgb} bind def +/col13 {0.000 0.690 0.000 srgb} bind def +/col14 {0.000 0.820 0.000 srgb} bind def +/col15 {0.000 0.560 0.560 srgb} bind def +/col16 {0.000 0.690 0.690 srgb} bind def +/col17 {0.000 0.820 0.820 srgb} bind def +/col18 {0.560 0.000 0.000 srgb} bind def +/col19 {0.690 0.000 0.000 srgb} bind def +/col20 {0.820 0.000 0.000 srgb} bind def +/col21 {0.560 0.000 0.560 srgb} bind def +/col22 {0.690 0.000 0.690 srgb} bind def +/col23 {0.820 0.000 0.820 srgb} bind def +/col24 {0.500 0.190 0.000 srgb} bind def +/col25 {0.630 0.250 0.000 srgb} bind def +/col26 {0.750 0.380 0.000 srgb} bind def +/col27 {1.000 0.500 0.500 srgb} bind def +/col28 {1.000 0.630 0.630 srgb} bind def +/col29 {1.000 0.750 0.750 srgb} bind def +/col30 {1.000 0.880 0.880 srgb} bind def +/col31 {1.000 0.840 0.000 srgb} bind def + +end +save +-28.0 181.0 translate +1 -1 scale + +/cp {closepath} bind def +/ef {eofill} bind def +/gr {grestore} bind def +/gs {gsave} bind def +/sa {save} bind def +/rs {restore} bind def +/l {lineto} bind def +/m {moveto} bind def +/rm {rmoveto} bind def +/n {newpath} bind def +/s {stroke} bind def +/sh {show} bind def +/slc {setlinecap} bind def +/slj {setlinejoin} bind def +/slw {setlinewidth} bind def +/srgb {setrgbcolor} bind def +/rot {rotate} bind def +/sc {scale} bind def +/sd {setdash} bind def +/ff {findfont} bind def +/sf {setfont} bind def +/scf {scalefont} bind def +/sw {stringwidth} bind def +/tr {translate} bind def +/tnt {dup dup currentrgbcolor + 4 -2 roll dup 1 exch sub 3 -1 roll mul add + 4 -2 roll dup 1 exch sub 3 -1 roll mul add + 4 -2 roll dup 1 exch sub 3 -1 roll mul add srgb} + bind def +/shd {dup dup currentrgbcolor 4 -2 roll mul 4 -2 roll mul + 4 -2 roll mul srgb} bind def +/$F2psBegin {$F2psDict begin /$F2psEnteredState save def} def +/$F2psEnd {$F2psEnteredState restore end} def +%%EndProlog + +$F2psBegin +10 setmiterlimit +n 0 792 m 0 0 l 612 0 l 612 792 l cp clip + 0.04800 0.04800 sc +/Helvetica ff 180.00 scf sf +975 1350 m +gs 1 -1 sc (Stack) col-1 sh gr +7.500 slw +% Polyline +n 600 3000 m 1800 3000 l gs col-1 s gr +% Polyline +n 600 2700 m 1800 2700 l gs col-1 s gr +% Polyline +gs clippath +2928 3495 m 3048 3525 l 2928 3555 l 3090 3555 l 3090 3495 l cp clip +n 1200 3150 m 2400 3150 l 2400 3525 l 3075 3525 l gs col-1 s gr gr + +% arrowhead +n 2928 3495 m 3048 3525 l 2928 3555 l col-1 s +% Polyline +gs clippath +2928 2820 m 3048 2850 l 2928 2880 l 3090 2880 l 3090 2820 l cp clip +n 1200 2850 m 3075 2850 l gs col-1 s gr gr + +% arrowhead +n 2928 2820 m 3048 2850 l 2928 2880 l col-1 s +% Polyline +n 3075 2700 m 4425 2700 l 4425 3075 l 3075 3075 l cp gs col-1 s gr +% Polyline +n 3075 3375 m 4425 3375 l 4425 3750 l 3075 3750 l cp gs col-1 s gr +/Helvetica ff 180.00 scf sf +3555 2955 m +gs 1 -1 sc (BCO) col-1 sh gr +/Helvetica ff 180.00 scf sf +3195 3630 m +gs 1 -1 sc (Return Value) col-1 sh gr +% Polyline +n 600 900 m 600 3300 l 1800 3300 l 1800 900 l gs col-1 s gr +$F2psEnd +rs diff --git a/docs/rts/hugs_ret2.pstex_t b/docs/rts/hugs_ret2.pstex_t new file mode 100644 index 0000000000..13208a3de1 --- /dev/null +++ b/docs/rts/hugs_ret2.pstex_t @@ -0,0 +1,13 @@ +\begin{picture}(0,0)% +\epsfig{file=hugs_ret2.pstex}% +\end{picture}% +\setlength{\unitlength}{0.00066700in}% +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +\begin{picture}(3849,2874)(589,-2923) +\end{picture} diff --git a/docs/rts/rts.tex b/docs/rts/rts.tex new file mode 100644 index 0000000000..158ae7e79a --- /dev/null +++ b/docs/rts/rts.tex @@ -0,0 +1,4683 @@ +% +% (c) The OBFUSCATION-THROUGH-GRATUITOUS-PREPROCESSOR-ABUSE Project, +% Glasgow University, 1990-1994 +% + +% TODO: +% +% o I (ADR) think it would be worth making the connection with CPS explicit. +% Now that we have explicit activation records (on the stack), we can +% explain the whole system in terms of CPS and tail calls --- with the +% one requirement that we carefuly distinguish stack-allocated objects +% from heap-allocated objects. + +% \documentstyle[preprint]{acmconf} +\documentclass[11pt]{article} +\oddsidemargin 0.1 in % Note that \oddsidemargin = \evensidemargin +\evensidemargin 0.1 in +\marginparwidth 0.85in % Narrow margins require narrower marginal notes +\marginparsep 0 in +\sloppy + +%\usepackage{epsfig} +\usepackage{shortvrb} +\MakeShortVerb{\@} + +%\newcommand{\note}[1]{{\em Note: #1}} +\newcommand{\note}[1]{{{\bf Note:}\sl #1}} +\newcommand{\ToDo}[1]{{{\bf ToDo:}\sl #1}} +\newcommand{\Arg}[1]{\mbox{${\tt arg}_{#1}$}} +\newcommand{\bottom}{\perp} + +\newcommand{\secref}[1]{Section~\ref{sec:#1}} +\newcommand{\figref}[1]{Figure~\ref{fig:#1}} +\newcommand{\Section}[2]{\section{#1}\label{sec:#2}} +\newcommand{\Subsection}[2]{\subsection{#1}\label{sec:#2}} +\newcommand{\Subsubsection}[2]{\subsubsection{#1}\label{sec:#2}} + +% DIMENSION OF TEXT: +\textheight 8.5 in +\textwidth 6.25 in + +\topmargin 0 in +\headheight 0 in +\headsep .25 in + + +\setlength{\parskip}{0.15cm} +\setlength{\parsep}{0.15cm} +\setlength{\topsep}{0cm} % Reduces space before and after verbatim, + % which is implemented using trivlist +\setlength{\parindent}{0cm} + +\renewcommand{\textfraction}{0.2} +\renewcommand{\floatpagefraction}{0.7} + +\begin{document} + +\title{The STG runtime system (revised)} +\author{Simon Peyton Jones \\ Microsoft Research Ltd., Cambridge \and +Simon Marlow \\ Microsoft Research Ltd., Cambridge \and +Alastair Reid \\ Yale University} + +\maketitle + +\tableofcontents +\newpage + +\part{Introduction} +\Section{Overview}{overview} + +This document describes the GHC/Hugs run-time system. It serves as +a Glasgow/Yale/Nottingham ``contract'' about what the RTS does. + +\Subsection{New features compared to GHC 3.xx}{new-features} + +\begin{itemize} +\item The RTS supports mixed compiled/interpreted execution, so +that a program can consist of a mixture of GHC-compiled and Hugs-interpreted +code. + +\item The RTS supports concurrency by default. +This has some costs (eg we can't do hardware stack checks) but +reduces the number of different configurations we need to support. + +\item CAFs are only retained if they are +reachable. Since they are referred to by implicit references buried +in code, this means that the garbage collector must traverse the whole +accessible code tree. This feature eliminates a whole class of painful +space leaks. + +\item A running thread has only one stack, which contains a mixture of +pointers and non-pointers. \secref{TSO} describes how we find out +which is which. (GHC has used two stacks for some while. Using one +stack instead of two reduces register pressure, reduces the size of +update frames, and eliminates ``stack-stubbing'' instructions.) + +\item The ``return in registers'' return convention has been dropped +because it was complicated and doesn't work well on register-poor +architectures. It has been partly replaced by unboxed tuples +(\secref{unboxed-tuples}) which allow the programmer to +explicitly state where results should be returned in registers (or on +the stack) instead of on the heap. + +\item Exceptions are supported by the RTS. + +\item Weak Pointers generalise the previously available Foreign Object +interface. + +\item The garbage collector supports a number of new features, +including a dynamically resizable heap and multiple generations with +aging within a generation. + +\end{itemize} + +\Subsection{Wish list}{wish-list} + +Here's a list of things we'd like to support in the future. +\begin{itemize} +\item Interrupts, speculative computation. + +\item +The SM could tune the size of the allocation arena, the number of +generations, etc taking into account residency, GC rate and page fault +rate. + +\item +We could trigger a GC when all threads are blocked waiting for IO if +the allocation arena (or some of the generations) are nearly full. + +\end{itemize} + +\Subsection{Configuration}{configuration} + +Some of the above features are expensive or less portable, so we +envision building a number of different configurations supporting +different subsets of the above features. + +You can make the following choices: +\begin{itemize} +\item +Support for parallelism. There are three mutually-exclusive choices. + +\begin{description} +\item[@SEQUENTIAL@] Support for concurrency but not for parallelism. +\item[@GRANSIM@] Concurrency support and simulated parallelism. +\item[@PARALLEL@] Concurrency support and real parallelism. +\end{description} + +\item @PROFILING@ adds cost-centre profiling. + +\item @TICKY@ gathers internal statistics (often known as ``ticky-ticky'' code). + +\item @DEBUG@ does internal consistency checks. + +\item Persistence. (well, not yet). + +\item +Which garbage collector to use. At the moment we +only anticipate one, however. +\end{itemize} + +\Subsection{Glossary}{glossary} + +\ToDo{This terminology is not used consistently within the document. +If you find something which disagrees with this terminology, fix the +usage.} + +In the type system, we have boxed and unboxed types. + +\begin{itemize} + +\item A \emph{pointed} type is one that contains $\bot$. Variables with +pointed types are the only things which can be lazily evaluated. In +the STG machine, this means that they are the only things that can be +\emph{entered} or \emph{updated} and it requires that they be boxed. + +\item An \emph{unpointed} type is one that does not contain $\bot$. +Variables with unpointed types are never delayed --- they are always +evaluated when they are constructed. In the STG machine, this means +that they cannot be \emph{entered} or \emph{updated}. Unpointed objects +may be boxed (like @Array#@) or unboxed (like @Int#@). + +\end{itemize} + +In the implementation, we have different kinds of objects: + +\begin{itemize} + +\item \emph{boxed} objects are heap objects used by the evaluators + +\item \emph{unboxed} objects are not heap allocated + +\item \emph{stack} objects are allocated on the stack + +\item \emph{closures} are objects which can be \emph{entered}. +They are always boxed and always have boxed types. +They may be in WHNF or they may be unevaluated. + +\item A \emph{thunk} is a (representation of) a value of a \emph{pointed} +type which is \emph{not} in WHNF. + +\item A \emph{value} is an object in WHNF. It can be pointed or unpointed. + +\end{itemize} + + + +At the hardware level, we have \emph{word}s and \emph{pointer}s. + +\begin{itemize} + +\item A \emph{word} is (at least) 32 bits and can hold either a signed +or an unsigned int. + +\item A \emph{pointer} is (at least) 32 bits and big enough to hold a +function pointer or a data pointer. + +\end{itemize} + +Occasionally, a field of a data structure must hold either a word or a +pointer. In such circumstances, it is \emph{not safe} to assume that +words and pointers are the same size. \ToDo{GHC currently makes words +the same size as pointers to reduce complexity in the code +generator/RTS. It would be useful to relax this restriction, and have +eg. 32-bit Ints on a 64-bit machine.} + +% should define terms like SRT, CAF, PAP, etc. here? --KSW 1999-03 + +\subsection{Subtle Dependencies} + +Some decisions have very subtle consequences which should be written +down in case we want to change our minds. + +\begin{itemize} + +\item + +If the garbage collector is allowed to shrink the stack of a thread, +we cannot omit the stack check in return continuations +(\secref{heap-and-stack-checks}). + +\item + +When we return to the scheduler, the top object on the stack is a closure. +The scheduler restarts the thread by entering the closure. + +\secref{hugs-return-convention} discusses how Hugs returns an +unboxed value to GHC and how GHC returns an unboxed value to Hugs. + +\item + +When we return to the scheduler, we need a few empty words on the stack +to store a closure to reenter. \secref{heap-and-stack-checks} +discusses who does the stack check and how much space they need. + +\item + +Heap objects never contain slop --- this is required if we want to +support mostly-copying garbage collection. + +This is a big problem when updating since the updatee is usually +bigger than an indirection object. The fix is to overwrite the end of +the updatee with ``slop objects'' (described in +\secref{slop-objects}). This is hard to arrange if we do +\emph{lazy} blackholing (\secref{lazy-black-holing}) so we +currently plan to blackhole an object when we push the update frame. + +% Idea: have specialised update code for various common sizes of +% updatee, the update frame hence encodes the length of the object. +% Specialised indirections will also encode the length of the object. A +% generic version of the update code will overwrite the slop with a slop +% object. We can do the same thing for blackhole objects, or just have +% a generic version that is the same size as an indirection and +% overwrite the slop with a slop object when blackholing. So: does this +% avoid the need to do eager black holing? + +\item + +Info tables for constructors contain enough information to decide which +return convention they use. This allows Hugs to use a single piece of +entry code for all constructors and insulates Hugs from changes in the +choice of return convention. + +\end{itemize} + +\Section{Source Language}{source-language} + +\Subsection{Explicit Allocation}{explicit-allocation} + +As in the original STG machine, (almost) all heap allocation is caused +by executing a let(rec). Since we no longer support the return in +registers convention for data constructors, constructors now cause heap +allocation and so they should be let-bound. + +For example, we now write +\begin{verbatim} +> cons = \ x xs -> let r = (:) x xs in r +@ +instead of +\begin{verbatim} +> cons = \ x xs -> (:) x xs +\end{verbatim} + +\note{For historical reasons, GHC doesn't use this syntax --- but it should.} + +\Subsection{Unboxed tuples}{unboxed-tuples} + +Functions can take multiple arguments as easily as they can take one +argument: there's no cost for adding another argument. But functions +can only return one result: the cost of adding a second ``result'' is +that the function must construct a tuple of ``results'' on the heap. +The assymetry is rather galling and can make certain programming +styles quite expensive. For example, consider a simple state transformer +monad: +\begin{verbatim} +> type S a = State -> (a,State) +> bindS m k s0 = case m s0 of { (a,s1) -> k a s1 } +> returnS a s = (a,s) +> getS s = (s,s) +> setS s _ = ((),s) +\end{verbatim} +Here, every use of @returnS@, @getS@ or @setS@ constructs a new tuple +in the heap which is instantly taken apart (and becomes garbage) by +the case analysis in @bind@. Even a short state-transformer program +will construct a lot of these temporary tuples. + +Unboxed tuples provide a way for the programmer to indicate that they +do not expect a tuple to be shared and that they do not expect it to +be allocated in the heap. Syntactically, unboxed tuples are just like +single constructor datatypes except for the annotation @unboxed@. +\begin{verbatim} +> data unboxed AAndState# a = AnS a State +> type S a = State -> AAndState# a +> bindS m k s0 = case m s0 of { AnS a s1 -> k a s1 } +> returnS a s = AnS a s +> getS s = AnS s s +> setS s _ = AnS () s +\end{verbatim} +Semantically, unboxed tuples are just unlifted tuples and are subject +to the same restrictions as other unpointed types. + +Operationally, unboxed tuples are never built on the heap. When +an unboxed tuple is returned, it is returned in multiple registers +or multiple stack slots. At first sight, this seems a little strange +but it's no different from passing double precision floats in two +registers. + +Notes: +\begin{itemize} +\item +Unboxed tuples can only have one constructor and that +thunks never have unboxed types --- so we'll never try to update an +unboxed constructor. The restriction to a single constructor is +largely to avoid garbage collection complications. + +\item +The core syntax does not allow variables to be bound to +unboxed tuples (ie in default case alternatives or as function arguments) +and does not allow unboxed tuples to be fields of other constructors. +However, there's no harm in allowing it in the source syntax as a +convenient, but easily removed, syntactic sugar. + +\item +The compiler generates a closure of the form +\begin{verbatim} +> c = \ x y z -> C x y z +\end{verbatim} +for every constructor (whether boxed or unboxed). + +This closure is normally used during desugaring to ensure that +constructors are saturated and to apply any strictness annotations. +They are also used when returning unboxed constructors to the machine +code evaluator from the bytecode evaluator and when a heap check fails +in a return continuation for an unboxed-tuple scrutinee. + +\end{itemize} + +\Subsection{STG Syntax}{stg-syntax} + + +\ToDo{Insert STG syntax with appropriate changes.} + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\part{System Overview} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +This part is concerned with defining the external interfaces of the +major components of the system; the next part is concerned with their +inner workings. + +The major components of the system are: +\begin{itemize} + +\item + +The evaluators (\secref{sm-overview}) are responsible for +evaluating heap objects. The system supports two evaluators: the +machine code evaluator; and the bytecode evaluator. + +\item + +The scheduler (\secref{scheduler-overview}) acts as the +coordinator for the whole system. It is responsible for switching +between evaluators, switching between threads, garbage collection, +communication between multiple processors, etc. + +\item + +The storage manager (\secref{evaluators-overview}) is +responsible for allocating blocks of contiguous memory and for garbage +collection. + +\item + +The loader (\secref{loader-overview}) is responsible for +loading machine code and bytecode files from the file system and for +resolving references between separately compiled modules. + +\item + +The compilers (\secref{compilers-overview}) generate machine +code and bytecode files which can be loaded by the loader. + +\end{itemize} + +\ToDo{Insert diagram showing all components underneath the scheduler +and communicating only with the scheduler} + + +\Section{The Evaluators}{evaluators-overview} + +There are two evaluators: a machine code evaluator and a bytecode +evaluator. The evaluators task is to evaluate code within a thread +until one of the following happens: + +\begin{itemize} +\item heap overflow +\item stack overflow +\item it is preempted +\item it blocks in one of the concurrency primitives +\item it performs a safe ccall +\item it needs to switch to the other evaluator. +\end{itemize} + +The evaluators expect to find a closure on top of the thread's stack +and terminate with a closure on top of the thread's stack. + +\Subsection{Evaluation Model}{evaluation-model} + +Whilst the evaluators differ internally, they share a common +evaluation model and many object representations. + +\Subsubsection{Heap objects}{heap-objects-overview} + +The choice of heap and stack objects used by the evaluators is tightly +bound to the evaluation model. This section provides an overview of +the most important heap and stack objects; further details are given +later. + +All heap objects look like this: + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Header} & \emph{Payload} \\ \hline +\end{tabular} +\end{center} + +The headers vary between different kinds of object but they all start +with a pointer to a pair consisting of an \emph{info table} and some +\emph{entry code}. The info table is used both by the evaluators and +by the storage manager and contains a @type@ field which identifies +which kind of heap object uses it and determines the interpretation of +the payload and of the other fields of the info table. The entry code +is some machine code used by the machine code evaluator to evaluate +closures and raises an error for other kinds of objects. + +The major kinds of heap object used are as follows. (For simplicity, +this description omits certain optimisations and extra fields required +by the garbage collector.) + +\begin{description} + +\item[Constructors] are used to represent data constructors. Their +payload consists of the fields of the constructor; the tag of the +constructor is stored in the info table. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@CONSTR@ & \emph{Fields} \\ \hline +\end{tabular} +\end{center} + +\item[Primitive objects] are used to represent objects with unlifted +types which are too large to fit in a register (or stack slot) or for +which sharing must be preserved. Primitive objects include large +objects such as multiple precision integers and immutable arrays and +mutable objects such as mutable arrays, mutable variables, MVar's, +IVar's and foreign object pointers. Since primitive objects are not +lifted, they cannot be entered. Their payload varies according to the +kind of object. + +\item[Function closures] are used to represent functions. Their +payload (if any) consists of the free variables of the function. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@FUN@ & \emph{Free Variables} \\ \hline +\end{tabular} +\end{center} + +Function closures are only generated by the machine code compiler. + +\item[Thunks] are used to represent unevaluated expressions which will +be updated with their result. Their payload (if any) consists of the +free variables of the function. The entry code for a thunk starts by +pushing an \emph{update frame} onto the stack. When evaluation of the +thunk completes, the update frame will cause the thunk to be +overwritten again with an \emph{indirection} to the result of the +thunk, which is always a constructor or a partial application. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@THUNK@ & \emph{Free Variables} \\ \hline +\end{tabular} +\end{center} + +Thunks are only generated by the machine code evaluator. + +\item[Byte-code Objects (@BCO@s)] are generated by the bytecode +compiler. In conjunction with \emph{updatable applications} and +\emph{non-updatable applications} they are used to represent +functions, unevaluated expressions and return addresses. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@BCO@ & \emph{Constant Pool} & \emph{Bytecodes} \\ \hline +\end{tabular} +\end{center} + +\item[Non-updatable (Partial) Applications] are used to represent the +application of a function to an insufficient number of arguments. +Their payload consists of the function and the arguments received so far. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@PAP@ & \emph{Function Closure} & \emph{Arguments} \\ \hline +\end{tabular} +\end{center} + +@PAP@s are used when a function is applied to too few arguments and by +code generated by the lambda-lifting phase of the bytecode compiler. + +\item[Updatable Applications] are used to represent the application of +a function to a sufficient number of arguments. Their payload +consists of the function and its arguments. + +Updateable applications are like thunks: on entering an updateable +application, the evaluators push an \emph{update frame} onto the stack +and overwrite the application with a \emph{black hole}; when +evaluation completes, the evaluators overwrite the application with an +\emph{indirection} to the result of the application. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@AP@ & \emph{Function Closure} & \emph{Arguments} \\ \hline +\end{tabular} +\end{center} + +@AP@s are only generated by the bytecode compiler. + +\item[Black holes] are used to mark updateable closures which are +currently being evaluated. ``Black holing'' an object cures a +potential space leak and detects certain classes of infinite loops. +More imporantly, black holes act as synchronisation objects between +separate threads: if a second thread tries to enter an updateable +closure which is already being evaluated, the second thread is added +to a list of blocked threads and the thread is suspended. + +When evaluation of the black-holed closure completes, the black hole +is overwritten with an indirection to the result of the closure and +any blocked threads are restored to the runnable queue. + +Closures are overwritten by black-holes during a ``lazy black-holing'' +phase which runs on each thread when it returns to the scheduler. +\ToDo{section describing lazy black-holing}. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@BLACKHOLE@ & \emph{Blocked threads} \\ \hline +\end{tabular} +\end{center} + +\ToDo{In a single threaded system, it's trivial to detect infinite +loops: reentering a BLACKHOLE is always an error. How easy is it in a +multi-threaded system?} + +\item[Indirections] are used to update an unevaluated closure with its +(usually fully evaluated) result in situations where it isn't possible +to perform an update in place. (In the current system, we always +update with an indirection to avoid duplicating the result when doing +an update in place.) + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@IND@ & \emph{Closure} \\ \hline +\end{tabular} +\end{center} + +Indirections needn't always point to a closure in WHNF. They can +point to a chain of indirections which point to an evaluated closure. + +\item[Thread State Objects (@TSO@s)] represent Haskell threads. Their +payload consists of some per-thread information such as the Thread ID +and the status of the thread (runnable, blocked etc.), and the +thread's stack. See @TSO.h@ for the full story. @TSO@s may be +resized by the scheduler if its stack is too small or too large. + +The thread stack grows downwards from higher to lower addresses. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@TSO@ & \emph{Thread info} & \emph{Stack} \\ \hline +\end{tabular} +\end{center} + +\end{description} + +\Subsubsection{Stack objects}{stack-objects-overview} + +The stack contains a mixture of \emph{pending arguments} and +\emph{stack objects}. + +Pending arguments are arguments to curried functions which have not +yet been incorporated into an activation frame. For example, when +evaluating @let { g x y = x + y; f x = g{x} } in f{3,4}@, the +evaluator pushes both arguments onto the stack and enters @f@. @f@ +only requires one argument so it leaves the second argument as a +\emph{pending argument}. The pending argument remains on the stack +until @f@ calls @g@ which requires two arguments: the argument passed +to it by @f@ and the pending argument which was passed to @f@. + +Unboxed pending arguments are always preceeded by a ``tag'' which says +how large the argument is. This allows the garbage collector to +locate pointers within the stack. + +There are three kinds of stack object: return addresses, update frames +and seq frames. All stack objects look like this + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Header} & \emph{Payload} \\ \hline +\end{tabular} +\end{center} + +As with heap objects, the header starts with a pointer to a pair +consisting of an \emph{info table} and some \emph{entry code}. + +\begin{description} + +\item[Return addresses] are used to cause selection and execution of +case alternatives when a constructor is returned. Return addresses +generated by the machine code compiler look like this: + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@RET_XXX@ & \emph{Free Variables of the case alternatives} \\ \hline +\end{tabular} +\end{center} + +The free variables are a mixture of pointers and non-pointers whose +layout is described by a bitmask in the info table. + +There are several kinds of @RET_XXX@ return address - see +\secref{activation-records} for the details. + +Return addresses generated by the bytecode compiler look like this: +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@BCO_RET@ & \emph{BCO} & \emph{Free Variables of the case alternatives} \\ \hline +\end{tabular} +\end{center} + +There is just one @BCO_RET@ info pointer. We avoid needing different +@BCO_RET@s for each stack layout by tagging unboxed free variables as +though they were pending arguments. + +\item[Update frames] are used to trigger updates. When an update +frame is entered, it overwrites the updatee with an indirection to the +result, restarts any threads blocked on the @BLACKHOLE@ and returns to +the stack object underneath the update frame. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@UPDATE_FRAME@ & \emph{Next Update Frame} & \emph{Updatee} \\ \hline +\end{tabular} +\end{center} + +\item[Seq frames] are used to implement the polymorphic @seq@ +primitive. They are a special kind of update frame, and are linked on +the update frame list. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@SEQ_FRAME@ & \emph{Next Update Frame} \\ \hline +\end{tabular} +\end{center} + +\item[Stop frames] are put on the bottom of each thread's stack, and +act as sentinels for the update frame list (i.e. the last update frame +points to the stop frame). Returning to a stop frame terminates the +thread. Stop frames have no payload: + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +@SEQ_FRAME@ \\ \hline +\end{tabular} +\end{center} + +\end{description} + +\Subsubsection{Case expressions}{case-expr-overview} + +In the STG language, all evaluation is triggered by evaluating a case +expression. When evaluating a case expression @case e of alts@, the +evaluators pushes a return address onto the stack and evaluate the +expression @e@. When @e@ eventually reduces to a constructor, the +return address on the stack is entered. The details of how the +constructor is passed to the return address and how the appropriate +case alternative is selected vary between evaluators. + +Case expressions for unboxed data types are essentially the same: the +case expression pushes a return address onto the stack before +evaluating the scrutinee; when a function returns an unboxed value, it +enters the return address on top of the stack. + + +\Subsubsection{Function applications}{fun-app-overview} + +In the STG language, all function calls are tail calls. The arguments +are pushed onto the stack and the function closure is entered. If any +arguments are unboxed, they must be tagged as unboxed pending +arguments. Entering a closure is just a special case of calling a +function with no arguments. + + +\Subsubsection{Let expressions}{let-expr-overview} + +In the STG language, almost all heap allocation is caused by let +expressions. Filling in the contents of a set of mutually recursive +heap objects is simple enough; the only difficulty is that once the +heap space has been allocated, the thread must not return to the +scheduler until after the objects are filled in. + + +\Subsubsection{Primitive operations}{primop-overview} + +\ToDo{} + +Most primops are simple, some aren't. + + + + + + +\Section{Scheduler}{scheduler-overview} + +The Scheduler is the heart of the run-time system. A running program +consists of a single running thread, and a list of runnable and +blocked threads. A thread is represented by a \emph{Thread Status +Object} (TSO), which contains a few words status information and a +stack. Except for the running thread, all threads have a closure on +top of their stack; the scheduler restarts a thread by entering an +evaluator which performs some reduction and returns to the scheduler. + +\Subsection{The scheduler's main loop}{scheduler-main-loop} + +The scheduler consists of a loop which chooses a runnable thread and +invokes one of the evaluators which performs some reduction and +returns. + +The scheduler also takes care of system-wide issues such as heap +overflow or communication with other processors (in the parallel +system) and thread-specific problems such as stack overflow. + +\Subsection{Creating a thread}{create-thread} + +Threads are created: + +\begin{itemize} + +\item + +When the scheduler is first invoked. + +\item + +When a message is received from another processor (I think). (Parallel +system only.) + +\item + +When a C program calls some Haskell code. + +\item + +By @forkIO@, @takeMVar@ and (maybe) other Concurrent Haskell primitives. + +\end{itemize} + + +\Subsection{Restarting a thread}{thread-restart} + +When the scheduler decides to run a thread, it has to decide which +evaluator to use. It does this by looking at the type of the closure +on top of the stack. +\begin{itemize} +\item @BCO@ $\Rightarrow$ bytecode evaluator +\item @FUN@ or @THUNK@ $\Rightarrow$ machine code evaluator +\item @CONSTR@ $\Rightarrow$ machine code evaluator +\item other $\Rightarrow$ either evaluator. +\end{itemize} + +The only surprise in the above is that the scheduler must enter the +machine code evaluator if there's a constructor on top of the stack. +This allows the bytecode evaluator to return a constructor to a +machine code return address by pushing the constructor on top of the +stack and returning to the scheduler. If the return address under the +constructor is @HUGS_RET@, the entry code for @HUGS_RET@ will +rearrange the stack so that the return @BCO@ is on top of the stack +and return to the scheduler which will then call the bytecode +evaluator. There is little point in trying to shorten this slightly +indirect route since it is will happen very rarely if at all. + +\note{As an optimisation, we could store the choice of evaluator in +the TSO status whenever we leave the evaluator. This is required for +any thread, no matter what state it is in (blocked, stack overflow, +etc). It isn't clear whether this would accomplish anything.} + +\Subsection{Returning from a thread}{thread-return} + +The evaluators return to the scheduler when any of the following +conditions arise: + +\begin{itemize} +\item A heap check fails, and a garbage collection is required. + +\item A stack check fails, and the scheduler must either enlarge the +current thread's stack, or flag an out of memory condition. + +\item A thread enters a closure built by the other evaluator. That +is, when the bytecode interpreter enters a closure compiled by GHC or +when the machine code evaluator enters a BCO. + +\item A thread returns to a return continuation built by the other +evaluator. That is, when the machine code evaluator returns to a +continuation built by Hugs or when the bytecode evaluator returns to a +continuation built by GHC. + +\item The evaluator needs to perform a ``safe'' C call +(\secref{c-calls}). + +\item The thread becomes blocked. This happens when a thread requires +the result of a computation currently being performed by another +thread, or it reads a synchronisation variable that is currently empty +(\secref{MVAR}). + +\item The thread is preempted (the preemption mechanism is described +in \secref{thread-preemption}). + +\item The thread terminates. +\end{itemize} + +Except when the thread terminates, the thread always terminates with a +closure on the top of the stack. The mechanism used to trigger the +world switch and the choice of closure left on top of the stack varies +according to which world is being left and what is being returned. + +\Subsubsection{Leaving the bytecode evaluator}{hugs-to-ghc-switch} + +\paragraph{Entering a machine code closure} + +When it enters a closure, the bytecode evaluator performs a switch +based on the type of closure (@AP@, @PAP@, @Ind@, etc). On entering a +machine code closure, it returns to the scheduler with the closure on +top of the stack. + +\paragraph{Returning a constructor} + +When it enters a constructor, the bytecode evaluator tests the return +continuation on top of the stack. If it is a machine code +continuation, it returns to the scheduler with the constructor on top +of the stack. + +\note{This is why the scheduler must enter the machine code evaluator +if it finds a constructor on top of the stack.} + +\paragraph{Returning an unboxed value} + +\note{Hugs doesn't support unboxed values in source programs but they +are used for a few complex primops.} + +When it returns an unboxed value, the bytecode evaluator tests the +return continuation on top of the stack. If it is a machine code +continuation, it returns to the scheduler with the tagged unboxed +value and a special closure on top of the stack. When the closure is +entered (by the machine code evaluator), it returns the unboxed value +on top of the stack to the return continuation under it. + +The runtime library for GHC provides one of these closures for each unboxed +type. Hugs cannot generate them itself since the entry code is really +very tricky. + +\paragraph{Heap/Stack overflow and preemption} + +The bytecode evaluator tests for heap/stack overflow and preemption +when entering a BCO and simply returns with the BCO on top of the +stack. + +\Subsubsection{Leaving the machine code evaluator}{ghc-to-hugs-switch} + +\paragraph{Entering a BCO} + +The entry code for a BCO pushes the BCO onto the stack and returns to +the scheduler. + +\paragraph{Returning a constructor} + +We avoid the need to test return addresses in the machine code +evaluator by pushing a special return address on top of a pointer to +the bytecode return continuation. \figref{hugs-return-stack1} +shows the state of the stack just before evaluating the scrutinee. + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| bco |--> BCO ++----------+ +| HUGS_RET | ++----------+ +\end{verbatim} +%\input{hugs_return1.pstex_t} +\end{center} +\caption{Stack layout for evaluating a scrutinee} +\label{fig:hugs-return-stack1} +\end{figure} + +This return address rearranges the stack so that the bco pointer is +above the constructor on the stack (as shown in +\figref{hugs-boxed-return}) and returns to the scheduler. + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| con |--> Constructor ++----------+ +| bco |--> BCO ++----------+ +\end{verbatim} +%\input{hugs_return2.pstex_t} +\end{center} +\caption{Stack layout for entering a Hugs return address} +\label{fig:hugs-boxed-return} +\end{figure} + +\paragraph{Returning an unboxed value} + +We avoid the need to test return addresses in the machine code +evaluator by pushing a special return address on top of a pointer to +the bytecode return continuation. This return address rearranges the +stack so that the bco pointer is above the tagged unboxed value (as +shown in \figref{hugs-entering-unboxed-return}) and returns to the +scheduler. + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| 1# | ++----------+ +| I# | ++----------+ +| bco |--> BCO ++----------+ +\end{verbatim} +%\input{hugs_return2.pstex_t} +\end{center} +\caption{Stack layout for returning an unboxed value} +\label{fig:hugs-entering-unboxed-return} +\end{figure} + +\paragraph{Heap/Stack overflow and preemption} + +\ToDo{} + + +\Subsection{Preempting a thread}{thread-preemption} + +Strictly speaking, threads cannot be preempted --- the scheduler +merely sets a preemption request flag which the thread must arrange to +test on a regular basis. When an evaluator finds that the preemption +request flag is set, it pushes an appropriate closure onto the stack +and returns to the scheduler. + +In the bytecode interpreter, the flag is tested whenever we enter a +closure. If the preemption flag is set, it leaves the closure on top +of the stack and returns to the scheduler. + +In the machine code evaluator, the flag is only tested when a heap or +stack check fails. This is less expensive than testing the flag on +entering every closure but runs the risk that a thread will enter an +infinite loop which does not allocate any space. If the flag is set, +the evaluator returns to the scheduler exactly as if a heap check had +failed. + +\Subsection{``Safe'' and ``unsafe'' C calls}{c-calls} + +There are two ways of calling C: + +\begin{description} + +\item[``Unsafe'' C calls] are used if the programer is certain that +the C function will not do anything dangerous. Unsafe C calls are +faster but must be hand-checked by the programmer. + +Dangerous things include: + +\begin{itemize} + +\item + +Call a system function such as @getchar@ which might block +indefinitely. This is dangerous because we don't want the entire +runtime system to block just because one thread blocks. + +\item + +Call an RTS function which will block on the RTS access semaphore. +This would lead to deadlock. + +\item + +Call a Haskell function. This is just a special case of calling an +RTS function. + +\end{itemize} + +Unsafe C calls are performed by pushing the arguments onto the C stack +and jumping to the C function's entry point. On exit, the result of +the function is in a register which is returned to the Haskell code as +an unboxed value. + +\item[``Safe'' C calls] are used if the programmer suspects that the +thread may do something dangerous. Safe C calls are relatively slow +but are less problematic. + +Safe C calls are performed by pushing the arguments onto the Haskell +stack, pushing a return continuation and returning a \emph{C function +descriptor} to the scheduler. The scheduler suspends the Haskell thread, +spawns a new operating system thread which pops the arguments off the +Haskell stack onto the C stack, calls the C function, pushes the +function result onto the Haskell stack and informs the scheduler that +the C function has completed and the Haskell thread is now runnable. + +\end{description} + +The bytecode evaluator will probably treat all C calls as being safe. + +\ToDo{It might be good for the programmer to indicate how the program +is unsafe. For example, if we distinguish between C functions which +might call Haskell functions and those which might block, we could +perform an unsafe call for blocking functions in a single-threaded +system or, perhaps, in a multi-threaded system which only happens to +have a single thread at the moment.} + + + +\Section{The Storage Manager}{sm-overview} + +The storage manager is responsible for managing the heap and all +objects stored in it. It provides special support for lazy evaluation +and for foreign function calls. + +\Subsection{SM support for lazy evaluation}{sm-lazy-evaluation} + +\begin{itemize} +\item + +Indirections are shorted out. + +\item + +Update frames pointing to unreachable objects are squeezed out. + +\ToDo{Part IV suggests this doesn't happen.} + +\item + +Adjacent update frames (for different closures) are compressed to a +single update frame pointing to a single black hole. + +\end{itemize} + + +\Subsection{SM support for foreign function calls}{sm-foreign-calls} + +\begin{itemize} + +\item + +Stable pointers allow other languages to access Haskell objects. + +\item + +Weak pointers and foreign objects provide finalisation support for +Haskell references to external objects. + +\end{itemize} + +\Subsection{Misc}{sm-misc} + +\begin{itemize} + +\item + +If the stack contains a large amount of free space, the storage +manager may shrink the stack. If it shrinks the stack, it guarantees +never to leave less than @MIN_SIZE_SHRUNKEN_STACK@ empty words on the +stack when it does so. + +\item + +For efficiency reasons, very large objects (eg large arrays and TSOs) +are not moved if possible. + +\end{itemize} + + +\Section{The Compilers}{compilers-overview} + +Need to describe interface files, format of bytecode files, symbols +defined by machine code files. + +\Subsection{Interface Files}{interface-files} + +Here's an example - but I don't know the grammar - ADR. +\begin{verbatim} +_interface_ Main 1 +_exports_ +Main main ; +_declarations_ +1 main _:_ IOBase.IO PrelBase.();; +\end{verbatim} + +\Subsection{Bytecode files}{bytecode-files} + +(All that matters here is what the loader sees.) + +\Subsection{Machine code files}{asm-files} + +(Again, all that matters is what the loader sees.) + +\Section{The Loader}{loader-overview} + +In a batch mode system, we can statically link all the modules +together. In an interactive system we need a loader which will +explicitly load and unload individual modules (or, perhaps, blocks of +mutually dependent modules) and resolve references between modules. + +While many operating systems provide support for dynamic loading and +will automatically resolve cross-module references for us, we generally +cannot rely on being able to load mutually dependent modules. + +A portable solution is to perform some of the linking ourselves. Each module +should provide three global symbols: +\begin{itemize} +\item +An initialisation routine. (Might also be used for finalisation.) +\item +A table of symbols it exports. +Entries in this table consist of the symbol name and the address of the +name's value. +\item +A table of symbols it imports. +Entries in this table consist of the symbol name and a list of references +to that symbol. +\end{itemize} + +On loading a group of modules, the loader adds the contents of the +export lists to a symbol table and then fills in all the references in the +import lists. + +References in import lists are of two types: +\begin{description} +\item[ References in machine code ] + +The most efficient approach is to patch the machine code directly, but +this will be a lot of work, very painful to port and rather fragile. + +Alternatively, the loader could store the value of each symbol in the +import table for each module and the compiled code can access all +external objects through the import table. This requires that the +import table be writable but does not require that the machine code or +info tables be writable. + +\item[ References in data structures (SRTs and static data constructors) ] + +Either we patch the SRTs and constructors directly or we somehow use +indirections through the symbol table. Patching the SRTs requires +that we make them writable and prevents us from making effective use +of virtual memories that use copy-on-write policies (this only makes a +difference if we want to run several copies of the same program +simultaneously). Using an indirection is possible but tricky. + +Note: We could avoid patching machine code if all references to +external references went through the SRT --- then we just have one +thing to patch. But the SRT always contains a pointer to the closure +rather than the fast entry point (say), so we'd take a big performance +hit for doing this. + +\end{description} + +Using the above scheme, all accesses to ``external'' objects involve a +layer of indirection. To avoid this overhead, the machine code +compiler might provide a way for the programmer to specify which +modules will be statically linked and which will be dynamically linked +--- the idea being that statically linked code and data will be +accessed directly. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\part{Internal details} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +This part is concerned with the internal details of the components +described in the previous part. + +The major components of the system are: +\begin{itemize} +\item The scheduler (\secref{scheduler-internals}) +\item The storage manager (\secref{storage-manager-internals}) +\item The evaluators +\item The loader +\item The compilers +\end{itemize} + +\Section{The Scheduler}{scheduler-internals} + +\ToDo{Detailed description of scheduler} + +Many heap objects contain fields allowing them to be inserted onto lists +during evaluation or during garbage collection. The lists required by +the evaluator and storage manager are as follows. + +\begin{itemize} + +\item 4 lists of threads: runnable threads, sleeping threads, threads +waiting for timeout and threads waiting for I/O. + +\item The \emph{mutables list} is a list of all objects in the old +generation which might contain pointers into the new generation. Most +of the objects on this list are indirections (\secref{IND}) +or ``mutable.'' (\secref{mutables}.) + +\item The \emph{Foreign Object list} is a list of all foreign objects + which have not yet been deallocated. (\secref{FOREIGN}.) + +\item The \emph{Spark pool} is a doubly(?) linked list of Spark objects +maintained by the parallel system. (\secref{SPARK}.) + +\item The \emph{Blocked Fetch list} (or +lists?). (\secref{BLOCKED_FETCH}.) + +\item For each thread, there is a list of all update frames on the +stack. (\secref{data-updates}.) + +\item The Stable Pointer Table is a table of pointers to objects which +are known to the outside world and must be retained by the garbage +collector even if they are not accessible from within the heap. + +\end{itemize} + +\ToDo{The links for these fields are usually inserted immediately +after the fixed header except ...} + + + +\Section{The Storage Manager}{storage-manager-internals} + +\subsection{Misc Text looking for a home} + +A \emph{value} may be: +\begin{itemize} +\item \emph{Boxed}, i.e.~represented indirectly by a pointer to a heap object (e.g.~foreign objects, arrays); or +\item \emph{Unboxed}, i.e.~represented directly by a bit-pattern in one or more registers (e.g.~@Int#@ and @Float#@). +\end{itemize} +All \emph{pointed} values are \emph{boxed}. + + +\Subsection{Heap Objects}{heap-objects} +\label{sec:fixed-header} + +\begin{figure} +\begin{center} +\input{closure} +\end{center} +\ToDo{Fix this picture} +\caption{A closure} +\label{fig:closure} +\end{figure} + +Every \emph{heap object} is a contiguous block of memory, consisting +of a fixed-format \emph{header} followed by zero or more \emph{data +words}. + +The header consists of the following fields: +\begin{itemize} +\item A one-word \emph{info pointer}, which points to +the object's static \emph{info table}. +\item Zero or more \emph{admin words} that support +\begin{itemize} +\item Profiling (notably a \emph{cost centre} word). + \note{We could possibly omit the cost centre word from some + administrative objects.} +\item Parallelism (e.g. GranSim keeps the object's global address here, +though GUM keeps a separate hash table). +\item Statistics (e.g. a word to track how many times a thunk is entered.). + +We add a Ticky word to the fixed-header part of closures. This is +used to indicate if a closure has been updated but not yet entered. It +is set when the closure is updated and cleared when subsequently +entered. \footnote{% NB: It is \emph{not} an ``entry count'', it is +an ``entries-after-update count.'' The commoning up of @CONST@, +@CHARLIKE@ and @INTLIKE@ closures is turned off(?) if this is +required. This has only been done for 2s collection. } + +\end{itemize} +\end{itemize} + +Most of the RTS is completely insensitive to the number of admin +words. The total size of the fixed header is given by +@sizeof(StgHeader)@. + +\Subsection{Info Tables}{info-tables} + +An \emph{info table} is a contiguous block of memory, laid out as follows: + +\begin{center} +\begin{tabular}{|r|l|} + \hline Parallelism Info & variable +\\ \hline Profile Info & variable +\\ \hline Debug Info & variable +\\ \hline Static reference table & pointer word (optional) +\\ \hline Storage manager layout info & pointer word +\\ \hline Closure flags & 8 bits +\\ \hline Closure type & 8 bits +\\ \hline Constructor Tag / SRT length & 16 bits +\\ \hline entry code +\\ \vdots +\end{tabular} +\end{center} + +On a 64-bit machine the tag, type and flags fields will all be doubled +in size, so the info table is a multiple of 64 bits. + +An info table has the following contents (working backwards in memory +addresses): + +\begin{itemize} + +\item The \emph{entry code} for the closure. This code appears +literally as the (large) last entry in the info table, immediately +preceded by the rest of the info table. An \emph{info pointer} always +points to the first byte of the entry code. + +\item A 16-bit constructor tag / SRT length. For a constructor info +table this field contains the tag of the constructor, in the range +$0..n-1$ where $n$ is the number of constructors in the datatype. +Otherwise, it contains the number of entries in this closure's Static +Reference Table (\secref{srt}). + +\item An 8-bit {\em closure type field}, which identifies what kind of +closure the object is. The various types of closure are described in +\secref{closures}. + +\item an 8-bit flags field, which holds various flags pertaining to +the closure type. + +\item A single pointer or word --- the {\em storage manager info +field}, contains auxiliary information describing the closure's +precise layout, for the benefit of the garbage collector and the code +that stuffs graph into packets for transmission over the network. +There are three kinds of layout information: + +\begin{itemize} +\item Standard layout information is for closures which place pointers +before non-pointers in instances of the closure (this applies to most +heap-based and static closures, but not activation records). The +layout information for standard closures is + + \begin{itemize} + \item Number of pointer fields (16 bits). + \item Number of non-pointer fields (16 bits). + \end{itemize} + +\item Activation records don't have pointers before non-pointers, +since stack-stubbing requires that the record has holes in it. The +layout is therefore represented by a bitmap in which each '1' bit +represents a non-pointer word. This kind of layout info is used for +@RET_SMALL@ and @RET_VEC_SMALL@ closures. + +\item If an activation record is longer than 32 words, then the layout +field contains a pointer to a bitmap record, consisting of a length +field followed by two or more bitmap words. This layout information +is used for @RET_BIG@ and @RET_VEC_BIG@ closures. + +\item Selector Thunks (\secref{THUNK_SELECTOR}) use the closure +layout field to hold the selector index, since the layout is always +known (the closure contains a single pointer field). +\end{itemize} + +\item A one-word {\em Static Reference Table} field. This field +points to the static reference table for the closure (\secref{srt}), +and is only present for the following closure types: + + \begin{itemize} + \item @FUN_*@ + \item @THUNK_*@ + \item @RET_*@ + \end{itemize} + +\ToDo{Expand the following explanation.} + +An SRT is basically a vector of pointers to static closures. A +top-level function or thunk will have an SRT (which might be empty), +which points to all the static closures referenced by that function or +thunk. Every non-top-level thunk or function also has an SRT, but +it'll be a sub-sequence of the top-level SRT, so we just store a +pointer and a length in the info table - the pointer points into the +middle of the larger SRT. + +At GC time, the garbage collector traverses the transitive closure of +all the SRTs reachable from the roots, and thereby discovers which +CAFs are live. + +\item \emph{Profiling info\/} + +\ToDo{The profiling info is completely bogus. I've not deleted it +from the document but I've commented it all out.} + +% change to \iftrue to uncomment this section +\iffalse + +Closure category records are attached to the info table of the +closure. They are declared with the info table. We put pointers to +these ClCat things in info tables. We need these ClCat things because +they are mutable, whereas info tables are immutable. Hashing will map +similar categories to the same hash value allowing statistics to be +grouped by closure category. + +Cost Centres and Closure Categories are hashed to provide indexes +against which arbitrary information can be stored. These indexes are +memoised in the appropriate cost centre or category record and +subsequent hashes avoided by the index routine (it simply returns the +memoised index). + +There are different features which can be hashed allowing information +to be stored for different groupings. Cost centres have the cost +centre recorded (using the pointer), module and group. Closure +categories have the closure description and the type +description. Records with the same feature will be hashed to the same +index value. + +The initialisation routines, @init_index_<feature>@, allocate a hash +table in which the cost centre / category records are stored. The +lower bound for the table size is taken from @max_<feature>_no@. They +return the actual table size used (the next power of 2). Unused +locations in the hash table are indicated by a 0 entry. Successive +@init_index_<feature>@ calls just return the actual table size. + +Calls to @index_<feature>@ will insert the cost centre / category +record in the @<feature>@ hash table, if not already inserted. The hash +index is memoised in the record and returned. + +CURRENTLY ONLY ONE MEMOISATION SLOT IS AVILABLE IN EACH RECORD SO +HASHING CAN ONLY BE DONE ON ONE FEATURE FOR EACH RECORD. This can be +easily relaxed at the expense of extra memoisation space or continued +rehashing. + +The initialisation routines must be called before initialisation of +the stacks and heap as they require to allocate storage. It is also +expected that the caller may want to allocate additional storage in +which to store profiling information based on the return table size +value(s). + +\begin{center} +\begin{tabular}{|l|} + \hline Hash Index +\\ \hline Selected +\\ \hline Kind +\\ \hline Description String +\\ \hline Type String +\\ \hline +\end{tabular} +\end{center} + +\begin{description} +\item[Hash Index] Memoised copy +\item[Selected] + Is this category selected (-1 == not memoised, selected? 0 or 1) +\item[Kind] +One of the following values (defined in CostCentre.lh): + +\begin{description} +\item[@CON_K@] +A constructor. +\item[@FN_K@] +A literal function. +\item[@PAP_K@] +A partial application. +\item[@THK_K@] +A thunk, or suspension. +\item[@BH_K@] +A black hole. +\item[@ARR_K@] +An array. +\item[@ForeignObj_K@] +A Foreign object (non-Haskell heap resident). +\item[@SPT_K@] +The Stable Pointer table. (There should only be one of these but it +represents a form of weak space leak since it can't shrink to meet +non-demand so it may be worth watching separately? ADR) +\item[@INTERNAL_KIND@] +Something internal to the runtime system. +\end{description} + + +\item[Description] Source derived string detailing closure description. +\item[Type] Source derived string detailing closure type. +\end{description} + +\fi % end of commented out stuff + +\item \emph{Parallelism info\/} +\ToDo{} + +\item \emph{Debugging info\/} +\ToDo{} + +\end{itemize} + + +%----------------------------------------------------------------------------- +\Subsection{Kinds of Heap Object}{closures} + +Heap objects can be classified in several ways, but one useful one is +this: +\begin{itemize} +\item +\emph{Static closures} occupy fixed, statically-allocated memory +locations, with globally known addresses. + +\item +\emph{Dynamic closures} are individually allocated in the heap. + +\item +\emph{Stack closures} are closures allocated within a thread's stack +(which is itself a heap object). Unlike other closures, there are +never any pointers to stack closures. Stack closures are discussed in +\secref{TSO}. + +\end{itemize} +A second useful classification is this: +\begin{itemize} + +\item \emph{Executive objects}, such as thunks and data constructors, +participate directly in a program's execution. They can be subdivided +into three kinds of objects according to their type: \begin{itemize} + +\item \emph{Pointed objects}, represent values of a \emph{pointed} +type (<.pointed types launchbury.>) --i.e.~a type that includes +$\bottom$ such as @Int@ or @Int# -> Int#@. + +\item \emph{Unpointed objects}, represent values of a \emph{unpointed} +type --i.e.~a type that does not include $\bottom$ such as @Int#@ or +@Array#@. + +\item \emph{Activation frames}, represent ``continuations''. They are +always stored on the stack and are never pointed to by heap objects or +passed as arguments. \note{It's not clear if this will still be true +once we support speculative evaluation.} + +\end{itemize} + +\item \emph{Administrative objects}, such as stack objects and thread +state objects, do not represent values in the original program. +\end{itemize} + +Only pointed objects can be entered. If an unpointed object is +entered the program will usually terminate with a fatal error. + +This section enumerates all the kinds of heap objects in the system. +Each is identified by a distinct closure type field in its info table. + +\begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|} +\hline + +closure type & Section \\ + +\hline +\emph{Pointed} \\ +\hline + +@CONSTR@ & \ref{sec:CONSTR} \\ +@CONSTR_p_n@ & \ref{sec:CONSTR} \\ +@CONSTR_STATIC@ & \ref{sec:CONSTR} \\ +@CONSTR_NOCAF_STATIC@ & \ref{sec:CONSTR} \\ + +@FUN@ & \ref{sec:FUN} \\ +@FUN_p_n@ & \ref{sec:FUN} \\ +@FUN_STATIC@ & \ref{sec:FUN} \\ + +@THUNK@ & \ref{sec:THUNK} \\ +@THUNK_p_n@ & \ref{sec:THUNK} \\ +@THUNK_STATIC@ & \ref{sec:THUNK} \\ +@THUNK_SELECTOR@ & \ref{sec:THUNK_SELECTOR} \\ + +@BCO@ & \ref{sec:BCO} \\ + +@AP_UPD@ & \ref{sec:AP_UPD} \\ +@PAP@ & \ref{sec:PAP} \\ + +@IND@ & \ref{sec:IND} \\ +@IND_OLDGEN@ & \ref{sec:IND} \\ +@IND_PERM@ & \ref{sec:IND} \\ +@IND_OLDGEN_PERM@ & \ref{sec:IND} \\ +@IND_STATIC@ & \ref{sec:IND} \\ + +@CAF_UNENTERED@ & \ref{sec:CAF} \\ +@CAF_ENTERED@ & \ref{sec:CAF} \\ +@CAF_BLACKHOLE@ & \ref{sec:CAF} \\ + +\hline +\emph{Unpointed} \\ +\hline + +@BLACKHOLE@ & \ref{sec:BLACKHOLE} \\ +@BLACKHOLE_BQ@ & \ref{sec:BLACKHOLE_BQ} \\ + +@MVAR@ & \ref{sec:MVAR} \\ + +@ARR_WORDS@ & \ref{sec:ARR_WORDS} \\ + +@MUTARR_PTRS@ & \ref{sec:MUT_ARR_PTRS} \\ +@MUTARR_PTRS_FROZEN@ & \ref{sec:MUT_ARR_PTRS_FROZEN} \\ + +@MUT_VAR@ & \ref{sec:MUT_VAR} \\ + +@WEAK@ & \ref{sec:WEAK} \\ +@FOREIGN@ & \ref{sec:FOREIGN} \\ +@STABLE_NAME@ & \ref{sec:STABLE_NAME} \\ +\hline +\end{tabular} + +Activation frames do not live (directly) on the heap --- but they have +a similar organisation. + +\begin{tabular}{|l|l|}\hline +closure type & Section \\ \hline +@RET_SMALL@ & \ref{sec:activation-records} \\ +@RET_VEC_SMALL@ & \ref{sec:activation-records} \\ +@RET_BIG@ & \ref{sec:activation-records} \\ +@RET_VEC_BIG@ & \ref{sec:activation-records} \\ +@UPDATE_FRAME@ & \ref{sec:activation-records} \\ +@CATCH_FRAME@ & \ref{sec:activation-records} \\ +@SEQ_FRAME@ & \ref{sec:activation-records} \\ +@STOP_FRAME@ & \ref{sec:activation-records} \\ +\hline +\end{tabular} + +There are also a number of administrative objects. It is an error to +enter one of these objects. + +\begin{tabular}{|l|l|}\hline +closure type & Section \\ \hline +@TSO@ & \ref{sec:TSO} \\ +@SPARK_OBJECT@ & \ref{sec:SPARK} \\ +@BLOCKED_FETCH@ & \ref{sec:BLOCKED_FETCH} \\ +@FETCHME@ & \ref{sec:FETCHME} \\ +\hline +\end{tabular} + +\Subsection{Predicates}{closure-predicates} + +The runtime system sometimes needs to be able to distinguish objects +according to their properties: is the object updateable? is it in weak +head normal form? etc. These questions can be answered by examining +the closure type field of the object's info table. + +We define the following predicates to detect families of related +info types. They are mutually exclusive and exhaustive. + +\begin{itemize} +\item @isCONSTR@ is true for @CONSTR@s. +\item @isFUN@ is true for @FUN@s. +\item @isTHUNK@ is true for @THUNK@s. +\item @isBCO@ is true for @BCO@s. +\item @isAP@ is true for @AP@s. +\item @isPAP@ is true for @PAP@s. +\item @isINDIRECTION@ is true for indirection objects. +\item @isBH@ is true for black holes. +\item @isFOREIGN_OBJECT@ is true for foreign objects. +\item @isARRAY@ is true for array objects. +\item @isMVAR@ is true for @MVAR@s. +\item @isIVAR@ is true for @IVAR@s. +\item @isFETCHME@ is true for @FETCHME@s. +\item @isSLOP@ is true for slop objects. +\item @isRET_ADDR@ is true for return addresses. +\item @isUPD_ADDR@ is true for update frames. +\item @isTSO@ is true for @TSO@s. +\item @isSTABLE_PTR_TABLE@ is true for the stable pointer table. +\item @isSPARK_OBJECT@ is true for spark objects. +\item @isBLOCKED_FETCH@ is true for blocked fetch objects. +\item @isINVALID_INFOTYPE@ is true for all other info types. + +\end{itemize} + +The following predicates detect other interesting properties: + +\begin{itemize} + +\item @isPOINTED@ is true if an object has a pointed type. + +If an object is pointed, the following predicates may be true +(otherwise they are false). @isWHNF@ and @isUPDATEABLE@ are +mutually exclusive. + +\begin{itemize} +\item @isWHNF@ is true if the object is in Weak Head Normal Form. +Note that unpointed objects are (arbitrarily) not considered to be in WHNF. + +@isWHNF@ is true for @PAP@s, @CONSTR@s, @FUN@s and all @BCO@s. + +\ToDo{Need to distinguish between whnf BCOs and non-whnf BCOs in their +closure type} + +\item @isUPDATEABLE@ is true if the object may be overwritten with an + indirection object. + +@isUPDATEABLE@ is true for @THUNK@s, @AP@s and @BH@s. + +\end{itemize} + +It is possible for a pointed object to be neither updatable nor in +WHNF. For example, indirections. + +\item @isUNPOINTED@ is true if an object has an unpointed type. +All such objects are boxed since only boxed objects have info pointers. + +It is true for @ARR_WORDS@, @ARR_PTRS@, @MUTVAR@, @MUTARR_PTRS@, +@MUTARR_PTRS_FROZEN@, @FOREIGN@ objects, @MVAR@s and @IVAR@s. + +\item @isACTIVATION_FRAME@ is true for activation frames of all sorts. + +It is true for return addresses and update frames. +\begin{itemize} +\item @isVECTORED_RETADDR@ is true for vectored return addresses. +\item @isDIRECT_RETADDR@ is true for direct return addresses. +\end{itemize} + +\item @isADMINISTRATIVE@ is true for administrative objects: +@TSO@s, the stable pointer table, spark objects and blocked fetches. + +\item @hasSRT@ is true if the info table for the object contains an +SRT pointer. + +@hasSRT@ is true for @THUNK@s, @FUN@s, and @RET@s. + +\end{itemize} + +\begin{itemize} + +\item @isSTATIC@ is true for any statically allocated closure. + +\item @isMUTABLE@ is true for objects with mutable pointer fields: + @MUT_ARR@s, @MUTVAR@s, @MVAR@s and @IVAR@s. + +\item @isSparkable@ is true if the object can (and should) be sparked. +It is true of updateable objects which are not in WHNF with the +exception of @THUNK_SELECTOR@s and black holes. + +\end{itemize} + +As a minor optimisation, we might use the top bits of the @INFO_TYPE@ +field to ``cache'' the answers to some of these predicates. + +An indirection either points to HNF (post update); or is result of +overwriting a FetchMe, in which case the thing fetched is either under +evaluation (BLACKHOLE), or by now an HNF. Thus, indirections get +NoSpark flag. + +\subsection{Closures (aka Pointed Objects)} + +An object can be entered iff it is a closure. + +\Subsubsection{Function closures}{FUN} + +Function closures represent lambda abstractions. For example, +consider the top-level declaration: +\begin{verbatim} + f = \x -> let g = \y -> x+y + in g x +\end{verbatim} +Both @f@ and @g@ are represented by function closures. The closure +for @f@ is \emph{static} while that for @g@ is \emph{dynamic}. + +The layout of a function closure is as follows: +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Pointers} & \emph{Non-pointers} \\ \hline +\end{tabular} +\end{center} + +The data words (pointers and non-pointers) are the free variables of +the function closure. The number of pointers and number of +non-pointers are stored in @info->layout.ptrs@ and +@info->layout.nptrs@ respecively. + +There are several different sorts of function closure, distinguished +by their closure type field: + +\begin{itemize} + +\item @FUN@: a vanilla, dynamically allocated on the heap. + +\item $@FUN_@p@_@np$: to speed up garbage collection a number of +specialised forms of @FUN@ are provided, for particular $(p,np)$ +pairs, where $p$ is the number of pointers and $np$ the number of +non-pointers. + +\item @FUN_STATIC@. Top-level, static, function closures (such as @f@ +above) have a different layout than dynamic ones: + +\begin{center} +\begin{tabular}{|l|l|l|}\hline +\emph{Fixed header} & \emph{Static object link} \\ \hline +\end{tabular} +\end{center} + +Static function closures have no free variables. (However they may +refer to other static closures; these references are recorded in the +function closure's SRT.) They have one field that is not present in +dynamic closures, the \emph{static object link} field. This is used +by the garbage collector in the same way that to-space is, to gather +closures that have been determined to be live but that have not yet +been scavenged. + +\note{Static function closures that have no static references, and +hence a null SRT pointer, don't need the static object link field. We +don't take advantage of this at the moment, but we could. See +@CONSTR\_NOCAF\_STATIC@.} +\end{itemize} + +Each lambda abstraction, $f$, in the STG program has its own private +info table. The following labels are relevant: + +\begin{itemize} + +\item $f$@_info@ is $f$'s info table. + +\item $f$@_entry@ is $f$'s slow entry point (i.e. the entry code of +its info table; so it will label the same byte as $f$@_info@). + +\item $f@_fast_@k$ is $f$'s fast entry point. $k$ is the number of +arguments $f$ takes; encoding this number in the fast-entry label +occasionally catches some nasty code-generation errors. + +\end{itemize} + +\Subsubsection{Data constructors}{CONSTR} + +Data-constructor closures represent values constructed with algebraic +data type constructors. The general layout of data constructors is +the same as that for function closures. That is + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Pointers} & \emph{Non-pointers} \\ \hline +\end{tabular} +\end{center} + +There are several different sorts of constructor: + +\begin{itemize} + +\item @CONSTR@: a vanilla, dynamically allocated constructor. + +\item @CONSTR_@$p$@_@$np$: just like $@FUN_@p@_@np$. + +\item @CONSTR_INTLIKE@. A dynamically-allocated heap object that +looks just like an @Int@. The garbage collector checks to see if it +can common it up with one of a fixed set of static int-like closures, +thus getting it out of the dynamic heap altogether. + +\item @CONSTR_CHARLIKE@: same deal, but for @Char@. + +\item @CONSTR_STATIC@ is similar to @FUN_STATIC@, with the +complication that the layout of the constructor must mimic that of a +dynamic constructor, because a static constructor might be returned to +some code that unpacks it. So its layout is like this: + +\begin{center} +\begin{tabular}{|l|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Pointers} & \emph{Non-pointers} & \emph{Static object link}\\ \hline +\end{tabular} +\end{center} + +The static object link, at the end of the closure, serves the same purpose +as that for @FUN_STATIC@. The pointers in the static constructor can point +only to other static closures. + +The static object link occurs last in the closure so that static +constructors can store their data fields in exactly the same place as +dynamic constructors. + +\item @CONSTR_NOCAF_STATIC@. A statically allocated data constructor +that guarantees not to point (directly or indirectly) to any CAF +(\secref{CAF}). This means it does not need a static object +link field. Since we expect that there might be quite a lot of static +constructors this optimisation makes sense. Furthermore, the @NOCAF@ +tag allows the compiler to indicate that no CAFs can be reached +anywhere \emph{even indirectly}. + +\end{itemize} + +For each data constructor $Con$, two info tables are generated: + +\begin{itemize} +\item $Con$@_con_info@ labels $Con$'s dynamic info table, +shared by all dynamic instances of the constructor. +\item $Con$@_static@ labels $Con$'s static info table, +shared by all static instances of the constructor. +\end{itemize} + +Each constructor also has a \emph{constructor function}, which is a +curried function which builds an instance of the constructor. The +constructor function has an info table labelled as @$Con$_info@, and +entry code pointed to by @$Con$_entry@. + +Nullary constructors are represented by a single static info table, +which everyone points to. Thus for a nullary constructor we can omit +the dynamic info table and the constructor function. + +\subsubsection{Thunks} +\label{sec:THUNK} +\label{sec:THUNK_SELECTOR} + +A thunk represents an expression that is not obviously in head normal +form. For example, consider the following top-level definitions: +\begin{verbatim} + range = between 1 10 + f = \x -> let ys = take x range + in sum ys +\end{verbatim} +Here the right-hand sides of @range@ and @ys@ are both thunks; the former +is static while the latter is dynamic. + +The layout of a thunk is the same as that for a function closure. +However, thunks must have a payload of at least @MIN_UPD_SIZE@ +words to allow it to be overwritten with a black hole and an +indirection. The compiler may have to add extra non-pointer fields to +satisfy this constraint. + +\begin{center} +\begin{tabular}{|l|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Pointers} & \emph{Non-pointers} \\ \hline +\end{tabular} +\end{center} + +The layout word in the info table contains the same information as for +function closures; that is, number of pointers and number of +non-pointers. + +A thunk differs from a function closure in that it can be updated. + +There are several forms of thunk: + +\begin{itemize} + +\item @THUNK@ and $@THUNK_@p@_@np$: vanilla, dynamically allocated +thunks. Dynamic thunks are overwritten with normal indirections +(@IND@), or old generation indirections (@IND_OLDGEN@): see +\secref{IND}. + +\item @THUNK_STATIC@. A static thunk is also known as a +\emph{constant applicative form}, or \emph{CAF}. Static thunks are +overwritten with static indirections. + +\begin{center} +\begin{tabular}{|l|l|}\hline +\emph{Fixed header} & \emph{Static object link}\\ \hline +\end{tabular} +\end{center} + +\item @THUNK_SELECTOR@ is a (dynamically allocated) thunk whose entry +code performs a simple selection operation from a data constructor +drawn from a single-constructor type. For example, the thunk +\begin{verbatim} + x = case y of (a,b) -> a +\end{verbatim} +is a selector thunk. A selector thunk is laid out like this: + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Selectee pointer} \\ \hline +\end{tabular} +\end{center} + +The layout word contains the byte offset of the desired word in the +selectee. Note that this is different from all other thunks. + +The garbage collector ``peeks'' at the selectee's tag (in its info +table). If it is evaluated, then it goes ahead and does the +selection, and then behaves just as if the selector thunk was an +indirection to the selected field. If it is not evaluated, it treats +the selector thunk like any other thunk of that shape. +[Implementation notes. Copying: only the evacuate routine needs to be +special. Compacting: only the PRStart (marking) routine needs to be +special.] + +There is a fixed set of pre-compiled selector thunks built into the +RTS, representing offsets from 0 to @MAX_SPEC_SELECTOR_THUNK@. The +info tables are labelled @__sel_$n$_upd_info@ where $n$ is the offset. +Non-updating versions are also built in, with info tables labelled +@__sel_$n$_noupd_info@. + +\end{itemize} + +The only label associated with a thunk is its info table: + +\begin{description} +\item[$f$@\_info@] is $f$'s info table. +\end{description} + + +\Subsubsection{Byte-code objects}{BCO} + +A Byte-Code Object (BCO) is a container for a a chunk of byte-code, +which can be executed by Hugs. The byte-code represents a +supercombinator in the program: when Hugs compiles a module, it +performs lambda lifting and each resulting supercombinator becomes a +byte-code object in the heap. + +BCOs are not updateable; the bytecode compiler represents updatable +thunks using a combination of @AP@s and @BCO@s. + +The semantics of BCOs are described in \secref{hugs-heap-objects}. A +BCO has the following structure: + +\begin{center} +\begin{tabular}{|l|l|l|l|l|l|} +\hline +\emph{Fixed Header} & \emph{Layout} & \emph{Offset} & \emph{Size} & +\emph{Literals} & \emph{Byte code} \\ +\hline +\end{tabular} +\end{center} + +\noindent where: +\begin{itemize} +\item The entry code is a static code fragment/info table that returns +to the scheduler to invoke Hugs (\secref{ghc-to-hugs-switch}). +\item \emph{Layout} contains the number of pointer literals in the +\emph{Literals} field. +\item \emph{Offset} is the offset to the byte code from the start of +the object. +\item \emph{Size} is the number of words of byte code in the object. +\item \emph{Literals} contains any pointer and non-pointer literals used in +the byte-codes (including jump addresses), pointers first. +\item \emph{Byte code} contains \emph{Size} words of non-pointer byte +code. +\end{itemize} + + +\Subsubsection{Partial applications}{PAP} + +A partial application (PAP) represents a function applied to too few +arguments. It is only built as a result of updating after an +argument-satisfaction check failure. A PAP has the following shape: + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\emph{Fixed header} & \emph{No of words of stack} & \emph{Function closure} & \emph{Stack chunk ...} \\ \hline +\end{tabular} +\end{center} + +The ``Stack chunk'' is a copy of the chunk of stack above the update +frame; ``No of words of stack'' tells how many words it consists of. +The function closure is (a pointer to) the closure for the function +whose argument-satisfaction check failed. + +In the normal case where a PAP is built as a result of an argument +satisfaction check failure, the stack chunk will just contain +``pending arguments'', ie. pointers and tagged non-pointers. It may +in fact also contain activation records, but not update frames, seq +frames, or catch frames. The reason is the garbage collector uses the +same code to scavenge a stack as it does to scavenge the payload of a +PAP, but an update frame contains a link to the next update frame in +the chain and this link would need to be relocated during garbage +collection. Revertible black holes and asynchronous exceptions use +the more general form of PAPs (see Section \ref{revertible-bh}). + +There is just one standard form of PAP. There is just one info table +too, called @PAP_info@. Its entry code simply copies the arg stack +chunk back on top of the stack and enters the function closure. (It +has to do a stack overflow test first.) + +There is just one way to build a PAP: by calling @stg_update_PAP@ with +the function closure in register @R1@ and the pending arguments on the +stack. The @stg_update_PAP@ function will build the PAP, perform the +update, and return to the next activation record on the stack. If +there are \emph{no} pending arguments on the stack, then no PAP need +be built: in this case @stg_update_PAP@ just overwrites the updatee +with an indirection to the function closure. + +PAPs are also used to implement Hugs functions (where the arguments +are free variables). PAPs generated by Hugs can be static so we need +both @PAP@ and @PAP_STATIC@. + +\Subsubsection{\texttt{AP\_UPD} objects}{AP_UPD} + +@AP_UPD@ objects are used to represent thunks built by Hugs, and to +save the currently-active computations when performing @raiseAsync()@. +The only +distinction between an @AP_UPD@ and a @PAP@ is that an @AP_UPD@ is +updateable. + +\begin{center} +\begin{tabular}{|l|l|l|l|} +\hline +\emph{Fixed Header} & \emph{No of stack words} & \emph{Function closure} & \emph{Stack chunk} \\ +\hline +\end{tabular} +\end{center} + +The entry code pushes an update frame, copies the arg stack chunk on +top of the stack, and enters the function closure. (It has to do a +stack overflow test first.) + +The ``stack chunk'' is a block of stack not containing update frames, +seq frames or catch frames (just like a PAP). In the case of Hugs, +the stack chunk will contain the free variables of the thunk, and the +function closure is (a pointer to) the closure for the thunk. The +argument stack may be empty if the thunk has no free variables. + +\note{Since @AP\_UPD@s are updateable, the @MIN\_UPD\_SIZE@ constraint applies here too.} + +\Subsubsection{Indirections}{IND} + +Indirection closures just point to other closures. They are introduced +when a thunk is updated to point to its value. The entry code for all +indirections simply enters the closure it points to. + +There are several forms of indirection: + +\begin{description} +\item[@IND@] is the vanilla, dynamically-allocated indirection. +It is removed by the garbage collector. It has the following +shape: +\begin{center} +\begin{tabular}{|l|l|l|}\hline +\emph{Fixed header} & \emph{Target closure} \\ \hline +\end{tabular} +\end{center} + +An @IND@ only exists in the youngest generation. In older +generations, we have @IND_OLDGEN@s. The update code +(@Upd_frame_$n$_entry@) checks whether the updatee is in the youngest +generation before deciding which kind of indirection to use. + +\item[@IND\_OLDGEN@] is the vanilla, dynamically-allocated indirection. +It is removed by the garbage collector. It has the following +shape: +\begin{center} +\begin{tabular}{|l|l|l|}\hline +\emph{Fixed header} & \emph{Target closure} & \emph{Mutable link field} \\ \hline +\end{tabular} +\end{center} +It contains a \emph{mutable link field} that is used to string together +mutable objects in each old generation. + +\item[@IND\_PERM@] +For lexical profiling, it is necessary to maintain cost centre +information in an indirection, so ``permanent indirections'' are +retained forever. Otherwise they are just like vanilla indirections. +\note{If a permanent indirection points to another permanent +indirection or a @CONST@ closure, it is possible to elide the indirection +since it will have no effect on the profiler.} + +\note{Do we still need @IND@ in the profiling build, or do we just +need @IND@ but its behaviour changes when profiling is on?} + +\item[@IND\_OLDGEN\_PERM@] +Just like an @IND_OLDGEN@, but sticks around like an @IND_PERM@. + +\item[@IND\_STATIC@] is used for overwriting CAFs when they have been +evaluated. Static indirections are not removed by the garbage +collector; and are statically allocated outside the heap (and should +stay there). Their static object link field is used just as for +@FUN_STATIC@ closures. + +\begin{center} +\begin{tabular}{|l|l|l|} +\hline +\emph{Fixed header} & \emph{Target closure} & \emph{Static link field} \\ +\hline +\end{tabular} +\end{center} + +\end{description} + +\subsubsection{Black holes and blocking queues} +\label{sec:BLACKHOLE} +\label{sec:BLACKHOLE_BQ} + +Black hole closures are used to overwrite closures currently being +evaluated. They inform the garbage collector that there are no live +roots in the closure, thus removing a potential space leak. + +Black holes also become synchronization points in the concurrent +world. When a thread attempts to enter a blackhole, it must wait for +the result of the computation, which is presumably in progress in +another thread. + +\note{In a single-threaded system, entering a black hole indicates an +infinite loop. In a concurrent system, entering a black hole +indicates an infinite loop only if the hole is being entered by the +same thread that originally entered the closure. It could also bring +about a deadlock situation where several threads are waiting +circularly on computations in progress.} + +There are two types of black hole: + +\begin{description} + +\item[@BLACKHOLE@] +A straightforward blackhole just consists of an info pointer and some +padding to allow updating with an @IND_OLDGEN@ if necessary. This +type of blackhole has no waiting threads. + +\begin{center} +\begin{tabular}{|l|l|l|} +\hline +\emph{Fixed header} & \emph{Padding} & \emph{Padding} \\ +\hline +\end{tabular} +\end{center} + +If we're doing \emph{eager blackholing} then a thunk's info pointer is +overwritten with @BLACKHOLE_info@ at the time of entry; hence the need +for blackholes to be small, otherwise we'd be overwriting part of the +thunk itself. + +\item[@BLACKHOLE\_BQ@] +When a thread enters a @BLACKHOLE@, it is turned into a @BLACKHOLE_BQ@ +(blocking queue), which contains a linked list of blocked threads in +addition to the info pointer. + +\begin{center} +\begin{tabular}{|l|l|l|} +\hline +\emph{Fixed header} & \emph{Blocked thread link} & \emph{Mutable link field} \\ +\hline +\end{tabular} +\end{center} + +The \emph{Blocked thread link} points to the TSO of the first thread +waiting for the value of this thunk. All subsequent TSOs in the list +are linked together using their @tso->link@ field, ending in +@END_TSO_QUEUE_closure@. + +Because new threads can be added to the \emph{Blocked thread link}, a +blocking queue is \emph{mutable}, so we need a mutable link field in +order to chain it on to a mutable list for the generational garbage +collector. + +\end{description} + +\Subsubsection{FetchMes}{FETCHME} + +In the parallel systems, FetchMes are used to represent pointers into +the global heap. When evaluated, the value they point to is read from +the global heap. + +\ToDo{Describe layout} + +Because there may be offsets into these arrays, a primitive array +cannot be handled as a FetchMe in the parallel system, but must be +shipped in its entirety if its parent closure is shipped. + + + +\Subsection{Unpointed Objects}{unpointed-objects} + +A variable of unpointed type is always bound to a \emph{value}, never +to a \emph{thunk}. For this reason, unpointed objects cannot be +entered. + +\subsubsection{Immutable objects} +\label{sec:ARR_WORDS} + +\begin{description} +\item[@ARR\_WORDS@] is a variable-sized object consisting solely of +non-pointers. It is used for arrays of all sorts of things (bytes, +words, floats, doubles... it doesn't matter). + +Strictly speaking, an @ARR_WORDS@ could be mutable, but because it +only contains non-pointers we don't need to track this fact. + +\begin{center} +\begin{tabular}{|c|c|c|c|} +\hline +\emph{Fixed Hdr} & \emph{No of non-pointers} & \emph{Non-pointers\ldots} \\ \hline +\end{tabular} +\end{center} +\end{description} + +\subsubsection{Mutable objects} +\label{sec:mutables} +\label{sec:MUT_VAR} +\label{sec:MUT_ARR_PTRS} +\label{sec:MUT_ARR_PTRS_FROZEN} +\label{sec:MVAR} + +Some of these objects are \emph{mutable}; they represent objects which +are explicitly mutated by Haskell code through the @ST@ or @IO@ +monads. They're not used for thunks which are updated precisely once. +Depending on the garbage collector, mutable closures may contain extra +header information which allows a generational collector to implement +the ``write barrier.'' + +Notice that mutable objects all have the same general layout: there is +a mutable link field as the second word after the header. This is so +that code to process old-generation mutable lists doesn't need to look +at the type of the object to determine where its link field is. + +\begin{description} + +\item[@MUT\_VAR@] is a mutable variable. +\begin{center} +\begin{tabular}{|c|c|c|} +\hline +\emph{Fixed Hdr} \emph{Pointer} & \emph{Mutable link} & \\ \hline +\end{tabular} +\end{center} + +\item[@MUT\_ARR\_PTRS@] is a mutable array of pointers. Such an array +may be \emph{frozen}, becoming an @MUT_ARR_PTRS_FROZEN@, with a +different info-table. + +\begin{center} +\begin{tabular}{|c|c|c|c|} +\hline +\emph{Fixed Hdr} & \emph{No of ptrs} & \emph{Mutable link} & \emph{Pointers\ldots} \\ \hline +\end{tabular} +\end{center} + +\item[@MUT\_ARR\_PTRS\_FROZEN@] This is the immutable version of +@MUT_ARR_PTRS@. It still has a mutable link field for two reasons: we +need to keep it on the mutable list for an old generation at least +until the next garbage collection, and it may become mutable again via +@thawArray@. + +\begin{center} +\begin{tabular}{|c|c|c|c|} +\hline +\emph{Fixed Hdr} & \emph{No of ptrs} & \emph{Mutable link} & \emph{Pointers\ldots} \\ \hline +\end{tabular} +\end{center} + +\item[@MVAR@] + +\begin{center} +\begin{tabular}{|l|l|l|l|l|} +\hline +\emph{Fixed header} & \emph{Head} & \emph{Mutable link} & \emph{Tail} +& \emph{Value}\\ +\hline +\end{tabular} +\end{center} + +\ToDo{MVars} + +\end{description} + + +\Subsubsection{Foreign objects}{FOREIGN} + +Here's what a ForeignObj looks like: + +\begin{center} +\begin{tabular}{|l|l|l|l|} +\hline +\emph{Fixed header} & \emph{Data} \\ +\hline +\end{tabular} +\end{center} + +A foreign object is simple a boxed pointer to an address outside the +Haskell heap, possible to @malloc@ed data. The only reason foreign +objects exist is so that we can track the lifetime of one using weak +pointers (see \secref{WEAK}) and run a finaliser when the foreign +object is unreachable. + +\subsubsection{Weak pointers} +\label{sec:WEAK} + +\begin{center} +\begin{tabular}{|l|l|l|l|l|} +\hline +\emph{Fixed header} & \emph{Key} & \emph{Value} & \emph{Finaliser} +& \emph{Link}\\ +\hline +\end{tabular} +\end{center} + +\ToDo{Weak poitners} + +\subsubsection{Stable names} +\label{sec:STABLE_NAME} + +\begin{center} +\begin{tabular}{|l|l|l|l|} +\hline +\emph{Fixed header} & \emph{Index} \\ +\hline +\end{tabular} +\end{center} + +\ToDo{Stable names} + +The remaining objects types are all administrative --- none of them +may be entered. + +\subsection{Other weird objects} +\label{sec:SPARK} +\label{sec:BLOCKED_FETCH} + +\begin{description} +\item[@BlockedFetch@ heap objects (`closures')] (parallel only) + +@BlockedFetch@s are inbound fetch messages blocked on local closures. +They arise as entries in a local blocking queue when a fetch has been +received for a local black hole. When awakened, we look at their +contents to figure out where to send a resume. + +A @BlockedFetch@ closure has the form: +\begin{center} +\begin{tabular}{|l|l|l|l|l|l|}\hline +\emph{Fixed header} & link & node & gtid & slot & weight \\ \hline +\end{tabular} +\end{center} + +\item[Spark Closures] (parallel only) + +Spark closures are used to link together all closures in the spark pool. When +the current processor is idle, it may choose to speculatively evaluate some of +the closures in the pool. It may also choose to delete sparks from the pool. +\begin{center} +\begin{tabular}{|l|l|l|l|l|l|}\hline +\emph{Fixed header} & \emph{Spark pool link} & \emph{Sparked closure} \\ \hline +\end{tabular} +\end{center} + +\item[Slop Objects]\label{sec:slop-objects} + +Slop objects are used to overwrite the end of an updatee if it is +larger than an indirection. Normal slop objects consist of an info +pointer a size word and a number of slop words. + +\begin{center} +\begin{tabular}{|l|l|l|l|l|l|}\hline +\emph{Info Pointer} & \emph{Size} & \emph{Slop Words} \\ \hline +\end{tabular} +\end{center} + +This is too large for single word slop objects which consist of a +single info table. + +Note that slop objects only contain an info pointer, not a standard +fixed header. This doesn't cause problems because slop objects are +always unreachable --- they can only be accessed by linearly scanning +the heap. + +\note{Currently we don't use slop objects because the storage manager +isn't reliant on objects being adjacent, but if we move to a ``mostly +copying'' style collector, this will become an issue.} + +\end{description} + +\Subsection{Thread State Objects (TSOs)}{TSO} + +In the multi-threaded system, the state of a suspended thread is +packed up into a Thread State Object (TSO) which contains all the +information needed to restart the thread and for the garbage collector +to find all reachable objects. When a thread is running, it may be +``unpacked'' into machine registers and various other memory locations +to provide faster access. + +Single-threaded systems don't really \emph{need\/} TSOs --- but they do +need some way to tell the storage manager about live roots so it is +convenient to use a single TSO to store the mutator state even in +single-threaded systems. + +Rather than manage TSOs' alloc/dealloc, etc., in some \emph{ad hoc} +way, we instead alloc/dealloc/etc them in the heap; then we can use +all the standard garbage-collection/fetching/flushing/etc machinery on +them. So that's why TSOs are ``heap objects,'' albeit very special +ones. +\begin{center} +\begin{tabular}{|l|l|} + \hline \emph{Fixed header} +\\ \hline \emph{Link field} +\\ \hline \emph{Mutable link field} +\\ \hline \emph{What next} +\\ \hline \emph{State} +\\ \hline \emph{Thread Id} +\\ \hline \emph{Exception Handlers} +\\ \hline \emph{Ticky Info} +\\ \hline \emph{Profiling Info} +\\ \hline \emph{Parallel Info} +\\ \hline \emph{GranSim Info} +\\ \hline \emph{Stack size} +\\ \hline \emph{Max Stack size} +\\ \hline \emph{Sp} +\\ \hline \emph{Su} +\\ \hline \emph{SpLim} +\\ \hline +\\ + \emph{Stack} +\\ +\\ \hline +\end{tabular} +\end{center} +The contents of a TSO are: +\begin{description} + +\item[\emph{Link field}] This is a pointer used to maintain a list of +threads with a similar state (e.g.~all runnable, all sleeping, all +blocked on the same black hole, all blocked on the same MVar, +etc.) + +\item[\emph{Mutable link field}] Because the stack is mutable by +definition, the generational collector needs to track TSOs in older +generations that may point into younger ones (which is just about any +TSO for a thread that has run recently). Hence the need for a mutable +link field (see \secref{mutables}). + +\item[\emph{What next}] +This field has five values: +\begin{description} +\item[@ThreadEnterGHC@] The thread can be started by entering the +closure pointed to by the word on the top of the stack. +\item[@ThreadRunGHC@] The thread can be started by jumping to the +address on the top of the stack. +\item[@ThreadEnterHugs@] The stack has a pointer to a Hugs-built +closure on top of the stack: enter the closure to run the thread. +\item[@ThreadKilled@] The thread has been killed (by @killThread#@). +It is probably still around because it is on some queue somewhere and +hasn't been garbage collected yet. +\item[@ThreadComplete@] The thread has finished. Its @TSO@ hasn't +been garbage collected yet. +\end{description} + +\item[\emph{Thread Id}] +This field contains a (not necessarily unique) integer that identifies +the thread. It can be used eg. for hashing. + +\item[\emph{Ticky Info}] Optional information for ``Ticky Ticky'' +statistics: @TSO_STK_HWM@ is the maximum number of words allocated to +this thread. + +\item[\emph{Profiling Info}] Optional information for profiling: +@TSO_CCC@ is the current cost centre. + +\item[\emph{Parallel Info}] +Optional information for parallel execution. + +% \begin{itemize} +% +% \item The types of threads (@TSO_TYPE@): +% \begin{description} +% \item[@T_MAIN@] Must be executed locally. +% \item[@T_REQUIRED@] A required thread -- may be exported. +% \item[@T_ADVISORY@] An advisory thread -- may be exported. +% \item[@T_FAIL@] A failure thread -- may be exported. +% \end{description} +% +% \item I've no idea what else +% +% \end{itemize} + +\item[\emph{GranSim Info}] +Optional information for gransim execution. + +% \item Optional information for GranSim execution: +% \begin{itemize} +% \item locked +% \item sparkname +% \item started at +% \item exported +% \item basic blocks +% \item allocs +% \item exectime +% \item fetchtime +% \item fetchcount +% \item blocktime +% \item blockcount +% \item global sparks +% \item local sparks +% \item queue +% \item priority +% \item clock (gransim light only) +% \end{itemize} +% +% +% Here are the various queues for GrAnSim-type events. +% +% Q_RUNNING +% Q_RUNNABLE +% Q_BLOCKED +% Q_FETCHING +% Q_MIGRATING +% + +\item[\emph{Stack Info}] Various fields contain information on the +stack: its current size, its maximum size (to avoid infinite loops +overflowing the memory), the current stack pointer (\emph{Sp}), the +current stack update frame pointer (\emph{Su}), and the stack limit +(\emph{SpLim}). The latter three fields are loaded into the relevant +registers when the thread is run. + +\item[\emph{Stack}] This is the actual stack for the thread, +\emph{Stack size} words long. It grows downwards from higher +addresses to lower addresses. When the stack overflows, it will +generally be relocated into larger premises unless \emph{Max stack +size} is reached. + +\end{description} + +The garbage collector needs to be able to find all the +pointers in a stack. How does it do this? + +\begin{itemize} + +\item Within the stack there are return addresses, pushed +by @case@ expressions. Below a return address (i.e. at higher +memory addresses, since the stack grows downwards) is a chunk +of stack that the return address ``knows about'', namely the +activation record of the currently running function. + +\item Below each such activation record is a \emph{pending-argument +section}, a chunk of +zero or more words that are the arguments to which the result +of the function should be applied. The return address does not +statically +``know'' how many pending arguments there are, or their types. +(For example, the function might return a result of type $\alpha$.) + +\item Below each pending-argument section is another return address, +and so on. Actually, there might be an update frame instead, but we +can consider update frames as a special case of a return address with +a well-defined activation record. + +\end{itemize} + +The game plan is this. The garbage collector walks the stack from the +top, traversing pending-argument sections and activation records +alternately. Next we discuss how it finds the pointers in each of +these two stack regions. + + +\Subsubsection{Activation records}{activation-records} + +An \emph{activation record} is a contiguous chunk of stack, +with a return address as its first word, followed by as many +data words as the return address ``knows about''. The return +address is actually a fully-fledged info pointer. It points +to an info table, replete with: + +\begin{itemize} +\item entry code (i.e. the code to return to). + +\item closure type is either @RET_SMALL/RET_VEC_SMALL@ or +@RET_BIG/RET_VEC_BIG@, depending on whether the activation record has +more than 32 data words (\note{64 for 8-byte-word architectures}) and +on whether to use a direct or a vectored return. + +\item the layout info for @RET_SMALL@ is a bitmap telling the layout +of the activation record, one bit per word. The least-significant bit +describes the first data word of the record (adjacent to the fixed +header) and so on. A ``@1@'' indicates a non-pointer, a ``@0@'' +indicates a pointer. We don't need to indicate exactly how many words +there are, because when we get to all zeros we can treat the rest of +the activation record as part of the next pending-argument region. + +For @RET_BIG@ the layout field points to a block of bitmap words, +starting with a word that tells how many words are in the block. + +\item the info table contains a Static Reference Table pointer for the +return address (\secref{srt}). +\end{itemize} + +The activation record is a fully fledged closure too. As well as an +info pointer, it has all the other attributes of a fixed header +(\secref{fixed-header}) including a saved cost centre which +is reloaded when the return address is entered. + +In other words, all the attributes of closures are needed for +activation records, so it's very convenient to make them look alike. + + +\Subsubsection{Pending arguments}{pending-args} + +So that the garbage collector can correctly identify pointers in +pending-argument sections we explicitly tag all non-pointers. Every +non-pointer in a pending-argument section is preceded (at the next +lower memory word) by a one-word byte count that says how many bytes +to skip over (excluding the tag word). + +The garbage collector traverses a pending argument section from the +top (i.e. lowest memory address). It looks at each word in turn: + +\begin{itemize} +\item If it is less than or equal to a small constant @ARGTAG_MAX@ +then it treats it as a tag heralding zero or more words of +non-pointers, so it just skips over them. + +\item If it points to the code segment, it must be a return +address, so we have come to the end of the pending-argument section. + +\item Otherwise it must be a bona fide heap pointer. +\end{itemize} + + +\Subsection{The Stable Pointer Table}{STABLEPTR_TABLE} + +A stable pointer is a name for a Haskell object which can be passed to +the external world. It is ``stable'' in the sense that the name does +not change when the Haskell garbage collector runs---in contrast to +the address of the object which may well change. + +A stable pointer is represented by an index into the +@StablePointerTable@. The Haskell garbage collector treats the +@StablePointerTable@ as a source of roots for GC. + +In order to provide efficient access to stable pointers and to be able +to cope with any number of stable pointers (eg $0 \ldots 100000$), the +table of stable pointers is an array stored on the heap and can grow +when it overflows. (Since we cannot compact the table by moving +stable pointers about, it seems unlikely that a half-empty table can +be reduced in size---this could be fixed if necessary by using a +hash table of some sort.) + +In general a stable pointer table closure looks like this: + +\begin{center} +\begin{tabular}{|l|l|l|l|l|l|l|l|l|l|l|} +\hline +\emph{Fixed header} & \emph{No of pointers} & \emph{Free} & $SP_0$ & \ldots & $SP_{n-1}$ +\\\hline +\end{tabular} +\end{center} + +The fields are: +\begin{description} + +\item[@NPtrs@:] number of (stable) pointers. + +\item[@Free@:] the byte offset (from the first byte of the object) of the first free stable pointer. + +\item[$SP_i$:] A stable pointer slot. If this entry is in use, it is +an ``unstable'' pointer to a closure. If this entry is not in use, it +is a byte offset of the next free stable pointer slot. + +\end{description} + +When a stable pointer table is evacuated +\begin{enumerate} +\item the free list entries are all set to @NULL@ so that the evacuation + code knows they're not pointers; + +\item The stable pointer slots are scanned linearly: non-@NULL@ slots +are evacuated and @NULL@-values are chained together to form a new free list. +\end{enumerate} + +There's no need to link the stable pointer table onto the mutable +list because we always treat it as a root. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\Subsection{Garbage Collecting CAFs}{CAF} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +% begin{direct quote from current paper} +A CAF (constant applicative form) is a top-level expression with no +arguments. The expression may need a large, even unbounded, amount of +storage when it is fully evaluated. + +CAFs are represented by closures in static memory that are updated +with indirections to objects in the heap space once the expression is +evaluated. Previous version of GHC maintained a list of all evaluated +CAFs and traversed them during GC, the result being that the storage +allocated by a CAF would reside in the heap until the program ended. +% end{direct quote from current paper} + +% begin{elaboration on why CAFs are very very bad} +Treating CAFs this way has two problems: +\begin{itemize} +\item +It can cause a very large space leak. For example, this program +should run in constant space but, instead, will run out of memory. +\begin{verbatim} +> main :: IO () +> main = print nats +> +> nats :: [Int] +> nats = [0..maxInt] +\end{verbatim} + +\item +Expressions with no arguments have very different space behaviour +depending on whether or not they occur at the top level. For example, +if we make \verb+nats+ a local definition, the space leak goes away +and the resulting program runs in constant space, as expected. +\begin{verbatim} +> main :: IO () +> main = print nats +> where +> nats :: [Int] +> nats = [0..maxInt] +\end{verbatim} + +This huge change in the operational behaviour of the program +is a problem for optimising compilers and for programmers. +For example, GHC will normally flatten a set of let bindings using +this transformation: +\begin{verbatim} +let x1 = let x2 = e2 in e1 ==> let x2 = e2 in let x1 = e1 +\end{verbatim} +but it does not do so if this would raise \verb+x2+ to the top level +since that may create a CAF. Many Haskell programmers avoid creating +large CAFs by adding a dummy argument to a CAF or by moving a CAF away +from the top level. + +\end{itemize} +% end{elaboration on why CAFs are very very bad} + +Solving the CAF problem requires different treatment in interactive +systems such as Hugs than in batch-mode systems such as GHC +\begin{itemize} +\item +In a batch-mode the program the runtime system is terminated +after every execution of the runtime system. In such systems, +the garbage collector can completely ``destroy'' a CAF when it +is no longer live --- in much the same way as it ``destroys'' +normal closures when they are no longer live. + +\item +In an interactive system, many expressions are evaluated without +restarting the runtime system between each evaluation. In such +systems, the garbage collector cannot completely ``destroy'' a CAF +when it is no longer live because, whilst it might not be required in +the evaluation of the current expression, it might be required in the +next evaluation. + +There are two possible behaviours we might want: +\begin{enumerate} +\item +When a CAF is no longer required for the current evaluation, the CAF +should be reverted to its original form. This behaviour ensures that +the operational behaviour of the interactive system is a reasonable +predictor of the operational behaviour of the batch-mode system. This +allows us to use Hugs for performance debugging (in particular, trying +to understand and reduce the heap usage of a program) --- an area of +increasing importance as Haskell is used more and more to solve ``real +problems'' in ``real problem domains''. + +\item +Even if a CAF is no longer required for the current evaluation, we might +choose to hang onto it by collecting it in the normal way. This keeps +the space leak but might be useful in a teaching environment when +trying to teach the difference between call by name evaluation (which +doesn't share work) and lazy evaluation (which does share work). + +\end{enumerate} + +It turns out that it is easy to support both styles of use, so the +runtime system provides a switch which lets us turn this on and off +during execution. \ToDo{What is this switch called?} It would also +be easy to provide a function \verb+RevertCAF+ to let the interpreter +revert any CAF it wanted between (but not during) executions, if we so +desired. Running \verb+RevertCAF+ during execution would lose some sharing +but is otherwise harmless. + +\end{itemize} + +% % begin{even more pointless observation?} +% The simplest fix would be to remove the special treatment of +% top level variables. This works but is very inefficient. +% ToDo: say why. +% (Note: delete this paragraph from final version.) +% % end{even more pointless observation?} + +% begin{pointless observation?} +An easy but inefficient fix to the CAF problem would be to make a +complete copy of the heap before every evaluation and discard the copy +after evaluation. This works but is inefficient. +% end{pointless observation?} + +An efficient way to achieve a similar effect is to revert all +updatable thunks to their original form as they become unnecessary for +the current evaluation. To do this, we modify the compiler to ensure +that the only updatable thunks generated by the compiler are CAFs and +we modify the garbage collector to revert entered CAFs to unentered +CAFs as their value becomes unnecessary. + + +\subsubsection{New Heap Objects} + +We add three new kinds of heap object: unentered CAF closures, entered +CAF objects and CAF blackholes. We first describe how they are +evaluated and then how they are garbage collected. +\begin{itemize} +\item +Unentered CAF closures contain a pointer to closure representing the +body of the CAF. The ``body closure'' is not updatable. + +Unentered CAF closures contain two unused fields to make them the same +size as entered CAF closures --- which allows us to perform an inplace +update. \ToDo{Do we have to add another kind of inplace update operation +to the storage manager interface or do we consider this to be internal +to the SM?} +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\verb+CAF_unentered+ & \emph{body closure} & \emph{unused} & \emph{unused} \\ \hline +\end{tabular} +\end{center} +When an unentered CAF is entered, we do the following: +\begin{itemize} +\item +allocate a CAF black hole; + +\item +push an update frame (to update the CAF black hole) onto the stack; + +\item +overwrite the CAF with an entered CAF object (see below) with the same +body and whose value field points to the black hole; + +\item +add the CAF to a list of all entered CAFs (called ``the CAF list''); +and + +\item +the closure representing the value of the CAF is entered. + +\end{itemize} + +When evaluation of the CAF body returns a value, the update frame +causes the CAF black hole to be updated with the value in the normal +way. + +\ToDo{Add a picture} + +\item +Entered CAF closures contain two pointers: a pointer to the CAF body +(the same as for unentered CAF closures); a pointer to the CAF value +(this is initialised with a CAF blackhole, as previously described); +and a link to the next CAF in the CAF list + +\ToDo{How is the end of the list marked? Null pointer or sentinel value?}. + +\begin{center} +\begin{tabular}{|l|l|l|l|}\hline +\verb+CAF_entered+ & \emph{body closure} & \emph{value} & \emph{link} \\ \hline +\end{tabular} +\end{center} +When an entered CAF is entered, it enters its value closure. + +\item +CAF blackholes are identical to normal blackholes except that they +have a different infotable. The only reason for having CAF blackholes +is to allow an optimisation of lazy blackholing where we stop scanning +the stack when we see the first {\em normal blackhole} but not +when we see a {\em CAF blackhole.} +\ToDo{The optimisation we want to allow should be described elsewhere +so that all we have to do here is describe the difference.} + +Instead of allocating a blackhole to update with the value of the CAF, +it might seem simpler to update the CAF directly. This would require +a new kind of update frame which would update the value field of the +CAF with a pointer to the value and wouldn't catch blackholes caused +by CAFs that depend on themselves so we chose not to do so. + +\end{itemize} + +\subsubsection{Garbage Collection} + +To avoid the space leak, each run of the garbage collector must revert +the entered CAFs which are not required to complete the current +evaluation (that is all the closures reachable from the set of +runnable threads and the stable pointer table). + +It does this by performing garbage collection in three phases: +\begin{enumerate} +\item +During the first phase, we ``mark'' all closures reachable from the +scheduler state. + +How we ``mark'' closures depends on the garbage collector. For +example, in a 2-space collector, closures are ``marked'' by copying +them into ``to-space'', overwriting them with a forwarding node and +``marking'' all the closures reachable from the copy. The only +requirements are that we can test whether a closure is marked and if a +closure is marked then so are all closures reachable from it. + +\ToDo{At present we say that the scheduler state includes any state +that Hugs may have. This is not true anymore.} + +Performing this phase first provides us with a cheap test for +execution closures: at this stage in execution, the execution closures +are precisely the marked closures. + +\item +During the second phase, we revert all unmarked CAFs on the CAF list +and remove them from the CAF list. + +Since the CAF list is exactly the set of all entered CAFs, this reverts +all entered CAFs which are not execution closures. + +\item +During the third phase, we mark all top level objects (including CAFs) +by calling \verb+MarkHugsRoots+ which will call \verb+MarkRoot+ for +each top level object known to Hugs. + +\end{enumerate} + +To implement the second style of interactive behaviour (where we +deliberately keep the CAF-related space leak), we simply omit the +second phase. Omitting the second phase causes the third phase to +mark any unmarked CAF value closures. + +So far, we have been describing a pure Hugs system which contains no +machine generated code. The main difference in a hybrid system is +that GHC-generated code is statically allocated in memory instead of +being dynamically allocated on the heap. We split both +\verb+CAF_unentered+ and \verb+CAF_entered+ into two versions: a +static and a dynamic version. The static and dynamic versions of each +CAF differ only in whether they are moved during garbage collection. +When reverting CAFs, we revert dynamic entered CAFs to dynamic +unentered CAFs and static entered CAFs to static unentered CAFs. + + + + +\Section{The Bytecode Evaluator}{bytecode-evaluator} + +This section describes how the Hugs interpreter interprets code in the +same environment as compiled code executes. Both evaluation models +use a common garbage collector, so they must agree on the form of +objects in the heap. + +Hugs interprets code by converting it to byte-code and applying a +byte-code interpreter to it. Wherever possible, we try to ensure that +the byte-code is all that is required to interpret a section of code. +This means not dynamically generating info tables, and hence we can +only have a small number of possible heap objects each with a statically +compiled info table. Similarly for stack objects: in fact we only +have one Hugs stack object, in which all information is tagged for the +garbage collector. + +There is, however, one exception to this rule. Hugs must generate +info tables for any constructors it is asked to compile, since the +alternative is to force a context-switch each time compiled code +enters a Hugs-built constructor, which would be prohibitively +expensive. + +We achieve this simplicity by forgoing some of the optimisations used +by compiled code: +\begin{itemize} +\item + +Whereas compiled code has five different ways of entering a closure +(\secref{ghc-fun-call}), interpreted code has only one. +The entry point for interpreted code behaves like slow entry points for +compiled code. + +\item + +We use just one info table for \emph{all\/} direct returns. +This introduces two problems: +\begin{enumerate} +\item How does the interpreter know what code to execute? + +Instead of pushing just a return address, we push a return BCO and a +trivial return address which just enters the return BCO. + +(In a purely interpreted system, we could avoid pushing the trivial +return address.) + +\item How can the garbage collector follow pointers within the +activation record? + +We could push a third word ---a bitmask describing the location of any +pointers within the record--- but, since we're already tagging unboxed +function arguments on the stack, we use the same mechanism for unboxed +values within the activation record. + +\ToDo{Do we have to stub out dead variables in the activation frame?} + +\end{enumerate} + +\item + +We trivially support vectored returns by pushing a return vector whose +entries are all the same. + +\item + +We avoid the need to build SRTs by putting bytecode objects on the +heap and restricting BCOs to a single basic block. + +\end{itemize} + +\Subsection{Hugs Info Tables}{hugs-info-tables} + +Hugs requires the following info tables and closures: +\begin{description} +\item [@HUGS\_RET@]. + +Contains both a vectored return table and a direct entry point. All +entry points are the same: they rearrange the stack to match the Hugs +return convention (\secref{hugs-return-convention}) and return to the +scheduler. When the scheduler restarts the thread, it will find a BCO +on top of the stack and will enter the Hugs interpreter. + +\item [@UPD\_RET@]. + +This is just the standard info table for an update frame. + +\item [Constructors]. + +The entry code for a constructor jumps to a generic entry point in the +runtime system which decides whether to do a vectored or unvectored +return depending on the shape of the constructor/type. This implies that +info tables must have enough info to make that decision. + +\item [@AP@ and @PAP@]. + +\item [Indirections]. + +\item [Selectors]. + +Hugs doesn't generate them itself but it ought to recognise them + +\item [Complex primops]. + +Some of the primops are too complex for GHC to generate inline. +Instead, these primops are hand-written and called as normal functions. +Hugs only needs to know their names and types but doesn't care whether +they are generated by GHC or by hand. Two things to watch: + +\begin{enumerate} +\item +Hugs must be able to enter these primops even if it is working on a +standalone system that does not support genuine GHC generated code. + +\item The complex primops often involve unboxed tuple types (which +Hugs does not support at the source level) so we cannot specify their +types in a Haskell source file. + +\end{enumerate} + +\end{description} + +\Subsection{Hugs Heap Objects}{hugs-heap-objects} + +\subsubsection{Byte-code objects} + +Compiled byte code lives on the global heap, in objects called +Byte-Code Objects (or BCOs). The layout of BCOs is described in +detail in \secref{BCO}, in this section we will describe +their semantics. + +Since byte-code lives on the heap, it can be garbage collected just +like any other heap-resident data. Hugs arranges that any BCO's +referred to by the Hugs symbol tables are treated as live objects by +the garbage collector. When a module is unloaded, the pointers to its +BCOs are removed from the symbol table, and the code will be garbage +collected some time later. + +A BCO represents a basic block of code --- the (only) entry points is +at the beginning of a BCO, and it is impossible to jump into the +middle of one. A BCO represents not only the code for a function, but +also its closure; a BCO can be entered just like any other closure. +Hugs performs lambda-lifting during compilation to byte-code, and each +top-level combinator becomes a BCO in the heap. + + +\subsubsection{Thunks and partial applications} + +A thunk consists of a code pointer, and values for the free variables +of that code. Since Hugs byte-code is lambda-lifted, free variables +become arguments and are expected to be on the stack by the called +function. + +Hugs represents updateable thunks with @AP_UPD@ objects applying a closure +to a list of arguments. (As for @PAP@s, unboxed arguments should be +preceded by a tag.) When it is entered, it pushes an update frame +followed by its payload on the stack, and enters the first word (which +will be a pointer to a BCO). The layout of @AP_UPD@ objects is described +in more detail in \secref{AP_UPD}. + +Partial applications are represented by @PAP@ objects, which are +non-updatable. + +\ToDo{Hugs Constructors}. + +\Subsection{Calling conventions}{hugs-calling-conventions} + +The calling convention for any byte-code function is straightforward: +\begin{itemize} +\item Push any arguments on the stack. +\item Push a pointer to the BCO. +\item Begin interpreting the byte code. +\end{itemize} + +In a system containing both GHC and Hugs, the bytecode interpreter +only has to be able to enter BCOs: everything else can be handled by +returning to the compiled world (as described in +\secref{hugs-to-ghc-switch}) and entering the closure +there. + +This would work but it would obviously be very inefficient if we +entered a @AP@ by switching worlds, entering the @AP@, pushing the +arguments and function onto the stack, and entering the function +which, likely as not, will be a byte-code object which we will enter +by \emph{returning} to the byte-code interpreter. To avoid such +gratuitious world switching, we choose to recognise certain closure +types as being ``standard'' --- and duplicate the entry code for the +``standard closures'' in the bytecode interpreter. + +A closure is said to be ``standard'' if its entry code is entirely +determined by its info table. \emph{Standard Closures} have the +desirable property that the byte-code interpreter can enter the +closure by simply ``interpreting'' the info table instead of switching +to the compiled world. The standard closures include: + +\begin{description} +\item[Constructor] To enter a constructor, we simply return (see +\secref{hugs-return-convention}). + +\item[Indirection] +To enter an indirection, we simply enter the object it points to +after possibly adjusting the current cost centre. + +\item[@AP@] + +To enter an @AP@, we push an update frame, push the +arguments, push the function and enter the function. +(Not forgetting a stack check at the start.) + +\item[@PAP@] + +To enter a @PAP@, we push the arguments, push the function and enter +the function. (Not forgetting a stack check at the start.) + +\item[Selector] + +To enter a selector (\secref{THUNK_SELECTOR}), we test whether the +selectee is a value. If so, we simply select the appropriate +component; if not, it's simplest to treat it as a GHC-built closure +--- though we could interpret it if we wanted. + +\end{description} + +The most obvious omissions from the above list are @BCO@s (which we +dealt with above) and GHC-built closures (which are covered in +\secref{hugs-to-ghc-switch}). + + +\Subsection{Return convention}{hugs-return-convention} + +When Hugs pushes a return address, it pushes both a pointer to the BCO +to return to, and a pointer to a static code fragment @HUGS_RET@ (this +is described in \secref{ghc-to-hugs-switch}). The +stack layout is shown in \figref{hugs-return-stack}. + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| bco |--> BCO ++----------+ +| HUGS_RET | ++----------+ +\end{verbatim} +%\input{hugs_ret.pstex_t} +\end{center} +\caption{Stack layout for a Hugs return address} +\label{fig:hugs-return-stack} +% this figure apparently duplicates {fig:hugs-return-stack1} earlier. +\end{figure} + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| con |--> CON ++----------+ +\end{verbatim} +%\input{hugs_ret2.pstex_t} +\end{center} +\caption{Stack layout on enterings a Hugs return address} +\label{fig:hugs-return2} +\end{figure} + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| 3# | ++----------+ +| I# | ++----------+ +\end{verbatim} +%\input{hugs_ret2.pstex_t} +\end{center} +\caption{Stack layout on entering a Hugs return address with an unboxed value} +\label{fig:hugs-return-int1} +\end{figure} + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| ghc_ret | ++----------+ +| con |--> CON ++----------+ +\end{verbatim} +%\input{hugs_ret3.pstex_t} +\end{center} +\caption{Stack layout on enterings a GHC return address} +\label{fig:hugs-return3} +\end{figure} + +\begin{figure}[ht] +\begin{center} +\begin{verbatim} +| stack | ++----------+ +| ghc_ret | ++----------+ +| 3# | ++----------+ +| I# | ++----------+ +| restart |--> id_Int#_closure ++----------+ +\end{verbatim} +%\input{hugs_ret2.pstex_t} +\end{center} +\caption{Stack layout on enterings a GHC return address with an unboxed value} +\label{fig:hugs-return-int} +\end{figure} + +When a Hugs byte-code sequence enters a closure, it examines the +return address on top of the stack. + +\begin{itemize} + +\item If the return address is @HUGS_RET@, pop the @HUGS_RET@ and the +bco for the continuation off the stack, push a pointer to the constructor onto +the stack and enter the BCO with the current object pointer set to the BCO +(\figref{hugs-return2}). + +\item If the top of the stack is not @HUGS_RET@, we need to do a world +switch as described in \secref{hugs-to-ghc-switch}. + +\end{itemize} + +\ToDo{This duplicates what we say about switching worlds +(\secref{switching-worlds}) - kill one or t'other.} + + +\ToDo{This was in the evaluation model part but it really belongs in +this part which is about the internal details of each of the major +sections.} + +\Subsection{Addressing Modes}{hugs-addressing-modes} + +To avoid potential alignment problems and simplify garbage collection, +all literal constants are stored in two tables (one boxed, the other +unboxed) within each BCO and are referred to by offsets into the tables. +Slots in the constant tables are word aligned. + +\ToDo{How big can the offsets be? Is the offset specified in the +address field or in the instruction?} + +Literals can have the following types: char, int, nat, float, double, +and pointer to boxed object. There is no real difference between +char, int, nat and float since they all occupy 32 bits --- but it +costs almost nothing to distinguish them and may improve portability +and simplify debugging. + +\Subsection{Compilation}{hugs-compilation} + + +\def\is{\mbox{\it is}} +\def\ts{\mbox{\it ts}} +\def\as{\mbox{\it as}} +\def\bs{\mbox{\it bs}} +\def\cs{\mbox{\it cs}} +\def\rs{\mbox{\it rs}} +\def\us{\mbox{\it us}} +\def\vs{\mbox{\it vs}} +\def\ws{\mbox{\it ws}} +\def\xs{\mbox{\it xs}} + +\def\e{\mbox{\it e}} +\def\alts{\mbox{\it alts}} +\def\fail{\mbox{\it fail}} +\def\panic{\mbox{\it panic}} +\def\ua{\mbox{\it ua}} +\def\obj{\mbox{\it obj}} +\def\bco{\mbox{\it bco}} +\def\tag{\mbox{\it tag}} +\def\entry{\mbox{\it entry}} +\def\su{\mbox{\it su}} + +\def\Ind#1{{\mbox{\it Ind}\ {#1}}} +\def\update#1{{\mbox{\it update}\ {#1}}} + +\def\next{$\Longrightarrow$} +\def\append{\mathrel{+\mkern-6mu+}} +\def\reverse{\mbox{\it reverse}} +\def\size#1{{\vert {#1} \vert}} +\def\arity#1{{\mbox{\it arity}{#1}}} + +\def\AP{\mbox{\it AP}} +\def\PAP{\mbox{\it PAP}} +\def\GHCRET{\mbox{\it GHCRET}} +\def\GHCOBJ{\mbox{\it GHCOBJ}} + +To make sense of the instructions, we need a sense of how they will be +used. Here is a small compiler for the STG language. + +\begin{verbatim} +> cg (f{a1, ... am}) = do +> pushAtom am; ... pushAtom a1 +> pushVar f +> SLIDE (m+1) |env| +> ENTER +> cg (let {x1=rhs1; ... xm=rhsm} in e) = do +> ALLOC x1 |rhs1|, ... ALLOC xm |rhsm| +> build x1 rhs1, ... build xm rhsm +> cg e +> cg (case e of alts) = do +> PUSHALTS (cgAlts alts) +> cg e + +> cgAlts { alt1; ... altm } = cgAlt alt1 $ ... $ cgAlt altm pmFail +> +> cgAlt (x@C{xs} -> e) fail = do +> TEST C fail +> HEAPCHECK (heapUse e) +> UNPACK xs +> cg e + +> build x (C{a1, ... am}) = do +> pushUntaggedAtom am; ... pushUntaggedAtom a1 +> PACK x C +> -- A useful optimisation +> build x ({v1, ... vm} \ {}. f{a1, ... am}) = do +> pushVar am; ... pushVar a1 +> pushVar f +> MKAP x m +> build x ({v1, ... vm} \ {}. e) = do +> pushVar vm; ... pushVar v1 +> PUSHBCO (cgRhs ({v1, ... vm} \ {}. e)) +> MKAP x m +> build x ({v1, ... vm} \ {x1, ... xm}. e) = do +> pushVar vm; ... pushVar v1 +> PUSHBCO (cgRhs ({v1, ... vm} \ {x1, ... xm}. e)) +> MKPAP x m + +> cgRhs (vs \ xs. e) = do +> ARGCHECK (xs ++ vs) -- can be omitted if xs == {} +> STACKCHECK min(stackUse e,heapOverflowSlop) +> HEAPCHECK (heapUse e) +> cg e + +> pushAtom x = pushVar x +> pushAtom i# = PUSHINT i# + +> pushVar x = if isGlobalVar x then PUSHGLOBAL x else PUSHLOCAL x + +> pushUntaggedAtom x = pushVar x +> pushUntaggedAtom i# = PUSHUNTAGGEDINT i# + +> pushVar x = if isGlobalVar x then PUSHGLOBAL x else PUSHLOCAL x +\end{verbatim} + +\ToDo{Is there an easy way to add semi-tagging? Would it be that different?} + +\ToDo{Optimise thunks of the form @f{x1,...xm}@ so that we build an AP directly} + +\Subsection{Instructions}{hugs-instructions} + +We specify the semantics of instructions using transition rules of +the form: + +\begin{tabular}{|llrrrrr|} +\hline + & $\is$ & $s$ & $\su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is'$ & $s'$ & $\su'$ & $h'$ & $hp'$ & $\sigma$ \\ +\hline +\end{tabular} + +where $\is$ is an instruction stream, $s$ is the stack, $\su$ is the +update frame pointer and $h$ is the heap. + + +\Subsection{Stack manipulation}{hugs-stack-manipulation} + +\begin{description} + +\item[ Push a global variable ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHGLOBAL $o$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $\sigma!o:s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[ Push a local variable ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHLOCAL $o$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s!o : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[ Push an unboxed int ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHINT $o$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $I\# : \sigma!o : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +The $I\#$ is a tag included for the benefit of the garbage collector. +Similar rules exist for floats, doubles, chars, etc. + +\item[ Push an unboxed int ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHUNTAGGEDINT $o$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $\sigma!o : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +Similar rules exist for floats, doubles, chars, etc. + +\item[ Delete environment from stack --- ready for tail call ]. + +\begin{tabular}{|llrrrrr|} +\hline + & SLIDE $m$ $n$ : $\is$ & $\as \append \bs \append \cs$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $\as \append \cs$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $\size{\as} = m$ and $\size{\bs} = n$. + + +\item[ Push a return address ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHALTS $o$:$\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $@HUGS_RET@:\sigma!o:s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[ Push a BCO ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PUSHBCO $o$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $\sigma!o : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\end{description} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\Subsection{Heap manipulation}{hugs-heap-manipulation} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{description} + +\item[ Allocate a heap object ]. + +\begin{tabular}{|llrrrrr|} +\hline + & ALLOC $m$ : $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $hp:s$ & $su$ & $h$ & $hp+m$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[ Build a constructor ]. + +\begin{tabular}{|llrrrrr|} +\hline + & PACK $o$ $o'$ : $\is$ & $\ws \append s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s$ & $su$ & $h[s!o \mapsto Pack C\{\ws\}]$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $C = \sigma!o'$ and $\size{\ws} = \arity{C}$. + +\item[ Build an AP or PAP ]. + +\begin{tabular}{|llrrrrr|} +\hline + & MKAP $o$ $m$:$\is$ & $f : \ws \append s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s$ & $su$ & $h[s!o \mapsto \AP(f,\ws)]$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $\size{\ws} = m$. + +\begin{tabular}{|llrrrrr|} +\hline + & MKPAP $o$ $m$:$\is$ & $f : \ws \append s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s$ & $su$ & $h[s!o \mapsto \PAP(f,\ws)]$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $\size{\ws} = m$. + +\item[ Unpacking a constructor ]. + +\begin{tabular}{|llrrrrr|} +\hline + & UNPACK : $is$ & $a : s$ & $su$ & $h[a \mapsto C\ \ws]$ & $hp$ & $\sigma$ \\ +\next & $is'$ & $(\reverse\ \ws) \append a : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +The $\reverse\ \ws$ looks expensive but, since the stack grows down +and the heap grows up, that's actually the cheap way of copying from +heap to stack. Looking at the compilation rules, you'll see that we +always push the args in reverse order. + +\end{description} + + +\Subsection{Entering a closure}{hugs-entering} + +\begin{description} + +\item[ Enter a BCO ]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : s$ & $su$ & $h[a \mapsto BCO\{\is\} ]$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $a : s$ & $su$ & $h$ & $hp$ & $a$ \\ +\hline +\end{tabular} + +\item[ Enter a PAP closure ]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : s$ & $su$ & $h[a \mapsto \PAP(f,\ws)]$ & $hp$ & $\sigma$ \\ +\next & [ENTER] & $f : \ws \append s$ & $su$ & $h$ & $hp$ & $???$ \\ +\hline +\end{tabular} + +\item[ Entering an AP closure ]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : s$ & $su$ & $h[a \mapsto \AP(f,ws)]$ & $hp$ & $\sigma$ \\ +\next & [ENTER] & $f : \ws \append @UPD_RET@:\su:a:s$ & $su'$ & $h$ & $hp$ & $???$ \\ +\hline +\end{tabular} + +Optimisations: +\begin{itemize} +\item Instead of blindly pushing an update frame for $a$, we can first test whether there's already + an update frame there. If so, overwrite the existing updatee with an indirection to $a$ and + overwrite the updatee field with $a$. (Overwriting $a$ with an indirection to the updatee also + works.) This results in update chains of maximum length 2. +\end{itemize} + + +\item[ Returning a constructor ]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : @HUGS_RET@ : \alts : s$ & $su$ & $h[a \mapsto C\{\ws\}]$ & $hp$ & $\sigma$ \\ +\next & $\alts.\entry$ & $a:s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + + +\item[ Entering an indirection node ]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : s$ & $su$ & $h[a \mapsto \Ind{a'}]$ & $hp$ & $\sigma$ \\ +\next & [ENTER] & $a' : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[Entering GHC closure]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : s$ & $su$ & $h[a \mapsto \GHCOBJ]$ & $hp$ & $\sigma$ \\ +\next & [ENTERGHC] & $a : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[Returning a constructor to GHC]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : \GHCRET : s$ & $su$ & $h[a \mapsto C \ws]$ & $hp$ & $\sigma$ \\ +\next & [ENTERGHC] & $a : \GHCRET : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\end{description} + + +\Subsection{Updates}{hugs-updates} + +\begin{description} + +\item[ Updating with a constructor]. + +\begin{tabular}{|llrrrrr|} +\hline + & [ENTER] & $a : @UPD_RET@ : ua : s$ & $su$ & $h[a \mapsto C\{\ws\}]$ & $hp$ & $\sigma$ \\ +\next & [ENTER] & $a \append s$ & $su$ & $h[au \mapsto \Ind{a}$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} + +\item[ Argument checks]. + +\begin{tabular}{|llrrrrr|} +\hline + & ARGCHECK $m$:$\is$ & $a : \as \append s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $a : \as \append s$ & $su$ & $h'$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $m \ge (su - sp)$ + +\begin{tabular}{|llrrrrr|} +\hline + & ARGCHECK $m$:$\is$ & $a : \as \append @UPD_RET@:su:ua:s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $a : \as \append s$ & $su$ & $h'$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $m < (su - sp)$ and + $h' = h[ua \mapsto \Ind{a'}, a' \mapsto \PAP(a,\reverse\ \as) ]$ + +Again, we reverse the list of values as we transfer them from the +stack to the heap --- reflecting the fact that the stack and heap grow +in different directions. + +\end{description} + +\Subsection{Branches}{hugs-branches} + +\begin{description} + +\item[ Testing a constructor ]. + +\begin{tabular}{|llrrrrr|} +\hline + & TEST $tag$ $is'$ : $is$ & $a : s$ & $su$ & $h[a \mapsto C\ \ws]$ & $hp$ & $\sigma$ \\ +\next & $is$ & $a : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $C.\tag = tag$ + +\begin{tabular}{|llrrrrr|} +\hline + & TEST $tag$ $is'$ : $is$ & $a : s$ & $su$ & $h[a \mapsto C\ \ws]$ & $hp$ & $\sigma$ \\ +\next & $is'$ & $a : s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +where $C.\tag \neq tag$ + +\end{description} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\Subsection{Heap and stack checks}{hugs-heap-stack-checks} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{tabular}{|llrrrrr|} +\hline + & STACKCHECK $stk$:$\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +if $s$ has $stk$ free slots. + +\begin{tabular}{|llrrrrr|} +\hline + & HEAPCHECK $hp$:$\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\next & $\is$ & $s$ & $su$ & $h$ & $hp$ & $\sigma$ \\ +\hline +\end{tabular} +\\ +if $h$ has $hp$ free slots. + +If either check fails, we push the current bco ($\sigma$) onto the +stack and return to the scheduler. When the scheduler has fixed the +problem, it pops the top object off the stack and reenters it. + + +Optimisations: +\begin{itemize} +\item The bytecode CHECK1000 conservatively checks for 1000 words of heap space and 1000 words of stack space. + We use it to reduce code space and instruction decoding time. +\item The bytecode HEAPCHECK1000 conservatively checks for 1000 words of heap space. + It is used in case alternatives. +\end{itemize} + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\Subsection{Primops}{hugs-primops} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\ToDo{primops take m words and return n words. The expect boxed arguments on the stack.} + + +\Section{The Machine Code Evaluator}{asm-evaluator} + +This section describes the framework in which compiled code evaluates +expressions. Only at certain points will compiled code need to be +able to talk to the interpreted world; these are discussed in +\secref{switching-worlds}. + +\Subsection{Calling conventions}{ghc-calling-conventions} + +\Subsubsection{The call/return registers}{ghc-regs} + +One of the problems in designing a virtual machine is that we want it +abstract away from tedious machine details but still reveal enough of +the underlying hardware that we can make sensible decisions about code +generation. A major problem area is the use of registers in +call/return conventions. On a machine with lots of registers, it's +cheaper to pass arguments and results in registers than to pass them +on the stack. On a machine with very few registers, it's cheaper to +pass arguments and results on the stack than to use ``virtual +registers'' in memory. We therefore use a hybrid system: the first +$n$ arguments or results are passed in registers; and the remaining +arguments or results are passed on the stack. For register-poor +architectures, it is important that we allow $n=0$. + +We'll label the arguments and results \Arg{1} \ldots \Arg{m} --- with +the understanding that \Arg{1} \ldots \Arg{n} are in registers and +\Arg{n+1} \ldots \Arg{m} are on top of the stack. + +Note that the mapping of arguments \Arg{1} \ldots \Arg{n} to machine +registers depends on the \emph{kinds} of the arguments. For example, +if the first argument is a Float, we might pass it in a different +register from if it is an Int. In fact, we might find that a given +architecture lets us pass varying numbers of arguments according to +their types. For example, if a CPU has 2 Int registers and 2 Float +registers then we could pass between 2 and 4 arguments in machine +registers --- depending on whether they all have the same kind or they +have different kinds. + +\Subsubsection{Entering closures}{entering-closures} + +To evaluate a closure we jump to the entry code for the closure +passing a pointer to the closure in \Arg{1} so that the entry code can +access its environment. + +\Subsubsection{Function call}{ghc-fun-call} + +The function-call mechanism is obviously crucial. There are five different +cases to consider: +\begin{enumerate} + +\item \emph{Known combinator (function with no free variables) and +enough arguments.} + +A fast call can be made: push excess arguments onto stack and jump to +function's \emph{fast entry point} passing arguments in \Arg{1} \ldots +\Arg{m}. + +The \emph{fast entry point} is only called with exactly the right +number of arguments (in \Arg{1} \ldots \Arg{m}) so it can instantly +start doing useful work without first testing whether it has enough +registers or having to pop them off the stack first. + +\item \emph{Known combinator and insufficient arguments.} + +A slow call can be made: push all arguments onto stack and jump to +function's \emph{slow entry point}. + +Any unpointed arguments which are pushed on the stack must be tagged. +This means pushing an extra word on the stack below the unpointed +words, containing the number of unpointed words above it. + +%Todo: forward ref about tagging? +%Todo: picture? + +The \emph{slow entry point} might be called with insufficient arguments +and so it must test whether there are enough arguments on the stack. +This \emph{argument satisfaction check} consists of checking that +@Su-Sp@ is big enough to hold all the arguments (including any tags). + +\begin{itemize} + +\item If the argument satisfaction check fails, it is because there is +one or more update frames on the stack before the rest of the +arguments that the function needs. In this case, we construct a PAP +(partial application, \secref{PAP}) containing the arguments +which are on the stack. The PAP construction code will return to the +update frame with the address of the PAP in \Arg{1}. + +\item If the argument satisfaction check succeeds, we jump to the fast +entry point with the arguments in \Arg{1} \ldots \Arg{arity}. + +If the fast entry point expects to receive some of \Arg{i} on the +stack, we can reduce the amount of movement required by making the +stack layout for the fast entry point look like the stack layout for +the slow entry point. Since the slow entry point is entered with the +first argument on the top of the stack and with tags in front of any +unpointed arguments, this means that if \Arg{i} is unpointed, there +should be space below it for a tag and that the highest numbered +argument should be passed on the top of the stack. + +We usually arrange that the fast entry point is placed immediately +after the slow entry point --- so we can just ``fall through'' to the +fast entry point without performing a jump. + +\end{itemize} + + +\item \emph{Known function closure (function with free variables) and +enough arguments.} + +A fast call can be made: push excess arguments onto stack and jump to +function's \emph{fast entry point} passing a pointer to closure in +\Arg{1} and arguments in \Arg{2} \ldots \Arg{m+1}. + +Like the fast entry point for a combinator, the fast entry point for a +closure is only called with appropriate values in \Arg{1} \ldots +\Arg{m+1} so we can start work straight away. The pointer to the +closure is used to access the free variables of the closure. + + +\item \emph{Known function closure and insufficient arguments.} + +A slow call can be made: push all arguments onto stack and jump to the +closure's slow entry point passing a pointer to the closure in \Arg{1}. + +Again, the slow entry point performs an argument satisfaction check +and either builds a PAP or pops the arguments off the stack into +\Arg{2} \ldots \Arg{m+1} and jumps to the fast entry point. + + +\item \emph{Unknown function closure, thunk or constructor.} + +Sometimes, the function being called is not statically identifiable. +Consider, for example, the @compose@ function: +\begin{verbatim} + compose f g x = f (g x) +\end{verbatim} +Since @f@ and @g@ are passed as arguments to @compose@, the latter has +to make a heap call. In a heap call the arguments are pushed onto the +stack, and the closure bound to the function is entered. In the +example, a thunk for @(g x)@ will be allocated, (a pointer to it) +pushed on the stack, and the closure bound to @f@ will be +entered. That is, we will jump to @f@s entry point passing @f@ in +\Arg{1}. If \Arg{1} is passed on the stack, it is pushed on top of +the thunk for @(g x)@. + +The \emph{entry code} for an updateable thunk (which must have arity 0) +pushes an update frame on the stack and starts executing the body of +the closure --- using \Arg{1} to access any free variables. This is +described in more detail in \secref{data-updates}. + +The \emph{entry code} for a non-updateable closure is just the +closure's slow entry point. + +\end{enumerate} + +In addition to the above considerations, if there are \emph{too many} +arguments then the extra arguments are simply pushed on the stack with +appropriate tags. + +To summarise, a closure's standard (slow) entry point performs the +following: + +\begin{description} +\item[Argument satisfaction check.] (function closure only) +\item[Stack overflow check.] +\item[Heap overflow check.] +\item[Copy free variables out of closure.] %Todo: why? +\item[Eager black holing.] (updateable thunk only) %Todo: forward ref. +\item[Push update frame.] +\item[Evaluate body of closure.] +\end{description} + + +\Subsection{Case expressions and return conventions}{return-conventions} + +The \emph{evaluation} of a thunk is always initiated by +a @case@ expression. For example: +\begin{verbatim} + case x of (a,b) -> E +\end{verbatim} + +The code for a @case@ expression looks like this: + +\begin{itemize} +\item Push the free variables of the branches on the stack (fv(@E@) in +this case). +\item Push a \emph{return address} on the stack. +\item Evaluate the scrutinee (@x@ in this case). +\end{itemize} + +Once evaluation of the scrutinee is complete, execution resumes at the +return address, which points to the code for the expression @E@. + +When execution resumes at the return point, there must be some {\em +return convention} that defines where the components of the pair, @a@ +and @b@, can be found. The return convention varies according to the +type of the scrutinee @x@: + +\begin{itemize} + +\item + +(A space for) the return address is left on the top of the stack. +Leaving the return address on the stack ensures that the top of the +stack contains a valid activation record +(\secref{activation-records}) --- should a garbage +collection be required. + +\item If @x@ has a boxed type (e.g.~a data constructor or a function), +a pointer to @x@ is returned in \Arg{1}. + +\ToDo{Warn that components of E should be extracted as soon as +possible to avoid a space leak.} + +\item If @x@ is an unboxed type (e.g.~@Int#@ or @Float#@), @x@ is +returned in \Arg{1} + +\item If @x@ is an unboxed tuple constructor, the components of @x@ +are returned in \Arg{1} \ldots \Arg{n} but no object is constructed in +the heap. + +When passing an unboxed tuple to a function, the components are +flattened out and passed in \Arg{1} \ldots \Arg{n} as usual. + +\end{itemize} + +\Subsection{Vectored Returns}{vectored-returns} + +Many algebraic data types have more than one constructor. For +example, the @Maybe@ type is defined like this: +\begin{verbatim} + data Maybe a = Nothing | Just a +\end{verbatim} +How does the return convention encode which of the two constructors is +being returned? A @case@ expression scrutinising a value of @Maybe@ +type would look like this: +\begin{verbatim} + case E of + Nothing -> ... + Just a -> ... +\end{verbatim} +Rather than pushing a return address before evaluating the scrutinee, +@E@, the @case@ expression pushes (a pointer to) a \emph{return +vector}, a static table consisting of two code pointers: one for the +@Just@ alternative, and one for the @Nothing@ alternative. + +\begin{itemize} + +\item + +The constructor @Nothing@ returns by jumping to the first item in the +return vector with a pointer to a (statically built) Nothing closure +in \Arg{1}. + +It might seem that we could avoid loading \Arg{1} in this case since the +first item in the return vector will know that @Nothing@ was returned +(and can easily access the Nothing closure in the (unlikely) event +that it needs it. The only reason we load \Arg{1} is in case we have to +perform an update (\secref{data-updates}). + +\item + +The constructor @Just@ returns by jumping to the second element of the +return vector with a pointer to the closure in \Arg{1}. + +\end{itemize} + +In this way no test need be made to see which constructor returns; +instead, execution resumes immediately in the appropriate branch of +the @case@. + +\Subsection{Direct Returns}{direct-returns} + +When a datatype has a large number of constructors, it may be +inappropriate to use vectored returns. The vector tables may be +large and sparse, and it may be better to identify the constructor +using a test-and-branch sequence on the tag. For this reason, we +provide an alternative return convention, called a \emph{direct +return}. + +In a direct return, the return address pushed on the stack really is a +code pointer. The returning code loads a pointer to the closure being +returned in \Arg{1} as usual, and also loads the tag into \Arg{2}. +The code at the return address will test the tag and jump to the +appropriate code for the case branch. If \Arg{2} isn't mapped to a +real machine register on this architecture, then we don't load it on a +return, instead using the tag directly from the info table. + +The choice of whether to use a vectored return or a direct return is +made on a type-by-type basis --- up to a certain maximum number of +constructors imposed by the update mechanism +(\secref{data-updates}). + +Single-constructor data types also use direct returns, although in +that case there is no need to return a tag in \Arg{2}. + +\ToDo{for a nullary constructor we needn't return a pointer to the +constructor in \Arg{1}.} + +\Subsection{Updates}{data-updates} + +The entry code for an updatable thunk (which must be of arity 0): + +\begin{itemize} +\item copies the free variables out of the thunk into registers or + onto the stack. +\item pushes an \emph{update frame} onto the stack. + +An update frame is a small activation record consisting of +\begin{center} +\begin{tabular}{|l|l|l|} +\hline +\emph{Fixed header} & \emph{Update Frame link} & \emph{Updatee} \\ +\hline +\end{tabular} +\end{center} + +\note{In the semantics part of the STG paper (section 5.6), an update +frame consists of everything down to the last update frame on the +stack. This would make sense too --- and would fit in nicely with +what we're going to do when we add support for speculative +evaluation.} +\ToDo{I think update frames contain cost centres sometimes} + +\item If we are doing ``eager blackholing,'' we then overwrite the +thunk with a black hole (\secref{BLACKHOLE}). Otherwise, we leave it +to the garbage collector to black hole the thunk. + +\item +Start evaluating the body of the expression. + +\end{itemize} + +When the expression finishes evaluation, it will enter the update +frame on the top of the stack. Since the returner doesn't know +whether it is entering a normal return address/vector or an update +frame, we follow exactly the same conventions as return addresses and +return vectors. That is, on entering the update frame: + +\begin{itemize} +\item The value of the thunk is in \Arg{1}. (Recall that only thunks +are updateable and that thunks return just one value.) + +\item If the data type is a direct-return type rather than a +vectored-return type, then the tag is in \Arg{2}. + +\item The update frame is still on the stack. +\end{itemize} + +We can safely share a single statically-compiled update function +between all types. However, the code must be able to handle both +vectored and direct-return datatypes. This is done by arranging that +the update code looks like this: + +\begin{verbatim} + | ^ | + | return vector | + |---------------| + | fixed-size | + | info table | + |---------------| <- update code pointer + | update code | + | v | +\end{verbatim} + +Each entry in the return vector (which is large enough to cover the +largest vectored-return type) points to the update code. + +The update code: +\begin{itemize} +\item overwrites the \emph{updatee} with an indirection to \Arg{1}; +\item loads @Su@ from the Update Frame link; +\item removes the update frame from the stack; and +\item enters \Arg{1}. +\end{itemize} + +We enter \Arg{1} again, having probably just come from there, because +it knows whether to perform a direct or vectored return. This could +be optimised by compiling special update code for each slot in the +return vector, which performs the correct return. + +\Subsection{Semi-tagging}{semi-tagging} + +When a @case@ expression evaluates a variable that might be bound +to a thunk it is often the case that the scrutinee is already evaluated. +In this case we have paid the penalty of (a) pushing the return address (or +return vector address) on the stack, (b) jumping through the info pointer +of the scrutinee, and (c) returning by an indirect jump through the +return address on the stack. + +If we knew that the scrutinee was already evaluated we could generate +(better) code which simply jumps to the appropriate branch of the +@case@ with a pointer to the scrutinee in \Arg{1}. (For direct +returns to multiconstructor datatypes, we might also load the tag into +\Arg{2}). + +An obvious idea, therefore, is to test dynamically whether the heap +closure is a value (using the tag in the info table). If not, we +enter the closure as usual; if so, we jump straight to the appropriate +alternative. Here, for example, is pseudo-code for the expression +@(case x of { (a,_,c) -> E }@: +\begin{verbatim} + \Arg{1} = <pointer to x>; + tag = \Arg{1}->entry->tag; + if (isWHNF(tag)) { + Sp--; \\ insert space for return address + goto ret; + } + push(ret); + goto \Arg{1}->entry; + + <info table for return address goes here> +ret: a = \Arg{1}->data1; \\ suck out a and c to avoid space leak + c = \Arg{1}->data3; + <code for E2> +\end{verbatim} +and here is the code for the expression @(case x of { [] -> E1; x:xs -> E2 }@: +\begin{verbatim} + \Arg{1} = <pointer to x>; + tag = \Arg{1}->entry->tag; + if (isWHNF(tag)) { + Sp--; \\ insert space for return address + goto retvec[tag]; + } + push(retinfo); + goto \Arg{1}->entry; + + .addr ret2 + .addr ret1 +retvec: \\ reversed return vector + <return info table for case goes here> +retinfo: + panic("Direct return into vectored case"); + +ret1: <code for E1> + +ret2: x = \Arg{1}->head; + xs = \Arg{1}->tail; + <code for E2> +\end{verbatim} +There is an obvious cost in compiled code size (but none in the size +of the bytecodes). There is also a cost in execution time if we enter +more thunks than data constructors. + +Both the direct and vectored returns are easily modified to chase chains +of indirections too. In the vectored case, this is most easily done by +making sure that @IND = TAG_1 - 1@, and adding an extra field to every +return vector. In the above example, the indirection code would be +\begin{verbatim} +ind: \Arg{1} = \Arg{1}->next; + goto ind_loop; +\end{verbatim} +where @ind_loop@ is the second line of code. + +Note that we have to leave space for a return address since the return +address expects to find one. If the body of the expression requires a +heap check, we will actually have to write the return address before +entering the garbage collector. + + +\Subsection{Heap and Stack Checks}{heap-and-stack-checks} + +The storage manager detects that it needs to garbage collect the old +generation when the evaluator requests a garbage collection without +having moved the heap pointer since the last garbage collection. It +is therefore important that the GC routines \emph{not} move the heap +pointer unless the heap check fails. This is different from what +happens in the current STG implementation. + +Assuming that the stack can never shrink, we perform a stack check +when we enter a closure but not when we return to a return +continuation. This doesn't work for heap checks because we cannot +predict what will happen to the heap if we call a function. + +If we wish to allow the stack to shrink, we need to perform a stack +check whenever we enter a return continuation. Most of these checks +could be eliminated if the storage manager guaranteed that a stack +would always have 1000 words (say) of space after it was shrunk. Then +we can omit stack checks for less than 1000 words in return +continuations. + +When an argument satisfaction check fails, we need to push the closure +(in R1) onto the stack - so we need to perform a stack check. The +problem is that the argument satisfaction check occurs \emph{before} +the stack check. The solution is that the caller of a slow entry +point or closure will guarantee that there is at least one word free +on the stack for the callee to use. + +Similarily, if a heap or stack check fails, we need to push the arguments +and closure onto the stack. If we just came from the slow entry point, +there's certainly enough space and it is the responsibility of anyone +using the fast entry point to guarantee that there is enough space. + +\ToDo{Be more precise about how much space is required - document it +in the calling convention section.} + +\Subsection{Handling interrupts/signals}{signals} + +\begin{verbatim} +May have to keep C stack pointer in register to placate OS? +May have to revert black holes - ouch! +\end{verbatim} + + + +\section{The Loader} +\section{The Compilers} + +\iffalse +\part{Old stuff - needs to be mined for useful info} + +\section{The Scheduler} + +The Scheduler is the heart of the run-time system. A running program +consists of a single running thread, and a list of runnable and +blocked threads. The running thread returns to the scheduler when any +of the following conditions arises: + +\begin{itemize} +\item A heap check fails, and a garbage collection is required +\item Compiled code needs to switch to interpreted code, and vice +versa. +\item The thread becomes blocked. +\item The thread is preempted. +\end{itemize} + +A running system has a global state, consisting of + +\begin{itemize} +\item @Hp@, the current heap pointer, which points to the next +available address in the Heap. +\item @HpLim@, the heap limit pointer, which points to the end of the +heap. +\item The Thread Preemption Flag, which is set whenever the currently +running thread should be preempted at the next opportunity. +\item A list of runnable threads. +\item A list of blocked threads. +\end{itemize} + +Each thread is represented by a Thread State Object (TSO), which is +described in detail in \secref{TSO}. + +The following is pseudo-code for the inner loop of the scheduler +itself. + +\begin{verbatim} +while (threads_exist) { + // handle global problems: GC, parallelism, etc + if (need_gc) gc(); + if (external_message) service_message(); + // deal with other urgent stuff + + pick a runnable thread; + do { + // enter object on top of stack + // if the top object is a BCO, we must enter it + // otherwise appply any heuristic we wish. + if (thread->stack[thread->sp]->info.type == BCO) { + status = runHugs(thread,&smInfo); + } else { + status = runGHC(thread,&smInfo); + } + switch (status) { // handle local problems + case (StackOverflow): enlargeStack; break; + case (Error e) : error(thread,e); break; + case (ExitWith e) : exit(e); break; + case (Yield) : break; + } + } while (thread_runnable); +} +\end{verbatim} + +\Subsection{Invoking the garbage collector}{ghc-invoking-gc} + +\Subsection{Putting the thread to sleep}{ghc-thread-sleeps} + +\Subsection{Calling C from Haskell}{ghc-ccall} + +We distinguish between "safe calls" where the programmer guarantees +that the C function will not call a Haskell function or, in a +multithreaded system, block for a long period of time and "unsafe +calls" where the programmer cannot make that guarantee. + +Safe calls are performed without returning to the scheduler and are +discussed elsewhere (\ToDo{discuss elsewhere}). + +Unsafe calls are performed by returning an array (outside the Haskell +heap) of arguments and a C function pointer to the scheduler. The +scheduler allocates a new thread from the operating system +(multithreaded system only), spawns a call to the function and +continues executing another thread. When the ccall completes, the +thread informs the scheduler and the scheduler adds the thread to the +runnable threads list. + +\ToDo{Describe this in more detail.} + + +\Subsection{Calling Haskell from C}{ghc-c-calls-haskell} + +When C calls a Haskell closure, it sends a message to the scheduler +thread. On receiving the message, the scheduler creates a new Haskell +thread, pushes the arguments to the C function onto the thread's stack +(with tags for unboxed arguments) pushes the Haskell closure and adds +the thread to the runnable list so that it can be entered in the +normal way. + +When the closure returns, the scheduler sends back a message which +awakens the (C) thread. + +\ToDo{Do we need to worry about the garbage collector deallocating the +thread if it gets blocked?} + +\Subsection{Switching Worlds}{switching-worlds} + +\ToDo{This has all changed: we always leave a closure on top of the +stack if we mean to continue executing it. The scheduler examines the +top of the stack and tries to guess which world we want to be in. If +it finds a @BCO@, it certainly enters Hugs, if it finds a @GHC@ +closure, it certainly enters GHC and if it finds a standard closure, +it is free to choose either one but it's probably best to enter GHC +for everything except @BCO@s and perhaps @AP@s.} + +Because this is a combined compiled/interpreted system, the +interpreter will sometimes encounter compiled code, and vice-versa. + +All world-switches go via the scheduler, ensuring that the world is in +a known state ready to enter either compiled code or the interpreter. +When a thread is run from the scheduler, the @whatNext@ field in the +TSO (\secref{TSO}) is checked to find out how to execute the +thread. + +\begin{itemize} +\item If @whatNext@ is set to @ReturnGHC@, we load up the required +registers from the TSO and jump to the address at the top of the user +stack. +\item If @whatNext@ is set to @EnterGHC@, we load up the required +registers from the TSO and enter the closure pointed to by the top +word of the stack. +\item If @whatNext@ is set to @EnterHugs@, we enter the top thing on +the stack, using the interpreter. +\end{itemize} + +There are four cases we need to consider: + +\begin{enumerate} +\item A GHC thread enters a Hugs-built closure. +\item A GHC thread returns to a Hugs-compiled return address. +\item A Hugs thread enters a GHC-built closure. +\item A Hugs thread returns to a Hugs-compiled return address. +\end{enumerate} + +GHC-compiled modules cannot call functions in a Hugs-compiled module +directly, because the compiler has no information about arities in the +external module. Therefore it must assume any top-level objects are +CAFs, and enter their closures. + +\ToDo{Hugs-built constructors?} + +We now examine the various cases one by one and describe how the +switch happens in each situation. + +\subsection{A GHC thread enters a Hugs-built closure} +\label{sec:ghc-to-hugs-switch} + +There is three possibilities: GHC has entered a @PAP@, or it has +entered a @AP@, or it has entered the BCO directly (for a top-level +function closure). @AP@s and @PAP@s are ``standard closures'' and +so do not require us to enter the bytecode interpreter. + +The entry code for a BCO does the following: + +\begin{itemize} +\item Push the address of the object entered on the stack. +\item Save the current state of the thread in its TSO. +\item Return to the scheduler, setting @whatNext@ to @EnterHugs@. +\end{itemize} + +BCO's for thunks and functions have the same entry conventions as +slow entry points: they expect to find their arguments on the stac +with unboxed arguments preceded by appropriate tags. + +\subsection{A GHC thread returns to a Hugs-compiled return address} +\label{sec:ghc-to-hugs-switch} + +Hugs return addresses are laid out as in \figref{hugs-return-stack}. +If GHC is returning, it will return to the address at the top of the +stack, namely @HUGS_RET@. The code at @HUGS_RET@ performs the +following: + +\begin{itemize} +\item pushes \Arg{1} (the return value) on the stack. +\item saves the thread state in the TSO +\item returns to the scheduler with @whatNext@ set to @EnterHugs@. +\end{itemize} + +\noindent When Hugs runs, it will enter the return value, which will +return using the correct Hugs convention +(\secref{hugs-return-convention}) to the return address underneath it +on the stack. + +\subsection{A Hugs thread enters a GHC-compiled closure} +\label{sec:hugs-to-ghc-switch} + +Hugs can recognise a GHC-built closure as not being one of the +following types of object: + +\begin{itemize} +\item A @BCO@, +\item A @AP@, +\item A @PAP@, +\item An indirection, or +\item A constructor. +\end{itemize} + +When Hugs is called on to enter a GHC closure, it executes the +following sequence of instructions: + +\begin{itemize} +\item Push the address of the closure on the stack. +\item Save the current state of the thread in the TSO. +\item Return to the scheduler, with the @whatNext@ field set to +@EnterGHC@. +\end{itemize} + +\subsection{A Hugs thread returns to a GHC-compiled return address} +\label{sec:hugs-to-ghc-switch} + +When Hugs encounters a return address on the stack that is not +@HUGS_RET@, it knows that a world-switch is required. At this point +the stack contains a pointer to the return value, followed by the GHC +return address. The following sequence is then performed: + +\begin{itemize} +\item save the state of the thread in the TSO. +\item return to the scheduler, setting @whatNext@ to @EnterGHC@. +\end{itemize} + +The first thing that GHC will do is enter the object on the top of the +stack, which is a pointer to the return value. This value will then +return itself to the return address using the GHC return convention. + + +\fi + + +\part{History} + +We're nuking the following: + +\begin{itemize} +\item + Two stacks + +\item + Return in registers. + This lets us remove update code pointers from info tables, + removes the need for phantom info tables, simplifies + semi-tagging, etc. + +\item + Threaded GC. + Careful analysis suggests that it doesn't buy us very much + and it is hard to work with. + + Eliminating threaded GCs eliminates the desire to share SMReps + so they are (once more) part of the Info table. + +\item + RetReg. + Doesn't buy us anything on a register-poor architecture and + isn't so important if we have semi-tagging. + +\begin{verbatim} + - Probably bad on register poor architecture + - Can avoid need to write return address to stack on reg rich arch. + - when a function does a small amount of work, doesn't + enter any other thunks and then returns. + eg entering a known constructor (but semitagging will catch this) + - Adds complications +\end{verbatim} + +\item + Update in place + + This lets us drop CONST closures and CHARLIKE closures (assuming we + don't support Unicode). The only point of these closures was to + avoid updating with an indirection. + + We also drop @MIN_UPD_SIZE@ --- all we need is space to insert an + indirection or a black hole. + +\item + STATIC SMReps are now called CONST + +\item + @MUTVAR@ is new + +\item The profiling ``kind'' field is now encoded in the @INFO_TYPE@ field. +This identifies the general sort of the closure for profiling purposes. + +\item Various papers describe deleting update frames for unreachable objects. + This has never been implemented and we don't plan to anytime soon. + +\end{itemize} + + +\end{document} + + diff --git a/docs/storage-mgt/Makefile b/docs/storage-mgt/Makefile new file mode 100644 index 0000000000..871766d4fc --- /dev/null +++ b/docs/storage-mgt/Makefile @@ -0,0 +1,37 @@ +# General makefile for Latex stuff + +dvi: sm.dvi rp.dvi ldv.dvi +ps: sm.ps rp.ps ldv.ps + +######## General rules +.SUFFIXES: +.PRECIOUS: %.tex %.ps %.bbl + +#%.dvi: %.tex $(addsuffix .tex, $(basename $(wildcard *.verb *.fig))) $(wildcard *.bib) +%.dvi: %.tex $(addsuffix .tex, $(basename $(wildcard *.verb))) $(wildcard *.bib) + latex $< + @if grep -s "\citation" $*.aux; then bibtex $*; fi + latex $< + latex $< + +%.ps: %.dvi + dvips -f < $< > $@ + +clean: + $(RM) *.aux *.log + +distclean: clean + $(RM) *.dvi *.ps *.bbl *.blg *.gz + +maintainer-clean: distclean + +# dummy targets +all: +boot: +install: +install-docs: +html: +chm: +HxS: + +# End of file diff --git a/docs/storage-mgt/architecture.eepic b/docs/storage-mgt/architecture.eepic new file mode 100644 index 0000000000..57ffd8fc99 --- /dev/null +++ b/docs/storage-mgt/architecture.eepic @@ -0,0 +1,55 @@ +\setlength{\unitlength}{0.00054167in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(5787,4014)(0,-10) +\path(2700,912)(5325,912)(5325,1212) + (2700,1212)(2700,912) +\path(2850,12)(5100,12)(5100,312) + (2850,312)(2850,12) +\path(2700,1812)(5325,1812)(5325,2112) + (2700,2112)(2700,1812) +\path(3825,2712)(5700,2712)(5700,3012) + (3825,3012)(3825,2712) +\path(3825,3687)(5625,3687)(5625,3987) + (3825,3987)(3825,3687) +\path(2625,3687)(3825,3687)(3825,3987) + (2625,3987)(2625,3687) +\path(3795.000,3357.000)(3825.000,3237.000)(3855.000,3357.000) +\path(3825,3237)(3825,3687) +\path(3855.000,3567.000)(3825.000,3687.000)(3795.000,3567.000) +\path(3795.000,1332.000)(3825.000,1212.000)(3855.000,1332.000) +\path(3825,1212)(3825,1812) +\path(3855.000,1692.000)(3825.000,1812.000)(3795.000,1692.000) +\path(1875,3237)(5775,3237)(5775,762) + (1875,762)(1875,3237) +\path(3855.000,642.000)(3825.000,762.000)(3795.000,642.000) +\path(3825,762)(3825,312) +\path(3795.000,432.000)(3825.000,312.000)(3855.000,432.000) +\path(2025,2712)(3525,2712)(3525,3012) + (2025,3012)(2025,2712) +\path(3195.000,2232.000)(3225.000,2112.000)(3255.000,2232.000) +\path(3225,2112)(3225,2712) +\path(3255.000,2592.000)(3225.000,2712.000)(3195.000,2592.000) +\path(4320.000,2232.000)(4350.000,2112.000)(4380.000,2232.000) +\path(4350,2112)(4350,2712) +\path(4380.000,2592.000)(4350.000,2712.000)(4320.000,2592.000) +\path(3525,2937)(3825,2937) +\path(3705.000,2907.000)(3825.000,2937.000)(3705.000,2967.000) +\path(3825,2787)(3525,2787) +\path(3645.000,2817.000)(3525.000,2787.000)(3645.000,2757.000) +\put(3225,1887){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}block allocator}}}}} +\put(3000,987){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}megablock allocator}}}}} +\put(3150,87){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}operating system}}}}} +\put(2700,3762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}mutatator}}}}} +\put(3900,3762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}runtime system}}}}} +\put(2100,2787){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}heap allocator}}}}} +\put(3975,2787){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}garbage collector}}}}} +\put(0,1962){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}storage manager}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/architecture.fig b/docs/storage-mgt/architecture.fig new file mode 100644 index 0000000000..563da78a53 --- /dev/null +++ b/docs/storage-mgt/architecture.fig @@ -0,0 +1,59 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +65.00 +Single +-2 +1200 2 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2400 4200 5025 4200 5025 3900 2400 3900 2400 4200 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2550 5100 4800 5100 4800 4800 2550 4800 2550 5100 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2400 3300 5025 3300 5025 3000 2400 3000 2400 3300 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 3525 2400 5400 2400 5400 2100 3525 2100 3525 2400 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 3525 1425 5325 1425 5325 1125 3525 1125 3525 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2325 1425 3525 1425 3525 1125 2325 1125 2325 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 3525 1875 3525 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 3525 3900 3525 3300 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 1575 1875 5475 1875 5475 4350 1575 4350 1575 1875 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 3525 4350 3525 4800 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 1725 2400 3225 2400 3225 2100 1725 2100 1725 2400 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 2925 3000 2925 2400 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 4050 3000 4050 2400 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 2175 3525 2175 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3525 2325 3225 2325 +4 0 0 50 0 0 12 0.0000 4 135 1110 2925 3225 block allocator\001 +4 0 0 50 0 0 12 0.0000 4 180 1515 2700 4125 megablock allocator\001 +4 0 0 50 0 0 12 0.0000 4 180 1305 2850 5025 operating system\001 +4 0 0 50 0 0 12 0.0000 4 105 735 2400 1350 mutatator\001 +4 0 0 50 0 0 12 0.0000 4 180 1170 3600 1350 runtime system\001 +4 0 0 50 0 0 12 0.0000 4 180 1065 1800 2325 heap allocator\001 +4 0 0 50 0 0 12 0.0000 4 180 1305 3675 2325 garbage collector\001 +4 0 0 50 0 0 12 0.0000 4 150 1260 -300 3150 storage manager\001 diff --git a/docs/storage-mgt/cacheprof_p.eps b/docs/storage-mgt/cacheprof_p.eps new file mode 100644 index 0000000000..94d3a5d0c2 --- /dev/null +++ b/docs/storage-mgt/cacheprof_p.eps @@ -0,0 +1,2083 @@ +%!PS-Adobe-2.0 EPSF-1.2 +%%Title: cacheprof_p -ghc-timing +RTS -H10m -K10m -p -hR -i1.0 -sstderr +%%Creator: Ghostscript ps2epsi from cacheprof_p.ps +%%CreationDate: Aug 23 18:51 +%%For:t-spark t-spark +%%Pages: 1 +%%DocumentFonts: Helvetica +%%BoundingBox: 72 107 505 756 +%%BeginPreview: 433 649 1 649 +% ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff80 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001ffffffffff080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000108000000084000000022000000022000000008000000000000000010000100001080 +% 800000000000000000000000100000000100000001cc0000000e600000006300000006300000000e000000000000000010420103c01080 +% 800000000000000000000000100000000100000001440000000a200000004900000004900000000b000000000000000010738106601080 +% 800000000000000000000003f80000003f800000012400000009200000004900000004900000003f800000000000000010588104201080 +% 8000000000000000000000000000000000000000011800000008c00000003e00000003e000000008000000000000000010488104201080 +% 800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000104c8106601080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010478102401080 +% 800000000000000000000006000000006000000003000000001800000000c00000000c0000000060000000000000000010000100001080 +% 800000000000002c00000001e00000001600000000f000000005800000003c00000002c00000001e000000000000000010420107201080 +% 800000000000006f000000031800000037800000018c0000000de00000006300000006f000000031800000000000000010738105a01080 +% 8000000000000045000000020800000022800000010400000008a000000041000000045000000020800000000000000010588105a01080 +% 8000000000000045000000021800000022800000010c00000008a000000043000000045000000021800000000000000010488104a01080 +% 800000000000003900000001f00000001c80000000f800000007200000003e00000003900000001f0000000000000000104c8107e01080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010478104001080 +% 800000000000003c00000001e00000001e00000000f000000007800000003c00000003c00000001e000000000000000010000103c01080 +% 8000000000000063000000031800000031800000018c0000000c6000000063000000063000000031800000000000000011c00106601080 +% 80000000000000410000000208000000208000000104000000082000000041000000041000000020800000000000000010800104201080 +% 8000000000000043000000021800000021800000010c000000086000000043000000043000000021800000000000000010000104201080 +% 800000000000003e00000001f00000001f00000000f800000007c00000003e00000003e00000001f000000000000000010000106601080 +% 800007800000003c00000001e00000001e00000000f000000007800000003c00000003c00000001e000000000000000010010102401080 +% 80000c6000000063000000031800000031800000018c0000000c6000000063000000063000000031800000000000000010010100001080 +% 800008200000004100000002080000002080000001040000000820000000410000000410000000208000000000000000107f8107fc1080 +% 8000086000000043000000021800000021800000010c000000086000000043000000043000000021800000000000000010000100201080 +% 800007c00000003e00000001f00000001f00000000f800000007c00000003e00000003e00000001f000000000200200010000100201080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200200010070100201080 +% 80000fe00000007f00000003f80000003f80000001fc0000000fe00000007f00000007f00000003f8000000003c8ff7c106d8107e01080 +% 800003000000001800000000c00000000c000000006000000003000000001800000001800000000c000000000225396010488101801080 +% 800003000000001800000000c00000000c000000006000000003000000001800000001800000000c0000000002253f3810488103c01080 +% 83c006800000003400000001a00000001a00000000d000000006800000003400000003400000001a0000000003273944106d8104a01080 +% 863008000000004000000002000000002000000001000000000800000000400000000400000000200000000003c23f78103f0104a01080 +% 84100000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000010000104e01080 +% 8430080000000080000000040000000020000000010000000010000000008000000004000000002000000000000c000010000102c01080 +% 83e00800000000800000000400000000200000000100000000100000000080000000040000000020000000000000000010010100001080 +% 80000800000000800000000400000000200000000100000000100000000080000000040000000020000000000000000010010100001080 +% 800008000000008000000004000000002000000001000000001000000000800000000400000000200000000000000000107f813fe01080 +% 8400fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe10000106601080 +% 83c00f00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000104201080 +% 86300e80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000106601080 +% 84100f60000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000103c01080 +% 84300f98000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011c00100001080 +% 83e00fd6000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010800100001080 +% 80000f61800000000000000000000000000000000000000000000000000000000000000000000000000000000000000010180107e01080 +% 80000ff14000000000000000000000000000000000000000000000000000000000000000000000000000000000000000101c0100201080 +% 80000f68300000000000000000000000000000000000000000000000000000000000000000000000000000000000000010120100201080 +% 80000fd55c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000010118103c01080 +% 80000baa0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000107fc106601080 +% 80000b7591c000000000000000000000000000000000000000000000000000000000000000000000000000000000000010100104201080 +% 80000fd2c02000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000104201080 +% 80000ff7755800000000000000000000000000000000000000000000000000000000000000000000000000000000000010180106601080 +% 80000da9b006000000000000000000000000000000000000000000000000000000000000000000000000000000000000101c0103c01080 +% 80000dd5d95180000000000000000000000000000000000000000000000000000000000000000000000000000000000010120100201080 +% 80000df46c0040000000000000000000000000000000000000000000000000000000000000000000000000000000000010118107fc1080 +% 80000ddb7755700000000000000000000000000000000000000000000000000000000000000000000000000000000000107fc100241080 +% 80000dea2b000c000000000000000000000000000000000000000000000000000000000000000000000000000000000010100100001080 +% 80000df5559113000000000000000000000000000000000000000000000000000000000000000000000000000000000010180110001080 +% 80000cfcaac000c000000000000000000000000000000000000000000000000000000000000000000000000000000000101c0110001080 +% 80000dd4d57555600000000000000000000000000000000000000000000000000000000000000000000000000000000010120110001080 +% 80000cbe42b000180000000000000000000000000000000000000000000000000000000000000000000000000000000010118110001080 +% 80000d765759511600000000000000000000000000000000000000000000000000000000000000000000000000000000107fc110001080 +% 80000c7b21ac00018000000000000000000000000000000000000000000000000000000000000000000000000000000010100110001080 +% 80000f5d15d755556000000000000000000000000000000000000000000000000000000000000000000000000000000010000110001080 +% 80000e7a906b0000100000000000000000000000000000000000000000000000000000000000000000000000000000001000013fe01080 +% 80000f558d7591111c00000000000000000000000000000000000000000000000000000000000000000000000000000010000106601080 +% 80000e3fca2ac0000300000000000000000000000000000000000000000000000000000000000000000000000000000010000104201080 +% 80000f7d4555755555c0000000000000000000000000000000000000000000000000000000000000000000000000000010000106601080 +% 80000e2ee20ab00000300000000000000000000000000000000000000000000000000000000000000000000000000000107fc103c01080 +% 80000f7f635559515158000000000000000000000000000000000000000000000000000000000000000000000000000010660100001080 +% 80000e27b122ac000006000000000000000000000000000000000000000000000000000000000000000000000000000010420100001080 +% 80000f55515757555555800000000000000000000000000000000000000000000000000000000000000000000000000010660100001080 +% 80000e1ff081ab0000006000000000000000000000000000000000000000000000000000000000000000000000000000103c0100001080 +% 80000f155855d5d11111180000000000000000000000000000000000000000000000000000000000000000000000000010000100801080 +% 80000e13b8626aa00000040000000000000000000000000000000000000000000000000000000000000000000000000010020100801080 +% 80000f5fd435755555555700000000000000000000000000000000000000000000000000000000000000000000000000121e0100801080 +% 80000f0bac201aa8000000c000000000000000000000000000000000000000000000000000000000000000000000000011f00100001080 +% 80000f1f561555555111513000000000000000000000000000000000000000000000000000000000000000000000000010700103c01080 +% 80001f0bfa122aaa0000000c000000000000000000000000000000000000000000000000000000000000000000000000101e0136601080 +% 80002d5d550d55555555555600000000000000000000000000000000000000000000000000000000000000000000000010020124201080 +% 80004d05eb0402aa8000000180000000000000000000000000000000000000000000000000000000000000000000000010020124201080 +% 80008d15f585555511111111600000000000000000000000000000000000000000000000000000000000000000000000107f813fe01080 +% 80008d07bb8223aaa00000001000000000000000000000000000000000000000000000000000000000000000000000001042011fe01080 +% 80004f55758355557555555558000000000000000000000000000000000000000000000000000000000000000000000010180100001080 +% 80002f0bee8200aab0000000060000000000000000000000000000000000000000000000000000000000000000000000103c0100001080 +% 80001f5d758355d559515151510000000000000000000000000000000000000000000000000000000000000000000000104a0107fc1080 +% 80000f0bfb81206aa8000000008000000000000000000000000000000000000000000000000000000000000000000000104a0100201080 +% 80000f5d7581555555555555556000000000000000000000000000000000000000000000000000000000000000000000104e0100201080 +% 80000f09ea81002aaa000000001000000000000000000000000000000000000000000000000000000000000000000000102c0100201080 +% 80000f1d758155755711111111180000000000000000000000000000000000000000000000000000000000000000000010000107e01080 +% 80000f0bbb80a23aab00000000060000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f5d7580d55555d55555555500000000000000000000000000000000000000000000000000000000000000000000106e0103c01080 +% 80000f19eec0801aaa8000000000c0000000000000000000000000000000000000000000000000000000000000000000104a0106601080 +% 80000f157540d55d55515111511160000000000000000000000000000000000000000000000000000000000000000000105a0104201080 +% 80000f13fbc0602aaaa0000000001000000000000000000000000000000000000000000000000000000000000000000010720104201080 +% 80000f57754055555575555555555c00000000000000000000000000000000000000000000000000000000000000000010000106601080 +% 80000f1beac04006aab0000000000200000000000000000000000000000000000000000000000000000000000000000010000102401080 +% 80000f17554055575559111111111100000000000000000000000000000000000000000000000000000000000000000010000100801080 +% 80000f93fbc02222aaa80000000000c0000000000000000000000000000000000000000000000000000000000000000010000100801080 +% 80000fd7554035555555555555555560000000000000000000000000000000000000000000000000000000000000000010400100801080 +% 80000fabeec02001aaaa000000000010000000000000000000000000000000000000000000000000000000000000000010660100001080 +% 80000ff755403555555551515151515c0000000000000000000000000000000000000000000000000000000000000000101c0100201080 +% 80000fa3fbc01020aaab0000000000020000000000000000000000000000000000000000000000000000000000000000103c0107f81080 +% 80000ff755401555d555d55555555555000000000000000000000000000000000000000000000000000000000000000010660104201080 +% 80000fabeac010006aaac00000000000c00000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fb5556015555555511111111111200000000000000000000000000000000000000000000000000000000000000010000107e41080 +% 80000fa7fba00a222aaaa00000000000100000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000ff555600d5575555555555555555c0000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fcfeee008001aaab00000000000020000000000000000000000000000000000000000000000000000000000000010000107e01080 +% 80000fd555600d555555591151115111510000000000000000000000000000000000000000000000000000000000000010000100201080 +% 80000fe7fba004223aaaac000000000000c0000000000000000000000000000000000000000000000000000000000000106e0100201080 +% 80000fd5556005555d555555555555555560000000000000000000000000000000000000000000000000000000000000104a0107e01080 +% 80000fefeae004000aaaaa00000000000010000000000000000000000000000000000000000000000000000000000000105a0100201080 +% 80000fd5556005555555551111111111111c00000000000000000000000000000000000000000000000000000000000010720100201080 +% 80001fe7fba0022226aaab0000000000000200000000000000000000000000000000000000000000000000000000000010000100201080 +% 80002ed555600355575555d555555555555500000000000000000000000000000000000000000000000000000000000010180107e01080 +% 80004e8feea0020002aaaac0000000000000c00000000000000000000000000000000000000000000000000000000000103c0100001080 +% 80008edd5560035557555551515151515151600000000000000000000000000000000000000000000000000000000000104a0100001080 +% 80008eabfbb0012021aaaaa0000000000000100000000000000000000000000000000000000000000000000000000000104a0107e41080 +% 80004edd5550015555d555555555555555555c0000000000000000000000000000000000000000000000000000000000104e0100001080 +% 80002eafeaf0008000aaaaa8000000000000020000000000000000000000000000000000000000000000000000000000102c0107e01080 +% 80001fdd555000d555555559111111111111118000000000000000000000000000000000000000000000000000000000103c0100201080 +% 84200fabfbb00062226aaaac00000000000000400000000000000000000000000000000000000000000000000000000010660100201080 +% 87300fdd555800555575555755555555555555600000000000000000000000000000000000000000000000000000000010420100201080 +% 85100f8feee80040001aaaab00000000000000180000000000000000000000000000000000000000000000000000000010420107e01080 +% 84900fd5555800355555555551115111511151140000000000000000000000000000000000000000000000000000000010660100001080 +% 84600fa7fbb80022202aaaaa80000000000000020000000000000000000000000000000000000000000000000000000010240103c01080 +% 80000fd55558001555555555555555555555555580000000000000000000000000000000000000000000000000000000103c0136601080 +% 8400ffafeae800100006aaaaa0000000000000004000000000000000000000000000000000000000000000000000000010660124201080 +% 80000fd57554000d5557555551111111111111113000000000000000000000000000000000000000000000000000000010420124201080 +% 83c00fa7bbbc000a2222aaaab000000000000000080000000000000000000000000000000000000000000000000000001042013fe01080 +% 86300fd57554000d555555555d55555555555555540000000000000000000000000000000000000000000000000000001066011fe01080 +% 84100f8feeac00040000aaaaac0000000000000003000000000000000000000000000000000000000000000000000000103c0100001080 +% 84300fd5755400055555d55557515151515151515180000000000000000000000000000000000000000000000000000010000100001080 +% 83e00fa7fbba000220206aaaaa000000000000000040000000000000000000000000000000000000000000000000000010000100001080 +% 80000fd57556000355557555555555555555555555700000000000000000000000000000000000000000000000000000107e0100001080 +% 80000fafeaea000100002aaaaa800000000000000008000000000000000000000000000000000000000000000000000010020101001080 +% 80000fd5755600015555555555511111111111111116000000000000000000000000000000000000000000000000000010020101001080 +% 80000fa7bbba000122223aaaaac00000000000000001000000000000000000000000000000000000000000000000000010020101001080 +% 80000fd575550000d5555d55557555555555555555558000000000000000000000000000000000000000000000000000107e0107e01080 +% 80000f8feeef0000800006aaaab00000000000000000600000000000000000000000000000000000000000000000000010000101001080 +% 80000fd57555000055555555555951115111511151115000000000000000000000000000000000000000000000000000103c0101001080 +% 80000fa7fbbb0000602222aaaaa80000000000000000080000000000000000000000000000000000000000000000000010660101001080 +% 80000fd5555500003555555555555555555555555555560000000000000000000000000000000000000000000000000010420100001080 +% 80000daffaeb0000200001aaaaaa0000000000000000010000000000000000000000000000000000000000000000000010420107fc1080 +% 80000dd555558000155555d55555111111111111111111c000000000000000000000000000000000000000000000000010660100441080 +% 80000da7bbbb8000122222aaaaab00000000000000000020000000000000000000000000000000000000000000000000107fc100441080 +% 80000dd555558000155555555555d555555555555555555000000000000000000000000000000000000000000000000010000100441080 +% 80000d8ffeae80000800002aaaaac000000000000000000c000000000000000000000000000000000000000000000000106e0100441080 +% 80000dd5555580000d555575555571515151515151515152000000000000000000000000000000000000000000000000104a0103cc1080 +% 80001da7fbbbc0000420203aaaaaa0000000000000000001000000000000000000000000000000000000000000000000105a0107b81080 +% 80002dd55555400005555555555555555555555555555555c0000000000000000000000000000000000000000000000010720100001080 +% 80004faffaeac0000200000aaaaaa800000000000000000020000000000000000000000000000000000000000000000010000100041080 +% 80008fd5555540000355555555555511111111111111111118000000000000000000000000000000000000000000000010000100041080 +% 80008fa3bbbba00001222222aaaaaa00000000000000000004000000000000000000000000000000000000000000000010000100041080 +% 80004d575d5560000155555755555555555555555555555557000000000000000000000000000000000000000000000010000107fc1080 +% 80002d4bfeeea00000800001aaaaab00000000000000000000800000000000000000000000000000000000000000000010000100041080 +% 80001d575555500000555555d5555591511151115111511151600000000000000000000000000000000000000000000010000100041080 +% 80000d63ffbbb000006220226aaaaac00000000000000000001800000000000000000000000000000000000000000000107f0100041080 +% 80000d7555555800003555557555557555555555555555555554000000000000000000000000000000000000000000001183c101001080 +% 80000f29feeae800001000002aaaaab0000000000000000000030000000000000000000000000000000000000000000010000103381080 +% 80000f35d755540000155555555555591111111111111111111180000000000000000000000000000000000000000000100001046c1080 +% 80000fa2bbbbbc00000a22222aaaaaac0000000000000000000060000000000000000000000000000000000000000000107fc104441080 +% 80000fd5d5555400000555555555555755555555555555555555580000000000000000000000000000000000000000001003c104441080 +% 80000f9affaeee000004000002aaaaab0000000000000000000004000000000000000000000000000000000000000000101e0104cc1080 +% 80000fd5555556000003555557555555d15151515151515151515300000000000000000000000000000000000000000010700107981080 +% 80000f927bbbbb000001202021aaaaaac00000000000000000000080000000000000000000000000000000000000000010700100001080 +% 80000fdd55d555000001555555d555557555555555555555555555600000000000000000000000000000000000000000101e0100001080 +% 80000f88bfeaea8000008000006aaaaab0000000000000000000001800000000000000000000000000000000000000001003c100001080 +% 80000f9d75d5558000005555557555555911111111111111111111140000000000000000000000000000000000000000107fc100001080 +% 80000f8a3bfbbb8000006222223aaaaaa80000000000000000000003000000000000000000000000000000000000000010000100801080 +% 80000fd5555555400000355555555555555555555555555555555555800000000000000000000000000000000000000010000100801080 +% 80000f849feeaec000001000000aaaaaaa00000000000000000000006000000000000000000000000000000000000000103fc100801080 +% 80000f95557555600000155555555555551151115111511151115111580000000000000000000000000000000000000010600100001080 +% 80000f863bbbbba0000008222022aaaaaa8000000000000000000000040000000000000000000000000000000000000010400100001080 +% 80000fd75d7555500000055555555555555555555555555555555555570000000000000000000000000000000000000010400107fc1080 +% 80000f82affaeaf0000004000001aaaaaaa000000000000000000000008000000000000000000000000000000000000010400100401080 +% 80000f935d555550000003555555d555555111111111111111111111116000000000000000000000000000000000000010600100401080 +% 80000e8227bbbbb80000012222226aaaaaa8000000000000000000000018000000000000000000000000000000000000103fc100401080 +% 80000ed5555d55580000015555557555555555555555555555555555555400000000000000000000000000000000000010004100401080 +% 80000ec18ffeeeac0000008000001aaaaaaa00000000000000000000000300000000000000000000000000000000000010004100401080 +% 80001ed1555d55540000005555555555555551515151515151515151515180000000000000000000000000000000000010004107fc1080 +% 80002ec123ffbbbc0000006020202aaaaaaa800000000000000000000000600000000000000000000000000000000000107fc100001080 +% 80004dd5d75555560000003555555555555555555555555555555555555558000000000000000000000000000000000010004100001080 +% 80008dc0abfeeaea00000010000002aaaaaac0000000000000000000000004000000000000000000000000000000000010004100101080 +% 80008dd1d75555550000001555555555555571111111111111111111111112000000000000000000000000000000000010004100101080 +% 80004fc0a3bfbbbb0000000a222223aaaaaab0000000000000000000000001000000000000000000000000000000000010000107f81080 +% 80002dd5d75555550000000d555555d555555d555555555555555555555555c00000000000000000000000000000000010000100001080 +% 80001fc08bfeaeef00000008000000aaaaaaa8000000000000000000000000200000000000000000000000000000000011c3c100001080 +% 80000dd15755555500000005555555555555551151115111511151115111511000000000000000000000000000000000107f0100001080 +% 80000fc123bfbbbb000000062022206aaaaaaa000000000000000000000000080000000000000000000000000000000010000100001080 +% 80000dd5575555550000000555555575555557555555555555555555555555540000000000000000000000000000000010000103f81080 +% 80000dc1abfeeaeb000000020000002aaaaaab000000000000000000000000020000000000000000000000000000000010000106181080 +% 80000dd1575555550000000355555555555555911111111111111111111111118000000000000000000000000000000010000104081080 +% 80000fc123bfbbbb000000022222223aaaaaaa800000000000000000000000004000000000000000000000000000000010000106181080 +% 80000dd557555555000000015555555d555555555555555555555555555555556000000000000000000000000000000010000103f01080 +% 80000dc18bfeeeae800000010000000aaaaaaac00000000000000000000000001000000000000000000000000000000010000100001080 +% 80000dd1575555558000000155555555555555715151515151515151515151515800000000000000000000000000000010000107e01080 +% 80000fc223ffbbbb80000000a0202026aaaaaab00000000000000000000000000400000000000000000000000000000010000100201080 +% 80000dd75755555580000000d5555557555555555555555555555555555555555700000000000000000000000000000010000100201080 +% 80000dc2abfeeaea8000000080000002aaaaaaa80000000000000000000000000080000000000000000000000000000010000107e01080 +% 80000dd3575555558000000055555555555555551111111111111111111111111140000000000000000000000000000010000100201080 +% 80000fc223bfbbbb8000000062222223aaaaaaac0000000000000000000000000020000000000000000000000000000010000100201080 +% 80000dd7575555558000000055555555d55555575555555555555555555555555550000000000000000000000000000010000100201080 +% 80000fc28bfeaeee8000000020000000aaaaaaab0000000000000000000000000008000000000000000000000000000010000107e01080 +% 80000dd3575555558000000035555555555555555111511151115111511151115116000000000000000000000000000010000100001080 +% 80000fc223bfbbbb80000000202220226aaaaaaa8000000000000000000000000001000000000000000000000000000010000100001080 +% 80000dd5575555554000000035555555755555555555555555555555555555555555800000000000000000000000000010000100001080 +% 80000dc4abfeeaeac0000000100000002aaaaaaac000000000000000000000000000400000000000000000000000000010000100001080 +% 80000dd5575555554000000015555555555555557111111111111111111111111111200000000000000000000000000010000100001080 +% 80000fc627bfbbbbc0000000122222223aaaaaaab000000000000000000000000000180000000000000000000000000010000100801080 +% 80001dd555555555400000000d5555555d5555555555555555555555555555555555540000000000000000000000000010000100801080 +% 81002dc68ffeeeaec0000000080000000aaaaaaaa800000000000000000000000000020000000000000000000000000010000100801080 +% 81c04dd555555555400000000d555555555555555551515151515151515151515151510000000000000000000000000010000100001080 +% 81608fc627ffbbbbc00000000420202026aaaaaaac00000000000000000000000000008000000000000000000000000010000100001080 +% 87f08dd5555555554000000005555555575555555755555555555555555555555555554000000000000000000000000010000107fc1080 +% 81004dc4affeeaeac00000000400000002aaaaaaab00000000000000000000000000003000000000000000000000000010000100401080 +% 80002dd5555555556000000003555555555555555511111111111111111111111111111800000000000000000000000010000100601080 +% 80001fca27bfbbbba00000000222222223aaaaaaaa80000000000000000000000000000400000000000000000000000010000100f01080 +% 8400fddd55555555600000000155555555d555555555555555555555555555555555555600000000000000000000000010000103981080 +% 80000fc88ffeaeeea00000000100000000aaaaaaaac0000000000000000000000000000180000000000000000000000010000106041080 +% 83c00ddd555555556000000001555555555555555571511151115111511151115111511140000000000000000000000010000104001080 +% 86300fca27bfbbbba000000000a22022206aaaaaaab0000000000000000000000000000020000000000000000000000010000100001080 +% 84100ddd555555556000000000d55555557555555555555555555555555555555555555550000000000000000000000010000100101080 +% 84300dc8affeeaeae000000000800000002aaaaaaaa800000000000000000000000000000c000000000000000000000010000100101080 +% 83e00ddd555555555000000000555555555555555555111111111111111111111111111112000000000000000000000010000107f81080 +% 80000fca27bfbbbbb000000000622222223aaaaaaaaa000000000000000000000000000001000000000000000000000010000100001080 +% 80000ddd555555555000000000555555555d55555557555555555555555555555555555555800000000000000000000010000100001080 +% 80000dca8ffeeeaef000000000200000000aaaaaaaab000000000000000000000000000000600000000000000000000010000100001080 +% 80000ddd555555555000000000355555555555555555d15151515151515151515151515151500000000000000000000010000103f81080 +% 80000fd227ffbbbbb0000000002020202022aaaaaaaa800000000000000000000000000000080000000000000000000010000106181080 +% 80000dd5555555555000000000155555555755555555555555555555555555555555555555540000000000000000000010000104081080 +% 80000dd8affeeaeae8000000001000000001aaaaaaaaa00000000000000000000000000000030000000000000000000010000106181080 +% 80000dd5555555555800000000155555555555555555711111111111111111111111111111118000000000000000000010000103f01080 +% 80000fd227bfbbbbb8000000000a22222222aaaaaaaab00000000000000000000000000000004000000000000000000010000100001080 +% 80000dd55555555558000000000d55555555d55555555d5555555555555555555555555555556000000000000000000010000100001080 +% 80000fd88ffeaeeea80000000004000000006aaaaaaaa80000000000000000000000000000001800000000000000000010000107e01080 +% 80000dd5555755555800000000055555555555555555551151115111511151115111511151115400000000000000000010000100201080 +% 80000fd227bbbbbbb80000000006202220222aaaaaaaaa0000000000000000000000000000000200000000000000000010000100201080 +% 80001dd5555755555800000000035555555575555555575555555555555555555555555555555500000000000000000010000107e01080 +% 80002db8affeeaeaec0000000002000000001aaaaaaaab00000000000000000000000000000000c0000000000000000010000100201080 +% 80004df5555755555400000000035555555555555555559111111111111111111111111111111120000000000000000010000100201080 +% 80008fb227bbbbbbbc0000000001222222222aaaaaaaaac000000000000000000000000000000010000000000000000010000100201080 +% 80008df557575555540000000001555555555555555555555555555555555555555555555555555c000000000000000010000107e01080 +% 80004dba8bfeeeaeee00000000008000000002aaaaaaaaa000000000000000000000000000000002000000000000000010000100001080 +% 80002df5575555555600000000005555555557555555555151515151515151515151515151515151800000000000000010000100001080 +% 80001fb223fbbbbbba00000000006020202021aaaaaaaaa800000000000000000000000000000000400000000000000010000100001080 +% 80000dfd555555555500000000003555555555d55555555555555555555555555555555555555555600000000000000010000100001080 +% 80000da8a9ffeaeaeb000000000010000000006aaaaaaaaa00000000000000000000000000000000180000000000000010000100801080 +% 80000dfd5555d5555580000000001555555555755555555511111111111111111111111111111111140000000000000010000100801080 +% 80000faa23bbbbbbbb80000000000a222222223aaaaaaaaa80000000000000000000000000000000030000000000000010000100801080 +% 80000df555d5d5555580000000000555555555555555555555555555555555555555555555555555558000000000000010000100001080 +% 80000fa48affeeeeaec00000000004000000000aaaaaaaaaa0000000000000000000000000000000006000000000000010000100001080 +% 80000df555d55555554000000000035555555555555555555111511151115111511151115111511151100000000000001000013fe01080 +% 80000fa622bbfbbbbba000000000012220222022aaaaaaaaa8000000000000000000000000000000000c00000000000010000106601080 +% 80000df5555555555560000000000155555555555555555555555555555555555555555555555555555600000000000010000104201080 +% 80000da2a8ffeaeaeae000000000008000000000aaaaaaaaaa000000000000000000000000000000000180000000000010000106601080 +% 80000df355557555555000000000005555555555d555555555111111111111111111111111111111111140000000000010000103c01080 +% 80000fa2227bbbbbbbb0000000000022222222226aaaaaaaaa800000000000000000000000000000000030000000000010000100001080 +% 80000df7557555555558000000000035555555557555555555555555555555555555555555555555555558000000000010000100001080 +% 80000da188bffeaeeea8000000000010000000001aaaaaaaaaa00000000000000000000000000000000004000000000010000100001080 +% 80000df155755555555800000000000d555555555d55555555515151515151515151515151515151515153000000000010000100001080 +% 80000fa1223bfbbbbbbc0000000000082020202026aaaaaaaaa80000000000000000000000000000000000800000000010000100801080 +% 80000df555555d555554000000000005555555555555555555555555555555555555555555555555555555600000000010000100801080 +% 80001da1a8bffaeaeaea0000000000020000000002aaaaaaaaaa0000000000000000000000000000000000100000000010000100801080 +% 80002df1d5555d5555560000000000035555555555555555555511111111111111111111111111111111111c0000000010000100001080 +% 80004fa0a22bbfbbbbba0000000000012222222222aaaaaaaaaa8000000000000000000000000000000000020000000010000100001080 +% 80008df5d55d55555555000000000000d55555555555555555555555555555555555555555555555555555558000000010000107fc1080 +% 80008fa08a8ffeeeaeef00000000000080000000002aaaaaaaaaa000000000000000000000000000000000004000000010000100201080 +% 80004df1d55d55555555000000000000555555555575555555555111511151115111511151115111511151116000000010000100201080 +% 80002fa0a22bffbbbbbb00000000000060222022203aaaaaaaaab000000000000000000000000000000000001800000010000100201080 +% 80001df5d55d57555555800000000000355555555555555555555d55555555555555555555555555555555555400000010000107e01080 +% 80000da0a8affeeaeaea80000000000020000000000aaaaaaaaaac00000000000000000000000000000000000200000010000100001080 +% 80000df1d55d5755555580000000000035555555555d555555555511111111111111111111111111111111111100000010000100001080 +% 80000fa0a22bbbbbbbbb800000000000122222222226aaaaaaaaaa00000000000000000000000000000000000080000010000107fc1080 +% 80000df5d55d57555555800000000000155555555555555555555555555555555555555555555555555555555560000010000100441080 +% 80000da0888ffeaeeeae800000000000080000000002aaaaaaaaaa80000000000000000000000000000000000010000010000100441080 +% 80000df1d55d575555554000000000000d55555555575555555555d1515151515151515151515151515151515158000010000100441080 +% 80000fa0a22bfbbbbbbbc00000000000082020202021aaaaaaaaaac0000000000000000000000000000000000004000010000100441080 +% 80000df5d55d57555555400000000000055555555555d55555555575555555555555555555555555555555555557000010000103cc1080 +% 80000da0a8affeeaeaeac00000000000040000000000aaaaaaaaaaa0000000000000000000000000000000000000800010000107b81080 +% 80000df1d55557555555400000000000035555555555555555555551111111111111111111111111111111111111400010000100001080 +% 80000fa0a227bbbbbbbbc000000000000222222222226aaaaaaaaaa8000000000000000000000000000000000000200010000100001080 +% 80000df5d5555755555560000000000003555555555575555555555d555555555555555555555555555555555555580010000100001080 +% 80000fa08a8ffeeeaeeea000000000000100000000002aaaaaaaaaac000000000000000000000000000000000000040010000100001080 +% 80000df1d55555555555600000000000015555555555555555555557511151115111511151115111511151115111520010000100801080 +% 80000fa0a227fbbbbbbba0000000000000a2202220223aaaaaaaaaab000000000000000000000000000000000000010010000100801080 +% 80000df5d5555555555560000000000000d5555555555d555555555555555555555555555555555555555555555555c010000100801080 +% 80001da0a8afffeaeaeae000000000000080000000000aaaaaaaaaaa800000000000000000000000000000000000002010000100001080 +% 80002df1d55555555555500000000000005555555555555555555555511111111111111111111111111111111111111010000100001080 +% 80004fa0a227bbbbbbbbb0000000000000622222222222aaaaaaaaaac00000000000000000000000000000000000000810000107e41080 +% 80008df5d55555d555555800000000000035555555555755555555557d5555555555555555555555555555555555555410000100001080 +% 80008fd0688bff7eeeaeee000000000000400000000002aaaaaaaaaaabf000000000000000000000000000000000001810000100001080 +% 80004fb97d55559d555555800000000000d555555555555555555555555ff1515151515151515151515151515151516010000100101080 +% 80002ee43e22fbc3bbbbbbf0000000000120202020202aaaaaaaaaaaaaaabf800000000000000000000000000000018010000100101080 +% 80001ff57dd555407555555c000000000155555555555d55555555555555557f5555555555555555555555555555560010000107f81080 +% 80000ffa1fe8bfe00eeaeaeb000000000200000000001aaaaaaaaaaaaaaaaaaafc00000000000000000000000000080010000100001080 +% 80000fdd1f7d5d7111d55555e00000000555555555555555555555555555555557f9111111111111111111111111300010000100001080 +% 80000ffe8ffe27b0003bbbbbb80000000622222222222aaaaaaaaaaaaaaaaaaaaaafe00000000000000000000000c00010000100001080 +% 80000fd55ff7575404075555570000000d55555555557555555555555555555555555fd555555555555555555557000010000104001080 +% 80000fff27ffcbf80000eeeeaec000001000000000006aaaaaaaaaaaaaaaaaaaaaaaaabf0000000000000000000c000010000100001080 +% 80000ff5f7ff75d911113d5555700000155555555555d555555555555555555555555555ff115111511151115130000010000100001080 +% 80000ffbf3fffe7c000007bbbbbe0000202220222022aaaaaaaaaaaaaaaaaaaaaaaaaaaaabf800000000000000c0000010000100001080 +% 83c00ff57fff5776404040f55555800055555555555555555555555555555555555555555557f555555555555700000010000103f81080 +% 86f00ffff5ffffbe0000001eeaeae000800000000001aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaafc0000000000c00000010000106181080 +% 84501ff56ffff77d11111113d5555c00d5555555555755555555555555555555555555555555557f111111113000000010000104081080 +% 84502ffbbbfffffb000000007bbbbb01222222222222aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaafe0000004000000010000106181080 +% 83a04ffd5dfff557c40444044d5555e355555555555555555555555555555555555555555555555555fd55558000000010000103f01080 +% 80008ffffeffffff8000000003aeeeba00000000000aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaff0060000000010000100001080 +% 80008ffd577ffd75d11111111155555d555555555575555555555555555555555555555555555555555ff9780000000010000100001080 +% 8400cffbfdffffff80000000007bbbb02020202021eaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabffe007800000000010000100001080 +% 80002ff55ffff5574040404040d555d555555555575555555555555555555555555555555555fffd5555f8000000000010000100001080 +% 83c01ffff7fffffe0000000001eaeb00000000003aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabfff0000000f00000000000010000100801080 +% 86300ff5fffff77f1111111111555d5555555555d55555555555555555555555555557fff111111111f000000000000010000100801080 +% 84100ffbbffffffc0000000003bbb2222222222eaaaaaaaaaaaaaaaaaaaaaaaaaafff800000000001f0000000000000010000100801080 +% 84300fd77ffd555c040404040555d5555555557555555555555555555555555fff55555555555555e00000000000000010000100001080 +% 83e00ffefffffff80000000006ef0000000001aaaaaaaaaaaaaaaaaaaaafffe0000000000000001e000000000000000010000100001080 +% 80000fddfff77751111111111d5d555555555f555555555555555555fff9511151115111511153e0000000000000000010000106e01080 +% 80000fb7ffffffa0000000000bb2202220227aaaaaaaaaaaaaaabfff000000000000000000003c00000000000000000010000104a01080 +% 80001f7fffd55d404040404055d555555557d5555555555557ffd55555555555555555555557c000000000000000000010000105a01080 +% 80002fdfffffffc0000000002b000000001eaaaaaaaaaafff80000000000000000000000007c0000000000000000000010000107201080 +% 80004fffff777591111111113d55555555755555557fff1111111111111111111111111117800000000000000000000010000100001080 +% 80008fbfffffbb00000000007222222223aaaaafffc0000000000000000000000000000078000000000000000000000010000100001080 +% 80008d7ff555550444044404d55555555d5555fd5555555555555555555555555555555780000000000000000000000010000106e01080 +% 80004dfffffffe0000000000800000001aaaab000000000000000000000000000000001800000000000000000000000010000104a01080 +% 80002d7ff5755711111111115555555575555751515151515151515151515151515151e000000000000000000000000010000105a01080 +% 80001dfffffffa0000000003202020206aaaac000000000000000000000000000000070000000000000000000000000010000107201080 +% 80000d7ff55556404040404755555555d5555d555555555555555555555555555555780000000000000000000000000010000100001080 +% 80000dbffffffc000000000400000003aaaaa0000000000000000000000000000000c00000000000000000000000000010000100201080 +% 80000d7fff7755111111111d55555555555551111111111111111111111111111117000000000000000000000000000010000107f81080 +% 80000fbfffffbc000000001a2222222aaaaa80000000000000000000000000000038000000000000000000000000000010000104201080 +% 80000d7ffd57540404040415555555555555555555555555555555555555555555c0000000000000000000000000000010000103c01080 +% 80000dbffffff800000000300000002aaaaa00000000000000000000000000000600000000000000000000000000000010000106601080 +% 80000d7ffd7759111111117555555555555551115111511151115111511151117800000000000000000000000000000010000104201080 +% 80000dbffffff800000000e2202220aaaab80000000000000000000000000001c000000000000000000000000000000010000104201080 +% 80000d7ffd575840404040d555555755557555555555555555555555555555560000000000000000000000000000000010000106601080 +% 80000dbffffff80000000180000006aaaac000000000000000000000000000380000000000000000000000000000000010000107fc1080 +% 80000d7fff7751111111135555555d55559111111111111111111111111111c00000000000000000000000000000000010000100001080 +% 80000fbfffffb0000000032222223aaaab000000000000000000000000000e000000000000000000000000000000000010000101801080 +% 80000d5ffd5554044404475555557555575555555555555555555555555570000000000000000000000000000000000010000103c01080 +% 80000dbffffff00000000c0000006aaaac00000000000000000000000001c0000000000000000000000000000000000010000104a01080 +% 80000d5ffd757111111115555555d555595151515151515151515151515e00000000000000000000000000000000000010000104a01080 +% 80000dbfffffe000000018202022aaaaa00000000000000000000000007000000000000000000000000000000000000010000104e01080 +% 80000d5ffd5560404040755555555555555555555555555555555555558000000000000000000000000000000000000010000102c01080 +% 80000dbfffffe00000006000000aaaaa8000000000000000000000000e0000000000000000000000000000000000000010000100001080 +% 80001d5fff7551111111755555555555111111111111111111111111700000000000000000000000000000000000000010000107e01080 +% 80002fbfffffc0000000e222222aaaaa000000000000000000000003800000000000000000000000000000000000000010000100201080 +% 80004f5ffd5d44040405d5555555555d55555555555555555555555c000000000000000000000000000000000000000010000100201080 +% 80008eafffffc0000002800001aaaab0000000000000000000000070000000000000000000000000000000000000000010000100001080 +% 80008f5ffd7d91111113555557555571511151115111511151115380000000000000000000000000000000000000000010000107e01080 +% 80004eaffffb80000006202226aaaac0000000000000000000000c00000000000000000000000000000000000000000010000100201080 +% 80002f5ffd5dc040404b55555d5555d5555555555555555555557000000000000000000000000000000000000000000010000100201080 +% 80001eafffff8000000c00001aaaab00000000000000000000018000000000000000000000000000000000000000000010000100001080 +% 80000f5fff7d1111111d5555755557111111111111111111111e0000000000000000000000000000000000000000000010000100001080 +% 80000eaffffb0000002a22226aaaac00000000000000000000300000000000000000000000000000000000000000000010000100001080 +% 80000f5ffd55440444355555d5555d55555555555555555555c00000000000000000000000000000000000000000000010000100001080 +% 80000eafffff000000600001aaaab000000000000000000006000000000000000000000000000000000000000000000010000100001080 +% 80000f5ff577111111d5555755557151515151515151515158000000000000000000000000000000000000000000000010000100001080 +% 80000eaffffa000001602026aaaac0000000000000000000e0000000000000000000000000000000000000000000000010000100001080 +% 80000f57f576404041d5555d55555555555555555555555700000000000000000000000000000000000000000000000010000100001080 +% 80000eaffffe00000300002aaaaa0000000000000000001c00000000000000000000000000000000000000000000000010000100001080 +% 80000f57f77711111555555555551111111111111111116000000000000000000000000000000000000000000000000010000100001080 +% 80000eaffffc0000062222aaaaa80000000000000000038000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f57404040d555555555555555555555555555c0000000000000000000000000000000000000000000000000010000100001080 +% 80000eaffffc0000180002aaaaa00000000000000000700000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f55511111d55555555515111511151115111800000000000000000000000000000000000000000000000000010000100001080 +% 80000eaffff8000030222aaaaa80000000000000000e000000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f55840407555555555555555555555555570000000000000000000000000000000000000000000000000000010000100001080 +% 80000eabfff80000a0002aaaae0000000000000001c0000000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f7591111d555555559111111111111111600000000000000000000000000000000000000000000000000000010000100001080 +% 80000eabfff00001a223aaaab0000000000000003800000000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f5d44406d55755557555555555555555c000000000000000000000000000000000000000000000000000000010000100001080 +% 80000eabfff000030006aaaac0000000000000070000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f5d11117555d5555d1515151515151580000000000000000000000000000000000000000000000000000000010000100001080 +% 80000eabffe0000c203aaaab00000000000000e00000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f57f5e0404d5575555755555555555557000000000000000000000000000000000000000000000000000000000010000100001080 +% 80001eabffe00018006aaaac0000000000001c000000000000000000000000000000000000000000000000000000000010000100001080 +% 80002f55f771113555d5555911111111111160000000000000000000000000000000000000000000000000000000000010000100001080 +% 80004eabffa0005223aaaab000000000000380000000000000000000000000000000000000000000000000000000000010000100001080 +% 80008f55f54404755755557555555555555c00000000000000000000000000000000000000000000000000000000000010000100001080 +% 80008eabffc000a006aaaac000000000003000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80004f55f55111d55d5555915111511151c000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80002faaffc001623aaaab0000000000070000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80001fd5f54041d57555575555555555580000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000faaffc001802aaaaa0000000000600000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fd57f5113555555551111111111800000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000feaffc00322aaaaa8000000000e000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f757dc447555555555555555570000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000efabfc00602aaaab000000000c0000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f757dd11d575555715151515300000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fbabfc00c26aaaac00000000c00000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f5d5dc04d5d5555d55555557000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000feebfc0181aaaaa800000018000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f5f5fd11d755555111111160000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fbeafc0322aaaaa000000180000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fd75fe435555555555555e00000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fefafe060aaaaac000003000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fd5dff17555555951115c000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000ffbafa061aaaab0000030000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fd5d760d75555755555c0000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000feaefe086aaaaa0000700000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86c00fd57771dd555551111800000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85c00ffbbbe13aaaaa80006000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400ff577e755555555558000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400feefbe22aaaab00060000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 82400ff55df355555751780000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83800ffbbfe6aaaaac00c00000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87c00ff557e555555d57000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400ffaeeedaaaab00c000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400ff557ff55555170000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87c01ffbbbfeaaaaa180000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80002ffd55fd55555600000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83804ffeaefaaaaa9800000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86408ffd55f55555e000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84408ffbbbeaaaab8000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84404ffd557555550000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 82c02ffaeaeaaaaa0000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83801ffd55d555560000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86400ff7bbeaaaaa0000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84400ff555d555540000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84400ff6efaaaaac0000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83800ff555d555540000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000ff7bbaaaaac0000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87c00fe5555555580000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80400fe6ebaaaaa80000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80400ff5575555580000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87c00fe7baaaaab00000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000fe5575555500000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83800fc6aeaaaab00000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86400fd5575555600000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84400fc7beaaaaa00000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84400fc55d5555600000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87f00f86eeaaaac00000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86c00f955d5555400000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85c00f87baaaaac00000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400f875d5555800000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 85400f82faaaaa800000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 82400f13555555800000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f03baaaab000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f43755555000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f02faaaab000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000f13755555000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8fe00e03eaaaaa000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 98300e07755556000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000e02eaaaaa000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87f00f13d55554000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80600c03eaaaac000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87800c43d55554000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 86000c02aaaaa8000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 81c01d13d55558000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80702c03aaaaa8000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87f04c07555550000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80008801aaaab0000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80008913555560000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83f04806aaab80000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 84002845555600000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8400180aaab800000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8400091d556000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 83f0081aab8000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000c35560000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8010086aac0000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80100955700000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 801008aac00000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87f009d7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 801009ac000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80100b70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 90100ec0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 98700d00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 87c00e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000003fff9fffcfffe7fff1fffcfffe7fff3fff8fffe7fff3fff9fffc7fff3fff9fffcfffe3fff9fffcfffe7fff00010000100001080 +% 80000003fff95554d556555514044d556400135558d55655553fff955545555355590404d556200095554d556555500010000100001080 +% 80000003fff9fffc8a8a7fff10004eeea400120008aaaa40013fff9fffc48a93fff90004aeee200090004aaaa400100010000100001080 +% 80000003fff97574d556555511114d556400135558d55651513fff977545555355591114d556200095554d556511500010000100001080 +% 80000003fff9fffca2227bbb10004bbba400122228aaaa40013fff9fffc62233fbb90004bbba200090224aaaa400100010000100001080 +% 80000003fff95554d556555510404d556400135558d55655553fff955545555355594044d556200095554d556555500010000100001080 +% 80000003fff9fffca8aa7fff10004aeae400120008aaaa40013fff9fffc4a8b3fff90004eaea200090004aaaa400100010000100001080 +% 80000003fff97774d556555511114d556400135558d55651113fff977745555355591114d556200095554d556511100010000100001080 +% 80000003fff9fffca2227bbb10004bbba400122228aaaa40013fff9fffc62233bbb90004bbba200092224aaaa400100010000100001080 +% 80000003fff95554d556555514044d556400135558d55655553fff955545555355594404d556200095554d556555500010000100001080 +% 80000003fff9fffc888a7fff10004eaee400120008aaaa40013fff9fffc68893fff90004eeae200090004aaaa400100010000100001080 +% 80000003fff97574d556555511114d556400135558d55651513fff957545555355591114d556200095554d556551500010000100001080 +% 80000003fff9fffca2227bfb10004bbba400122028aaaa40013fff9fffc62233fbf90004bbba200090204aaaa400100010000100001080 +% 80000003fff95554d556555510404d556400135558d55655553fff955545555355594044d556200095554d556555500010000100001080 +% 80000003fff9fffcfffe7fff1fffcfffe7fff3fff8fffe7fff3fff9fffc7fff3fff9fffcfffe3fff9fffcfffe7fff00010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000380000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 800000004400fe003f801fc00fe007f001fc00fe007f003f800fe007f003f801fc007f003f801fc00fe003f801fc000010000100001080 +% 800000008201830060c0306018300c180306018300c18060c018300c18060c030600c18060c030601830060c0306000010000100001080 +% 80000000820000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8000000082002c00000001c0000000000084002c00000010800000000000b000580000000f000200010000000020000010000100001080 +% 80000000c6006f0000800f200020001000e6006f0001001cc00020001001bc00de0001001bc0038001c000080038000010000100001080 +% 800000007c00450000800a200020001000a200450001001440002000100114008a000100114002c001600008002c000010000100001080 +% 80000000000045001fc00f2007f003f800920045003f80124007f003f80114008a003f8011400fe007f001fc00fe000010000100001080 +% 8000000002003900000007c000000000008c003900000011800000000000e400720000000e800200010000000020000010000100001080 +% 80000000020001000b00058003c00110001c002c0000000040000000000038008400000010800440030004040058000010000100001080 +% 80000000020041001bc00de006f0031800f2006f00010010400020001001e400e60001001cc00c6004f0061c00de000010000100001080 +% 80000000fe007900114008a00450024800a200450001001e4000200010014400a200010014400920049001f0008a000010000100001080 +% 8000000002000d00114008a00450024800f20045003f80034007f003f801e40092003f801240092004900000008a000010004100001080 +% 80000000020003000e40072003a001f0007c003900000000c00000000000f8008c000000118007c003e000800072000010004100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000001980000000010004100001080 +% 80000000fe01010040402020010001100202010100000040400010008004040202002100404020201010013402020000107fc100001080 +% 800000001001870061c030e001c00318030e018700010061c0041000e0061c030e00398061c030e018700124030e000010004100001080 +% 8000000010007c001f000f800160024800f8007c0001001f00079000b001f000f80028801f000f8007c0012400f8000010004100001080 +% 8000000010006c00ff00040007f0024800000064003f80ff0000d003f8019000d800248008000c80000001c800d8000010004100001080 +% 80000000fe005c001b000cc0010001f000f80054000000190000300080015000b800230019800a8007c0000400b8000010000100001080 +% 8000000000005400110009a01010080800080054008080110010100808015000a800808013400a800400000c00a80000107fc100001080 +% 80000000000054001100092018700c380008005400c380110018700c38015000a800c38012400a800400003000a8000010020100001080 +% 80000000fe0024001f00092007c003e000f8007c003e000e0007c003e001f00048003e0012400f8007c001e00048000010020100001080 +% 800000009201040000000e400000000000080000000000000000000000000002080000001c800000000000380208000010020100001080 +% 800000009201380019000000000000000008000000360000000000036000e002700036000000070006c0000c02700000107e0100001080 +% 800000009200e0001500002007c003e000f8007c002e001f0007c002e0019001c0002e0000400c8005c0000001c0000010000100001080 +% 8000000092003800150000600400002000c80004002a000100040002a001100070002a0000c00880054000800070000010000100001080 +% 8000000000000400150001800400002000a80004002a000e00040002a001100008002a00030008800540019800080000107e0100001080 +% 80000000000000001f000f0007c003e000a8007c0012001f0007c0012001fc00000012001e000fe0024001340000000010400100001080 +% 80000000fe007c00000001c006c0002000a80000008200150006c0082000e000f8008200038007000380012400f8000010400100001080 +% 80000000120004001f00006005c0002000f8007c009c00150005c009c001900008009c0000c00c8007c001240008000010600100001080 +% 800000001200040001000400054003e0000000040070001f00054007000110000800700008000880054001c800080000107e0100001080 +% 8000000012007c0000000cc00540032000000004001c000000054001c0011000f8001c00198008800540000000f8000010000100001080 +% 80000000f20000001b0009a0024002a000fa007c000200ff000240002001fc000000020013400fe007c000040000000010000100001080 +% 80000000ee00040017000920038002a0000000000000001900038000000000000800000012400000100000040008000010000100001080 +% 8000000000007f001500092007c002a000f80038003e00110007c003e0007000fe003e00124003801000000400fe000010000100001080 +% 800000000000440015000e40054003e000080064000200110005400020008800880002001c800440100001fc0088000010400100001080 +% 8000000000007f000900000005400000000800440002000e0005400020010400fe000200000008201000000400fe000010780100001080 +% 80000000000004000e00002007c003f800f80044003e00190007c003e001040008003e00004008201000000400080000100f0100001080 +% 80000000000004001f000020000000c000000038000200150000000020010400080002000040082010000000000800001009c100001080 +% 8000000000007c001500002001c000c000000004003f801500100003f8018c00f8003f8000400c6007f001fc00f800001009c100001080 +% 800000000000000015000fe0022001a00000007f00220015001000022000c800000022001fc006400640012400000000100f0100001080 +% 80000000000100001f00002004100200000000440000001f00100000000070020000000000400380044001240200000010780100001080 +% 800000000001000000000020041001c000000064003f800000100003f800880200003f8000400440044001240200000010400100001080 +% 800000000001000000000000041003e0000000540002000000100000200104020000020000000820038001240200000010000100001080 +% 800000000001000000000fe0063002a0000000540002001f0017f00020010402000002001fc0082007f0000002000000107e0100001080 +% 800000000001000000000920032002a000000054003e000100064003e001040200003e0012400820064001fc0200000010400100001080 +% 80000000000100000000092001c003e00000007c0000001b0004400000018c020000000012400c60044000180200000010400100001080 +% 800000000000420000000920022000e00000000000800017000440080000c8008400000012400640044001e00084000010600100001080 +% 800000000000730000000920041001100000000400800015000380080001b000e6003f8012400d800380018000e60000107e0100001080 +% 800000000000510000000000041002080000007f008000150000000800017000a200200000000b800000007000a20000103c0100001080 +% 800000000000490000000000041002080000007c008000090007f008000150009200200000000a800000001c0092000013660100001080 +% 800000000000460000000fe0063002080000007c0080000e00064008000150008c0020001fc00a80000001fc008c000012420100001080 +% 8000000000000000000000c0032003180000005400a1001f0004400a100090000000000001800480000000000000000012420100001080 +% 800000000000c00000000f00038001900000005400398015000440039804000180003e801e002000000000000000000013fe0100001080 +% 800000000000000000000c00064000e00000007c0028801500038002880400000000000018002000000000000000000011fe0100001080 +% 80000000000064000000038004400110000000000024801f000000024804000000003e0007002000000000000000000010000100001080 +% 8000000000005400000000e0044002080000010000230000000c000230040000f800020001c02000000000000000000010000100001080 +% 800000000000540000000fe007f0020800000100000000000000000000040000800002001fc02000000000000000000010000100001080 +% 8000000000005400000000000000020800000100000000000006c0000004000080003e0000002000000000000000000010000100001080 +% 8000000000007c00000018000380031800000100006000000005c00600003000f800000030000180000000000000000010420100001080 +% 80000000000038000000000007c001900000010000000000000540000001e000d8001c0000000f00000000000000000010738100001080 +% 8000000000006400000000000540000000000100003e000000054003e001c000b8003e001b000e00000000000000000010588100001080 +% 800000000000440000000f80054001c000000000000200000002400020003000a8002a0017000180000000000000000010488100001080 +% 80000000000044000000080007c003200000007d00020000001040002001e000a8002a0015000f000000000000000000104c8100001080 +% 8000000000007f000000080006c0022000000000003e000000138003e001c00048003e0015000e00000000000000000010478100001080 +% 800000000000000000000f8005c002200000007c00020000000e0000200030007000000009000180000000000000000010000100001080 +% 800000000000380000000000054003f800000004000200000003800020000000f800000041000000000000000000000010210100001080 +% 800000000000640000000d80054001c000000004003e000000004003e001f000a80000004e000f80000000000000000010718100001080 +% 800000000000440000000b80024003e00000007c000000000000000000001000a800000038000080000000000000000010408100001080 +% 800000000000440000000a80038002a00000006c003200000007c0032001fc00f80000000e000fe0000000000000000010448100001080 +% 8000000000007f0000000a80064002a00000005c002a000000004002a000600000000000010003000000000000000000104c8100001080 +% 800000000000000000000480044003e000000054002a000000004002a000600200000000000003000000000000000000103b0100001080 +% 8000000000001c00000007000440000000000054002a00000007c002a000d002000000001f000680000000000000000010000100001080 +% 800000000000220000000f8002c0036000000024003e000000000003e00100020000000001000800000000000000000010000100001080 +% 800000000000410000000a80000002e000000000003f800000004003f80000020000000001000000000000000000000010000100001080 +% 800000000000410000000a8007c002a00000007c000c00000007f000c0030002000000001f000000000000000000000010000100001080 +% 800000000000410000000f80004002a000000004000c000000044000c00000020000000000000000000000000000000010000100001080 +% 8000000000006300000020000000012000000004001a000000000001a0000000fe00000001000000000000000000000010010100001080 +% 800000000000320000002000000001c00000007c002000000007f0020001f000c80000001fc00000000000000000000010010100001080 +% 8000000000001c000000200007d0032000000000001c000000004001c001000088000000110000000000000000000000107f8100001080 +% 8000000000002200000020003fc0022000000000003e000000004003e0010000880000001fc00000000000000000000010000100001080 +% 8000000000004100000020000640022000000000002a00000007c002a001f0007000000001000000000000000000000010000100001080 +% 8000000000004100000020000440016000000000002a000000000002a0000000fe00000001000000000000000000000010000100001080 +% 800000000000410000000fe00440000000000000003e00000007f003e001b000c80000001f000000000000000000000010008100001080 +% 800000000000630000000c80038003e0000000000000000000040000000170008800000000000000000000000000000010008100001080 +% 8000000000003200000008800040002000000000000e000000040000e00150008800000040000000000000000000000010708100001080 +% 8000000000006c000000088007f000000000000000110000000400011001500070000000400000000000000000000000101c8100001080 +% 8000000000005c0000000700044003e8000000000020800000000002080090000000000040000000000000000000000010028100001080 +% 800000000000540000000fe003801fe00000000000208000000000020800e0000000000040000000000000000000000010018100001080 +% 800000000000540000000c800640032000000000002080000007d0020801f0000000000040000000000000000000000010000100001080 +% 80000000000024000000088004400220000000000031800000000003180150000000000050800000000000000000000010420100001080 +% 8000000000010000000008800440022000000000001900000007c00190015000000000001cc00000000000000000000010000100001080 +% 800000000001000000000700038001c000000000000e000000004000e001f0000000000014400000000000000000000010000100001080 +% 80000000000100000000000000000000000000000011000000004001100400000000000012400000000000000000000010210100001080 +% 8000000000010000000000000000002000000000002080000007c002080400000000000011800000000000000000000010718100001080 +% 80000000000100000000000007c003f8000000000020800000038002080400000000000000000000000000000000000010408100001080 +% 8000000000010000000000000040022000000000002080000007c002080400000000000000000000000000000000000010448100001080 +% 8000000000000c000000000006c001c00000000000318000000540031804000000000000000000000000000000000000104c8100001080 +% 80000000000078000000000005c003200000000000190000000540019004000000000000000000000000000000000000103b0100001080 +% 8000000000007000000000000540022000000000001c00000007c001c001fc000000000000000000000000000000000010008100001080 +% 8000000000000c000000000005400220000000000032000000000003200190000000000000000000000000000000000010008100001080 +% 800000000000780000000000024001c0000000000022000000000002200110000000000000000000000000000000000010708100001080 +% 800000000000700000000000000000000000000000220000000000022001100000000000000000000000000000000000101c8100001080 +% 8000000000000c0000000000000003e000000000003f800000000003f800e0000000000000000000000000000000000010028100001080 +% 800000000000000000000000000000200000000000000000000000000001fc000000000000000000000000000000000010018100001080 +% 8000000000007c00000000000000036000000000001c000000000001c00190000000000000000000000000000000000010000100001080 +% 800000000000040000000000000002e000000000003e000000000003e00110000000000000000000000000000000000010000100001080 +% 8000000000007f0000000000000002a000000000002a000000000002a00110000000000000000000000000000000000010000100001080 +% 800000000000180000000000000002a000000000002a000000000002a000e0000000000000000000000000000000000010000100001080 +% 8000000000001800000000000000012000000000003e000000000003e00000000000000000000000000000000000000010420100001080 +% 80000000000034000000000000000000000000000036000000000003600000000000000000000000000000000000000010738100001080 +% 8000000000004000000000000000000000000000002e000000000002e00000000000000000000000000000000000000010588100001080 +% 8000000000000000000000000000000000000000002a000000000002a00000000000000000000000000000000000000010488100001080 +% 8000000000000000000000000000000000000000002a000000000002a000000000000000000000000000000000000000104c8100001080 +% 80000000000000000000000000000000000000000012000000000001200000000000000000000000000000000000000010478100001080 +% 8000000000000000000000000000000000000000001c000000000001c00000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000032000000000003200000000000000000000000000000000000000010000100001080 +% 800000000000000000000000000000000000000000220000000000022000000000000000000000000000000000000000103f8100001080 +% 80000000000000000000000000000000000000000022000000000002200000000000000000000000000000000000000010618100001080 +% 80000000000000000000000000000000000000000016000000000001600000000000000000000000000000000000000010408100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010618100001080 +% 8000000000000000000000000000000000000000003e000000000003e000000000000000000000000000000000000000103f0100001080 +% 80000000000000000000000000000000000000000002000000000000200000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000103f8100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010618100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010408100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010618100001080 +% 800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000103f0100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010010100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010010100001080 +% 800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000107f8100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000100001080 +% 8000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001ffffffffff080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% 80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080 +% ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff80 +%%EndImage +%%EndPreview +save +countdictstack +mark +newpath +/showpage {} def +/setpagedevice {pop} def +%%EndProlog +%%Page 1 1 + +-90 rotate +-756.000000 72.000000 translate +/HE10 /Helvetica findfont 10 scalefont def +/HE12 /Helvetica findfont 12 scalefont def +newpath +0 0 moveto +0 432.000000 rlineto +648.000000 0 rlineto +0 -432.000000 rlineto +closepath +0.500000 setlinewidth +stroke +newpath +5.000000 387.000000 moveto +0 40.000000 rlineto +638.000000 0 rlineto +0 -40.000000 rlineto +closepath +0.500000 setlinewidth +stroke +5.000000 407.000000 moveto +638.000000 0 rlineto +stroke +HE12 setfont +11.000000 413.000000 moveto +(cacheprof_p -ghc-timing +RTS -H10m -K10m -p -hR -i1.0 -sstderr +) show +HE12 setfont +11.000000 393.000000 moveto +(22,191,444 bytes x seconds (MUT)) +show +HE12 setfont +(Thu Aug 23 17:37 2001) +dup stringwidth pop +637.000000 +exch sub +393.000000 moveto +show +45.000000 20.000000 moveto +431.338567 0 rlineto +0.500000 setlinewidth +stroke +HE10 setfont +(seconds (MUT)) +dup stringwidth pop +476.338567 +exch sub +5.000000 moveto +show +45.000000 20.000000 moveto +0 -4 rlineto +stroke +HE10 setfont +(0.0) +dup stringwidth pop +2 div +45.000000 exch sub +5.000000 moveto +show +135.712632 20.000000 moveto +0 -4 rlineto +stroke +HE10 setfont +(2.0) +dup stringwidth pop +2 div +135.712632 exch sub +5.000000 moveto +show +226.425265 20.000000 moveto +0 -4 rlineto +stroke +HE10 setfont +(4.0) +dup stringwidth pop +2 div +226.425265 exch sub +5.000000 moveto +show +317.137897 20.000000 moveto +0 -4 rlineto +stroke +HE10 setfont +(6.0) +dup stringwidth pop +2 div +317.137897 exch sub +5.000000 moveto +show +45.000000 20.000000 moveto +0 362.000000 rlineto +0.500000 setlinewidth +stroke +gsave +HE10 setfont +(bytes) +dup stringwidth pop +382.000000 +exch sub +40.000000 exch +translate +90 rotate +0 0 moveto +show +grestore +45.000000 20.000000 moveto +-4 0 rlineto +stroke +HE10 setfont +(0k) +dup stringwidth +2 div +20.000000 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 56.751299 moveto +-4 0 rlineto +stroke +HE10 setfont +(500k) +dup stringwidth +2 div +56.751299 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 93.502598 moveto +-4 0 rlineto +stroke +HE10 setfont +(1,000k) +dup stringwidth +2 div +93.502598 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 130.253897 moveto +-4 0 rlineto +stroke +HE10 setfont +(1,500k) +dup stringwidth +2 div +130.253897 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 167.005196 moveto +-4 0 rlineto +stroke +HE10 setfont +(2,000k) +dup stringwidth +2 div +167.005196 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 203.756494 moveto +-4 0 rlineto +stroke +HE10 setfont +(2,500k) +dup stringwidth +2 div +203.756494 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 240.507793 moveto +-4 0 rlineto +stroke +HE10 setfont +(3,000k) +dup stringwidth +2 div +240.507793 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 277.259092 moveto +-4 0 rlineto +stroke +HE10 setfont +(3,500k) +dup stringwidth +2 div +277.259092 exch sub +exch +40.000000 exch sub +exch +moveto +show +45.000000 314.010391 moveto +-4 0 rlineto +stroke +HE10 setfont +(4,000k) +dup stringwidth +2 div +314.010391 exch sub +exch +40.000000 exch sub +exch +moveto +show +481.338567 30.238095 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.000000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 32.238095 moveto +(OTHER) show +481.338567 47.476190 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.200000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 49.476190 moveto +((57)synth_2,addCCs_wrk) show +481.338567 64.714286 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.600000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 66.714286 moveto +((15)parse) show +481.338567 81.952381 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.300000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 83.952381 moveto +((95)SYSTEM,use_bb) show +481.338567 99.190476 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.900000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 101.190476 moveto +((164)useCCdescriptors) show +481.338567 116.428571 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.400000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 118.428571 moveto +((133)makeCCdescriptors) show +481.338567 133.666667 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +1.000000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 135.666667 moveto +((29)main) show +481.338567 150.904762 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.700000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 152.904762 moveto +((55)annotate_insn) show +481.338567 168.142857 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.500000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 170.142857 moveto +((111)synth_2,makeCCdescr) show +481.338567 185.380952 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.800000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 187.380952 moveto +((27)preparse) show +481.338567 202.619048 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.000000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 204.619048 moveto +((117)use_bb,synthLine) show +481.338567 219.857143 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.200000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 221.857143 moveto +((114)synth_2,makeCCdescr) show +481.338567 237.095238 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.600000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 239.095238 moveto +((59)addCCs_wrk,use_bb) show +481.338567 254.333333 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.300000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 256.333333 moveto +((52)synth_2,use_bb) show +481.338567 271.571429 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.900000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 273.571429 moveto +((112)synthLine) show +481.338567 288.809524 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.400000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 290.809524 moveto +((62)SYSTEM,synth_2) show +481.338567 306.047619 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +1.000000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 308.047619 moveto +((43)addCCs_wrk) show +481.338567 323.285714 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.700000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 325.285714 moveto +((48)use_bb) show +481.338567 340.523810 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.500000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 342.523810 moveto +((1)SYSTEM) show +481.338567 357.761905 moveto +0 14 rlineto +14 0 rlineto +0 -14 rlineto +closepath +gsave +0.800000 setgray +fill +grestore +stroke +HE10 setfont +500.338567 359.761905 moveto +((45)synth_2) show +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 20.000000 lineto +125.280680 20.000000 lineto +160.658606 20.000000 lineto +191.954465 20.000000 lineto +221.889633 20.000000 lineto +249.556986 20.000000 lineto +275.410086 20.000000 lineto +298.541808 20.000000 lineto +316.230771 20.000000 lineto +330.291229 20.000000 lineto +356.144329 20.000000 lineto +386.533061 20.000000 lineto +421.457425 20.000000 lineto +459.556730 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 20.618304 lineto +421.457425 28.827368 lineto +386.533061 20.618304 lineto +356.144329 20.618304 lineto +330.291229 20.618304 lineto +316.230771 28.577459 lineto +298.541808 20.618304 lineto +275.410086 20.618304 lineto +249.556986 20.618304 lineto +221.889633 20.618304 lineto +191.954465 20.618304 lineto +160.658606 20.618304 lineto +125.280680 20.618304 lineto +88.542064 20.618304 lineto +45.000000 20.000000 lineto +closepath +gsave +0.000000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 20.618304 lineto +125.280680 20.618304 lineto +160.658606 20.618304 lineto +191.954465 20.618304 lineto +221.889633 20.618304 lineto +249.556986 20.618304 lineto +275.410086 20.618304 lineto +298.541808 20.618304 lineto +316.230771 28.577459 lineto +330.291229 20.618304 lineto +356.144329 20.618304 lineto +386.533061 20.618304 lineto +421.457425 28.827368 lineto +459.556730 20.618304 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 20.618304 lineto +421.457425 28.827368 lineto +386.533061 20.618304 lineto +356.144329 20.654467 lineto +330.291229 21.323929 lineto +316.230771 28.577459 lineto +298.541808 21.736719 lineto +275.410086 21.736719 lineto +249.556986 21.736719 lineto +221.889633 21.736719 lineto +191.954465 21.736719 lineto +160.658606 21.700556 lineto +125.280680 21.571780 lineto +88.542064 20.955240 lineto +45.000000 20.000000 lineto +closepath +gsave +0.200000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 20.955240 lineto +125.280680 21.571780 lineto +160.658606 21.700556 lineto +191.954465 21.736719 lineto +221.889633 21.736719 lineto +249.556986 21.736719 lineto +275.410086 21.736719 lineto +298.541808 21.736719 lineto +316.230771 28.577459 lineto +330.291229 21.323929 lineto +356.144329 20.654467 lineto +386.533061 20.618304 lineto +421.457425 28.827368 lineto +459.556730 20.618304 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 20.618304 lineto +421.457425 28.827368 lineto +386.533061 20.618304 lineto +356.144329 20.654467 lineto +330.291229 21.323929 lineto +316.230771 28.577459 lineto +298.541808 23.214416 lineto +275.410086 23.213534 lineto +249.556986 23.213534 lineto +221.889633 23.074761 lineto +191.954465 23.074761 lineto +160.658606 21.700556 lineto +125.280680 22.833966 lineto +88.542064 21.955757 lineto +45.000000 20.000000 lineto +closepath +gsave +0.600000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 21.955757 lineto +125.280680 22.833966 lineto +160.658606 21.700556 lineto +191.954465 23.074761 lineto +221.889633 23.074761 lineto +249.556986 23.213534 lineto +275.410086 23.213534 lineto +298.541808 23.214416 lineto +316.230771 28.577459 lineto +330.291229 21.323929 lineto +356.144329 20.654467 lineto +386.533061 20.618304 lineto +421.457425 28.827368 lineto +459.556730 20.618304 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 20.618304 lineto +421.457425 28.827368 lineto +386.533061 20.618304 lineto +356.144329 20.654467 lineto +330.291229 21.323929 lineto +316.230771 38.031951 lineto +298.541808 23.214416 lineto +275.410086 23.213534 lineto +249.556986 23.213534 lineto +221.889633 23.074761 lineto +191.954465 23.074761 lineto +160.658606 21.700556 lineto +125.280680 22.833966 lineto +88.542064 21.955757 lineto +45.000000 20.000000 lineto +closepath +gsave +0.300000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 21.955757 lineto +125.280680 22.833966 lineto +160.658606 21.700556 lineto +191.954465 23.074761 lineto +221.889633 23.074761 lineto +249.556986 23.213534 lineto +275.410086 23.213534 lineto +298.541808 23.214416 lineto +316.230771 38.031951 lineto +330.291229 21.323929 lineto +356.144329 20.654467 lineto +386.533061 20.618304 lineto +421.457425 28.827368 lineto +459.556730 20.618304 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.040972 lineto +421.457425 28.827368 lineto +386.533061 20.618304 lineto +356.144329 20.654467 lineto +330.291229 21.323929 lineto +316.230771 38.031951 lineto +298.541808 23.214416 lineto +275.410086 23.213534 lineto +249.556986 23.213534 lineto +221.889633 23.074761 lineto +191.954465 23.074761 lineto +160.658606 21.700556 lineto +125.280680 22.833966 lineto +88.542064 21.955757 lineto +45.000000 20.000000 lineto +closepath +gsave +0.900000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 21.955757 lineto +125.280680 22.833966 lineto +160.658606 21.700556 lineto +191.954465 23.074761 lineto +221.889633 23.074761 lineto +249.556986 23.213534 lineto +275.410086 23.213534 lineto +298.541808 23.214416 lineto +316.230771 38.031951 lineto +330.291229 21.323929 lineto +356.144329 20.654467 lineto +386.533061 20.618304 lineto +421.457425 28.827368 lineto +459.556730 31.040972 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.040972 lineto +421.457425 41.485985 lineto +386.533061 20.620068 lineto +356.144329 20.656231 lineto +330.291229 21.325693 lineto +316.230771 38.031951 lineto +298.541808 23.214416 lineto +275.410086 23.213534 lineto +249.556986 23.213534 lineto +221.889633 23.074761 lineto +191.954465 23.074761 lineto +160.658606 21.700556 lineto +125.280680 22.833966 lineto +88.542064 21.955757 lineto +45.000000 20.000000 lineto +closepath +gsave +0.400000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 21.955757 lineto +125.280680 22.833966 lineto +160.658606 21.700556 lineto +191.954465 23.074761 lineto +221.889633 23.074761 lineto +249.556986 23.213534 lineto +275.410086 23.213534 lineto +298.541808 23.214416 lineto +316.230771 38.031951 lineto +330.291229 21.325693 lineto +356.144329 20.656231 lineto +386.533061 20.620068 lineto +421.457425 41.485985 lineto +459.556730 31.040972 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.124282 lineto +386.533061 21.258364 lineto +356.144329 21.294528 lineto +330.291229 21.963989 lineto +316.230771 39.278849 lineto +298.541808 24.455433 lineto +275.410086 24.454551 lineto +249.556986 24.454551 lineto +221.889633 24.315779 lineto +191.954465 24.315779 lineto +160.658606 22.941574 lineto +125.280680 24.074984 lineto +88.542064 23.196775 lineto +45.000000 20.000000 lineto +closepath +gsave +1.000000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 23.196775 lineto +125.280680 24.074984 lineto +160.658606 22.941574 lineto +191.954465 24.315779 lineto +221.889633 24.315779 lineto +249.556986 24.454551 lineto +275.410086 24.454551 lineto +298.541808 24.455433 lineto +316.230771 39.278849 lineto +330.291229 21.963989 lineto +356.144329 21.294528 lineto +386.533061 21.258364 lineto +421.457425 42.124282 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.124282 lineto +386.533061 22.023968 lineto +356.144329 22.922757 lineto +330.291229 23.872705 lineto +316.230771 41.524501 lineto +298.541808 26.559960 lineto +275.410086 26.340334 lineto +249.556986 26.038679 lineto +221.889633 25.642353 lineto +191.954465 25.245439 lineto +160.658606 23.546647 lineto +125.280680 24.593618 lineto +88.542064 23.493137 lineto +45.000000 20.000000 lineto +closepath +gsave +0.700000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 23.493137 lineto +125.280680 24.593618 lineto +160.658606 23.546647 lineto +191.954465 25.245439 lineto +221.889633 25.642353 lineto +249.556986 26.038679 lineto +275.410086 26.340334 lineto +298.541808 26.559960 lineto +316.230771 41.524501 lineto +330.291229 23.872705 lineto +356.144329 22.922757 lineto +386.533061 22.023968 lineto +421.457425 42.124282 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 31.368500 lineto +356.144329 28.080582 lineto +330.291229 25.593548 lineto +316.230771 41.524501 lineto +298.541808 26.559960 lineto +275.410086 26.340334 lineto +249.556986 26.038679 lineto +221.889633 25.642353 lineto +191.954465 25.245439 lineto +160.658606 23.546647 lineto +125.280680 24.593618 lineto +88.542064 23.493137 lineto +45.000000 20.000000 lineto +closepath +gsave +0.500000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 23.493137 lineto +125.280680 24.593618 lineto +160.658606 23.546647 lineto +191.954465 25.245439 lineto +221.889633 25.642353 lineto +249.556986 26.038679 lineto +275.410086 26.340334 lineto +298.541808 26.559960 lineto +316.230771 41.524501 lineto +330.291229 25.593548 lineto +356.144329 28.080582 lineto +386.533061 31.368500 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 31.368500 lineto +356.144329 28.080582 lineto +330.291229 25.593548 lineto +316.230771 41.524501 lineto +298.541808 32.677140 lineto +275.410086 32.867659 lineto +249.556986 27.036551 lineto +221.889633 29.209581 lineto +191.954465 32.384894 lineto +160.658606 24.587738 lineto +125.280680 24.605379 lineto +88.542064 29.373933 lineto +45.000000 20.000000 lineto +closepath +gsave +0.800000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 29.373933 lineto +125.280680 24.605379 lineto +160.658606 24.587738 lineto +191.954465 32.384894 lineto +221.889633 29.209581 lineto +249.556986 27.036551 lineto +275.410086 32.867659 lineto +298.541808 32.677140 lineto +316.230771 41.524501 lineto +330.291229 25.593548 lineto +356.144329 28.080582 lineto +386.533061 31.368500 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 35.123306 lineto +356.144329 36.264949 lineto +330.291229 35.930071 lineto +316.230771 53.392818 lineto +298.541808 32.677140 lineto +275.410086 32.867659 lineto +249.556986 27.036551 lineto +221.889633 29.209581 lineto +191.954465 32.384894 lineto +160.658606 24.587738 lineto +125.280680 24.605379 lineto +88.542064 29.373933 lineto +45.000000 20.000000 lineto +closepath +gsave +0.000000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 29.373933 lineto +125.280680 24.605379 lineto +160.658606 24.587738 lineto +191.954465 32.384894 lineto +221.889633 29.209581 lineto +249.556986 27.036551 lineto +275.410086 32.867659 lineto +298.541808 32.677140 lineto +316.230771 53.392818 lineto +330.291229 35.930071 lineto +356.144329 36.264949 lineto +386.533061 35.123306 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 39.445259 lineto +356.144329 44.770669 lineto +330.291229 47.872773 lineto +316.230771 65.212036 lineto +298.541808 32.677140 lineto +275.410086 32.867659 lineto +249.556986 27.036551 lineto +221.889633 29.209581 lineto +191.954465 32.384894 lineto +160.658606 24.587738 lineto +125.280680 24.605379 lineto +88.542064 29.373933 lineto +45.000000 20.000000 lineto +closepath +gsave +0.200000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 29.373933 lineto +125.280680 24.605379 lineto +160.658606 24.587738 lineto +191.954465 32.384894 lineto +221.889633 29.209581 lineto +249.556986 27.036551 lineto +275.410086 32.867659 lineto +298.541808 32.677140 lineto +316.230771 65.212036 lineto +330.291229 47.872773 lineto +356.144329 44.770669 lineto +386.533061 39.445259 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 39.445259 lineto +356.144329 44.770669 lineto +330.291229 47.872773 lineto +316.230771 65.212036 lineto +298.541808 45.475412 lineto +275.410086 44.560452 lineto +249.556986 37.761462 lineto +221.889633 37.820558 lineto +191.954465 38.949558 lineto +160.658606 29.876397 lineto +125.280680 28.824428 lineto +88.542064 32.072361 lineto +45.000000 20.000000 lineto +closepath +gsave +0.600000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 32.072361 lineto +125.280680 28.824428 lineto +160.658606 29.876397 lineto +191.954465 38.949558 lineto +221.889633 37.820558 lineto +249.556986 37.761462 lineto +275.410086 44.560452 lineto +298.541808 45.475412 lineto +316.230771 65.212036 lineto +330.291229 47.872773 lineto +356.144329 44.770669 lineto +386.533061 39.445259 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 41.769117 lineto +356.144329 48.957965 lineto +330.291229 55.205392 lineto +316.230771 65.212036 lineto +298.541808 55.455301 lineto +275.410086 53.812665 lineto +249.556986 46.138112 lineto +221.889633 45.465122 lineto +191.954465 45.850276 lineto +160.658606 35.922721 lineto +125.280680 33.236054 lineto +88.542064 34.512353 lineto +45.000000 20.000000 lineto +closepath +gsave +0.300000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 34.512353 lineto +125.280680 33.236054 lineto +160.658606 35.922721 lineto +191.954465 45.850276 lineto +221.889633 45.465122 lineto +249.556986 46.138112 lineto +275.410086 53.812665 lineto +298.541808 55.455301 lineto +316.230771 65.212036 lineto +330.291229 55.205392 lineto +356.144329 48.957965 lineto +386.533061 41.769117 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.278637 lineto +386.533061 57.202311 lineto +356.144329 78.538645 lineto +330.291229 96.945459 lineto +316.230771 105.991277 lineto +298.541808 55.455301 lineto +275.410086 53.812665 lineto +249.556986 46.138112 lineto +221.889633 45.465122 lineto +191.954465 45.850276 lineto +160.658606 35.922721 lineto +125.280680 33.236054 lineto +88.542064 34.512353 lineto +45.000000 20.000000 lineto +closepath +gsave +0.900000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 34.512353 lineto +125.280680 33.236054 lineto +160.658606 35.922721 lineto +191.954465 45.850276 lineto +221.889633 45.465122 lineto +249.556986 46.138112 lineto +275.410086 53.812665 lineto +298.541808 55.455301 lineto +316.230771 105.991277 lineto +330.291229 96.945459 lineto +356.144329 78.538645 lineto +386.533061 57.202311 lineto +421.457425 42.278637 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.279519 lineto +386.533061 57.202311 lineto +356.144329 78.538645 lineto +330.291229 96.945459 lineto +316.230771 125.536500 lineto +298.541808 83.504186 lineto +275.410086 79.552981 lineto +249.556986 69.332298 lineto +221.889633 65.722732 lineto +191.954465 63.020776 lineto +160.658606 49.723863 lineto +125.280680 43.050415 lineto +88.542064 40.162351 lineto +45.000000 20.000000 lineto +closepath +gsave +0.400000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 40.162351 lineto +125.280680 43.050415 lineto +160.658606 49.723863 lineto +191.954465 63.020776 lineto +221.889633 65.722732 lineto +249.556986 69.332298 lineto +275.410086 79.552981 lineto +298.541808 83.504186 lineto +316.230771 125.536500 lineto +330.291229 96.945459 lineto +356.144329 78.538645 lineto +386.533061 57.202311 lineto +421.457425 42.279519 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.353610 lineto +386.533061 58.459793 lineto +356.144329 79.759964 lineto +330.291229 97.497317 lineto +316.230771 125.675567 lineto +298.541808 138.035176 lineto +275.410086 128.812658 lineto +249.556986 111.126757 lineto +221.889633 101.413535 lineto +191.954465 91.637396 lineto +160.658606 70.610949 lineto +125.280680 55.169523 lineto +88.542064 45.801764 lineto +45.000000 20.000000 lineto +closepath +gsave +1.000000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 45.801764 lineto +125.280680 55.169523 lineto +160.658606 70.610949 lineto +191.954465 91.637396 lineto +221.889633 101.413535 lineto +249.556986 111.126757 lineto +275.410086 128.812658 lineto +298.541808 138.035176 lineto +316.230771 125.675567 lineto +330.291229 97.497317 lineto +356.144329 79.759964 lineto +386.533061 58.459793 lineto +421.457425 42.353610 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 31.679269 lineto +421.457425 42.353610 lineto +386.533061 70.083200 lineto +356.144329 103.293144 lineto +330.291229 132.934977 lineto +316.230771 172.460852 lineto +298.541808 182.646548 lineto +275.410086 169.982933 lineto +249.556986 148.411390 lineto +221.889633 133.790548 lineto +191.954465 118.811600 lineto +160.658606 93.317371 lineto +125.280680 71.339506 lineto +88.542064 55.072500 lineto +45.000000 20.000000 lineto +closepath +gsave +0.700000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 55.072500 lineto +125.280680 71.339506 lineto +160.658606 93.317371 lineto +191.954465 118.811600 lineto +221.889633 133.790548 lineto +249.556986 148.411390 lineto +275.410086 169.982933 lineto +298.541808 182.646548 lineto +316.230771 172.460852 lineto +330.291229 132.934977 lineto +356.144329 103.293144 lineto +386.533061 70.083200 lineto +421.457425 42.353610 lineto +459.556730 31.679269 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 51.559663 lineto +421.457425 63.454442 lineto +386.533061 89.964183 lineto +356.144329 123.181477 lineto +330.291229 152.819782 lineto +316.230771 341.486250 lineto +298.541808 226.567291 lineto +275.410086 210.440821 lineto +249.556986 185.053905 lineto +221.889633 166.048486 lineto +191.954465 146.420940 lineto +160.658606 117.332434 lineto +125.280680 90.502222 lineto +88.542064 65.399909 lineto +45.000000 20.000000 lineto +closepath +gsave +0.500000 setgray +fill +grestore +stroke +45.000000 20.000000 moveto +45.000000 20.000000 lineto +88.542064 65.399909 lineto +125.280680 90.502222 lineto +160.658606 117.332434 lineto +191.954465 146.420940 lineto +221.889633 166.048486 lineto +249.556986 185.053905 lineto +275.410086 210.440821 lineto +298.541808 226.567291 lineto +316.230771 341.486250 lineto +330.291229 152.819782 lineto +356.144329 123.181477 lineto +386.533061 89.964183 lineto +421.457425 63.454442 lineto +459.556730 51.559663 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +476.338567 20.000000 lineto +459.556730 51.559663 lineto +421.457425 63.528532 lineto +386.533061 140.979690 lineto +356.144329 216.546240 lineto +330.291229 287.298665 lineto +316.230771 348.276126 lineto +298.541808 382.000000 lineto +275.410086 353.331635 lineto +249.556986 315.505140 lineto +221.889633 281.057412 lineto +191.954465 246.046949 lineto +160.658606 196.150446 lineto +125.280680 147.445272 lineto +88.542064 98.116209 lineto +45.000000 20.000000 lineto +closepath +gsave +0.800000 setgray +fill +grestore +stroke +84.542064 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +121.280680 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +156.658606 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +187.954465 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +217.889633 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +245.556986 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +271.410086 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +294.541808 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +312.230771 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +326.291229 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +352.144329 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +382.533061 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +417.457425 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +455.556730 20.000000 moveto +4.000000 -4.000000 rlineto +4.000000 4.000000 rlineto +closepath +gsave +1.0 setgray +fill +grestore +stroke +showpage +%%Trailer +cleartomark +countdictstack exch sub { end } repeat +restore +%%EOF diff --git a/docs/storage-mgt/code.sty b/docs/storage-mgt/code.sty new file mode 100644 index 0000000000..f5ec2f59ee --- /dev/null +++ b/docs/storage-mgt/code.sty @@ -0,0 +1,83 @@ +
+% I have enclosed code.sty, which achieves 99% of what you want without
+% the need for a separate preprocessor. At the start of your document
+% you write "\makeatactive". From then on, inline code is written as @\x
+% -> x_1 & y@. The only difference with what you are used to, is that
+% instead of
+%
+% @
+% foo :: Int -> Int
+% foo = \n -> n+1
+% @
+%
+% you have to write
+%
+% \begin{code}
+% foo :: Int -> Int
+% foo = \n -> n+1
+% \end{code}
+%
+% and that you cannot use @ in \section{} and \caption{}. For the paper that occured twice, in which case I had to replace @...@ b y \texttt{...}.
+%
+%
+% code.sty --- nice verbatim mode for code
+
+\def\icode{%
+ \relax\ifmmode\hbox\else\leavevmode\null\fi
+ \bgroup
+ %\begingroup
+ \@noligs
+ \verbatim@font
+ \verb@eol@error
+ \let\do\@makeother \dospecials
+ \@vobeyspaces
+ \frenchspacing
+ \@icode}
+\def\@icode#1{%
+ \catcode`#1\active
+ \lccode`\~`#1%
+ \lowercase{\let~\icode@egroup}}
+\def\icode@egroup{%
+ %\endgroup}
+ \egroup}
+
+% The \makeatactive command:
+% makes @ active, in such a way that @...@ behaves as \icode@...@:
+{
+\catcode`@=\active
+\gdef\makeatactive{
+ \catcode`@=\active \def@{\icode@}
+ % Since @ becomes active, it has to be taken care of in verbatim-modes:
+ \let\olddospecials\dospecials \def\dospecials{\do\@\olddospecials}}
+}
+% \gdef\makeatother{\g@remfrom@specials{\@}\@makeother\@}
+\gdef\makeatother{\@makeother\@}
+
+\newcommand\codetabwidth{42pt}
+{\catcode`\^^I=\active%
+\gdef\@vobeytab{\catcode`\^^I\active\let^^I\@xobeytab}}
+\def\@xobeytab{\leavevmode\penalty10000\hskip\codetabwidth}
+
+\begingroup \catcode `|=0 \catcode `[= 1
+\catcode`]=2 \catcode `\{=12 \catcode `\}=12
+\catcode`\\=12 |gdef|@xcode#1\end{code}[#1|end[code]]
+|endgroup
+\def\@code{\trivlist \item\relax
+ \if@minipage\else\vskip\parskip\fi
+ \leftskip\@totalleftmargin\rightskip\z@skip
+ \parindent\z@\parfillskip\@flushglue\parskip\z@skip
+ \@@par
+ \@tempswafalse
+ \def\par{%
+ \if@tempswa
+ \leavevmode \null \@@par\penalty\interlinepenalty
+ \else
+ \@tempswatrue
+ \ifhmode\@@par\penalty\interlinepenalty\fi
+ \fi}%
+ \obeylines \verbatim@font \@noligs
+ \let\do\@makeother \dospecials
+ \everypar \expandafter{\the\everypar \unpenalty}%
+}
+\def\code{\@code \frenchspacing\@vobeytab\@vobeyspaces \@xcode}
+\def\endcode{\if@newlist \leavevmode\fi\endtrivlist}
diff --git a/docs/storage-mgt/freelist.eepic b/docs/storage-mgt/freelist.eepic new file mode 100644 index 0000000000..f87d939649 --- /dev/null +++ b/docs/storage-mgt/freelist.eepic @@ -0,0 +1,104 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(9912,7369)(0,-10) +\path(1125,6067)(2100,6067) +\path(1980.000,6037.000)(2100.000,6067.000)(1980.000,6097.000) +\path(5025,6367)(6000,6367)(6000,5167) + (5025,5167)(5025,6367) +\path(4650,6367)(5025,6367)(5025,5167) + (4650,5167)(4650,6367) +\path(3675,6367)(4650,6367)(4650,5167) + (3675,5167)(3675,6367) +\path(6600,6367)(7575,6367)(7575,5167) + (6600,5167)(6600,6367) +\path(8925,6367)(9900,6367)(9900,5167) + (8925,5167)(8925,6367) +\path(7575,6367)(8550,6367)(8550,5167) + (7575,5167)(7575,6367) +\path(8550,6367)(8925,6367)(8925,5167) + (8550,5167)(8550,6367) +\path(2100,6367)(3675,6367)(3675,5167) + (2100,5167)(2100,6367) +\path(2850,6217)(2850,6667)(6600,6667)(6600,6367) +\path(6570.000,6487.000)(6600.000,6367.000)(6630.000,6487.000) +\path(4425,6217)(4425,6967)(7575,6967)(7575,6367) +\path(7545.000,6487.000)(7575.000,6367.000)(7605.000,6487.000) +\path(5700,6217)(5700,7342)(8925,7342)(8925,6367) +\path(8895.000,6487.000)(8925.000,6367.000)(8955.000,6487.000) +\path(4350,5317)(4350,4792)(2100,4792)(2100,5167) +\path(2130.000,5047.000)(2100.000,5167.000)(2070.000,5047.000) +\path(5625,5317)(5625,4492)(2100,4492)(2100,5167) +\path(2130.000,5047.000)(2100.000,5167.000)(2070.000,5047.000) +\path(3000,5917)(3000,6667) +\path(5025,2842)(6000,2842)(6000,1642) + (5025,1642)(5025,2842) +\path(4650,2842)(5025,2842)(5025,1642) + (4650,1642)(4650,2842) +\path(3675,2842)(4650,2842)(4650,1642) + (3675,1642)(3675,2842) +\path(6600,2842)(7575,2842)(7575,1642) + (6600,1642)(6600,2842) +\path(8925,2842)(9900,2842)(9900,1642) + (8925,1642)(8925,2842) +\path(7575,2842)(8550,2842)(8550,1642) + (7575,1642)(7575,2842) +\path(8550,2842)(8925,2842)(8925,1642) + (8550,1642)(8550,2842) +\path(2100,2842)(3675,2842)(3675,1642) + (2100,1642)(2100,2842) +\path(2850,2692)(2850,3142)(6600,3142)(6600,2842) +\path(6570.000,2962.000)(6600.000,2842.000)(6630.000,2962.000) +\path(4425,2692)(4425,3442)(7575,3442)(7575,2842) +\path(7545.000,2962.000)(7575.000,2842.000)(7605.000,2962.000) +\path(5700,2692)(5700,3817)(8925,3817)(8925,2842) +\path(8895.000,2962.000)(8925.000,2842.000)(8955.000,2962.000) +\path(4350,1792)(4350,1267)(2100,1267)(2100,1642) +\path(2130.000,1522.000)(2100.000,1642.000)(2070.000,1522.000) +\path(5625,1792)(5625,967)(2100,967)(2100,1642) +\path(2130.000,1522.000)(2100.000,1642.000)(2070.000,1522.000) +\path(3000,2392)(3000,3142) +\path(2250,5317)(1650,5317)(1650,2542)(2100,2542) +\path(1980.000,2512.000)(2100.000,2542.000)(1980.000,2572.000) +\path(2250,1792)(1650,1792)(1650,142)(2325,142) +\path(2205.000,112.000)(2325.000,142.000)(2205.000,172.000) +\put(0,5992){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free\_list}}}}} +\put(8625,5917){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(4725,5767){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(3750,5242){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(5100,5242){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(2175,6142){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(2175,5842){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3750,6142){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(5100,6142){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(7800,6442){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(6825,6442){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(9150,6442){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(2175,5542){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_1$}}}}} +\put(3750,5842){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free=0}}}}} +\put(5100,5842){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free=0}}}}} +\put(8625,2392){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(4725,2242){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(3750,1717){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(5100,1717){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(2175,2617){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(2175,2317){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3750,2617){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(5100,2617){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(7800,2917){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(6825,2917){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(9150,2917){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(3750,2317){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free=0}}}}} +\put(5100,2317){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free=0}}}}} +\put(2325,5242){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(2325,1717){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(2475,67){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}next block group}}}}} +\put(2175,2017){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_2$}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/freelist.fig b/docs/storage-mgt/freelist.fig new file mode 100644 index 0000000000..d8debffd7c --- /dev/null +++ b/docs/storage-mgt/freelist.fig @@ -0,0 +1,116 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5325 1725 6300 1725 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9225 1425 10200 1425 10200 2625 9225 2625 9225 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8850 1425 9225 1425 9225 2625 8850 2625 8850 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 1425 8850 1425 8850 2625 7875 2625 7875 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 10800 1425 11775 1425 11775 2625 10800 2625 10800 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 13125 1425 14100 1425 14100 2625 13125 2625 13125 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 11775 1425 12750 1425 12750 2625 11775 2625 11775 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 12750 1425 13125 1425 13125 2625 12750 2625 12750 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 1425 7875 1425 7875 2625 6300 2625 6300 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 1575 7050 1125 10800 1125 10800 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 8625 1575 8625 825 11775 825 11775 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 9900 1575 9900 450 13125 450 13125 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 8550 2475 8550 3000 6300 3000 6300 2625 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 9825 2475 9825 3300 6300 3300 6300 2625 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 1875 7200 1125 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9225 4950 10200 4950 10200 6150 9225 6150 9225 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8850 4950 9225 4950 9225 6150 8850 6150 8850 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 4950 8850 4950 8850 6150 7875 6150 7875 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 10800 4950 11775 4950 11775 6150 10800 6150 10800 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 13125 4950 14100 4950 14100 6150 13125 6150 13125 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 11775 4950 12750 4950 12750 6150 11775 6150 11775 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 12750 4950 13125 4950 13125 6150 12750 6150 12750 4950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 4950 7875 4950 7875 6150 6300 6150 6300 4950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 5100 7050 4650 10800 4650 10800 4950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 8625 5100 8625 4350 11775 4350 11775 4950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 9900 5100 9900 3975 13125 3975 13125 4950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 8550 6000 8550 6525 6300 6525 6300 6150 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 9825 6000 9825 6825 6300 6825 6300 6150 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 5400 7200 4650 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 6450 2475 5850 2475 5850 5250 6300 5250 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 6450 6000 5850 6000 5850 7650 6525 7650 +4 0 0 50 0 0 17 0.0000 4 195 825 4200 1800 free_list\001 +4 0 0 50 0 0 17 0.0000 4 30 180 12825 1875 ...\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8925 2025 ...\001 +4 0 0 50 0 0 17 0.0000 4 165 390 7950 2550 link\001 +4 0 0 50 0 0 17 0.0000 4 165 390 9300 2550 link\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 1650 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 1950 free\001 +4 0 0 50 0 0 17 0.0000 4 150 435 7950 1650 start\001 +4 0 0 50 0 0 17 0.0000 4 150 435 9300 1650 start\001 +4 0 0 50 0 0 17 0.0000 4 165 540 12000 1350 block\001 +4 0 0 50 0 0 17 0.0000 4 165 540 11025 1350 block\001 +4 0 0 50 0 0 17 0.0000 4 165 540 13350 1350 block\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 2250 blocks=n_1\001 +4 0 0 50 0 0 17 0.0000 4 165 645 7950 1950 free=0\001 +4 0 0 50 0 0 17 0.0000 4 165 645 9300 1950 free=0\001 +4 0 0 50 0 0 17 0.0000 4 30 180 12825 5400 ...\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8925 5550 ...\001 +4 0 0 50 0 0 17 0.0000 4 165 390 7950 6075 link\001 +4 0 0 50 0 0 17 0.0000 4 165 390 9300 6075 link\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 5175 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 5475 free\001 +4 0 0 50 0 0 17 0.0000 4 150 435 7950 5175 start\001 +4 0 0 50 0 0 17 0.0000 4 150 435 9300 5175 start\001 +4 0 0 50 0 0 17 0.0000 4 165 540 12000 4875 block\001 +4 0 0 50 0 0 17 0.0000 4 165 540 11025 4875 block\001 +4 0 0 50 0 0 17 0.0000 4 165 540 13350 4875 block\001 +4 0 0 50 0 0 17 0.0000 4 165 645 7950 5475 free=0\001 +4 0 0 50 0 0 17 0.0000 4 165 645 9300 5475 free=0\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 2550 link\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 6075 link\001 +4 0 0 50 0 0 17 0.0000 4 225 1650 6675 7725 next block group\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 5775 blocks=n_2\001 diff --git a/docs/storage-mgt/gen.eepic b/docs/storage-mgt/gen.eepic new file mode 100644 index 0000000000..b50d691395 --- /dev/null +++ b/docs/storage-mgt/gen.eepic @@ -0,0 +1,57 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(9849,5907)(0,-10) +\path(3237,5562)(4212,5562)(4212,4062) + (3237,4062)(3237,5562) +\path(4212,5562)(5187,5562)(5187,4062) + (4212,4062)(4212,5562) +\path(5187,5562)(6162,5562)(6162,4062) + (5187,4062)(5187,5562) +\path(6162,5562)(7137,5562)(7137,4062) + (6162,4062)(6162,5562) +\put(5487,4737){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(4812,5712){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step[]}}}}} +\path(7812,2712)(9837,2712)(9837,2112) + (7812,2112)(7812,2712) +\put(7887,2862){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}END\_MUT\_LIST}}}}} +\path(6687,312)(7812,2412) +\path(7781.778,2292.056)(7812.000,2412.000)(7728.889,2320.389) +\path(6687,2412)(7812,2412) +\path(7692.000,2382.000)(7812.000,2412.000)(7692.000,2442.000) +\put(6012,312){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(6012,2412){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\path(4662,312)(5787,312) +\path(5667.000,282.000)(5787.000,312.000)(5667.000,342.000) +\path(3237,612)(5262,612)(5262,12) + (3237,12)(3237,612) +\path(4662,2412)(5787,2412) +\path(5667.000,2382.000)(5787.000,2412.000)(5667.000,2442.000) +\path(3237,2712)(5262,2712)(5262,2112) + (3237,2112)(3237,2712) +\put(3387,237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}mut\_link}}}}} +\put(3312,762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}StgMutClosure}}}}} +\put(3387,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}mut\_link}}}}} +\put(3312,2862){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}StgMutClosure}}}}} +\path(912,3012)(2487,3012)(2487,4887)(3237,4887) +\path(3117.000,4857.000)(3237.000,4887.000)(3117.000,4917.000) +\path(1212,2412)(3237,2412) +\path(3117.000,2382.000)(3237.000,2412.000)(3117.000,2442.000) +\path(1737,2112)(2487,2112)(2487,312)(3237,312) +\path(3117.000,282.000)(3237.000,312.000)(3117.000,342.000) +\path(12,3462)(1887,3462)(1887,1962) + (12,1962)(12,3462) +\put(87,3237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}no}}}}} +\put(237,3612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}generation}}}}} +\put(87,2937){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}steps}}}}} +\put(87,2637){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}n\_steps}}}}} +\put(87,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}mut\_list}}}}} +\put(87,2052){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}mut\_once\_list}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/gen.fig b/docs/storage-mgt/gen.fig new file mode 100644 index 0000000000..086a335819 --- /dev/null +++ b/docs/storage-mgt/gen.fig @@ -0,0 +1,71 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +6 5250 900 9150 2775 +6 5250 1275 9150 2775 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 5250 1275 6225 1275 6225 2775 5250 2775 5250 1275 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6225 1275 7200 1275 7200 2775 6225 2775 6225 1275 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7200 1275 8175 1275 8175 2775 7200 2775 7200 1275 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8175 1275 9150 1275 9150 2775 8175 2775 8175 1275 +4 0 0 50 0 0 17 0.0000 4 30 360 7500 2100 ......\001 +-6 +4 0 0 50 0 0 17 0.0000 4 225 540 6825 1125 step[]\001 +-6 +6 5250 3750 11850 6825 +6 8025 3750 11850 6525 +6 9825 3750 11850 4725 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9825 4125 11850 4125 11850 4725 9825 4725 9825 4125 +4 0 0 50 0 0 17 0.0000 4 195 1815 9900 3975 END_MUT_LIST\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 8700 6525 9825 4425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 8700 4425 9825 4425 +4 0 0 50 0 0 17 0.0000 4 30 360 8025 6525 ......\001 +4 0 0 50 0 0 17 0.0000 4 30 360 8025 4425 ......\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6675 6525 7800 6525 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 5250 6225 7275 6225 7275 6825 5250 6825 5250 6225 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6675 4425 7800 4425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 5250 4125 7275 4125 7275 4725 5250 4725 5250 4125 +4 0 0 50 0 0 17 0.0000 4 195 900 5400 6600 mut_link\001 +4 0 0 50 0 0 17 0.0000 4 225 1515 5325 6075 StgMutClosure\001 +4 0 0 50 0 0 17 0.0000 4 195 900 5400 4500 mut_link\001 +4 0 0 50 0 0 17 0.0000 4 225 1515 5325 3975 StgMutClosure\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 2925 3825 4500 3825 4500 1950 5250 1950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3225 4425 5250 4425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 3750 4725 4500 4725 4500 6525 5250 6525 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2025 3375 3900 3375 3900 4875 2025 4875 2025 3375 +4 0 0 50 0 0 17 0.0000 4 120 240 2100 3600 no\001 +4 0 0 50 0 0 17 0.0000 4 225 1035 2250 3225 generation\001 +4 0 0 50 0 0 17 0.0000 4 210 480 2100 3900 steps\001 +4 0 0 50 0 0 17 0.0000 4 210 720 2100 4200 n_steps\001 +4 0 0 50 0 0 17 0.0000 4 195 825 2100 4500 mut_list\001 +4 0 0 50 0 0 17 0.0000 4 195 1395 2100 4785 mut_once_list\001 diff --git a/docs/storage-mgt/generation.eepic b/docs/storage-mgt/generation.eepic new file mode 100644 index 0000000000..bea5a8c6ec --- /dev/null +++ b/docs/storage-mgt/generation.eepic @@ -0,0 +1,62 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(8153,4017)(0,-10) +\path(5025,3687)(6375,3687) +\path(6255.000,3657.000)(6375.000,3687.000)(6255.000,3717.000) +\path(2775,3687)(4125,3687) +\path(4005.000,3657.000)(4125.000,3687.000)(4005.000,3717.000) +\path(1875,3912)(2775,3912)(2775,3462) + (1875,3462)(1875,3912) +\path(4125,3912)(5025,3912)(5025,3462) + (4125,3462)(4125,3912) +\path(6375,3912)(7275,3912)(7275,3462) + (6375,3462)(6375,3912) +\path(5025,2187)(6375,2187) +\path(6255.000,2157.000)(6375.000,2187.000)(6255.000,2217.000) +\path(2775,2187)(4125,2187) +\path(4005.000,2157.000)(4125.000,2187.000)(4005.000,2217.000) +\path(4125,2412)(5025,2412)(5025,1962) + (4125,1962)(4125,2412) +\path(6375,2412)(7275,2412)(7275,1962) + (6375,1962)(6375,2412) +\path(1875,2412)(2775,2412)(2775,1962) + (1875,1962)(1875,2412) +\path(1875,912)(2775,912)(2775,462) + (1875,462)(1875,912) +\path(7275,3687)(8025,3687)(8025,3012) + (2325,3012)(2325,2412) +\path(2295.000,2532.000)(2325.000,2412.000)(2355.000,2532.000) +\path(7275,2187)(8025,2187)(8025,1512)(5025,1512) +\path(5145.000,1542.000)(5025.000,1512.000)(5145.000,1482.000) +\path(4125,1512)(2325,1512)(2325,912) +\path(2295.000,1032.000)(2325.000,912.000)(2355.000,1032.000) +\path(2895.000,717.000)(2775.000,687.000)(2895.000,657.000) +\path(2775,687)(3525,687)(3525,12) + (2325,12)(2325,462) +\put(5550,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3225,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}GC}}}}} +\put(1950,3612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step 0}}}}} +\put(4200,3612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step 1}}}}} +\put(6450,3612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step $S$}}}}} +\put(5550,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3225,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}GC}}}}} +\put(4200,2112){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step 1}}}}} +\put(6450,2112){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step $S$}}}}} +\put(1950,2112){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step 0}}}}} +\put(1950,612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step 0}}}}} +\put(7800,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}GC}}}}} +\put(3225,837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}GC}}}}} +\put(4500,1512){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(0,3612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}generation 0}}}}} +\put(0,2112){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}generation 1}}}}} +\put(0,612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}generation $G$}}}}} +\put(450,1512){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/generation.fig b/docs/storage-mgt/generation.fig new file mode 100644 index 0000000000..e91ed6d4c6 --- /dev/null +++ b/docs/storage-mgt/generation.fig @@ -0,0 +1,65 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9150 3150 10500 3150 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6900 3150 8250 3150 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6000 2925 6900 2925 6900 3375 6000 3375 6000 2925 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8250 2925 9150 2925 9150 3375 8250 3375 8250 2925 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 10500 2925 11400 2925 11400 3375 10500 3375 10500 2925 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 9150 4650 10500 4650 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6900 4650 8250 4650 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8250 4425 9150 4425 9150 4875 8250 4875 8250 4425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 10500 4425 11400 4425 11400 4875 10500 4875 10500 4425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6000 4425 6900 4425 6900 4875 6000 4875 6000 4425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6000 5925 6900 5925 6900 6375 6000 6375 6000 5925 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 5 + 0 0 1.00 60.00 120.00 + 11400 3150 12150 3150 12150 3825 6450 3825 6450 4425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 11400 4650 12150 4650 12150 5325 9150 5325 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 8250 5325 6450 5325 6450 5925 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 0 1 5 + 0 0 1.00 60.00 120.00 + 6900 6150 7650 6150 7650 6825 6450 6825 6450 6375 +4 0 0 50 0 0 12 0.0000 4 15 270 9675 3000 ......\001 +4 0 0 50 0 0 17 0.0000 4 165 345 7350 3000 GC\001 +4 0 0 50 0 0 17 0.0000 4 225 570 6075 3225 step 0\001 +4 0 0 50 0 0 17 0.0000 4 225 570 8325 3225 step 1\001 +4 0 0 50 0 0 17 0.0000 4 225 585 10575 3225 step S\001 +4 0 0 50 0 0 12 0.0000 4 15 270 9675 4500 ......\001 +4 0 0 50 0 0 17 0.0000 4 165 345 7350 4500 GC\001 +4 0 0 50 0 0 17 0.0000 4 225 570 8325 4725 step 1\001 +4 0 0 50 0 0 17 0.0000 4 225 585 10575 4725 step S\001 +4 0 0 50 0 0 17 0.0000 4 225 570 6075 4725 step 0\001 +4 0 0 50 0 0 17 0.0000 4 225 570 6075 6225 step 0\001 +4 0 0 50 0 0 17 0.0000 4 165 345 11925 3000 GC\001 +4 0 0 50 0 0 17 0.0000 4 165 345 7350 6000 GC\001 +4 0 0 50 0 0 12 0.0000 4 15 270 8625 5325 ......\001 +4 0 0 50 0 0 17 0.0000 4 225 1215 4125 3225 generation 0\001 +4 0 0 50 0 0 17 0.0000 4 225 1215 4125 4725 generation 1\001 +4 0 0 50 0 0 17 0.0000 4 225 1275 4125 6225 generation G\001 +4 0 0 50 0 0 12 0.0000 4 15 270 4575 5325 ......\001 diff --git a/docs/storage-mgt/largeobjectpool.eepic b/docs/storage-mgt/largeobjectpool.eepic new file mode 100644 index 0000000000..9c198fd279 --- /dev/null +++ b/docs/storage-mgt/largeobjectpool.eepic @@ -0,0 +1,70 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(10212,4689)(0,-10) +\path(6900,4362)(10200,4362)(10200,3162) + (6900,3162)(6900,4362) +\path(7020.000,3792.000)(6900.000,3762.000)(7020.000,3732.000) +\path(6900,3762)(10050,3762) +\path(9930.000,3732.000)(10050.000,3762.000)(9930.000,3792.000) +\path(10050,4362)(10050,3162) +\put(8100,4437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\put(7875,3912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single object}}}}} +\path(6900,2262)(10200,2262)(10200,1062) + (6900,1062)(6900,2262) +\path(7020.000,1692.000)(6900.000,1662.000)(7020.000,1632.000) +\path(6900,1662)(10050,1662) +\path(9930.000,1632.000)(10050.000,1662.000)(9930.000,1692.000) +\path(10050,2262)(10050,1062) +\put(8100,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\put(7875,1812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single object}}}}} +\path(2550,4062)(3375,4062) +\path(3255.000,4032.000)(3375.000,4062.000)(3255.000,4092.000) +\path(3405.000,1182.000)(3525.000,1212.000)(3405.000,1242.000) +\path(3525,1212)(2925,1212)(2925,12)(3375,12) +\path(3255.000,-18.000)(3375.000,12.000)(3255.000,42.000) +\path(3405.000,3282.000)(3525.000,3312.000)(3405.000,3342.000) +\path(3525,3312)(2925,3312)(2925,2112)(3375,2112) +\path(3255.000,2082.000)(3375.000,2112.000)(3255.000,2142.000) +\path(3375,4362)(4950,4362)(4950,3162) + (3375,3162)(3375,4362) +\path(4275,3912)(4275,4662) +\path(4950,4362)(5400,4362)(5400,3162) + (4950,3162)(4950,4362) +\path(5400,4362)(5850,4362)(5850,3162) + (5400,3162)(5400,4362) +\path(5850,4362)(6300,4362)(6300,3162) + (5850,3162)(5850,4362) +\path(3375,2262)(4950,2262)(4950,1062) + (3375,1062)(3375,2262) +\path(4125,2112)(4125,2562)(6900,2562)(6900,2262) +\path(6870.000,2382.000)(6900.000,2262.000)(6930.000,2382.000) +\path(4275,1812)(4275,2562) +\path(4950,2262)(5400,2262)(5400,1062) + (4950,1062)(4950,2262) +\path(5400,2262)(5850,2262)(5850,1062) + (5400,1062)(5400,2262) +\path(5850,2262)(6300,2262)(6300,1062) + (5850,1062)(5850,2262) +\path(4125,4212)(4125,4662)(6900,4662)(6900,4362) +\path(6870.000,4482.000)(6900.000,4362.000)(6930.000,4482.000) +\put(3600,12){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3450,4137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3450,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3450,3537){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_1$}}}}} +\put(3600,3237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(5550,3762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(3450,2037){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3450,1737){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3600,1137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(5550,1662){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(3450,1437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_2$}}}}} +\put(0,3987){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}g0s0-$>$large\_objects}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/largeobjectpool.fig b/docs/storage-mgt/largeobjectpool.fig new file mode 100644 index 0000000000..6c49ff03f1 --- /dev/null +++ b/docs/storage-mgt/largeobjectpool.fig @@ -0,0 +1,82 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +6 9825 1125 13125 2625 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9825 1425 13125 1425 13125 2625 9825 2625 9825 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9825 2025 12975 2025 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12975 1425 12975 2625 +4 0 0 50 0 0 17 0.0000 4 165 630 11025 1350 blocks\001 +4 0 0 50 0 0 17 0.0000 4 225 1230 10800 1875 single object\001 +-6 +6 9825 3225 13125 4725 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9825 3525 13125 3525 13125 4725 9825 4725 9825 3525 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9825 4125 12975 4125 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12975 3525 12975 4725 +4 0 0 50 0 0 17 0.0000 4 165 630 11025 3450 blocks\001 +4 0 0 50 0 0 17 0.0000 4 225 1230 10800 3975 single object\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5475 1725 6300 1725 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 4 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 6450 4575 5850 4575 5850 5775 6300 5775 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 4 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 6450 2475 5850 2475 5850 3675 6300 3675 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 1425 7875 1425 7875 2625 6300 2625 6300 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 1875 7200 1125 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 1425 8325 1425 8325 2625 7875 2625 7875 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8325 1425 8775 1425 8775 2625 8325 2625 8325 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8775 1425 9225 1425 9225 2625 8775 2625 8775 1425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 3525 7875 3525 7875 4725 6300 4725 6300 3525 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 3675 7050 3225 9825 3225 9825 3525 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 3975 7200 3225 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 3525 8325 3525 8325 4725 7875 4725 7875 3525 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8325 3525 8775 3525 8775 4725 8325 4725 8325 3525 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8775 3525 9225 3525 9225 4725 8775 4725 8775 3525 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 1575 7050 1125 9825 1125 9825 1425 +4 0 0 50 0 0 17 0.0000 4 30 360 6525 5775 ......\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 1650 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 1950 free\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 2250 blocks=n_1\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 2550 link\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8475 2025 ...\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 3750 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 4050 free\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 4650 link\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8475 4125 ...\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 4350 blocks=n_2\001 +4 0 0 50 0 0 17 0.0000 4 225 2010 2925 1800 g0s0->large_objects\001 diff --git a/docs/storage-mgt/ldv.eepic b/docs/storage-mgt/ldv.eepic new file mode 100644 index 0000000000..aa41327aa5 --- /dev/null +++ b/docs/storage-mgt/ldv.eepic @@ -0,0 +1,41 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(6036,3169)(0,-10) +\path(1692,3142)(1692,2692)(3342,2692) +\path(1692,2317)(1692,2692) +\path(1722.000,2572.000)(1692.000,2692.000)(1662.000,2572.000) +\path(4992,2317)(4992,2692) +\path(5022.000,2572.000)(4992.000,2692.000)(4962.000,2572.000) +\path(4992,2692)(4992,3142) +\path(3342,3142)(3342,2692)(4992,2692) +\path(3342,2317)(3342,2692) +\path(3372.000,2572.000)(3342.000,2692.000)(3312.000,2572.000) +\path(42,3142)(42,2692)(1692,2692) +\path(42,2317)(42,2692) +\path(72.000,2572.000)(42.000,2692.000)(12.000,2572.000) +\put(1992,2767){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}use}}}}} +\put(342,2767){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}lag}}}}} +\put(117,2092){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}created}}}}} +\put(3642,2767){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}drag}}}}} +\put(1767,2092){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}first used}}}}} +\put(3417,2092){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}last used}}}}} +\put(5067,2092){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}destroyed}}}}} +\path(4992,292)(4992,667) +\path(5022.000,547.000)(4992.000,667.000)(4962.000,547.000) +\path(4992,667)(4992,1117) +\path(1692,667)(3342,667)(4992,667) +\path(42,1117)(42,667)(1692,667) +\path(42,292)(42,667) +\path(72.000,547.000)(42.000,667.000)(12.000,547.000) +\put(117,67){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}created}}}}} +\put(5067,67){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}destroyed}}}}} +\put(1992,742){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}void}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/ldv.fig b/docs/storage-mgt/ldv.fig new file mode 100644 index 0000000000..772411c289 --- /dev/null +++ b/docs/storage-mgt/ldv.fig @@ -0,0 +1,53 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +6 3600 3375 9675 4500 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 3 + 5325 3375 5325 3825 6975 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 5325 4200 5325 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 8625 4200 8625 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 2 + 8625 3825 8625 3375 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 3 + 6975 3375 6975 3825 8625 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 6975 4200 6975 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 3 + 3675 3375 3675 3825 5325 3825 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3675 4200 3675 3825 +4 0 0 50 0 0 17 0.0000 4 120 315 5625 3750 use\001 +4 0 0 50 0 0 17 0.0000 4 225 300 3975 3750 lag\001 +4 0 0 50 0 0 17 0.0000 4 165 705 3750 4425 created\001 +4 0 0 50 0 0 17 0.0000 4 225 435 7275 3750 drag\001 +4 0 0 50 0 0 17 0.0000 4 165 915 5400 4425 first used\001 +4 0 0 50 0 0 17 0.0000 4 165 840 7050 4425 last used\001 +4 0 0 50 0 0 17 0.0000 4 225 945 8700 4425 destroyed\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 8625 6225 8625 5850 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 2 + 8625 5850 8625 5400 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 3 + 5325 5850 6975 5850 8625 5850 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 3 + 3675 5400 3675 5850 5325 5850 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 3675 6225 3675 5850 +4 0 0 50 0 0 17 0.0000 4 165 705 3750 6450 created\001 +4 0 0 50 0 0 17 0.0000 4 225 945 8700 6450 destroyed\001 +4 0 0 50 0 0 17 0.0000 4 165 435 5625 5775 void\001 diff --git a/docs/storage-mgt/ldv.tex b/docs/storage-mgt/ldv.tex new file mode 100644 index 0000000000..936407c701 --- /dev/null +++ b/docs/storage-mgt/ldv.tex @@ -0,0 +1,695 @@ +\documentclass{article} +\usepackage{code,a4wide} + +\usepackage{graphics,epsfig,epic,eepic,epsfig} + +\setlength{\parskip}{0.25cm} +\setlength{\parsep}{0.25cm} +\setlength{\topsep}{0cm} +\setlength{\parindent}{0cm} +\renewcommand{\textfraction}{0.2} +\renewcommand{\floatpagefraction}{0.7} + + +% Terminology +\newcommand{\block}{block} +\newcommand{\Block}{Block} +\newcommand{\segment}{segment} +\newcommand{\Segment}{Segment} +\newcommand{\step}{step} +\newcommand{\Step}{Step} + +\newcommand{\note}[1]{{\em $\spadesuit$ #1}} + +\begin{document} +\title{Implementation of Lag/Drag/Void/Use Profiling} +\author{Sungwoo Park \\ Simon Marlow} + +\makeatactive +\maketitle + +\section{Lag/Drag/Void/Use Profiling} + +\emph{Lag/Drag/Void/Use} (LDVU) profiling~\cite{RR} is a profiling technique +which yields a summary of the biography of all the dynamic closures created +during program execution. +In this profiling scheme, +the biography of a closure is determined by four important events associated +with the closure: \emph{creation}, \emph{first use}, +\emph{last use}, and \emph{destruction} (see Figure~\ref{fig-ldv}). +The intervals between these successive events correspond to three phases +for the closure: \emph{lag} (between creation and first use), +\emph{use} (between first use and last use), and +\emph{drag} (between last use and destruction). +If the closure is never used, it is considered to remain in the \emph{void} +phase all its lifetime. + +\begin{figure}[ht] +\begin{center} +\input{ldv.eepic} +\caption{The biography of a closure} +\label{fig-ldv} +\end{center} +\end{figure} + +The LDVU profiler regularly performs heap censuses during program execution. +Each time a heap census is performed, the LDVU profiler increments a global +time, which is used for timing all the events (such as creation and destruction +of a closure) occurring during program execution. +Hence, for instance, all closures creating between two successive heap censuses +have the same creation time and belong to the same \emph{generation}.\footnote{In +this document, a generation is related with heap censuses, not garbage collections +as in other profiling schemes.} +After the program terminates, it yields a post-mortem report on how much +of the \emph{live} graph is in one of the four phases at the moment of each +heap census. + +It must be emphasized that the LDVU profiler considers only live closures; +it should not take into consideration dead closures which do not constitute +the graph. Therefore, the result of LDVU profiling does not depend on the +frequency of garbage collections. + +This document describes the implementation of LDVU profiling on the Glasgow +Haskell Compiler runtime system.\footnote{Unless otherwise noted, all identifiers +are defined in @LdvProfile.c@}. + +\section{An Overview of the Implementation} + +Every closure is augmented with an additional word in its profiling header +to accommodate three additional pieces of information: +1) state flag indicating whether the closure has been used at least once or not. +2) creation time; 3) time of most recent use if any so far. +We refer to such a word as an LDV word. + +The LDVU profiler maintains a global time, stored in @ldvTime@. +It is incremented each time a heap census is performed. +During a heap census, the profiler scans all live closures and computes the +following: +1) the total size of all closures which have never been used; +2) the total size of all closures which have been used at least once +in the past.\footnote{There is another category of closures, namely, +\emph{inherently used} closures. We will explain +in Section~\ref{sec-heap-censuses}.} +It is not until the whole program execution finishes that the profiler +can actually decide the total size corresponding to each of the four phases for +a particular heap census. It is only when a closure is destroyed that the profiler +can determine how long the closure has been in a specific phase. +Therefore, it is not sufficient to perform heap censuses periodically in order to +compute the profiling statistics: the runtime system needs to intercept +all events associated with any closures and update necessary information. + +All events associated with closures are handled by one of the three +macros defined +in @includes/StgLdv.h@: @LDV_recordCreate()@, @LDV_recordUse()@, and +@LDV_recordDead()@. + +\begin{itemize} +\item{@LDV_recordCreate()@} is called when a closure is created and updates its +creation time field. + +\item{@LDV_recordUse()@} is called when a closure is used and updates its most recent +use time field. + +\item{@LDV_recordDead()@} is called when a closure @c@ is removed from the graph. +It does not update its LDV word (because @c@ is about to be destroyed). +Instead, it updates the statistics on LDVU profiling according to the following +observation: +if @c@ has never been used (which is indicated by the state flag in its LDV +word), +@c@ contributes to the void phase from its creation time to the last census +time; if @c@ was used at least once (which is also indicated by the state flag), +@c@ contributes to the @drag@ phase after its last use time. +\end{itemize} + +At the end of the program execution, the profiler performs a last census during +which all closures in the heap are declared to be dead and @LDV_recordDead()@ +is invoked on each of them. +Then, the profiler computes the final statistics. + +\section{LDV Words} + +We choose to share the LDV word for both retainer profiling and LDVU +profiling, which cannot take place simultaneously. +This is the reason why there is a +union structure inside the @StgProHeader@ structure. +The field @hp.ldvw@ in the @StgProfHeader@ structure corresponds to the LDV +word: +\begin{code} +typedef struct { + ... + union { + retainerSet *rs; // Retainer Set + StgWord ldvw; // Lag/Drag/Void Word + } hp; +} StgProfHeader; +\end{code} +For instance, the LDV word of a closure @c@ can now be accessed with +@c->header.prof.hp.ldvw@ (or by @LDVW(c)@ where @LDVW()@ is a macro in +@includes/StgLdvProf.h@). + +An LDV word is divided into three fields, whose position is specified +by three constants in @includes/StgLdvProf.h@: +\begin{itemize} +\item{@LDV_STATE_MASK@} corresponds to the state flag. +\item{@LDV_CREATE_MASK@} corresponds to the creation time. +\item{@LDV_LAST_MASK@} corresponds to the most recent use time. +\end{itemize} +The constant @LDV_SHIFT@ specifies how many bits are allocated for +creation time or most recent use time. +For instance, the creation time of a closure @c@ can be obtained by +@(LDVW(c) & LDV_CREATE_MASK) >> LDV_SHIFT@. + +The creation time field and the most recent use time field can be set only by the +macros @LDV_recordCreate()@ and @LDV_recordUse()@. +@LDV_recordCreate()@ must be called whenever a new dynamic closure is created, +and this is handily accomplished by rewriting the macro @SET_PROF_HDR()@ +(in @includes/ClosureMacros.h@) (we do not need to change @SET_STATIC_PROF_HDR()@ +because static closures are not involved in LDVU profiling at all): + +\begin{code} +#define SET_PROF_HDR(c,ccs_) \ + ((c)->header.prof.ccs = ccs_, \ + LDV_recordCreate((c))) +\end{code} + +There are a few cases in which the info table of a closure changes through +an explicit invocation of @SET_INFO()@ or a direct assignment to its @header.info@ +field: 1) an indirection closure is replaced by an old-generation +indirection closure; 2) a thunk is replaced by a blackhole; 3) a thunk is replaced +by an indirection closure when its evaluation result becomes available. + +\emph{We regard such a situation as +the destruction of an old closure followed by the creation of a new closure +at the same memory address.}\footnote{This would be unnecessary if the two closures +are of the same size, but it is not always the case. We choose to distinguish +the two closures for the sake of consistency.} +For instance, when an @IND_PERM@ closure is replaced by an @IND_OLDGEN_PERM@ +closures (during scavenging in @GC.c@), we wrap the invocation of @SET_INFO()@ with +the invocations of @LDV_recordDead()@ and @LDV_recordCreate()@ as follows +(@LDV_recordDead()@ requires the actual size of the closures being destroyed): + +\begin{code} + LDV_recordDead((StgClosure *)p, sizeofW(StgInd) - sizeofW(StgProfHeader)); + SET_INFO(((StgClosure *)p), &stg_IND_OLDGEN_PERM_info); + LDV_recordCreate((StgClosure *)p); +\end{code} + +\textbf{To do:} +A direct assignment to the @header.info@ field implies that its cost centre +field is not initialized. This is no problem in the case of @EVACUATED@ closures +because they will +not be used again after a garbage collection. However, I am not sure if this is safe +for @BLACKHOLE_BQ@ closures (in @StgMiscClosures.hc@) when retainer profiling, +which employs cost centre stacks, is going on. +If it is safe, please leave a comment there. + +@LDV_recordUse()@ is called on a closure whenever it is used, or \emph{entered}. +Its state flag changes if necessary to indicate that it has been used, and +the current global time is stored in its last use time field. + +\section{Global Time \texttt{ldvTime} and Retainer Profiling} + +The global time, stored in @ldvTime@, records the current time period. +It is initialized to $1$ and incremented after each time a heap census +is completed through an invocation of @LdvCensus()@. Note that each +value of @ldvTime@ represents a time \emph{period}, not a point in +time. + +All closures created between two successive invocations of +@LdvCensus()@ have the same creation time. If a closure is used at +least once between two successive heap censuses, we consider the +closure to be in the use phase during the corresponding time period +(because we just set its last use time field to the current value of +@ldvTime@ whenever it is used). Notice that a closure with a creation +time $t_c$ may be destroyed before the actual heap census for time +$t_c$ and thus may \emph{not} be observed during the heap census for +time $t_c$. Such a closure does not show up in the profile at all. + +In addition, the value of @ldvTime@ indicates which of LDVU profiling +and retainer profiling is currently active: during LDVU profiling, it +is initialized to $1$ in @initLdvProfiling()@ and then increments as +LDVU profiling proceeds; during retainer profiling, however, it is +always fixed to $0$. Thus, wherever a piece of code shared by both +retainer profiling and LDVU profiling comes to play, we usually need +to first examine the value of @ldvTime@ if necessary. For instance, +consider the macro @LDV_recordUse()@: + +\begin{code} +#define LDV_recordUse(c) \ + if (ldvTime > 0) \ + LDVW((c)) = (LDVW((c)) & LDV_CREATE_MASK) | ldvTime | LDV_STATE_USE; +\end{code} + +If retainer profiling is being performed, @ldvTime@ is equal to $0$, +and @LDV_recordUse()@ causes no side effect.\footnote{Due to this +interference with LDVU profiling, retainer profiling slows down a bit; +for instance, checking @ldvTime@ against $0$ in the above example +would always evaluate to @rtsFalse@ during retainer profiling. +However, this is the price to be paid for our decision not to employ a +separate field for LDVU profiling.} + +As another example, consider @LDV_recordCreate()@: + +\begin{code} +#define LDV_recordCreate(c) \ + LDVW((c)) = (ldvTime << LDV_SHIFT) | LDV_STATE_CREATE +\end{code} + +The above definition of @LDV_recordCreate()@ works without any problem +even for retainer profiling: during retainer profiling, +a retainer set field (@hp.ldvw@) must be initialized to a null pointer. +Since @ldvTime@ is fixed to $0$, @LDV_recordCreate()@ initializes +retainer set fields correctly. + +\section{Heap Censuses} +\label{sec-heap-censuses} + +The LDVU profiler performs heap censuses periodically by invoking the +function @LdvCensus()@. Because we need to know exactly which +closures in the heap are live at census time, we always precede the +census with a major garbage collection. + +During a census, we examine each closure one by one and compute the +following three quantities: + +\begin{enumerate} +\item the total size of all \emph{inherently used} closures. +\item the total size of all closures which have not been used (yet). +\item the total size of all closures which have been used at least once. +\end{enumerate} + +For most closures, a \emph{use} consists of entering the closure. For +unlifted objects which are never entered (eg. @ARR_WORDS@), it would +be difficult to determine their points of use because such points are +scattered around the implementation in various primitive operations. +For this reason we consider all unlifted objects as ``inherently +used''. The following types of closures are considered to be +inherently used: @TSO@, @MVAR@, @MUT_ARR_PTRS@, @MUT_ARR_PTRS_FROZEN@, +@ARR_WORDS@, @WEAK@, @MUT_VAR@, @MUT_CONS@, @FOREIGN@, @BCO@, and +@STABLE_NAME@. + +The three quantities are stored in an @LdvGenInfo@ array @gi[]@. +@gi[]@ is indexed by time period. For instance, @gi[ldvTime]@ stores +the three quantaties for the current global time period. The +structure @LdvGenInfo@ is defined as follows: + +\begin{code} +typedef struct { + ... + int inherentlyUsed; // total size of 'inherently used' closures + int notUsed; // total size of 'not used yet' closures + int used; // total size of 'used at least once' closures + ... +} LdvGenInfo; +\end{code} + +The above three quantities account for mutually exclusive sets of closures. +In other words, if a closure is not inherently used, it belongs to +either the second or the third. + +\subsection{Taking a Census of the Live Heap} + +During a heap census, we need to visit every live closure once, so we +perform a linear scan of the live heap after a major GC. We can take +advantage of the following facts to implement a linear scan for heap +censuses: + +\begin{itemize} +\item The nursery is empty. The small object pool and the large object pool, + however, may \emph{not} be empty. This is because the garbage collector + invokes @scheduleFinalizer()@ after removing dead closures, and + @scheduleFinalizer()@ may create new closures through @allocate()@. +\item @IND@, @IND_OLDGEN@, and @EVACUATED@ closures do not appear in +the live heap. +\end{itemize} + +There is one small complication when traversing the live heap: the +garbage collector may have replaced @WEAK@ objects with @DEAD_WEAK@ +objects, which have a smaller size and hence leave some space before +the next object. To avoid this problem we change the size of +@DEAD_WEAK@ objects to match that of @WEAK@ objects when profiling is +enabled (see @StgMiscClosures.hc@). + +\section{Destruction of Closures} + +In order to compute the total size of closures for each of the four +phases, we must report the destruction of every closure (except +inherently used closures) to the LDVU profiler by invoking +@LDV_recordDead()@. @LDV_recordDead()@ must not be called on any +inherently used closure because any invocation of @LDV_recordDead()@ +affects the statistics regarding void and drag phases, which no +inherently used closure can be in. + +@LDV_recordDead()@ updates two fields @voidNew@ and @dragNew@ in the +@LdvGenInfo@ array @gi[]@: + +\begin{code} +typedef struct { + ... + int voidNew; + int dragnew; + ... +} LdvGenInfo; +\end{code} + +@gi[ldvTime].voidNew@ accumulates the size of all closures satisfying +the following two conditions: 1) observed during the heap census at +time @ldvTime@; 2) now known to have been in the void phase at time +@ldvTime@. It is updated when a closure which has never been used is +destroyed. Suppose that a closure @c@ which has never been used is +about to be destroyed. If its creation time is $t_c$, we judge that +@c@ has been in the void phase all its lifetime, namely, from time +$t_c$ to @ldvTime@. Since @c@ will not be observed during the next +heap census, which corresponds to time @ldvTime@, @c@ contributes to +the void phase of times $t_c$ through @ldvTime@ - 1. Therefore, we +increase the @voidNew@ field of @gi[@$t_c$@]@ through @gi[ldvTime - 1]@ + by the size of @c@.\footnote{In the actual implementation, we +update @gi[$t_c$]@ and @gi[ldvTime]@ (not @gi[ldvTime@$ - $1@]@) only: +@gi[$t_c$]@ and @gi[ldvTime]@ are increased and decreased by the size +of @c@, respectively. After finishing the program execution, we can +correctly adjust all the fields as follows: @gi[$t_c$]@ is computed as +$\sum_{i=0}^{t_c}$@gi[$i$]@. } + +@gi[ldvTime].dragNew@ accumulates the size of all closures satisfying the following +two conditions: 1) observed during the heap census at time @ldvTime@; +2) now known to have been in the drag phase at time @ldvTime@. +It is updated when a closure which has been used at least once is destroyed. +Suppose that a closure @c@ which has been used last at time $t_l$ is about to +be destroyed. +We judge that @c@ has been in the drag phase from time $t_l + 1$ to +time @ldvTime@$ - 1$ (if $t_l + 1 > $@ldvTime@$ - 1$, nothing happens). +Therefore, we increase the @dragNew@ field of @gi[@$t_l + 1$@]@ through +@gi[ldvTime@$ - 1$@]@ +by the size of @c@.\footnote{As in the case of @voidNew@, we update +@gi[@$t_l + 1$@]@ and @gi[ldvTime]@ only.} + +Now we need to find out all the cases of closure destruction. +There are four cases in which a closure is destroyed: + +\begin{enumerate} +\item A closure is overwritten with a blackhole: + @UPD_BH_UPDATABLE()@ and @UPD_BH_SINGLE_ENTRY()@ in @includes/StgMacros.h@, + @threadLazyBlackHole()@ and @threadSqueezeStack()@ in @GC.c@, + the entry code for @BLACKHOLE@ closures in @StgMiscClosures.hc@ (a + @BLACKHOLE@ closure is changed into a @BLACKHOLE_BQ@ closure). + We call either @LDV_recordDead()@ or @LDV_recordDead_FILL_SLOP_DYNAMIC()@. + +\item A weak pointer is overwritten with a dead weak pointer: + @finalizzeWeakzh_fast()@ in @PrimOps.hc@, + @finalizeWeakPointersNow()@ and @scheduleFinalizers()@ in @Weak.c@. + Since a weak pointer is inherently used, we do not call @LDV_recordDead()@. + +\item A closure is overwritten with an indirection closure: + @updateWithIndirection()@ and @updateWithPermIndirection()@ in @Storage.h@, + @scavenge()@ in @GC.c@, in which an @IND_PERM@ closure is explicitly replaced + with an @IND_OLDGEN_PERM@ closure during scavenging. + We call either @LDV_recordDead()@ or @LDV_recordDead_FILL_SLOP_DYNAMIC()@. + +\item Closures are removed permanently from the graph during garbage +collections. We locate and dispose of all dead closures by linearly +scanning the from-space right before tidying up. This is feasible +because any closures which is about to be removed from the graph still +remains in the from-space until tidying up is completed. The next +subsection explains how to implement this idea. +\end{enumerate} + +\subsection{Linear scan of the from-space during garbage collections} + +In order to implement linear scan of the from-space during a garbage collection +(before tidying up), +we need to take into consideration the following facts: + +\begin{itemize} +\item The pointer @free@ of a block in the nursery may incorrectly point to +a byte past its actual boundary. +This happens because +the Haskell mutator first increases @hpLim@ without comparing it with the +actual boundary when allocating fresh memory for a new closure. +@hpLim@ is later assigned to the pointer @free@ of the corresponding memory +block, which means that during a heap census, the pointer @hpLim@ may not +be trusted. +Notice that @hpLim@ is not available during LDVU profiling; it is valid +only during the Haskell mutator time. + +\item The from-space may well contain a good number of @EVACUATED@ closures, +and they must be skipped over. + +\item The from-space includes the nursery. +Furthermore, a closure in the nursery may not necessarily be adjacent to the next +closure because slop words may lie between the two closures; +the Haskell mutator may allocate more space than actually needed in the +nursery when creating a closure, potentially leaving slop words. +\end{itemize} + +The first problem is easily solved by limiting the scan up to the +actual block boundary for each nursery block (see +@processNurseryForDead()@). In other words, for a nursery block +descriptor @bd@, whichever of @bd->start@$ + $@BLOCK_SIZE_W@ and +@bd->free@ is smaller is used as the actual boundary. + +We solve the second problem by exploiting LDV words of @EVACUATED@ +closures: we store the size of an evacuated closure, which now resides +in the to-space, in the LDV word of the new @EVACUATED@ closure +occupying its memory. This is easily implemented by inserting a call +to the macro @SET_EVACUAEE_FOR_LDV()@ in @copy()@ and @copyPart()@ (in +@GC.c@). Thus, when we encounter an @EVACUATED@ closure while +linearly scanning the nursery, we can skip a correct number of words +by referring to its LDV word. + +The third problem could be partially solved by always monitoring @Hp@ +during the Haskell mutator time: whenever @Hp@ is increased, we fill +with zeroes as many words as the change of @HP@. Then, we could skip +any trailing zero words when linearly scanning the nursery. +Alternatively we could initialize the entire nursery with zeroes after +each garbage collection and not worry about any change made to @Hp@ +during the Haskell mutator time. The number of zero words to be +written to the nursery could be reduced in the first approach, for we +do not have to fill the header for a new closure. Nevertheless we +choose to employ the second approach because it simplifies the +implementation code significantly (see @resetNurseries()@ in +@Storage.c@). Moreover, the second approach compensates for its +redundant initialization cost by providing faster execution (due to a +single memory write loop in contrast to frequent memory write loops in +the first approach). Also, we attribute the initialization cost to +the runtime system and thus the Haskell mutator behavior is little +affected. + +There is further complication though: occasionally a closure is +overwritten with a closure of a smaller size, leaving some slop +between itself and the next closure in the heap. There are two cases: + +\begin{enumerate} +\item A closure is overwritten with a blackhole. +\item A closure is overwritten with an indirection closure. +\end{enumerate} + +In either case, an existing closure is destroyed after being replaced +with a new closure. If the two closures are of the same size, no slop +words are introduced and we only need to invoke @LDV_recordDead()@ on +the existing closure, which cannot be an inherently used closure. If +not, that is, the new closure is smaller than the existing closure +(the opposite cannot happen), we need to fill one or more slop words +with zeroes as well as invoke @LDV_recordDead()@ on the existing +closure. The macro @LDV_recordDead_FILL_SLOP_DYNAMIC()@ accomplishes +these two tasks: it determines the size of the existing closure, +invokes @LDV_recordDead()@, and fills the slop words with zeroes. +After excluding all cases in which the two closures are of the same +size, we invoke @LDV_recordDead_FILL_SLOP_DYNAMIC()@ only from: + +\begin{enumerate} +\item @threadLazyBlackHole()@ and @threadSqueezeStack()@ in @GC.c@ +(for lazy blackholing), +\item @UPD_BH_UPDATABLE()@ and @UPD_BH_SINGLE_ENTRY()@ in +@includes/StgMacros.h@ (for eager blackholing, which isn't the +default), +\item @updateWithIndirection()@ and @updateWithPermIndirection()@ +in @Storage.h@.\footnote{Actually slop words created in +@updateWithIndirection()@ cannot survive major garbage collections. +Still we invoke @LDV\_recordDead\_FILL\_SLOP\_DYNAMIC()@ to support linear +scan of the heap during a garbage collection, which is discussed in the next +section.} +\end{enumerate} + +The linear scan of the from-space is initiated by the garbage +collector. From the function @LdvCensusForDead()@, every dead closure +in the from-space is visited through an invocation of +@processHeapClosureForDead()@. + +\subsection{Final scan of the heap} + +Since a closure surviving the final garbage collection is implicitly destroyed +when the runtime system shuts down, we must invoke @processHeapClosureForDead@ +on \emph{every} closure in the heap once more after the final garbage collection. +The function @LdvCensusKillAll()@, which is invoked from @shutdownHaskell()@ +in @RtsStartup.c@, traverses the entire heap and visits each closure. +It also stops LDVU profiling by resetting @ldvTime@ to $0$. + +It may be that after LDVU profiling stops, new closures may be created +and even garbage collections may be performed. +We choose to ignore these closures because they are all concerned about +finalizing weak pointers (in @finalizeWeakPointersNow()@). +It can be catastrophic to invoke @LdvCensusKillAll()@ after finishing +@finalizeWeakPointersNow()@: @finalizeWeakPointersNow()@ calls +@rts_evalIO()@, which is essentially initiating a new program execution, +and no assumptions made upon LDVU profiling hold any longer. + +\section{Time of Use} + +In order to yield correct LDVU profiling results, we must make sure +that @LDV_recordUse()@ be called on a closure whenever it is used; +otherwise, most of closures would be reported to be in the void phase. +@includes/StgLdvProf.h@ provides an entry macro @LDV_ENTER@ which +expands to @LDV_recordUse()@. The compiler arranges to invoke +@LDV_ENTER@ in the entry code for every dynamic closure it generates +code for (constructors, thunks and functions). We also have to add +@LDV_ENTER@ calls to the closures statically compiled into the RTS: +@PAP@s, @AP_UPD@s, standard thunk forms (in @StgStdThunks.hc@, and +several others in @StgMiscClosures.hc@. + +\section{Computing Final Statistics} + +After the final scan of the heap, we can accurately determine the total +size of closures in one of the four phases at the moment of each heap census. +The structure @LdvGenInfo@ is augmented with two additional fields +@voidTotal@ and @dragTotal@: + +\begin{code} +typedef struct { + ... + int voidTotal; + int dragTotal; + ... +} LdvGenInfo; +\end{code} + +@gi[@$i$@].voidTotal@ and @gi[@$i$@].dragTotal@ are computed +from @gi[@$i$@].voidNew@ and @gi[@$i$@].dragNew@, respectively.\footnote{Due +to a slight optimization described before, @gi[@$i$@].voidTotal@ is actually +computed as $\sum_{1 \leq j \leq i}$@gi[@$j$@].voidNew@. +@gi[@$i$@].dragTotal@ is computed in a similar way.} +Then, the total size of closures in the lag phase @gi[@$i$@].lagTotal@ is computed +as @gi[@$i$@].notUsed@$-$@gi[@$i$@].voidTotal@ (because any unused closure +is either in the void phase or in the lag phase). +Similarly, +the total size of closures in the use phase @gi[@$i$@].useTotal@ is computed +as @gi[@$i$@].used@$-$@gi[@$i$@].dragTotal@ (because any used closure +is either in the use phase or in the drag phase). +@endLdvProfiling()@, called from @endHeapProfiling@ in @ProfHeap.c@, computes these +final statistics. + +\section{Usage} + +The runtime system option @-hL@ tells the executable program to +perform LDVU profiling and produce a @.hp@ file: + +\begin{code} +$ Foo.out +RTS -hL +\end{code} + +The option @-i@ can be used to +specify a desired interval at which LDVU profiling is performed. +The default and minimum value is half a second: + +\begin{code} +$ Foo.out +RTS -hL -i2.5 -RTS +\end{code} + +The @.hp@ file can be supplied to the @hp2ps@ program to create a postscript +file showing the progress of LDVU profiling in a graph: + +\begin{code} +$ hp2ps Foo.hs +$ gv Foo.ps +\end{code} + +The horizontal axis of the graph is in the Haskell mutator time, which excludes +the runtime system time such as garbage collection time and LDVU profiling +time. +The Haskell mutator runs a bit slower than it would without performing +LDVU profiling, but the difference is minute. +Also, the timer employed to periodically perform retainer profiling +is not perfectly accurate. Therefore, the result may slightly vary for each +execution of retainer profiling. + +\textbf{To do:} Currently the LDVU profiling is not supported with @-G1@ option. + +\textbf{To do:} When we perform LDVU profiling, the Haskell mutator time seems to +be affected by @-S@ or @-s@ runtime option. For instance, the following +two options should result in nearly same profiling outputs, but +the second run (without @-Sstderr@ option) spends almost twice as +long in the Haskell mutator as the first run: +1) @+RTS -Sstderr -hL -RTS@; 2) @+RTS -hL -RTS@. +This is quite a subtle bug because this wierd phenomenon is not +observed in retainer profiling, yet the implementation of +@mut_user_time_during_LDV()@ is completely analogous to that of +@mut_user_time_during_RP()@. The overall shapes of the resultant graphs +are almost the same, though. + +\section{Files} + +This section gives a summary of changes made to the GHC in +implementing LDVU profiling. +Only three files (@includes/StgLdvProf.h@, @LdvProfile.c@, and +@LdvProfile.h@) are new, and all others exist in the GHC. + +@\includes@ directory: + +\begin{description} +\item[StgLdvProf.h] defines type @LdvGenInfo@, constants, and macros related +with LDVU profiling. +\item[ClosureMacros.h] changes macro @SET_PROF_HDR()@. +\item[Stg.h] includes th header file @StgLdvProf.h@. +\item[StgMacros.h] changes macros @UPD_BH_UPDATABLE()@ and @UPD_BH_SINGLE_ENTRY()@. +\end{description} + +@\rts@ directory: + +\begin{description} +\item[GC.c] invokes @LdvCensusForDead()@ before tidying up, sets @hasBeenAnyGC@ to + @rtsTrue@, and changes @copy()@ and @copyPart()@. + Invokes @LDV_recordDead()@ and @LDV_recordDead_FILL_SLOP_DYNAMIC()@. +\item[Itimer.c] changes @handle_tick()@. +\item[LdvProfile.c] implements the LDVU profiling engine. +\item[LdvProfile.h] is the header for @LdvProfile.c@. +\item[PrimOps.hc] changes @finalizzeWeakzh_fast()@. +\item[ProfHeap.c] changes @initHeapProfiling()@ and @endHeapProfiling()@. +\item[Profiling.c] changes @initProfilingLogFile@ and @report_ccs_profiling()@. +\item[Proftimer.c] declares @ticks_to_retainer_ldv_profiling@, + @performRetainerLdvProfiling@, and @doContextSwitch@. +\item[Proftimer.h] is the header for @Proftimer.c@. Defines @PROFILING_MIN_PERIOD@, + which specifies the minimum profiling period and the default profiling period. +%\item[RtsAPI.c] implements @setProfileHeader()@. +\item[RtsFlags.c] + sets @RtsFlags.ProfFlags.doHeapProfile@, + adds a string to @usage_text[]@ in @setupRtsFlags()@. +\item[RtsFlags.h] defines constants @HEAP_BY_LDV@ and @LDVchar@. +\item[RtsStartup.c] changes @shutDownHaskell()@. +\item[Schedule.c] changes @schedule()@. +\item[Stats.c] + declares @LDV_start_time@, @LDV_tot_time@, @LDVe_start_time@, + @LDVe_tot_time@. + Changes @mut_user_time_during_GC()@, @mut_user_time()@, + @stat_startExit()@, + @stat_endExit()@, and + @stat_exit()@. + Defines + @mut_user_time_during_LDV()@, + @stat_startLDV()@, and + @stat_endLDV()@. +\item[Stats.h] is hte header for @Stats.c@. +\item[StgMiscClosures.hc] inserts entry macros in + @stg_IND_entry()@, @stg_IND_PERM_entry()@, @stg_IND_OLDGEN_entry()@, + @stg_IND_OLDGEN_PERM_entry()@, @stg_BLACKHOLE_entry()@, @stg_BLACKHOLE_BQ_entry()@, + and @stg_CAF_BLACKHOLE_entry()@. + Invokes @LDV_recordDead()@ in @stg_BLACKHOLE_entry@. + Redefines @stg_DEAD_WEAK_info@. +\item[Storage.c] changes @initStorage()@, @resetNurseries()@, and @allocNursery()@. +\item[Storage.h] changes @updateWithIndirection()@ and @updateWithPermIndirection()@. +\item[Updates.hc] inserts entry macros in @stg_PAP_entry()@ and @stg_AP_UPD_entry()@. +\item[Weak.c] changes @scheduleFinalizers()@. +\end{description} + +\bibliographystyle{plain} +\bibliography{reference} + +\end{document} diff --git a/docs/storage-mgt/megablock.eepic b/docs/storage-mgt/megablock.eepic new file mode 100644 index 0000000000..922226945b --- /dev/null +++ b/docs/storage-mgt/megablock.eepic @@ -0,0 +1,35 @@ +\setlength{\unitlength}{0.00054167in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(6849,1539)(0,-10) +\put(687,1062){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(687,837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}descriptor}}}}} +\path(612,1512)(1737,1512)(1737,462) + (612,462)(612,1512) +\path(4062,1512)(5187,1512)(5187,462) + (4062,462)(4062,1512) +\path(12,1512)(6837,1512)(6837,462) + (12,462)(12,1512) +\path(2337,1512)(2337,462) +\path(132.000,192.000)(12.000,162.000)(132.000,132.000) +\path(12,162)(2337,162) +\path(2217.000,132.000)(2337.000,162.000)(2217.000,192.000) +\path(2457.000,192.000)(2337.000,162.000)(2457.000,132.000) +\path(2337,162)(6837,162) +\path(6717.000,132.000)(6837.000,162.000)(6717.000,192.000) +\path(2337,12)(2337,312) +\put(237,912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(1962,912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(2862,912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(5637,912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(4362,912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}block}}}}} +\put(312,237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}block descriptors}}}}} +\put(4212,237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{8}{9.6}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/megablock.fig b/docs/storage-mgt/megablock.fig new file mode 100644 index 0000000000..8116c841b5 --- /dev/null +++ b/docs/storage-mgt/megablock.fig @@ -0,0 +1,40 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +65.00 +Single +-2 +1200 2 +6 3000 3675 4125 4725 +6 3075 3975 3900 4425 +4 0 0 50 0 0 12 0.0000 4 135 405 3075 4125 block\001 +4 0 0 50 0 0 12 0.0000 4 180 765 3075 4350 descriptor\001 +-6 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 3000 3675 4125 3675 4125 4725 3000 4725 3000 3675 +-6 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6450 3675 7575 3675 7575 4725 6450 4725 6450 3675 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2400 3675 9225 3675 9225 4725 2400 4725 2400 3675 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 4725 3675 4725 4725 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 2400 5025 4725 5025 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 4725 5025 9225 5025 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 4725 5175 4725 4875 +4 0 0 50 0 0 12 0.0000 4 15 135 2625 4275 ...\001 +4 0 0 50 0 0 12 0.0000 4 15 135 4350 4275 ...\001 +4 0 0 50 0 0 12 0.0000 4 15 270 5250 4275 ......\001 +4 0 0 50 0 0 12 0.0000 4 15 270 8025 4275 ......\001 +4 0 0 50 0 0 12 0.0000 4 135 405 6750 4275 block\001 +4 0 0 50 0 0 12 0.0000 4 180 1305 2700 4950 block descriptors\001 +4 0 0 50 0 0 12 0.0000 4 135 495 6600 4950 blocks\001 diff --git a/docs/storage-mgt/nursery.eepic b/docs/storage-mgt/nursery.eepic new file mode 100644 index 0000000000..9b06c6e0a3 --- /dev/null +++ b/docs/storage-mgt/nursery.eepic @@ -0,0 +1,89 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(11262,7914)(0,-10) +\path(4575,7137)(6150,7137)(6150,5937) + (4575,5937)(4575,7137) +\path(5325,6987)(5325,7437)(7950,7437)(7950,7137) +\path(7920.000,7257.000)(7950.000,7137.000)(7980.000,7257.000) +\path(11025,7137)(11025,5937) +\path(5475,6687)(5475,7437) +\path(7950,7137)(11250,7137)(11250,5937) + (7950,5937)(7950,7137) +\path(5475,6687)(5475,7887)(11025,7887)(11025,7137) +\path(10995.000,7257.000)(11025.000,7137.000)(11055.000,7257.000) +\path(4725,6087)(4125,6087)(4125,5562) +\path(4095.000,5682.000)(4125.000,5562.000)(4155.000,5682.000) +\path(8070.000,6567.000)(7950.000,6537.000)(8070.000,6507.000) +\path(7950,6537)(11025,6537) +\path(10905.000,6507.000)(11025.000,6537.000)(10905.000,6567.000) +\path(4125,5112)(4125,4587)(4500,4587) +\path(4380.000,4557.000)(4500.000,4587.000)(4380.000,4617.000) +\path(4500,4662)(6075,4662)(6075,3462) + (4500,3462)(4500,4662) +\path(5250,4512)(5250,4962)(7875,4962)(7875,4662) +\path(7845.000,4782.000)(7875.000,4662.000)(7905.000,4782.000) +\path(5400,4212)(5400,4962) +\path(7875,4662)(11175,4662)(11175,3462) + (7875,3462)(7875,4662) +\path(4650,3612)(4050,3612)(4050,2112) +\path(4020.000,2232.000)(4050.000,2112.000)(4080.000,2232.000) +\path(5400,4212)(5400,5412)(7875,5412)(7875,4662) +\path(7845.000,4782.000)(7875.000,4662.000)(7905.000,4782.000) +\path(7995.000,4092.000)(7875.000,4062.000)(7995.000,4032.000) +\path(7875,4062)(9750,4062) +\path(9630.000,4032.000)(9750.000,4062.000)(9630.000,4092.000) +\path(9750,4662)(9750,3462) +\path(9150,2787)(9750,2787)(9750,3462) +\path(9780.000,3342.000)(9750.000,3462.000)(9720.000,3342.000) +\path(9525,2337)(11175,2337)(11175,3462) +\path(11205.000,3342.000)(11175.000,3462.000)(11145.000,3342.000) +\path(3300,4737)(3300,4362)(4500,4362) +\path(4380.000,4332.000)(4500.000,4362.000)(4380.000,4392.000) +\path(3375,7212)(3375,6837)(4575,6837) +\path(4455.000,6807.000)(4575.000,6837.000)(4455.000,6867.000) +\path(4050,1662)(4050,1137)(4425,1137) +\path(4305.000,1107.000)(4425.000,1137.000)(4305.000,1167.000) +\path(4425,1212)(6000,1212)(6000,12) + (4425,12)(4425,1212) +\path(5175,1062)(5175,1512)(7800,1512)(7800,1212) +\path(7770.000,1332.000)(7800.000,1212.000)(7830.000,1332.000) +\path(5325,762)(5325,1512) +\path(7800,1212)(11100,1212)(11100,12) + (7800,12)(7800,1212) +\path(5325,762)(5325,1962)(7800,1962)(7800,1212) +\path(7770.000,1332.000)(7800.000,1212.000)(7830.000,1332.000) +\put(4650,6912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(4650,6612){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(4800,6012){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(4650,6312){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(8625,7287){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(8625,6687){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\put(3900,5337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(4575,4437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(4575,4137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(4725,3537){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(4575,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(8550,4812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(8025,4212){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\put(9975,4212){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(9975,3927){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}memory}}}}} +\put(8625,2712){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}Hp}}}}} +\put(8625,2262){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}HpLim}}}}} +\put(0,4887){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}MainRegTable.rCurrentNursery}}}}} +\put(750,7362){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}MainRegTable.rNursery}}}}} +\put(3825,1887){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(4500,987){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(4500,687){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(4500,387){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(8475,1362){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(8775,762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free memory}}}}} +\put(4500,87){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link=NULL}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/nursery.fig b/docs/storage-mgt/nursery.fig new file mode 100644 index 0000000000..6a4b60fb82 --- /dev/null +++ b/docs/storage-mgt/nursery.fig @@ -0,0 +1,107 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 1425 7875 1425 7875 2625 6300 2625 6300 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 1575 7050 1125 9675 1125 9675 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12750 1425 12750 2625 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 1875 7200 1125 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9675 1425 12975 1425 12975 2625 9675 2625 9675 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7200 1875 7200 675 12750 675 12750 1425 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 6450 2475 5850 2475 5850 3000 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9675 2025 12750 2025 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 5850 3450 5850 3975 6225 3975 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6225 3900 7800 3900 7800 5100 6225 5100 6225 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 6975 4050 6975 3600 9600 3600 9600 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7125 4350 7125 3600 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9600 3900 12900 3900 12900 5100 9600 5100 9600 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 6375 4950 5775 4950 5775 6450 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7125 4350 7125 3150 9600 3150 9600 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9600 4500 11475 4500 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 11475 3900 11475 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 10875 5775 11475 5775 11475 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 11250 6225 12900 6225 12900 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 5025 3825 5025 4200 6225 4200 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 5100 1350 5100 1725 6300 1725 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 5775 6900 5775 7425 6150 7425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6150 7350 7725 7350 7725 8550 6150 8550 6150 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 6900 7500 6900 7050 9525 7050 9525 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7050 7800 7050 7050 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9525 7350 12825 7350 12825 8550 9525 8550 9525 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 7800 7050 6600 9525 6600 9525 7350 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 1650 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 1950 free\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 2550 link\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6375 2250 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10350 1275 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 10350 1875 used memory\001 +4 0 0 50 0 0 17 0.0000 4 30 360 5625 3225 ......\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6300 4125 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6300 4425 free\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6450 5025 link\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6300 4725 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10275 3750 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 9750 4350 used memory\001 +4 0 0 50 0 0 17 0.0000 4 165 390 11700 4350 free\001 +4 0 0 50 0 0 17 0.0000 4 180 825 11700 4635 memory\001 +4 0 0 50 0 0 17 0.0000 4 225 300 10350 5850 Hp\001 +4 0 0 50 0 0 17 0.0000 4 225 720 10350 6300 HpLim\001 +4 0 0 50 0 0 17 0.0000 4 225 3180 1725 3675 MainRegTable.rCurrentNursery\001 +4 0 0 50 0 0 17 0.0000 4 225 2415 2475 1200 MainRegTable.rNursery\001 +4 0 0 50 0 0 17 0.0000 4 30 360 5550 6675 ......\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6225 7575 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6225 7875 free\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6225 8175 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10200 7200 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1275 10500 7800 free memory\001 +4 0 0 50 0 0 17 0.0000 4 165 1185 6225 8475 link=NULL\001 diff --git a/docs/storage-mgt/reference.bib b/docs/storage-mgt/reference.bib new file mode 100644 index 0000000000..48fa520b27 --- /dev/null +++ b/docs/storage-mgt/reference.bib @@ -0,0 +1,14 @@ +@inproceedings {CN, + author = {Colin Runciman and Niklas Rojemo}, + title = {New Dimensions in heap profiling}, + booktitle = "", + pages = "", + year = "1994" } + +@inproceedings {RR, + author = {Niklas Rojemo and Colin Runciman}, + title = {Lag, drag, void and use - heap profiling and space-efficient compilation revisited}, + booktitle = "", + pages = "", + year = "1996" } + diff --git a/docs/storage-mgt/rp.tex b/docs/storage-mgt/rp.tex new file mode 100644 index 0000000000..2055894282 --- /dev/null +++ b/docs/storage-mgt/rp.tex @@ -0,0 +1,1102 @@ +\documentclass{article} +\usepackage{code,a4wide} + +\usepackage{graphics,epsfig,epic,eepic,epsfig} + +\setlength{\parskip}{0.25cm} +\setlength{\parsep}{0.25cm} +\setlength{\topsep}{0cm} +\setlength{\parindent}{0cm} +\renewcommand{\textfraction}{0.2} +\renewcommand{\floatpagefraction}{0.7} + + +% Terminology +\newcommand{\block}{block} +\newcommand{\Block}{Block} +\newcommand{\segment}{segment} +\newcommand{\Segment}{Segment} +\newcommand{\step}{step} +\newcommand{\Step}{Step} + +\newcommand{\note}[1]{{\em $\spadesuit$ #1}} + +\begin{document} +\title{Implementation of Retainer Profiling} +\author{Sungwoo Park and Simon Peyton-Jones} + +\makeatactive +\maketitle + +\section{Retainer Profiling} + +Retainer profiling~\cite{CN} is a profiling technique which is based upon a +special view of production and consumption of heap objects at runtime: +while producers build heap objects to form new pieces of graph, +consumers hold pointers to these heap objects, or \emph{retain} them, so +that they are not freed during garbage collections. +On this basis, we refereed to such consumers as \emph{retainers}. +Notice that an object can have more than one retainer because it can +be pointed to by multiple objects. + +For each live object in the heap, retainer profiling computes +all its retainers, or its \emph{retainer set}. +A naive implementation of retainer profiling could consider every +immediate ancestor of an object as its retainer. +Although this approach appears to provide an accurate report on the +relationship between retainers and retainees, the result can hardly be useful. +For instance, it is pointless to scrutinize a list and treat each cons +cell as a retainer of the following cons cell. +This observation suggests that we need to have a way of designating only +certain types of objects as candidates for retainers. +In other words, we need to employ an oracle which tells whether a given +object can be a retainer or not. + +Since no retainer of a particular object needs to be using the +object actively, we can find all its retainers simply by traversing +the graph. In other words, we do not have to distinguish those retainers +actively exploiting it from other retainers just holding pointers +to it (either directly or indirectly). +Thus, retainer profiling can be accomplished simply by traversing the +graph. + +Figure~\ref{fig-retaineralgorithm} shows the algorithm for retainer +profiling. The traversal begins at every root, and proceeds +in a depth first manner (or a breadth first manner). +The function @R()@ returns the \emph{identity} of a given retainer such +as its information table address or +the name of the module which creates it. +Notice that the retainer identity function does not need to be a +one-to-one mapping: +multiple objects can share the same identity. +Such a retainer identity function reduces the cost of traversal. +For instance, when an object +is reached from several retainers which share the same identity, we need to +consider only the first visit to the object. +In other words, whichever retainer (among those sharing the same identity) +leads to the object for the first time affects the retainer set of the object +in consideration +and all the other retainers can be ignored. +Thus, the more coarse the function @R()@ is, the less +it costs to traverse the graph for retainer profiling. +The function @isRetainer()@ tells whether a given object is a retainer or not. +Hence, the behavior of the retainer profiling algorithm is parameterized +over: 1) the set of roots; 2) the function @R()@; 3) the function +@isRetainer()@. + +One important invariant on the function @R()@ is that its return value +must be consistent for a given retainer. In other words, @R()@ must return +the same value for a given retainer no matter it is invoked. +For this reason, the memory address of a retainer, for instance, cannot be used as +its retainer identity because its location may change during garbage collections. + +\begin{figure}[ht] +\begin{center} +\begin{code} +for every root r + retain(r, r) + +R(r) = + the identity of r + +isRetainer(c) = + if c is a retainer then + true + else + false + +retain(c, r) = + if R(r) is a member of c.retainerSet then + return + add R(r) to c.retainerSet + if isRetainer(c) then + r' := c + else + r' := r + for every successor c' of c + retain(c', r') +\end{code} +\caption{Retainer profiling algorithm} +\label{fig-retaineralgorithm} +\end{center} +\end{figure} + +Another way of formulating retainer profiling is in terms of fixed point +equations on retainer sets. +To be specific, given the two functions @isRetainer()@ and @R()@, +the retainer set of every object is computed as the least fixed point +solution of the following equations: +\begin{itemize} +\item For every root @r@, +\begin{center} + @R(r)@ $\in$ @r.retainerSet@. +\end{center} +\item For every reachable object @c@, +\begin{center} +$\bigcup_{\mathit{each\ ancestor\ @a@\ of\ @c@}}$ @from(a)@ $\subseteq$ +@c.retainerSet@ +\end{center} +where @from(a)@ returns retainer(s) obtainable from @a@: +\begin{center} +@from(a)@ = if @isRetainer(a)@ then $\{@a@\}$ else @a.retainerSet@. +\end{center} +\end{itemize} + +This document describes the implementation of retainer profiling on +the Glasgow Haskell Compiler runtime system. +It explains every detail of the development so that it can be (hopefully) +a complete maintenance guide. +A secondary goal is to help (hopefully) those who wish to extend the system +to implement another profiling scheme.\footnote{Unless otherwise mentioned, +all identifiers are defined in @RetainerProfile.c@ or @RetainerSet.c@.} + +\section{Installing the GHC} + +Installing the GHC is done as follows: + +\begin{enumerate} +\item Get the source code from the CVS repository. +\begin{code} +./ cvs checkout fpconfig +./fptools/ cvs update -d CONTRIB common-rts distrib docs ghc glafp-utils + hslibs literate mhms mk nofib testsuite +\end{code} + +\item Set up the basic configuration. +\begin{code} +./fptools/ autoconf +./fptools/ghc/ autoconf +./fptools/ configure +\end{code} + +\item Set up the configuration for development and debugging. +\begin{code} +./fptools/mk vi build.mk + GhcHcOpts = -O -fasm -Rghc-timing + SplitObjs = NO + GhcRtsHcOpts = + GhcRtsCcOpts = -g + STRIP =: +\end{code} +@GhcLibWays@ tells the compiler to build the code for profiling as well. +@GhcRtsHcOpts@ has additional flags for @gcc@ when compiling @.hc@ files. +@GhcRtsCcOpts@ has additional flags for @gcc@ when compiling @.c@ files. +Since we will implement retainer profiling in @.c@ files, we turn on the +debugging flag @-g@. +The empty setting for @STRIP@ tells the compiler not to remove source code +information (generated due to the @-g@ option) from executable files so that +they can be examined with @gdb@. + +\item Remove unnecessary files if needed and build everything. +\begin{code} +./fptools/ make +\end{code} +\end{enumerate} + +\section{Adding Retainer Set Fields} + +Since every Haskell closure now needs to store its retainer set at runtime, +it must be augmented with a new field, +namely, a \emph{retainer set field}. +This section explains how to add such a field to Haskell closures. +It should be clear how to generalize the idea for adding +any number of new fields.\footnote{The GHC provides two +ways of building executable programs from +source files: normal way and profiling way. +We are concerned only about profiling way, and all the pieces of code +implementing profiling way are wrapped by the @PROFILING@ +pre-processing directive (as in @\#ifdef PROFILING@). +Therefore, all the additions and changes that we make to the source code +are assumed to be wrapped by the @PROFILING@ pre-processing +directive as well unless otherwise mentioned.} + +\subsection{Adding a new field to Haskell closures} + +We want to add a retainer set field of type @retainerSet@ to every +closure, so we create a new file @includes/StgRetainerProf.h@ where +we define the type @retainerSet@. +The actual definition of @retainerSet@ will be given later. + +\begin{code} +/* includes/StgRetainerProf.h */ +typedef ... retainerSet; +\end{code} + +We make type @retainerSet@ to be publicly available by including +@includes/StgRetainerProf.h@ itself to @includes/Stg.h@ (not wrapped +by @PROFILING@). + +\begin{code} +/* includes/Stg.h */ +#include "StgRetainerProf.h" +\end{code} + +Then we add a retainer set field @rs@ to the @StgProfHeader@ structure. + +\begin{code} +/* include/Closures.h */ +typedef struct { + CostCentreStack *ccs; + retainerSet *rs; +} StgProfHeader; +\end{code} + +Now every closure @c@ (including static closures) has a retainer set field, +which can be accessed with @c->header.prof.rs@ (where @c@ denotes the +address of the closure). + +\subsection{Changing constants} + +We are ready to reflect the new size of Haskell closures to other part +of the source code. +This is accomplished by changing a few constants which specify the size +of certain types of closures and their layout. + +When building the runtime system, the @gcc@ compiler correctly figures out +the size of every structure on its own. +However, +GHC simply reads @includes/Constants.h@ to to determine the size of +closures assumed by the runtime system. +Thus, we must change the constants used by the GHC itself (as opposed to +the runtime system). They are all found in @includes/Constants.h@. +We increase each of them by 1 to reflect the retainer set field which is one +word long: +\begin{code} +/* includes/Constants.h */ +#define PROF_HDR_SIZE 2 +#define SCC_UF_SIZE 5 +#define SCC_SEQ_FRAME_SIZE 4 +\end{code} +@PROF_HDR_SIZE@ denotes the size of the structure @StgProfHeader@, which +is now two words long. +@SCC_UF_SIZE@ and @SCC_SEQ_FRAME_SIZE@ denote the size of the structures +@StgUpdateFrame@ and @StgSeqFrame@ (in @includes/Closures.h@) in +words. + +Now we must rebuild the GHC so that, when executed, the code generated by +the GHC must now allocate one more word for the retainer set field than before. + +\begin{code} +./fptools/ghc/ make boot +./fptools/ghc/ make +\end{code} + +The second command @make boot@ instructs the build system to analyze +the source code dependency so that the next execution of @make@ correctly +finds all required files. + +Next we change four bitmap constants which specify the layout of +certain types of closures. +As an example, let us consider @RET_BITMAP@, which specifies the layout +of thunk selectors (corresponding to closure type @THUNK_SELECTOR@). +Without a retainer set field, there is only one non-pointer (represented +by $1$) followed by one or more pointers (represented by $0$) in the closure +body and the bitmap representation is $0b1$, or $1$. +With a retainer set field, which is not a pointer to another closure and thus +represented by $1$, there are two non-pointers, and the bitmap representation +is $0b11$, or $3$. Notice that the bitmap is interpreted in reverse order: +the least significant bit corresponds to the first word in the closure body, +and the second least significant bit to the second word, and so forth. +The same rule applies to the other three bitmap constants: +@CATCH_FRAME_BITMAP@ (for closure type @CATCH_FRAME@ and structure +@StgCatchFrame@), +@STOP_THREAD_BITMAP@ (for closure type @STOP_FRAME@ and structure +@StgStopFrame@), and +@UPD_FRAME_BITMAP@ (for closure type @UPDATE_FRAME@ and structure +@StgUpdateFrame@). + +\begin{code} +/* rts/StgStdThunks.hc */ +#define RET_BITMAP 3 +/* rts/Exception.hc */ +#define CATCH_FRAME_BITMAP 15 +/* rts/StgStartup.hc */ +#define STOP_THREAD_BITMAP 3 +/* rts/updates.hc */ +#define UPD_FRAME_BITMAP 7 +\end{code} + +For most closure types, the new definition of @StgProfHeader@ is +automatically propagated to their corresponding structures. +However, there are six closures types which are not affected by +@StgProfHeader@. They are all stack closures: +@RET_DYN@, @RET_BCO@, @RET_SMALL@, @RET_VEC_SMALL@, @RET_BIG@, and +@RET_VEC_BIG@. +If you want a new field to be added to these closures, you may +have to modify their corresponding structures. + +\textbf{To do:} Presently the above changes introduce two bug in the +runtime system. +First, @nofib/real/symalg@ ends up with a division-by-zero +exception if we add a new field. +Second, the runtime system option @-auto-all@ clashes in some test files +in the @nofib@ testing suite (e.g., @spectral/expert@). + +\subsection{Initialization code} + +When a new closure is allocated, its retainer set field may have to be +initialized according to the way that retainer profiling is implemented. +For instance, we could use as an initial value a pointer to an empty retainer +set. +Alternatively we could assign a null pointer to indicate that its retainer +set has not been computed yet, which we adopt in our implementation. +In either case, we have to visit the new closure and execute some initialization +code on it so that its retainer set field is set to an appropriate value. + +There are three parts in the source code which need to be modified. +dynamic closure initialization, static closure initialization, +and update frame initialization. +The first is accomplished by modifying the macro @SET_PROF_HDR()@ (in +@include/ClosureMacros.h@). When a closure @c@ is created at runtime, +@SET_PROF_HDR()@ is invoked immediately with @c@ as its first argument. +Thus, the following code initializes the retainer set field of every +dynamic closure to a null pointer. + +\begin{code} +/* include/ClosureMacros.h */ +#define SET_PROF_HDR(c,ccs_) \ + ((c)->header.prof.ccs = ccs_, (c)->header.prof.rs = NULL) +\end{code} + +Similarly, the macro @SET_STATIC_PROF_HDR()@ (in the +same file) specifies how the retainer set field of every static closure +is initialized, which is rewritten as follows: + +\begin{code} +/* include/ClosureMacros.h */ +#define SET_STATIC_PROF_HDR(ccs_) \ + prof : { ccs : ccs_, rs : NULL }, +\end{code} + +\textbf{Obsolete:} Dynamic closures created through explicit C function invocations +(in @RtsAPI.c@) are now initialized by @SET_HDR()@. + +%There is another way of creating dynamic closures through explicit C +%function invocations at runtime. +%Such functions are all defined in @RtsAPI.c@: @rts_mkChar()@, @rts_mkInt()@, +%@rts_mkWord()@, and so forth. +%Each function allocates memory for a new closure, +%initializes it, and returns its address. +%Therefore, we can simply insert in each function another initialization code +%for retainer set fields. +%To this end, we define a macro @setRetainerField()@ and insert it +%in each function: +% +%\begin{code} +%#define setRetainerField(p) \ +% (p)->header.prof.rs = NULL +%\end{code} +% +%For instance, @rts_mkChar()@ is now defined as follows: +% +%\begin{code} +%/* RtsAPI.c */ +%HaskellObj +%rts_mkChar (HsChar c) +%{ +% StgClosure *p = (StgClosure *)allocate(CONSTR_sizeW(0,1)); +% ... +% setRetainerField(p); +% return p; +%} +%\end{code} + +Finally we may need to initialize the retainer set field of an update frame +(stack closure) when it is pushed onto the stack for the first time. +For instance, if we want to initialize the retainer set field of update +frames to a null pointer, we can rewrite the macro @PUSH_STD_CCCS()@ +(in @includes/Updates.h@) as follows: + +\begin{code} +/* includes/Updates.h */ +#define PUSH_STD_CCCS(frame) \ + (frame->header.prof.ccs = CCCS, frame->header.prof.rs = NULL) +\end{code} + +In our implementation of retainer profiling, however, the retainer set field is not +used for any stack closure. +Hence, the above modification is entirely unnecessary. +Also, update frames are the only exception to the standard way of creating +stack closures: all the other types of stack closures with a retainer set +field are eventually initialized by +the macro @SET\_HDR()@ (in @includes/ClosureMacros.h@), which in turn +invokes @SET\_PROF\_HDR()@. This is not the case for update frames. +Compare @PUSH\_UPD\_FRAME()@ (in @includes/Updates.h@) and +@PUSH\_SEQ\_FRAME()@ (in @includes/StgMacros.h@) for clarification. + +\section{Retainer Sets} + +At the end of retainer profiling, every live closure (except stack +closures, for which we do not compute retainer sets) is associated with +a retainer set; there can be no closure without an associated retainer set +because every live closure is visited during traversal. +Since many closures may well be associated with a common retainer set, +we want to avoid creating any particular retainer set more than once. +This section presents the details of manipulating retainer sets in our +implementation. + +\subsection{Identity of a retainer} + +The function @R()@ in Figure~\ref{fig-retaineralgorithm} returns +the identity of a retainer. In order to implement it, we need +a type for retainer identity. +The type @retainer@ (in @includes/StgRetainerProf.h@) is introduced for +this purpose. + +There are various ways of defining the type @retainer@. +For instance, we can designate the information table address of a retainer as +its identity as follows: + +\begin{code} +struct _StgInfoTable; +typedef struct _StgInfoTable *retainer; +\end{code} + +We can also use the cost centre stack associated with the retainer: + +\begin{code} +typedef CostCentreStack *retainer; +\end{code} + +The function @R()@ is embodied as the function @getRetainerFrom()@ in the +implementation, whose type is @(StgClosure *)@ $\rightarrow$ @retainer@. +It is straightforward to define @getRetainerFrom()@ according to the definition +of @retainer@, as illustrated below: + +\begin{code} +retainer getRetainerFrom(StgClosure *c) { return get_itbl(c); } +retainer getRetainerFrom(StgClosure *c) { return c->header.prof.ccs; } +\end{code} + +\subsection{Retainer sets and the cost function} + +A retainer set is stored in the structure @retainerSet@ +(in @includes/StgRetainerProf.h@): + +\begin{code} +typedef struct _retainerSet { + nat num; + nat cost; + ... + int id; + retainer element[0]; +} retainerSet; +\end{code} + +The field @num@ gives the number of retainers in the retainer set, which +are all stored in the array @element[]@. Thus, the size of @element[]@ +is assumed to be @num@. +The field @cost@ gives the sum of the \emph{costs} of those closures +associated with the retainer set: if a closure @c@ is +associated with the retainer set, that is, if @c@ is retained by each +retainer in the retainer set and none else, +the cost of @c@ is added to the field @cost@. +The field @id@ gives a unique identification number for the retainer set. + +The interface to @retainerSet@ is as follows +(see @RetainerSet.h@): + +\begin{description} +\item[@void initializeAllRetainerSet(void)@] initializes the store for retainer sets. +\item[@void refreshAllRetainerSet(void)@] refreshes each retainer set by setting +its @cost@ field to zero. This function does destroy any retainer set. +\item[@void closeAllRetainerSet(void)@] destroys all retainer sets and closes +the store for retainer sets. +\item[@retainerSet *singleton(retainer r)@] returns a singleton retainer set +consisting of @r@ alone. If such a retainer set already exists, no new retainer +set is created. Otherwise, a new retainer set is created. +\item[@retainerSet *addElement(retainer r, retainerSet *rs)@] returns a retainer set +@rs@ augmented with @r@. If such a retainer set already exists, no new retainer set +is created. Otherwise, a new retainer set is created. +\item[@rtsBool isMember(retainer r, retainerSet *rs)@] returns a boolean value +indicating whether @r@ is a member of @rs@. +\item[@void traverseAllRetainerSet(void (*f)(retainerSet *))@] invokes the function +@f@ on every retainer set created. +\item[@void printRetainerSetShort(FILE *, retainerSet *)@] prints a single retainer +set. +\item[@void outputRetainerSet(FILE *, nat *allCost, nat *numSet)@] prints all +retainer sets. Stores the sum of all their costs in @*allCost@ and the number +of them in @*numSet@. +\item[@void outputAllRetainerSet(FILE *)@] prints all retainer sets. +\end{description} + +We also define a \emph{cost function}, which returns the cost of a given closure, +in order to compute the field @cost@. +The cost function can be defined in several ways. +A typical definition is on the size of a closure, which results in +the field @cost@ accumulating the size of all the closures retained by a +retainer set. +If we just want to count the number of closures retained by the +retainer set, we can simply set the cost of every closure to one regardless +of its closure type. +Furthermore, we can define the cost function flexibly according to +the closure type. +For instance, we can set the size of any static closure to zero so that +it is not taken into account at all in computing the field @cost@. +Notice that static closures are also visited during traversal because they +may lead to other dynamic closures (e.g., static indirection closures of +closure type @IND_STATIC@). +This is especially desirable because we usually focus on the heap use. +We can also selectively choose certain dynamic closure types not to contribute +to the field @cost@. + +In our implementation, there are two functions related with the cost function: +@cost()@ and @costPure()@. +@cost()@ returns the size of the entire memory allocated for a given closure +(even including the two fields in the structure @StgProfHeader@). +It returns zero for static closures. +@costPure()@ returns the size of the memory which would be allocated for +a given closure with no profiling. +It is defined in terms of @cost()@, and it suffices to change only @cost()@ +when a new scheme for the cost function is desired. +@costPure()@ is put to actual use in computing the field @cost@ because it +effectively hides the memory overhead incurred by profiling. + +\subsection{Implementation} + +The algorithm for retainer profiling in Figure~\ref{fig-retaineralgorithm} +adds at most one retainer to an existing retainer set (or an empty retainer set) +at any moment; it does not require a retainer set union operation. +This observation simplifies the implementation, and +we employ the following two functions for creating new retainer sets: +@singleton()@, which creates a singleton retainer set, and +@addElement()@, which adds an element to an existing retainer set. + +It is a frequent operation during retainer profiling to search for a retainer +set, which may or may not exist, built from a given retainer set and a +particular retainer. +To efficiently implement this operation, +we choose to store all retainer sets in a hash table and +the structure @retainerSet@ is now extended with two new fields +@hashKey@ and @link@. +The field @hashKey@ stores the hash key which is obtained +from the retainers in a retainer set. +The field @link@ points to the next retainer set in the same bucket: + +\begin{code} +typedef struct _retainerSet { + ... + StgWord hashKey; + struct _retainerSet *link; + ... +} retainerSet; +\end{code} + +The hashing function must be defined in such a way that a retainer set +can have only one unique hash key regardless of the order its elements +are stored, i.e., the hashing function must be additive. + +It is often observed that two successive executions of retainer profiling share +a number of retainer sets in common, especially if the two executions are +close in time. +This also implies that the number of all retainer sets which can be created +at any moment does not grow indefinitely regardless of the interval at which +retainer profiling is performed; it does not grow commensurately with the +number of times retainer profiling is executed. +This observation eliminates the need to free the memory allocated for +retainer sets; we can simply set the @cost@ field of every retainer set +to zero after each retainer profiling and reuse it during the next time. + +\section{Graph Traversal} + +At the heart of retainer profiling lies \emph{graph traversal}; +the algorithm in Figure~\ref{fig-retaineralgorithm} is supposed to visit +every closure in the graph at least once and yield statistics on the heap use. +Since only live closures are reachable from the root, the algorithm +does not deal with dead closures. + +This section presents details on how to achieve an efficient implementation of +graph traversal without incurring extra memory overhead and compromising speed. + +\subsection{Goal} + +Traversing a graph itself can be done in a straightforward way; +we choose either depth first search or breadth first search, and traverse +the graph starting from a given set of roots. +After a complete traversal, each live closure @c@ (including static closures) +has an associated retainer set, whose address is stored in the field +@c->header.prof.rs@. + +A real complication arises when retainer profiling is performed once again: +all live closures which have survived all garbage collections since +the previous retainer profiling +still have an associated retainer set (indicated by +a non-null pointer in their retainer set field), which is no longer +valid. Any new closure created since then has +a null pointer in its retainer set field at the beginning of retainer +profiling and will become associated with a retainer set. +Thus, we can no longer distinguish valid retainer set fields +from invalid ones. + +A simple remedy is to linearly scan the heap at the beginning of each +retainer profiling and set all retainer set fields to a null pointer. +It resets the retainer set field of each dynamic closure, whether it is +live or not with respect to the given set of root. +This is feasible because any closure in the heap directly adjoins the +next closure, if any. +The problem is that we have no way of visiting all live static closures, +for which we compute retainer sets. + +A moment of thought, however, reveals that we can completely avoid computing +retainer sets for static closures. This is because retainer profiling is +concerned only about the heap, which consists of dynamic closures and no +static closures. In other words, we can treat every static closure as +a bridge connecting two dynamic closures. +For instance, if a dynamic closure @c@$_1$ has a pointer to a static +closure @s@ and @c@ has a pointer to another dynamic closure @c@$_2$, +we can think of the pointer in @c@$_1$ as a direct pointer to @c@$_2$. +The big problem is that if the graph has a cycle containing static closures, +an infinite loop occurs. In other words, we have no way of telling whether +a static closure has been visited or not and are forced to compute +retainer sets for static closures as well.\footnote{For instance, +a static closure is allowed to have a self-reference in its SRT, which +is also followed during retainer profiling.} + +Another remedy is to stores in every closure a time stamp for the +retainer set field. The time stamp indicates whether the retainer +set field is valid or no longer valid (i.e., it is for the previous +retainer profiling). +At the cost of one extra field in each closure, we can achieve an +elegant implementation with little complication. +However, it turns out that the memory overhead is too big.\footnote{A typical +dynamic closure is only two or three words long.} +Thus, our goal is to stick to the definition of the structure @StgProfHeader@ +given earlier and yet to achieve an elegant solution. + +\subsection{Basic plan} + +Since we visit every live object and update its retainer set field, +any retainer set field can either be valid (the corresponding retainer +set is valid) or point to a retainer set created during the previous +retainer profiling. +In order to distinguish valid retainer set fields +from invalid ones, we exploit the least significant bit of the retainer +set field: we maintain a one bit mark which flips over every time +retainer profiling is performed, and judge that a retainer set field is +valid only if its least significant bit matches the mark. +The variable @flip@ serves for this purpose. +The macros @isRetainerSetFieldValid()@ tests if the retainer set field +of a give closure @c@ is valid: + +\begin{code} +#define isRetainerSetFieldValid(c) \ + ((((StgWord)(c)->header.prof.rs & 1) ^ flip) == 0) +\end{code} + +As an example, a retainer set field can be set to a null value conforming +the current value of @flip@ by the macro @setRetainerSetToNull()@: + +\begin{code} +#define setRetainerSetToNull(c) \ + (c)->header.prof.rs = (retainerSet *)((StgWord)NULL | flip) +\end{code} + +Now, when a dynamic closure @c@ is created, its retainer set field is +initialized to a null value conforming to the current value of +@flip@:\footnote{Actually this is not mandatory: even when the null +value does not conform to the current value of @flip@, it will be replaced +by a correct null value when @c@ is visited later.} + +\begin{code} +extern StgWord flip; +#define SET_PROF_HDR(c,ccs_) \ + ((c)->header.prof.ccs = ccs_, (c)->header.prof.rs = (retainerSet *)((StgWord)NULL | flip)) +\end{code} + +We do not need to revise @SET_STATIC_PROF_HDR()@ if the initial value of +@flip@ is set to $0$.\footnote{For the same reason, an initial value $1$ +does not compromise the correctness of the implementation.} + +\subsection{Set of roots} + +The set of roots consists of all thread closures (running, sleeping, or +blocked) existing at the beginning of a retainer profiling. +It is handily obtained in an indirect way by invoking the function +@GetRoots()@ (in @Schedule.c@) with an appropriate argument, which must be +a function: +@GetRoots()@ invokes on each root known to the runtime system its argument. +Thus, we implement a function @retainClosure()@, which initiates traversal +from a given root and updates the retainer set of every closure reachable +from the root, +and invokes @GetRoots()@ with @retainClosure@ as an argument. + +In addition to the thread closures, weak pointers are also considered +as roots; they may not be reachable from any thread closure yet are still +being in used. +A weak pointer has three pointer fields: @key@, @value@, and +@finalizer@ (see the structure @StgWeak@ in @includes/Closures.h@). +It turns out that these pointers may not be valid all the time: +at a certain point during execution, for instance, the pointer @key@ may point +to a dead closure. +However, right after a major garbage collection, all the three pointers are +guaranteed to be valid, i.e., they all point to live closures. +This facilitates the handling of weak pointers if we choose to +perform retainer profiling immediately after a major garbage collection. +All weak pointers are found in the linked list @weak_ptr_list@ +(in @Weak.c@). + +See the function @computeRetainerSet()@ for details. + +\subsection{Static closures} + +When a live dynamic closure @c@ is visited for the first time during traversal, +its retainer set field is checked against the current value of @flip@. +If it was created at some point since the previous retainer profiling, +its retainer set field is already set to a correct null value. +Otherwise, it must have been visited +during the previous retainer profiling and thus its retainer set field is +invalid and will be set to a correct null value. +Therefore it is unnecessary to visit all dynamic closures and set their +retainer set field to a correct null value at the beginning of each retainer +profiling. + +However, this operation is required for static closures. +The reason is that a static closure, which is never garbage collected, +may appear alternately in the set of live closures. +In other words, a currently live static closure may become dead and +be resuscitated again. +Therefore, for a static closure, it does not help to check if its +retainer set field conforms to the current value of @flip@. +For instance, +if a static closure happens to belong to the set of live closures every other +retainer profiling, its retainer set field will never set to a null value, +which is disastrous. +Therefore, we choose to visit all live static closures at the beginning +of each retainer profiling and set their retainer set field to a +correct null value. + +In order to find all live static closures, we have each retainer +profiling preceded by a major garbage collection, which knows all live +static closures.\footnote{This is a heavy +restriction on retainer profiling, which makes retainer profiling partially +dependent on garbage collection. +However, it does not affect any retainer profiling result because +retainer profiling considers only live closures, which survive any +garbage collection.} +To be specific, the garbage collector builds a linked list +@scavenged_static_objects@ (in @GC.c@) during a major garbage collection, +which stores all live static closures of our interest. +\footnote{ +A static closure of closure type @IND\_STATIC@ may be put in the +list @mut\_once\_list@ of the oldest generation, instead of the list +@scavenged\_static\_objects@. +In our implementation, such a closure is just skipped over because it +contains only a pointer to a dynamic closure, and we do not compute +its retainer set. +Thus, there is no need to traverse the list @mut\_once\_list@ of the oldest +generation.} +Since it destroys the linked list after finishing the major garbage collection +(by invoking the function @zero_static_object_list()@ with +@scavenged_static_objects@ as its argument), +we traverse the linked list to set the retainer set field of each +live static closure to a correct null value before its destruction. +This is done by invoking the function +@resetStaticObjectForRetainerProfiling()@. + +\textbf{To do:} In the current implemenation, if a static closure has no child +(e.g., @CONSTR_NOCAF_STATIC@, @THUNK_STATIC@ with an empty SRT, and +@FUN_STATIC@ with an empty SRT), we do not compute its retainer set (because +there is no need to do). This slight optimization allows us to render +retainer profiling no longer dependent on garbage collection due to the +following propoerty: + +\begin{center} +A static closure can alternately appear and disappear in the set of live +closures across multiple executions of retainer profiling if and only if +it has an empty SRT and no child. +\end{center} + +Then we can completely eliminate the function +@resetStaticObjectForRetainerProfiling()@. + +\subsection{Traversal} + +The traversal proceeds in a depth first manner and is implemented +with two mutually recursive functions: @retainStack()@ and @retainerClosure()@. +@retainerStack()@ can be invoked on dynamic closures holding a stack chunk: +closure types @TSO@, @PAP@, and @AP_UPD@. +It in turn invokes @retainerClosure()@ on each closure reachable from +stack closures in the stack chunk. Notice that it does not invoke +@retainerClosure()@ on those stack closures because we do not compute +retainer sets for stack closures. +@retainerClosure()@ iteratively traverses all live closures reachable +from a given closure. +It maintains its own stack to record the next scan position in every closure +currently under consideration.\footnote{A recursive version of +@retainerClosure()@ could be implemented easily. +@retainerClosure()@ in our implementation is an iterative version.} +When it encounters a closure holding a stack chunk, it invokes @retainerStack()@ +on that closure. +Hence, +the traversal is triggered simply by invoking @retainerClosure()@ on every root. + +\textbf{To do:} +The correctness of retainer profiling is subject to the correctness +of the two macros @IS_ARG_TAG()@ and @LOOKS_LIKE_GHC_INFO()@ +(see @retainStack()@). Since +@LOOKS_LIKE_GHC_INFO()@ is a bit precarious macro, so I believe that +the current implementation may not be quite safe. Also, @scavenge_stack()@ +in @GC.c@ also exploits this macro in order to identify shallow pointers. +This can be a serious problem if a stack chunk contains some +word which looks like a pointer but is actually not a pointer. + +\subsection{Sanity check} + +Since we assume that a retainer profiling is preceded by a major garbage +collection, +we expect that the size of all the dynamic closures visited during +any retainer profiling adds up exactly to the total size of the heap. +In fact, this is not the case; there can be closures not reachable from +the set of roots yet residing in the heap even after a major garbage +collection. + +First, a dead weak pointer remains in the heap until its finalizer +finishes. Although its finalizer thread closure is part of the set of roots, +the dead weak pointer itself is not reachable from any root. +Since it cannot be visited during retainer profiling anyhow, we do not +need to located it and set its retainer set field +appropriately.\footnote{Dead weak pointers are identified with their +information table @stg\_DEAD\_WEAK\_info@ (in @StgMiscClosures.hc@). +Notice that their closure type is @CONSTR@, \emph{not} @WEAK@; +their information table is replaced by @stg\_DEAD\_WEAK\_info@ in the +function @scheduleFinalizers()@ (in @GC.c@).} + +Second, +mutable variables (of closure type @MUT_VAR@) may remain in the heap +even when they are not reachable from the set of roots while +dynamic closures pointed to by them must be live.\footnote{I do not +understand clearly why this happens :(} +Since such mutable variables may become live again (in the sense that +they become reachable from the set of roots), we must locate them +and set their retainer set field appropriately after each retainer +profiling. This is handily accomplished by traversing the list +@mut_once_list@ in every generation. + +\section{Retainer Profiling Schemes} + +A retainer profiling scheme specifies \emph{what} retainer profiling +yields (as opposed to \emph{how} retainer profiling computes the retainer +set for every live object). +It is determined primarily by the meaning of retainer identity, +that is, the type @retainer@ (in @includes/StgRetainerProf.h@). +The function @getRetainerFrom()@ must be defined according to the +definition of the type @retainer@. + +In order for a new retain profiling scheme to fully work, we need to follow +four steps: + +\begin{enumerate} +\item Define the type @retainer@ as desired. +\item Write @getRetainerFrom()@. +\item Write two hashing functions @hashkeySingletone()@ and + @hashKeyAddElement()@, which return the hash key from a single + retainer and a retainer set with another retainer, respectively. +\item Write two printing functions @printRetainer()@ and + @printRetainerSetShort()@. + These functions are employed when a retainer or a retainer set is + printed in the output file. +\end{enumerate} + +In our implementation, we use cost centre stacks for retainer identity: + +\begin{code} +typedef CostCentreStack *retainer; +\end{code} +\begin{code} +retainer getRetainerFrom(StgClosure *c) { return c->header.prof.ccs; } +\end{code} +\begin{code} +void printRetainer(FILE *f, retainer cc) +{ + fprintf(f,"%s[%s]", cc->label, cc->module); +} +\end{code} + +\textbf{To do:} All the closures created by @rts_mk...()@ in @RtsAPI.c@ are given +@CCS_SYSTEM@ as their cost centre stacks. This may not be accurate indeed, +and, for instance, @CCCS@ may be a better choice than @CCS_SYSTEM@. + +\section{Usage} + +Since cost centre stacks are used as retainer identity, a source program +must be given proper cost centre annotations by programmers. +Alternatively, +we can ask the compiler to automatically insert cost centre annotations. +For instance, the compiler option @-auto-all@ inserts a cost centre +annotation around every top-level function as shown below +(the @-p@ option is a must +because we must build the executable file in a profiling way): + +\begin{code} +$ ghc-inplace -o Foo.out -p -auto-all Foo.hs +\end{code} + +The runtime system option @-hR@ tells the executable program to +gather profiling statistics and report them in a @.prof@ file: + +\begin{code} +$ Foo.out +RTS -hR -RTS +\end{code} + +The option @-i@ can be used to +specify a desired interval at which retainer profiling is performed. +The default and minimum value is half a second: + +\begin{code} +$ Foo.out +RTS -hR -i2.5 -RTS +\end{code} + +Then, two text files are generated: a @.prof@ file and a @.hp@ file. +The @.prof@ file records the progress of retainer profiling: +for each retainer profiling performed during program execution, +it shows +the Haskell mutator time (as opposed to the user time) at which +the retainer profiling starts, +the average number of times a closure is visited, +the sum of costs assigned to all retainer sets (obtained from the field +@cost@ in each retainer set), +and the number of all retainer sets created \emph{since} the beginning +of program execution. +A typical entry in a @.prof@ file looks like: + +\begin{code} +Retainer Profiling: 3, at 3.530000 seconds + Average number of visits per object = 1.687765 + Current total costs = 801844 + Number of retainer sets = 118 +\end{code} + +The sum of costs assigned to all retainer sets may \emph{not} be equal to the +size of the heap. +The discrepancy is attributed to those live object which are not reachable +from the set of roots. +Still it is a good estimate of the size of the heap at the moment when +the retainer profiling was performed. + +The @.prof@ file also shows the contents of every retainer set which +has been assigned a positive cost (i.e., the field @cost@) at least once; +not every retainer set created is assigned a positive cost because quite +a few retainer sets are created as intermediate retainer sets before +creating a real retainer set. This results from the restriction on the way +retainer sets are created (only one retainer can be added to an existing +retainer set at a time). + +An example of the contents of a retainer set is: + +\begin{code} +SET 71 = {<doFile[Main],main[Main],MAIN[MAIN]>, <synth_2[Main],doFile[Main],main[Main],MAIN[MAIN]>} +\end{code} + +The retainer set has an identification number $71$. +It is associated with two retainers, whose retainer identities are shown +inside angle brackets @<...>@. +For instance, the first retainer is created when the cost centre stack +is @doFile[Main],main[Main],MAIN[MAIN]@, shown from the top to the bottom. +Each entry in angle brackets consists of a cost centre name (e.g., @doFile@) +and its module name (e.g., @Main@). + +The @.hp@ file can be supplied to the @hp2ps@ program to create a postscript +file showing the progress of retainer profiling in a graph: + +\begin{code} +$ hp2ps Foo.hs +$ gv Foo.ps +\end{code} + +An example of such a graph is shown in Figure~\ref{fig-cacheprof}. +It shows the cost assigned to each retainer set at the point +when a retainer profiling is performed (marked by a corresponding inverted +triangles on the horizontal axis). +The abbreviated contents of each retainer set is displayed in the right column. +Due to the space limitation, +it shows only topmost cost centres (without module names) +instead of printing the full contents. +For instance, @(71)doFile,synth_2@ corresponds to a retainer set shown above +(@71@ is its identification number). +The contents may be truncated if it is too long. + +Notice that the time is in the Haskell mutator time, which excludes +the runtime system time such as garbage collection time and retainer profiling +time. Thus, the actual execution takes longer than indicated in the +graph. Also, the timer employed to periodically perform retainer profiling +is not perfectly accurate. Therefore, the result may slightly vary for each +execution of retainer profiling. + +\begin{figure}[ht] +\centering +\epsfig{file=cacheprof_p.eps,width=5in} +\caption{A graph showing the progress of retainer profiling} +\label{fig-cacheprof} +\end{figure} + +\section{Comparision with nhc} + +\section{Files} + +This section gives a summary of changes made to the GHC in +implementing retainer profiling. +Only three files (@includes/StgRetainerProf.h@, @RetainerProfile.c@, and +@RetainerProfile.h@) are new, and all others exist in the GHC. + +@\includes@ directory: + +\begin{description} +\item[StgRetainerProf.h] defines types @retainer@ and @retainerSet@. +\item[Stg.h] includes the header file @StgRetainerProf.h@. +\item[Closures.h] changes structure @StgProfHeader@. +\item[Constants.h] changes constants @PROF_HDR_SIZE@, @SCC_UF_SIZE@, and + @SCC_SEQ_FRAME_SIZE@. +\item[ClosureMacros.h] changes macros @SET_PROF_HDR()@ and + @SET_STATIC_PROF_HDR()@. +\item[Updates.h] changes macro @PUSH_STD_CCCS()@. +\end{description} + +@\rts@ directory: + +\begin{description} +\item[Exception.hc] changes constant @CATCH_FRAME_BITMAP@, +\item[StgStartup.hc] changes constant @STOP_THREAD_BITMAP@. +\item[StgStdThunks.hc] changes constant @RET_BITMAP@. +\item[Updates.hc] changes constant @UPD_FRAME_BITMAP@. +\item[RetainerProfile.c] implements the retainer profiling engine. +\item[RetainerProfile.h] is the header for @RetainerProfile.c@. +\item[RetainerSet.c] implements the abstract datatype @retainerSet@. +\item[RetainerSet.h] defines the interface for @retainerSet@. +\item[GC.c] invokes @resetStaticObjectForRetainerProfiling()@ in + @GarbageCollect()@. +\item[Itimer.c] changes @handle_tick()@. +\item[ProfHeap.c] changes @initHeapProfiling()@ and @endHeapProfiling()@. +\item[Profiling.c] changes @initProfilingLogFile()@ and + @report_ccs_profiling()@. +\item[Proftimer.c] declares @ticks_to_retainer_profiling@, + @performRetainerProfiling@, and @doContextSwitch@. +\item[Proftimer.h] is the header for @Proftimer.c@. Defines @PROFILING_MIN_PERIOD@, + which specifies the minimum profiling period and the default profiling period. +%\item[RtsAPI.c] implements @setRetainerField()@. +\item[RtsFlags.c] + sets @RtsFlags.ProfFlags.doHeapProfile@ and + adds a string to @usage_text[]@ in @setupRtsFlags()@. +\item[RtsFlags.h] defines constants @HEAP_BY_RETAINER@ and @RETAINERchar@. +\item[RtsStartup.c] includes the header file @RetainerProfile.h@. + Changes @shutdownHaskell()@. +\item[Schedule.c] changes @schedule()@. +\item[Stats.c] + declares @RP_start_time@, @RP_tot_time@, @RPe_start_time@, + @RPe_tot_time@. + Changes @mut_user_time_during_GC()@, @mut_user_time()@, + @stat_startExit()@, + @stat_endExit()@, and + @stat_exit()@. + Defines + @mut_user_time_during_RP()@, + @stat_startRP()@, and + @stat_endRP()@. +\item[Stats.h] is the header for @Stats.c@. +\item[StgMiscClosures.hc] redefines @stg_DEAD_WEAK_info@. +\item[Storage.c] changes @initStorage()@, @memInventory()@. +\end{description} + +\bibliographystyle{plain} +\bibliography{reference} + +\end{document} diff --git a/docs/storage-mgt/sm.tex b/docs/storage-mgt/sm.tex new file mode 100644 index 0000000000..9dee565c7d --- /dev/null +++ b/docs/storage-mgt/sm.tex @@ -0,0 +1,995 @@ +\documentclass{article} +\usepackage{code,a4wide} + +\usepackage{graphics,epsfig,epic,eepic} + +\setlength{\parskip}{0.25cm} +\setlength{\parsep}{0.25cm} +\setlength{\topsep}{0cm} +\setlength{\parindent}{0cm} +\renewcommand{\textfraction}{0.2} +\renewcommand{\floatpagefraction}{0.7} + + +% Terminology +\newcommand{\block}{block} +\newcommand{\Block}{Block} +\newcommand{\segment}{segment} +\newcommand{\Segment}{Segment} +\newcommand{\step}{step} +\newcommand{\Step}{Step} + +\newcommand{\note}[1]{{\em $\spadesuit$ #1}} + +\begin{document} +\title{The GHC Storage Manager} +\author{Simon Peyton-Jones and Sungwoo Park} + +\makeatactive +\maketitle +\section{Introduction} + +This document describes the details of the GHC storage manager, including +the interface and implementation of each of its components. + +\section{Goals} + +Storage management goals are: +\begin{itemize} +\item Generational collection, supporting multiple generations. +\item The ability to pin the allocation +area into a few pages that we hope will fit entirely in the cache. +\item Allows objects to age within a generation before getting promoted. +\item Heap can grow as needed, rather than having to be pre-sized + by the programmer. +\item We support mark/sweep/compact collection for older generations. +This is a Good Thing when the live memory approaches the available +physical memory, because it reduces paging. +\item Little OS support needed. No @mmap()@ etc. All that we require is + the ability to call @malloc()@ to allocate a new chunk of memory. + There can be intervening ``sandbars'' allocated by other programs + (e.g. DLLs or other @malloc()@'d structures) between chunks of heap. +\end{itemize} + +Language-support goals are: +\begin{itemize} +\item The garbage collector ``shorts out'' indirection objects introduced +by the mutator (notably when overwriting a thunk with an indirection). +\item The garbage collector executes selector thunks. +For example, a thunk for +@(fst x)@ where @x@ is a pointer to a pair @(a,b)@ would be +evaluated by the garbage collector to just @a@. This is an important +strategy for plugging space leaks. +\item The garbage collector traversese the code tree, as well as +the heap data structures, to find which CAFs are live. This is a royal pain. +\item The garbage collector finalises some objects (typically a tiny minority). +At the moment ``finalisation'' means ``call a C routine when this thing +dies'' but it would be more general to schedule a call to a Haskell +procedure. +\end{itemize} + +Instrumentation goals are: +\begin{itemize} +\item The garbage collector can gather heap-census information for profiling. +To this end we can force GC to happen more often than it otherwise would, +and the collector can gather information about the type and cost-centre +associated with each heap object. +\end{itemize} + +\section{The architecture of the storage manager} + +The storage manager is a component of the GHC system which is responsible +for allocating fresh memory for new objects and reclaiming memory +that is no longer used. +It is built on a layered architecture and consists of four main parts: +\emph{megablock allocator}, \emph{block allocator}, \emph{heap allocator}, +and \emph{garbage collector} (Figure~\ref{fig-architecture}). +The megablock allocator communicates directly with the underlying +operating system and forms the lowest level of the storage manager. +The heap allocator and garbage collector lie in the topmost level of +the storage manager and process requests from +the mutator (the Haskell realm at the runtime) and the runtime system. +The block allocator lies between the two levels. + +\begin{figure}[ht] +\begin{center} +\input{architecture.eepic} +\caption{The overall architecture of the storage manager} +\label{fig-architecture} +\end{center} +\end{figure} + +\section{The megablock allocator} + +% need more elaboration - Sung +The megablock allocator implements a direct interface to the underlying +operating system. +It can request a chunk of physical memory of a fixed size, +which is called a \emph{megablock}, from the operating system and returns it +to the block allocator. A new megablock is not initialized by the +megablock allocator; it is later initialized by the block allocator. + +\subsection{Interface} + +\begin{description} +\item[@void *getMBlock()@] allocates a single megablock and returns its +starting address. +\item[@void *getMBlocks(nat n)@] allocates @n@ contiguous megablocks +and returns their starting address. +\end{description} + +\subsection{Implementation} + +Since the megablock allocator communicates directly with the underlying +operating system, its implementation relies on memory allocation functions +provided by the operating system; thus, the implementation varies between +platforms. +However, every megablock is always of a fixed size $2^M$ and aligned on a +$2^M$ boundary, regardless of the platform +(@MBLOCK_SIZE@ in @include/Constants.h@ defines the size of megablocks). +@mblocks_allocated@ in @MBlock.c@ stores the number of megablocks allocated. + +For implementation details, see @MBlock.c@, @MBlock.h@, @include/Block.h@. + +\section{The block allocator} + +The block allocator divides a megablock returned by the megablock allocator +into a contiguous group of \emph{block descriptors} followed by another +contiguous group of \emph{blocks}. + +A block is a contiguous chunk of $2^K$ bytes, starting on +a $2^K$-byte boundary (@BLOCK_SIZE@ in +@include/Constants.h@ defines the size of blocks). +Each block has its own associated block descriptor, which records the +current state of the block. + +Figure~\ref{fig-megablock} shows a megablock after initialization by the +megablock allocator. +Block descriptors occupy the lower address space and blocks the higher address +space in the megablock. +A block is the unit of allocation for the block allocator. +That is, the block allocator hands over store to the heap allocator in multiples of +one block, where multiple heap objects may be allocated. +A contiguous group of blocks, which is called a \emph{block group}, can be +directly handed over to the heap allocator to reduce inter-block +linkage costs. +The first block of a block group is called the \emph{group head}.\footnote{ +An alternative design has the block descriptor at the start of each block. +This makes it easy to locate the block descriptor corresponding to a particular +block, but is pessimistic for cache locality when fiddling with block descriptors. +It also means that only the first block in a contiguous chunk of blocks can +have a block descriptor. This in turn makes it difficult to achieve an +efficient mostly-copying conservative (MCC) garbage collector.} +Since block descriptors are ordered linearly, we can always locate a block +descriptor corresponding to a particular block from the starting address +of the block. + +\begin{figure}[ht] +\begin{center} +\input{megablock.eepic} +\caption{A megablock after initialization} +\label{fig-megablock} +\end{center} +\end{figure} + +\subsection{Interface} + +\begin{description} +\item[@typedef struct bdescr@] is the type of block descriptors. +\item[@void initBlockAllocator(void)@] initializes the block allocator. +\item[@bdescr *allocBlock(void)@] requests a single block and returns +the address of its block descriptor. +\item[@bdescr *allocGroup(nat n)@] allocates a block group of size @n@ +and returns the address of the block descriptor for the group head. +\item[@void freeGroup(bdescr *p)@] frees the block group where @p@ points +to the block descriptor of the group head, and places it in a pool of +free block groups. +\item[@bdescr *Bdescr(StgPtr p)@] takes a pointer @p@ to any byte within +a block and returns a pointer to its block descriptor. It is implemented as +an @inline@ procedure. +\end{description} + +\subsection{Block descriptors} + +A block descriptor has the following structure, defined in +@include/Blocks.h@: + +\begin{code} +typedef struct _bdescr { + StgPtr start; + StgPtr free; + StgWord32 blocks; + struct _bdescr *link; + /* additional fields */ +} bdescr; +\end{code} + +The fields of a block descriptor have the following purposes: + +\begin{description} +\item[@start@] points to the first byte of the corresponding block. +\item[@free@] For a group head, @free@ points to the first free byte in +the block group. For a non-group head, @free@ is set to zero to identify +the corresponding block as a non-group head. +\item[@blocks@] For a group head, @blocks@ stores the number of blocks +in the block group. It is not used for non-group heads. +\item[@link@] For a group head, @link@ is used to chain all individual +blocks or block groups together. For a non-group head, @link@ points +to the block descriptor of the group head. +\end{description} + +\subsection{Implementation} + +The block allocator maintains a linked list of free block groups, whose head +is stored in @free_list@ in @BlockAlloc.c@ (Figure~\ref{fig-freelist}). +When @allocBlock()@ or @allocGroup()@ is called, the block allocator +scans the linked list from @free_list@ and finds the first block group +which can handle the request. +If such a block group exists, it takes off the requested number of blocks +from the block group, creates a new block group from them, +initializes it if needed, and returns it to the caller. +The rest of the old block group, if any, is linked back to the list of free block +groups as another block group. +If such a block group does not exist, the block allocator requests a megablock +from the megablock allocator and processes the request using the new megablock. + +For implementation details, see @BlockAlloc.c@ and @include/Block.h@. + +\begin{figure}[ht] +\begin{center} +\input{freelist.eepic} +\caption{Linked list of free block groups} +\label{fig-freelist} +\end{center} +\end{figure} + +\section{Heap allocator} + +The role of the heap allocator in the storage manager is to allocate fresh +memory upon requests from the mutator and the runtime system. +Memory allocation takes place frequently during the execution of Haskell +programs, and hence its efficiency is crucial to the overall performance. +To handle requests from the mutator and the runtime system efficiently, +the heap allocator maintains three different memory stores, +each of which has its own purpose. + +The first store is the \emph{nursery}, where typical Haskell +objects are born. +The mutator itself can allocate fresh memory directly in the nursery +without invoking an interface function: +the configuration of the nursery is always revealed to the mutator and can even +be changed by the mutator when it allocates fresh memory from the nursery +on its own. +Thus, although the small overhead in manipulating the nursery results in fast +memory allocation, it is up to the mutator to keep the nursery in an +uncorrupted state. + +The second and the third are the \emph{small object pool} and the +\emph{large object pool}. +The heap allocator provides a common interface function to be shared by both stores: +the size of fresh memory requested, which is passed as an argument to the +interface function, determines which of the two stores to be used. +The interface function can be called by both the mutator and the runtime system. + +\subsection{Interface} + +\begin{description} +\item[@void initStorage(void)@] initializes the storage manager. @Storage.c@. +\item[@void allocNurseries(void)@] creates and initializes the nursery. +@Storage.c@. +\item[@void resetNurseries(void)@] re-initializes the nursery. @Storage.c@. +\item[@OpenNursery(hp, hplim)@] opens an allocation area in the nursery and sets +@hp@ and @hplim@ appropriately. +Then the caller can freely use the memory from @hp@ to @hpLim@. +A macro in @include/StgStorage.h@. +\item[@CloseNursery(hp)@] closes the current allocation area beginning at @hp@ +and returns it to the storage manager. +A macro in @include/StgStorage.h@. +\item[@ExtendNursery(hp, hplim)@] closes the current allocation area and +tries to find a new allocation area in the nursery. +If it succeeds, it sets @hp@ and @hplim@ appropriately and returns @rtsTrue@; +otherwise, it returns @rtsFalse@, +which means that the nursery has been exhausted. +The new allocation area is not necessarily contiguous with the old one. +A macro in @Storage.h@. +\item[@StgPtr allocate(nat n)@] allocates @n@ words from either the small +object pool or the large object pool, depending on the argument @n@, +and returns a pointer to the first byte. It \emph{always} succeeds. +@Storage.c@. +\end{description} + +\subsection{Implementation} + +The nursery is implemented with a fixed number of blocks (@nursery_blocks@ +in @Storage.c@ specifies the number of blocks). +Each of these blocks forms its own block group, and they are all linked together +by @allocNurseries()@. +The blocks in the nursery are carefully allocated in a contiguous address +range so that they fit next to each other in the cache. +They are never freed. + +A single block called the \emph{active block} provides the allocation area for +the mutator at any moment. +When the free space left in the active block is not enough for the request from +the mutator, the heap allocator sets the @free@ field in the corresponding +block descriptor to the first free byte in the block and moves the allocation +area to the next block. + +Figure~\ref{fig-nursery} shows the configuration of the nursery during +the mutator time. +The head of the linked list is stored in @MainRegTable.rNursery@, and +the address of the block descriptor of the active block is stored +in @MainRegTable.rCurrentNursery@. +@Hp@, defined as @MainRegTable.rHp@, points to the byte before the first byte of +the current allocation area in the active block. +@HpLim@, defines as @MainRegTable.rHpLim@, marks the boundary of the current +allocation area: +it points to the last byte in the current allocation area, and thus +all the bytes of memory addresses from @Hp@$ + 1$ to @HpLim@ are free. +The mutator can obtain fresh memory simply by adjusting @Hp@ as long as the new +value of @Hp@ does not exceed @HpLim@. For instance, if the mutator +increases @Hp@ by @n@, it can now store an object of size up to @n@ at the +address pointed to by the old value of @Hp@$ + 1$. + +When the runtime system runs, none of the above four pointers +(@MainRegTable.rNursery@, @MainRegTable.rCurrentNursery@, @Hp@ and @HpLim@) are +valid; they are simply aliases to registers. +Instead @g0s0->blocks@\footnote{@g0s0->blocks@ is valid all the time, even during +the mutator time. The meaning of @g0s0@ is explained in the next section.} +can be used to retrieve the head of the linked list, and +the @free@ field in each block descriptor points to the first free byte +in its corresponding block.\footnote{To be precise, this is \emph{not} the +case: a @free@ field may point to a byte past its actual boundary. +This happens because +the mutator first increases @hpLim@ without comparing it with the +actual boundary when allocating fresh memory, +and later assigns @hpLim@ to the @free@ of the corresponding block.} +@Hp@ and @HpLim@ are not saved because they can be inferred from @free@ fields +of the blocks descriptors in the nursery. + +\begin{figure}[ht] +\begin{center} +\input{nursery.eepic} +\caption{Nursery during the mutator time} +\label{fig-nursery} +\end{center} +\end{figure} + +The small object pool is implemented with a linked list of block groups, +each of which consists of a single block (Figure~\ref{fig-smallobjectpool}). +The head of the linked list is stored in @small_alloc_list@ in @Storage.c@. + +\begin{figure}[ht] +\begin{center} +\input{smallobjectpool.eepic} +\caption{Small object pool} +\label{fig-smallobjectpool} +\end{center} +\end{figure} + +The allocation in the small object pool is done in the same way as in the +nursery; @alloc_Hp@ and @alloc_HpLim@ (both defined in @Storage.c@) +point to the first free byte and the boundary of the small object pool, +respectively. +Thus, when @allocate()@ is called and the heap allocator decides to +allocate fresh memory in the small object pool, it simply increases @alloc_Hp@ +by the size of memory requested. +If the allocation cannot be done in the current small object pool, the +heap allocator calls @allocBlock()@ to obtain a new block from the block +allocator, puts it to the head of the linked list, and +sets @alloc_Hp@ and @alloc_HpLim@ appropriately. + +The large object pool is also implemented with a (doubly) linked list of block +groups (Figure~\ref{fig-largeobjectpool}). +The difference from the small object pool is that each block group stores only +a single object: each time the argument to @allocate()@ is +greater than a threshold value (computed from @LARGE_OBJECT_THRESHOLD@ +in @include/Constants.h@), a new block group accommodating the requested size +is created to store a single object. +The new block group is put to the head of the list. +The head of the linked list is available as @g0s0->large_objects@. + +\begin{figure}[ht] +\begin{center} +\input{largeobjectpool.eepic} +\caption{Large object pool} +\label{fig-largeobjectpool} +\end{center} +\end{figure} + +For implementation details, see @Storage.c@ and @include/StgStorage.h@. + +\section{Garbage collector} + +The garbage collector finds all the objects unreachable from a given set of +roots and frees the memory allocated to them. By invoking the +garbage collector regularly, the storage manager prevents the heap from +growing indefinitely and allows Haskell programs to be executed at a +reasonable memory cost. + +The garbage collector in the storage manager is based upon the generational +garbage collection algorithm. +The storage manager records the age for every object in the heap. +An object surviving one garbage collection grows old by one \emph{step}, +and an object surviving a certain number of garbage collections +is promoted to the next \emph{generation}. +That is, a step can be defined as a collection of objects which have survived +the same number of garbage collections (or a collection of objects which are +born at some point between two particular successive garbage collections), +and a generation as a group of steps belonging to a certain range of ages. +Notice that the unit of garbage collections is not step but generation: +a garbage collection applies to all the steps in a generation, and we cannot +perform a garbage collection just on part of a generation. +Furthermore, if a particular generation is garbage collected, so are +all the younger generations.\footnote{Some +authors define a generation as the set of +all the objects created between two particular garbage collection and +an object cannot change its generation (e.g., 1960's, 1970's, and so on). +In this document, +an object can change its generation as it survives garbage collections +(e.g., teenagers, 20's, and so on).} + +Figure~\ref{fig-generation} illustrates how an object grows old. +Every object is created in step $0$ of generation $0$. +As it survives garbage collections, it is moved to the next step of the +same generation until it is finally promoted to +step $0$ of the next generation: +during a garbage collection of generation $g < G$, live objects from +step $s < S_g$ are moved to step $s + 1$, and live objects from +the last step $S_g$ are promoted to step $0$ in generation $g + 1$. +Live objects in step $0$ of generation $G$ stay in the same step; +the oldest generation maintains only one step because there is no point +in aging objects in the oldest generation. +In this way, objects are given a decent chance of dying before being +promoted to the next generation. + +\begin{figure}[ht] +\begin{center} +\input{generation.eepic} +\caption{Evolution of objects through garbage collections} +\label{fig-generation} +\end{center} +\end{figure} + +The main reason that we separate steps from generations is to +reduce the cost of maintaining \emph{backward inter-generational pointers}, +that is, pointers from older generations to younger generations. +Suppose that a garbage collection applies to all generations $0$ +through $g$. If an object @O@ in one of these generations is pointed to +by another object in generation $g' > g$, we cannot free the object @O@ +even though generation $g'$ is out of consideration. Consequently +we have to track backward inter-generational pointers to perform garbage +collections correctly. +Since maintaining backward pointers is costly, we +choose to track backward inter-generational pointers only; +we do not track backward inter-step pointers. + +By grouping all the objects created between two garbage collections +and grouping multiple age groups into one generation, the garbage +collector makes an efficient use of heap memory. + +\subsection{Interface} + +\begin{description} +%\item[@StgClosure *MarkRoot(StgClosure *root)@] informs the garbage collector +%that @root@ is an object in the root set. It returns the new location of +%the object. @GC.c@. +\item[@void *mark\_root(StgClosure **root)@] informs the garbage collector +that @*root@ is an object in the root set. It replaces @*root@ by +the new location of the object. @GC.c@. +\item[@void GarbageCollect(void (*get\_roots)(evac\_fn), rtsBool force\_major\_gc)@] +performs a garbage collection. +@get_roots()@ is a function which is called by the garbage collector when +it wishes to find all the objects in the root set (other than those +it can find itself). +Therefore it is incumbent on the caller to find the root set. +@force_major_gc@ specifies whether a major garbage collection is required +or not. If a major garbage collection is not required, the garbage collector +decides an oldest generation $g$ to garbage collect on its own. +@GC.c@. +\item[@rtsBool doYouWantToGC(void)@] returns @rtsTrue@ if the garbage +collector is ready to perform a garbage collection. Specifically, it returns +@rtsTrue@ if the number of allocated blocks since the last garbage collection +(@alloc_blocks@ in @Storage.c@) exceeds an approximate limit +(@alloc_blocks_lim@ in @Storage.c@). +@Storage.h@. +\item[@void recordMutable(StgMutClosure *p)@] informs the garbage collector +that a previously immutable object @p@ has become mutable. +The garbage collector then puts the object @p@ in the list @mut_list@ of the +generation to which it belongs.\footnote{It is easy to +locate the generation to which a dynamic object belongs from its address: +we can identify the block in which the object resides from its address, +and the corresponding block descriptor stores pointers +to the step and the generation (@gen@ and @step@ fields in the @bdescr@ +structure) to which it belongs.} +It suffices to call @RecordMutable()@ only once for any object. + +For an object which is genuinely mutable (e.g., mutable arrays), +it is permanently recorded as mutable. +On the other hand, +an object which is temporarily mutable (e.g., frozen arrays), +can be dropped from the list @mut_list@ once its pointer has been dealt with +during garbage collections. @Storage.h@. +\item[@void recordOldToNewPtrs(StgMutClosure *p)@] puts the object @p@ in the +list @mut_once_list@ of the generation to which it belongs. +\item[@void newCAF(StgClosure *caf)@] puts the CAF @caf@ either +in the list @caf_list@ or +in the list @mut_once_list@ of the oldest generation, +depending on whether it is dynamically loaded or not. +\end{description} + +\subsection{Steps} + +A step has the following structure, defined in +@include/StgStorage.h@: + +\begin{code} +typedef struct _step { + unsigned int no; + bdescr *blocks; + unsigned int n_blocks; + bdescr *large_objects; + /* additional fields */ +} step; +\end{code} + +The fields of a step have the following purposes (Figure~\ref{fig-step}): + +\begin{description} +\item[@no@] indicates the age within its generation. +$0$ indicates the youngest step in a generation. +\item[@blocks@] is a linked list of all the blocks in this step +which contain small objects. +Each block forms its own block group. +\item[@n\_blocks@] is the number of blocks in the linked list @blocks@. +\item[@large\_objects@] is a (doubly) linked list of all the block groups +in this step which contain large objects. +Each block group stores only a single object. +\end{description} + +\begin{figure}[ht] +\begin{center} +\input{step.eepic} +\caption{Memory layout of a step} +\label{fig-step} +\end{center} +\end{figure} + +The linked list @blocks@ of step $s$ in generation $g$ is created +during a garbage collection +from live small objects of step $s - 1$ in the same generation +(or the last step in the previous generation if $s = 0$). +The @free@ field in every block descriptor never changes because +no objects are added after the garbage collection; new objects are created +only in step $0$ in generation $0$. +Likewise, the linked list @large_objects@ is created during a +garbage collection from live large objects of the previous step. + +There are three exceptions to the above rules. +First, both @blocks@ and @large_objects@ of +step $0$ in generation $0$ are not filled with new objects during a garbage +collection. +They are simply re-initialized by the garbage collector and +grow during during the execution of a program as new objects are +created. +Step $0$ in generation $0$ is accessible via a global variable @g0s0@, +and this is the reason why the large object pool (described in the previous +section) is indeed stored in @g0s0->large_objects@. +For the same reason, @MainRegTable.rNursery@ holds the same address as +@g0s0->blocks@ during the mutator time. +Second, @blocks@ of step $1$ in generation $0$ is created not only from +the nursery (@blocks@ of step $0$ in the same generation) but also from the +small object pool. In other words, all the live small objects created since +the previous garbage collection, either directly by the mutator or indirectly +through @allocate()@, are gathered together in the same linked list. +Finally, step $0$ of the oldest generation serves the source for itself during +any garbage collection, i.e., $S_G = 1$, because there exists no older step. + +\subsection{Generations} + +A generation has the following structure, defined in +@include/StgStorage.h@: + +\begin{code} +typedef struct _generation { + unsigned int no; + step *steps; + unsigned int n_steps; + unsigned int max_blocks; + StgMutClosure *mut_list; + StgMutClosure *mut_once_list; + /* additional fields */ +} generation; +\end{code} + +The fields of a generation have the following purposes (Figure~\ref{fig-gen}): + +\begin{description} +\item[@no@] is the generation number. +\item[@steps@] points to an array of @step@ structures. @steps[@$i$@]@ +corresponds to step $i$ in this generation, i.e., +@steps[@$i$@].no@ is equal to $i$. +\item[@n\_steps@] is the number of @step@ structures in the array pointed to +by @steps@. +\item[@max\_blocks@] is the maximum number of blocks allowed in step $0$ of +this generation. If the number of blocks allocated +in step @0@ exceeds @max_blocks@, +this generation is garbage collected during the next garbage collection. +\item[@mut\_list@] links all mutable objects in this generation, that is, +objects whose contents can be updated and hence may contain pointers to +younger generations. +Every object in this linked list is a dynamic object residing in the heap +and has a structure compatible with @StgMutClosure@. +The structure @StgMutClosure@ (@includes/Closures.h@) has a field +@mut_link@ (called a mutable link field) of type @StgMutClosure *@, which +points to the next object in this linked list. +The end mark of this linked list is a pointer to a statically allocated object +@END_MUT_LIST@ (@StoragePriv.h@). +\item[@mut\_once\_list@] links objects in this generation whose contents +cannot be updated any more but may already have pointers to younger generations. +As with @mut_list@, it links only those objects whose structure is compatible +with @StgMutClosure@ and ends with @END_MUT_LIST@. +\end{description} + +\begin{figure}[ht] +\begin{center} +\input{gen.eepic} +\caption{Memory layout of a generation} +\label{fig-gen} +\end{center} +\end{figure} + +The garbage collector maintains an array @generations@ of @generation@ structure +(defined in @Storage.c@), whose size is stored in a runtime system flag +(@RtsFlags.GcFlags.generations@). +The generation number of each generation coincides with its index into +the array @generations@, i.e., @generations[@$i$@].no@ is equal to $i$. + +As mentioned before, lists of objects which may have pointers to younger +generations are kept per generation, not per step. The youngest generation, +accessible via a global variable @g0@, does not keep such a list because it +does not have younger generations. + +The oldest generation, accessible via a global variable @oldest_gen@, may +contain static objects (as opposed to dynamic objects residing in the heap) +in its list @mut_once_list@. This happens when a static +thunk, also known as a \emph{constant applicative form} (CAF), is entered. +When a CAF (corresponding to closure type @THUNK_STATIC@, defined +in @includes/ClosureTypes.h@) is entered, +it is first put in the list @mut_once_list@ of the oldest generation +and then overwritten with an appropriate static indirection object +(corresponding to closure type @IND_STATIC@).\footnote{Actually a static +indirection object does not have a @mut\_link@ field. +We use its @static\_link@ field as a substitute for @mut\_link@. +See the structure @StgIndStatic@ in @include/Closures.h@.}\footnote{For +details of this operation, see the macro @UPD\_CAF()@ in @includes/Updates.h@} +If the CAF is dynamically loaded (e.g., in an interactive environment), it is +instead put in a separate linked list @caf_list@ +(declared in @Storage.c@). + +The evaluation result of the +CAF is stored in a separate dynamic object in the heap and the static +indirection object has a pointer to the dynamic object. +Thus, the new static indirection object is put in the list +@mut_once_list@ of the oldest generation (or the list @caf_list@) so that the +dynamic object is not removed during the next garbage collection. +Once it is created, the static indirection object remains unaltered, which +is the reason why it is put in the @mut_once_list@ list, not in the +@mut_list@ list. +Since the static indirection object survives any garbage collection (because +it comes from a static object) and would be eventually moved to the oldest +generation, +we put it in the @mut_once_list@ of the oldest generation as soon +as it is created. + +\subsection{Implementation} + +The overall structure of a garbage collection is as follows: + +\begin{enumerate} +\item[(1)] Initialize. +\item[(2)] Scavenge lists @mut_once_list@ and @mut_list@ if necessary. +\item[(3)] Scavenge CAFs. +\item[(4)] Evacuate roots. +\item[(5)] Scavenge objects. +\item[(6)] Tidy up. +\end{enumerate} + +\subsubsection{(1) Initialization} + +During initialization, the garbage collector first decides which generation +to garbage collect. +Specifically, +if the argument @force_major_gc@ to @GarbageCollect()@ is @rtsFalse@, +it decides the greatest generation number $N$ such +that the number of blocks allocated in step $0$ of generation $N$ exceeds +@generations[@$N$@].max_blocks@. +If the argument @force_major_gc@ to @GarbageCollect()@ is @rtsTrue@, +$N$ is set to the greatest generation number, namely, +$@RtsFlags.GcFlags.generations@ - 1$. +The garbage collector considers up to generation $N$ for garbage collection. +A major garbage collection takes place if $N$ is set to +$@RtsFlags.GcFlags.generations@ - 1$ during this process. + +Then, the garbage collector initialize the \emph{to-space} (as opposed to +\emph{from-space}) for each step of +each generation, which is complete with an \emph{allocation pointer} and +an \emph{sweep pointer}. +The to-space of a step is the memory to which any object belonging to the +step can be copied when it survives a garbage collection. +For instance, a live object in step $s$ of generation $g$ can first be copied +to the to-space associated with step $s$, which eventually becomes +associated with the next step $s + 1$ (or step $0$ of the next generation) +during tidying up. +This operation effectively moves an object to the next step if it survives +a garbage collection. +The allocation pointer points to the next free in the to-space while +the sweep pointer points to the next object considered for scavenging. + +During major garbage collections, +the static link field of every static object indicates whether it has +been visited by the garbage collector or not. +Therefore, the static link field of every static object must have +a null value before a major garbage collection starts. +The list @mut_once_list@ of the oldest generation may contain static +indirection objects, and thus +the garbage collector invokes @zero_mutable_list()@ on the list, +Although this breaks up the list, it does not cause any problem because +the list is not employed during major garbage collections. + +\subsubsection{\tt evacuate()} + +The function @evacuate()@ (defined in @GC.c@), which +is called eventually for every live object +(including even static objects reachable from roots), +moves an object to +a safe place so as not to be garbage collected. +Before invoking the function @evacuate()@ on an object @o@, the caller specifies +a \emph{desired generation} for @o@ in a variable @evac_gen@ +(declared in @GC.c@). +The desired generation is the youngest generation to which the caller wishes +@o@ to be evacuated; the garbage collector should evacuate @o@ to a +generation no younger than the desired generation. + +Depending on @evac_gen@ and the generation $M$ where @o@ currently resides, +@evacuate()@ behaves itself as follows: +\begin{itemize} +\item If @evac_gen@ $\leq M$ and $N < M$, it does nothing because @o@ is already + in a generation no younger than @evac_gen@. +\item If @evac_gen@ $\leq M \leq N$, it evacuates @o@ to the to-space of the +step to which @o@ currently belongs. @o@ will be moved to the next step later. +@recordMutable()@ may be invoked on @o@ depending on its type (e.g., @MVAR@). +\item If $M <$ @evac_gen@, @o@ is evacuated to the to-space of step $0$ + of generation @even_gen@, which accomplishes the request. + This happens even when $N \leq$ @evac_gen@. Therefore, those generations + which are not considered for garbage collection may still be augmented + with new objects during garbage collection. + @recordMutable()@ may be invoked on @o@ depending on its type. +\end{itemize} +If @o@ has already been evacuated, @evacuate()@ either does nothing (when +@even_gen@ $\leq M$) or reports +a failure to evacuate @o@ by setting the flag @failed_to_evac@ (declared +in @GC.c@). + +Evacuating a large object is handled by @evacuate_large()@. +Since it is costly to allocate new memory blocks and copy all the contents +of the object, the garbage collector simply removes the object form +the list @large_alloc_list@ of its step and links it to another list, +from which it will be scavenged later. + +\subsubsection{Set of roots for garbage collection} +Part of the set of roots for garbage collection is obtained indirectly by +invoking the function +@get_roots()@, an argument to @GarbageCollect()@: the garbage collector +invokes @get_roots()@ with @mark_root()@ as an argument, and @get_roots()@ +in turn invokes @mark_root()@ on each of known roots. +The rest of the set of roots is obtained from the lists @mut_list@ and +@mut_once_list@ of generation $N + 1$ through the oldest generation: +any objects in these lists may have pointers to objects in generations +$0$ to $N$, and thus must be considered as a root. +If a major garbage collection takes place, no @mut_list@ and @mut_once_list@ +lists are consider for scavenging and step (2) is skipped. +The entire set of roots is now specified by @get_roots()@ alone. + +\subsubsection{(2) Scavenging lists {\tt mut\_once\_list} and {\tt mut\_list}} + +Since the roots obtained from the lists @mut_list@ and @mut_once_list@ are +already in generations $N' > N$, we only have to scavenge them. +That is, it suffices to invoke @evacuate()@ once on each object +which is currently pointed to by an object in these lists. + +When scavenging an object @r@ in the list @mut_once_list@ of generation $M$, +the desired generation is set to $M$ for each object @o@ pointed +to by @r@ before invoking @evacuate()@. +The rationale is that the contents of @r@ cannot be updated any more, +and thus @r@ is always survived by @o@; @o@ is live as long as @r@ is. +Therefore, we wish @r@ to be evacuated to the same generation $M$ as @r@ +currently resides (not to its next step). +If the evacuation succeeds (indicated by a @rtsFalse@ value of a variable +@failed_to_evac@, declared in @GC.c@) for every object @o@, @r@ is removed +from the list @mut_once_list@ because it does not hold any backward +inter-generational pointers.\footnote{It turns out that @r@ can have only +one such object @o@. The type of @r@ is one of the following: +@IND\_OLDGEN@, @IND\_OLDGEN\_PERM@, @IND\_STATIC@, and @MUT\_VAR@.} + +Scavenging a list @mut_list@ is similar to the case of @mut_once_list@. +When scavenging an object @r@ in the list @mut_list@ of generation $M$, +the desired generation is set to $M$ for each object pointed to by @r@ +if @r@ is known to be immutable (e.g., @MUT_ARR_PTRS_FROZEN@, +@IND_OLDGEN@) +or to $0$ if @r@ is still mutable (e.g., @MUT_ARR_PTRS@, @MUT_VAR@). +The list @mut_once_list@ is also adjusted if it is safe to remove @r@ from +@mut_list@. + +\subsubsection{(3) Scavenging CAFs} + +When a dynamically loaded CAF is entered, it it first put to the list +@caf_list@ and then overwritten with a static indirection object. +The evaluation result of the CAF is stored in a dynamic object in the heap +and the static indirection object stores a pointer to the dynamic object. +Although the static indirection object (or the CAF) itself is never freed, +it may be removed later from the @caf_list@ when it is reverted to the +original CAF, and the dynamic object may not be live afterwards. +Hence, we treat the dynamic object just as normal dynamic objects and +set the desired generation to $0$. + +\subsubsection{(4) Evacuating roots} + +Evacuating roots (other than those in the lists @mut_once_list@ and +@mut_list@) is simply done by invoking @get_roots()@ with @mark_root()@ +as an argument. +Since these roots are normal dynamic objects, we set the desired generation +to $0$. + +\subsubsection{(5) Scavenging} + +The garbage collector scavenges all the objects in the to-space of +each step (by invoking @evacuate()@ on each object reachable from them) +until every sweep pointer has reached its corresponding +allocation pointer. +It repeatedly examines all the to-spaces because not only sweep pointers +but also allocation pointers change during scavenging: +when an object @r@ is scavenged, each object reachable from +@r@ is evacuated to a certain to-space, which increases the corresponding +allocation pointer, and +the sweep pointer of the to-space which currently contains @r@ +increases as well upon finishing scavenging the object @r@. +Thus, the garbage collector cannot anticipate in advance how many times +it needs to scan through all the to-spaces; it keeps scavenging until +no objects are left to be scavenged. + +\subsubsection{Scavenging static objects} + +Since it is possible for dynamic objects to point to static objects, +the garbage collector may invoke @evacuate()@ on static objects +while scavenging dynamic objects in to-spaces. +This complicates the garbage collector because +static objects cannot be evacuated in general yet +they may have pointers to dynamic objects, which must be evacuated. +Thus the garbage collector needs to at least scavenge live static objects +(as opposed to those static objects currently not reachable from roots). + +When a minor garbage collection is performed, any invocation of +@evacuate()@ on static objects is simply ignored. +Furthermore, no static object is considered for scavenging +(except those in the list @mut_once_list@ of the oldest generation during). +Still all dynamic objects which are marked as live due to static objects +are safely evacuated. +The reason is that we can reach all such dynamic objects from +indirection static objects stored in the list +@mut_once_list@ of the oldest generation, which is scavenged during step (2), +and the list @caf_list@. +In other words, in order to evacuate all such dynamic objects, it is +sufficient to evacuate all dynamic objects reachable from +static indirection objects in +the list @mut_once_list@ of the oldest generation and the list @caf_list@. +However, the garbage collector may unnecessarily scavenge certain static +indirection objects which are no longer used. +They are not scavenged during a major garbage collection, however. + +During a major garbage collection, +if an invocation of @evacuate()@ on a static object @r@ is made, +the garbage collector first checks whether @r@ needs to be scavenged or not. +If its SRT (Static Reference Table) is empty and it has no other pointers, +no dynamic objects are reachable from @r@ and it is ignored.\footnote{If +no dynamic objects are reachable from a static object @r@ (even indirectly +via multiple static objects), +@r@ is not stored in \emph{any} SRT table because it would be no use attempting +to follow any pointers in @r@.} +Otherwise, it is put in the list @static_objects@. +At the beginning of each scavenging loop in step (5), +the garbage collector invokes @scavenge_static()@ if the list @static_objects@ +is not empty. +@scavenge_static()@ scavenges the static objects in the list @static_objects@ +by invoking @evacuate()@ on every object reachable from them. +The desired generation is set to the oldest generation (because any +dynamic object directly pointed to by a static object lives +forever). +These static objects are then put in another list @scavenged_static_objects@ +and removed from the list @static_objects@. +For a static indirection object, if the evacuation +fails, it is put back to the list @mut_once_list@ of the oldest generation; +it can be thought of as a CAF just entered. + +After a major garbage collection, therefore, the list @scavenged_static_objects@ +links all live static objects except for static indirection objects put back +to the list @mut_once_list@ of the oldest generation. +Dynamically loaded CAFs are found in the list @caf_list@. + +\subsubsection{(6) Tidying up} + +The garbage collector tidies up the heap by +moving the to-space of each step to the next step. +It also re-initialize the small object pool (which now does not contain +any live objects), frees any large objects which have not been scavenged, +and invokes @resetNurseries()@. +If a major garbage collection has been performed, it +invokes @zero_static_object_list()@ on the list @scavenged_static_objects@ +so that all static objects +(other than those in the list @mut_once_list@ of the oldest generation) +have a null static link field again. + +At this point, both the small allocation pool and the large object pool are +empty. Upon the exit from @GarbageCollect()@, however, they may not +be empty any more because the garbage collector invokes @scheduleFinalizer()@ +before exiting, which tries to run pending finalizers on dead weak pointers and +may create new objects through @allocate()@. +The nursery still remains intact. + +The heap may contain extra objects which are not reachable from the roots +used during the garbage collection: 1) weak head pointers; 2) dead +weak head pointers. Weak head pointers can be tracked from +the list @weak_ptr_list@ (declared in @Weak.c@). However, there is no way +of reaching dead weak pointers; they will be garbage collected during the +next garbage collection. + +For implementation details, see @GC.c@. + +\section{State of the heap allocator and the garbage collector} + +The state of the heap allocator and the garbage collector is fully specified by the +following variables: + +\begin{description} +\item[@small\_alloc\_list@] is the header of the small object pool. +\item[@alloc\_Hp@] points to the first free byte in the small object pool. +\item[@alloc\_HpLim@] points to the boundary of the small object pool. +\item[@generations@] is the array of @generation@ structures. +\item[@RtsFlags.GcFlags.generations@] specifies the number of elements in +the array @generations@. +\item[@caf\_list@] links dynamically loaded CAFs. +\end{description} + +\textbf{To do:} check if this is a complete list. + +The following variables are derivable, but they are given special purposes: + +\begin{description} +\item[@g0s0@] points to step 0 of the youngest generation. +\item[@oldest\_gen@] points to the oldest generation. +\item[@g0s0->blocks@] is the header of the nursery. +\item[@g0s0->large\_blocks@] is the header of the large object pool. +\end{description} + +\section{Miscellaneous notes} + +\begin{itemize} +\item To see how to add new fields to Haskell closures, +see the document on the implementation of retainer profiling +(section `Adding Retainer Set Fields'). + +\item To see how to traverse the graph and visit every live closure, +see the document on the implementation of retainer profiling +(section `Graph Traversal'). + +\item To see how to linearly scan the heap at any random moment during +program execution, see the document on the implementation of LDVU profiling +(section `Heap Censuses'). + +\item To see how to linearly scan the from-space during garbage collections, +see the document on the implementation of LDVU profiling +(section `Destruction of Closures'). + +\end{itemize} + +\end{document} diff --git a/docs/storage-mgt/smallobjectpool.eepic b/docs/storage-mgt/smallobjectpool.eepic new file mode 100644 index 0000000000..0ccf61c3fb --- /dev/null +++ b/docs/storage-mgt/smallobjectpool.eepic @@ -0,0 +1,65 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(10062,5607)(0,-10) +\path(3375,5262)(4950,5262)(4950,4062) + (3375,4062)(3375,5262) +\path(4125,5112)(4125,5562)(6750,5562)(6750,5262) +\path(6720.000,5382.000)(6750.000,5262.000)(6780.000,5382.000) +\path(6750,5262)(10050,5262)(10050,4062) + (6750,4062)(6750,5262) +\path(6870.000,4692.000)(6750.000,4662.000)(6870.000,4632.000) +\path(6750,4662)(8625,4662) +\path(8505.000,4632.000)(8625.000,4662.000)(8505.000,4692.000) +\path(8625,5262)(8625,4062) +\path(8025,3387)(8625,3387)(8625,4062) +\path(8655.000,3942.000)(8625.000,4062.000)(8595.000,3942.000) +\path(8400,2937)(10050,2937)(10050,4062) +\path(10080.000,3942.000)(10050.000,4062.000)(10020.000,3942.000) +\path(3525,4212)(2925,4212)(2925,2712) +\path(2895.000,2832.000)(2925.000,2712.000)(2955.000,2832.000) +\path(1950,4962)(3375,4962) +\path(3255.000,4932.000)(3375.000,4962.000)(3255.000,4992.000) +\path(2925,2262)(2925,1737)(3300,1737) +\path(3180.000,1707.000)(3300.000,1737.000)(3180.000,1767.000) +\path(3300,1812)(4875,1812)(4875,612) + (3300,612)(3300,1812) +\path(4050,1662)(4050,2112)(6675,2112)(6675,1812) +\path(6645.000,1932.000)(6675.000,1812.000)(6705.000,1932.000) +\path(9750,1812)(9750,612) +\path(6675,1812)(9975,1812)(9975,612) + (6675,612)(6675,1812) +\path(3450,762)(2850,762)(2850,237) +\path(2820.000,357.000)(2850.000,237.000)(2880.000,357.000) +\path(6795.000,1242.000)(6675.000,1212.000)(6795.000,1182.000) +\path(6675,1212)(9750,1212) +\path(9630.000,1182.000)(9750.000,1212.000)(9630.000,1242.000) +\path(3900,1362)(5850,1362)(5850,12) + (9750,12)(9750,612) +\path(9780.000,492.000)(9750.000,612.000)(9720.000,492.000) +\put(3450,5037){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3600,4137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(3450,4437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(7425,5412){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(6900,4812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\put(8850,4812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(8850,4527){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}memory}}}}} +\put(2700,2487){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(0,4887){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}small\_alloc\_list}}}}} +\put(6825,3312){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}alloc\_Hp}}}}} +\put(6600,2862){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}alloc\_HpLim}}}}} +\put(3375,1587){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3525,687){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(3375,987){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(7350,1962){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(7350,1362){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\put(2625,12){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3375,1302){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/smallobjectpool.fig b/docs/storage-mgt/smallobjectpool.fig new file mode 100644 index 0000000000..afcfe9862d --- /dev/null +++ b/docs/storage-mgt/smallobjectpool.fig @@ -0,0 +1,74 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6225 3900 7800 3900 7800 5100 6225 5100 6225 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 6975 4050 6975 3600 9600 3600 9600 3900 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9600 3900 12900 3900 12900 5100 9600 5100 9600 3900 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9600 4500 11475 4500 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 11475 3900 11475 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 10875 5775 11475 5775 11475 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 3 + 0 0 1.00 60.00 120.00 + 11250 6225 12900 6225 12900 5100 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 6375 4950 5775 4950 5775 6450 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2 + 0 0 1.00 60.00 120.00 + 4800 4200 6225 4200 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 5775 6900 5775 7425 6150 7425 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6150 7350 7725 7350 7725 8550 6150 8550 6150 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 6900 7500 6900 7050 9525 7050 9525 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12600 7350 12600 8550 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9525 7350 12825 7350 12825 8550 9525 8550 9525 7350 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 6300 8400 5700 8400 5700 8925 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9525 7950 12600 7950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 5 + 0 0 1.00 60.00 120.00 + 6750 7800 8700 7800 8700 9150 12600 9150 12600 8550 +4 0 0 50 0 0 17 0.0000 4 150 435 6300 4125 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6450 5025 link\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6300 4725 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10275 3750 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 9750 4350 used memory\001 +4 0 0 50 0 0 17 0.0000 4 165 390 11700 4350 free\001 +4 0 0 50 0 0 17 0.0000 4 180 825 11700 4635 memory\001 +4 0 0 50 0 0 17 0.0000 4 30 360 5550 6675 ......\001 +4 0 0 50 0 0 17 0.0000 4 195 1575 2850 4275 small_alloc_list\001 +4 0 0 50 0 0 17 0.0000 4 225 900 9675 5850 alloc_Hp\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 9450 6300 alloc_HpLim\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6225 7575 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 8475 link\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6225 8175 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10200 7200 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 10200 7800 used memory\001 +4 0 0 50 0 0 17 0.0000 4 30 360 5475 9150 ......\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6225 7860 free\001 diff --git a/docs/storage-mgt/step.eepic b/docs/storage-mgt/step.eepic new file mode 100644 index 0000000000..d5af2b7b04 --- /dev/null +++ b/docs/storage-mgt/step.eepic @@ -0,0 +1,121 @@ +\setlength{\unitlength}{0.00050000in} +% +\begingroup\makeatletter\ifx\SetFigFont\undefined% +\gdef\SetFigFont#1#2#3#4#5{% + \reset@font\fontsize{#1}{#2pt}% + \fontfamily{#3}\fontseries{#4}\fontshape{#5}% + \selectfont}% +\fi\endgroup% +{\renewcommand{\dashlinestretch}{30} +\begin{picture}(10749,10689)(0,-10) +\path(7437,4362)(10737,4362)(10737,3162) + (7437,3162)(7437,4362) +\path(7557.000,3792.000)(7437.000,3762.000)(7557.000,3732.000) +\path(7437,3762)(10587,3762) +\path(10467.000,3732.000)(10587.000,3762.000)(10467.000,3792.000) +\path(10587,4362)(10587,3162) +\put(8637,4437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\put(8412,3912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single object}}}}} +\path(7437,2262)(10737,2262)(10737,1062) + (7437,1062)(7437,2262) +\path(7557.000,1692.000)(7437.000,1662.000)(7557.000,1632.000) +\path(7437,1662)(10587,1662) +\path(10467.000,1632.000)(10587.000,1662.000)(10467.000,1692.000) +\path(10587,2262)(10587,1062) +\put(8637,2337){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\put(8412,1812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single object}}}}} +\path(3912,2262)(5487,2262)(5487,1062) + (3912,1062)(3912,2262) +\path(4662,2112)(4662,2562)(7437,2562)(7437,2262) +\path(7407.000,2382.000)(7437.000,2262.000)(7467.000,2382.000) +\path(4812,1812)(4812,2562) +\path(5487,2262)(5937,2262)(5937,1062) + (5487,1062)(5487,2262) +\path(5937,2262)(6387,2262)(6387,1062) + (5937,1062)(5937,2262) +\path(6387,2262)(6837,2262)(6837,1062) + (6387,1062)(6387,2262) +\put(3987,2037){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3987,1737){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(4137,1137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(6087,1662){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(3987,1437){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_2$}}}}} +\path(3912,9912)(5487,9912)(5487,8712) + (3912,8712)(3912,9912) +\path(4662,9762)(4662,10212)(7287,10212)(7287,9912) +\path(7257.000,10032.000)(7287.000,9912.000)(7317.000,10032.000) +\path(10362,9912)(10362,8712) +\path(4812,9462)(4812,10212) +\path(7287,9912)(10587,9912)(10587,8712) + (7287,8712)(7287,9912) +\path(4812,9462)(4812,10662)(10362,10662)(10362,9912) +\path(10332.000,10032.000)(10362.000,9912.000)(10392.000,10032.000) +\path(7407.000,9342.000)(7287.000,9312.000)(7407.000,9282.000) +\path(7287,9312)(10362,9312) +\path(10242.000,9282.000)(10362.000,9312.000)(10242.000,9342.000) +\put(3987,9687){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3987,9387){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(4137,8787){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(3987,9087){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(7962,10062){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(7962,9462){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\path(3462,7587)(3462,7062)(3837,7062) +\path(3717.000,7032.000)(3837.000,7062.000)(3717.000,7092.000) +\path(3912,7362)(5487,7362)(5487,6162) + (3912,6162)(3912,7362) +\path(4662,7212)(4662,7662)(7287,7662)(7287,7362) +\path(7257.000,7482.000)(7287.000,7362.000)(7317.000,7482.000) +\path(10362,7362)(10362,6162) +\path(4812,6912)(4812,7662) +\path(7287,7362)(10587,7362)(10587,6162) + (7287,6162)(7287,7362) +\path(4812,6912)(4812,8112)(10362,8112)(10362,7362) +\path(10332.000,7482.000)(10362.000,7362.000)(10392.000,7482.000) +\path(7407.000,6792.000)(7287.000,6762.000)(7407.000,6732.000) +\path(7287,6762)(10362,6762) +\path(10242.000,6732.000)(10362.000,6762.000)(10242.000,6792.000) +\put(3237,7812){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3987,7137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3987,6837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3987,6537){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=1}}}}} +\put(7962,7512){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}single block}}}}} +\put(7962,6912){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}used memory}}}}} +\put(3987,6237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link=NULL}}}}} +\path(4062,8862)(3462,8862)(3462,8112) +\path(3432.000,8232.000)(3462.000,8112.000)(3492.000,8232.000) +\path(3942.000,1182.000)(4062.000,1212.000)(3942.000,1242.000) +\path(4062,1212)(3462,1212)(3462,12)(3912,12) +\path(3792.000,-18.000)(3912.000,12.000)(3792.000,42.000) +\path(3942.000,3282.000)(4062.000,3312.000)(3942.000,3342.000) +\path(4062,3312)(3462,3312)(3462,2112)(3912,2112) +\path(3792.000,2082.000)(3912.000,2112.000)(3792.000,2142.000) +\path(3912,4362)(5487,4362)(5487,3162) + (3912,3162)(3912,4362) +\path(4812,3912)(4812,4662) +\path(5487,4362)(5937,4362)(5937,3162) + (5487,3162)(5487,4362) +\path(5937,4362)(6387,4362)(6387,3162) + (5937,3162)(5937,4362) +\path(6387,4362)(6837,4362)(6837,3162) + (6387,3162)(6387,4362) +\path(4662,4212)(4662,4662)(7437,4662)(7437,4362) +\path(7407.000,4482.000)(7437.000,4362.000)(7467.000,4482.000) +\path(12,6087)(1737,6087)(1737,4887) + (12,4887)(12,6087) +\path(987,5637)(2637,5637)(2637,9612)(3912,9612) +\path(3792.000,9582.000)(3912.000,9612.000)(3792.000,9642.000) +\path(1587,5037)(2637,5037)(2637,4062)(3912,4062) +\path(3792.000,4032.000)(3912.000,4062.000)(3792.000,4092.000) +\put(4137,12){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}......}}}}} +\put(3987,4137){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}start}}}}} +\put(3987,3837){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}free}}}}} +\put(3987,3537){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks=$n_1$}}}}} +\put(4137,3237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}link}}}}} +\put(6087,3762){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}...}}}}} +\put(462,6237){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}step}}}}} +\put(87,5562){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}blocks}}}}} +\put(87,5862){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}no}}}}} +\put(87,5262){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}n\_blocks}}}}} +\put(87,4962){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{\rmdefault}{\mddefault}{\updefault}large\_object}}}}} +\end{picture} +} diff --git a/docs/storage-mgt/step.fig b/docs/storage-mgt/step.fig new file mode 100644 index 0000000000..af9661f2be --- /dev/null +++ b/docs/storage-mgt/step.fig @@ -0,0 +1,154 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +60.00 +Single +-2 +1200 2 +6 9825 1650 13125 3150 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9825 1950 13125 1950 13125 3150 9825 3150 9825 1950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9825 2550 12975 2550 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12975 1950 12975 3150 +4 0 0 50 0 0 17 0.0000 4 165 630 11025 1875 blocks\001 +4 0 0 50 0 0 17 0.0000 4 225 1230 10800 2400 single object\001 +-6 +6 6300 3750 13125 5250 +6 9825 3750 13125 5250 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9825 4050 13125 4050 13125 5250 9825 5250 9825 4050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9825 4650 12975 4650 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12975 4050 12975 5250 +4 0 0 50 0 0 17 0.0000 4 165 630 11025 3975 blocks\001 +4 0 0 50 0 0 17 0.0000 4 225 1230 10800 4500 single object\001 +-6 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 4050 7875 4050 7875 5250 6300 5250 6300 4050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 4200 7050 3750 9825 3750 9825 4050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 4500 7200 3750 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 4050 8325 4050 8325 5250 7875 5250 7875 4050 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8325 4050 8775 4050 8775 5250 8325 5250 8325 4050 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8775 4050 9225 4050 9225 5250 8775 5250 8775 4050 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 4275 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 4575 free\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 5175 link\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8475 4650 ...\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 4875 blocks=n_2\001 +-6 +6 5625 -4350 12975 150 +6 6300 -4350 12975 -2400 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 -3600 7875 -3600 7875 -2400 6300 -2400 6300 -3600 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 -3450 7050 -3900 9675 -3900 9675 -3600 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12750 -3600 12750 -2400 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 -3150 7200 -3900 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9675 -3600 12975 -3600 12975 -2400 9675 -2400 9675 -3600 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7200 -3150 7200 -4350 12750 -4350 12750 -3600 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9675 -3000 12750 -3000 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 -3375 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 -3075 free\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 -2475 link\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6375 -2775 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10350 -3750 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 10350 -3150 used memory\001 +-6 +6 5625 -1800 12975 150 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 5850 -1275 5850 -750 6225 -750 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 -1050 7875 -1050 7875 150 6300 150 6300 -1050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 -900 7050 -1350 9675 -1350 9675 -1050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 12750 -1050 12750 150 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 -600 7200 -1350 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 9675 -1050 12975 -1050 12975 150 9675 150 9675 -1050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7200 -600 7200 -1800 12750 -1800 12750 -1050 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 2 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 9675 -450 12750 -450 +4 0 0 50 0 0 17 0.0000 4 30 360 5625 -1500 ......\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 -825 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 -525 free\001 +4 0 0 50 0 0 17 0.0000 4 165 885 6375 -225 blocks=1\001 +4 0 0 50 0 0 17 0.0000 4 225 1185 10350 -1200 single block\001 +4 0 0 50 0 0 17 0.0000 4 225 1320 10350 -600 used memory\001 +4 0 0 50 0 0 17 0.0000 4 165 1185 6375 75 link=NULL\001 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 3 + 0 0 1.00 60.00 120.00 + 6450 -2550 5850 -2550 5850 -1800 +-6 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 4 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 6450 5100 5850 5100 5850 6300 6300 6300 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 1 4 + 0 0 1.00 60.00 120.00 + 0 0 1.00 60.00 120.00 + 6450 3000 5850 3000 5850 4200 6300 4200 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 6300 1950 7875 1950 7875 3150 6300 3150 6300 1950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 + 7200 2400 7200 1650 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 7875 1950 8325 1950 8325 3150 7875 3150 7875 1950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8325 1950 8775 1950 8775 3150 8325 3150 8325 1950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 8775 1950 9225 1950 9225 3150 8775 3150 8775 1950 +2 1 0 1 0 7 50 0 -1 0.000 0 0 7 1 0 4 + 0 0 1.00 60.00 120.00 + 7050 2100 7050 1650 9825 1650 9825 1950 +2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5 + 2400 225 4125 225 4125 1425 2400 1425 2400 225 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 3375 675 5025 675 5025 -3300 6300 -3300 +2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 4 + 0 0 1.00 60.00 120.00 + 3975 1275 5025 1275 5025 2250 6300 2250 +4 0 0 50 0 0 17 0.0000 4 30 360 6525 6300 ......\001 +4 0 0 50 0 0 17 0.0000 4 150 435 6375 2175 start\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6375 2475 free\001 +4 0 0 50 0 0 17 0.0000 4 195 1125 6375 2775 blocks=n_1\001 +4 0 0 50 0 0 17 0.0000 4 165 390 6525 3075 link\001 +4 0 0 50 0 0 17 0.0000 4 30 180 8475 2550 ...\001 +4 0 0 50 0 0 17 0.0000 4 210 390 2850 75 step\001 +4 0 0 50 0 0 17 0.0000 4 165 630 2475 750 blocks\001 +4 0 0 50 0 0 17 0.0000 4 120 240 2475 450 no\001 +4 0 0 50 0 0 17 0.0000 4 195 870 2475 1050 n_blocks\001 +4 0 0 50 0 0 17 0.0000 4 225 1200 2475 1350 large_object\001 diff --git a/docs/users_guide/5-00-notes.xml b/docs/users_guide/5-00-notes.xml new file mode 100644 index 0000000000..28712472c6 --- /dev/null +++ b/docs/users_guide/5-00-notes.xml @@ -0,0 +1,207 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="release-5-00"> + <title>Release notes for version 5.00 (April 2001)</title> + + <sect2> + <title>User-visible compiler changes</title> + <itemizedlist> + <listitem> + <para>GHCi, the new interactive environment on top of GHC, has + been added (<xref linkend="ghci">).</para> + </listitem> + <listitem> + <para>New <option>––make</option> flag added (<xref + linkend="make-mode">).</para> + </listitem> + <listitem> + <para>The native code generator now supports Sparc in addition + to x86.</para> + </listitem> + <listitem> + <para>We now make it clear which options can be placed in an + OPTIONS pragma. See <xref + linkend="static-dynamic-flags">.</para> + </listitem> + <listitem> + <para><option>-fglasgow-exts</option> no longer implies + <option>-package lang</option>.</para> + </listitem> + <listitem> + <para><option>-noC</option> is no more.</para> + </listitem> + <listitem> + <para><option>-hi</option> and <option>-nohi</option> are no more.</para> + </listitem> + <listitem> + <para>The concept of “packages” has been + generalised and extended. Packages may be installed or + removed from an existing GHC installation using the new + <command>ghc-pkg</command> tool. See <xref + linkend="packages">.</para> + </listitem> + <listitem> + <para>Initial unicode support: the <literal>Char</literal> + type is now 31 bits. We don't yet have support for unicode + I/O.</para> + </listitem> + <listitem> + <para><option>-v</option> now takes an optional numeric + argument indicating the level of verbosity (<xref + linkend="options-help">). <option>-dshow-passes</option> has + been removed.</para> + </listitem> + <listitem> + <para>Parallel list comprehensions added. See <xref + linkend="parallel-list-comprehensions">.</para> + </listitem> + <listitem> + <para>Functional dependencies are now fully implemented. + </para> + </listitem> + <listitem> + <para>Profiling: please use + <literal>{-# SCC ".." #-}</literal> + rather than <literal>_scc_ "..."</literal>. The latter + will be phased out in due course.</para> + </listitem> + <listitem> + <para>A new experimental optimisation, SpecConstr, is turned + on with <literal>-O2</literal>.</para> + </listitem> + <listitem> + <para>Please report bugs using the <ulink + url="http://sourceforge.net/projects/ghc/">SourceForge bug + tracker</ulink> instead of + <email>glasgow-haskell-bugs@haskell.org</email> if + possible.</para> + </listitem> + <listitem> + <para>Documentation changes: there's now a useful Flag + Reference section, see <xref linkend="flag-reference">.</para> + </listitem> + <listitem> + <para>Many, many, bugfixes.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>New experimental features</title> + + <itemizedlist> + <listitem> + <para>A “front panel” for GHC-compiled programs + displays real-time graphs of memory behaviour in a GTK+ + window. You need to recompile the RTS with front panel + support to use this.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>FFI changes</title> + <itemizedlist> + <listitem> + <para><command>hsc2hs</command> added (<xref linkend="hsc2hs">).</para> + </listitem> + <listitem> + <para>FFI libraries have been updated to the latest proposal + from the FFI task force. Too many changes to list here, see + the docs: <xref linkend="sec-Foreign">.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible library changes</title> + <itemizedlist> + <listitem> + <para><function>putMVar</function> now blocks if the + <literal>MVar</literal> is already full. The + <literal>PutFullMVar</literal> exception no longer exists. + A non-blocking version of <function>putMVar</function>, + <function>tryPutMVar</function>, has been added (<xref + linkend="sec-MVars">).</para> + </listitem> + <listitem> + <para>The <literal>Int</literal> and + <literal>Integer</literal> types now have instances of + <literal>Bits</literal> (<xref linkend="sec-Bits">).</para> + </listitem> + <listitem> + <para>Package <literal>hssource</literal> has been added. It + contains a Haskell 98 abstract syntax, parser, lexer and pretty + printer. No documentation yet.</para> + </listitem> + <listitem> + <para>The methods <literal>fromInt</literal> and + <literal>toInt</literal>, which used to be in class + <literal>Num</literal> but exported from module + <literal>Int</literal>, are no longer in class + <literal>Num</literal>. They're still available from module + <literal>Int</literal>, however.</para> + + <para>In most cases, there should be no benefit from using + <literal>fromInt</literal> instead of + <literal>fromIntegral</literal>, which is specialised for all + integral types.</para> + </listitem> + <listitem> + <para>New modules: DiffArray (<xref linkend="sec-DiffArray">), + StorableArray (<xref linkend="sec-StorableArray">), + MonadList, MonadCont (no documentation yet).</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Build system changes</title> + + <itemizedlist> + <listitem> + <para>The <literal>WithGhcHc</literal> setting in + <literal>build.mk</literal> has been replaced by the + <literal>––with-ghc=<replaceable>ghc</replaceable></literal> + option to <literal>configure</literal>. The new option + <emphasis>must</emphasis> be used if you intend to use + anything except “<literal>ghc</literal>” to + bootstrap GHC, in order that the build system can figure out + what version of GHC you're using.</para> + </listitem> + <listitem> + <para>Source distributions are now made by doing <literal>make + distclean</literal> in a build tree, instead of requiring a + linked build tree.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Internal changes</title> + <itemizedlist> + <listitem> + <para>Many internal compiler changes: too many to list + here.</para> + </listitem> + <listitem> + <para>The old perl driver has been removed and replaced by a + driver in the compiler proper.</para> + </listitem> + <listitem> + <para>We now use GMP 3 instead of GMP 2 for + arbitrary-precision integer support.</para> + </listitem> + <listitem> + <para>Several libraries rewritten to use the FFI.</para> + </listitem> + </itemizedlist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/5-02-notes.xml b/docs/users_guide/5-02-notes.xml new file mode 100644 index 0000000000..a8bc83a4ba --- /dev/null +++ b/docs/users_guide/5-02-notes.xml @@ -0,0 +1,57 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="release-5-04"> + <title>Release notes for version 5.04</title> + + <sect2> + <title>User-visible compiler changes</title> + <itemizedlist> + <listitem> + <para></para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible interpreter (GHCi) changes</title> + <itemizedlist> + <listitem> + <para></para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible library changes</title> + <itemizedlist> + <listitem> + <para></para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>New experimental features</title> + <itemizedlist> + <listitem> + <para></para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Internal changes</title> + <itemizedlist> + <listitem> + <para></para> + </listitem> + </itemizedlist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/5-04-notes.xml b/docs/users_guide/5-04-notes.xml new file mode 100644 index 0000000000..91b8dcf606 --- /dev/null +++ b/docs/users_guide/5-04-notes.xml @@ -0,0 +1,288 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="release-5-04"> + <title>Release notes for version 5.04</title> + + <sect2> + <title>User-visible compiler changes</title> + <itemizedlist> + <listitem> + <para>Full support for MacOS X, including fully optimized compilation, has been added. Only a native + code generator and support for <option>-split-objs</option> is still missing. + Everything else needs more testing, but should work.</para> + </listitem> + <listitem> + <para><literal>ghc-pkg</literal>: new options + <option>--auto-ghci-libs</option>, + <option>-u</option>/<option>--update-package</option>, + <option>--force</option>, and + <option>-i</option>/<option>--input-file</option>, and + suppport for expanding environment variables in package + descriptions. See <xref linkend="packages">).</para> + </listitem> + <listitem> + <para>The latest version of the FFI spec is fully supported. + The syntax of FFI declarations has changed accordingly. The + old syntax is still accepted for the time being, but will + elicit a warning from the compiler.</para> + </listitem> + <listitem> + <para>New option: <option>-F</option> specifies a user-defined + preprocessing phase (see <xref linkend="pre-processor">).</para> + </listitem> + <listitem> + <para>Major overhaul of the heap profiling subsystem, with new + facilities for retainer profiling and biographical profiling + (ala nhc98, albeit with a couple of omissions). The syntax of + the runtime heap-profiling options has changed. See <xref + linkend="prof-heap">.</para> + </listitem> + <listitem> + <para>The type system now supports full rank-N types + (previously only limited rank-2 types were supported). See + <xref linkend="universal-quantification">.</para> + </listitem> + <listitem> + <para>Explicit kind annotations can now be given on any + binding occurrence of a type variable. See <xref + linkend="sec-kinding">.</para> + </listitem> + <listitem> + <para>The handling of type synonyms has been rationalised. + See <xref linkend="type-synonyms">.</para> + </listitem> + <listitem> + <para>Fixes for several space leaks in the compiler itself + (these fixes were also merged into 5.02.3).</para> + </listitem> + <listitem> + <para>It is now possible to derive arbitrary classes for + newtypes. See <xref linkend="newtype-deriving">.</para> + </listitem> + <listitem> + <para>Deadlock is now an exception, rather than a return + status from the scheduler. See the module + <literal>Control.Exception</literal> in the library + documentation for more details.</para> + </listitem> + <listitem> + <para>The syntax and behaviour of <literal>RULE</literal> + pragmas has changed slightly. See <xref + linkend="rewrite-rules">.</para> + </listitem> + <listitem> + <para>Interface files are now in a binary format to reduce + compilation times. To view an interface file in plain text, + use the <option>--show-iface</option> flag.</para> + </listitem> + <listitem> + <para>A restriction on the form of class declarations has been + lifted. In Haskell 98, it is illegal for class method types + to mention constraints on the class type variable. eg.</para> + +<programlisting> + class Seq s a where + elem :: Eq a => a -> s a -> Bool +</programlisting> + + <para>This restriction has now been lifted in GHC.</para> + </listitem> + <listitem> + <para>Main threads can now receive the + <literal>BlockedOnDeadMVar</literal> exception in the same way + as other threads.</para> + </listitem> + <listitem> + <para>The <option>-fall-strict</option> flag never really + worked, and has been removed.</para> + </listitem> + <listitem> + <para>The syntax of <literal>.hi-boot</literal> files is now + much clearer and Haskell-like. See <xref + linkend="mutual-recursion">.</para> + </listitem> + <listitem> + <para>There is a new flag <option>-fffi</option> which enables + FFI support without turning on the rest of the GHC + extensions.</para> + </listitem> + <listitem> + <para>The syntax for implicit parameter bindings has changed. + Previously the keyword <literal>with</literal> was used to + introduce implicit bindings, but now implicit bindings may be + introduced using <literal>let</literal> (see <xref + linkend="implicit-parameters">). As a result of this, + <literal>with</literal> is no longer a keyword when + <option>-fglasgow-exts</option> is turned on.</para> + + <para>The option <literal>-fwith</literal> may be used to + restore the old behaviour.</para> + </listitem> + <listitem> + <para>Infix type constructors are now allowed, and must begin + with a colon (as with data constructors). See <xref + linkend="infix-tycons">.</para> + </listitem> + <listitem> + <para>The <literal>do</literal>-notation syntax is now + rebindable in the same way as other built-in syntax. See + <xref linkend="rebindable-syntax">.</para> + </listitem> + <listitem> + <para>Support for using “frameworks” on + Darwin/MacOS X has been added. See the + <option>-framework</option> option in <xref + linkend="options-linker">, and the + <literal>framework_dirs</literal> field of a package spec in + <xref linkend="package-management">.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible interpreter (GHCi) changes</title> + <itemizedlist> + <listitem> + <para>New commands: <literal>:browse</literal>, <literal>:set + args</literal>, <literal>:set prog</literal>, <literal>:show + bindings</literal>, and <literal>:show modules</literal> (see + <xref linkend="ghci-commands">).</para> + </listitem> + <listitem> + <para>There is a much more flexible mechanism for manipulating + the scope for expressions typed at the prompt. For example, + one can now have both the <literal>Prelude</literal> and the + exports of several compiled modules in scope at the same + time. See <xref linkend="ghci-scope">.</para> + </listitem> + <listitem> + <para>GHCi now supports <literal>foreign import + "wrapper"</literal> FFI declarations.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible library changes</title> + <itemizedlist> + <listitem> + <para>GHC is in the process of moving to a new hierarchical + set of libraries. At the moment, we have two sets of + libraries, both described in accompanying documents:</para> + <itemizedlist> + <listitem> + <para>The “new libraries” which are + hierarchical and consist of the following packages: + <literal>base</literal>, <literal>haskell98</literal>, + <literal>haskell-src</literal>, and + <literal>network</literal>. Broadly speaking, + <literal>base</literal> contains the + <literal>Prelude</literal>, standard libraries and most of + the contents of the old <literal>lang</literal> + package. By default, the <literal>base</literal> and + <literal>haskell98</literal> packages are enabled.</para> + </listitem> + + <listitem> + <para>The <literal>hslibs</literal>, most of which are now + deprecated. Where possible, new code should be written to + use the new libraries instead. </para> + + <para>The following libraries in <literal>hslibs</literal> + have not moved yet:</para> + <itemizedlist> + <listitem> + <para>The packages <literal>win32</literal>, + <literal>xlib</literal>, <literal>graphics</literal>, + and <literal>posix</literal>.</para> + </listitem> + <listitem> + <para>The Edison libraries in the + <literal>data</literal> package.</para> + </listitem> + <listitem> + <para>In the <literal>lang</literal> package, the + modules <literal>TimeExts</literal>, + <literal>DirectoryExts</literal>, + <literal>SystemExts</literal>, and + <literal>NumExts</literal>.</para> + </listitem> + <listitem> + <para>The HaXml libraries in the + <literal>text</literal> package.</para> + </listitem> + <listitem> + <para>In the <literal>util</literal> package, the + modules <literal>MD5</literal>, + <literal>Select</literal>, <literal>Memo</literal>, + <literal>Observe</literal>, and + <literal>Readline</literal>.</para> + </listitem> + </itemizedlist> + + <para>All other libraries from <literal>hslibs</literal> + either have equivalents in the new libraries (see the + <literal>hslibs</literal> docs for details), or were + already deprecated and hence were not moved into the new + hierarchy.</para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> + <para>The <literal>Read</literal> class is now based on a + parsing combinator library which is vastly more efficient than + the previous one. See the modules + <literal>Text.Read</literal>. + <literal>Text.ParserCombinators.ReadP</literal>, and + <literal>Text.ParserCombinators.ReadPrec</literal> in the + library documentation.</para> + + <para>The code generated by the compiler for derived + <literal>Read</literal> instances should be much shorter than + before.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>New experimental features</title> + <itemizedlist> + <listitem> + <para>Linear implicit parameters. See <xref + linkend="linear-implicit-parameters">.</para> + </listitem> + <listitem> + <para>The RTS has support for running in a multi-threaded + environment and making non-blocking (from Haskell's point of + view) calls to foreign C functions which would normally block. + To enable this behaviour, configure with the + <option>--enable-threaded-rts</option> option.</para> + </listitem> + <listitem> + <para>The compiler can now read in files containing Core + syntax (such as those produced by the + <option>-fext-core</option> option) and compile them. Input + files with the <literal>.hcr</literal> file extension are + assumed to contain Core syntax.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Internal changes</title> + <itemizedlist> + <listitem> + <para>Happy 1.13 is now required to build GHC, because of the + change in names of certain libraries.</para> + </listitem> + </itemizedlist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/6.0-notes.xml b/docs/users_guide/6.0-notes.xml new file mode 100644 index 0000000000..e07bc890f2 --- /dev/null +++ b/docs/users_guide/6.0-notes.xml @@ -0,0 +1,319 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="release-6-0"> + <title>Release notes for version 6.0</title> + + <sect2> + <title>User-visible compiler changes</title> + <itemizedlist> + <listitem> + <para>Template Haskell, a new feature for compile-time + metaprogramming has been introduced. See <xref + linkend="template-haskell"/>.</para> + </listitem> + <listitem> + <para>INLINE pragmas on methods in class or instance + declarations now work properly.</para> + </listitem> + <listitem> + <para>Recursive do-notation (aka <literal>mdo</literal>) is + now supported. See <xref linkend="mdo-notation"/>.</para> + </listitem> + <listitem> + <para>There is now a native code generator for PowerPC + platforms.</para> + </listitem> + <listitem> + <para>Profiling: the <option>-xt</option> RTS option enables + inclusion of thread stacks in a heap profile. See <xref + linkend="rts-options-heap-prof"/>.</para> + </listitem> + <listitem> + <para>Non-blocking I/O is now supported on Windows.</para> + </listitem> + <listitem> + <para>The <ulink url="../libraries/base/Data.Dynamic.html#Typeable"><literal>Typeable</literal></ulink> class can now be + derived, and the implementation of <literal>Typeable</literal> + is now more efficient.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible interpreter (GHCi) changes</title> + <itemizedlist> + <listitem> + <para>Loading a <literal>Main</literal> module that does not + define <literal>main</literal> is no longer an error, although + GHCi will still emit a warning in this case.</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>User-visible library changes</title> + <itemizedlist> + <listitem> + <para>Hierarchical libraries are now available without needing + to specify an explicit <option>-package</option> flag. There + are some exceptions to this rule (see <xref + linkend="using-packages"/>), but if you stick to GHCi and + <option>--make</option> mode then there will normally be no + need to specify <option>-package</option> options at + all.</para> + + <para>Non-hierarchical libraries + (i.e. <literal>hslibs</literal> libraries) still need to be + explicitly requested with <option>-package</option> + options.</para> + </listitem> + + <listitem> + <para>The <literal>Posix</literal> library has been rewritten. + It is now a hierarchical library rooted at + <literal>System.Posix</literal>, and has some additions aimed + at supporting the latest revision of the POSIX standard (IEEE + Std 1003.1-2001). See the <ulink + url="../libraries/unix/index.html"><literal>unix</literal> + package</ulink> for details.</para> + + <para>The old <literal>posix</literal> package is still + available for backwards compatibility, but is deprecated and + will be removed in a future release.</para> + </listitem> + + <listitem> + <para><ulink url="../libraries/base/Data.IORef.html"><literal>Data.IORef</literal></ulink>: Added <literal>atomicModifyIORef</literal>.</para> + </listitem> + + <listitem> + <para><ulink url="../libraries/base/System.Cmd.html"><literal>System.Cmd</literal></ulink>: Added <literal>rawSystem</literal>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/System.Environment.html"><literal>System.Environment</literal></ulink>: + Added <literal>withArgs</literal> and <literal>withProgName</literal>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/network/Network.Socket.html"><literal>Network.Socket</literal></ulink>: + Added <literal>sendFd</literal> and <literal>recvFd</literal>.</para> + </listitem> + + <listitem> + <para>The <literal>Readline</literal> library has moved to + <ulink + url="../libraries/readline/System.Console.Readline.html"><literal>System.Console.Readline</literal></ulink>, + and is in a package of its own + (<literal>readline</literal>).</para> + </listitem> + + <listitem> + <para>The non-hierarchical versions of the FFI libraries are + now all available without needing to specify <literal>-package + lang</literal> (they are actually now in the + <literal>haskell98</literal> package, which is available by + default).</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/network/Network.BSD.html"><literal>Network.BSD</literal></ulink>: + <literal>symlink</literal> and <literal>readline</literal> are + now deprecated; use + <literal>System.Posix.createSymbolicLink</literal> and + <literal>System.Posix.readSymbolicLink</literal> + respectively.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Control.Exception.html"><literal>Control.Exception</literal></ulink>: + Added <literal>mapException</literal>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.Dynamic.html"><literal>Data.Dynamic</literal></ulink>: + various changes to make the implementation of + <literal>Typeable</literal> more efficient. This entails some + changes to the interface, and affects how instances of + <literal>Typeable</literal> are defined.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.Tree.html"><literal>Data.Tree</literal></ulink> + is a new library for trees.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.Graph.html"><literal>Data.Graph</literal></ulink> + is a new library for graphs.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/System.IO.html"><literal>System.IO</literal></ulink>: + Removed <literal>bracket</literal> and + <literal>bracket_</literal> (use the versions from + <literal>Control.Exception</literal> instead).</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/System.IO.html"><literal>System.IO</literal></ulink>: + The <literal>IOError</literal> type is now a synonym for + <literal>IOException</literal>, whereas previously it was a + synonym for <literal>Exception</literal>. This has various + consequences, one of which is that the types of + <literal>System.IO.catch</literal> and + <literal>Control.Exception.catch</literal> are now different + (useful, because they do different things).</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/System.IO.Error.html"><literal>System.IO.Error</literal></ulink>: + added <literal>annotateIOError</literal>, + <literal>modifyIOError</literal>, and <literal>ioeSet{ErrorType,ErrorString,Handle,FileName}</literal>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Text.ParserCombinators.ReadP.html"><literal>Text.ParserCombinators.ReadP</literal></ulink>: + lots of updates.</para> + </listitem> + + <listitem> + <para><literal>Control.Monad.Monoid</literal> is now <ulink url="../libraries/base/Data.Monoid.html"><literal>Data.Monoid</literal></ulink>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.PackedString.html"><literal>Data.PackedString</literal></ulink>: + added <literal>joinPS</literal>, <literal>unwordsPS</literal> + and <literal>unlinesPS</literal>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.HashTable.html"><literal>Data.HashTable</literal></ulink> + is a new dynamic hash-table implementation.</para> + </listitem> + + <listitem> + <para>Added <ulink + url="../libraries/unix/System.Sendfile.html"><literal>System.Sendfile</literal></ulink>.</para> + </listitem> + + <listitem> + <para>Added <ulink + url="../libraries/base/Foreign.Marshal.Pool.html"><literal>Foreign.Marshal.Pool</literal></ulink>.</para> + </listitem> + + <listitem> + <para><ulink + url="../libraries/base/Data.Bits.html"><literal>Data.Bits</literal></ulink>: + <literal>shiftL</literal>, <literal>shiftR</literal>, + <literal>rotateL</literal>, and <literal>rotateR</literal> are + now methods of the <literal>Bite</literal> class.</para> + </listitem> + + <listitem> + <para>The FFI libraries now conform to the latest version of + the FFI spec:</para> + <itemizedlist> + <listitem> + <para>Added <ulink + url="../libraries/base/Foreign.ForeignPtr.html#mallocForeignPtr"><literal>Foreign.ForeignPtr.mallocForeignPtr</literal></ulink> + and friends.</para> + </listitem> + <listitem> + <para>Finalizers added to a <literal>ForeignPtr</literal> + with <literal>addForeignPtrFinalizer</literal> are now run + in strict order; namely the reverse of the order they were + added.</para> + </listitem> + <listitem> + <para><literal>Foreign.C.TypesISO</literal> has been + merged into <ulink + url="../libraries/base/Foreign.C.Types.html"><literal>Foreign.C.Types</literal></ulink>.</para> + </listitem> + </itemizedlist> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Experimental features</title> + <itemizedlist> + <listitem> + <para>The <literal>Data</literal> class provides for generic + data traversals and folds; see <ulink + url="../libraries/base/Data.Generics.html"><literal>Data.Generics</literal></ulink>. + <literal>Data</literal> can be derived for arbitrary + datatypes. The <literal>Data</literal> class is still + experimental, so its contents may change in the future.</para> + </listitem> + <listitem> + <para>Several bugs have been fixed in the threaded RTS, and it + should now be rather more robust (it should still be + considered experimental, however).</para> + </listitem> + </itemizedlist> + </sect2> + + <sect2> + <title>Internal changes</title> + <itemizedlist> + <listitem> + <para>Sweeping changes to the compiler and runtime system to + change the evaluation model from <quote>push/enter</quote> to + <quote>eval/apply</quote>. The bottom line is that the + compiler is now more portable and some of the complexity is + now more centralised, while performance and binary sizes + remain about the same.</para> + + <para>A paper describing these changes can be found <ulink + url="http://research.microsoft.com/~simonpj/papers/eval-apply">here</ulink>.</para> + </listitem> + <listitem> + <para>The test suite is now driven by a Python script and is + rather more flexible and robust. It now supports building + tests several different "ways", and as a result we now run + each test with optimisation, profiling, native code + generation, and GHCi in addition to the vanilla way.</para> + </listitem> + <listitem> + <para>The build system now supports bootstrapping the compiler + in a single build tree. By default, typing + <literal>make</literal> at the top level will bootstrap the + compiler once to create a stage-2 compiler. See the Building + Guide for more details.</para> + </listitem> + <listitem> + <para>The RTS debugging flags are no longer represented by a + bitfield and now have single-character names. For example, to + turn on scheduler debugging output, use <literal>-Ds</literal> + rather than <literal>-D1</literal>.</para> + </listitem> + <listitem> + <para>The compiler no longer requires any packages from + <literal>hslibs</literal> to bootstrap. It is enough to + compile <literal>fptools/libraries</literal> before building + the stage 2 compiler.</para> + </listitem> + </itemizedlist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/Makefile b/docs/users_guide/Makefile new file mode 100644 index 0000000000..f0a31fb705 --- /dev/null +++ b/docs/users_guide/Makefile @@ -0,0 +1,7 @@ +TOP = ../.. +include $(TOP)/mk/boilerplate.mk + +XML_DOC = users_guide +INSTALL_XML_DOC = users_guide + +include $(TOP)/mk/target.mk diff --git a/docs/users_guide/bugs.xml b/docs/users_guide/bugs.xml new file mode 100644 index 0000000000..ab0b9be7b9 --- /dev/null +++ b/docs/users_guide/bugs.xml @@ -0,0 +1,400 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="bugs-and-infelicities"> + <title>Known bugs and infelicities</title> + + <sect1 id="vs-Haskell-defn"> + <title>Haskell 98 vs. Glasgow Haskell: language non-compliance +</title> + + <indexterm><primary>GHC vs the Haskell 98 language</primary></indexterm> + <indexterm><primary>Haskell 98 language vs GHC</primary></indexterm> + + <para>This section lists Glasgow Haskell infelicities in its + implementation of Haskell 98. See also the “when things + go wrong” section (<xref linkend="wrong"/>) for information + about crashes, space leaks, and other undesirable phenomena.</para> + + <para>The limitations here are listed in Haskell Report order + (roughly).</para> + + <sect2 id="haskell98-divergence"> + <title>Divergence from Haskell 98</title> + + + <sect3 id="infelicities-lexical"> + <title>Lexical syntax</title> + + <itemizedlist> + <listitem> + <para>The Haskell report specifies that programs may be + written using Unicode. GHC only accepts the ISO-8859-1 + character set at the moment.</para> + </listitem> + + <listitem> + <para>Certain lexical rules regarding qualified identifiers + are slightly different in GHC compared to the Haskell + report. When you have + <replaceable>module</replaceable><literal>.</literal><replaceable>reservedop</replaceable>, + such as <literal>M.\</literal>, GHC will interpret it as a + single qualified operator rather than the two lexemes + <literal>M</literal> and <literal>.\</literal>.</para> + </listitem> + </itemizedlist> + </sect3> + + <sect3 id="infelicities-syntax"> + <title>Context-free syntax</title> + + <itemizedlist> + <listitem> + <para>GHC is a little less strict about the layout rule when used + in <literal>do</literal> expressions. Specifically, the + restriction that "a nested context must be indented further to + the right than the enclosing context" is relaxed to allow the + nested context to be at the same level as the enclosing context, + if the enclosing context is a <literal>do</literal> + expression.</para> + + <para>For example, the following code is accepted by GHC: + +<programlisting> +main = do args <- getArgs + if null args then return [] else do + ps <- mapM process args + mapM print ps</programlisting> + + </para> + </listitem> + + <listitem> + <para>GHC doesn't do fixity resolution in expressions during + parsing. For example, according to the Haskell report, the + following expression is legal Haskell: +<programlisting> + let x = 42 in x == 42 == True</programlisting> + and parses as: +<programlisting> + (let x = 42 in x == 42) == True</programlisting> + + because according to the report, the <literal>let</literal> + expression <quote>extends as far to the right as + possible</quote>. Since it can't extend past the second + equals sign without causing a parse error + (<literal>==</literal> is non-fix), the + <literal>let</literal>-expression must terminate there. GHC + simply gobbles up the whole expression, parsing like this: +<programlisting> + (let x = 42 in x == 42 == True)</programlisting> + + The Haskell report is arguably wrong here, but nevertheless + it's a difference between GHC & Haskell 98.</para> + </listitem> + </itemizedlist> + </sect3> + + <sect3 id="infelicities-exprs-pats"> + <title>Expressions and patterns</title> + + <para>None known.</para> + </sect3> + + <sect3 id="infelicities-decls"> + <title>Declarations and bindings</title> + + <para>None known.</para> + </sect3> + + <sect3 id="infelicities-Modules"> + <title>Module system and interface files</title> + + <para>None known.</para> + </sect3> + + <sect3 id="infelicities-numbers"> + <title>Numbers, basic types, and built-in classes</title> + + <variablelist> + <varlistentry> + <term>Multiply-defined array elements—not checked:</term> + <listitem> + <para>This code fragment should + elicit a fatal error, but it does not: + +<programlisting> +main = print (array (1,1) [(1,2), (1,3)])</programlisting> +GHC's implementation of <literal>array</literal> takes the value of an +array slot from the last (index,value) pair in the list, and does no +checking for duplicates. The reason for this is efficiency, pure and simple. + </para> + </listitem> + </varlistentry> + </variablelist> + + </sect3> + + <sect3 id="infelicities-Prelude"> + <title>In <literal>Prelude</literal> support</title> + + <variablelist> + <varlistentry> + <term>Arbitrary-sized tuples</term> + <listitem> + <para>Tuples are currently limited to size 100. HOWEVER: + standard instances for tuples (<literal>Eq</literal>, + <literal>Ord</literal>, <literal>Bounded</literal>, + <literal>Ix</literal> <literal>Read</literal>, and + <literal>Show</literal>) are available + <emphasis>only</emphasis> up to 16-tuples.</para> + + <para>This limitation is easily subvertible, so please ask + if you get stuck on it.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Read</literal>ing integers</term> + <listitem> + <para>GHC's implementation of the + <literal>Read</literal> class for integral types accepts + hexadecimal and octal literals (the code in the Haskell + 98 report doesn't). So, for example, +<programlisting>read "0xf00" :: Int</programlisting> + works in GHC.</para> + <para>A possible reason for this is that <literal>readLitChar</literal> accepts hex and + octal escapes, so it seems inconsistent not to do so for integers too.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>isAlpha</literal></term> + <listitem> + <para>The Haskell 98 definition of <literal>isAlpha</literal> + is:</para> + +<programlisting>isAlpha c = isUpper c || isLower c</programlisting> + + <para>GHC's implementation diverges from the Haskell 98 + definition in the sense that Unicode alphabetic characters which + are neither upper nor lower case will still be identified as + alphabetic by <literal>isAlpha</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + </sect3> + </sect2> + + <sect2 id="haskell98-undefined"> + <title>GHC's interpretation of undefined behaviour in + Haskell 98</title> + + <para>This section documents GHC's take on various issues that are + left undefined or implementation specific in Haskell 98.</para> + + <variablelist> + <varlistentry> + <term> + The <literal>Char</literal> type + <indexterm><primary><literal>Char</literal></primary><secondary>size of</secondary></indexterm> + </term> + <listitem> + <para>Following the ISO-10646 standard, + <literal>maxBound :: Char</literal> in GHC is + <literal>0x10FFFF</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + Sized integral types + <indexterm><primary><literal>Int</literal></primary><secondary>size of</secondary></indexterm> + </term> + <listitem> + <para>In GHC the <literal>Int</literal> type follows the + size of an address on the host architecture; in other words + it holds 32 bits on a 32-bit machine, and 64-bits on a + 64-bit machine.</para> + + <para>Arithmetic on <literal>Int</literal> is unchecked for + overflow<indexterm><primary>overflow</primary><secondary><literal>Int</literal></secondary> + </indexterm>, so all operations on <literal>Int</literal> happen + modulo + 2<superscript><replaceable>n</replaceable></superscript> + where <replaceable>n</replaceable> is the size in bits of + the <literal>Int</literal> type.</para> + + <para>The <literal>fromInteger</literal><indexterm><primary><literal>fromInteger</literal></primary> + </indexterm>function (and hence + also <literal>fromIntegral</literal><indexterm><primary><literal>fromIntegral</literal></primary> + </indexterm>) is a special case when + converting to <literal>Int</literal>. The value of + <literal>fromIntegral x :: Int</literal> is given by taking + the lower <replaceable>n</replaceable> bits of <literal>(abs + x)</literal>, multiplied by the sign of <literal>x</literal> + (in 2's complement <replaceable>n</replaceable>-bit + arithmetic). This behaviour was chosen so that for example + writing <literal>0xffffffff :: Int</literal> preserves the + bit-pattern in the resulting <literal>Int</literal>.</para> + + + <para>Negative literals, such as <literal>-3</literal>, are + specified by (a careful reading of) the Haskell Report as + meaning <literal>Prelude.negate (Prelude.fromInteger 3)</literal>. + So <literal>-2147483648</literal> means <literal>negate (fromInteger 2147483648)</literal>. + Since <literal>fromInteger</literal> takes the lower 32 bits of the representation, + <literal>fromInteger (2147483648::Integer)</literal>, computed at type <literal>Int</literal> is + <literal>-2147483648::Int</literal>. The <literal>negate</literal> operation then + overflows, but it is unchecked, so <literal>negate (-2147483648::Int)</literal> is just + <literal>-2147483648</literal>. In short, one can write <literal>minBound::Int</literal> as + a literal with the expected meaning (but that is not in general guaranteed. + </para> + + <para>The <literal>fromIntegral</literal> function also + preserves bit-patterns when converting between the sized + integral types (<literal>Int8</literal>, + <literal>Int16</literal>, <literal>Int32</literal>, + <literal>Int64</literal> and the unsigned + <literal>Word</literal> variants), see the modules + <literal>Data.Int</literal> and <literal>Data.Word</literal> + in the library documentation.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Unchecked float arithmetic</term> + <listitem> + <para>Operations on <literal>Float</literal> and + <literal>Double</literal> numbers are + <emphasis>unchecked</emphasis> for overflow, underflow, and + other sad occurrences. (note, however that some + architectures trap floating-point overflow and + loss-of-precision and report a floating-point exception, + probably terminating the + program)<indexterm><primary>floating-point + exceptions</primary></indexterm>.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + </sect1> + + + <sect1 id="bugs"> + <title>Known bugs or infelicities</title> + + <para>The bug tracker lists bugs that have been reported in GHC but not + yet fixed: see the <ulink url="http://sourceforge.net/projects/ghc/">SourceForge GHC + page</ulink>. In addition to those, GHC also has the following known bugs + or infelicities. These bugs are more permanent; it is unlikely that + any of them will be fixed in the short term.</para> + + <sect2 id="bugs-ghc"> + <title>Bugs in GHC</title> + + <itemizedlist> + <listitem> + <para> GHC can warn about non-exhaustive or overlapping + patterns (see <xref linkend="options-sanity"/>), and usually + does so correctly. But not always. It gets confused by + string patterns, and by guards, and can then emit bogus + warnings. The entire overlap-check code needs an overhaul + really.</para> + </listitem> + + <listitem> + <para>GHC does not allow you to have a data type with a context + that mentions type variables that are not data type parameters. + For example: +<programlisting> + data C a b => T a = MkT a +</programlisting> + so that <literal>MkT</literal>'s type is +<programlisting> + MkT :: forall a b. C a b => a -> T a +</programlisting> + In principle, with a suitable class declaration with a functional dependency, + it's possible that this type is not ambiguous; but GHC nevertheless rejects + it. The type variables mentioned in the context of the data type declaration must + be among the type parameters of the data type.</para> + </listitem> + + <listitem> + <para>GHC's inliner can be persuaded into non-termination + using the standard way to encode recursion via a data type:</para> +<programlisting> + data U = MkU (U -> Bool) + + russel :: U -> Bool + russel u@(MkU p) = not $ p u + + x :: Bool + x = russel (MkU russel) +</programlisting> + + <para>We have never found another class of programs, other + than this contrived one, that makes GHC diverge, and fixing + the problem would impose an extra overhead on every + compilation. So the bug remains un-fixed. There is more + background in <ulink + url="http://research.microsoft.com/~simonpj/Papers/inlining"> + Secrets of the GHC inliner</ulink>.</para> + </listitem> + + <listitem> + <para>GHC does not keep careful track of + what instance declarations are 'in scope' if they come from other packages. + Instead, all instance declarations that GHC has seen in other + packages are all in scope everywhere, whether or not the + module from that package is used by the command-line + expression. This bug affects only the <option>--make</option> mode and + GHCi.</para> + </listitem> + + </itemizedlist> + </sect2> + + <sect2 id="bugs-ghci"> + <title>Bugs in GHCi (the interactive GHC)</title> + <itemizedlist> + <listitem> + <para>GHCi does not respect the <literal>default</literal> + declaration in the module whose scope you are in. Instead, + for expressions typed at the command line, you always get the + default default-type behaviour; that is, + <literal>default(Int,Double)</literal>.</para> + + <para>It would be better for GHCi to record what the default + settings in each module are, and use those of the 'current' + module (whatever that is).</para> + </listitem> + + <listitem> + <para>On Windows, there's a GNU ld/BFD bug + whereby it emits bogus PE object files that have more than + 0xffff relocations. When GHCi tries to load a package affected by this + bug, you get an error message of the form +<screen> +Loading package javavm ... linking ... WARNING: Overflown relocation field (# relocs found: 30765) +</screen> + The last time we looked, this bug still + wasn't fixed in the BFD codebase, and there wasn't any + noticeable interest in fixing it when we reported the bug + back in 2001 or so. + </para> + <para>The workaround is to split up the .o files that make up + your package into two or more .o's, along the lines of + how the "base" package does it.</para> + </listitem> + </itemizedlist> + </sect2> + </sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/debugging.xml b/docs/users_guide/debugging.xml new file mode 100644 index 0000000000..a325389d46 --- /dev/null +++ b/docs/users_guide/debugging.xml @@ -0,0 +1,599 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="options-debugging"> + <title>Debugging the compiler</title> + + <indexterm><primary>debugging options (for GHC)</primary></indexterm> + + <para>HACKER TERRITORY. HACKER TERRITORY. (You were warned.)</para> + + <sect2 id="dumping-output"> + <title>Dumping out compiler intermediate structures</title> + + <indexterm><primary>dumping GHC intermediates</primary></indexterm> + <indexterm><primary>intermediate passes, output</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-ddump-</option><replaceable>pass</replaceable> + <indexterm><primary><option>-ddump</option> options</primary></indexterm> + </term> + <listitem> + <para>Make a debugging dump after pass + <literal><pass></literal> (may be common enough to need + a short form…). You can get all of these at once + (<emphasis>lots</emphasis> of output) by using + <option>-v5</option>, or most of them with + <option>-v4</option>. Some of the most useful ones + are:</para> + + <variablelist> + <varlistentry> + <term> + <option>-ddump-parsed</option>: + <indexterm><primary><option>-ddump-parsed</option></primary></indexterm> + </term> + <listitem> + <para>parser output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-rn</option>: + <indexterm><primary><option>-ddump-rn</option></primary></indexterm> + </term> + <listitem> + <para>renamer output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-tc</option>: + <indexterm><primary><option>-ddump-tc</option></primary></indexterm> + </term> + <listitem> + <para>typechecker output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-types</option>: + <indexterm><primary><option>-ddump-types</option></primary></indexterm> + </term> + <listitem> + <para>Dump a type signature for each value defined at + the top level of the module. The list is sorted + alphabetically. Using <option>-dppr-debug</option> + dumps a type signature for all the imported and + system-defined things as well; useful for debugging the + compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-deriv</option>: + <indexterm><primary><option>-ddump-deriv</option></primary></indexterm> + </term> + <listitem> + <para>derived instances</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-ds</option>: + <indexterm><primary><option>-ddump-ds</option></primary></indexterm> + </term> + <listitem> + <para>desugarer output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-spec</option>: + <indexterm><primary><option>-ddump-spec</option></primary></indexterm> + </term> + <listitem> + <para>output of specialisation pass</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-rules</option>: + <indexterm><primary><option>-ddump-rules</option></primary></indexterm> + </term> + <listitem> + <para>dumps all rewrite rules (including those generated + by the specialisation pass)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-simpl</option>: + <indexterm><primary><option>-ddump-simpl</option></primary></indexterm> + </term> + <listitem> + <para>simplifier output (Core-to-Core passes)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-inlinings</option>: + <indexterm><primary><option>-ddump-inlinings</option></primary></indexterm> + </term> + <listitem> + <para>inlining info from the simplifier</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-usagesp</option>: + <indexterm><primary><option>-ddump-usagesp</option></primary></indexterm> + </term> + <listitem> + <para>UsageSP inference pre-inf and output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-cpranal</option>: + <indexterm><primary><option>-ddump-cpranal</option></primary></indexterm> + </term> + <listitem> + <para>CPR analyser output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-stranal</option>: + <indexterm><primary><option>-ddump-stranal</option></primary></indexterm> + </term> + <listitem> + <para>strictness analyser output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-cse</option>: + <indexterm><primary><option>-ddump-cse</option></primary></indexterm> + </term> + <listitem> + <para>CSE pass output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-workwrap</option>: + <indexterm><primary><option>-ddump-workwrap</option></primary></indexterm> + </term> + <listitem> + <para>worker/wrapper split output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-occur-anal</option>: + <indexterm><primary><option>-ddump-occur-anal</option></primary></indexterm> + </term> + <listitem> + <para>`occurrence analysis' output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-sat</option>: + <indexterm><primary><option>-ddump-sat</option></primary></indexterm> + </term> + <listitem> + <para>output of “saturate” pass</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-stg</option>: + <indexterm><primary><option>-ddump-stg</option></primary></indexterm> + </term> + <listitem> + <para>output of STG-to-STG passes</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-absC</option>: + <indexterm><primary><option>-ddump-absC</option></primary></indexterm> + </term> + <listitem> + <para><emphasis>un</emphasis>flattened Abstract C</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-flatC</option>: + <indexterm><primary><option>-ddump-flatC</option></primary></indexterm> + </term> + <listitem> + <para><emphasis>flattened</emphasis> Abstract C</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-realC</option>: + <indexterm><primary><option>-ddump-realC</option></primary></indexterm> + </term> + <listitem> + <para>same as what goes to the C compiler</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-stix</option>: + <indexterm><primary><option>-ddump-stix</option></primary></indexterm> + </term> + <listitem> + <para>native-code generator intermediate form</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-asm</option>: + <indexterm><primary><option>-ddump-asm</option></primary></indexterm> + </term> + <listitem> + <para>assembly language from the native-code generator</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-bcos</option>: + <indexterm><primary><option>-ddump-bcos</option></primary></indexterm> + </term> + <listitem> + <para>byte code compiler output</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-foreign</option>: + <indexterm><primary><option>-ddump-foreign</option></primary></indexterm> + </term> + <listitem> + <para>dump foreign export stubs</para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dverbose-core2core</option> + <indexterm><primary><option>-dverbose-core2core</option></primary></indexterm> + </term> + <term> + <option>-dverbose-stg2stg</option> + <indexterm><primary><option>-dverbose-stg2stg</option></primary></indexterm> + </term> + <listitem> + <para>Show the output of the intermediate Core-to-Core and + STG-to-STG passes, respectively. (<emphasis>Lots</emphasis> + of output!) So: when we're really desperate:</para> + + <screen> +% ghc -noC -O -ddump-simpl -dverbose-simpl -dcore-lint Foo.hs +</screen> + + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-simpl-iterations</option>: + <indexterm><primary><option>-ddump-simpl-iterations</option></primary></indexterm> + </term> + <listitem> + <para>Show the output of each <emphasis>iteration</emphasis> + of the simplifier (each run of the simplifier has a maximum + number of iterations, normally 4). Used when even + <option>-dverbose-simpl</option> doesn't cut it.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dppr-debug</option> + <indexterm><primary><option>-dppr-debug</option></primary></indexterm> + </term> + <listitem> + <para>Debugging output is in one of several + “styles.” Take the printing of types, for + example. In the “user” style (the default), the + compiler's internal ideas about types are presented in + Haskell source-level syntax, insofar as possible. In the + “debug” style (which is the default for + debugging output), the types are printed in with explicit + foralls, and variables have their unique-id attached (so you + can check for things that look the same but aren't). This + flag makes debugging output appear in the more verbose debug + style.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dppr-user-length</option> + <indexterm><primary><option>-dppr-user-length</option></primary></indexterm> + </term> + <listitem> + <para>In error messages, expressions are printed to a + certain “depth”, with subexpressions beyond the + depth replaced by ellipses. This flag sets the + depth.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-simpl-stats</option> + <indexterm><primary><option>-ddump-simpl-stats option</option></primary></indexterm> + </term> + <listitem> + <para>Dump statistics about how many of each kind of + transformation too place. If you add + <option>-dppr-debug</option> you get more detailed + information.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-rn-trace</option> + <indexterm><primary><option>-ddump-rn-trace</option></primary></indexterm> + </term> + <listitem> + <para>Make the renamer be *real* chatty about what it is + upto.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-rn-stats</option> + <indexterm><primary><option>-dshow-rn-stats</option></primary></indexterm> + </term> + <listitem> + <para>Print out summary of what kind of information the renamer + had to bring in.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dshow-unused-imports</option> + <indexterm><primary><option>-dshow-unused-imports</option></primary></indexterm> + </term> + <listitem> + <para>Have the renamer report what imports does not + contribute.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="checking-consistency"> + <title>Checking for consistency</title> + + <indexterm><primary>consistency checks</primary></indexterm> + <indexterm><primary>lint</primary></indexterm> + + <variablelist> + + <varlistentry> + <term> + <option>-dcore-lint</option> + <indexterm><primary><option>-dcore-lint</option></primary></indexterm> + </term> + <listitem> + <para>Turn on heavyweight intra-pass sanity-checking within + GHC, at Core level. (It checks GHC's sanity, not yours.)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dstg-lint</option>: + <indexterm><primary><option>-dstg-lint</option></primary></indexterm> + </term> + <listitem> + <para>Ditto for STG level. (NOTE: currently doesn't work).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dusagesp-lint</option>: + <indexterm><primary><option>-dstg-lint</option></primary></indexterm> + </term> + <listitem> + <para>Turn on checks around UsageSP inference + (<option>-fusagesp</option>). This verifies various simple + properties of the results of the inference, and also warns + if any identifier with a used-once annotation before the + inference has a used-many annotation afterwards; this could + indicate a non-worksafe transformation is being + applied.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2> + <title>How to read Core syntax (from some <option>-ddump</option> + flags)</title> + + <indexterm><primary>reading Core syntax</primary></indexterm> + <indexterm><primary>Core syntax, how to read</primary></indexterm> + + <para>Let's do this by commenting an example. It's from doing + <option>-ddump-ds</option> on this code: + +<programlisting> +skip2 m = m : skip2 (m+2) +</programlisting> + + Before we jump in, a word about names of things. Within GHC, + variables, type constructors, etc., are identified by their + “Uniques.” These are of the form `letter' plus + `number' (both loosely interpreted). The `letter' gives some idea + of where the Unique came from; e.g., <literal>_</literal> + means “built-in type variable”; <literal>t</literal> + means “from the typechecker”; <literal>s</literal> + means “from the simplifier”; and so on. The `number' + is printed fairly compactly in a `base-62' format, which everyone + hates except me (WDP).</para> + + <para>Remember, everything has a “Unique” and it is + usually printed out when debugging, in some form or another. So + here we go…</para> + +<programlisting> +Desugared: +Main.skip2{-r1L6-} :: _forall_ a$_4 =>{{Num a$_4}} -> a$_4 -> [a$_4] + +--# `r1L6' is the Unique for Main.skip2; +--# `_4' is the Unique for the type-variable (template) `a' +--# `{{Num a$_4}}' is a dictionary argument + +_NI_ + +--# `_NI_' means "no (pragmatic) information" yet; it will later +--# evolve into the GHC_PRAGMA info that goes into interface files. + +Main.skip2{-r1L6-} = + /\ _4 -> \ d.Num.t4Gt -> + let { + {- CoRec -} + +.t4Hg :: _4 -> _4 -> _4 + _NI_ + +.t4Hg = (+{-r3JH-} _4) d.Num.t4Gt + + fromInt.t4GS :: Int{-2i-} -> _4 + _NI_ + fromInt.t4GS = (fromInt{-r3JX-} _4) d.Num.t4Gt + +--# The `+' class method (Unique: r3JH) selects the addition code +--# from a `Num' dictionary (now an explicit lambda'd argument). +--# Because Core is 2nd-order lambda-calculus, type applications +--# and lambdas (/\) are explicit. So `+' is first applied to a +--# type (`_4'), then to a dictionary, yielding the actual addition +--# function that we will use subsequently... + +--# We play the exact same game with the (non-standard) class method +--# `fromInt'. Unsurprisingly, the type `Int' is wired into the +--# compiler. + + lit.t4Hb :: _4 + _NI_ + lit.t4Hb = + let { + ds.d4Qz :: Int{-2i-} + _NI_ + ds.d4Qz = I#! 2# + } in fromInt.t4GS ds.d4Qz + +--# `I# 2#' is just the literal Int `2'; it reflects the fact that +--# GHC defines `data Int = I# Int#', where Int# is the primitive +--# unboxed type. (see relevant info about unboxed types elsewhere...) + +--# The `!' after `I#' indicates that this is a *saturated* +--# application of the `I#' data constructor (i.e., not partially +--# applied). + + skip2.t3Ja :: _4 -> [_4] + _NI_ + skip2.t3Ja = + \ m.r1H4 -> + let { ds.d4QQ :: [_4] + _NI_ + ds.d4QQ = + let { + ds.d4QY :: _4 + _NI_ + ds.d4QY = +.t4Hg m.r1H4 lit.t4Hb + } in skip2.t3Ja ds.d4QY + } in + :! _4 m.r1H4 ds.d4QQ + + {- end CoRec -} + } in skip2.t3Ja +</programlisting> + + <para>(“It's just a simple functional language” is an + unregisterised trademark of Peyton Jones Enterprises, plc.)</para> + + </sect2> + + <sect2 id="unreg"> + <title>Unregisterised compilation</title> + <indexterm><primary>unregisterised compilation</primary></indexterm> + + <para>The term "unregisterised" really means "compile via vanilla + C", disabling some of the platform-specific tricks that GHC + normally uses to make programs go faster. When compiling + unregisterised, GHC simply generates a C file which is compiled + via gcc.</para> + + <para>Unregisterised compilation can be useful when porting GHC to + a new machine, since it reduces the prerequisite tools to + <command>gcc</command>, <command>as</command>, and + <command>ld</command> and nothing more, and furthermore the amount + of platform-specific code that needs to be written in order to get + unregisterised compilation going is usually fairly small.</para> + + <variablelist> + <varlistentry> + <term> + <option>-unreg</option>: + <indexterm><primary><option>-unreg</option></primary></indexterm> + </term> + <listitem> + <para>Compile via vanilla ANSI C only, turning off + platform-specific optimisations. NOTE: in order to use + <option>-unreg</option>, you need to have a set of libraries + (including the RTS) built for unregisterised compilation. + This amounts to building GHC with way "u" enabled.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/ffi-chap.xml b/docs/users_guide/ffi-chap.xml new file mode 100644 index 0000000000..e1374c4610 --- /dev/null +++ b/docs/users_guide/ffi-chap.xml @@ -0,0 +1,411 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!-- FFI docs as a chapter --> + +<chapter id="ffi"> + <title> +Foreign function interface (FFI) + </title> + + <para>GHC (mostly) conforms to the Haskell 98 Foreign Function Interface + Addendum 1.0, whose definition is available from <ulink url="http://haskell.org/"><literal>http://haskell.org/</literal></ulink>.</para> + + <para>To enable FFI support in GHC, give the <option>-fffi</option><indexterm><primary><option>-fffi</option></primary> + </indexterm>flag, or +the <option>-fglasgow-exts</option><indexterm><primary><option>-fglasgow-exts</option></primary> + </indexterm> flag which implies <option>-fffi</option> +.</para> + + <para>The FFI support in GHC diverges from the Addendum in the following ways:</para> + + <itemizedlist> + <listitem> + <para>Syntactic forms and library functions proposed in earlier versions + of the FFI are still supported for backwards compatibility.</para> + </listitem> + + <listitem> + <para>GHC implements a number of GHC-specific extensions to the FFI + Addendum. These extensions are described in <xref linkend="sec-ffi-ghcexts" />, but please note that programs using + these features are not portable. Hence, these features should be + avoided where possible.</para> + </listitem> + </itemizedlist> + + <para>The FFI libraries are documented in the accompanying library + documentation; see for example the <literal>Foreign</literal> + module.</para> + + <sect1 id="sec-ffi-ghcexts"> + <title>GHC extensions to the FFI Addendum</title> + + <para>The FFI features that are described in this section are specific to + GHC. Avoid them where possible to not compromise the portability of the + resulting code.</para> + + <sect2> + <title>Unboxed types</title> + + <para>The following unboxed types may be used as basic foreign types + (see FFI Addendum, Section 3.2): <literal>Int#</literal>, + <literal>Word#</literal>, <literal>Char#</literal>, + <literal>Float#</literal>, <literal>Double#</literal>, + <literal>Addr#</literal>, <literal>StablePtr# a</literal>, + <literal>MutableByteArray#</literal>, <literal>ForeignObj#</literal>, + and <literal>ByteArray#</literal>.</para> + </sect2> + + </sect1> + + <sect1 id="sec-ffi-ghc"> + <title>Using the FFI with GHC</title> + + <para>The following sections also give some hints and tips on the + use of the foreign function interface in GHC.</para> + + <sect2 id="foreign-export-ghc"> + <title>Using <literal>foreign export</literal> and <literal>foreign import ccall "wrapper"</literal> with GHC</title> + + <indexterm><primary><literal>foreign export + </literal></primary><secondary>with GHC</secondary> + </indexterm> + + <para>When GHC compiles a module (say <filename>M.hs</filename>) + which uses <literal>foreign export</literal> or + <literal>foreign import "wrapper"</literal>, it generates two + additional files, <filename>M_stub.c</filename> and + <filename>M_stub.h</filename>. GHC will automatically compile + <filename>M_stub.c</filename> to generate + <filename>M_stub.o</filename> at the same time.</para> + + <para>For a plain <literal>foreign export</literal>, the file + <filename>M_stub.h</filename> contains a C prototype for the + foreign exported function, and <filename>M_stub.c</filename> + contains its definition. For example, if we compile the + following module:</para> + +<programlisting> +module Foo where + +foreign export ccall foo :: Int -> IO Int + +foo :: Int -> IO Int +foo n = return (length (f n)) + +f :: Int -> [Int] +f 0 = [] +f n = n:(f (n-1))</programlisting> + + <para>Then <filename>Foo_stub.h</filename> will contain + something like this:</para> + +<programlisting> +#include "HsFFI.h" +extern HsInt foo(HsInt a0);</programlisting> + + <para>and <filename>Foo_stub.c</filename> contains the + compiler-generated definition of <literal>foo()</literal>. To + invoke <literal>foo()</literal> from C, just <literal>#include + "Foo_stub.h"</literal> and call <literal>foo()</literal>.</para> + + <para>The <filename>foo_stub.c</filename> and + <filename>foo_stub.h</filename> files can be redirected using the + <option>-stubdir</option> option; see <xref linkend="options-output" + />.</para> + + <sect3 id="using-own-main"> + <title>Using your own <literal>main()</literal></title> + + <para>Normally, GHC's runtime system provides a + <literal>main()</literal>, which arranges to invoke + <literal>Main.main</literal> in the Haskell program. However, + you might want to link some Haskell code into a program which + has a main function written in another language, say C. In + order to do this, you have to initialize the Haskell runtime + system explicitly.</para> + + <para>Let's take the example from above, and invoke it from a + standalone C program. Here's the C code:</para> + +<programlisting> +#include <stdio.h> +#include "HsFFI.h" + +#ifdef __GLASGOW_HASKELL__ +#include "foo_stub.h" +#endif + +#ifdef __GLASGOW_HASKELL__ +extern void __stginit_Foo ( void ); +#endif + +int main(int argc, char *argv[]) +{ + int i; + + hs_init(&argc, &argv); +#ifdef __GLASGOW_HASKELL__ + hs_add_root(__stginit_Foo); +#endif + + for (i = 0; i < 5; i++) { + printf("%d\n", foo(2500)); + } + + hs_exit(); + return 0; +}</programlisting> + + <para>We've surrounded the GHC-specific bits with + <literal>#ifdef __GLASGOW_HASKELL__</literal>; the rest of the + code should be portable across Haskell implementations that + support the FFI standard.</para> + + <para>The call to <literal>hs_init()</literal> + initializes GHC's runtime system. Do NOT try to invoke any + Haskell functions before calling + <literal>hs_init()</literal>: strange things will + undoubtedly happen.</para> + + <para>We pass <literal>argc</literal> and + <literal>argv</literal> to <literal>hs_init()</literal> + so that it can separate out any arguments for the RTS + (i.e. those arguments between + <literal>+RTS...-RTS</literal>).</para> + + <para>Next, we call + <function>hs_add_root</function><indexterm><primary><function>hs_add_root</function></primary> + </indexterm>, a GHC-specific interface which is required to + initialise the Haskell modules in the program. The argument + to <function>hs_add_root</function> should be the name of the + initialization function for the "root" module in your program + - in other words, the module which directly or indirectly + imports all the other Haskell modules in the program. In a + standalone Haskell program the root module is normally + <literal>Main</literal>, but when you are using Haskell code + from a library it may not be. If your program has multiple + root modules, then you can call + <function>hs_add_root</function> multiple times, one for each + root. The name of the initialization function for module + <replaceable>M</replaceable> is + <literal>__stginit_<replaceable>M</replaceable></literal>, and + it may be declared as an external function symbol as in the + code above.</para> + + <para>After we've finished invoking our Haskell functions, we + can call <literal>hs_exit()</literal>, which + terminates the RTS. It runs any outstanding finalizers and + generates any profiling or stats output that might have been + requested.</para> + + <para>There can be multiple calls to + <literal>hs_init()</literal>, but each one should be matched + by one (and only one) call to + <literal>hs_exit()</literal><footnote><para>The outermost + <literal>hs_exit()</literal> will actually de-initialise the + system. NOTE that currently GHC's runtime cannot reliably + re-initialise after this has happened.</para> + </footnote>.</para> + + <para>NOTE: when linking the final program, it is normally + easiest to do the link using GHC, although this isn't + essential. If you do use GHC, then don't forget the flag + <option>-no-hs-main</option><indexterm><primary><option>-no-hs-main</option></primary> + </indexterm>, otherwise GHC will try to link + to the <literal>Main</literal> Haskell module.</para> + </sect3> + + <sect3 id="foreign-export-dynamic-ghc"> + <title>Using <literal>foreign import ccall "wrapper"</literal> with GHC</title> + + <indexterm><primary><literal>foreign import + ccall "wrapper"</literal></primary><secondary>with GHC</secondary> + </indexterm> + + <para>When <literal>foreign import ccall "wrapper"</literal> is used + in a Haskell module, The C stub file <filename>M_stub.c</filename> + generated by GHC contains small helper functions used by the code + generated for the imported wrapper, so it must be linked in to the + final program. When linking the program, remember to include + <filename>M_stub.o</filename> in the final link command line, or + you'll get link errors for the missing function(s) (this isn't + necessary when building your program with <literal>ghc + ––make</literal>, as GHC will automatically link in the + correct bits).</para> + </sect3> + </sect2> + + <sect2 id="glasgow-foreign-headers"> + <title>Using function headers</title> + + <indexterm><primary>C calls, function headers</primary></indexterm> + + <para>When generating C (using the <option>-fvia-C</option> + directive), one can assist the C compiler in detecting type + errors by using the <option>-#include</option> directive + (<xref linkend="options-C-compiler"/>) to provide + <filename>.h</filename> files containing function + headers.</para> + + <para>For example,</para> + +<programlisting> +#include "HsFFI.h" + +void initialiseEFS (HsInt size); +HsInt terminateEFS (void); +HsForeignObj emptyEFS(void); +HsForeignObj updateEFS (HsForeignObj a, HsInt i, HsInt x); +HsInt lookupEFS (HsForeignObj a, HsInt i); +</programlisting> + + <para>The types <literal>HsInt</literal>, + <literal>HsForeignObj</literal> etc. are described in the H98 FFI + Addendum.</para> + + <para>Note that this approach is only + <emphasis>essential</emphasis> for returning + <literal>float</literal>s (or if <literal>sizeof(int) != + sizeof(int *)</literal> on your architecture) but is a Good + Thing for anyone who cares about writing solid code. You're + crazy not to do it.</para> + +<para> +What if you are importing a module from another package, and +a cross-module inlining exposes a foreign call that needs a supporting +<option>-#include</option>? If the imported module is from the same package as +the module being compiled, you should supply all the <option>-#include</option> +that you supplied when compiling the imported module. If the imported module comes +from another package, you won't necessarily know what the appropriate +<option>-#include</option> options are; but they should be in the package +configuration, which GHC knows about. So if you are building a package, remember +to put all those <option>-#include</option> options into the package configuration. +See the <literal>c_includes</literal> field in <xref linkend="package-management"/>. +</para> + +<para> +It is also possible, according the FFI specification, to put the +<option>-#include</option> option in the foreign import +declaration itself: +<programlisting> + foreign import "foo.h f" f :: Int -> IO Int +</programlisting> +When compiling this module, GHC will generate a C file that includes +the specified <option>-#include</option>. However, GHC +<emphasis>disables</emphasis> cross-module inlining for such foreign +calls, because it doesn't transport the <option>-#include</option> +information across module boundaries. (There is no fundamental reason for this; +it was just tiresome to implement. The wrapper, which unboxes the arguments +etc, is still inlined across modules.) So if you want the foreign call itself +to be inlined across modules, use the command-line and package-configuration +<option>-#include</option> mechanism. +</para> + + <sect3 id="finding-header-files"> + <title>Finding Header files</title> + + <para>Header files named by the <option>-#include</option> + option or in a <literal>foreign import</literal> declaration + are searched for using the C compiler's usual search path. + You can add directories to this search path using the + <option>-I</option> option (see <xref + linkend="c-pre-processor"/>).</para> + + <para>Note: header files are ignored unless compiling via C. + If you had been compiling your code using the native code + generator (the default) and suddenly switch to compiling via + C, then you can get unexpected errors about missing include + files. Compiling via C is enabled automatically when certain + options are given (eg. <option>-O</option> and + <option>-prof</option> both enable + <option>-fvia-C</option>).</para> + </sect3> + + </sect2> + + <sect2> + <title>Memory Allocation</title> + + <para>The FFI libraries provide several ways to allocate memory + for use with the FFI, and it isn't always clear which way is the + best. This decision may be affected by how efficient a + particular kind of allocation is on a given compiler/platform, + so this section aims to shed some light on how the different + kinds of allocation perform with GHC.</para> + + <variablelist> + <varlistentry> + <term><literal>alloca</literal> and friends</term> + <listitem> + <para>Useful for short-term allocation when the allocation + is intended to scope over a given <literal>IO</literal> + computation. This kind of allocation is commonly used + when marshalling data to and from FFI functions.</para> + + <para>In GHC, <literal>alloca</literal> is implemented + using <literal>MutableByteArray#</literal>, so allocation + and deallocation are fast: much faster than C's + <literal>malloc/free</literal>, but not quite as fast as + stack allocation in C. Use <literal>alloca</literal> + whenever you can.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>mallocForeignPtr</literal></term> + <listitem> + <para>Useful for longer-term allocation which requires + garbage collection. If you intend to store the pointer to + the memory in a foreign data structure, then + <literal>mallocForeignPtr</literal> is + <emphasis>not</emphasis> a good choice, however.</para> + + <para>In GHC, <literal>mallocForeignPtr</literal> is also + implemented using <literal>MutableByteArray#</literal>. + Although the memory is pointed to by a + <literal>ForeignPtr</literal>, there are no actual + finalizers involved (unless you add one with + <literal>addForeignPtrFinalizer</literal>), and the + deallocation is done using GC, so + <literal>mallocForeignPtr</literal> is normally very + cheap.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>malloc/free</literal></term> + <listitem> + <para>If all else fails, then you need to resort to + <literal>Foreign.malloc</literal> and + <literal>Foreign.free</literal>. These are just wrappers + around the C functions of the same name, and their + efficiency will depend ultimately on the implementations + of these functions in your platform's C library. We + usually find <literal>malloc</literal> and + <literal>free</literal> to be significantly slower than + the other forms of allocation above.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>Foreign.Marshal.Pool</literal></term> + <listitem> + <para>Pools are currently implemented using + <literal>malloc/free</literal>, so while they might be a + more convenient way to structure your memory allocation + than using one of the other forms of allocation, they + won't be any more efficient. We do plan to provide an + improved-performance implementation of Pools in the + future, however.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + </sect1> +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/flags.xml b/docs/users_guide/flags.xml new file mode 100644 index 0000000000..e288da2fb2 --- /dev/null +++ b/docs/users_guide/flags.xml @@ -0,0 +1,1894 @@ +<?xml version="1.0" encoding="iso-8859-1"?> + <sect1 id="flag-reference"> + <title>Flag reference</title> + + <para>This section is a quick-reference for GHC's command-line + flags. For each flag, we also list its static/dynamic status (see + <xref linkend="static-dynamic-flags"/>), and the flag's opposite + (if available).</para> + + <sect2> + <title>Help and verbosity options</title> + + <para><xref linkend="options-help"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-?</option></entry> + <entry>help</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-help</option></entry> + <entry>help</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-v</option></entry> + <entry>verbose mode (equivalent to <option>-v3</option>)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-v</option><replaceable>n</replaceable></entry> + <entry>set verbosity level</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-V</option></entry> + <entry>display GHC version</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>––version</option></entry> + <entry>display GHC version</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>––numeric-version</option></entry> + <entry>display GHC version (numeric only)</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>––print-libdir</option></entry> + <entry>display GHC library directory</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ferror-spans</option></entry> + <entry>output full span in error messages</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-Rghc-timing</option></entry> + <entry>Summarise timing stats for GHC (same as <literal>+RTS -tstderr</literal>)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + + </sect2> + <sect2> + <title>Which phases to run</title> + + <para><xref linkend="options-order"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-E</option></entry> + <entry>Stop after preprocessing (<literal>.hspp</literal> file)</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-C</option></entry> + <entry>Stop after generating C (<literal>.hc</literal> file)</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-S</option></entry> + <entry>Stop after generating assembly (<literal>.s</literal> file)</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-c</option></entry> + <entry>Do not link</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-x</option> <replaceable>suffix</replaceable></entry> + <entry>Override default behaviour for source files</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Alternative modes of operation</title> + + <para><xref linkend="modes"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>--interactive</option></entry> + <entry>Interactive mode - normally used by just running <command>ghci</command></entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>--make</option></entry> + <entry>Build a multi-module Haskell program, automatically figuring out dependencies. Likely to be much easier, and faster, than using <command>make</command>.</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-e <replaceable>expr</replaceable></option></entry> + <entry>Evaluate <replaceable>expr</replaceable></entry> + <entry>mode</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-M</option></entry> + <entry>Generate dependency information suitable for use in a <filename>Makefile</filename>.</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Redirecting output</title> + + <para><xref linkend="options-output"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-hcsuf</option> <replaceable>suffix</replaceable></entry> + <entry>set the suffix to use for intermediate C files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-hidir</option> <replaceable>dir</replaceable></entry> + <entry>set directory for interface files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-hisuf</option> <replaceable>suffix</replaceable></entry> + <entry>set the suffix to use for interface files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-o</option> <replaceable>filename</replaceable></entry> + <entry>set output filename</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-odir</option> <replaceable>dir</replaceable></entry> + <entry>set output directory</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ohi</option> <replaceable>filename</replaceable></entry> + <entry>set the filename in which to put the interface</entry> + <entry>dynamic</entry> + <entry></entry> + </row> + <row> + <entry><option>-osuf</option> <replaceable>suffix</replaceable></entry> + <entry>set the output file suffix</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-stubdir</option> <replaceable>dir</replaceable></entry> + <entry>redirect FFi stub files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Keeping intermediate files</title> + + <para><xref linkend="keeping-intermediates"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-keep-hc-file</option></entry> + <entry>retain intermediate <literal>.hc</literal> files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-keep-s-file</option></entry> + <entry>retain intermediate <literal>.s</literal> files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-keep-raw-s-file</option></entry> + <entry>retain intermediate <literal>.raw_s</literal> files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-keep-tmp-files</option></entry> + <entry>retain all intermediate temporary files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Temporary files</title> + + <para><xref linkend="temp-files"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-tmpdir</option></entry> + <entry>set the directory for temporary files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Finding imports</title> + + <para><xref linkend="search-path"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-i</option><replaceable>dir1</replaceable>:<replaceable>dir2</replaceable>:...</entry> + <entry>add <replaceable>dir</replaceable>, + <replaceable>dir2</replaceable>, etc. to import path</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-i</option></entry> + <entry>Empty the import directory list</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Interface file options</title> + + <para><xref linkend="hi-options"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-ddump-hi</option></entry> + <entry>Dump the new interface to stdout</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-hi-diffs</option></entry> + <entry>Show the differences vs. the old interface</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-minimal-imports</option></entry> + <entry>Dump a minimal set of imports</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>--show-iface</option> <replaceable>file</replaceable></entry> + <entry>Read the interface in + <replaceable>file</replaceable> and dump it as text to + <literal>stdout</literal>.</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Recompilation checking</title> + + <para><xref linkend="recomp"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-no-recomp</option></entry> + <entry>Turn off recompilation checking; implied by any + <option>-ddump-X</option> option</entry> + <entry>dynamic</entry> + <entry><option>-recomp</option></entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Interactive-mode options</title> + + <para><xref linkend="ghci-dot-files"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-ignore-dot-ghci</option></entry> + <entry>Disable reading of <filename>.ghci</filename> files</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-read-dot-ghci</option></entry> + <entry>Enable reading of <filename>.ghci</filename> files</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Packages</title> + + <para><xref linkend="packages"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-package</option> <replaceable>P</replaceable></entry> + <entry>Expose package <replaceable>P</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-hide-all-packages</option></entry> + <entry>Hide all packages by default</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-hide-package</option> <replaceable>name</replaceable></entry> + <entry>Hide package <replaceable>P</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ignore-package</option> <replaceable>name</replaceable></entry> + <entry>Ignore package <replaceable>P</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-package-conf</option> <replaceable>file</replaceable></entry> + <entry>Load more packages from <replaceable>file</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-no-user-package-conf</option></entry> + <entry>Don't load the user's package config file.</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Language options</title> + + <para><xref linkend="options-language"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-fallow-overlapping-instances</option></entry> + <entry>Enable overlapping instances</entry> + <entry>dynamic</entry> + <entry><option>-fno-allow-overlapping-instances</option></entry> + </row> + <row> + <entry><option>-fallow-undecidable-instances</option></entry> + <entry>Enable undecidable instances</entry> + <entry>dynamic</entry> + <entry><option>-fno-allow-undecidable-instances</option></entry> + </row> + <row> + <entry><option>-fallow-incoherent-instances</option></entry> + <entry>Enable incoherent instances. + Implies <option>-fallow-overlapping-instances</option> </entry> + <entry>dynamic</entry> + <entry><option>-fno-allow-incoherent-instances</option></entry> + </row> + <row> + <entry><option>-farrows</option></entry> + <entry>Enable arrow notation extension</entry> + <entry>dynamic</entry> + <entry><option>-fno-arrows</option></entry> + </row> + <row> + <entry><option>-fcontext-stack</option><replaceable>n</replaceable></entry> + <entry>set the limit for context reduction</entry> + <entry>static</entry> + <entry><option>-</option></entry> + </row> + <row> + <entry><option>-ffi</option> or <option>-fffi</option></entry> + <entry>Enable foreign function interface (implied by + <option>-fglasgow-exts</option>)</entry> + <entry>dynamic</entry> + <entry><option>-fno-ffi</option></entry> + </row> + <row> + <entry><option>-fgenerics</option></entry> + <entry>Enable generics</entry> + <entry>dynamic</entry> + <entry><option>-fno-fgenerics</option></entry> + </row> + <row> + <entry><option>-fglasgow-exts</option></entry> + <entry>Enable most language extensions</entry> + <entry>dynamic</entry> + <entry><option>-fno-glasgow-exts</option></entry> + </row> + <row> + <entry><option>-fimplicit-params</option></entry> + <entry>Enable Implicit Parameters. + Implied by <option>-fglasgow-exts</option>.</entry> + <entry>dynamic</entry> + <entry><option>-fno-implicit-params</option></entry> + </row> + <row> + <entry><option>-firrefutable-tuples</option></entry> + <entry>Make tuple pattern matching irrefutable</entry> + <entry>dynamic</entry> + <entry><option>-fno-irrefutable-tuples</option></entry> + </row> + <row> + <entry><option>-fno-implicit-prelude</option></entry> + <entry>Don't implicitly <literal>import Prelude</literal></entry> + <entry>dynamic</entry> + <entry><option>-fimplicit-prelude</option></entry> + </row> + <row> + <entry><option>-fno-monomorphism-restriction</option></entry> + <entry>Disable the monomorphism restriction</entry> + <entry>dynamic</entry> + <entry><option>-fmonomorphism-restriction</option></entry> + </row> + <row> + <entry><option>-fscoped-type-variables</option></entry> + <entry>Enable lexically-scoped type variables. + Implied by <option>-fglasgow-exts</option>.</entry> + <entry>dynamic</entry> + <entry><option>-fno-scoped-type-variables</option></entry> + </row> + <row> + <entry><option>-fth</option></entry> + <entry>Enable Template Haskell. + Implied by <option>-fglasgow-exts</option>.</entry> + <entry>dynamic</entry> + <entry><option>-fno-th</option></entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Warnings</title> + + <para>(<xref linkend="options-sanity"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-W</option></entry> + <entry>enable normal warnings</entry> + <entry>dynamic</entry> + <entry><option>-w</option></entry> + </row> + <row> + <entry><option>-w</option></entry> + <entry>disable all warnings</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-Wall</option></entry> + <entry>enable all warnings</entry> + <entry>dynamic</entry> + <entry><option>-w</option></entry> + </row> + <row> + <entry><option>-Werror</option></entry> + <entry>make warnings fatal</entry> + <entry>dynamic</entry> + <entry></entry> + </row> + + <row> + <entry><option>-fwarn-deprecations</option></entry> + <entry>warn about uses of functions & types that are deprecated</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-deprecations</option></entry> + </row> + + <row> + <entry><option>-fwarn-duplicate-exports</option></entry> + <entry>warn when an entity is exported multiple times</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-duplicate-exports</option></entry> + </row> + + <row> + <entry><option>-fwarn-hi-shadowing</option></entry> + <entry>warn when a <literal>.hi</literal> file in the + current directory shadows a library</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-hi-shadowing</option></entry> + </row> + + <row> + <entry><option>-fwarn-incomplete-patterns</option></entry> + <entry>warn when a pattern match could fail</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-incomplete-patterns</option></entry> + </row> + + <row> + <entry><option>-fwarn-incomplete-record-updates</option></entry> + <entry>warn when a record update could fail</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-incomplete-record-updates</option></entry> + </row> + + <row> + <entry><option>-fwarn-misc</option></entry> + <entry>enable miscellaneous warnings</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-misc</option></entry> + </row> + + <row> + <entry><option>-fwarn-missing-fields</option></entry> + <entry>warn when fields of a record are uninitialised</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-missing-fields</option></entry> + </row> + + <row> + <entry><option>-fwarn-missing-methods</option></entry> + <entry>warn when class methods are undefined</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-missing-methods</option></entry> + </row> + + <row> + <entry><option>-fwarn-missing-signatures</option></entry> + <entry>warn about top-level functions without signatures</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-missing-signatures</option></entry> + </row> + + <row> + <entry><option>-fwarn-name-shadowing</option></entry> + <entry>warn when names are shadowed</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-name-shadowing</option></entry> + </row> + + <row> + <entry><option>-fwarn-oprhans</option></entry> + <entry>warn when the module contains "orphan" instance declarations + or rewrite rules</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-orphans</option></entry> + </row> + + <row> + <entry><option>-fwarn-overlapping-patterns</option></entry> + <entry>warn about overlapping patterns</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-overlapping-patterns</option></entry> + </row> + + <row> + <entry><option>-fwarn-simple-patterns</option></entry> + <entry>warn about lambda-patterns that can fail</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-simple-patterns</option></entry> + </row> + + <row> + <entry><option>-fwarn-type-defaults</option></entry> + <entry>warn when defaulting happens</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-type-defaults</option></entry> + </row> + + <row> + <entry><option>-fwarn-unused-binds</option></entry> + <entry>warn about bindings that are unused</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-unused-binds</option></entry> + </row> + + <row> + <entry><option>-fwarn-unused-imports</option></entry> + <entry>warn about unnecessary imports</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-unused-imports</option></entry> + </row> + + <row> + <entry><option>-fwarn-unused-matches</option></entry> + <entry>warn about variables in patterns that aren't used</entry> + <entry>dynamic</entry> + <entry><option>-fno-warn-unused-matches</option></entry> + </row> + + </tbody> + </tgroup> + </informaltable> + + </sect2> + <sect2> + <title>Optimisation levels</title> + + <para><xref linkend="options-optimise"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-O</option></entry> + <entry>Enable default optimisation (level 1)</entry> + <entry>dynamic</entry> + <entry><option>-O0</option></entry> + </row> + <row> + <entry><option>-O</option><replaceable>n</replaceable></entry> + <entry>Set optimisation level <replaceable>n</replaceable></entry> + <entry>dynamic</entry> + <entry><option>-O0</option></entry> + </row> + </tbody> + </tgroup> + </informaltable> + + </sect2> + <sect2> + <title>Individual optimisations</title> + + <para><xref linkend="options-f"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-fcase-merge</option></entry> + <entry>Enable case-merging</entry> + <entry>dynamic</entry> + <entry><option>-fno-case-merge</option></entry> + </row> + + <row> + <entry><option>-fdicts-strict</option></entry> + <entry>Make dictionaries strict</entry> + <entry>static</entry> + <entry><option>-fno-dicts-strict</option></entry> + </row> + + <row> + <entry><option>-fdo-eta-reduction</option></entry> + <entry>Enable eta-reduction</entry> + <entry>dynamic</entry> + <entry><option>-fno-do-eta-reduction</option></entry> + </row> + + <row> + <entry><option>-fdo-lambda-eta-expansion</option></entry> + <entry>Enable lambda eta-reduction</entry> + <entry>dynamic</entry> + <entry><option>-fno-do-lambda-eta-expansion</option></entry> + </row> + + <row> + <entry><option>-fexcess-precision</option></entry> + <entry>Enable excess intermediate precision</entry> + <entry>dynamic</entry> + <entry><option>-fno-excess-precision</option></entry> + </row> + + <row> + <entry><option>-frules-off</option></entry> + <entry>Switch off all rewrite rules (including rules + generated by automatic specialisation of overloaded functions)</entry> + <entry>static</entry> + <entry><option>-frules-off</option></entry> + </row> + + <row> + <entry><option>-fignore-asserts</option></entry> + <entry>Ignore assertions in the source</entry> + <entry>dynamic</entry> + <entry><option>-fno-ignore-asserts</option></entry> + </row> + + <row> + <entry><option>-fignore-interface-pragmas</option></entry> + <entry>Ignore pragmas in interface files</entry> + <entry>dynamic</entry> + <entry><option>-fno-ignore-interface-pragmas</option></entry> + </row> + + <row> + <entry><option>-fliberate-case-threshold</option></entry> + <entry>Tweak the liberate-case optimisation (default: 10)</entry> + <entry>static</entry> + <entry><option>-fno-liberate-case-threshold</option></entry> + </row> + + <row> + <entry><option>-fomit-interface-pragmas</option></entry> + <entry>Don't generate interface pragmas</entry> + <entry>dynamic</entry> + <entry><option>-fno-omit-interface-pragmas</option></entry> + </row> + + <row> + <entry><option>-fmax-worker-args</option></entry> + <entry>If a worker has that many arguments, none will be + unpacked anymore (default: 10)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-fmax-simplifier-iterations</option></entry> + <entry>Set the max iterations for the simplifier</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-fno-state-hack</option></entry> + <entry>Turn off the "state hack" whereby any lambda with a real-world state token + as argument is considered to be single-entry. Hence OK to inline things inside it.</entry> + <entry>static</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-fno-cse</option></entry> + <entry>Turn off common sub-expression</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-fno-full-laziness</option></entry> + <entry>Turn off full laziness (floating bindings outwards).</entry> + <entry>dynamic</entry> + <entry>-ffull-laziness</entry> + </row> + + <row> + <entry><option>-fno-pre-inlining</option></entry> + <entry>Turn off pre-inlining</entry> + <entry>static</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-fno-strictness</option></entry> + <entry>Turn off strictness analysis</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + + <row> + <entry><option>-funbox-strict-fields</option></entry> + <entry>Flatten strict constructor fields</entry> + <entry>dynamic</entry> + <entry><option>-fno-unbox-strict-fields</option></entry> + </row> + + <row> + <entry><option>-funfolding-creation-threshold</option></entry> + <entry>Tweak unfolding settings</entry> + <entry>static</entry> + <entry><option>-fno-unfolding-creation-threshold</option></entry> + </row> + + <row> + <entry><option>-funfolding-fun-discount</option></entry> + <entry>Tweak unfolding settings</entry> + <entry>static</entry> + <entry><option>-fno-unfolding-fun-discount</option></entry> + </row> + + <row> + <entry><option>-funfolding-keeness-factor</option></entry> + <entry>Tweak unfolding settings</entry> + <entry>static</entry> + <entry><option>-fno-unfolding-keeness-factor</option></entry> + </row> + + <row> + <entry><option>-funfolding-update-in-place</option></entry> + <entry>Tweak unfolding settings</entry> + <entry>static</entry> + <entry><option>-fno-unfolding-update-in-place</option></entry> + </row> + + <row> + <entry><option>-funfolding-use-threshold</option></entry> + <entry>Tweak unfolding settings</entry> + <entry>static</entry> + <entry><option>-fno-unfolding-use-threshold</option></entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Profiling options</title> + + <para><xref linkend="profiling"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-auto</option></entry> + <entry>Auto-add <literal>_scc_</literal>s to all + exported functions</entry> + <entry>static</entry> + <entry><option>-no-auto</option></entry> + </row> + <row> + <entry><option>-auto-all</option></entry> + <entry>Auto-add <literal>_scc_</literal>s to all + top-level functions</entry> + <entry>static</entry> + <entry><option>-no-auto-all</option></entry> + </row> + <row> + <entry><option>-auto-dicts</option></entry> + <entry>Auto-add <literal>_scc_</literal>s to all dictionaries</entry> + <entry>static</entry> + <entry><option>-no-auto-dicts</option></entry> + </row> + <row> + <entry><option>-caf-all</option></entry> + <entry>Auto-add <literal>_scc_</literal>s to all CAFs</entry> + <entry>static</entry> + <entry><option>-no-caf-all</option></entry> + </row> + <row> + <entry><option>-prof</option></entry> + <entry>Turn on profiling</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ticky</option></entry> + <entry>Turn on ticky-ticky profiling</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Parallelism options</title> + + <para><xref linkend="sec-using-parallel"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-gransim</option></entry> + <entry>Enable GRANSIM</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-parallel</option></entry> + <entry>Enable Parallel Haskell</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-smp</option></entry> + <entry>Enable SMP support</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>C pre-processor options</title> + + <para><xref linkend="c-pre-processor"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-cpp</option></entry> + <entry>Run the C pre-processor on Haskell source files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-D</option><replaceable>symbol</replaceable><optional>=<replaceable>value</replaceable></optional></entry> + <entry>Define a symbol in the C pre-processor</entry> + <entry>dynamic</entry> + <entry><option>-U</option><replaceable>symbol</replaceable></entry> + </row> + <row> + <entry><option>-U</option><replaceable>symbol</replaceable></entry> + <entry>Undefine a symbol in the C pre-processor</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-I</option><replaceable>dir</replaceable></entry> + <entry>Add <replaceable>dir</replaceable> to the + directory search list for <literal>#include</literal> files</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>C compiler options</title> + + <para><xref linkend="options-C-compiler"/></para> + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-#include</option> <replaceable>file</replaceable></entry> + <entry>Include <replaceable>file</replaceable> when + compiling the <filename>.hc</filename> file</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Code generation options</title> + + <para><xref linkend="options-codegen"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-fasm</option></entry> + <entry>Use the native code generator</entry> + <entry>dynamic</entry> + <entry>-fvia-C</entry> + </row> + <row> + <entry><option>-fvia-C</option></entry> + <entry>Compile via C</entry> + <entry>dynamic</entry> + <entry>-fasm</entry> + </row> + <row> + <entry><option>-fno-code</option></entry> + <entry>Omit code generation</entry> + <entry>mode</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Linking options</title> + + <para><xref linkend="options-linker"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-dynamic</option></entry> + <entry>Use dynamic Haskell libraries (if available)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-framework</option> <replaceable>name</replaceable></entry> + <entry>On Darwin/MacOS X only, link in the framework <replaceable>name</replaceable>. + This option corresponds to the <option>-framework</option> option for Apple's Linker.</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-framework-path</option> <replaceable>name</replaceable></entry> + <entry>On Darwin/MacOS X only, add <replaceable>dir</replaceable> to the list of + directories searched for frameworks. + This option corresponds to the <option>-F</option> option for Apple's Linker.</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-l</option><replaceable>lib</replaceable></entry> + <entry>Link in library <replaceable>lib</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-L</option><replaceable>dir</replaceable></entry> + <entry>Add <replaceable>dir</replaceable> to the list of + directories searched for libraries</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-main-is</option></entry> + <entry>Set main function</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>--mk-dll</option></entry> + <entry>DLL-creation mode (Windows only)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-no-hs-main</option></entry> + <entry>Don't assume this program contains <literal>main</literal></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-no-link</option></entry> + <entry>Omit linking</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-package</option> <replaceable>name</replaceable></entry> + <entry>Link in package <replaceable>name</replaceable></entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-split-objs</option></entry> + <entry>Split objects (for libraries)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-static</option></entry> + <entry>Use static Haskell libraries</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-threaded</option></entry> + <entry>Use the threaded runtime</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-debug</option></entry> + <entry>Use the debugging runtime</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Replacing phases</title> + + <para><xref linkend="replacing-phases"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-pgmL</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the literate pre-processor</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgmP</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the C + pre-processor (with <option>-cpp</option> only)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgmc</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the C compiler</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgma</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the assembler</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgml</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the linker</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgmdll</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the DLL generator</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgmdep</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the dependency generator</entry> + <entry>dyanmic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-pgmF</option> <replaceable>cmd</replaceable></entry> + <entry>Use <replaceable>cmd</replaceable> as the pre-processor + (with <option>-F</option> only)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + <indexterm><primary><option>-pgmL</option></primary></indexterm> + <indexterm><primary><option>-pgmP</option></primary></indexterm> + <indexterm><primary><option>-pgmc</option></primary></indexterm> + <indexterm><primary><option>-pgma</option></primary></indexterm> + <indexterm><primary><option>-pgml</option></primary></indexterm> + <indexterm><primary><option>-pgmdll</option></primary></indexterm> + <indexterm><primary><option>-pgmdep</option></primary></indexterm> + <indexterm><primary><option>-pgmF</option></primary></indexterm> + + </sect2> + + <sect2> + <title>Forcing options to particular phases</title> + + <para><xref linkend="forcing-options-through"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-optL</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the literate pre-processor</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optP</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to cpp (with + <option>-cpp</option> only)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optF</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the + custom pre-processor</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optc</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the C compiler</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-opta</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the assembler</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optl</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the linker</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optdll</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the DLL generator</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-optdep</option> <replaceable>option</replaceable></entry> + <entry>pass <replaceable>option</replaceable> to the dependency generator</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Platform-specific options</title> + + <para><xref linkend="options-platform"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-mv8</option></entry> + <entry>(SPARC only) enable version 8 support</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-monly-[32]-regs</option></entry> + <entry>(x86 only) give some registers back to the C compiler</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + + <sect2> + <title>External core file options</title> + + <para><xref linkend="ext-core"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-fext-core</option></entry> + <entry>Generate <filename>.hcr</filename> external Core files</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + + <sect2> + <title>Compiler debugging options</title> + + <para><xref linkend="options-debugging"/></para> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-dcore-lint</option></entry> + <entry>Turn on internal sanity checking</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-absC</option></entry> + <entry>Dump abstract C</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-asm</option></entry> + <entry>Dump assembly</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-bcos</option></entry> + <entry>Dump interpreter byte code</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-cpranal</option></entry> + <entry>Dump output from CPR analysis</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-cse</option></entry> + <entry>Dump CSE output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-deriv</option></entry> + <entry>Dump deriving output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-ds</option></entry> + <entry>Dump desugarer output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-flatC</option></entry> + <entry>Dump “flat” C</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-foreign</option></entry> + <entry>Dump <literal>foreign export</literal> stubs</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-inlinings</option></entry> + <entry>Dump inlining info</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-occur-anal</option></entry> + <entry>Dump occurrence analysis output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-parsed</option></entry> + <entry>Dump parse tree</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-realC</option></entry> + <entry>Dump “real” C</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-rn</option></entry> + <entry>Dump renamer output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-rules</option></entry> + <entry>Dump rules</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-sat</option></entry> + <entry>Dump saturated output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-simpl</option></entry> + <entry>Dump final simplifier output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-simpl-iterations</option></entry> + <entry>Dump output from each simplifier iteration</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-spec</option></entry> + <entry>Dump specialiser output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-stg</option></entry> + <entry>Dump final STG</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-stranal</option></entry> + <entry>Dump strictness analyser output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-tc</option></entry> + <entry>Dump typechecker output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-types</option></entry> + <entry>Dump type signatures</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-usagesp</option></entry> + <entry>Dump UsageSP analysis output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-worker-wrapper</option></entry> + <entry>Dump worker-wrapper output</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-rn-trace</option></entry> + <entry>Trace renamer</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-rn-stats</option></entry> + <entry>Renamer stats</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-stix</option></entry> + <entry>Native code generator intermediate form</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-ddump-simpl-stats</option></entry> + <entry>Dump simplifier stats</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dppr-debug</option></entry> + <entry>Turn on debug printing (more verbose)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dppr-noprags</option></entry> + <entry>Don't output pragma info in dumps</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dppr-user-length</option></entry> + <entry>Set the depth for printing expressions in error msgs</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dsource-stats</option></entry> + <entry>Dump haskell source stats</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dstg-lint</option></entry> + <entry>STG pass sanity checking</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dstg-stats</option></entry> + <entry>Dump STG stats</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dusagesp-lint</option></entry> + <entry>UsageSP sanity checker</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dverbose-core2core</option></entry> + <entry>Show output from each core-to-core pass</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dverbose-stg2stg</option></entry> + <entry>Show output from each STG-to-STG pass</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-unreg</option></entry> + <entry>Enable unregisterised compilation</entry> + <entry>static</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + + <sect2> + <title>Misc compiler options</title> + + <informaltable> + <tgroup cols="4" align="left" colsep="1" rowsep="1"> + <thead> + <row> + <entry>Flag</entry> + <entry>Description</entry> + <entry>Static/Dynamic</entry> + <entry>Reverse</entry> + </row> + </thead> + <tbody> + <row> + <entry><option>-femit-extern-decls</option></entry> + <entry>???</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-fno-hi-version-check</option></entry> + <entry>Don't complain about <literal>.hi</literal> file mismatches</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-dno-black-holing</option></entry> + <entry>Turn off black holing (probably doesn't work)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-fno-method-sharing</option></entry> + <entry>Don't share specialisations of overloaded functions</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-fhistory-size</option></entry> + <entry>Set simplification history size</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-funregisterised</option></entry> + <entry>Unregisterised compilation (use <option>-unreg</option> instead)</entry> + <entry>static</entry> + <entry>-</entry> + </row> + <row> + <entry><option>-fno-asm-mangling</option></entry> + <entry>Turn off assembly mangling (use <option>-unreg</option> instead)</entry> + <entry>dynamic</entry> + <entry>-</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </sect2> + </sect1> + + +<!-- +Still to document: + +Misc: + , ( "H" , HasArg (setHeapSize . fromIntegral . decodeSize) ) + + -Bdir +--> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/ghci.xml b/docs/users_guide/ghci.xml new file mode 100644 index 0000000000..786815d484 --- /dev/null +++ b/docs/users_guide/ghci.xml @@ -0,0 +1,1500 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="ghci"> + <title>Using GHCi</title> + <indexterm><primary>GHCi</primary></indexterm> + <indexterm><primary>interpreter</primary><see>GHCi</see></indexterm> + <indexterm><primary>interactive</primary><see>GHCi</see></indexterm> + + <para>GHCi<footnote> + <para>The ‘i’ stands for “Interactive”</para> + </footnote> + is GHC's interactive environment, in which Haskell expressions can + be interactively evaluated and programs can be interpreted. If + you're familiar with <ulink url="http://www.haskell.org/hugs/">Hugs</ulink><indexterm><primary>Hugs</primary> + </indexterm>, then you'll be right at home with GHCi. However, GHCi + also has support for interactively loading compiled code, as well as + supporting all<footnote><para>except <literal>foreign export</literal>, at the moment</para> + </footnote> the language extensions that GHC provides.</para> + <indexterm><primary>FFI</primary><secondary>GHCi support</secondary></indexterm> + <indexterm><primary>Foreign Function Interface</primary><secondary>GHCi support</secondary></indexterm> + + <sect1> + <title>Introduction to GHCi</title> + + <para>Let's start with an example GHCi session. You can fire up + GHCi with the command <literal>ghci</literal>:</para> + +<screen> +$ ghci + ___ ___ _ + / _ \ /\ /\/ __(_) + / /_\// /_/ / / | | GHC Interactive, version 5.04, for Haskell 98. +/ /_\\/ __ / /___| | http://www.haskell.org/ghc/ +\____/\/ /_/\____/|_| Type :? for help. + +Loading package base ... linking ... done. +Loading package haskell98 ... linking ... done. +Prelude> +</screen> + + <para>There may be a short pause while GHCi loads the prelude and + standard libraries, after which the prompt is shown. If we follow + the instructions and type <literal>:?</literal> for help, we + get:</para> + +<screen> + Commands available from the prompt: + + <stmt> evaluate/run <stmt> + :add <filename> ... add module(s) to the current target set + :browse [*]<module> display the names defined by <module> + :cd <dir> change directory to <dir> + :def <cmd> <expr> define a command :<cmd> + :help, :? display this list of commands + :info [<name> ...] display information about the given names + :load <filename> ... load module(s) and their dependents + :module [+/-] [*]<mod> ... set the context for expression evaluation + :reload reload the current module set + + :set <option> ... set options + :set args <arg> ... set the arguments returned by System.getArgs + :set prog <progname> set the value returned by System.getProgName + :set prompt <prompt> set the prompt used in GHCi + + :show modules show the currently loaded modules + :show bindings show the current bindings made at the prompt + + :ctags [<file>] create tags file for Vi (default: "tags") + :etags [<file>] create tags file for Emacs (defauilt: "TAGS") + :type <expr> show the type of <expr> + :kind <type> show the kind of <type> + :undef <cmd> undefine user-defined command :<cmd> + :unset <option> ... unset options + :quit exit GHCi + :!<command> run the shell command <command> + + Options for `:set' and `:unset': + + +r revert top-level expressions after each evaluation + +s print timing/memory stats after each evaluation + +t print type after evaluation + -<flags> most GHC command line flags can also be set here + (eg. -v2, -fglasgow-exts, etc.) +</screen> + + <para>We'll explain most of these commands as we go along. For + Hugs users: many things work the same as in Hugs, so you should be + able to get going straight away.</para> + + <para>Haskell expressions can be typed at the prompt:</para> + <indexterm><primary>prompt</primary><secondary>GHCi</secondary> + </indexterm> + +<screen> +Prelude> 1+2 +3 +Prelude> let x = 42 in x / 9 +4.666666666666667 +Prelude> +</screen> + + <para>GHCi interprets the whole line as an expression to evaluate. + The expression may not span several lines - as soon as you press + enter, GHCi will attempt to evaluate it.</para> + </sect1> + + <sect1> + <title>Loading source files</title> + + <para>Suppose we have the following Haskell source code, which we + place in a file <filename>Main.hs</filename>:</para> + +<programlisting> +main = print (fac 20) + +fac 0 = 1 +fac n = n * fac (n-1) +</programlisting> + + <para>You can save <filename>Main.hs</filename> anywhere you like, + but if you save it somewhere other than the current + directory<footnote><para>If you started up GHCi from the command + line then GHCi's current directory is the same as the current + directory of the shell from which it was started. If you started + GHCi from the “Start” menu in Windows, then the + current directory is probably something like + <filename>C:\Documents and Settings\<replaceable>user + name</replaceable></filename>.</para> </footnote> then we will + need to change to the right directory in GHCi:</para> + +<screen> +Prelude> :cd <replaceable>dir</replaceable> +</screen> + + <para>where <replaceable>dir</replaceable> is the directory (or + folder) in which you saved <filename>Main.hs</filename>.</para> + + <para>To load a Haskell source file into GHCi, use the + <literal>:load</literal> command:</para> + <indexterm><primary><literal>:load</literal></primary></indexterm> + +<screen> +Prelude> :load Main +Compiling Main ( Main.hs, interpreted ) +Ok, modules loaded: Main. +*Main> +</screen> + + <para>GHCi has loaded the <literal>Main</literal> module, and the + prompt has changed to “<literal>*Main></literal>” to + indicate that the current context for expressions typed at the + prompt is the <literal>Main</literal> module we just loaded (we'll + explain what the <literal>*</literal> means later in <xref + linkend="ghci-scope"/>). So we can now type expressions involving + the functions from <filename>Main.hs</filename>:</para> + +<screen> +*Main> fac 17 +355687428096000 +</screen> + + <para>Loading a multi-module program is just as straightforward; + just give the name of the “topmost” module to the + <literal>:load</literal> command (hint: <literal>:load</literal> + can be abbreviated to <literal>:l</literal>). The topmost module + will normally be <literal>Main</literal>, but it doesn't have to + be. GHCi will discover which modules are required, directly or + indirectly, by the topmost module, and load them all in dependency + order.</para> + + <sect2 id="ghci-modules-filenames"> + <title>Modules vs. filenames</title> + <indexterm><primary>modules</primary><secondary>and filenames</secondary></indexterm> + <indexterm><primary>filenames</primary><secondary>of modules</secondary></indexterm> + + <para>Question: How does GHC find the filename which contains + module <replaceable>M</replaceable>? Answer: it looks for the + file <literal><replaceable>M</replaceable>.hs</literal>, or + <literal><replaceable>M</replaceable>.lhs</literal>. This means + that for most modules, the module name must match the filename. + If it doesn't, GHCi won't be able to find it.</para> + + <para>There is one exception to this general rule: when you load + a program with <literal>:load</literal>, or specify it when you + invoke <literal>ghci</literal>, you can give a filename rather + than a module name. This filename is loaded if it exists, and + it may contain any module you like. This is particularly + convenient if you have several <literal>Main</literal> modules + in the same directory and you can't call them all + <filename>Main.hs</filename>.</para> + + <para>The search path for finding source files is specified with + the <option>-i</option> option on the GHCi command line, like + so:</para> +<screen>ghci -i<replaceable>dir<subscript>1</subscript></replaceable>:...:<replaceable>dir<subscript>n</subscript></replaceable></screen> + + <para>or it can be set using the <literal>:set</literal> command + from within GHCi (see <xref + linkend="ghci-cmd-line-options"/>)<footnote><para>Note that in + GHCi, and <option>––make</option> mode, the <option>-i</option> + option is used to specify the search path for + <emphasis>source</emphasis> files, whereas in standard + batch-compilation mode the <option>-i</option> option is used to + specify the search path for interface files, see <xref + linkend="search-path"/>.</para> </footnote></para> + + <para>One consequence of the way that GHCi follows dependencies + to find modules to load is that every module must have a source + file. The only exception to the rule is modules that come from + a package, including the <literal>Prelude</literal> and standard + libraries such as <literal>IO</literal> and + <literal>Complex</literal>. If you attempt to load a module for + which GHCi can't find a source file, even if there are object + and interface files for the module, you'll get an error + message.</para> + </sect2> + + <sect2> + <title>Making changes and recompilation</title> + <indexterm><primary><literal>:reload</literal></primary></indexterm> + + <para>If you make some changes to the source code and want GHCi + to recompile the program, give the <literal>:reload</literal> + command. The program will be recompiled as necessary, with GHCi + doing its best to avoid actually recompiling modules if their + external dependencies haven't changed. This is the same + mechanism we use to avoid re-compiling modules in the batch + compilation setting (see <xref linkend="recomp"/>).</para> + </sect2> + </sect1> + + <sect1 id="ghci-compiled"> + <title>Loading compiled code</title> + <indexterm><primary>compiled code</primary><secondary>in GHCi</secondary></indexterm> + + <para>When you load a Haskell source module into GHCi, it is + normally converted to byte-code and run using the interpreter. + However, interpreted code can also run alongside compiled code in + GHCi; indeed, normally when GHCi starts, it loads up a compiled + copy of the <literal>base</literal> package, which contains the + <literal>Prelude</literal>.</para> + + <para>Why should we want to run compiled code? Well, compiled + code is roughly 10x faster than interpreted code, but takes about + 2x longer to produce (perhaps longer if optimisation is on). So + it pays to compile the parts of a program that aren't changing + very often, and use the interpreter for the code being actively + developed.</para> + + <para>When loading up source files with <literal>:load</literal>, + GHCi looks for any corresponding compiled object files, and will + use one in preference to interpreting the source if possible. For + example, suppose we have a 4-module program consisting of modules + A, B, C, and D. Modules B and C both import D only, + and A imports both B & C:</para> +<screen> + A + / \ + B C + \ / + D +</screen> + <para>We can compile D, then load the whole program, like this:</para> +<screen> +Prelude> :! ghc -c D.hs +Prelude> :load A +Skipping D ( D.hs, D.o ) +Compiling C ( C.hs, interpreted ) +Compiling B ( B.hs, interpreted ) +Compiling A ( A.hs, interpreted ) +Ok, modules loaded: A, B, C, D. +*Main> +</screen> + + <para>In the messages from the compiler, we see that it skipped D, + and used the object file <filename>D.o</filename>. The message + <literal>Skipping</literal> <replaceable>module</replaceable> + indicates that compilation for <replaceable>module</replaceable> + isn't necessary, because the source and everything it depends on + is unchanged since the last compilation.</para> + + <para>At any time you can use the command + <literal>:show modules</literal> + to get a list of the modules currently loaded + into GHCi:</para> + +<screen> +*Main> :show modules +D ( D.hs, D.o ) +C ( C.hs, interpreted ) +B ( B.hs, interpreted ) +A ( A.hs, interpreted ) +*Main></screen> + + <para>If we now modify the source of D (or pretend to: using Unix + command <literal>touch</literal> on the source file is handy for + this), the compiler will no longer be able to use the object file, + because it might be out of date:</para> + +<screen> +*Main> :! touch D.hs +*Main> :reload +Compiling D ( D.hs, interpreted ) +Skipping C ( C.hs, interpreted ) +Skipping B ( B.hs, interpreted ) +Skipping A ( A.hs, interpreted ) +Ok, modules loaded: A, B, C, D. +*Main> +</screen> + + <para>Note that module D was compiled, but in this instance + because its source hadn't really changed, its interface remained + the same, and the recompilation checker determined that A, B and C + didn't need to be recompiled.</para> + + <para>So let's try compiling one of the other modules:</para> + +<screen> +*Main> :! ghc -c C.hs +*Main> :load A +Compiling D ( D.hs, interpreted ) +Compiling C ( C.hs, interpreted ) +Compiling B ( B.hs, interpreted ) +Compiling A ( A.hs, interpreted ) +Ok, modules loaded: A, B, C, D. +</screen> + + <para>We didn't get the compiled version of C! What happened? + Well, in GHCi a compiled module may only depend on other compiled + modules, and in this case C depends on D, which doesn't have an + object file, so GHCi also rejected C's object file. Ok, so let's + also compile D:</para> + +<screen> +*Main> :! ghc -c D.hs +*Main> :reload +Ok, modules loaded: A, B, C, D. +</screen> + + <para>Nothing happened! Here's another lesson: newly compiled + modules aren't picked up by <literal>:reload</literal>, only + <literal>:load</literal>:</para> + +<screen> +*Main> :load A +Skipping D ( D.hs, D.o ) +Skipping C ( C.hs, C.o ) +Compiling B ( B.hs, interpreted ) +Compiling A ( A.hs, interpreted ) +Ok, modules loaded: A, B, C, D. +</screen> + + <para>HINT: since GHCi will only use a compiled object file if it + can sure that the compiled version is up-to-date, a good technique + when working on a large program is to occasionally run + <literal>ghc ––make</literal> to compile the whole project (say + before you go for lunch :-), then continue working in the + interpreter. As you modify code, the new modules will be + interpreted, but the rest of the project will remain + compiled.</para> + + </sect1> + + <sect1> + <title>Interactive evaluation at the prompt</title> + + <para>When you type an expression at the prompt, GHCi immediately + evaluates and prints the result: +<screen> +Prelude> reverse "hello" +"olleh" +Prelude> 5+5 +10 +</screen> +</para> + +<sect2><title>I/O actions at the prompt</title> + +<para>GHCi does more than simple expression evaluation at the prompt. +If you type something of type <literal>IO a</literal> for some + <literal>a</literal>, then GHCi <emphasis>executes</emphasis> it + as an IO-computation. +<screen> +Prelude> "hello" +"hello" +Prelude> putStrLn "hello" +hello +</screen> +Furthermore, GHCi will print the result of the I/O action if (and only +if): +<itemizedlist> + <listitem><para>The result type is an instance of <literal>Show</literal>.</para></listitem> + <listitem><para>The result type is not + <literal>()</literal>.</para></listitem> +</itemizedlist> +For example, remembering that <literal>putStrLn :: String -> IO ()</literal>: +<screen> +Prelude> putStrLn "hello" +hello +Prelude> do { putStrLn "hello"; return "yes" } +hello +"yes" +</screen> +</para></sect2> + + <sect2> + <title>Using <literal>do-</literal>notation at the prompt</title> + <indexterm><primary>do-notation</primary><secondary>in GHCi</secondary></indexterm> + <indexterm><primary>statements</primary><secondary>in GHCi</secondary></indexterm> + + <para>GHCi actually accepts <firstterm>statements</firstterm> + rather than just expressions at the prompt. This means you can + bind values and functions to names, and use them in future + expressions or statements.</para> + + <para>The syntax of a statement accepted at the GHCi prompt is + exactly the same as the syntax of a statement in a Haskell + <literal>do</literal> expression. However, there's no monad + overloading here: statements typed at the prompt must be in the + <literal>IO</literal> monad. +<screen> +Prelude> x <- return 42 +42 +Prelude> print x +42 +Prelude> +</screen> + The statement <literal>x <- return 42</literal> means + “execute <literal>return 42</literal> in the + <literal>IO</literal> monad, and bind the result to + <literal>x</literal>”. We can then use + <literal>x</literal> in future statements, for example to print + it as we did above.</para> + + <para>GHCi will print the result of a statement if and only if: + <itemizedlist> + <listitem> + <para>The statement is not a binding, or it is a monadic binding + (<literal>p <- e</literal>) that binds exactly one + variable.</para> + </listitem> + <listitem> + <para>The variable's type is not polymorphic, is not + <literal>()</literal>, and is an instance of + <literal>Show</literal></para> + </listitem> + </itemizedlist> + </para> + + <para>Of course, you can also bind normal non-IO expressions + using the <literal>let</literal>-statement:</para> +<screen> +Prelude> let x = 42 +Prelude> x +42 +Prelude> +</screen> + <para>Another important difference between the two types of binding + is that the monadic bind (<literal>p <- e</literal>) is + <emphasis>strict</emphasis> (it evaluates <literal>e</literal>), + whereas with the <literal>let</literal> form, the expression + isn't evaluated immediately:</para> +<screen> +Prelude> let x = error "help!" +Prelude> print x +*** Exception: help! +Prelude> +</screen> + + <para>Note that <literal>let</literal> bindings do not automatically + print the value bound, unlike monadic bindings.</para> + + <para>Any exceptions raised during the evaluation or execution + of the statement are caught and printed by the GHCi command line + interface (for more information on exceptions, see the module + <literal>Control.Exception</literal> in the libraries + documentation).</para> + + <para>Every new binding shadows any existing bindings of the + same name, including entities that are in scope in the current + module context.</para> + + <para>WARNING: temporary bindings introduced at the prompt only + last until the next <literal>:load</literal> or + <literal>:reload</literal> command, at which time they will be + simply lost. However, they do survive a change of context with + <literal>:module</literal>: the temporary bindings just move to + the new location.</para> + + <para>HINT: To get a list of the bindings currently in scope, use the + <literal>:show bindings</literal> command:</para> + +<screen> +Prelude> :show bindings +x :: Int +Prelude></screen> + + <para>HINT: if you turn on the <literal>+t</literal> option, + GHCi will show the type of each variable bound by a statement. + For example:</para> + <indexterm><primary><literal>+t</literal></primary></indexterm> +<screen> +Prelude> :set +t +Prelude> let (x:xs) = [1..] +x :: Integer +xs :: [Integer] +</screen> + + </sect2> + + <sect2 id="ghci-scope"> + <title>What's really in scope at the prompt?</title> + + <para>When you type an expression at the prompt, what + identifiers and types are in scope? GHCi provides a flexible + way to control exactly how the context for an expression is + constructed. Let's start with the simple cases; when you start + GHCi the prompt looks like this:</para> + +<screen>Prelude></screen> + + <para>Which indicates that everything from the module + <literal>Prelude</literal> is currently in scope. If we now + load a file into GHCi, the prompt will change:</para> + +<screen> +Prelude> :load Main.hs +Compiling Main ( Main.hs, interpreted ) +*Main> +</screen> + + <para>The new prompt is <literal>*Main</literal>, which + indicates that we are typing expressions in the context of the + top-level of the <literal>Main</literal> module. Everything + that is in scope at the top-level in the module + <literal>Main</literal> we just loaded is also in scope at the + prompt (probably including <literal>Prelude</literal>, as long + as <literal>Main</literal> doesn't explicitly hide it).</para> + + <para>The syntax + <literal>*<replaceable>module</replaceable></literal> indicates + that it is the full top-level scope of + <replaceable>module</replaceable> that is contributing to the + scope for expressions typed at the prompt. Without the + <literal>*</literal>, just the exports of the module are + visible.</para> + + <para>We're not limited to a single module: GHCi can combine + scopes from multiple modules, in any mixture of + <literal>*</literal> and non-<literal>*</literal> forms. GHCi + combines the scopes from all of these modules to form the scope + that is in effect at the prompt. For technical reasons, GHCi + can only support the <literal>*</literal>-form for modules which + are interpreted, so compiled modules and package modules can + only contribute their exports to the current scope.</para> + + <para>The scope is manipulated using the + <literal>:module</literal> command. For example, if the current + scope is <literal>Prelude</literal>, then we can bring into + scope the exports from the module <literal>IO</literal> like + so:</para> + +<screen> +Prelude> :module +IO +Prelude,IO> hPutStrLn stdout "hello\n" +hello +Prelude,IO> +</screen> + + <para>(Note: <literal>:module</literal> can be shortened to + <literal>:m</literal>). The full syntax of the + <literal>:module</literal> command is:</para> + +<screen> +:module <optional>+|-</optional> <optional>*</optional><replaceable>mod<subscript>1</subscript></replaceable> ... <optional>*</optional><replaceable>mod<subscript>n</subscript></replaceable> +</screen> + + <para>Using the <literal>+</literal> form of the + <literal>module</literal> commands adds modules to the current + scope, and <literal>-</literal> removes them. Without either + <literal>+</literal> or <literal>-</literal>, the current scope + is replaced by the set of modules specified. Note that if you + use this form and leave out <literal>Prelude</literal>, GHCi + will assume that you really wanted the + <literal>Prelude</literal> and add it in for you (if you don't + want the <literal>Prelude</literal>, then ask to remove it with + <literal>:m -Prelude</literal>).</para> + + <para>The scope is automatically set after a + <literal>:load</literal> command, to the most recently loaded + "target" module, in a <literal>*</literal>-form if possible. + For example, if you say <literal>:load foo.hs bar.hs</literal> + and <filename>bar.hs</filename> contains module + <literal>Bar</literal>, then the scope will be set to + <literal>*Bar</literal> if <literal>Bar</literal> is + interpreted, or if <literal>Bar</literal> is compiled it will be + set to <literal>Prelude,Bar</literal> (GHCi automatically adds + <literal>Prelude</literal> if it isn't present and there aren't + any <literal>*</literal>-form modules).</para> + + <para>With multiple modules in scope, especially multiple + <literal>*</literal>-form modules, it is likely that name + clashes will occur. Haskell specifies that name clashes are + only reported when an ambiguous identifier is used, and GHCi + behaves in the same way for expressions typed at the + prompt.</para> + + <sect3> + <title>Qualified names</title> + + <para>To make life slightly easier, the GHCi prompt also + behaves as if there is an implicit <literal>import + qualified</literal> declaration for every module in every + package, and every module currently loaded into GHCi.</para> + </sect3> + </sect2> + + + <sect2> + <title>The <literal>it</literal> variable</title> + <indexterm><primary><literal>it</literal></primary> + </indexterm> + + <para>Whenever an expression (or a non-binding statement, to be + precise) is typed at the prompt, GHCi implicitly binds its value + to the variable <literal>it</literal>. For example:</para> +<screen> +Prelude> 1+2 +3 +Prelude> it * 2 +6 +</screen> + <para>What actually happens is that GHCi typechecks the + expression, and if it doesn't have an <literal>IO</literal> type, + then it transforms it as follows: an expression + <replaceable>e</replaceable> turns into +<screen> + let it = <replaceable>e</replaceable>; + print it +</screen> + which is then run as an IO-action.</para> + + <para>Hence, the original expression must have a type which is an + instance of the <literal>Show</literal> class, or GHCi will + complain:</para> + +<screen> +Prelude> id +No instance for `Show (a -> a)' +arising from use of `print' +in a `do' expression pattern binding: print it +</screen> + + <para>The error message contains some clues as to the + transformation happening internally.</para> + + <para>If the expression was instead of type <literal>IO a</literal> for + some <literal>a</literal>, then <literal>it</literal> will be + bound to the result of the <literal>IO</literal> computation, + which is of type <literal>a</literal>. eg.:</para> +<screen> +Prelude> Time.getClockTime +Prelude> print it +Wed Mar 14 12:23:13 GMT 2001 +</screen> + + <para>The corresponding translation for an IO-typed + <replaceable>e</replaceable> is +<screen> + it <- <replaceable>e</replaceable> +</screen> + </para> + + <para>Note that <literal>it</literal> is shadowed by the new + value each time you evaluate a new expression, and the old value + of <literal>it</literal> is lost.</para> + + </sect2> + + <sect2> + <title>Type defaulting in GHCi</title> + <indexterm><primary>Type default</primary></indexterm> + <indexterm><primary><literal>Show</literal> class</primary></indexterm> + <para> + Consider this GHCi session: +<programlisting> + ghci> reverse [] +</programlisting> + What should GHCi do? Strictly speaking, the program is ambiguous. <literal>show (reverse [])</literal> + (which is what GHCi computes here) has type <literal>Show a => a</literal> and how that displays depends + on the type <literal>a</literal>. For example: +<programlisting> + ghci> (reverse []) :: String + "" + ghci> (reverse []) :: [Int] + [] +</programlisting> + However, it is tiresome for the user to have to specify the type, so GHCi extends Haskell's type-defaulting + rules (Section 4.3.4 of the Haskell 98 Report (Revised)) as follows. The + standard rules take each group of constraints <literal>(C1 a, C2 a, ..., Cn + a)</literal> for each type variable <literal>a</literal>, and defaults the + type variable if + <itemizedlist> + <listitem><para> The type variable <literal>a</literal> + appears in no other constraints </para></listitem> + <listitem><para> All the classes <literal>Ci</literal> are standard.</para></listitem> + <listitem><para> At least one of the classes <literal>Ci</literal> is + numeric.</para></listitem> + </itemizedlist> + At the GHCi prompt, the second and third rules are relaxed as follows + (differences italicised): + <itemizedlist> + <listitem><para> <emphasis>All</emphasis> of the classes + <literal>Ci</literal> are single-parameter type classes.</para></listitem> + <listitem><para> At least one of the classes <literal>Ci</literal> is + numeric, <emphasis>or is <literal>Show</literal>, + <literal>Eq</literal>, or <literal>Ord</literal></emphasis>.</para></listitem> + </itemizedlist> + </para> + </sect2> + </sect1> + + <sect1 id="ghci-invocation"> + <title>Invoking GHCi</title> + <indexterm><primary>invoking</primary><secondary>GHCi</secondary></indexterm> + <indexterm><primary><option>––interactive</option></primary></indexterm> + + <para>GHCi is invoked with the command <literal>ghci</literal> or + <literal>ghc ––interactive</literal>. One or more modules or + filenames can also be specified on the command line; this + instructs GHCi to load the specified modules or filenames (and all + the modules they depend on), just as if you had said + <literal>:load <replaceable>modules</replaceable></literal> at the + GHCi prompt (see <xref linkend="ghci-commands"/>). For example, to + start GHCi and load the program whose topmost module is in the + file <literal>Main.hs</literal>, we could say:</para> + +<screen> +$ ghci Main.hs +</screen> + + <para>Most of the command-line options accepted by GHC (see <xref + linkend="using-ghc"/>) also make sense in interactive mode. The ones + that don't make sense are mostly obvious; for example, GHCi + doesn't generate interface files, so options related to interface + file generation won't have any effect.</para> + + <sect2> + <title>Packages</title> + <indexterm><primary>packages</primary><secondary>with GHCi</secondary></indexterm> + + <para>Most packages (see <xref linkend="using-packages"/>) are + available without needing to specify any extra flags at all: + they will be automatically loaded the first time they are + needed.</para> + + <para>For non-auto packages, however, you need to request the + package be loaded by using the <literal>-package</literal> flag:</para> + +<screen> +$ ghci -package data + ___ ___ _ + / _ \ /\ /\/ __(_) + / /_\// /_/ / / | | GHC Interactive, version 5.05, for Haskell 98. +/ /_\\/ __ / /___| | http://www.haskell.org/ghc/ +\____/\/ /_/\____/|_| Type :? for help. + +Loading package base ... linking ... done. +Loading package haskell98 ... linking ... done. +Loading package lang ... linking ... done. +Loading package concurrent ... linking ... done. +Loading package readline ... linking ... done. +Loading package unix ... linking ... done. +Loading package posix ... linking ... done. +Loading package util ... linking ... done. +Loading package data ... linking ... done. +Prelude> +</screen> + + <para>The following command works to load new packages into a + running GHCi:</para> + +<screen> +Prelude> :set -package <replaceable>name</replaceable> +</screen> + + <para>But note that doing this will cause all currently loaded + modules to be unloaded, and you'll be dumped back into the + <literal>Prelude</literal>.</para> + </sect2> + + <sect2> + <title>Extra libraries</title> + <indexterm><primary>libraries</primary><secondary>with GHCi</secondary></indexterm> + + <para>Extra libraries may be specified on the command line using + the normal <literal>-l<replaceable>lib</replaceable></literal> + option. (The term <emphasis>library</emphasis> here refers to + libraries of foreign object code; for using libraries of Haskell + source code, see <xref linkend="ghci-modules-filenames"/>.) For + example, to load the “m” library:</para> + +<screen> +$ ghci -lm +</screen> + + <para>On systems with <literal>.so</literal>-style shared + libraries, the actual library loaded will the + <filename>lib<replaceable>lib</replaceable>.so</filename>. GHCi + searches the following places for libraries, in this order:</para> + + <itemizedlist> + <listitem> + <para>Paths specified using the + <literal>-L<replaceable>path</replaceable></literal> + command-line option,</para> + </listitem> + <listitem> + <para>the standard library search path for your system, + which on some systems may be overridden by setting the + <literal>LD_LIBRARY_PATH</literal> environment + variable.</para> + </listitem> + </itemizedlist> + + <para>On systems with <literal>.dll</literal>-style shared + libraries, the actual library loaded will be + <filename><replaceable>lib</replaceable>.dll</filename>. Again, + GHCi will signal an error if it can't find the library.</para> + + <para>GHCi can also load plain object files + (<literal>.o</literal> or <literal>.obj</literal> depending on + your platform) from the command-line. Just add the name the + object file to the command line.</para> + + <para>Ordering of <option>-l</option> options matters: a library + should be mentioned <emphasis>before</emphasis> the libraries it + depends on (see <xref linkend="options-linker"/>).</para> + </sect2> + + </sect1> + + <sect1 id="ghci-commands"> + <title>GHCi commands</title> + + <para>GHCi commands all begin with + ‘<literal>:</literal>’ and consist of a single command + name followed by zero or more parameters. The command name may be + abbreviated, as long as the abbreviation is not ambiguous. All of + the builtin commands, with the exception of + <literal>:unset</literal> and <literal>:undef</literal>, may be + abbreviated to a single letter.</para> + + <variablelist> + <varlistentry> + <term> + <literal>:add</literal> <replaceable>module</replaceable> ... + <indexterm><primary><literal>:add</literal></primary></indexterm> + </term> + <listitem> + <para>Add <replaceable>module</replaceable>(s) to the + current <firstterm>target set</firstterm>, and perform a + reload.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:browse</literal> <optional><literal>*</literal></optional><replaceable>module</replaceable> ... + <indexterm><primary><literal>:browse</literal></primary></indexterm> + </term> + <listitem> + <para>Displays the identifiers defined by the module + <replaceable>module</replaceable>, which must be either + loaded into GHCi or be a member of a package. If the + <literal>*</literal> symbol is placed before the module + name, then <emphasis>all</emphasis> the identifiers defined + in <replaceable>module</replaceable> are shown; otherwise + the list is limited to the exports of + <replaceable>module</replaceable>. The + <literal>*</literal>-form is only available for modules + which are interpreted; for compiled modules (including + modules from packages) only the non-<literal>*</literal> + form of <literal>:browse</literal> is available.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:cd</literal> <replaceable>dir</replaceable> + <indexterm><primary><literal>:cd</literal></primary></indexterm> + </term> + <listitem> + <para>Changes the current working directory to + <replaceable>dir</replaceable>. A + ‘<literal>˜</literal>’ symbol at the + beginning of <replaceable>dir</replaceable> will be replaced + by the contents of the environment variable + <literal>HOME</literal>.</para> + + <para>NOTE: changing directories causes all currently loaded + modules to be unloaded. This is because the search path is + usually expressed using relative directories, and changing + the search path in the middle of a session is not + supported.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:def</literal> <replaceable>name</replaceable> <replaceable>expr</replaceable> + <indexterm><primary><literal>:def</literal></primary></indexterm> + </term> + <listitem> + <para>The command <literal>:def</literal> + <replaceable>name</replaceable> + <replaceable>expr</replaceable> defines a new GHCi command + <literal>:<replaceable>name</replaceable></literal>, + implemented by the Haskell expression + <replaceable>expr</replaceable>, which must have type + <literal>String -> IO String</literal>. When + <literal>:<replaceable>name</replaceable> + <replaceable>args</replaceable></literal> is typed at the + prompt, GHCi will run the expression + <literal>(<replaceable>name</replaceable> + <replaceable>args</replaceable>)</literal>, take the + resulting <literal>String</literal>, and feed it back into + GHCi as a new sequence of commands. Separate commands in + the result must be separated by + ‘<literal>\n</literal>’.</para> + + <para>That's all a little confusing, so here's a few + examples. To start with, here's a new GHCi command which + doesn't take any arguments or produce any results, it just + outputs the current date & time:</para> + +<screen> +Prelude> let date _ = Time.getClockTime >>= print >> return "" +Prelude> :def date date +Prelude> :date +Fri Mar 23 15:16:40 GMT 2001 +</screen> + + <para>Here's an example of a command that takes an argument. + It's a re-implementation of <literal>:cd</literal>:</para> + +<screen> +Prelude> let mycd d = Directory.setCurrentDirectory d >> return "" +Prelude> :def mycd mycd +Prelude> :mycd .. +</screen> + + <para>Or I could define a simple way to invoke + “<literal>ghc ––make Main</literal>” in the + current directory:</para> + +<screen> +Prelude> :def make (\_ -> return ":! ghc ––make Main") +</screen> + + <para>We can define a command that reads GHCi input from a + file. This might be useful for creating a set of bindings + that we want to repeatedly load into the GHCi session:</para> + +<screen> +Prelude> :def . readFile +Prelude> :. cmds.ghci +</screen> + + <para>Notice that we named the command + <literal>:.</literal>, by analogy with the + ‘<literal>.</literal>’ Unix shell command that + does the same thing.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:help</literal> + <indexterm><primary><literal>:help</literal></primary></indexterm> + </term> + <term> + <literal>:?</literal> + <indexterm><primary><literal>:?</literal></primary></indexterm> + </term> + <listitem> + <para>Displays a list of the available commands.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:info</literal> <replaceable>name</replaceable> ... + <indexterm><primary><literal>:info</literal></primary></indexterm> + </term> + <listitem> + <para>Displays information about the given name(s). For + example, if <replaceable>name</replaceable> is a class, then + the class methods and their types will be printed; if + <replaceable>name</replaceable> is a type constructor, then + its definition will be printed; if + <replaceable>name</replaceable> is a function, then its type + will be printed. If <replaceable>name</replaceable> has + been loaded from a source file, then GHCi will also display + the location of its definition in the source.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:load</literal> <replaceable>module</replaceable> ... + <indexterm><primary><literal>:load</literal></primary></indexterm> + </term> + <listitem> + <para>Recursively loads the specified + <replaceable>module</replaceable>s, and all the modules they + depend on. Here, each <replaceable>module</replaceable> + must be a module name or filename, but may not be the name + of a module in a package.</para> + + <para>All previously loaded modules, except package modules, + are forgotten. The new set of modules is known as the + <firstterm>target set</firstterm>. Note that + <literal>:load</literal> can be used without any arguments + to unload all the currently loaded modules and + bindings.</para> + + <para>After a <literal>:load</literal> command, the current + context is set to:</para> + + <itemizedlist> + <listitem> + <para><replaceable>module</replaceable>, if it was loaded + successfully, or</para> + </listitem> + <listitem> + <para>the most recently successfully loaded module, if + any other modules were loaded as a result of the current + <literal>:load</literal>, or</para> + </listitem> + <listitem> + <para><literal>Prelude</literal> otherwise.</para> + </listitem> + </itemizedlist> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:module <optional>+|-</optional> <optional>*</optional><replaceable>mod<subscript>1</subscript></replaceable> ... <optional>*</optional><replaceable>mod<subscript>n</subscript></replaceable></literal> + <indexterm><primary><literal>:module</literal></primary></indexterm> + </term> + <listitem> + <para>Sets or modifies the current context for statements + typed at the prompt. See <xref linkend="ghci-scope"/> for + more details.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:quit</literal> + <indexterm><primary><literal>:quit</literal></primary></indexterm> + </term> + <listitem> + <para>Quits GHCi. You can also quit by typing a control-D + at the prompt.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:reload</literal> + <indexterm><primary><literal>:reload</literal></primary></indexterm> + </term> + <listitem> + <para>Attempts to reload the current target set (see + <literal>:load</literal>) if any of the modules in the set, + or any dependent module, has changed. Note that this may + entail loading new modules, or dropping modules which are no + longer indirectly required by the target.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:set</literal> <optional><replaceable>option</replaceable>...</optional> + <indexterm><primary><literal>:set</literal></primary></indexterm> + </term> + <listitem> + <para>Sets various options. See <xref linkend="ghci-set"/> + for a list of available options. The + <literal>:set</literal> command by itself shows which + options are currently set.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:set</literal> <literal>args</literal> <replaceable>arg</replaceable> ... + <indexterm><primary><literal>:set args</literal></primary></indexterm> + </term> + <listitem> + <para>Sets the list of arguments which are returned when the + program calls <literal>System.getArgs</literal><indexterm><primary>getArgs</primary> + </indexterm>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:set</literal> <literal>prog</literal> <replaceable>prog</replaceable> + <indexterm><primary><literal>:set prog</literal></primary></indexterm> + </term> + <listitem> + <para>Sets the string to be returned when the program calls + <literal>System.getProgName</literal><indexterm><primary>getProgName</primary> + </indexterm>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:set</literal> <literal>prompt</literal> <replaceable>prompt</replaceable> + </term> + <listitem> + <para>Sets the string to be used as the prompt in GHCi. + Inside <replaceable>prompt</replaceable>, the sequence + <literal>%s</literal> is replaced by the names of the + modules currently in scope, and <literal>%%</literal> is + replaced by <literal>%</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:show bindings</literal> + <indexterm><primary><literal>:show bindings</literal></primary></indexterm> + </term> + <listitem> + <para>Show the bindings made at the prompt and their + types.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:show modules</literal> + <indexterm><primary><literal>:show modules</literal></primary></indexterm> + </term> + <listitem> + <para>Show the list of modules currently load.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:ctags</literal> <optional><replaceable>filename</replaceable></optional> + <literal>:etags</literal> <optional><replaceable>filename</replaceable></optional> + <indexterm><primary><literal>:etags</literal></primary> + </indexterm> + <indexterm><primary><literal>:etags</literal></primary> + </indexterm> + </term> + <listitem> + <para>Generates a “tags” file for Vi-style editors + (<literal>:ctags</literal>) or Emacs-style editors (<literal>etags</literal>). If + no filename is specified, the defaulit <filename>tags</filename> or + <filename>TAGS</filename> is + used, respectively. Tags for all the functions, constructors and + types in the currently loaded modules are created. All modules must + be interpreted for these commands to work.</para> + <para>See also <xref linkend="hasktags" />.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:type</literal> <replaceable>expression</replaceable> + <indexterm><primary><literal>:type</literal></primary></indexterm> + </term> + <listitem> + <para>Infers and prints the type of + <replaceable>expression</replaceable>, including explicit + forall quantifiers for polymorphic types. The monomorphism + restriction is <emphasis>not</emphasis> applied to the + expression during type inference.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:kind</literal> <replaceable>type</replaceable> + <indexterm><primary><literal>:kind</literal></primary></indexterm> + </term> + <listitem> + <para>Infers and prints the kind of + <replaceable>type</replaceable>. The latter can be an arbitrary + type expression, including a partial application of a type constructor, + such as <literal>Either Int</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:undef</literal> <replaceable>name</replaceable> + <indexterm><primary><literal>:undef</literal></primary></indexterm> + </term> + <listitem> + <para>Undefines the user-defined command + <replaceable>name</replaceable> (see <literal>:def</literal> + above).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:unset</literal> <replaceable>option</replaceable>... + <indexterm><primary><literal>:unset</literal></primary></indexterm> + </term> + <listitem> + <para>Unsets certain options. See <xref linkend="ghci-set"/> + for a list of available options.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>:!</literal> <replaceable>command</replaceable>... + <indexterm><primary><literal>:!</literal></primary></indexterm> + <indexterm><primary>shell commands</primary><secondary>in GHCi</secondary></indexterm> + </term> + <listitem> + <para>Executes the shell command + <replaceable>command</replaceable>.</para> + </listitem> + </varlistentry> + + </variablelist> + </sect1> + + <sect1 id="ghci-set"> + <title>The <literal>:set</literal> command</title> + <indexterm><primary><literal>:set</literal></primary></indexterm> + + <para>The <literal>:set</literal> command sets two types of + options: GHCi options, which begin with + ‘<literal>+</literal>” and “command-line” + options, which begin with ‘-’. </para> + + <para>NOTE: at the moment, the <literal>:set</literal> command + doesn't support any kind of quoting in its arguments: quotes will + not be removed and cannot be used to group words together. For + example, <literal>:set -DFOO='BAR BAZ'</literal> will not do what + you expect.</para> + + <sect2> + <title>GHCi options</title> + <indexterm><primary>options</primary><secondary>GHCi</secondary> + </indexterm> + + <para>GHCi options may be set using <literal>:set</literal> and + unset using <literal>:unset</literal>.</para> + + <para>The available GHCi options are:</para> + + <variablelist> + <varlistentry> + <term> + <literal>+r</literal> + <indexterm><primary><literal>+r</literal></primary></indexterm> + <indexterm><primary>CAFs</primary><secondary>in GHCi</secondary></indexterm> + <indexterm><primary>Constant Applicative Form</primary><see>CAFs</see></indexterm> + </term> + <listitem> + <para>Normally, any evaluation of top-level expressions + (otherwise known as CAFs or Constant Applicative Forms) in + loaded modules is retained between evaluations. Turning + on <literal>+r</literal> causes all evaluation of + top-level expressions to be discarded after each + evaluation (they are still retained + <emphasis>during</emphasis> a single evaluation).</para> + + <para>This option may help if the evaluated top-level + expressions are consuming large amounts of space, or if + you need repeatable performance measurements.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>+s</literal> + <indexterm><primary><literal>+s</literal></primary></indexterm> + </term> + <listitem> + <para>Display some stats after evaluating each expression, + including the elapsed time and number of bytes allocated. + NOTE: the allocation figure is only accurate to the size + of the storage manager's allocation area, because it is + calculated at every GC. Hence, you might see values of + zero if no GC has occurred.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>+t</literal> + <indexterm><primary><literal>+t</literal></primary></indexterm> + </term> + <listitem> + <para>Display the type of each variable bound after a + statement is entered at the prompt. If the statement is a + single expression, then the only variable binding will be + for the variable + ‘<literal>it</literal>’.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="ghci-cmd-line-options"> + <title>Setting GHC command-line options in GHCi</title> + + <para>Normal GHC command-line options may also be set using + <literal>:set</literal>. For example, to turn on + <option>-fglasgow-exts</option>, you would say:</para> + +<screen> +Prelude> :set -fglasgow-exts +</screen> + + <para>Any GHC command-line option that is designated as + <firstterm>dynamic</firstterm> (see the table in <xref + linkend="flag-reference"/>), may be set using + <literal>:set</literal>. To unset an option, you can set the + reverse option:</para> + <indexterm><primary>dynamic</primary><secondary>options</secondary></indexterm> + +<screen> +Prelude> :set -fno-glasgow-exts +</screen> + + <para><xref linkend="flag-reference"/> lists the reverse for each + option where applicable.</para> + + <para>Certain static options (<option>-package</option>, + <option>-I</option>, <option>-i</option>, and + <option>-l</option> in particular) will also work, but some may + not take effect until the next reload.</para> + <indexterm><primary>static</primary><secondary>options</secondary></indexterm> + </sect2> + </sect1> + + <sect1 id="ghci-dot-files"> + <title>The <filename>.ghci</filename> file</title> + <indexterm><primary><filename>.ghci</filename></primary><secondary>file</secondary> + </indexterm> + <indexterm><primary>startup</primary><secondary>files, GHCi</secondary> + </indexterm> + + <para>When it starts, GHCi always reads and executes commands from + <filename>$HOME/.ghci</filename>, followed by + <filename>./.ghci</filename>.</para> + + <para>The <filename>.ghci</filename> in your home directory is + most useful for turning on favourite options (eg. <literal>:set + +s</literal>), and defining useful macros. Placing a + <filename>.ghci</filename> file in a directory with a Haskell + project is a useful way to set certain project-wide options so you + don't have to type them everytime you start GHCi: eg. if your + project uses GHC extensions and CPP, and has source files in three + subdirectories A B and C, you might put the following lines in + <filename>.ghci</filename>:</para> + +<screen> +:set -fglasgow-exts -cpp +:set -iA:B:C +</screen> + + <para>(Note that strictly speaking the <option>-i</option> flag is + a static one, but in fact it works to set it using + <literal>:set</literal> like this. The changes won't take effect + until the next <literal>:load</literal>, though.)</para> + + <para>Two command-line options control whether the + <filename>.ghci</filename> files are read:</para> + + <variablelist> + <varlistentry> + <term> + <option>-ignore-dot-ghci</option> + <indexterm><primary><option>-ignore-dot-ghci</option></primary></indexterm> + </term> + <listitem> + <para>Don't read either <filename>./.ghci</filename> or + <filename>$HOME/.ghci</filename> when starting up.</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-read-dot-ghci</option> + <indexterm><primary><option>-read-dot-ghci</option></primary></indexterm> + </term> + <listitem> + <para>Read <filename>.ghci</filename> and + <filename>$HOME/.ghci</filename>. This is normally the + default, but the <option>-read-dot-ghci</option> option may + be used to override a previous + <option>-ignore-dot-ghci</option> option.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect1> + + <sect1> + <title>FAQ and Things To Watch Out For</title> + + <variablelist> + <varlistentry> + <term>The interpreter can't load modules with foreign export + declarations!</term> + <listitem> + <para>Unfortunately not. We haven't implemented it yet. + Please compile any offending modules by hand before loading + them into GHCi.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>-O</literal> doesn't work with GHCi! + <indexterm><primary><option>-O</option></primary></indexterm> + </term> + <listitem> + <para>For technical reasons, the bytecode compiler doesn't + interact well with one of the optimisation passes, so we + have disabled optimisation when using the interpreter. This + isn't a great loss: you'll get a much bigger win by + compiling the bits of your code that need to go fast, rather + than interpreting them with optimisation turned on.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Unboxed tuples don't work with GHCi</term> + <listitem> + <para>That's right. You can always compile a module that + uses unboxed tuples and load it into GHCi, however. + (Incidentally the previous point, namely that + <literal>-O</literal> is incompatible with GHCi, is because + the bytecode compiler can't deal with unboxed + tuples).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Concurrent threads don't carry on running when GHCi is + waiting for input.</term> + <listitem> + <para>No, they don't. This is because the Haskell binding + to the GNU readline library doesn't support reading from the + terminal in a non-blocking way, which is required to work + properly with GHC's concurrency model.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>After using <literal>getContents</literal>, I can't use + <literal>stdin</literal> again until I do + <literal>:load</literal> or <literal>:reload</literal>.</term> + + <listitem> + <para>This is the defined behaviour of + <literal>getContents</literal>: it puts the stdin Handle in + a state known as <firstterm>semi-closed</firstterm>, wherein + any further I/O operations on it are forbidden. Because I/O + state is retained between computations, the semi-closed + state persists until the next <literal>:load</literal> or + <literal>:reload</literal> command.</para> + + <para>You can make <literal>stdin</literal> reset itself + after every evaluation by giving GHCi the command + <literal>:set +r</literal>. This works because + <literal>stdin</literal> is just a top-level expression that + can be reverted to its unevaluated state in the same way as + any other top-level expression (CAF).</para> + </listitem> + </varlistentry> + + </variablelist> + </sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/glasgow_exts.xml b/docs/users_guide/glasgow_exts.xml new file mode 100644 index 0000000000..beaaad616a --- /dev/null +++ b/docs/users_guide/glasgow_exts.xml @@ -0,0 +1,6264 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<para> +<indexterm><primary>language, GHC</primary></indexterm> +<indexterm><primary>extensions, GHC</primary></indexterm> +As with all known Haskell systems, GHC implements some extensions to +the language. They are all enabled by options; by default GHC +understands only plain Haskell 98. +</para> + +<para> +Some of the Glasgow extensions serve to give you access to the +underlying facilities with which we implement Haskell. Thus, you can +get at the Raw Iron, if you are willing to write some non-portable +code at a more primitive level. You need not be “stuck” +on performance because of the implementation costs of Haskell's +“high-level” features—you can always code +“under” them. In an extreme case, you can write all your +time-critical code in C, and then just glue it together with Haskell! +</para> + +<para> +Before you get too carried away working at the lowest level (e.g., +sloshing <literal>MutableByteArray#</literal>s around your +program), you may wish to check if there are libraries that provide a +“Haskellised veneer” over the features you want. The +separate <ulink url="../libraries/index.html">libraries +documentation</ulink> describes all the libraries that come with GHC. +</para> + +<!-- LANGUAGE OPTIONS --> + <sect1 id="options-language"> + <title>Language options</title> + + <indexterm><primary>language</primary><secondary>option</secondary> + </indexterm> + <indexterm><primary>options</primary><secondary>language</secondary> + </indexterm> + <indexterm><primary>extensions</primary><secondary>options controlling</secondary> + </indexterm> + + <para>These flags control what variation of the language are + permitted. Leaving out all of them gives you standard Haskell + 98.</para> + + <para>NB. turning on an option that enables special syntax + <emphasis>might</emphasis> cause working Haskell 98 code to fail + to compile, perhaps because it uses a variable name which has + become a reserved word. So, together with each option below, we + list the special syntax which is enabled by this option. We use + notation and nonterminal names from the Haskell 98 lexical syntax + (see the Haskell 98 Report). There are two classes of special + syntax:</para> + + <itemizedlist> + <listitem> + <para>New reserved words and symbols: character sequences + which are no longer available for use as identifiers in the + program.</para> + </listitem> + <listitem> + <para>Other special syntax: sequences of characters that have + a different meaning when this particular option is turned + on.</para> + </listitem> + </itemizedlist> + + <para>We are only listing syntax changes here that might affect + existing working programs (i.e. "stolen" syntax). Many of these + extensions will also enable new context-free syntax, but in all + cases programs written to use the new syntax would not be + compilable without the option enabled.</para> + + <variablelist> + + <varlistentry> + <term> + <option>-fglasgow-exts</option>: + <indexterm><primary><option>-fglasgow-exts</option></primary></indexterm> + </term> + <listitem> + <para>This simultaneously enables all of the extensions to + Haskell 98 described in <xref + linkend="ghc-language-features"/>, except where otherwise + noted. </para> + + <para>New reserved words: <literal>forall</literal> (only in + types), <literal>mdo</literal>.</para> + + <para>Other syntax stolen: + <replaceable>varid</replaceable>{<literal>#</literal>}, + <replaceable>char</replaceable><literal>#</literal>, + <replaceable>string</replaceable><literal>#</literal>, + <replaceable>integer</replaceable><literal>#</literal>, + <replaceable>float</replaceable><literal>#</literal>, + <replaceable>float</replaceable><literal>##</literal>, + <literal>(#</literal>, <literal>#)</literal>, + <literal>|)</literal>, <literal>{|</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ffi</option> and <option>-fffi</option>: + <indexterm><primary><option>-ffi</option></primary></indexterm> + <indexterm><primary><option>-fffi</option></primary></indexterm> + </term> + <listitem> + <para>This option enables the language extension defined in the + Haskell 98 Foreign Function Interface Addendum plus deprecated + syntax of previous versions of the FFI for backwards + compatibility.</para> + + <para>New reserved words: <literal>foreign</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-monomorphism-restriction</option>: + <indexterm><primary><option>-fno-monomorphism-restriction</option></primary></indexterm> + </term> + <listitem> + <para> Switch off the Haskell 98 monomorphism restriction. + Independent of the <option>-fglasgow-exts</option> + flag. </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fallow-overlapping-instances</option> + <indexterm><primary><option>-fallow-overlapping-instances</option></primary></indexterm> + </term> + <term> + <option>-fallow-undecidable-instances</option> + <indexterm><primary><option>-fallow-undecidable-instances</option></primary></indexterm> + </term> + <term> + <option>-fallow-incoherent-instances</option> + <indexterm><primary><option>-fallow-incoherent-instances</option></primary></indexterm> + </term> + <term> + <option>-fcontext-stack</option> + <indexterm><primary><option>-fcontext-stack</option></primary></indexterm> + </term> + <listitem> + <para> See <xref linkend="instance-decls"/>. Only relevant + if you also use <option>-fglasgow-exts</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-finline-phase</option> + <indexterm><primary><option>-finline-phase</option></primary></indexterm> + </term> + <listitem> + <para>See <xref linkend="rewrite-rules"/>. Only relevant if + you also use <option>-fglasgow-exts</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-farrows</option> + <indexterm><primary><option>-farrows</option></primary></indexterm> + </term> + <listitem> + <para>See <xref linkend="arrow-notation"/>. Independent of + <option>-fglasgow-exts</option>.</para> + + <para>New reserved words/symbols: <literal>rec</literal>, + <literal>proc</literal>, <literal>-<</literal>, + <literal>>-</literal>, <literal>-<<</literal>, + <literal>>>-</literal>.</para> + + <para>Other syntax stolen: <literal>(|</literal>, + <literal>|)</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fgenerics</option> + <indexterm><primary><option>-fgenerics</option></primary></indexterm> + </term> + <listitem> + <para>See <xref linkend="generic-classes"/>. Independent of + <option>-fglasgow-exts</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fno-implicit-prelude</option></term> + <listitem> + <para><indexterm><primary>-fno-implicit-prelude + option</primary></indexterm> GHC normally imports + <filename>Prelude.hi</filename> files for you. If you'd + rather it didn't, then give it a + <option>-fno-implicit-prelude</option> option. The idea is + that you can then import a Prelude of your own. (But don't + call it <literal>Prelude</literal>; the Haskell module + namespace is flat, and you must not conflict with any + Prelude module.)</para> + + <para>Even though you have not imported the Prelude, most of + the built-in syntax still refers to the built-in Haskell + Prelude types and values, as specified by the Haskell + Report. For example, the type <literal>[Int]</literal> + still means <literal>Prelude.[] Int</literal>; tuples + continue to refer to the standard Prelude tuples; the + translation for list comprehensions continues to use + <literal>Prelude.map</literal> etc.</para> + + <para>However, <option>-fno-implicit-prelude</option> does + change the handling of certain built-in syntax: see <xref + linkend="rebindable-syntax"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fimplicit-params</option></term> + <listitem> + <para>Enables implicit parameters (see <xref + linkend="implicit-parameters"/>). Currently also implied by + <option>-fglasgow-exts</option>.</para> + + <para>Syntax stolen: + <literal>?<replaceable>varid</replaceable></literal>, + <literal>%<replaceable>varid</replaceable></literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fscoped-type-variables</option></term> + <listitem> + <para>Enables lexically-scoped type variables (see <xref + linkend="scoped-type-variables"/>). Implied by + <option>-fglasgow-exts</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fth</option></term> + <listitem> + <para>Enables Template Haskell (see <xref + linkend="template-haskell"/>). Currently also implied by + <option>-fglasgow-exts</option>.</para> + + <para>Syntax stolen: <literal>[|</literal>, + <literal>[e|</literal>, <literal>[p|</literal>, + <literal>[d|</literal>, <literal>[t|</literal>, + <literal>$(</literal>, + <literal>$<replaceable>varid</replaceable></literal>.</para> + </listitem> + </varlistentry> + + </variablelist> + </sect1> + +<!-- UNBOXED TYPES AND PRIMITIVE OPERATIONS --> +<!-- included from primitives.sgml --> +<!-- &primitives; --> +<sect1 id="primitives"> + <title>Unboxed types and primitive operations</title> + +<para>GHC is built on a raft of primitive data types and operations. +While you really can use this stuff to write fast code, + we generally find it a lot less painful, and more satisfying in the + long run, to use higher-level language features and libraries. With + any luck, the code you write will be optimised to the efficient + unboxed version in any case. And if it isn't, we'd like to know + about it.</para> + +<para>We do not currently have good, up-to-date documentation about the +primitives, perhaps because they are mainly intended for internal use. +There used to be a long section about them here in the User Guide, but it +became out of date, and wrong information is worse than none.</para> + +<para>The Real Truth about what primitive types there are, and what operations +work over those types, is held in the file +<filename>fptools/ghc/compiler/prelude/primops.txt.pp</filename>. +This file is used directly to generate GHC's primitive-operation definitions, so +it is always correct! It is also intended for processing into text.</para> + +<para> Indeed, +the result of such processing is part of the description of the + <ulink + url="http://haskell.cs.yale.edu/ghc/docs/papers/core.ps.gz">External + Core language</ulink>. +So that document is a good place to look for a type-set version. +We would be very happy if someone wanted to volunteer to produce an SGML +back end to the program that processes <filename>primops.txt</filename> so that +we could include the results here in the User Guide.</para> + +<para>What follows here is a brief summary of some main points.</para> + +<sect2 id="glasgow-unboxed"> +<title>Unboxed types +</title> + +<para> +<indexterm><primary>Unboxed types (Glasgow extension)</primary></indexterm> +</para> + +<para>Most types in GHC are <firstterm>boxed</firstterm>, which means +that values of that type are represented by a pointer to a heap +object. The representation of a Haskell <literal>Int</literal>, for +example, is a two-word heap object. An <firstterm>unboxed</firstterm> +type, however, is represented by the value itself, no pointers or heap +allocation are involved. +</para> + +<para> +Unboxed types correspond to the “raw machine” types you +would use in C: <literal>Int#</literal> (long int), +<literal>Double#</literal> (double), <literal>Addr#</literal> +(void *), etc. The <emphasis>primitive operations</emphasis> +(PrimOps) on these types are what you might expect; e.g., +<literal>(+#)</literal> is addition on +<literal>Int#</literal>s, and is the machine-addition that we all +know and love—usually one instruction. +</para> + +<para> +Primitive (unboxed) types cannot be defined in Haskell, and are +therefore built into the language and compiler. Primitive types are +always unlifted; that is, a value of a primitive type cannot be +bottom. We use the convention that primitive types, values, and +operations have a <literal>#</literal> suffix. +</para> + +<para> +Primitive values are often represented by a simple bit-pattern, such +as <literal>Int#</literal>, <literal>Float#</literal>, +<literal>Double#</literal>. But this is not necessarily the case: +a primitive value might be represented by a pointer to a +heap-allocated object. Examples include +<literal>Array#</literal>, the type of primitive arrays. A +primitive array is heap-allocated because it is too big a value to fit +in a register, and would be too expensive to copy around; in a sense, +it is accidental that it is represented by a pointer. If a pointer +represents a primitive value, then it really does point to that value: +no unevaluated thunks, no indirections…nothing can be at the +other end of the pointer than the primitive value. +A numerically-intensive program using unboxed types can +go a <emphasis>lot</emphasis> faster than its “standard” +counterpart—we saw a threefold speedup on one example. +</para> + +<para> +There are some restrictions on the use of primitive types: +<itemizedlist> +<listitem><para>The main restriction +is that you can't pass a primitive value to a polymorphic +function or store one in a polymorphic data type. This rules out +things like <literal>[Int#]</literal> (i.e. lists of primitive +integers). The reason for this restriction is that polymorphic +arguments and constructor fields are assumed to be pointers: if an +unboxed integer is stored in one of these, the garbage collector would +attempt to follow it, leading to unpredictable space leaks. Or a +<function>seq</function> operation on the polymorphic component may +attempt to dereference the pointer, with disastrous results. Even +worse, the unboxed value might be larger than a pointer +(<literal>Double#</literal> for instance). +</para> +</listitem> +<listitem><para> You cannot bind a variable with an unboxed type +in a <emphasis>top-level</emphasis> binding. +</para></listitem> +<listitem><para> You cannot bind a variable with an unboxed type +in a <emphasis>recursive</emphasis> binding. +</para></listitem> +<listitem><para> You may bind unboxed variables in a (non-recursive, +non-top-level) pattern binding, but any such variable causes the entire +pattern-match +to become strict. For example: +<programlisting> + data Foo = Foo Int Int# + + f x = let (Foo a b, w) = ..rhs.. in ..body.. +</programlisting> +Since <literal>b</literal> has type <literal>Int#</literal>, the entire pattern +match +is strict, and the program behaves as if you had written +<programlisting> + data Foo = Foo Int Int# + + f x = case ..rhs.. of { (Foo a b, w) -> ..body.. } +</programlisting> +</para> +</listitem> +</itemizedlist> +</para> + +</sect2> + +<sect2 id="unboxed-tuples"> +<title>Unboxed Tuples +</title> + +<para> +Unboxed tuples aren't really exported by <literal>GHC.Exts</literal>, +they're available by default with <option>-fglasgow-exts</option>. An +unboxed tuple looks like this: +</para> + +<para> + +<programlisting> +(# e_1, ..., e_n #) +</programlisting> + +</para> + +<para> +where <literal>e_1..e_n</literal> are expressions of any +type (primitive or non-primitive). The type of an unboxed tuple looks +the same. +</para> + +<para> +Unboxed tuples are used for functions that need to return multiple +values, but they avoid the heap allocation normally associated with +using fully-fledged tuples. When an unboxed tuple is returned, the +components are put directly into registers or on the stack; the +unboxed tuple itself does not have a composite representation. Many +of the primitive operations listed in <literal>primops.txt.pp</literal> return unboxed +tuples. +In particular, the <literal>IO</literal> and <literal>ST</literal> monads use unboxed +tuples to avoid unnecessary allocation during sequences of operations. +</para> + +<para> +There are some pretty stringent restrictions on the use of unboxed tuples: +<itemizedlist> +<listitem> + +<para> +Values of unboxed tuple types are subject to the same restrictions as +other unboxed types; i.e. they may not be stored in polymorphic data +structures or passed to polymorphic functions. + +</para> +</listitem> +<listitem> + +<para> +No variable can have an unboxed tuple type, nor may a constructor or function +argument have an unboxed tuple type. The following are all illegal: + + +<programlisting> + data Foo = Foo (# Int, Int #) + + f :: (# Int, Int #) -> (# Int, Int #) + f x = x + + g :: (# Int, Int #) -> Int + g (# a,b #) = a + + h x = let y = (# x,x #) in ... +</programlisting> +</para> +</listitem> +</itemizedlist> +</para> +<para> +The typical use of unboxed tuples is simply to return multiple values, +binding those multiple results with a <literal>case</literal> expression, thus: +<programlisting> + f x y = (# x+1, y-1 #) + g x = case f x x of { (# a, b #) -> a + b } +</programlisting> +You can have an unboxed tuple in a pattern binding, thus +<programlisting> + f x = let (# p,q #) = h x in ..body.. +</programlisting> +If the types of <literal>p</literal> and <literal>q</literal> are not unboxed, +the resulting binding is lazy like any other Haskell pattern binding. The +above example desugars like this: +<programlisting> + f x = let t = case h x o f{ (# p,q #) -> (p,q) + p = fst t + q = snd t + in ..body.. +</programlisting> +Indeed, the bindings can even be recursive. +</para> + +</sect2> +</sect1> + + +<!-- ====================== SYNTACTIC EXTENSIONS ======================= --> + +<sect1 id="syntax-extns"> +<title>Syntactic extensions</title> + + <!-- ====================== HIERARCHICAL MODULES ======================= --> + + <sect2 id="hierarchical-modules"> + <title>Hierarchical Modules</title> + + <para>GHC supports a small extension to the syntax of module + names: a module name is allowed to contain a dot + <literal>‘.’</literal>. This is also known as the + “hierarchical module namespace” extension, because + it extends the normally flat Haskell module namespace into a + more flexible hierarchy of modules.</para> + + <para>This extension has very little impact on the language + itself; modules names are <emphasis>always</emphasis> fully + qualified, so you can just think of the fully qualified module + name as <quote>the module name</quote>. In particular, this + means that the full module name must be given after the + <literal>module</literal> keyword at the beginning of the + module; for example, the module <literal>A.B.C</literal> must + begin</para> + +<programlisting>module A.B.C</programlisting> + + + <para>It is a common strategy to use the <literal>as</literal> + keyword to save some typing when using qualified names with + hierarchical modules. For example:</para> + +<programlisting> +import qualified Control.Monad.ST.Strict as ST +</programlisting> + + <para>For details on how GHC searches for source and interface + files in the presence of hierarchical modules, see <xref + linkend="search-path"/>.</para> + + <para>GHC comes with a large collection of libraries arranged + hierarchically; see the accompanying library documentation. + There is an ongoing project to create and maintain a stable set + of <quote>core</quote> libraries used by several Haskell + compilers, and the libraries that GHC comes with represent the + current status of that project. For more details, see <ulink + url="http://www.haskell.org/~simonmar/libraries/libraries.html">Haskell + Libraries</ulink>.</para> + + </sect2> + + <!-- ====================== PATTERN GUARDS ======================= --> + +<sect2 id="pattern-guards"> +<title>Pattern guards</title> + +<para> +<indexterm><primary>Pattern guards (Glasgow extension)</primary></indexterm> +The discussion that follows is an abbreviated version of Simon Peyton Jones's original <ulink url="http://research.microsoft.com/~simonpj/Haskell/guards.html">proposal</ulink>. (Note that the proposal was written before pattern guards were implemented, so refers to them as unimplemented.) +</para> + +<para> +Suppose we have an abstract data type of finite maps, with a +lookup operation: + +<programlisting> +lookup :: FiniteMap -> Int -> Maybe Int +</programlisting> + +The lookup returns <function>Nothing</function> if the supplied key is not in the domain of the mapping, and <function>(Just v)</function> otherwise, +where <varname>v</varname> is the value that the key maps to. Now consider the following definition: +</para> + +<programlisting> +clunky env var1 var2 | ok1 && ok2 = val1 + val2 +| otherwise = var1 + var2 +where + m1 = lookup env var1 + m2 = lookup env var2 + ok1 = maybeToBool m1 + ok2 = maybeToBool m2 + val1 = expectJust m1 + val2 = expectJust m2 +</programlisting> + +<para> +The auxiliary functions are +</para> + +<programlisting> +maybeToBool :: Maybe a -> Bool +maybeToBool (Just x) = True +maybeToBool Nothing = False + +expectJust :: Maybe a -> a +expectJust (Just x) = x +expectJust Nothing = error "Unexpected Nothing" +</programlisting> + +<para> +What is <function>clunky</function> doing? The guard <literal>ok1 && +ok2</literal> checks that both lookups succeed, using +<function>maybeToBool</function> to convert the <function>Maybe</function> +types to booleans. The (lazily evaluated) <function>expectJust</function> +calls extract the values from the results of the lookups, and binds the +returned values to <varname>val1</varname> and <varname>val2</varname> +respectively. If either lookup fails, then clunky takes the +<literal>otherwise</literal> case and returns the sum of its arguments. +</para> + +<para> +This is certainly legal Haskell, but it is a tremendously verbose and +un-obvious way to achieve the desired effect. Arguably, a more direct way +to write clunky would be to use case expressions: +</para> + +<programlisting> +clunky env var1 var1 = case lookup env var1 of + Nothing -> fail + Just val1 -> case lookup env var2 of + Nothing -> fail + Just val2 -> val1 + val2 +where + fail = val1 + val2 +</programlisting> + +<para> +This is a bit shorter, but hardly better. Of course, we can rewrite any set +of pattern-matching, guarded equations as case expressions; that is +precisely what the compiler does when compiling equations! The reason that +Haskell provides guarded equations is because they allow us to write down +the cases we want to consider, one at a time, independently of each other. +This structure is hidden in the case version. Two of the right-hand sides +are really the same (<function>fail</function>), and the whole expression +tends to become more and more indented. +</para> + +<para> +Here is how I would write clunky: +</para> + +<programlisting> +clunky env var1 var1 + | Just val1 <- lookup env var1 + , Just val2 <- lookup env var2 + = val1 + val2 +...other equations for clunky... +</programlisting> + +<para> +The semantics should be clear enough. The qualifiers are matched in order. +For a <literal><-</literal> qualifier, which I call a pattern guard, the +right hand side is evaluated and matched against the pattern on the left. +If the match fails then the whole guard fails and the next equation is +tried. If it succeeds, then the appropriate binding takes place, and the +next qualifier is matched, in the augmented environment. Unlike list +comprehensions, however, the type of the expression to the right of the +<literal><-</literal> is the same as the type of the pattern to its +left. The bindings introduced by pattern guards scope over all the +remaining guard qualifiers, and over the right hand side of the equation. +</para> + +<para> +Just as with list comprehensions, boolean expressions can be freely mixed +with among the pattern guards. For example: +</para> + +<programlisting> +f x | [y] <- x + , y > 3 + , Just z <- h y + = ... +</programlisting> + +<para> +Haskell's current guards therefore emerge as a special case, in which the +qualifier list has just one element, a boolean expression. +</para> +</sect2> + + <!-- ===================== Recursive do-notation =================== --> + +<sect2 id="mdo-notation"> +<title>The recursive do-notation +</title> + +<para> The recursive do-notation (also known as mdo-notation) is implemented as described in +"A recursive do for Haskell", +Levent Erkok, John Launchbury", +Haskell Workshop 2002, pages: 29-37. Pittsburgh, Pennsylvania. +</para> +<para> +The do-notation of Haskell does not allow <emphasis>recursive bindings</emphasis>, +that is, the variables bound in a do-expression are visible only in the textually following +code block. Compare this to a let-expression, where bound variables are visible in the entire binding +group. It turns out that several applications can benefit from recursive bindings in +the do-notation, and this extension provides the necessary syntactic support. +</para> +<para> +Here is a simple (yet contrived) example: +</para> +<programlisting> +import Control.Monad.Fix + +justOnes = mdo xs <- Just (1:xs) + return xs +</programlisting> +<para> +As you can guess <literal>justOnes</literal> will evaluate to <literal>Just [1,1,1,...</literal>. +</para> + +<para> +The Control.Monad.Fix library introduces the <literal>MonadFix</literal> class. It's definition is: +</para> +<programlisting> +class Monad m => MonadFix m where + mfix :: (a -> m a) -> m a +</programlisting> +<para> +The function <literal>mfix</literal> +dictates how the required recursion operation should be performed. If recursive bindings are required for a monad, +then that monad must be declared an instance of the <literal>MonadFix</literal> class. +For details, see the above mentioned reference. +</para> +<para> +The following instances of <literal>MonadFix</literal> are automatically provided: List, Maybe, IO. +Furthermore, the Control.Monad.ST and Control.Monad.ST.Lazy modules provide the instances of the MonadFix class +for Haskell's internal state monad (strict and lazy, respectively). +</para> +<para> +There are three important points in using the recursive-do notation: +<itemizedlist> +<listitem><para> +The recursive version of the do-notation uses the keyword <literal>mdo</literal> (rather +than <literal>do</literal>). +</para></listitem> + +<listitem><para> +You should <literal>import Control.Monad.Fix</literal>. +(Note: Strictly speaking, this import is required only when you need to refer to the name +<literal>MonadFix</literal> in your program, but the import is always safe, and the programmers +are encouraged to always import this module when using the mdo-notation.) +</para></listitem> + +<listitem><para> +As with other extensions, ghc should be given the flag <literal>-fglasgow-exts</literal> +</para></listitem> +</itemizedlist> +</para> + +<para> +The web page: <ulink url="http://www.cse.ogi.edu/PacSoft/projects/rmb">http://www.cse.ogi.edu/PacSoft/projects/rmb</ulink> +contains up to date information on recursive monadic bindings. +</para> + +<para> +Historical note: The old implementation of the mdo-notation (and most +of the existing documents) used the name +<literal>MonadRec</literal> for the class and the corresponding library. +This name is not supported by GHC. +</para> + +</sect2> + + + <!-- ===================== PARALLEL LIST COMPREHENSIONS =================== --> + + <sect2 id="parallel-list-comprehensions"> + <title>Parallel List Comprehensions</title> + <indexterm><primary>list comprehensions</primary><secondary>parallel</secondary> + </indexterm> + <indexterm><primary>parallel list comprehensions</primary> + </indexterm> + + <para>Parallel list comprehensions are a natural extension to list + comprehensions. List comprehensions can be thought of as a nice + syntax for writing maps and filters. Parallel comprehensions + extend this to include the zipWith family.</para> + + <para>A parallel list comprehension has multiple independent + branches of qualifier lists, each separated by a `|' symbol. For + example, the following zips together two lists:</para> + +<programlisting> + [ (x, y) | x <- xs | y <- ys ] +</programlisting> + + <para>The behavior of parallel list comprehensions follows that of + zip, in that the resulting list will have the same length as the + shortest branch.</para> + + <para>We can define parallel list comprehensions by translation to + regular comprehensions. Here's the basic idea:</para> + + <para>Given a parallel comprehension of the form: </para> + +<programlisting> + [ e | p1 <- e11, p2 <- e12, ... + | q1 <- e21, q2 <- e22, ... + ... + ] +</programlisting> + + <para>This will be translated to: </para> + +<programlisting> + [ e | ((p1,p2), (q1,q2), ...) <- zipN [(p1,p2) | p1 <- e11, p2 <- e12, ...] + [(q1,q2) | q1 <- e21, q2 <- e22, ...] + ... + ] +</programlisting> + + <para>where `zipN' is the appropriate zip for the given number of + branches.</para> + + </sect2> + +<sect2 id="rebindable-syntax"> +<title>Rebindable syntax</title> + + + <para>GHC allows most kinds of built-in syntax to be rebound by + the user, to facilitate replacing the <literal>Prelude</literal> + with a home-grown version, for example.</para> + + <para>You may want to define your own numeric class + hierarchy. It completely defeats that purpose if the + literal "1" means "<literal>Prelude.fromInteger + 1</literal>", which is what the Haskell Report specifies. + So the <option>-fno-implicit-prelude</option> flag causes + the following pieces of built-in syntax to refer to + <emphasis>whatever is in scope</emphasis>, not the Prelude + versions: + + <itemizedlist> + <listitem> + <para>An integer literal <literal>368</literal> means + "<literal>fromInteger (368::Integer)</literal>", rather than + "<literal>Prelude.fromInteger (368::Integer)</literal>". +</para> </listitem> + + <listitem><para>Fractional literals are handed in just the same way, + except that the translation is + <literal>fromRational (3.68::Rational)</literal>. +</para> </listitem> + + <listitem><para>The equality test in an overloaded numeric pattern + uses whatever <literal>(==)</literal> is in scope. +</para> </listitem> + + <listitem><para>The subtraction operation, and the + greater-than-or-equal test, in <literal>n+k</literal> patterns + use whatever <literal>(-)</literal> and <literal>(>=)</literal> are in scope. + </para></listitem> + + <listitem> + <para>Negation (e.g. "<literal>- (f x)</literal>") + means "<literal>negate (f x)</literal>", both in numeric + patterns, and expressions. + </para></listitem> + + <listitem> + <para>"Do" notation is translated using whatever + functions <literal>(>>=)</literal>, + <literal>(>>)</literal>, and <literal>fail</literal>, + are in scope (not the Prelude + versions). List comprehensions, mdo (<xref linkend="mdo-notation"/>), and parallel array + comprehensions, are unaffected. </para></listitem> + + <listitem> + <para>Arrow + notation (see <xref linkend="arrow-notation"/>) + uses whatever <literal>arr</literal>, + <literal>(>>>)</literal>, <literal>first</literal>, + <literal>app</literal>, <literal>(|||)</literal> and + <literal>loop</literal> functions are in scope. But unlike the + other constructs, the types of these functions must match the + Prelude types very closely. Details are in flux; if you want + to use this, ask! + </para></listitem> + </itemizedlist> +In all cases (apart from arrow notation), the static semantics should be that of the desugared form, +even if that is a little unexpected. For emample, the +static semantics of the literal <literal>368</literal> +is exactly that of <literal>fromInteger (368::Integer)</literal>; it's fine for +<literal>fromInteger</literal> to have any of the types: +<programlisting> +fromInteger :: Integer -> Integer +fromInteger :: forall a. Foo a => Integer -> a +fromInteger :: Num a => a -> Integer +fromInteger :: Integer -> Bool -> Bool +</programlisting> +</para> + + <para>Be warned: this is an experimental facility, with + fewer checks than usual. Use <literal>-dcore-lint</literal> + to typecheck the desugared program. If Core Lint is happy + you should be all right.</para> + +</sect2> +</sect1> + + +<!-- TYPE SYSTEM EXTENSIONS --> +<sect1 id="type-extensions"> +<title>Type system extensions</title> + + +<sect2> +<title>Data types and type synonyms</title> + +<sect3 id="nullary-types"> +<title>Data types with no constructors</title> + +<para>With the <option>-fglasgow-exts</option> flag, GHC lets you declare +a data type with no constructors. For example:</para> + +<programlisting> + data S -- S :: * + data T a -- T :: * -> * +</programlisting> + +<para>Syntactically, the declaration lacks the "= constrs" part. The +type can be parameterised over types of any kind, but if the kind is +not <literal>*</literal> then an explicit kind annotation must be used +(see <xref linkend="sec-kinding"/>).</para> + +<para>Such data types have only one value, namely bottom. +Nevertheless, they can be useful when defining "phantom types".</para> +</sect3> + +<sect3 id="infix-tycons"> +<title>Infix type constructors, classes, and type variables</title> + +<para> +GHC allows type constructors, classes, and type variables to be operators, and +to be written infix, very much like expressions. More specifically: +<itemizedlist> +<listitem><para> + A type constructor or class can be an operator, beginning with a colon; e.g. <literal>:*:</literal>. + The lexical syntax is the same as that for data constructors. + </para></listitem> +<listitem><para> + Data type and type-synonym declarations can be written infix, parenthesised + if you want further arguments. E.g. +<screen> + data a :*: b = Foo a b + type a :+: b = Either a b + class a :=: b where ... + + data (a :**: b) x = Baz a b x + type (a :++: b) y = Either (a,b) y +</screen> + </para></listitem> +<listitem><para> + Types, and class constraints, can be written infix. For example + <screen> + x :: Int :*: Bool + f :: (a :=: b) => a -> b + </screen> + </para></listitem> +<listitem><para> + A type variable can be an (unqualified) operator e.g. <literal>+</literal>. + The lexical syntax is the same as that for variable operators, excluding "(.)", + "(!)", and "(*)". In a binding position, the operator must be + parenthesised. For example: +<programlisting> + type T (+) = Int + Int + f :: T Either + f = Left 3 + + liftA2 :: Arrow (~>) + => (a -> b -> c) -> (e ~> a) -> (e ~> b) -> (e ~> c) + liftA2 = ... +</programlisting> + </para></listitem> +<listitem><para> + Back-quotes work + as for expressions, both for type constructors and type variables; e.g. <literal>Int `Either` Bool</literal>, or + <literal>Int `a` Bool</literal>. Similarly, parentheses work the same; e.g. <literal>(:*:) Int Bool</literal>. + </para></listitem> +<listitem><para> + Fixities may be declared for type constructors, or classes, just as for data constructors. However, + one cannot distinguish between the two in a fixity declaration; a fixity declaration + sets the fixity for a data constructor and the corresponding type constructor. For example: +<screen> + infixl 7 T, :*: +</screen> + sets the fixity for both type constructor <literal>T</literal> and data constructor <literal>T</literal>, + and similarly for <literal>:*:</literal>. + <literal>Int `a` Bool</literal>. + </para></listitem> +<listitem><para> + Function arrow is <literal>infixr</literal> with fixity 0. (This might change; I'm not sure what it should be.) + </para></listitem> + +</itemizedlist> +</para> +</sect3> + +<sect3 id="type-synonyms"> +<title>Liberalised type synonyms</title> + +<para> +Type synonyms are like macros at the type level, and +GHC does validity checking on types <emphasis>only after expanding type synonyms</emphasis>. +That means that GHC can be very much more liberal about type synonyms than Haskell 98: +<itemizedlist> +<listitem> <para>You can write a <literal>forall</literal> (including overloading) +in a type synonym, thus: +<programlisting> + type Discard a = forall b. Show b => a -> b -> (a, String) + + f :: Discard a + f x y = (x, show y) + + g :: Discard Int -> (Int,Bool) -- A rank-2 type + g f = f Int True +</programlisting> +</para> +</listitem> + +<listitem><para> +You can write an unboxed tuple in a type synonym: +<programlisting> + type Pr = (# Int, Int #) + + h :: Int -> Pr + h x = (# x, x #) +</programlisting> +</para></listitem> + +<listitem><para> +You can apply a type synonym to a forall type: +<programlisting> + type Foo a = a -> a -> Bool + + f :: Foo (forall b. b->b) +</programlisting> +After expanding the synonym, <literal>f</literal> has the legal (in GHC) type: +<programlisting> + f :: (forall b. b->b) -> (forall b. b->b) -> Bool +</programlisting> +</para></listitem> + +<listitem><para> +You can apply a type synonym to a partially applied type synonym: +<programlisting> + type Generic i o = forall x. i x -> o x + type Id x = x + + foo :: Generic Id [] +</programlisting> +After expanding the synonym, <literal>foo</literal> has the legal (in GHC) type: +<programlisting> + foo :: forall x. x -> [x] +</programlisting> +</para></listitem> + +</itemizedlist> +</para> + +<para> +GHC currently does kind checking before expanding synonyms (though even that +could be changed.) +</para> +<para> +After expanding type synonyms, GHC does validity checking on types, looking for +the following mal-formedness which isn't detected simply by kind checking: +<itemizedlist> +<listitem><para> +Type constructor applied to a type involving for-alls. +</para></listitem> +<listitem><para> +Unboxed tuple on left of an arrow. +</para></listitem> +<listitem><para> +Partially-applied type synonym. +</para></listitem> +</itemizedlist> +So, for example, +this will be rejected: +<programlisting> + type Pr = (# Int, Int #) + + h :: Pr -> Int + h x = ... +</programlisting> +because GHC does not allow unboxed tuples on the left of a function arrow. +</para> +</sect3> + + +<sect3 id="existential-quantification"> +<title>Existentially quantified data constructors +</title> + +<para> +The idea of using existential quantification in data type declarations +was suggested by Perry, and implemented in Hope+ (Nigel Perry, <emphasis>The Implementation +of Practical Functional Programming Languages</emphasis>, PhD Thesis, University of +London, 1991). It was later formalised by Laufer and Odersky +(<emphasis>Polymorphic type inference and abstract data types</emphasis>, +TOPLAS, 16(5), pp1411-1430, 1994). +It's been in Lennart +Augustsson's <command>hbc</command> Haskell compiler for several years, and +proved very useful. Here's the idea. Consider the declaration: +</para> + +<para> + +<programlisting> + data Foo = forall a. MkFoo a (a -> Bool) + | Nil +</programlisting> + +</para> + +<para> +The data type <literal>Foo</literal> has two constructors with types: +</para> + +<para> + +<programlisting> + MkFoo :: forall a. a -> (a -> Bool) -> Foo + Nil :: Foo +</programlisting> + +</para> + +<para> +Notice that the type variable <literal>a</literal> in the type of <function>MkFoo</function> +does not appear in the data type itself, which is plain <literal>Foo</literal>. +For example, the following expression is fine: +</para> + +<para> + +<programlisting> + [MkFoo 3 even, MkFoo 'c' isUpper] :: [Foo] +</programlisting> + +</para> + +<para> +Here, <literal>(MkFoo 3 even)</literal> packages an integer with a function +<function>even</function> that maps an integer to <literal>Bool</literal>; and <function>MkFoo 'c' +isUpper</function> packages a character with a compatible function. These +two things are each of type <literal>Foo</literal> and can be put in a list. +</para> + +<para> +What can we do with a value of type <literal>Foo</literal>?. In particular, +what happens when we pattern-match on <function>MkFoo</function>? +</para> + +<para> + +<programlisting> + f (MkFoo val fn) = ??? +</programlisting> + +</para> + +<para> +Since all we know about <literal>val</literal> and <function>fn</function> is that they +are compatible, the only (useful) thing we can do with them is to +apply <function>fn</function> to <literal>val</literal> to get a boolean. For example: +</para> + +<para> + +<programlisting> + f :: Foo -> Bool + f (MkFoo val fn) = fn val +</programlisting> + +</para> + +<para> +What this allows us to do is to package heterogenous values +together with a bunch of functions that manipulate them, and then treat +that collection of packages in a uniform manner. You can express +quite a bit of object-oriented-like programming this way. +</para> + +<sect4 id="existential"> +<title>Why existential? +</title> + +<para> +What has this to do with <emphasis>existential</emphasis> quantification? +Simply that <function>MkFoo</function> has the (nearly) isomorphic type +</para> + +<para> + +<programlisting> + MkFoo :: (exists a . (a, a -> Bool)) -> Foo +</programlisting> + +</para> + +<para> +But Haskell programmers can safely think of the ordinary +<emphasis>universally</emphasis> quantified type given above, thereby avoiding +adding a new existential quantification construct. +</para> + +</sect4> + +<sect4> +<title>Type classes</title> + +<para> +An easy extension is to allow +arbitrary contexts before the constructor. For example: +</para> + +<para> + +<programlisting> +data Baz = forall a. Eq a => Baz1 a a + | forall b. Show b => Baz2 b (b -> b) +</programlisting> + +</para> + +<para> +The two constructors have the types you'd expect: +</para> + +<para> + +<programlisting> +Baz1 :: forall a. Eq a => a -> a -> Baz +Baz2 :: forall b. Show b => b -> (b -> b) -> Baz +</programlisting> + +</para> + +<para> +But when pattern matching on <function>Baz1</function> the matched values can be compared +for equality, and when pattern matching on <function>Baz2</function> the first matched +value can be converted to a string (as well as applying the function to it). +So this program is legal: +</para> + +<para> + +<programlisting> + f :: Baz -> String + f (Baz1 p q) | p == q = "Yes" + | otherwise = "No" + f (Baz2 v fn) = show (fn v) +</programlisting> + +</para> + +<para> +Operationally, in a dictionary-passing implementation, the +constructors <function>Baz1</function> and <function>Baz2</function> must store the +dictionaries for <literal>Eq</literal> and <literal>Show</literal> respectively, and +extract it on pattern matching. +</para> + +<para> +Notice the way that the syntax fits smoothly with that used for +universal quantification earlier. +</para> + +</sect4> + +<sect4> +<title>Record Constructors</title> + +<para> +GHC allows existentials to be used with records syntax as well. For example: + +<programlisting> +data Counter a = forall self. NewCounter + { _this :: self + , _inc :: self -> self + , _display :: self -> IO () + , tag :: a + } +</programlisting> +Here <literal>tag</literal> is a public field, with a well-typed selector +function <literal>tag :: Counter a -> a</literal>. The <literal>self</literal> +type is hidden from the outside; any attempt to apply <literal>_this</literal>, +<literal>_inc</literal> or <literal>_output</literal> as functions will raise a +compile-time error. In other words, <emphasis>GHC defines a record selector function +only for fields whose type does not mention the existentially-quantified variables</emphasis>. +(This example used an underscore in the fields for which record selectors +will not be defined, but that is only programming style; GHC ignores them.) +</para> + +<para> +To make use of these hidden fields, we need to create some helper functions: + +<programlisting> +inc :: Counter a -> Counter a +inc (NewCounter x i d t) = NewCounter + { _this = i x, _inc = i, _display = d, tag = t } + +display :: Counter a -> IO () +display NewCounter{ _this = x, _display = d } = d x +</programlisting> + +Now we can define counters with different underlying implementations: + +<programlisting> +counterA :: Counter String +counterA = NewCounter + { _this = 0, _inc = (1+), _display = print, tag = "A" } + +counterB :: Counter String +counterB = NewCounter + { _this = "", _inc = ('#':), _display = putStrLn, tag = "B" } + +main = do + display (inc counterA) -- prints "1" + display (inc (inc counterB)) -- prints "##" +</programlisting> + +In GADT declarations (see <xref linkend="gadt"/>), the explicit +<literal>forall</literal> may be omitted. For example, we can express +the same <literal>Counter a</literal> using GADT: + +<programlisting> +data Counter a where + NewCounter { _this :: self + , _inc :: self -> self + , _display :: self -> IO () + , tag :: a + } + :: Counter a +</programlisting> + +At the moment, record update syntax is only supported for Haskell 98 data types, +so the following function does <emphasis>not</emphasis> work: + +<programlisting> +-- This is invalid; use explicit NewCounter instead for now +setTag :: Counter a -> a -> Counter a +setTag obj t = obj{ tag = t } +</programlisting> + +</para> + +</sect4> + + +<sect4> +<title>Restrictions</title> + +<para> +There are several restrictions on the ways in which existentially-quantified +constructors can be use. +</para> + +<para> + +<itemizedlist> +<listitem> + +<para> + When pattern matching, each pattern match introduces a new, +distinct, type for each existential type variable. These types cannot +be unified with any other type, nor can they escape from the scope of +the pattern match. For example, these fragments are incorrect: + + +<programlisting> +f1 (MkFoo a f) = a +</programlisting> + + +Here, the type bound by <function>MkFoo</function> "escapes", because <literal>a</literal> +is the result of <function>f1</function>. One way to see why this is wrong is to +ask what type <function>f1</function> has: + + +<programlisting> + f1 :: Foo -> a -- Weird! +</programlisting> + + +What is this "<literal>a</literal>" in the result type? Clearly we don't mean +this: + + +<programlisting> + f1 :: forall a. Foo -> a -- Wrong! +</programlisting> + + +The original program is just plain wrong. Here's another sort of error + + +<programlisting> + f2 (Baz1 a b) (Baz1 p q) = a==q +</programlisting> + + +It's ok to say <literal>a==b</literal> or <literal>p==q</literal>, but +<literal>a==q</literal> is wrong because it equates the two distinct types arising +from the two <function>Baz1</function> constructors. + + +</para> +</listitem> +<listitem> + +<para> +You can't pattern-match on an existentially quantified +constructor in a <literal>let</literal> or <literal>where</literal> group of +bindings. So this is illegal: + + +<programlisting> + f3 x = a==b where { Baz1 a b = x } +</programlisting> + +Instead, use a <literal>case</literal> expression: + +<programlisting> + f3 x = case x of Baz1 a b -> a==b +</programlisting> + +In general, you can only pattern-match +on an existentially-quantified constructor in a <literal>case</literal> expression or +in the patterns of a function definition. + +The reason for this restriction is really an implementation one. +Type-checking binding groups is already a nightmare without +existentials complicating the picture. Also an existential pattern +binding at the top level of a module doesn't make sense, because it's +not clear how to prevent the existentially-quantified type "escaping". +So for now, there's a simple-to-state restriction. We'll see how +annoying it is. + +</para> +</listitem> +<listitem> + +<para> +You can't use existential quantification for <literal>newtype</literal> +declarations. So this is illegal: + + +<programlisting> + newtype T = forall a. Ord a => MkT a +</programlisting> + + +Reason: a value of type <literal>T</literal> must be represented as a +pair of a dictionary for <literal>Ord t</literal> and a value of type +<literal>t</literal>. That contradicts the idea that +<literal>newtype</literal> should have no concrete representation. +You can get just the same efficiency and effect by using +<literal>data</literal> instead of <literal>newtype</literal>. If +there is no overloading involved, then there is more of a case for +allowing an existentially-quantified <literal>newtype</literal>, +because the <literal>data</literal> version does carry an +implementation cost, but single-field existentially quantified +constructors aren't much use. So the simple restriction (no +existential stuff on <literal>newtype</literal>) stands, unless there +are convincing reasons to change it. + + +</para> +</listitem> +<listitem> + +<para> + You can't use <literal>deriving</literal> to define instances of a +data type with existentially quantified data constructors. + +Reason: in most cases it would not make sense. For example:# + +<programlisting> +data T = forall a. MkT [a] deriving( Eq ) +</programlisting> + +To derive <literal>Eq</literal> in the standard way we would need to have equality +between the single component of two <function>MkT</function> constructors: + +<programlisting> +instance Eq T where + (MkT a) == (MkT b) = ??? +</programlisting> + +But <varname>a</varname> and <varname>b</varname> have distinct types, and so can't be compared. +It's just about possible to imagine examples in which the derived instance +would make sense, but it seems altogether simpler simply to prohibit such +declarations. Define your own instances! +</para> +</listitem> + +</itemizedlist> + +</para> + +</sect4> +</sect3> + +</sect2> + + + +<sect2 id="multi-param-type-classes"> +<title>Class declarations</title> + +<para> +This section, and the next one, documents GHC's type-class extensions. +There's lots of background in the paper <ulink +url="http://research.microsoft.com/~simonpj/Papers/type-class-design-space" >Type +classes: exploring the design space</ulink > (Simon Peyton Jones, Mark +Jones, Erik Meijer). +</para> +<para> +All the extensions are enabled by the <option>-fglasgow-exts</option> flag. +</para> + +<sect3> +<title>Multi-parameter type classes</title> +<para> +Multi-parameter type classes are permitted. For example: + + +<programlisting> + class Collection c a where + union :: c a -> c a -> c a + ...etc. +</programlisting> + +</para> +</sect3> + +<sect3> +<title>The superclasses of a class declaration</title> + +<para> +There are no restrictions on the context in a class declaration +(which introduces superclasses), except that the class hierarchy must +be acyclic. So these class declarations are OK: + + +<programlisting> + class Functor (m k) => FiniteMap m k where + ... + + class (Monad m, Monad (t m)) => Transform t m where + lift :: m a -> (t m) a +</programlisting> + + +</para> +<para> +As in Haskell 98, The class hierarchy must be acyclic. However, the definition +of "acyclic" involves only the superclass relationships. For example, +this is OK: + + +<programlisting> + class C a where { + op :: D b => a -> b -> b + } + + class C a => D a where { ... } +</programlisting> + + +Here, <literal>C</literal> is a superclass of <literal>D</literal>, but it's OK for a +class operation <literal>op</literal> of <literal>C</literal> to mention <literal>D</literal>. (It +would not be OK for <literal>D</literal> to be a superclass of <literal>C</literal>.) +</para> +</sect3> + + + + +<sect3 id="class-method-types"> +<title>Class method types</title> + +<para> +Haskell 98 prohibits class method types to mention constraints on the +class type variable, thus: +<programlisting> + class Seq s a where + fromList :: [a] -> s a + elem :: Eq a => a -> s a -> Bool +</programlisting> +The type of <literal>elem</literal> is illegal in Haskell 98, because it +contains the constraint <literal>Eq a</literal>, constrains only the +class type variable (in this case <literal>a</literal>). +GHC lifts this restriction. +</para> + + +</sect3> +</sect2> + +<sect2 id="functional-dependencies"> +<title>Functional dependencies +</title> + +<para> Functional dependencies are implemented as described by Mark Jones +in “<ulink url="http://www.cse.ogi.edu/~mpj/pubs/fundeps.html">Type Classes with Functional Dependencies</ulink>”, Mark P. Jones, +In Proceedings of the 9th European Symposium on Programming, +ESOP 2000, Berlin, Germany, March 2000, Springer-Verlag LNCS 1782, +. +</para> +<para> +Functional dependencies are introduced by a vertical bar in the syntax of a +class declaration; e.g. +<programlisting> + class (Monad m) => MonadState s m | m -> s where ... + + class Foo a b c | a b -> c where ... +</programlisting> +There should be more documentation, but there isn't (yet). Yell if you need it. +</para> + +<sect3><title>Rules for functional dependencies </title> +<para> +In a class declaration, all of the class type variables must be reachable (in the sense +mentioned in <xref linkend="type-restrictions"/>) +from the free variables of each method type. +For example: + +<programlisting> + class Coll s a where + empty :: s + insert :: s -> a -> s +</programlisting> + +is not OK, because the type of <literal>empty</literal> doesn't mention +<literal>a</literal>. Functional dependencies can make the type variable +reachable: +<programlisting> + class Coll s a | s -> a where + empty :: s + insert :: s -> a -> s +</programlisting> + +Alternatively <literal>Coll</literal> might be rewritten + +<programlisting> + class Coll s a where + empty :: s a + insert :: s a -> a -> s a +</programlisting> + + +which makes the connection between the type of a collection of +<literal>a</literal>'s (namely <literal>(s a)</literal>) and the element type <literal>a</literal>. +Occasionally this really doesn't work, in which case you can split the +class like this: + + +<programlisting> + class CollE s where + empty :: s + + class CollE s => Coll s a where + insert :: s -> a -> s +</programlisting> +</para> +</sect3> + + +<sect3> +<title>Background on functional dependencies</title> + +<para>The following description of the motivation and use of functional dependencies is taken +from the Hugs user manual, reproduced here (with minor changes) by kind +permission of Mark Jones. +</para> +<para> +Consider the following class, intended as part of a +library for collection types: +<programlisting> + class Collects e ce where + empty :: ce + insert :: e -> ce -> ce + member :: e -> ce -> Bool +</programlisting> +The type variable e used here represents the element type, while ce is the type +of the container itself. Within this framework, we might want to define +instances of this class for lists or characteristic functions (both of which +can be used to represent collections of any equality type), bit sets (which can +be used to represent collections of characters), or hash tables (which can be +used to represent any collection whose elements have a hash function). Omitting +standard implementation details, this would lead to the following declarations: +<programlisting> + instance Eq e => Collects e [e] where ... + instance Eq e => Collects e (e -> Bool) where ... + instance Collects Char BitSet where ... + instance (Hashable e, Collects a ce) + => Collects e (Array Int ce) where ... +</programlisting> +All this looks quite promising; we have a class and a range of interesting +implementations. Unfortunately, there are some serious problems with the class +declaration. First, the empty function has an ambiguous type: +<programlisting> + empty :: Collects e ce => ce +</programlisting> +By "ambiguous" we mean that there is a type variable e that appears on the left +of the <literal>=></literal> symbol, but not on the right. The problem with +this is that, according to the theoretical foundations of Haskell overloading, +we cannot guarantee a well-defined semantics for any term with an ambiguous +type. +</para> +<para> +We can sidestep this specific problem by removing the empty member from the +class declaration. However, although the remaining members, insert and member, +do not have ambiguous types, we still run into problems when we try to use +them. For example, consider the following two functions: +<programlisting> + f x y = insert x . insert y + g = f True 'a' +</programlisting> +for which GHC infers the following types: +<programlisting> + f :: (Collects a c, Collects b c) => a -> b -> c -> c + g :: (Collects Bool c, Collects Char c) => c -> c +</programlisting> +Notice that the type for f allows the two parameters x and y to be assigned +different types, even though it attempts to insert each of the two values, one +after the other, into the same collection. If we're trying to model collections +that contain only one type of value, then this is clearly an inaccurate +type. Worse still, the definition for g is accepted, without causing a type +error. As a result, the error in this code will not be flagged at the point +where it appears. Instead, it will show up only when we try to use g, which +might even be in a different module. +</para> + +<sect4><title>An attempt to use constructor classes</title> + +<para> +Faced with the problems described above, some Haskell programmers might be +tempted to use something like the following version of the class declaration: +<programlisting> + class Collects e c where + empty :: c e + insert :: e -> c e -> c e + member :: e -> c e -> Bool +</programlisting> +The key difference here is that we abstract over the type constructor c that is +used to form the collection type c e, and not over that collection type itself, +represented by ce in the original class declaration. This avoids the immediate +problems that we mentioned above: empty has type <literal>Collects e c => c +e</literal>, which is not ambiguous. +</para> +<para> +The function f from the previous section has a more accurate type: +<programlisting> + f :: (Collects e c) => e -> e -> c e -> c e +</programlisting> +The function g from the previous section is now rejected with a type error as +we would hope because the type of f does not allow the two arguments to have +different types. +This, then, is an example of a multiple parameter class that does actually work +quite well in practice, without ambiguity problems. +There is, however, a catch. This version of the Collects class is nowhere near +as general as the original class seemed to be: only one of the four instances +for <literal>Collects</literal> +given above can be used with this version of Collects because only one of +them---the instance for lists---has a collection type that can be written in +the form c e, for some type constructor c, and element type e. +</para> +</sect4> + +<sect4><title>Adding functional dependencies</title> + +<para> +To get a more useful version of the Collects class, Hugs provides a mechanism +that allows programmers to specify dependencies between the parameters of a +multiple parameter class (For readers with an interest in theoretical +foundations and previous work: The use of dependency information can be seen +both as a generalization of the proposal for `parametric type classes' that was +put forward by Chen, Hudak, and Odersky, or as a special case of Mark Jones's +later framework for "improvement" of qualified types. The +underlying ideas are also discussed in a more theoretical and abstract setting +in a manuscript [implparam], where they are identified as one point in a +general design space for systems of implicit parameterization.). + +To start with an abstract example, consider a declaration such as: +<programlisting> + class C a b where ... +</programlisting> +which tells us simply that C can be thought of as a binary relation on types +(or type constructors, depending on the kinds of a and b). Extra clauses can be +included in the definition of classes to add information about dependencies +between parameters, as in the following examples: +<programlisting> + class D a b | a -> b where ... + class E a b | a -> b, b -> a where ... +</programlisting> +The notation <literal>a -> b</literal> used here between the | and where +symbols --- not to be +confused with a function type --- indicates that the a parameter uniquely +determines the b parameter, and might be read as "a determines b." Thus D is +not just a relation, but actually a (partial) function. Similarly, from the two +dependencies that are included in the definition of E, we can see that E +represents a (partial) one-one mapping between types. +</para> +<para> +More generally, dependencies take the form <literal>x1 ... xn -> y1 ... ym</literal>, +where x1, ..., xn, and y1, ..., yn are type variables with n>0 and +m>=0, meaning that the y parameters are uniquely determined by the x +parameters. Spaces can be used as separators if more than one variable appears +on any single side of a dependency, as in <literal>t -> a b</literal>. Note that a class may be +annotated with multiple dependencies using commas as separators, as in the +definition of E above. Some dependencies that we can write in this notation are +redundant, and will be rejected because they don't serve any useful +purpose, and may instead indicate an error in the program. Examples of +dependencies like this include <literal>a -> a </literal>, +<literal>a -> a a </literal>, +<literal>a -> </literal>, etc. There can also be +some redundancy if multiple dependencies are given, as in +<literal>a->b</literal>, + <literal>b->c </literal>, <literal>a->c </literal>, and +in which some subset implies the remaining dependencies. Examples like this are +not treated as errors. Note that dependencies appear only in class +declarations, and not in any other part of the language. In particular, the +syntax for instance declarations, class constraints, and types is completely +unchanged. +</para> +<para> +By including dependencies in a class declaration, we provide a mechanism for +the programmer to specify each multiple parameter class more precisely. The +compiler, on the other hand, is responsible for ensuring that the set of +instances that are in scope at any given point in the program is consistent +with any declared dependencies. For example, the following pair of instance +declarations cannot appear together in the same scope because they violate the +dependency for D, even though either one on its own would be acceptable: +<programlisting> + instance D Bool Int where ... + instance D Bool Char where ... +</programlisting> +Note also that the following declaration is not allowed, even by itself: +<programlisting> + instance D [a] b where ... +</programlisting> +The problem here is that this instance would allow one particular choice of [a] +to be associated with more than one choice for b, which contradicts the +dependency specified in the definition of D. More generally, this means that, +in any instance of the form: +<programlisting> + instance D t s where ... +</programlisting> +for some particular types t and s, the only variables that can appear in s are +the ones that appear in t, and hence, if the type t is known, then s will be +uniquely determined. +</para> +<para> +The benefit of including dependency information is that it allows us to define +more general multiple parameter classes, without ambiguity problems, and with +the benefit of more accurate types. To illustrate this, we return to the +collection class example, and annotate the original definition of <literal>Collects</literal> +with a simple dependency: +<programlisting> + class Collects e ce | ce -> e where + empty :: ce + insert :: e -> ce -> ce + member :: e -> ce -> Bool +</programlisting> +The dependency <literal>ce -> e</literal> here specifies that the type e of elements is uniquely +determined by the type of the collection ce. Note that both parameters of +Collects are of kind *; there are no constructor classes here. Note too that +all of the instances of Collects that we gave earlier can be used +together with this new definition. +</para> +<para> +What about the ambiguity problems that we encountered with the original +definition? The empty function still has type Collects e ce => ce, but it is no +longer necessary to regard that as an ambiguous type: Although the variable e +does not appear on the right of the => symbol, the dependency for class +Collects tells us that it is uniquely determined by ce, which does appear on +the right of the => symbol. Hence the context in which empty is used can still +give enough information to determine types for both ce and e, without +ambiguity. More generally, we need only regard a type as ambiguous if it +contains a variable on the left of the => that is not uniquely determined +(either directly or indirectly) by the variables on the right. +</para> +<para> +Dependencies also help to produce more accurate types for user defined +functions, and hence to provide earlier detection of errors, and less cluttered +types for programmers to work with. Recall the previous definition for a +function f: +<programlisting> + f x y = insert x y = insert x . insert y +</programlisting> +for which we originally obtained a type: +<programlisting> + f :: (Collects a c, Collects b c) => a -> b -> c -> c +</programlisting> +Given the dependency information that we have for Collects, however, we can +deduce that a and b must be equal because they both appear as the second +parameter in a Collects constraint with the same first parameter c. Hence we +can infer a shorter and more accurate type for f: +<programlisting> + f :: (Collects a c) => a -> a -> c -> c +</programlisting> +In a similar way, the earlier definition of g will now be flagged as a type error. +</para> +<para> +Although we have given only a few examples here, it should be clear that the +addition of dependency information can help to make multiple parameter classes +more useful in practice, avoiding ambiguity problems, and allowing more general +sets of instance declarations. +</para> +</sect4> +</sect3> +</sect2> + +<sect2 id="instance-decls"> +<title>Instance declarations</title> + +<sect3 id="instance-rules"> +<title>Relaxed rules for instance declarations</title> + +<para>An instance declaration has the form +<screen> + instance ( <replaceable>assertion</replaceable><subscript>1</subscript>, ..., <replaceable>assertion</replaceable><subscript>n</subscript>) => <replaceable>class</replaceable> <replaceable>type</replaceable><subscript>1</subscript> ... <replaceable>type</replaceable><subscript>m</subscript> where ... +</screen> +The part before the "<literal>=></literal>" is the +<emphasis>context</emphasis>, while the part after the +"<literal>=></literal>" is the <emphasis>head</emphasis> of the instance declaration. +</para> + +<para> +In Haskell 98 the head of an instance declaration +must be of the form <literal>C (T a1 ... an)</literal>, where +<literal>C</literal> is the class, <literal>T</literal> is a type constructor, +and the <literal>a1 ... an</literal> are distinct type variables. +Furthermore, the assertions in the context of the instance declaration +must be of the form <literal>C a</literal> where <literal>a</literal> +is a type variable that occurs in the head. +</para> +<para> +The <option>-fglasgow-exts</option> flag loosens these restrictions +considerably. Firstly, multi-parameter type classes are permitted. Secondly, +the context and head of the instance declaration can each consist of arbitrary +(well-kinded) assertions <literal>(C t1 ... tn)</literal> subject only to the +following rules: +<orderedlist> +<listitem><para> +For each assertion in the context: +<orderedlist> +<listitem><para>No type variable has more occurrences in the assertion than in the head</para></listitem> +<listitem><para>The assertion has fewer constructors and variables (taken together + and counting repetitions) than the head</para></listitem> +</orderedlist> +</para></listitem> + +<listitem><para>The coverage condition. For each functional dependency, +<replaceable>tvs</replaceable><subscript>left</subscript> <literal>-></literal> +<replaceable>tvs</replaceable><subscript>right</subscript>, of the class, +every type variable in +S(<replaceable>tvs</replaceable><subscript>right</subscript>) must appear in +S(<replaceable>tvs</replaceable><subscript>left</subscript>), where S is the +substitution mapping each type variable in the class declaration to the +corresponding type in the instance declaration. +</para></listitem> +</orderedlist> +These restrictions ensure that context reduction terminates: each reduction +step makes the problem smaller by at least one +constructor. For example, the following would make the type checker +loop if it wasn't excluded: +<programlisting> + instance C a => C a where ... +</programlisting> +For example, these are OK: +<programlisting> + instance C Int [a] -- Multiple parameters + instance Eq (S [a]) -- Structured type in head + + -- Repeated type variable in head + instance C4 a a => C4 [a] [a] + instance Stateful (ST s) (MutVar s) + + -- Head can consist of type variables only + instance C a + instance (Eq a, Show b) => C2 a b + + -- Non-type variables in context + instance Show (s a) => Show (Sized s a) + instance C2 Int a => C3 Bool [a] + instance C2 Int a => C3 [a] b +</programlisting> +But these are not: +<programlisting> + -- Context assertion no smaller than head + instance C a => C a where ... + -- (C b b) has more more occurrences of b than the head + instance C b b => Foo [b] where ... +</programlisting> +</para> + +<para> +The same restrictions apply to instances generated by +<literal>deriving</literal> clauses. Thus the following is accepted: +<programlisting> + data MinHeap h a = H a (h a) + deriving (Show) +</programlisting> +because the derived instance +<programlisting> + instance (Show a, Show (h a)) => Show (MinHeap h a) +</programlisting> +conforms to the above rules. +</para> + +<para> +A useful idiom permitted by the above rules is as follows. +If one allows overlapping instance declarations then it's quite +convenient to have a "default instance" declaration that applies if +something more specific does not: +<programlisting> + instance C a where + op = ... -- Default +</programlisting> +</para> +</sect3> + +<sect3 id="undecidable-instances"> +<title>Undecidable instances</title> + +<para> +Sometimes even the rules of <xref linkend="instance-rules"/> are too onerous. +For example, sometimes you might want to use the following to get the +effect of a "class synonym": +<programlisting> + class (C1 a, C2 a, C3 a) => C a where { } + + instance (C1 a, C2 a, C3 a) => C a where { } +</programlisting> +This allows you to write shorter signatures: +<programlisting> + f :: C a => ... +</programlisting> +instead of +<programlisting> + f :: (C1 a, C2 a, C3 a) => ... +</programlisting> +The restrictions on functional dependencies (<xref +linkend="functional-dependencies"/>) are particularly troublesome. +It is tempting to introduce type variables in the context that do not appear in +the head, something that is excluded by the normal rules. For example: +<programlisting> + class HasConverter a b | a -> b where + convert :: a -> b + + data Foo a = MkFoo a + + instance (HasConverter a b,Show b) => Show (Foo a) where + show (MkFoo value) = show (convert value) +</programlisting> +This is dangerous territory, however. Here, for example, is a program that would make the +typechecker loop: +<programlisting> + class D a + class F a b | a->b + instance F [a] [[a]] + instance (D c, F a c) => D [a] -- 'c' is not mentioned in the head +</programlisting> +Similarly, it can be tempting to lift the coverage condition: +<programlisting> + class Mul a b c | a b -> c where + (.*.) :: a -> b -> c + + instance Mul Int Int Int where (.*.) = (*) + instance Mul Int Float Float where x .*. y = fromIntegral x * y + instance Mul a b c => Mul a [b] [c] where x .*. v = map (x.*.) v +</programlisting> +The third instance declaration does not obey the coverage condition; +and indeed the (somewhat strange) definition: +<programlisting> + f = \ b x y -> if b then x .*. [y] else y +</programlisting> +makes instance inference go into a loop, because it requires the constraint +<literal>(Mul a [b] b)</literal>. +</para> +<para> +Nevertheless, GHC allows you to experiment with more liberal rules. If you use +the experimental flag <option>-fallow-undecidable-instances</option> +<indexterm><primary>-fallow-undecidable-instances +option</primary></indexterm>, you can use arbitrary +types in both an instance context and instance head. Termination is ensured by having a +fixed-depth recursion stack. If you exceed the stack depth you get a +sort of backtrace, and the opportunity to increase the stack depth +with <option>-fcontext-stack</option><emphasis>N</emphasis>. +</para> + +</sect3> + + +<sect3 id="instance-overlap"> +<title>Overlapping instances</title> +<para> +In general, <emphasis>GHC requires that that it be unambiguous which instance +declaration +should be used to resolve a type-class constraint</emphasis>. This behaviour +can be modified by two flags: <option>-fallow-overlapping-instances</option> +<indexterm><primary>-fallow-overlapping-instances +</primary></indexterm> +and <option>-fallow-incoherent-instances</option> +<indexterm><primary>-fallow-incoherent-instances +</primary></indexterm>, as this section discusses.</para> +<para> +When GHC tries to resolve, say, the constraint <literal>C Int Bool</literal>, +it tries to match every instance declaration against the +constraint, +by instantiating the head of the instance declaration. For example, consider +these declarations: +<programlisting> + instance context1 => C Int a where ... -- (A) + instance context2 => C a Bool where ... -- (B) + instance context3 => C Int [a] where ... -- (C) + instance context4 => C Int [Int] where ... -- (D) +</programlisting> +The instances (A) and (B) match the constraint <literal>C Int Bool</literal>, +but (C) and (D) do not. When matching, GHC takes +no account of the context of the instance declaration +(<literal>context1</literal> etc). +GHC's default behaviour is that <emphasis>exactly one instance must match the +constraint it is trying to resolve</emphasis>. +It is fine for there to be a <emphasis>potential</emphasis> of overlap (by +including both declarations (A) and (B), say); an error is only reported if a +particular constraint matches more than one. +</para> + +<para> +The <option>-fallow-overlapping-instances</option> flag instructs GHC to allow +more than one instance to match, provided there is a most specific one. For +example, the constraint <literal>C Int [Int]</literal> matches instances (A), +(C) and (D), but the last is more specific, and hence is chosen. If there is no +most-specific match, the program is rejected. +</para> +<para> +However, GHC is conservative about committing to an overlapping instance. For example: +<programlisting> + f :: [b] -> [b] + f x = ... +</programlisting> +Suppose that from the RHS of <literal>f</literal> we get the constraint +<literal>C Int [b]</literal>. But +GHC does not commit to instance (C), because in a particular +call of <literal>f</literal>, <literal>b</literal> might be instantiate +to <literal>Int</literal>, in which case instance (D) would be more specific still. +So GHC rejects the program. If you add the flag <option>-fallow-incoherent-instances</option>, +GHC will instead pick (C), without complaining about +the problem of subsequent instantiations. +</para> +<para> +The willingness to be overlapped or incoherent is a property of +the <emphasis>instance declaration</emphasis> itself, controlled by the +presence or otherwise of the <option>-fallow-overlapping-instances</option> +and <option>-fallow-incoherent-instances</option> flags when that mdodule is +being defined. Neither flag is required in a module that imports and uses the +instance declaration. Specifically, during the lookup process: +<itemizedlist> +<listitem><para> +An instance declaration is ignored during the lookup process if (a) a more specific +match is found, and (b) the instance declaration was compiled with +<option>-fallow-overlapping-instances</option>. The flag setting for the +more-specific instance does not matter. +</para></listitem> +<listitem><para> +Suppose an instance declaration does not matche the constraint being looked up, but +does unify with it, so that it might match when the constraint is further +instantiated. Usually GHC will regard this as a reason for not committing to +some other constraint. But if the instance declaration was compiled with +<option>-fallow-incoherent-instances</option>, GHC will skip the "does-it-unify?" +check for that declaration. +</para></listitem> +</itemizedlist> +All this makes it possible for a library author to design a library that relies on +overlapping instances without the library client having to know. +</para> +<para>The <option>-fallow-incoherent-instances</option> flag implies the +<option>-fallow-overlapping-instances</option> flag, but not vice versa. +</para> +</sect3> + +<sect3> +<title>Type synonyms in the instance head</title> + +<para> +<emphasis>Unlike Haskell 98, instance heads may use type +synonyms</emphasis>. (The instance "head" is the bit after the "=>" in an instance decl.) +As always, using a type synonym is just shorthand for +writing the RHS of the type synonym definition. For example: + + +<programlisting> + type Point = (Int,Int) + instance C Point where ... + instance C [Point] where ... +</programlisting> + + +is legal. However, if you added + + +<programlisting> + instance C (Int,Int) where ... +</programlisting> + + +as well, then the compiler will complain about the overlapping +(actually, identical) instance declarations. As always, type synonyms +must be fully applied. You cannot, for example, write: + + +<programlisting> + type P a = [[a]] + instance Monad P where ... +</programlisting> + + +This design decision is independent of all the others, and easily +reversed, but it makes sense to me. + +</para> +</sect3> + + +</sect2> + +<sect2 id="type-restrictions"> +<title>Type signatures</title> + +<sect3><title>The context of a type signature</title> +<para> +Unlike Haskell 98, constraints in types do <emphasis>not</emphasis> have to be of +the form <emphasis>(class type-variable)</emphasis> or +<emphasis>(class (type-variable type-variable ...))</emphasis>. Thus, +these type signatures are perfectly OK +<programlisting> + g :: Eq [a] => ... + g :: Ord (T a ()) => ... +</programlisting> +</para> +<para> +GHC imposes the following restrictions on the constraints in a type signature. +Consider the type: + +<programlisting> + forall tv1..tvn (c1, ...,cn) => type +</programlisting> + +(Here, we write the "foralls" explicitly, although the Haskell source +language omits them; in Haskell 98, all the free type variables of an +explicit source-language type signature are universally quantified, +except for the class type variables in a class declaration. However, +in GHC, you can give the foralls if you want. See <xref linkend="universal-quantification"/>). +</para> + +<para> + +<orderedlist> +<listitem> + +<para> + <emphasis>Each universally quantified type variable +<literal>tvi</literal> must be reachable from <literal>type</literal></emphasis>. + +A type variable <literal>a</literal> is "reachable" if it it appears +in the same constraint as either a type variable free in in +<literal>type</literal>, or another reachable type variable. +A value with a type that does not obey +this reachability restriction cannot be used without introducing +ambiguity; that is why the type is rejected. +Here, for example, is an illegal type: + + +<programlisting> + forall a. Eq a => Int +</programlisting> + + +When a value with this type was used, the constraint <literal>Eq tv</literal> +would be introduced where <literal>tv</literal> is a fresh type variable, and +(in the dictionary-translation implementation) the value would be +applied to a dictionary for <literal>Eq tv</literal>. The difficulty is that we +can never know which instance of <literal>Eq</literal> to use because we never +get any more information about <literal>tv</literal>. +</para> +<para> +Note +that the reachability condition is weaker than saying that <literal>a</literal> is +functionally dependent on a type variable free in +<literal>type</literal> (see <xref +linkend="functional-dependencies"/>). The reason for this is there +might be a "hidden" dependency, in a superclass perhaps. So +"reachable" is a conservative approximation to "functionally dependent". +For example, consider: +<programlisting> + class C a b | a -> b where ... + class C a b => D a b where ... + f :: forall a b. D a b => a -> a +</programlisting> +This is fine, because in fact <literal>a</literal> does functionally determine <literal>b</literal> +but that is not immediately apparent from <literal>f</literal>'s type. +</para> +</listitem> +<listitem> + +<para> + <emphasis>Every constraint <literal>ci</literal> must mention at least one of the +universally quantified type variables <literal>tvi</literal></emphasis>. + +For example, this type is OK because <literal>C a b</literal> mentions the +universally quantified type variable <literal>b</literal>: + + +<programlisting> + forall a. C a b => burble +</programlisting> + + +The next type is illegal because the constraint <literal>Eq b</literal> does not +mention <literal>a</literal>: + + +<programlisting> + forall a. Eq b => burble +</programlisting> + + +The reason for this restriction is milder than the other one. The +excluded types are never useful or necessary (because the offending +context doesn't need to be witnessed at this point; it can be floated +out). Furthermore, floating them out increases sharing. Lastly, +excluding them is a conservative choice; it leaves a patch of +territory free in case we need it later. + +</para> +</listitem> + +</orderedlist> + +</para> +</sect3> + +<sect3 id="hoist"> +<title>For-all hoisting</title> +<para> +It is often convenient to use generalised type synonyms (see <xref linkend="type-synonyms"/>) at the right hand +end of an arrow, thus: +<programlisting> + type Discard a = forall b. a -> b -> a + + g :: Int -> Discard Int + g x y z = x+y +</programlisting> +Simply expanding the type synonym would give +<programlisting> + g :: Int -> (forall b. Int -> b -> Int) +</programlisting> +but GHC "hoists" the <literal>forall</literal> to give the isomorphic type +<programlisting> + g :: forall b. Int -> Int -> b -> Int +</programlisting> +In general, the rule is this: <emphasis>to determine the type specified by any explicit +user-written type (e.g. in a type signature), GHC expands type synonyms and then repeatedly +performs the transformation:</emphasis> +<programlisting> + <emphasis>type1</emphasis> -> forall a1..an. <emphasis>context2</emphasis> => <emphasis>type2</emphasis> +==> + forall a1..an. <emphasis>context2</emphasis> => <emphasis>type1</emphasis> -> <emphasis>type2</emphasis> +</programlisting> +(In fact, GHC tries to retain as much synonym information as possible for use in +error messages, but that is a usability issue.) This rule applies, of course, whether +or not the <literal>forall</literal> comes from a synonym. For example, here is another +valid way to write <literal>g</literal>'s type signature: +<programlisting> + g :: Int -> Int -> forall b. b -> Int +</programlisting> +</para> +<para> +When doing this hoisting operation, GHC eliminates duplicate constraints. For +example: +<programlisting> + type Foo a = (?x::Int) => Bool -> a + g :: Foo (Foo Int) +</programlisting> +means +<programlisting> + g :: (?x::Int) => Bool -> Bool -> Int +</programlisting> +</para> +</sect3> + + +</sect2> + +<sect2 id="implicit-parameters"> +<title>Implicit parameters</title> + +<para> Implicit parameters are implemented as described in +"Implicit parameters: dynamic scoping with static types", +J Lewis, MB Shields, E Meijer, J Launchbury, +27th ACM Symposium on Principles of Programming Languages (POPL'00), +Boston, Jan 2000. +</para> + +<para>(Most of the following, stil rather incomplete, documentation is +due to Jeff Lewis.)</para> + +<para>Implicit parameter support is enabled with the option +<option>-fimplicit-params</option>.</para> + +<para> +A variable is called <emphasis>dynamically bound</emphasis> when it is bound by the calling +context of a function and <emphasis>statically bound</emphasis> when bound by the callee's +context. In Haskell, all variables are statically bound. Dynamic +binding of variables is a notion that goes back to Lisp, but was later +discarded in more modern incarnations, such as Scheme. Dynamic binding +can be very confusing in an untyped language, and unfortunately, typed +languages, in particular Hindley-Milner typed languages like Haskell, +only support static scoping of variables. +</para> +<para> +However, by a simple extension to the type class system of Haskell, we +can support dynamic binding. Basically, we express the use of a +dynamically bound variable as a constraint on the type. These +constraints lead to types of the form <literal>(?x::t') => t</literal>, which says "this +function uses a dynamically-bound variable <literal>?x</literal> +of type <literal>t'</literal>". For +example, the following expresses the type of a sort function, +implicitly parameterized by a comparison function named <literal>cmp</literal>. +<programlisting> + sort :: (?cmp :: a -> a -> Bool) => [a] -> [a] +</programlisting> +The dynamic binding constraints are just a new form of predicate in the type class system. +</para> +<para> +An implicit parameter occurs in an expression using the special form <literal>?x</literal>, +where <literal>x</literal> is +any valid identifier (e.g. <literal>ord ?x</literal> is a valid expression). +Use of this construct also introduces a new +dynamic-binding constraint in the type of the expression. +For example, the following definition +shows how we can define an implicitly parameterized sort function in +terms of an explicitly parameterized <literal>sortBy</literal> function: +<programlisting> + sortBy :: (a -> a -> Bool) -> [a] -> [a] + + sort :: (?cmp :: a -> a -> Bool) => [a] -> [a] + sort = sortBy ?cmp +</programlisting> +</para> + +<sect3> +<title>Implicit-parameter type constraints</title> +<para> +Dynamic binding constraints behave just like other type class +constraints in that they are automatically propagated. Thus, when a +function is used, its implicit parameters are inherited by the +function that called it. For example, our <literal>sort</literal> function might be used +to pick out the least value in a list: +<programlisting> + least :: (?cmp :: a -> a -> Bool) => [a] -> a + least xs = fst (sort xs) +</programlisting> +Without lifting a finger, the <literal>?cmp</literal> parameter is +propagated to become a parameter of <literal>least</literal> as well. With explicit +parameters, the default is that parameters must always be explicit +propagated. With implicit parameters, the default is to always +propagate them. +</para> +<para> +An implicit-parameter type constraint differs from other type class constraints in the +following way: All uses of a particular implicit parameter must have +the same type. This means that the type of <literal>(?x, ?x)</literal> +is <literal>(?x::a) => (a,a)</literal>, and not +<literal>(?x::a, ?x::b) => (a, b)</literal>, as would be the case for type +class constraints. +</para> + +<para> You can't have an implicit parameter in the context of a class or instance +declaration. For example, both these declarations are illegal: +<programlisting> + class (?x::Int) => C a where ... + instance (?x::a) => Foo [a] where ... +</programlisting> +Reason: exactly which implicit parameter you pick up depends on exactly where +you invoke a function. But the ``invocation'' of instance declarations is done +behind the scenes by the compiler, so it's hard to figure out exactly where it is done. +Easiest thing is to outlaw the offending types.</para> +<para> +Implicit-parameter constraints do not cause ambiguity. For example, consider: +<programlisting> + f :: (?x :: [a]) => Int -> Int + f n = n + length ?x + + g :: (Read a, Show a) => String -> String + g s = show (read s) +</programlisting> +Here, <literal>g</literal> has an ambiguous type, and is rejected, but <literal>f</literal> +is fine. The binding for <literal>?x</literal> at <literal>f</literal>'s call site is +quite unambiguous, and fixes the type <literal>a</literal>. +</para> +</sect3> + +<sect3> +<title>Implicit-parameter bindings</title> + +<para> +An implicit parameter is <emphasis>bound</emphasis> using the standard +<literal>let</literal> or <literal>where</literal> binding forms. +For example, we define the <literal>min</literal> function by binding +<literal>cmp</literal>. +<programlisting> + min :: [a] -> a + min = let ?cmp = (<=) in least +</programlisting> +</para> +<para> +A group of implicit-parameter bindings may occur anywhere a normal group of Haskell +bindings can occur, except at top level. That is, they can occur in a <literal>let</literal> +(including in a list comprehension, or do-notation, or pattern guards), +or a <literal>where</literal> clause. +Note the following points: +<itemizedlist> +<listitem><para> +An implicit-parameter binding group must be a +collection of simple bindings to implicit-style variables (no +function-style bindings, and no type signatures); these bindings are +neither polymorphic or recursive. +</para></listitem> +<listitem><para> +You may not mix implicit-parameter bindings with ordinary bindings in a +single <literal>let</literal> +expression; use two nested <literal>let</literal>s instead. +(In the case of <literal>where</literal> you are stuck, since you can't nest <literal>where</literal> clauses.) +</para></listitem> + +<listitem><para> +You may put multiple implicit-parameter bindings in a +single binding group; but they are <emphasis>not</emphasis> treated +as a mutually recursive group (as ordinary <literal>let</literal> bindings are). +Instead they are treated as a non-recursive group, simultaneously binding all the implicit +parameter. The bindings are not nested, and may be re-ordered without changing +the meaning of the program. +For example, consider: +<programlisting> + f t = let { ?x = t; ?y = ?x+(1::Int) } in ?x + ?y +</programlisting> +The use of <literal>?x</literal> in the binding for <literal>?y</literal> does not "see" +the binding for <literal>?x</literal>, so the type of <literal>f</literal> is +<programlisting> + f :: (?x::Int) => Int -> Int +</programlisting> +</para></listitem> +</itemizedlist> +</para> + +</sect3> + +<sect3><title>Implicit parameters and polymorphic recursion</title> + +<para> +Consider these two definitions: +<programlisting> + len1 :: [a] -> Int + len1 xs = let ?acc = 0 in len_acc1 xs + + len_acc1 [] = ?acc + len_acc1 (x:xs) = let ?acc = ?acc + (1::Int) in len_acc1 xs + + ------------ + + len2 :: [a] -> Int + len2 xs = let ?acc = 0 in len_acc2 xs + + len_acc2 :: (?acc :: Int) => [a] -> Int + len_acc2 [] = ?acc + len_acc2 (x:xs) = let ?acc = ?acc + (1::Int) in len_acc2 xs +</programlisting> +The only difference between the two groups is that in the second group +<literal>len_acc</literal> is given a type signature. +In the former case, <literal>len_acc1</literal> is monomorphic in its own +right-hand side, so the implicit parameter <literal>?acc</literal> is not +passed to the recursive call. In the latter case, because <literal>len_acc2</literal> +has a type signature, the recursive call is made to the +<emphasis>polymoprhic</emphasis> version, which takes <literal>?acc</literal> +as an implicit parameter. So we get the following results in GHCi: +<programlisting> + Prog> len1 "hello" + 0 + Prog> len2 "hello" + 5 +</programlisting> +Adding a type signature dramatically changes the result! This is a rather +counter-intuitive phenomenon, worth watching out for. +</para> +</sect3> + +<sect3><title>Implicit parameters and monomorphism</title> + +<para>GHC applies the dreaded Monomorphism Restriction (section 4.5.5 of the +Haskell Report) to implicit parameters. For example, consider: +<programlisting> + f :: Int -> Int + f v = let ?x = 0 in + let y = ?x + v in + let ?x = 5 in + y +</programlisting> +Since the binding for <literal>y</literal> falls under the Monomorphism +Restriction it is not generalised, so the type of <literal>y</literal> is +simply <literal>Int</literal>, not <literal>(?x::Int) => Int</literal>. +Hence, <literal>(f 9)</literal> returns result <literal>9</literal>. +If you add a type signature for <literal>y</literal>, then <literal>y</literal> +will get type <literal>(?x::Int) => Int</literal>, so the occurrence of +<literal>y</literal> in the body of the <literal>let</literal> will see the +inner binding of <literal>?x</literal>, so <literal>(f 9)</literal> will return +<literal>14</literal>. +</para> +</sect3> +</sect2> + +<sect2 id="linear-implicit-parameters"> +<title>Linear implicit parameters</title> +<para> +Linear implicit parameters are an idea developed by Koen Claessen, +Mark Shields, and Simon PJ. They address the long-standing +problem that monads seem over-kill for certain sorts of problem, notably: +</para> +<itemizedlist> +<listitem> <para> distributing a supply of unique names </para> </listitem> +<listitem> <para> distributing a supply of random numbers </para> </listitem> +<listitem> <para> distributing an oracle (as in QuickCheck) </para> </listitem> +</itemizedlist> + +<para> +Linear implicit parameters are just like ordinary implicit parameters, +except that they are "linear" -- that is, they cannot be copied, and +must be explicitly "split" instead. Linear implicit parameters are +written '<literal>%x</literal>' instead of '<literal>?x</literal>'. +(The '/' in the '%' suggests the split!) +</para> +<para> +For example: +<programlisting> + import GHC.Exts( Splittable ) + + data NameSupply = ... + + splitNS :: NameSupply -> (NameSupply, NameSupply) + newName :: NameSupply -> Name + + instance Splittable NameSupply where + split = splitNS + + + f :: (%ns :: NameSupply) => Env -> Expr -> Expr + f env (Lam x e) = Lam x' (f env e) + where + x' = newName %ns + env' = extend env x x' + ...more equations for f... +</programlisting> +Notice that the implicit parameter %ns is consumed +<itemizedlist> +<listitem> <para> once by the call to <literal>newName</literal> </para> </listitem> +<listitem> <para> once by the recursive call to <literal>f</literal> </para></listitem> +</itemizedlist> +</para> +<para> +So the translation done by the type checker makes +the parameter explicit: +<programlisting> + f :: NameSupply -> Env -> Expr -> Expr + f ns env (Lam x e) = Lam x' (f ns1 env e) + where + (ns1,ns2) = splitNS ns + x' = newName ns2 + env = extend env x x' +</programlisting> +Notice the call to 'split' introduced by the type checker. +How did it know to use 'splitNS'? Because what it really did +was to introduce a call to the overloaded function 'split', +defined by the class <literal>Splittable</literal>: +<programlisting> + class Splittable a where + split :: a -> (a,a) +</programlisting> +The instance for <literal>Splittable NameSupply</literal> tells GHC how to implement +split for name supplies. But we can simply write +<programlisting> + g x = (x, %ns, %ns) +</programlisting> +and GHC will infer +<programlisting> + g :: (Splittable a, %ns :: a) => b -> (b,a,a) +</programlisting> +The <literal>Splittable</literal> class is built into GHC. It's exported by module +<literal>GHC.Exts</literal>. +</para> +<para> +Other points: +<itemizedlist> +<listitem> <para> '<literal>?x</literal>' and '<literal>%x</literal>' +are entirely distinct implicit parameters: you + can use them together and they won't intefere with each other. </para> +</listitem> + +<listitem> <para> You can bind linear implicit parameters in 'with' clauses. </para> </listitem> + +<listitem> <para>You cannot have implicit parameters (whether linear or not) + in the context of a class or instance declaration. </para></listitem> +</itemizedlist> +</para> + +<sect3><title>Warnings</title> + +<para> +The monomorphism restriction is even more important than usual. +Consider the example above: +<programlisting> + f :: (%ns :: NameSupply) => Env -> Expr -> Expr + f env (Lam x e) = Lam x' (f env e) + where + x' = newName %ns + env' = extend env x x' +</programlisting> +If we replaced the two occurrences of x' by (newName %ns), which is +usually a harmless thing to do, we get: +<programlisting> + f :: (%ns :: NameSupply) => Env -> Expr -> Expr + f env (Lam x e) = Lam (newName %ns) (f env e) + where + env' = extend env x (newName %ns) +</programlisting> +But now the name supply is consumed in <emphasis>three</emphasis> places +(the two calls to newName,and the recursive call to f), so +the result is utterly different. Urk! We don't even have +the beta rule. +</para> +<para> +Well, this is an experimental change. With implicit +parameters we have already lost beta reduction anyway, and +(as John Launchbury puts it) we can't sensibly reason about +Haskell programs without knowing their typing. +</para> + +</sect3> + +<sect3><title>Recursive functions</title> +<para>Linear implicit parameters can be particularly tricky when you have a recursive function +Consider +<programlisting> + foo :: %x::T => Int -> [Int] + foo 0 = [] + foo n = %x : foo (n-1) +</programlisting> +where T is some type in class Splittable.</para> +<para> +Do you get a list of all the same T's or all different T's +(assuming that split gives two distinct T's back)? +</para><para> +If you supply the type signature, taking advantage of polymorphic +recursion, you get what you'd probably expect. Here's the +translated term, where the implicit param is made explicit: +<programlisting> + foo x 0 = [] + foo x n = let (x1,x2) = split x + in x1 : foo x2 (n-1) +</programlisting> +But if you don't supply a type signature, GHC uses the Hindley +Milner trick of using a single monomorphic instance of the function +for the recursive calls. That is what makes Hindley Milner type inference +work. So the translation becomes +<programlisting> + foo x = let + foom 0 = [] + foom n = x : foom (n-1) + in + foom +</programlisting> +Result: 'x' is not split, and you get a list of identical T's. So the +semantics of the program depends on whether or not foo has a type signature. +Yikes! +</para><para> +You may say that this is a good reason to dislike linear implicit parameters +and you'd be right. That is why they are an experimental feature. +</para> +</sect3> + +</sect2> + +<sect2 id="sec-kinding"> +<title>Explicitly-kinded quantification</title> + +<para> +Haskell infers the kind of each type variable. Sometimes it is nice to be able +to give the kind explicitly as (machine-checked) documentation, +just as it is nice to give a type signature for a function. On some occasions, +it is essential to do so. For example, in his paper "Restricted Data Types in Haskell" (Haskell Workshop 1999) +John Hughes had to define the data type: +<screen> + data Set cxt a = Set [a] + | Unused (cxt a -> ()) +</screen> +The only use for the <literal>Unused</literal> constructor was to force the correct +kind for the type variable <literal>cxt</literal>. +</para> +<para> +GHC now instead allows you to specify the kind of a type variable directly, wherever +a type variable is explicitly bound. Namely: +<itemizedlist> +<listitem><para><literal>data</literal> declarations: +<screen> + data Set (cxt :: * -> *) a = Set [a] +</screen></para></listitem> +<listitem><para><literal>type</literal> declarations: +<screen> + type T (f :: * -> *) = f Int +</screen></para></listitem> +<listitem><para><literal>class</literal> declarations: +<screen> + class (Eq a) => C (f :: * -> *) a where ... +</screen></para></listitem> +<listitem><para><literal>forall</literal>'s in type signatures: +<screen> + f :: forall (cxt :: * -> *). Set cxt Int +</screen></para></listitem> +</itemizedlist> +</para> + +<para> +The parentheses are required. Some of the spaces are required too, to +separate the lexemes. If you write <literal>(f::*->*)</literal> you +will get a parse error, because "<literal>::*->*</literal>" is a +single lexeme in Haskell. +</para> + +<para> +As part of the same extension, you can put kind annotations in types +as well. Thus: +<screen> + f :: (Int :: *) -> Int + g :: forall a. a -> (a :: *) +</screen> +The syntax is +<screen> + atype ::= '(' ctype '::' kind ') +</screen> +The parentheses are required. +</para> +</sect2> + + +<sect2 id="universal-quantification"> +<title>Arbitrary-rank polymorphism +</title> + +<para> +Haskell type signatures are implicitly quantified. The new keyword <literal>forall</literal> +allows us to say exactly what this means. For example: +</para> +<para> +<programlisting> + g :: b -> b +</programlisting> +means this: +<programlisting> + g :: forall b. (b -> b) +</programlisting> +The two are treated identically. +</para> + +<para> +However, GHC's type system supports <emphasis>arbitrary-rank</emphasis> +explicit universal quantification in +types. +For example, all the following types are legal: +<programlisting> + f1 :: forall a b. a -> b -> a + g1 :: forall a b. (Ord a, Eq b) => a -> b -> a + + f2 :: (forall a. a->a) -> Int -> Int + g2 :: (forall a. Eq a => [a] -> a -> Bool) -> Int -> Int + + f3 :: ((forall a. a->a) -> Int) -> Bool -> Bool +</programlisting> +Here, <literal>f1</literal> and <literal>g1</literal> are rank-1 types, and +can be written in standard Haskell (e.g. <literal>f1 :: a->b->a</literal>). +The <literal>forall</literal> makes explicit the universal quantification that +is implicitly added by Haskell. +</para> +<para> +The functions <literal>f2</literal> and <literal>g2</literal> have rank-2 types; +the <literal>forall</literal> is on the left of a function arrow. As <literal>g2</literal> +shows, the polymorphic type on the left of the function arrow can be overloaded. +</para> +<para> +The function <literal>f3</literal> has a rank-3 type; +it has rank-2 types on the left of a function arrow. +</para> +<para> +GHC allows types of arbitrary rank; you can nest <literal>forall</literal>s +arbitrarily deep in function arrows. (GHC used to be restricted to rank 2, but +that restriction has now been lifted.) +In particular, a forall-type (also called a "type scheme"), +including an operational type class context, is legal: +<itemizedlist> +<listitem> <para> On the left of a function arrow </para> </listitem> +<listitem> <para> On the right of a function arrow (see <xref linkend="hoist"/>) </para> </listitem> +<listitem> <para> As the argument of a constructor, or type of a field, in a data type declaration. For +example, any of the <literal>f1,f2,f3,g1,g2</literal> above would be valid +field type signatures.</para> </listitem> +<listitem> <para> As the type of an implicit parameter </para> </listitem> +<listitem> <para> In a pattern type signature (see <xref linkend="scoped-type-variables"/>) </para> </listitem> +</itemizedlist> +There is one place you cannot put a <literal>forall</literal>: +you cannot instantiate a type variable with a forall-type. So you cannot +make a forall-type the argument of a type constructor. So these types are illegal: +<programlisting> + x1 :: [forall a. a->a] + x2 :: (forall a. a->a, Int) + x3 :: Maybe (forall a. a->a) +</programlisting> +Of course <literal>forall</literal> becomes a keyword; you can't use <literal>forall</literal> as +a type variable any more! +</para> + + +<sect3 id="univ"> +<title>Examples +</title> + +<para> +In a <literal>data</literal> or <literal>newtype</literal> declaration one can quantify +the types of the constructor arguments. Here are several examples: +</para> + +<para> + +<programlisting> +data T a = T1 (forall b. b -> b -> b) a + +data MonadT m = MkMonad { return :: forall a. a -> m a, + bind :: forall a b. m a -> (a -> m b) -> m b + } + +newtype Swizzle = MkSwizzle (Ord a => [a] -> [a]) +</programlisting> + +</para> + +<para> +The constructors have rank-2 types: +</para> + +<para> + +<programlisting> +T1 :: forall a. (forall b. b -> b -> b) -> a -> T a +MkMonad :: forall m. (forall a. a -> m a) + -> (forall a b. m a -> (a -> m b) -> m b) + -> MonadT m +MkSwizzle :: (Ord a => [a] -> [a]) -> Swizzle +</programlisting> + +</para> + +<para> +Notice that you don't need to use a <literal>forall</literal> if there's an +explicit context. For example in the first argument of the +constructor <function>MkSwizzle</function>, an implicit "<literal>forall a.</literal>" is +prefixed to the argument type. The implicit <literal>forall</literal> +quantifies all type variables that are not already in scope, and are +mentioned in the type quantified over. +</para> + +<para> +As for type signatures, implicit quantification happens for non-overloaded +types too. So if you write this: + +<programlisting> + data T a = MkT (Either a b) (b -> b) +</programlisting> + +it's just as if you had written this: + +<programlisting> + data T a = MkT (forall b. Either a b) (forall b. b -> b) +</programlisting> + +That is, since the type variable <literal>b</literal> isn't in scope, it's +implicitly universally quantified. (Arguably, it would be better +to <emphasis>require</emphasis> explicit quantification on constructor arguments +where that is what is wanted. Feedback welcomed.) +</para> + +<para> +You construct values of types <literal>T1, MonadT, Swizzle</literal> by applying +the constructor to suitable values, just as usual. For example, +</para> + +<para> + +<programlisting> + a1 :: T Int + a1 = T1 (\xy->x) 3 + + a2, a3 :: Swizzle + a2 = MkSwizzle sort + a3 = MkSwizzle reverse + + a4 :: MonadT Maybe + a4 = let r x = Just x + b m k = case m of + Just y -> k y + Nothing -> Nothing + in + MkMonad r b + + mkTs :: (forall b. b -> b -> b) -> a -> [T a] + mkTs f x y = [T1 f x, T1 f y] +</programlisting> + +</para> + +<para> +The type of the argument can, as usual, be more general than the type +required, as <literal>(MkSwizzle reverse)</literal> shows. (<function>reverse</function> +does not need the <literal>Ord</literal> constraint.) +</para> + +<para> +When you use pattern matching, the bound variables may now have +polymorphic types. For example: +</para> + +<para> + +<programlisting> + f :: T a -> a -> (a, Char) + f (T1 w k) x = (w k x, w 'c' 'd') + + g :: (Ord a, Ord b) => Swizzle -> [a] -> (a -> b) -> [b] + g (MkSwizzle s) xs f = s (map f (s xs)) + + h :: MonadT m -> [m a] -> m [a] + h m [] = return m [] + h m (x:xs) = bind m x $ \y -> + bind m (h m xs) $ \ys -> + return m (y:ys) +</programlisting> + +</para> + +<para> +In the function <function>h</function> we use the record selectors <literal>return</literal> +and <literal>bind</literal> to extract the polymorphic bind and return functions +from the <literal>MonadT</literal> data structure, rather than using pattern +matching. +</para> +</sect3> + +<sect3> +<title>Type inference</title> + +<para> +In general, type inference for arbitrary-rank types is undecidable. +GHC uses an algorithm proposed by Odersky and Laufer ("Putting type annotations to work", POPL'96) +to get a decidable algorithm by requiring some help from the programmer. +We do not yet have a formal specification of "some help" but the rule is this: +</para> +<para> +<emphasis>For a lambda-bound or case-bound variable, x, either the programmer +provides an explicit polymorphic type for x, or GHC's type inference will assume +that x's type has no foralls in it</emphasis>. +</para> +<para> +What does it mean to "provide" an explicit type for x? You can do that by +giving a type signature for x directly, using a pattern type signature +(<xref linkend="scoped-type-variables"/>), thus: +<programlisting> + \ f :: (forall a. a->a) -> (f True, f 'c') +</programlisting> +Alternatively, you can give a type signature to the enclosing +context, which GHC can "push down" to find the type for the variable: +<programlisting> + (\ f -> (f True, f 'c')) :: (forall a. a->a) -> (Bool,Char) +</programlisting> +Here the type signature on the expression can be pushed inwards +to give a type signature for f. Similarly, and more commonly, +one can give a type signature for the function itself: +<programlisting> + h :: (forall a. a->a) -> (Bool,Char) + h f = (f True, f 'c') +</programlisting> +You don't need to give a type signature if the lambda bound variable +is a constructor argument. Here is an example we saw earlier: +<programlisting> + f :: T a -> a -> (a, Char) + f (T1 w k) x = (w k x, w 'c' 'd') +</programlisting> +Here we do not need to give a type signature to <literal>w</literal>, because +it is an argument of constructor <literal>T1</literal> and that tells GHC all +it needs to know. +</para> + +</sect3> + + +<sect3 id="implicit-quant"> +<title>Implicit quantification</title> + +<para> +GHC performs implicit quantification as follows. <emphasis>At the top level (only) of +user-written types, if and only if there is no explicit <literal>forall</literal>, +GHC finds all the type variables mentioned in the type that are not already +in scope, and universally quantifies them.</emphasis> For example, the following pairs are +equivalent: +<programlisting> + f :: a -> a + f :: forall a. a -> a + + g (x::a) = let + h :: a -> b -> b + h x y = y + in ... + g (x::a) = let + h :: forall b. a -> b -> b + h x y = y + in ... +</programlisting> +</para> +<para> +Notice that GHC does <emphasis>not</emphasis> find the innermost possible quantification +point. For example: +<programlisting> + f :: (a -> a) -> Int + -- MEANS + f :: forall a. (a -> a) -> Int + -- NOT + f :: (forall a. a -> a) -> Int + + + g :: (Ord a => a -> a) -> Int + -- MEANS the illegal type + g :: forall a. (Ord a => a -> a) -> Int + -- NOT + g :: (forall a. Ord a => a -> a) -> Int +</programlisting> +The latter produces an illegal type, which you might think is silly, +but at least the rule is simple. If you want the latter type, you +can write your for-alls explicitly. Indeed, doing so is strongly advised +for rank-2 types. +</para> +</sect3> +</sect2> + + + + +<sect2 id="scoped-type-variables"> +<title>Scoped type variables +</title> + +<para> +A <emphasis>lexically scoped type variable</emphasis> can be bound by: +<itemizedlist> +<listitem><para>A declaration type signature (<xref linkend="decl-type-sigs"/>)</para></listitem> +<listitem><para>A pattern type signature (<xref linkend="pattern-type-sigs"/>)</para></listitem> +<listitem><para>A result type signature (<xref linkend="result-type-sigs"/>)</para></listitem> +</itemizedlist> +For example: +<programlisting> +f (xs::[a]) = ys ++ ys + where + ys :: [a] + ys = reverse xs +</programlisting> +The pattern <literal>(xs::[a])</literal> includes a type signature for <varname>xs</varname>. +This brings the type variable <literal>a</literal> into scope; it scopes over +all the patterns and right hand sides for this equation for <function>f</function>. +In particular, it is in scope at the type signature for <varname>y</varname>. +</para> + +<para> +At ordinary type signatures, such as that for <varname>ys</varname>, any type variables +mentioned in the type signature <emphasis>that are not in scope</emphasis> are +implicitly universally quantified. (If there are no type variables in +scope, all type variables mentioned in the signature are universally +quantified, which is just as in Haskell 98.) In this case, since <varname>a</varname> +is in scope, it is not universally quantified, so the type of <varname>ys</varname> is +the same as that of <varname>xs</varname>. In Haskell 98 it is not possible to declare +a type for <varname>ys</varname>; a major benefit of scoped type variables is that +it becomes possible to do so. +</para> + +<para> +Scoped type variables are implemented in both GHC and Hugs. Where the +implementations differ from the specification below, those differences +are noted. +</para> + +<para> +So much for the basic idea. Here are the details. +</para> + +<sect3> +<title>What a scoped type variable means</title> +<para> +A lexically-scoped type variable is simply +the name for a type. The restriction it expresses is that all occurrences +of the same name mean the same type. For example: +<programlisting> + f :: [Int] -> Int -> Int + f (xs::[a]) (y::a) = (head xs + y) :: a +</programlisting> +The pattern type signatures on the left hand side of +<literal>f</literal> express the fact that <literal>xs</literal> +must be a list of things of some type <literal>a</literal>; and that <literal>y</literal> +must have this same type. The type signature on the expression <literal>(head xs)</literal> +specifies that this expression must have the same type <literal>a</literal>. +<emphasis>There is no requirement that the type named by "<literal>a</literal>" is +in fact a type variable</emphasis>. Indeed, in this case, the type named by "<literal>a</literal>" is +<literal>Int</literal>. (This is a slight liberalisation from the original rather complex +rules, which specified that a pattern-bound type variable should be universally quantified.) +For example, all of these are legal:</para> + +<programlisting> + t (x::a) (y::a) = x+y*2 + + f (x::a) (y::b) = [x,y] -- a unifies with b + + g (x::a) = x + 1::Int -- a unifies with Int + + h x = let k (y::a) = [x,y] -- a is free in the + in k x -- environment + + k (x::a) True = ... -- a unifies with Int + k (x::Int) False = ... + + w :: [b] -> [b] + w (x::a) = x -- a unifies with [b] +</programlisting> + +</sect3> + +<sect3> +<title>Scope and implicit quantification</title> + +<para> + +<itemizedlist> +<listitem> + +<para> +All the type variables mentioned in a pattern, +that are not already in scope, +are brought into scope by the pattern. We describe this set as +the <emphasis>type variables bound by the pattern</emphasis>. +For example: +<programlisting> + f (x::a) = let g (y::(a,b)) = fst y + in + g (x,True) +</programlisting> +The pattern <literal>(x::a)</literal> brings the type variable +<literal>a</literal> into scope, as well as the term +variable <literal>x</literal>. The pattern <literal>(y::(a,b))</literal> +contains an occurrence of the already-in-scope type variable <literal>a</literal>, +and brings into scope the type variable <literal>b</literal>. +</para> +</listitem> + +<listitem> +<para> +The type variable(s) bound by the pattern have the same scope +as the term variable(s) bound by the pattern. For example: +<programlisting> + let + f (x::a) = <...rhs of f...> + (p::b, q::b) = (1,2) + in <...body of let...> +</programlisting> +Here, the type variable <literal>a</literal> scopes over the right hand side of <literal>f</literal>, +just like <literal>x</literal> does; while the type variable <literal>b</literal> scopes over the +body of the <literal>let</literal>, and all the other definitions in the <literal>let</literal>, +just like <literal>p</literal> and <literal>q</literal> do. +Indeed, the newly bound type variables also scope over any ordinary, separate +type signatures in the <literal>let</literal> group. +</para> +</listitem> + + +<listitem> +<para> +The type variables bound by the pattern may be +mentioned in ordinary type signatures or pattern +type signatures anywhere within their scope. + +</para> +</listitem> + +<listitem> +<para> + In ordinary type signatures, any type variable mentioned in the +signature that is in scope is <emphasis>not</emphasis> universally quantified. + +</para> +</listitem> + +<listitem> + +<para> + Ordinary type signatures do not bring any new type variables +into scope (except in the type signature itself!). So this is illegal: + +<programlisting> + f :: a -> a + f x = x::a +</programlisting> + +It's illegal because <varname>a</varname> is not in scope in the body of <function>f</function>, +so the ordinary signature <literal>x::a</literal> is equivalent to <literal>x::forall a.a</literal>; +and that is an incorrect typing. + +</para> +</listitem> + +<listitem> +<para> +The pattern type signature is a monotype: +</para> + +<itemizedlist> +<listitem> <para> +A pattern type signature cannot contain any explicit <literal>forall</literal> quantification. +</para> </listitem> + +<listitem> <para> +The type variables bound by a pattern type signature can only be instantiated to monotypes, +not to type schemes. +</para> </listitem> + +<listitem> <para> +There is no implicit universal quantification on pattern type signatures (in contrast to +ordinary type signatures). +</para> </listitem> + +</itemizedlist> + +</listitem> + +<listitem> +<para> + +The type variables in the head of a <literal>class</literal> or <literal>instance</literal> declaration +scope over the methods defined in the <literal>where</literal> part. For example: + + +<programlisting> + class C a where + op :: [a] -> a + + op xs = let ys::[a] + ys = reverse xs + in + head ys +</programlisting> + + +(Not implemented in Hugs yet, Dec 98). +</para> +</listitem> + +</itemizedlist> + +</para> + +</sect3> + +<sect3 id="decl-type-sigs"> +<title>Declaration type signatures</title> +<para>A declaration type signature that has <emphasis>explicit</emphasis> +quantification (using <literal>forall</literal>) brings into scope the +explicitly-quantified +type variables, in the definition of the named function(s). For example: +<programlisting> + f :: forall a. [a] -> [a] + f (x:xs) = xs ++ [ x :: a ] +</programlisting> +The "<literal>forall a</literal>" brings "<literal>a</literal>" into scope in +the definition of "<literal>f</literal>". +</para> +<para>This only happens if the quantification in <literal>f</literal>'s type +signature is explicit. For example: +<programlisting> + g :: [a] -> [a] + g (x:xs) = xs ++ [ x :: a ] +</programlisting> +This program will be rejected, because "<literal>a</literal>" does not scope +over the definition of "<literal>f</literal>", so "<literal>x::a</literal>" +means "<literal>x::forall a. a</literal>" by Haskell's usual implicit +quantification rules. +</para> +</sect3> + +<sect3 id="pattern-type-sigs"> +<title>Where a pattern type signature can occur</title> + +<para> +A pattern type signature can occur in any pattern. For example: +<itemizedlist> + +<listitem> +<para> +A pattern type signature can be on an arbitrary sub-pattern, not +just on a variable: + + +<programlisting> + f ((x,y)::(a,b)) = (y,x) :: (b,a) +</programlisting> + + +</para> +</listitem> +<listitem> + +<para> + Pattern type signatures, including the result part, can be used +in lambda abstractions: + +<programlisting> + (\ (x::a, y) :: a -> x) +</programlisting> +</para> +</listitem> +<listitem> + +<para> + Pattern type signatures, including the result part, can be used +in <literal>case</literal> expressions: + +<programlisting> + case e of { ((x::a, y) :: (a,b)) -> x } +</programlisting> + +Note that the <literal>-></literal> symbol in a case alternative +leads to difficulties when parsing a type signature in the pattern: in +the absence of the extra parentheses in the example above, the parser +would try to interpret the <literal>-></literal> as a function +arrow and give a parse error later. + +</para> + +</listitem> + +<listitem> +<para> +To avoid ambiguity, the type after the “<literal>::</literal>” in a result +pattern signature on a lambda or <literal>case</literal> must be atomic (i.e. a single +token or a parenthesised type of some sort). To see why, +consider how one would parse this: + + +<programlisting> + \ x :: a -> b -> x +</programlisting> + + +</para> +</listitem> + +<listitem> + +<para> + Pattern type signatures can bind existential type variables. +For example: + + +<programlisting> + data T = forall a. MkT [a] + + f :: T -> T + f (MkT [t::a]) = MkT t3 + where + t3::[a] = [t,t,t] +</programlisting> + + +</para> +</listitem> + + +<listitem> + +<para> +Pattern type signatures +can be used in pattern bindings: + +<programlisting> + f x = let (y, z::a) = x in ... + f1 x = let (y, z::Int) = x in ... + f2 (x::(Int,a)) = let (y, z::a) = x in ... + f3 :: (b->b) = \x -> x +</programlisting> + +In all such cases, the binding is not generalised over the pattern-bound +type variables. Thus <literal>f3</literal> is monomorphic; <literal>f3</literal> +has type <literal>b -> b</literal> for some type <literal>b</literal>, +and <emphasis>not</emphasis> <literal>forall b. b -> b</literal>. +In contrast, the binding +<programlisting> + f4 :: b->b + f4 = \x -> x +</programlisting> +makes a polymorphic function, but <literal>b</literal> is not in scope anywhere +in <literal>f4</literal>'s scope. + +</para> +</listitem> +</itemizedlist> +</para> +<para>Pattern type signatures are completely orthogonal to ordinary, separate +type signatures. The two can be used independently or together.</para> + +</sect3> + +<sect3 id="result-type-sigs"> +<title>Result type signatures</title> + +<para> +The result type of a function can be given a signature, thus: + + +<programlisting> + f (x::a) :: [a] = [x,x,x] +</programlisting> + + +The final <literal>:: [a]</literal> after all the patterns gives a signature to the +result type. Sometimes this is the only way of naming the type variable +you want: + + +<programlisting> + f :: Int -> [a] -> [a] + f n :: ([a] -> [a]) = let g (x::a, y::a) = (y,x) + in \xs -> map g (reverse xs `zip` xs) +</programlisting> + +</para> +<para> +The type variables bound in a result type signature scope over the right hand side +of the definition. However, consider this corner-case: +<programlisting> + rev1 :: [a] -> [a] = \xs -> reverse xs + + foo ys = rev (ys::[a]) +</programlisting> +The signature on <literal>rev1</literal> is considered a pattern type signature, not a result +type signature, and the type variables it binds have the same scope as <literal>rev1</literal> +itself (i.e. the right-hand side of <literal>rev1</literal> and the rest of the module too). +In particular, the expression <literal>(ys::[a])</literal> is OK, because the type variable <literal>a</literal> +is in scope (otherwise it would mean <literal>(ys::forall a.[a])</literal>, which would be rejected). +</para> +<para> +As mentioned above, <literal>rev1</literal> is made monomorphic by this scoping rule. +For example, the following program would be rejected, because it claims that <literal>rev1</literal> +is polymorphic: +<programlisting> + rev1 :: [b] -> [b] + rev1 :: [a] -> [a] = \xs -> reverse xs +</programlisting> +</para> + +<para> +Result type signatures are not yet implemented in Hugs. +</para> + +</sect3> + +</sect2> + +<sect2 id="deriving-typeable"> +<title>Deriving clause for classes <literal>Typeable</literal> and <literal>Data</literal></title> + +<para> +Haskell 98 allows the programmer to add "<literal>deriving( Eq, Ord )</literal>" to a data type +declaration, to generate a standard instance declaration for classes specified in the <literal>deriving</literal> clause. +In Haskell 98, the only classes that may appear in the <literal>deriving</literal> clause are the standard +classes <literal>Eq</literal>, <literal>Ord</literal>, +<literal>Enum</literal>, <literal>Ix</literal>, <literal>Bounded</literal>, <literal>Read</literal>, and <literal>Show</literal>. +</para> +<para> +GHC extends this list with two more classes that may be automatically derived +(provided the <option>-fglasgow-exts</option> flag is specified): +<literal>Typeable</literal>, and <literal>Data</literal>. These classes are defined in the library +modules <literal>Data.Typeable</literal> and <literal>Data.Generics</literal> respectively, and the +appropriate class must be in scope before it can be mentioned in the <literal>deriving</literal> clause. +</para> +<para>An instance of <literal>Typeable</literal> can only be derived if the +data type has seven or fewer type parameters, all of kind <literal>*</literal>. +The reason for this is that the <literal>Typeable</literal> class is derived using the scheme +described in +<ulink url="http://research.microsoft.com/%7Esimonpj/papers/hmap/gmap2.ps"> +Scrap More Boilerplate: Reflection, Zips, and Generalised Casts +</ulink>. +(Section 7.4 of the paper describes the multiple <literal>Typeable</literal> classes that +are used, and only <literal>Typeable1</literal> up to +<literal>Typeable7</literal> are provided in the library.) +In other cases, there is nothing to stop the programmer writing a <literal>TypableX</literal> +class, whose kind suits that of the data type constructor, and +then writing the data type instance by hand. +</para> +</sect2> + +<sect2 id="newtype-deriving"> +<title>Generalised derived instances for newtypes</title> + +<para> +When you define an abstract type using <literal>newtype</literal>, you may want +the new type to inherit some instances from its representation. In +Haskell 98, you can inherit instances of <literal>Eq</literal>, <literal>Ord</literal>, +<literal>Enum</literal> and <literal>Bounded</literal> by deriving them, but for any +other classes you have to write an explicit instance declaration. For +example, if you define + +<programlisting> + newtype Dollars = Dollars Int +</programlisting> + +and you want to use arithmetic on <literal>Dollars</literal>, you have to +explicitly define an instance of <literal>Num</literal>: + +<programlisting> + instance Num Dollars where + Dollars a + Dollars b = Dollars (a+b) + ... +</programlisting> +All the instance does is apply and remove the <literal>newtype</literal> +constructor. It is particularly galling that, since the constructor +doesn't appear at run-time, this instance declaration defines a +dictionary which is <emphasis>wholly equivalent</emphasis> to the <literal>Int</literal> +dictionary, only slower! +</para> + + +<sect3> <title> Generalising the deriving clause </title> +<para> +GHC now permits such instances to be derived instead, so one can write +<programlisting> + newtype Dollars = Dollars Int deriving (Eq,Show,Num) +</programlisting> + +and the implementation uses the <emphasis>same</emphasis> <literal>Num</literal> dictionary +for <literal>Dollars</literal> as for <literal>Int</literal>. Notionally, the compiler +derives an instance declaration of the form + +<programlisting> + instance Num Int => Num Dollars +</programlisting> + +which just adds or removes the <literal>newtype</literal> constructor according to the type. +</para> +<para> + +We can also derive instances of constructor classes in a similar +way. For example, suppose we have implemented state and failure monad +transformers, such that + +<programlisting> + instance Monad m => Monad (State s m) + instance Monad m => Monad (Failure m) +</programlisting> +In Haskell 98, we can define a parsing monad by +<programlisting> + type Parser tok m a = State [tok] (Failure m) a +</programlisting> + +which is automatically a monad thanks to the instance declarations +above. With the extension, we can make the parser type abstract, +without needing to write an instance of class <literal>Monad</literal>, via + +<programlisting> + newtype Parser tok m a = Parser (State [tok] (Failure m) a) + deriving Monad +</programlisting> +In this case the derived instance declaration is of the form +<programlisting> + instance Monad (State [tok] (Failure m)) => Monad (Parser tok m) +</programlisting> + +Notice that, since <literal>Monad</literal> is a constructor class, the +instance is a <emphasis>partial application</emphasis> of the new type, not the +entire left hand side. We can imagine that the type declaration is +``eta-converted'' to generate the context of the instance +declaration. +</para> +<para> + +We can even derive instances of multi-parameter classes, provided the +newtype is the last class parameter. In this case, a ``partial +application'' of the class appears in the <literal>deriving</literal> +clause. For example, given the class + +<programlisting> + class StateMonad s m | m -> s where ... + instance Monad m => StateMonad s (State s m) where ... +</programlisting> +then we can derive an instance of <literal>StateMonad</literal> for <literal>Parser</literal>s by +<programlisting> + newtype Parser tok m a = Parser (State [tok] (Failure m) a) + deriving (Monad, StateMonad [tok]) +</programlisting> + +The derived instance is obtained by completing the application of the +class to the new type: + +<programlisting> + instance StateMonad [tok] (State [tok] (Failure m)) => + StateMonad [tok] (Parser tok m) +</programlisting> +</para> +<para> + +As a result of this extension, all derived instances in newtype + declarations are treated uniformly (and implemented just by reusing +the dictionary for the representation type), <emphasis>except</emphasis> +<literal>Show</literal> and <literal>Read</literal>, which really behave differently for +the newtype and its representation. +</para> +</sect3> + +<sect3> <title> A more precise specification </title> +<para> +Derived instance declarations are constructed as follows. Consider the +declaration (after expansion of any type synonyms) + +<programlisting> + newtype T v1...vn = T' (S t1...tk vk+1...vn) deriving (c1...cm) +</programlisting> + +where + <itemizedlist> +<listitem><para> + <literal>S</literal> is a type constructor, +</para></listitem> +<listitem><para> + The <literal>t1...tk</literal> are types, +</para></listitem> +<listitem><para> + The <literal>vk+1...vn</literal> are type variables which do not occur in any of + the <literal>ti</literal>, and +</para></listitem> +<listitem><para> + The <literal>ci</literal> are partial applications of + classes of the form <literal>C t1'...tj'</literal>, where the arity of <literal>C</literal> + is exactly <literal>j+1</literal>. That is, <literal>C</literal> lacks exactly one type argument. +</para></listitem> +<listitem><para> + None of the <literal>ci</literal> is <literal>Read</literal>, <literal>Show</literal>, + <literal>Typeable</literal>, or <literal>Data</literal>. These classes + should not "look through" the type or its constructor. You can still + derive these classes for a newtype, but it happens in the usual way, not + via this new mechanism. +</para></listitem> +</itemizedlist> +Then, for each <literal>ci</literal>, the derived instance +declaration is: +<programlisting> + instance ci (S t1...tk vk+1...v) => ci (T v1...vp) +</programlisting> +where <literal>p</literal> is chosen so that <literal>T v1...vp</literal> is of the +right <emphasis>kind</emphasis> for the last parameter of class <literal>Ci</literal>. +</para> +<para> + +As an example which does <emphasis>not</emphasis> work, consider +<programlisting> + newtype NonMonad m s = NonMonad (State s m s) deriving Monad +</programlisting> +Here we cannot derive the instance +<programlisting> + instance Monad (State s m) => Monad (NonMonad m) +</programlisting> + +because the type variable <literal>s</literal> occurs in <literal>State s m</literal>, +and so cannot be "eta-converted" away. It is a good thing that this +<literal>deriving</literal> clause is rejected, because <literal>NonMonad m</literal> is +not, in fact, a monad --- for the same reason. Try defining +<literal>>>=</literal> with the correct type: you won't be able to. +</para> +<para> + +Notice also that the <emphasis>order</emphasis> of class parameters becomes +important, since we can only derive instances for the last one. If the +<literal>StateMonad</literal> class above were instead defined as + +<programlisting> + class StateMonad m s | m -> s where ... +</programlisting> + +then we would not have been able to derive an instance for the +<literal>Parser</literal> type above. We hypothesise that multi-parameter +classes usually have one "main" parameter for which deriving new +instances is most interesting. +</para> +<para>Lastly, all of this applies only for classes other than +<literal>Read</literal>, <literal>Show</literal>, <literal>Typeable</literal>, +and <literal>Data</literal>, for which the built-in derivation applies (section +4.3.3. of the Haskell Report). +(For the standard classes <literal>Eq</literal>, <literal>Ord</literal>, +<literal>Ix</literal>, and <literal>Bounded</literal> it is immaterial whether +the standard method is used or the one described here.) +</para> +</sect3> + +</sect2> + +<sect2 id="typing-binds"> +<title>Generalised typing of mutually recursive bindings</title> + +<para> +The Haskell Report specifies that a group of bindings (at top level, or in a +<literal>let</literal> or <literal>where</literal>) should be sorted into +strongly-connected components, and then type-checked in dependency order +(<ulink url="http://haskell.org/onlinereport/decls.html#sect4.5.1">Haskell +Report, Section 4.5.1</ulink>). +As each group is type-checked, any binders of the group that +have +an explicit type signature are put in the type environment with the specified +polymorphic type, +and all others are monomorphic until the group is generalised +(<ulink url="http://haskell.org/onlinereport/decls.html#sect4.5.2">Haskell Report, Section 4.5.2</ulink>). +</para> + +<para>Following a suggestion of Mark Jones, in his paper +<ulink url="http://www.cse.ogi.edu/~mpj/thih/">Typing Haskell in +Haskell</ulink>, +GHC implements a more general scheme. If <option>-fglasgow-exts</option> is +specified: +<emphasis>the dependency analysis ignores references to variables that have an explicit +type signature</emphasis>. +As a result of this refined dependency analysis, the dependency groups are smaller, and more bindings will +typecheck. For example, consider: +<programlisting> + f :: Eq a => a -> Bool + f x = (x == x) || g True || g "Yes" + + g y = (y <= y) || f True +</programlisting> +This is rejected by Haskell 98, but under Jones's scheme the definition for +<literal>g</literal> is typechecked first, separately from that for +<literal>f</literal>, +because the reference to <literal>f</literal> in <literal>g</literal>'s right +hand side is ingored by the dependency analysis. Then <literal>g</literal>'s +type is generalised, to get +<programlisting> + g :: Ord a => a -> Bool +</programlisting> +Now, the defintion for <literal>f</literal> is typechecked, with this type for +<literal>g</literal> in the type environment. +</para> + +<para> +The same refined dependency analysis also allows the type signatures of +mutually-recursive functions to have different contexts, something that is illegal in +Haskell 98 (Section 4.5.2, last sentence). With +<option>-fglasgow-exts</option> +GHC only insists that the type signatures of a <emphasis>refined</emphasis> group have identical +type signatures; in practice this means that only variables bound by the same +pattern binding must have the same context. For example, this is fine: +<programlisting> + f :: Eq a => a -> Bool + f x = (x == x) || g True + + g :: Ord a => a -> Bool + g y = (y <= y) || f True +</programlisting> +</para> +</sect2> + +</sect1> +<!-- ==================== End of type system extensions ================= --> + +<!-- ====================== Generalised algebraic data types ======================= --> + +<sect1 id="gadt"> +<title>Generalised Algebraic Data Types</title> + +<para>Generalised Algebraic Data Types (GADTs) generalise ordinary algebraic data types by allowing you +to give the type signatures of constructors explicitly. For example: +<programlisting> + data Term a where + Lit :: Int -> Term Int + Succ :: Term Int -> Term Int + IsZero :: Term Int -> Term Bool + If :: Term Bool -> Term a -> Term a -> Term a + Pair :: Term a -> Term b -> Term (a,b) +</programlisting> +Notice that the return type of the constructors is not always <literal>Term a</literal>, as is the +case with ordinary vanilla data types. Now we can write a well-typed <literal>eval</literal> function +for these <literal>Terms</literal>: +<programlisting> + eval :: Term a -> a + eval (Lit i) = i + eval (Succ t) = 1 + eval t + eval (IsZero t) = eval t == 0 + eval (If b e1 e2) = if eval b then eval e1 else eval e2 + eval (Pair e1 e2) = (eval e1, eval e2) +</programlisting> +These and many other examples are given in papers by Hongwei Xi, and Tim Sheard. +</para> +<para> The extensions to GHC are these: +<itemizedlist> +<listitem><para> + Data type declarations have a 'where' form, as exemplified above. The type signature of +each constructor is independent, and is implicitly universally quantified as usual. Unlike a normal +Haskell data type declaration, the type variable(s) in the "<literal>data Term a where</literal>" header +have no scope. Indeed, one can write a kind signature instead: +<programlisting> + data Term :: * -> * where ... +</programlisting> +or even a mixture of the two: +<programlisting> + data Foo a :: (* -> *) -> * where ... +</programlisting> +The type variables (if given) may be explicitly kinded, so we could also write the header for <literal>Foo</literal> +like this: +<programlisting> + data Foo a (b :: * -> *) where ... +</programlisting> +</para></listitem> + +<listitem><para> +There are no restrictions on the type of the data constructor, except that the result +type must begin with the type constructor being defined. For example, in the <literal>Term</literal> data +type above, the type of each constructor must end with <literal> ... -> Term ...</literal>. +</para></listitem> + +<listitem><para> +You can use record syntax on a GADT-style data type declaration: + +<programlisting> + data Term a where + Lit { val :: Int } :: Term Int + Succ { num :: Term Int } :: Term Int + Pred { num :: Term Int } :: Term Int + IsZero { arg :: Term Int } :: Term Bool + Pair { arg1 :: Term a + , arg2 :: Term b + } :: Term (a,b) + If { cnd :: Term Bool + , tru :: Term a + , fls :: Term a + } :: Term a +</programlisting> +For every constructor that has a field <literal>f</literal>, (a) the type of +field <literal>f</literal> must be the same; and (b) the +result type of the constructor must be the same; both modulo alpha conversion. +Hence, in our example, we cannot merge the <literal>num</literal> and <literal>arg</literal> +fields above into a +single name. Although their field types are both <literal>Term Int</literal>, +their selector functions actually have different types: + +<programlisting> + num :: Term Int -> Term Int + arg :: Term Bool -> Term Int +</programlisting> + +At the moment, record updates are not yet possible with GADT, so support is +limited to record construction, selection and pattern matching: + +<programlisting> + someTerm :: Term Bool + someTerm = IsZero { arg = Succ { num = Lit { val = 0 } } } + + eval :: Term a -> a + eval Lit { val = i } = i + eval Succ { num = t } = eval t + 1 + eval Pred { num = t } = eval t - 1 + eval IsZero { arg = t } = eval t == 0 + eval Pair { arg1 = t1, arg2 = t2 } = (eval t1, eval t2) + eval t@If{} = if eval (cnd t) then eval (tru t) else eval (fls t) +</programlisting> + +</para></listitem> + +<listitem><para> +You can use strictness annotations, in the obvious places +in the constructor type: +<programlisting> + data Term a where + Lit :: !Int -> Term Int + If :: Term Bool -> !(Term a) -> !(Term a) -> Term a + Pair :: Term a -> Term b -> Term (a,b) +</programlisting> +</para></listitem> + +<listitem><para> +You can use a <literal>deriving</literal> clause on a GADT-style data type +declaration, but only if the data type could also have been declared in +Haskell-98 syntax. For example, these two declarations are equivalent +<programlisting> + data Maybe1 a where { + Nothing1 :: Maybe a ; + Just1 :: a -> Maybe a + } deriving( Eq, Ord ) + + data Maybe2 a = Nothing2 | Just2 a + deriving( Eq, Ord ) +</programlisting> +This simply allows you to declare a vanilla Haskell-98 data type using the +<literal>where</literal> form without losing the <literal>deriving</literal> clause. +</para></listitem> + +<listitem><para> +Pattern matching causes type refinement. For example, in the right hand side of the equation +<programlisting> + eval :: Term a -> a + eval (Lit i) = ... +</programlisting> +the type <literal>a</literal> is refined to <literal>Int</literal>. (That's the whole point!) +A precise specification of the type rules is beyond what this user manual aspires to, but there is a paper +about the ideas: "Wobbly types: practical type inference for generalised algebraic data types", on Simon PJ's home page.</para> + +<para> The general principle is this: <emphasis>type refinement is only carried out based on user-supplied type annotations</emphasis>. +So if no type signature is supplied for <literal>eval</literal>, no type refinement happens, and lots of obscure error messages will +occur. However, the refinement is quite general. For example, if we had: +<programlisting> + eval :: Term a -> a -> a + eval (Lit i) j = i+j +</programlisting> +the pattern match causes the type <literal>a</literal> to be refined to <literal>Int</literal> (because of the type +of the constructor <literal>Lit</literal>, and that refinement also applies to the type of <literal>j</literal>, and +the result type of the <literal>case</literal> expression. Hence the addition <literal>i+j</literal> is legal. +</para> +</listitem> +</itemizedlist> +</para> + +<para>Notice that GADTs generalise existential types. For example, these two declarations are equivalent: +<programlisting> + data T a = forall b. MkT b (b->a) + data T' a where { MKT :: b -> (b->a) -> T' a } +</programlisting> +</para> +</sect1> + +<!-- ====================== End of Generalised algebraic data types ======================= --> + +<!-- ====================== TEMPLATE HASKELL ======================= --> + +<sect1 id="template-haskell"> +<title>Template Haskell</title> + +<para>Template Haskell allows you to do compile-time meta-programming in Haskell. There is a "home page" for +Template Haskell at <ulink url="http://www.haskell.org/th/"> +http://www.haskell.org/th/</ulink>, while +the background to +the main technical innovations is discussed in "<ulink +url="http://research.microsoft.com/~simonpj/papers/meta-haskell"> +Template Meta-programming for Haskell</ulink>" (Proc Haskell Workshop 2002). +The details of the Template Haskell design are still in flux. Make sure you +consult the <ulink url="http://www.haskell.org/ghc/docs/latest/html/libraries/index.html">online library reference material</ulink> +(search for the type ExpQ). +[Temporary: many changes to the original design are described in + <ulink url="http://research.microsoft.com/~simonpj/tmp/notes2.ps">"http://research.microsoft.com/~simonpj/tmp/notes2.ps"</ulink>. +Not all of these changes are in GHC 6.2.] +</para> + +<para> The first example from that paper is set out below as a worked example to help get you started. +</para> + +<para> +The documentation here describes the realisation in GHC. (It's rather sketchy just now; +Tim Sheard is going to expand it.) +</para> + + <sect2> + <title>Syntax</title> + + <para> Template Haskell has the following new syntactic + constructions. You need to use the flag + <option>-fth</option><indexterm><primary><option>-fth</option></primary> + </indexterm>to switch these syntactic extensions on + (<option>-fth</option> is currently implied by + <option>-fglasgow-exts</option>, but you are encouraged to + specify it explicitly).</para> + + <itemizedlist> + <listitem><para> + A splice is written <literal>$x</literal>, where <literal>x</literal> is an + identifier, or <literal>$(...)</literal>, where the "..." is an arbitrary expression. + There must be no space between the "$" and the identifier or parenthesis. This use + of "$" overrides its meaning as an infix operator, just as "M.x" overrides the meaning + of "." as an infix operator. If you want the infix operator, put spaces around it. + </para> + <para> A splice can occur in place of + <itemizedlist> + <listitem><para> an expression; the spliced expression must + have type <literal>Q Exp</literal></para></listitem> + <listitem><para> a list of top-level declarations; ; the spliced expression must have type <literal>Q [Dec]</literal></para></listitem> + <listitem><para> [Planned, but not implemented yet.] a + type; the spliced expression must have type <literal>Q Typ</literal>.</para></listitem> + </itemizedlist> + (Note that the syntax for a declaration splice uses "<literal>$</literal>" not "<literal>splice</literal>" as in + the paper. Also the type of the enclosed expression must be <literal>Q [Dec]</literal>, not <literal>[Q Dec]</literal> + as in the paper.) + </para></listitem> + + + <listitem><para> + A expression quotation is written in Oxford brackets, thus: + <itemizedlist> + <listitem><para> <literal>[| ... |]</literal>, where the "..." is an expression; + the quotation has type <literal>Expr</literal>.</para></listitem> + <listitem><para> <literal>[d| ... |]</literal>, where the "..." is a list of top-level declarations; + the quotation has type <literal>Q [Dec]</literal>.</para></listitem> + <listitem><para> [Planned, but not implemented yet.] <literal>[t| ... |]</literal>, where the "..." is a type; + the quotation has type <literal>Type</literal>.</para></listitem> + </itemizedlist></para></listitem> + + <listitem><para> + Reification is written thus: + <itemizedlist> + <listitem><para> <literal>reifyDecl T</literal>, where <literal>T</literal> is a type constructor; this expression + has type <literal>Dec</literal>. </para></listitem> + <listitem><para> <literal>reifyDecl C</literal>, where <literal>C</literal> is a class; has type <literal>Dec</literal>.</para></listitem> + <listitem><para> <literal>reifyType f</literal>, where <literal>f</literal> is an identifier; has type <literal>Typ</literal>.</para></listitem> + <listitem><para> Still to come: fixities </para></listitem> + + </itemizedlist></para> + </listitem> + + + </itemizedlist> +</sect2> + +<sect2> <title> Using Template Haskell </title> +<para> +<itemizedlist> + <listitem><para> + The data types and monadic constructor functions for Template Haskell are in the library + <literal>Language.Haskell.THSyntax</literal>. + </para></listitem> + + <listitem><para> + You can only run a function at compile time if it is imported from another module. That is, + you can't define a function in a module, and call it from within a splice in the same module. + (It would make sense to do so, but it's hard to implement.) + </para></listitem> + + <listitem><para> + The flag <literal>-ddump-splices</literal> shows the expansion of all top-level splices as they happen. + </para></listitem> + <listitem><para> + If you are building GHC from source, you need at least a stage-2 bootstrap compiler to + run Template Haskell. A stage-1 compiler will reject the TH constructs. Reason: TH + compiles and runs a program, and then looks at the result. So it's important that + the program it compiles produces results whose representations are identical to + those of the compiler itself. + </para></listitem> +</itemizedlist> +</para> +<para> Template Haskell works in any mode (<literal>--make</literal>, <literal>--interactive</literal>, + or file-at-a-time). There used to be a restriction to the former two, but that restriction + has been lifted. +</para> +</sect2> + +<sect2> <title> A Template Haskell Worked Example </title> +<para>To help you get over the confidence barrier, try out this skeletal worked example. + First cut and paste the two modules below into "Main.hs" and "Printf.hs":</para> + +<programlisting> + +{- Main.hs -} +module Main where + +-- Import our template "pr" +import Printf ( pr ) + +-- The splice operator $ takes the Haskell source code +-- generated at compile time by "pr" and splices it into +-- the argument of "putStrLn". +main = putStrLn ( $(pr "Hello") ) + + +{- Printf.hs -} +module Printf where + +-- Skeletal printf from the paper. +-- It needs to be in a separate module to the one where +-- you intend to use it. + +-- Import some Template Haskell syntax +import Language.Haskell.TH + +-- Describe a format string +data Format = D | S | L String + +-- Parse a format string. This is left largely to you +-- as we are here interested in building our first ever +-- Template Haskell program and not in building printf. +parse :: String -> [Format] +parse s = [ L s ] + +-- Generate Haskell source code from a parsed representation +-- of the format string. This code will be spliced into +-- the module which calls "pr", at compile time. +gen :: [Format] -> ExpQ +gen [D] = [| \n -> show n |] +gen [S] = [| \s -> s |] +gen [L s] = stringE s + +-- Here we generate the Haskell code for the splice +-- from an input format string. +pr :: String -> ExpQ +pr s = gen (parse s) +</programlisting> + +<para>Now run the compiler (here we are a Cygwin prompt on Windows): +</para> +<programlisting> +$ ghc --make -fth main.hs -o main.exe +</programlisting> + +<para>Run "main.exe" and here is your output:</para> + +<programlisting> +$ ./main +Hello +</programlisting> + +</sect2> + +</sect1> + +<!-- ===================== Arrow notation =================== --> + +<sect1 id="arrow-notation"> +<title>Arrow notation +</title> + +<para>Arrows are a generalization of monads introduced by John Hughes. +For more details, see +<itemizedlist> + +<listitem> +<para> +“Generalising Monads to Arrows”, +John Hughes, in <citetitle>Science of Computer Programming</citetitle> 37, +pp67–111, May 2000. +</para> +</listitem> + +<listitem> +<para> +“<ulink url="http://www.soi.city.ac.uk/~ross/papers/notation.html">A New Notation for Arrows</ulink>”, +Ross Paterson, in <citetitle>ICFP</citetitle>, Sep 2001. +</para> +</listitem> + +<listitem> +<para> +“<ulink url="http://www.soi.city.ac.uk/~ross/papers/fop.html">Arrows and Computation</ulink>”, +Ross Paterson, in <citetitle>The Fun of Programming</citetitle>, +Palgrave, 2003. +</para> +</listitem> + +</itemizedlist> +and the arrows web page at +<ulink url="http://www.haskell.org/arrows/"><literal>http://www.haskell.org/arrows/</literal></ulink>. +With the <option>-farrows</option> flag, GHC supports the arrow +notation described in the second of these papers. +What follows is a brief introduction to the notation; +it won't make much sense unless you've read Hughes's paper. +This notation is translated to ordinary Haskell, +using combinators from the +<ulink url="../libraries/base/Control-Arrow.html"><literal>Control.Arrow</literal></ulink> +module. +</para> + +<para>The extension adds a new kind of expression for defining arrows: +<screen> +<replaceable>exp</replaceable><superscript>10</superscript> ::= ... + | proc <replaceable>apat</replaceable> -> <replaceable>cmd</replaceable> +</screen> +where <literal>proc</literal> is a new keyword. +The variables of the pattern are bound in the body of the +<literal>proc</literal>-expression, +which is a new sort of thing called a <firstterm>command</firstterm>. +The syntax of commands is as follows: +<screen> +<replaceable>cmd</replaceable> ::= <replaceable>exp</replaceable><superscript>10</superscript> -< <replaceable>exp</replaceable> + | <replaceable>exp</replaceable><superscript>10</superscript> -<< <replaceable>exp</replaceable> + | <replaceable>cmd</replaceable><superscript>0</superscript> +</screen> +with <replaceable>cmd</replaceable><superscript>0</superscript> up to +<replaceable>cmd</replaceable><superscript>9</superscript> defined using +infix operators as for expressions, and +<screen> +<replaceable>cmd</replaceable><superscript>10</superscript> ::= \ <replaceable>apat</replaceable> ... <replaceable>apat</replaceable> -> <replaceable>cmd</replaceable> + | let <replaceable>decls</replaceable> in <replaceable>cmd</replaceable> + | if <replaceable>exp</replaceable> then <replaceable>cmd</replaceable> else <replaceable>cmd</replaceable> + | case <replaceable>exp</replaceable> of { <replaceable>calts</replaceable> } + | do { <replaceable>cstmt</replaceable> ; ... <replaceable>cstmt</replaceable> ; <replaceable>cmd</replaceable> } + | <replaceable>fcmd</replaceable> + +<replaceable>fcmd</replaceable> ::= <replaceable>fcmd</replaceable> <replaceable>aexp</replaceable> + | ( <replaceable>cmd</replaceable> ) + | (| <replaceable>aexp</replaceable> <replaceable>cmd</replaceable> ... <replaceable>cmd</replaceable> |) + +<replaceable>cstmt</replaceable> ::= let <replaceable>decls</replaceable> + | <replaceable>pat</replaceable> <- <replaceable>cmd</replaceable> + | rec { <replaceable>cstmt</replaceable> ; ... <replaceable>cstmt</replaceable> [;] } + | <replaceable>cmd</replaceable> +</screen> +where <replaceable>calts</replaceable> are like <replaceable>alts</replaceable> +except that the bodies are commands instead of expressions. +</para> + +<para> +Commands produce values, but (like monadic computations) +may yield more than one value, +or none, and may do other things as well. +For the most part, familiarity with monadic notation is a good guide to +using commands. +However the values of expressions, even monadic ones, +are determined by the values of the variables they contain; +this is not necessarily the case for commands. +</para> + +<para> +A simple example of the new notation is the expression +<screen> +proc x -> f -< x+1 +</screen> +We call this a <firstterm>procedure</firstterm> or +<firstterm>arrow abstraction</firstterm>. +As with a lambda expression, the variable <literal>x</literal> +is a new variable bound within the <literal>proc</literal>-expression. +It refers to the input to the arrow. +In the above example, <literal>-<</literal> is not an identifier but an +new reserved symbol used for building commands from an expression of arrow +type and an expression to be fed as input to that arrow. +(The weird look will make more sense later.) +It may be read as analogue of application for arrows. +The above example is equivalent to the Haskell expression +<screen> +arr (\ x -> x+1) >>> f +</screen> +That would make no sense if the expression to the left of +<literal>-<</literal> involves the bound variable <literal>x</literal>. +More generally, the expression to the left of <literal>-<</literal> +may not involve any <firstterm>local variable</firstterm>, +i.e. a variable bound in the current arrow abstraction. +For such a situation there is a variant <literal>-<<</literal>, as in +<screen> +proc x -> f x -<< x+1 +</screen> +which is equivalent to +<screen> +arr (\ x -> (f x, x+1)) >>> app +</screen> +so in this case the arrow must belong to the <literal>ArrowApply</literal> +class. +Such an arrow is equivalent to a monad, so if you're using this form +you may find a monadic formulation more convenient. +</para> + +<sect2> +<title>do-notation for commands</title> + +<para> +Another form of command is a form of <literal>do</literal>-notation. +For example, you can write +<screen> +proc x -> do + y <- f -< x+1 + g -< 2*y + let z = x+y + t <- h -< x*z + returnA -< t+z +</screen> +You can read this much like ordinary <literal>do</literal>-notation, +but with commands in place of monadic expressions. +The first line sends the value of <literal>x+1</literal> as an input to +the arrow <literal>f</literal>, and matches its output against +<literal>y</literal>. +In the next line, the output is discarded. +The arrow <function>returnA</function> is defined in the +<ulink url="../libraries/base/Control-Arrow.html"><literal>Control.Arrow</literal></ulink> +module as <literal>arr id</literal>. +The above example is treated as an abbreviation for +<screen> +arr (\ x -> (x, x)) >>> + first (arr (\ x -> x+1) >>> f) >>> + arr (\ (y, x) -> (y, (x, y))) >>> + first (arr (\ y -> 2*y) >>> g) >>> + arr snd >>> + arr (\ (x, y) -> let z = x+y in ((x, z), z)) >>> + first (arr (\ (x, z) -> x*z) >>> h) >>> + arr (\ (t, z) -> t+z) >>> + returnA +</screen> +Note that variables not used later in the composition are projected out. +After simplification using rewrite rules (see <xref linkend="rewrite-rules"/>) +defined in the +<ulink url="../libraries/base/Control-Arrow.html"><literal>Control.Arrow</literal></ulink> +module, this reduces to +<screen> +arr (\ x -> (x+1, x)) >>> + first f >>> + arr (\ (y, x) -> (2*y, (x, y))) >>> + first g >>> + arr (\ (_, (x, y)) -> let z = x+y in (x*z, z)) >>> + first h >>> + arr (\ (t, z) -> t+z) +</screen> +which is what you might have written by hand. +With arrow notation, GHC keeps track of all those tuples of variables for you. +</para> + +<para> +Note that although the above translation suggests that +<literal>let</literal>-bound variables like <literal>z</literal> must be +monomorphic, the actual translation produces Core, +so polymorphic variables are allowed. +</para> + +<para> +It's also possible to have mutually recursive bindings, +using the new <literal>rec</literal> keyword, as in the following example: +<programlisting> +counter :: ArrowCircuit a => a Bool Int +counter = proc reset -> do + rec output <- returnA -< if reset then 0 else next + next <- delay 0 -< output+1 + returnA -< output +</programlisting> +The translation of such forms uses the <function>loop</function> combinator, +so the arrow concerned must belong to the <literal>ArrowLoop</literal> class. +</para> + +</sect2> + +<sect2> +<title>Conditional commands</title> + +<para> +In the previous example, we used a conditional expression to construct the +input for an arrow. +Sometimes we want to conditionally execute different commands, as in +<screen> +proc (x,y) -> + if f x y + then g -< x+1 + else h -< y+2 +</screen> +which is translated to +<screen> +arr (\ (x,y) -> if f x y then Left x else Right y) >>> + (arr (\x -> x+1) >>> f) ||| (arr (\y -> y+2) >>> g) +</screen> +Since the translation uses <function>|||</function>, +the arrow concerned must belong to the <literal>ArrowChoice</literal> class. +</para> + +<para> +There are also <literal>case</literal> commands, like +<screen> +case input of + [] -> f -< () + [x] -> g -< x+1 + x1:x2:xs -> do + y <- h -< (x1, x2) + ys <- k -< xs + returnA -< y:ys +</screen> +The syntax is the same as for <literal>case</literal> expressions, +except that the bodies of the alternatives are commands rather than expressions. +The translation is similar to that of <literal>if</literal> commands. +</para> + +</sect2> + +<sect2> +<title>Defining your own control structures</title> + +<para> +As we're seen, arrow notation provides constructs, +modelled on those for expressions, +for sequencing, value recursion and conditionals. +But suitable combinators, +which you can define in ordinary Haskell, +may also be used to build new commands out of existing ones. +The basic idea is that a command defines an arrow from environments to values. +These environments assign values to the free local variables of the command. +Thus combinators that produce arrows from arrows +may also be used to build commands from commands. +For example, the <literal>ArrowChoice</literal> class includes a combinator +<programlisting> +ArrowChoice a => (<+>) :: a e c -> a e c -> a e c +</programlisting> +so we can use it to build commands: +<programlisting> +expr' = proc x -> do + returnA -< x + <+> do + symbol Plus -< () + y <- term -< () + expr' -< x + y + <+> do + symbol Minus -< () + y <- term -< () + expr' -< x - y +</programlisting> +(The <literal>do</literal> on the first line is needed to prevent the first +<literal><+> ...</literal> from being interpreted as part of the +expression on the previous line.) +This is equivalent to +<programlisting> +expr' = (proc x -> returnA -< x) + <+> (proc x -> do + symbol Plus -< () + y <- term -< () + expr' -< x + y) + <+> (proc x -> do + symbol Minus -< () + y <- term -< () + expr' -< x - y) +</programlisting> +It is essential that this operator be polymorphic in <literal>e</literal> +(representing the environment input to the command +and thence to its subcommands) +and satisfy the corresponding naturality property +<screen> +arr k >>> (f <+> g) = (arr k >>> f) <+> (arr k >>> g) +</screen> +at least for strict <literal>k</literal>. +(This should be automatic if you're not using <function>seq</function>.) +This ensures that environments seen by the subcommands are environments +of the whole command, +and also allows the translation to safely trim these environments. +The operator must also not use any variable defined within the current +arrow abstraction. +</para> + +<para> +We could define our own operator +<programlisting> +untilA :: ArrowChoice a => a e () -> a e Bool -> a e () +untilA body cond = proc x -> + if cond x then returnA -< () + else do + body -< x + untilA body cond -< x +</programlisting> +and use it in the same way. +Of course this infix syntax only makes sense for binary operators; +there is also a more general syntax involving special brackets: +<screen> +proc x -> do + y <- f -< x+1 + (|untilA (increment -< x+y) (within 0.5 -< x)|) +</screen> +</para> + +</sect2> + +<sect2> +<title>Primitive constructs</title> + +<para> +Some operators will need to pass additional inputs to their subcommands. +For example, in an arrow type supporting exceptions, +the operator that attaches an exception handler will wish to pass the +exception that occurred to the handler. +Such an operator might have a type +<screen> +handleA :: ... => a e c -> a (e,Ex) c -> a e c +</screen> +where <literal>Ex</literal> is the type of exceptions handled. +You could then use this with arrow notation by writing a command +<screen> +body `handleA` \ ex -> handler +</screen> +so that if an exception is raised in the command <literal>body</literal>, +the variable <literal>ex</literal> is bound to the value of the exception +and the command <literal>handler</literal>, +which typically refers to <literal>ex</literal>, is entered. +Though the syntax here looks like a functional lambda, +we are talking about commands, and something different is going on. +The input to the arrow represented by a command consists of values for +the free local variables in the command, plus a stack of anonymous values. +In all the prior examples, this stack was empty. +In the second argument to <function>handleA</function>, +this stack consists of one value, the value of the exception. +The command form of lambda merely gives this value a name. +</para> + +<para> +More concretely, +the values on the stack are paired to the right of the environment. +So operators like <function>handleA</function> that pass +extra inputs to their subcommands can be designed for use with the notation +by pairing the values with the environment in this way. +More precisely, the type of each argument of the operator (and its result) +should have the form +<screen> +a (...(e,t1), ... tn) t +</screen> +where <replaceable>e</replaceable> is a polymorphic variable +(representing the environment) +and <replaceable>ti</replaceable> are the types of the values on the stack, +with <replaceable>t1</replaceable> being the <quote>top</quote>. +The polymorphic variable <replaceable>e</replaceable> must not occur in +<replaceable>a</replaceable>, <replaceable>ti</replaceable> or +<replaceable>t</replaceable>. +However the arrows involved need not be the same. +Here are some more examples of suitable operators: +<screen> +bracketA :: ... => a e b -> a (e,b) c -> a (e,c) d -> a e d +runReader :: ... => a e c -> a' (e,State) c +runState :: ... => a e c -> a' (e,State) (c,State) +</screen> +We can supply the extra input required by commands built with the last two +by applying them to ordinary expressions, as in +<screen> +proc x -> do + s <- ... + (|runReader (do { ... })|) s +</screen> +which adds <literal>s</literal> to the stack of inputs to the command +built using <function>runReader</function>. +</para> + +<para> +The command versions of lambda abstraction and application are analogous to +the expression versions. +In particular, the beta and eta rules describe equivalences of commands. +These three features (operators, lambda abstraction and application) +are the core of the notation; everything else can be built using them, +though the results would be somewhat clumsy. +For example, we could simulate <literal>do</literal>-notation by defining +<programlisting> +bind :: Arrow a => a e b -> a (e,b) c -> a e c +u `bind` f = returnA &&& u >>> f + +bind_ :: Arrow a => a e b -> a e c -> a e c +u `bind_` f = u `bind` (arr fst >>> f) +</programlisting> +We could simulate <literal>if</literal> by defining +<programlisting> +cond :: ArrowChoice a => a e b -> a e b -> a (e,Bool) b +cond f g = arr (\ (e,b) -> if b then Left e else Right e) >>> f ||| g +</programlisting> +</para> + +</sect2> + +<sect2> +<title>Differences with the paper</title> + +<itemizedlist> + +<listitem> +<para>Instead of a single form of arrow application (arrow tail) with two +translations, the implementation provides two forms +<quote><literal>-<</literal></quote> (first-order) +and <quote><literal>-<<</literal></quote> (higher-order). +</para> +</listitem> + +<listitem> +<para>User-defined operators are flagged with banana brackets instead of +a new <literal>form</literal> keyword. +</para> +</listitem> + +</itemizedlist> + +</sect2> + +<sect2> +<title>Portability</title> + +<para> +Although only GHC implements arrow notation directly, +there is also a preprocessor +(available from the +<ulink url="http://www.haskell.org/arrows/">arrows web page</ulink>) +that translates arrow notation into Haskell 98 +for use with other Haskell systems. +You would still want to check arrow programs with GHC; +tracing type errors in the preprocessor output is not easy. +Modules intended for both GHC and the preprocessor must observe some +additional restrictions: +<itemizedlist> + +<listitem> +<para> +The module must import +<ulink url="../libraries/base/Control-Arrow.html"><literal>Control.Arrow</literal></ulink>. +</para> +</listitem> + +<listitem> +<para> +The preprocessor cannot cope with other Haskell extensions. +These would have to go in separate modules. +</para> +</listitem> + +<listitem> +<para> +Because the preprocessor targets Haskell (rather than Core), +<literal>let</literal>-bound variables are monomorphic. +</para> +</listitem> + +</itemizedlist> +</para> + +</sect2> + +</sect1> + +<!-- ==================== ASSERTIONS ================= --> + +<sect1 id="sec-assertions"> +<title>Assertions +<indexterm><primary>Assertions</primary></indexterm> +</title> + +<para> +If you want to make use of assertions in your standard Haskell code, you +could define a function like the following: +</para> + +<para> + +<programlisting> +assert :: Bool -> a -> a +assert False x = error "assertion failed!" +assert _ x = x +</programlisting> + +</para> + +<para> +which works, but gives you back a less than useful error message -- +an assertion failed, but which and where? +</para> + +<para> +One way out is to define an extended <function>assert</function> function which also +takes a descriptive string to include in the error message and +perhaps combine this with the use of a pre-processor which inserts +the source location where <function>assert</function> was used. +</para> + +<para> +Ghc offers a helping hand here, doing all of this for you. For every +use of <function>assert</function> in the user's source: +</para> + +<para> + +<programlisting> +kelvinToC :: Double -> Double +kelvinToC k = assert (k >= 0.0) (k+273.15) +</programlisting> + +</para> + +<para> +Ghc will rewrite this to also include the source location where the +assertion was made, +</para> + +<para> + +<programlisting> +assert pred val ==> assertError "Main.hs|15" pred val +</programlisting> + +</para> + +<para> +The rewrite is only performed by the compiler when it spots +applications of <function>Control.Exception.assert</function>, so you +can still define and use your own versions of +<function>assert</function>, should you so wish. If not, import +<literal>Control.Exception</literal> to make use +<function>assert</function> in your code. +</para> + +<para> +GHC ignores assertions when optimisation is turned on with the + <option>-O</option><indexterm><primary><option>-O</option></primary></indexterm> flag. That is, expressions of the form +<literal>assert pred e</literal> will be rewritten to +<literal>e</literal>. You can also disable assertions using the + <option>-fignore-asserts</option> + option<indexterm><primary><option>-fignore-asserts</option></primary> + </indexterm>.</para> + +<para> +Assertion failures can be caught, see the documentation for the +<literal>Control.Exception</literal> library for the details. +</para> + +</sect1> + + +<!-- =============================== PRAGMAS =========================== --> + + <sect1 id="pragmas"> + <title>Pragmas</title> + + <indexterm><primary>pragma</primary></indexterm> + + <para>GHC supports several pragmas, or instructions to the + compiler placed in the source code. Pragmas don't normally affect + the meaning of the program, but they might affect the efficiency + of the generated code.</para> + + <para>Pragmas all take the form + +<literal>{-# <replaceable>word</replaceable> ... #-}</literal> + + where <replaceable>word</replaceable> indicates the type of + pragma, and is followed optionally by information specific to that + type of pragma. Case is ignored in + <replaceable>word</replaceable>. The various values for + <replaceable>word</replaceable> that GHC understands are described + in the following sections; any pragma encountered with an + unrecognised <replaceable>word</replaceable> is (silently) + ignored.</para> + + <sect2 id="deprecated-pragma"> + <title>DEPRECATED pragma</title> + <indexterm><primary>DEPRECATED</primary> + </indexterm> + + <para>The DEPRECATED pragma lets you specify that a particular + function, class, or type, is deprecated. There are two + forms. + + <itemizedlist> + <listitem> + <para>You can deprecate an entire module thus:</para> +<programlisting> + module Wibble {-# DEPRECATED "Use Wobble instead" #-} where + ... +</programlisting> + <para>When you compile any module that import + <literal>Wibble</literal>, GHC will print the specified + message.</para> + </listitem> + + <listitem> + <para>You can deprecate a function, class, type, or data constructor, with the + following top-level declaration:</para> +<programlisting> + {-# DEPRECATED f, C, T "Don't use these" #-} +</programlisting> + <para>When you compile any module that imports and uses any + of the specified entities, GHC will print the specified + message.</para> + <para> You can only depecate entities declared at top level in the module + being compiled, and you can only use unqualified names in the list of + entities being deprecated. A capitalised name, such as <literal>T</literal> + refers to <emphasis>either</emphasis> the type constructor <literal>T</literal> + <emphasis>or</emphasis> the data constructor <literal>T</literal>, or both if + both are in scope. If both are in scope, there is currently no way to deprecate + one without the other (c.f. fixities <xref linkend="infix-tycons"/>).</para> + </listitem> + </itemizedlist> + Any use of the deprecated item, or of anything from a deprecated + module, will be flagged with an appropriate message. However, + deprecations are not reported for + (a) uses of a deprecated function within its defining module, and + (b) uses of a deprecated function in an export list. + The latter reduces spurious complaints within a library + in which one module gathers together and re-exports + the exports of several others. + </para> + <para>You can suppress the warnings with the flag + <option>-fno-warn-deprecations</option>.</para> + </sect2> + + <sect2 id="include-pragma"> + <title>INCLUDE pragma</title> + + <para>The <literal>INCLUDE</literal> pragma is for specifying the names + of C header files that should be <literal>#include</literal>'d into + the C source code generated by the compiler for the current module (if + compiling via C). For example:</para> + +<programlisting> +{-# INCLUDE "foo.h" #-} +{-# INCLUDE <stdio.h> #-}</programlisting> + + <para>The <literal>INCLUDE</literal> pragma(s) must appear at the top of + your source file with any <literal>OPTIONS_GHC</literal> + pragma(s).</para> + + <para>An <literal>INCLUDE</literal> pragma is the preferred alternative + to the <option>-#include</option> option (<xref + linkend="options-C-compiler" />), because the + <literal>INCLUDE</literal> pragma is understood by other + compilers. Yet another alternative is to add the include file to each + <literal>foreign import</literal> declaration in your code, but we + don't recommend using this approach with GHC.</para> + </sect2> + + <sect2 id="inline-noinline-pragma"> + <title>INLINE and NOINLINE pragmas</title> + + <para>These pragmas control the inlining of function + definitions.</para> + + <sect3 id="inline-pragma"> + <title>INLINE pragma</title> + <indexterm><primary>INLINE</primary></indexterm> + + <para>GHC (with <option>-O</option>, as always) tries to + inline (or “unfold”) functions/values that are + “small enough,” thus avoiding the call overhead + and possibly exposing other more-wonderful optimisations. + Normally, if GHC decides a function is “too + expensive” to inline, it will not do so, nor will it + export that unfolding for other modules to use.</para> + + <para>The sledgehammer you can bring to bear is the + <literal>INLINE</literal><indexterm><primary>INLINE + pragma</primary></indexterm> pragma, used thusly:</para> + +<programlisting> +key_function :: Int -> String -> (Bool, Double) + +#ifdef __GLASGOW_HASKELL__ +{-# INLINE key_function #-} +#endif +</programlisting> + + <para>(You don't need to do the C pre-processor carry-on + unless you're going to stick the code through HBC—it + doesn't like <literal>INLINE</literal> pragmas.)</para> + + <para>The major effect of an <literal>INLINE</literal> pragma + is to declare a function's “cost” to be very low. + The normal unfolding machinery will then be very keen to + inline it.</para> + + <para>Syntactically, an <literal>INLINE</literal> pragma for a + function can be put anywhere its type signature could be + put.</para> + + <para><literal>INLINE</literal> pragmas are a particularly + good idea for the + <literal>then</literal>/<literal>return</literal> (or + <literal>bind</literal>/<literal>unit</literal>) functions in + a monad. For example, in GHC's own + <literal>UniqueSupply</literal> monad code, we have:</para> + +<programlisting> +#ifdef __GLASGOW_HASKELL__ +{-# INLINE thenUs #-} +{-# INLINE returnUs #-} +#endif +</programlisting> + + <para>See also the <literal>NOINLINE</literal> pragma (<xref + linkend="noinline-pragma"/>).</para> + </sect3> + + <sect3 id="noinline-pragma"> + <title>NOINLINE pragma</title> + + <indexterm><primary>NOINLINE</primary></indexterm> + <indexterm><primary>NOTINLINE</primary></indexterm> + + <para>The <literal>NOINLINE</literal> pragma does exactly what + you'd expect: it stops the named function from being inlined + by the compiler. You shouldn't ever need to do this, unless + you're very cautious about code size.</para> + + <para><literal>NOTINLINE</literal> is a synonym for + <literal>NOINLINE</literal> (<literal>NOINLINE</literal> is + specified by Haskell 98 as the standard way to disable + inlining, so it should be used if you want your code to be + portable).</para> + </sect3> + + <sect3 id="phase-control"> + <title>Phase control</title> + + <para> Sometimes you want to control exactly when in GHC's + pipeline the INLINE pragma is switched on. Inlining happens + only during runs of the <emphasis>simplifier</emphasis>. Each + run of the simplifier has a different <emphasis>phase + number</emphasis>; the phase number decreases towards zero. + If you use <option>-dverbose-core2core</option> you'll see the + sequence of phase numbers for successive runs of the + simplifier. In an INLINE pragma you can optionally specify a + phase number, thus:</para> + + <itemizedlist> + <listitem> + <para>You can say "inline <literal>f</literal> in Phase 2 + and all subsequent phases": +<programlisting> + {-# INLINE [2] f #-} +</programlisting> + </para> + </listitem> + + <listitem> + <para>You can say "inline <literal>g</literal> in all + phases up to, but not including, Phase 3": +<programlisting> + {-# INLINE [~3] g #-} +</programlisting> + </para> + </listitem> + + <listitem> + <para>If you omit the phase indicator, you mean "inline in + all phases".</para> + </listitem> + </itemizedlist> + + <para>You can use a phase number on a NOINLINE pragma too:</para> + + <itemizedlist> + <listitem> + <para>You can say "do not inline <literal>f</literal> + until Phase 2; in Phase 2 and subsequently behave as if + there was no pragma at all": +<programlisting> + {-# NOINLINE [2] f #-} +</programlisting> + </para> + </listitem> + + <listitem> + <para>You can say "do not inline <literal>g</literal> in + Phase 3 or any subsequent phase; before that, behave as if + there was no pragma": +<programlisting> + {-# NOINLINE [~3] g #-} +</programlisting> + </para> + </listitem> + + <listitem> + <para>If you omit the phase indicator, you mean "never + inline this function".</para> + </listitem> + </itemizedlist> + + <para>The same phase-numbering control is available for RULES + (<xref linkend="rewrite-rules"/>).</para> + </sect3> + </sect2> + + <sect2 id="language-pragma"> + <title>LANGUAGE pragma</title> + + <indexterm><primary>LANGUAGE</primary><secondary>pragma</secondary></indexterm> + <indexterm><primary>pragma</primary><secondary>LANGUAGE</secondary></indexterm> + + <para>This allows language extensions to be enabled in a portable way. + It is the intention that all Haskell compilers support the + <literal>LANGUAGE</literal> pragma with the same syntax, although not + all extensions are supported by all compilers, of + course. The <literal>LANGUAGE</literal> pragma should be used instead + of <literal>OPTIONS_GHC</literal>, if possible.</para> + + <para>For example, to enable the FFI and preprocessing with CPP:</para> + +<programlisting>{-# LANGUAGE ForeignFunctionInterface, CPP #-}</programlisting> + + <para>Any extension from the <literal>Extension</literal> type defined in + <ulink + url="../libraries/Cabal/Language-Haskell-Extension.html"><literal>Language.Haskell.Extension</literal></ulink> may be used. GHC will report an error if any of the requested extensions are not supported.</para> + </sect2> + + + <sect2 id="line-pragma"> + <title>LINE pragma</title> + + <indexterm><primary>LINE</primary><secondary>pragma</secondary></indexterm> + <indexterm><primary>pragma</primary><secondary>LINE</secondary></indexterm> + <para>This pragma is similar to C's <literal>#line</literal> + pragma, and is mainly for use in automatically generated Haskell + code. It lets you specify the line number and filename of the + original code; for example</para> + +<programlisting>{-# LINE 42 "Foo.vhs" #-}</programlisting> + + <para>if you'd generated the current file from something called + <filename>Foo.vhs</filename> and this line corresponds to line + 42 in the original. GHC will adjust its error messages to refer + to the line/file named in the <literal>LINE</literal> + pragma.</para> + </sect2> + + <sect2 id="options-pragma"> + <title>OPTIONS_GHC pragma</title> + <indexterm><primary>OPTIONS_GHC</primary> + </indexterm> + <indexterm><primary>pragma</primary><secondary>OPTIONS_GHC</secondary> + </indexterm> + + <para>The <literal>OPTIONS_GHC</literal> pragma is used to specify + additional options that are given to the compiler when compiling + this source file. See <xref linkend="source-file-options"/> for + details.</para> + + <para>Previous versions of GHC accepted <literal>OPTIONS</literal> rather + than <literal>OPTIONS_GHC</literal>, but that is now deprecated.</para> + </sect2> + + <sect2 id="rules"> + <title>RULES pragma</title> + + <para>The RULES pragma lets you specify rewrite rules. It is + described in <xref linkend="rewrite-rules"/>.</para> + </sect2> + + <sect2 id="specialize-pragma"> + <title>SPECIALIZE pragma</title> + + <indexterm><primary>SPECIALIZE pragma</primary></indexterm> + <indexterm><primary>pragma, SPECIALIZE</primary></indexterm> + <indexterm><primary>overloading, death to</primary></indexterm> + + <para>(UK spelling also accepted.) For key overloaded + functions, you can create extra versions (NB: more code space) + specialised to particular types. Thus, if you have an + overloaded function:</para> + +<programlisting> + hammeredLookup :: Ord key => [(key, value)] -> key -> value +</programlisting> + + <para>If it is heavily used on lists with + <literal>Widget</literal> keys, you could specialise it as + follows:</para> + +<programlisting> + {-# SPECIALIZE hammeredLookup :: [(Widget, value)] -> Widget -> value #-} +</programlisting> + + <para>A <literal>SPECIALIZE</literal> pragma for a function can + be put anywhere its type signature could be put.</para> + + <para>A <literal>SPECIALIZE</literal> has the effect of generating + (a) a specialised version of the function and (b) a rewrite rule + (see <xref linkend="rewrite-rules"/>) that rewrites a call to the + un-specialised function into a call to the specialised one.</para> + + <para>The type in a SPECIALIZE pragma can be any type that is less + polymorphic than the type of the original function. In concrete terms, + if the original function is <literal>f</literal> then the pragma +<programlisting> + {-# SPECIALIZE f :: <type> #-} +</programlisting> + is valid if and only if the defintion +<programlisting> + f_spec :: <type> + f_spec = f +</programlisting> + is valid. Here are some examples (where we only give the type signature + for the original function, not its code): +<programlisting> + f :: Eq a => a -> b -> b + {-# SPECIALISE f :: Int -> b -> b #-} + + g :: (Eq a, Ix b) => a -> b -> b + {-# SPECIALISE g :: (Eq a) => a -> Int -> Int #-} + + h :: Eq a => a -> a -> a + {-# SPECIALISE h :: (Eq a) => [a] -> [a] -> [a] #-} +</programlisting> +The last of these examples will generate a +RULE with a somewhat-complex left-hand side (try it yourself), so it might not fire very +well. If you use this kind of specialisation, let us know how well it works. +</para> + +<para>A <literal>SPECIALIZE</literal> pragma can optionally be followed with a +<literal>INLINE</literal> or <literal>NOINLINE</literal> pragma, optionally +followed by a phase, as described in <xref linkend="inline-noinline-pragma"/>. +The <literal>INLINE</literal> pragma affects the specialised verison of the +function (only), and applies even if the function is recursive. The motivating +example is this: +<programlisting> +-- A GADT for arrays with type-indexed representation +data Arr e where + ArrInt :: !Int -> ByteArray# -> Arr Int + ArrPair :: !Int -> Arr e1 -> Arr e2 -> Arr (e1, e2) + +(!:) :: Arr e -> Int -> e +{-# SPECIALISE INLINE (!:) :: Arr Int -> Int -> Int #-} +{-# SPECIALISE INLINE (!:) :: Arr (a, b) -> Int -> (a, b) #-} +(ArrInt _ ba) !: (I# i) = I# (indexIntArray# ba i) +(ArrPair _ a1 a2) !: i = (a1 !: i, a2 !: i) +</programlisting> +Here, <literal>(!:)</literal> is a recursive function that indexes arrays +of type <literal>Arr e</literal>. Consider a call to <literal>(!:)</literal> +at type <literal>(Int,Int)</literal>. The second specialisation will fire, and +the specialised function will be inlined. It has two calls to +<literal>(!:)</literal>, +both at type <literal>Int</literal>. Both these calls fire the first +specialisation, whose body is also inlined. The result is a type-based +unrolling of the indexing function.</para> +<para>Warning: you can make GHC diverge by using <literal>SPECIALISE INLINE</literal> +on an ordinarily-recursive function.</para> + + <para>Note: In earlier versions of GHC, it was possible to provide your own + specialised function for a given type: + +<programlisting> +{-# SPECIALIZE hammeredLookup :: [(Int, value)] -> Int -> value = intLookup #-} +</programlisting> + + This feature has been removed, as it is now subsumed by the + <literal>RULES</literal> pragma (see <xref linkend="rule-spec"/>).</para> + + </sect2> + +<sect2 id="specialize-instance-pragma"> +<title>SPECIALIZE instance pragma +</title> + +<para> +<indexterm><primary>SPECIALIZE pragma</primary></indexterm> +<indexterm><primary>overloading, death to</primary></indexterm> +Same idea, except for instance declarations. For example: + +<programlisting> +instance (Eq a) => Eq (Foo a) where { + {-# SPECIALIZE instance Eq (Foo [(Int, Bar)]) #-} + ... usual stuff ... + } +</programlisting> +The pragma must occur inside the <literal>where</literal> part +of the instance declaration. +</para> +<para> +Compatible with HBC, by the way, except perhaps in the placement +of the pragma. +</para> + +</sect2> + + <sect2 id="unpack-pragma"> + <title>UNPACK pragma</title> + + <indexterm><primary>UNPACK</primary></indexterm> + + <para>The <literal>UNPACK</literal> indicates to the compiler + that it should unpack the contents of a constructor field into + the constructor itself, removing a level of indirection. For + example:</para> + +<programlisting> +data T = T {-# UNPACK #-} !Float + {-# UNPACK #-} !Float +</programlisting> + + <para>will create a constructor <literal>T</literal> containing + two unboxed floats. This may not always be an optimisation: if + the <function>T</function> constructor is scrutinised and the + floats passed to a non-strict function for example, they will + have to be reboxed (this is done automatically by the + compiler).</para> + + <para>Unpacking constructor fields should only be used in + conjunction with <option>-O</option>, in order to expose + unfoldings to the compiler so the reboxing can be removed as + often as possible. For example:</para> + +<programlisting> +f :: T -> Float +f (T f1 f2) = f1 + f2 +</programlisting> + + <para>The compiler will avoid reboxing <function>f1</function> + and <function>f2</function> by inlining <function>+</function> + on floats, but only when <option>-O</option> is on.</para> + + <para>Any single-constructor data is eligible for unpacking; for + example</para> + +<programlisting> +data T = T {-# UNPACK #-} !(Int,Int) +</programlisting> + + <para>will store the two <literal>Int</literal>s directly in the + <function>T</function> constructor, by flattening the pair. + Multi-level unpacking is also supported:</para> + +<programlisting> +data T = T {-# UNPACK #-} !S +data S = S {-# UNPACK #-} !Int {-# UNPACK #-} !Int +</programlisting> + + <para>will store two unboxed <literal>Int#</literal>s + directly in the <function>T</function> constructor. The + unpacker can see through newtypes, too.</para> + + <para>If a field cannot be unpacked, you will not get a warning, + so it might be an idea to check the generated code with + <option>-ddump-simpl</option>.</para> + + <para>See also the <option>-funbox-strict-fields</option> flag, + which essentially has the effect of adding + <literal>{-# UNPACK #-}</literal> to every strict + constructor field.</para> + </sect2> + +</sect1> + +<!-- ======================= REWRITE RULES ======================== --> + +<sect1 id="rewrite-rules"> +<title>Rewrite rules + +<indexterm><primary>RULES pragma</primary></indexterm> +<indexterm><primary>pragma, RULES</primary></indexterm> +<indexterm><primary>rewrite rules</primary></indexterm></title> + +<para> +The programmer can specify rewrite rules as part of the source program +(in a pragma). GHC applies these rewrite rules wherever it can, provided (a) +the <option>-O</option> flag (<xref linkend="options-optimise"/>) is on, +and (b) the <option>-frules-off</option> flag +(<xref linkend="options-f"/>) is not specified. +</para> + +<para> +Here is an example: + +<programlisting> + {-# RULES + "map/map" forall f g xs. map f (map g xs) = map (f.g) xs + #-} +</programlisting> + +</para> + +<sect2> +<title>Syntax</title> + +<para> +From a syntactic point of view: + +<itemizedlist> +<listitem> + +<para> + There may be zero or more rules in a <literal>RULES</literal> pragma. +</para> +</listitem> + +<listitem> + +<para> + Each rule has a name, enclosed in double quotes. The name itself has +no significance at all. It is only used when reporting how many times the rule fired. +</para> +</listitem> + +<listitem> +<para> +A rule may optionally have a phase-control number (see <xref linkend="phase-control"/>), +immediately after the name of the rule. Thus: +<programlisting> + {-# RULES + "map/map" [2] forall f g xs. map f (map g xs) = map (f.g) xs + #-} +</programlisting> +The "[2]" means that the rule is active in Phase 2 and subsequent phases. The inverse +notation "[~2]" is also accepted, meaning that the rule is active up to, but not including, +Phase 2. +</para> +</listitem> + + +<listitem> + +<para> + Layout applies in a <literal>RULES</literal> pragma. Currently no new indentation level +is set, so you must lay out your rules starting in the same column as the +enclosing definitions. +</para> +</listitem> + +<listitem> + +<para> + Each variable mentioned in a rule must either be in scope (e.g. <function>map</function>), +or bound by the <literal>forall</literal> (e.g. <function>f</function>, <function>g</function>, <function>xs</function>). The variables bound by +the <literal>forall</literal> are called the <emphasis>pattern</emphasis> variables. They are separated +by spaces, just like in a type <literal>forall</literal>. +</para> +</listitem> +<listitem> + +<para> + A pattern variable may optionally have a type signature. +If the type of the pattern variable is polymorphic, it <emphasis>must</emphasis> have a type signature. +For example, here is the <literal>foldr/build</literal> rule: + +<programlisting> +"fold/build" forall k z (g::forall b. (a->b->b) -> b -> b) . + foldr k z (build g) = g k z +</programlisting> + +Since <function>g</function> has a polymorphic type, it must have a type signature. + +</para> +</listitem> +<listitem> + +<para> +The left hand side of a rule must consist of a top-level variable applied +to arbitrary expressions. For example, this is <emphasis>not</emphasis> OK: + +<programlisting> +"wrong1" forall e1 e2. case True of { True -> e1; False -> e2 } = e1 +"wrong2" forall f. f True = True +</programlisting> + +In <literal>"wrong1"</literal>, the LHS is not an application; in <literal>"wrong2"</literal>, the LHS has a pattern variable +in the head. +</para> +</listitem> +<listitem> + +<para> + A rule does not need to be in the same module as (any of) the +variables it mentions, though of course they need to be in scope. +</para> +</listitem> +<listitem> + +<para> + Rules are automatically exported from a module, just as instance declarations are. +</para> +</listitem> + +</itemizedlist> + +</para> + +</sect2> + +<sect2> +<title>Semantics</title> + +<para> +From a semantic point of view: + +<itemizedlist> +<listitem> + +<para> +Rules are only applied if you use the <option>-O</option> flag. +</para> +</listitem> + +<listitem> +<para> + Rules are regarded as left-to-right rewrite rules. +When GHC finds an expression that is a substitution instance of the LHS +of a rule, it replaces the expression by the (appropriately-substituted) RHS. +By "a substitution instance" we mean that the LHS can be made equal to the +expression by substituting for the pattern variables. + +</para> +</listitem> +<listitem> + +<para> + The LHS and RHS of a rule are typechecked, and must have the +same type. + +</para> +</listitem> +<listitem> + +<para> + GHC makes absolutely no attempt to verify that the LHS and RHS +of a rule have the same meaning. That is undecidable in general, and +infeasible in most interesting cases. The responsibility is entirely the programmer's! + +</para> +</listitem> +<listitem> + +<para> + GHC makes no attempt to make sure that the rules are confluent or +terminating. For example: + +<programlisting> + "loop" forall x,y. f x y = f y x +</programlisting> + +This rule will cause the compiler to go into an infinite loop. + +</para> +</listitem> +<listitem> + +<para> + If more than one rule matches a call, GHC will choose one arbitrarily to apply. + +</para> +</listitem> +<listitem> +<para> + GHC currently uses a very simple, syntactic, matching algorithm +for matching a rule LHS with an expression. It seeks a substitution +which makes the LHS and expression syntactically equal modulo alpha +conversion. The pattern (rule), but not the expression, is eta-expanded if +necessary. (Eta-expanding the expression can lead to laziness bugs.) +But not beta conversion (that's called higher-order matching). +</para> + +<para> +Matching is carried out on GHC's intermediate language, which includes +type abstractions and applications. So a rule only matches if the +types match too. See <xref linkend="rule-spec"/> below. +</para> +</listitem> +<listitem> + +<para> + GHC keeps trying to apply the rules as it optimises the program. +For example, consider: + +<programlisting> + let s = map f + t = map g + in + s (t xs) +</programlisting> + +The expression <literal>s (t xs)</literal> does not match the rule <literal>"map/map"</literal>, but GHC +will substitute for <varname>s</varname> and <varname>t</varname>, giving an expression which does match. +If <varname>s</varname> or <varname>t</varname> was (a) used more than once, and (b) large or a redex, then it would +not be substituted, and the rule would not fire. + +</para> +</listitem> +<listitem> + +<para> + In the earlier phases of compilation, GHC inlines <emphasis>nothing +that appears on the LHS of a rule</emphasis>, because once you have substituted +for something you can't match against it (given the simple minded +matching). So if you write the rule + +<programlisting> + "map/map" forall f,g. map f . map g = map (f.g) +</programlisting> + +this <emphasis>won't</emphasis> match the expression <literal>map f (map g xs)</literal>. +It will only match something written with explicit use of ".". +Well, not quite. It <emphasis>will</emphasis> match the expression + +<programlisting> +wibble f g xs +</programlisting> + +where <function>wibble</function> is defined: + +<programlisting> +wibble f g = map f . map g +</programlisting> + +because <function>wibble</function> will be inlined (it's small). + +Later on in compilation, GHC starts inlining even things on the +LHS of rules, but still leaves the rules enabled. This inlining +policy is controlled by the per-simplification-pass flag <option>-finline-phase</option><emphasis>n</emphasis>. + +</para> +</listitem> +<listitem> + +<para> + All rules are implicitly exported from the module, and are therefore +in force in any module that imports the module that defined the rule, directly +or indirectly. (That is, if A imports B, which imports C, then C's rules are +in force when compiling A.) The situation is very similar to that for instance +declarations. +</para> +</listitem> + +</itemizedlist> + +</para> + +</sect2> + +<sect2> +<title>List fusion</title> + +<para> +The RULES mechanism is used to implement fusion (deforestation) of common list functions. +If a "good consumer" consumes an intermediate list constructed by a "good producer", the +intermediate list should be eliminated entirely. +</para> + +<para> +The following are good producers: + +<itemizedlist> +<listitem> + +<para> + List comprehensions +</para> +</listitem> +<listitem> + +<para> + Enumerations of <literal>Int</literal> and <literal>Char</literal> (e.g. <literal>['a'..'z']</literal>). +</para> +</listitem> +<listitem> + +<para> + Explicit lists (e.g. <literal>[True, False]</literal>) +</para> +</listitem> +<listitem> + +<para> + The cons constructor (e.g <literal>3:4:[]</literal>) +</para> +</listitem> +<listitem> + +<para> + <function>++</function> +</para> +</listitem> + +<listitem> +<para> + <function>map</function> +</para> +</listitem> + +<listitem> +<para> + <function>filter</function> +</para> +</listitem> +<listitem> + +<para> + <function>iterate</function>, <function>repeat</function> +</para> +</listitem> +<listitem> + +<para> + <function>zip</function>, <function>zipWith</function> +</para> +</listitem> + +</itemizedlist> + +</para> + +<para> +The following are good consumers: + +<itemizedlist> +<listitem> + +<para> + List comprehensions +</para> +</listitem> +<listitem> + +<para> + <function>array</function> (on its second argument) +</para> +</listitem> +<listitem> + +<para> + <function>length</function> +</para> +</listitem> +<listitem> + +<para> + <function>++</function> (on its first argument) +</para> +</listitem> + +<listitem> +<para> + <function>foldr</function> +</para> +</listitem> + +<listitem> +<para> + <function>map</function> +</para> +</listitem> +<listitem> + +<para> + <function>filter</function> +</para> +</listitem> +<listitem> + +<para> + <function>concat</function> +</para> +</listitem> +<listitem> + +<para> + <function>unzip</function>, <function>unzip2</function>, <function>unzip3</function>, <function>unzip4</function> +</para> +</listitem> +<listitem> + +<para> + <function>zip</function>, <function>zipWith</function> (but on one argument only; if both are good producers, <function>zip</function> +will fuse with one but not the other) +</para> +</listitem> +<listitem> + +<para> + <function>partition</function> +</para> +</listitem> +<listitem> + +<para> + <function>head</function> +</para> +</listitem> +<listitem> + +<para> + <function>and</function>, <function>or</function>, <function>any</function>, <function>all</function> +</para> +</listitem> +<listitem> + +<para> + <function>sequence_</function> +</para> +</listitem> +<listitem> + +<para> + <function>msum</function> +</para> +</listitem> +<listitem> + +<para> + <function>sortBy</function> +</para> +</listitem> + +</itemizedlist> + +</para> + + <para> +So, for example, the following should generate no intermediate lists: + +<programlisting> +array (1,10) [(i,i*i) | i <- map (+ 1) [0..9]] +</programlisting> + +</para> + +<para> +This list could readily be extended; if there are Prelude functions that you use +a lot which are not included, please tell us. +</para> + +<para> +If you want to write your own good consumers or producers, look at the +Prelude definitions of the above functions to see how to do so. +</para> + +</sect2> + +<sect2 id="rule-spec"> +<title>Specialisation +</title> + +<para> +Rewrite rules can be used to get the same effect as a feature +present in earlier versions of GHC. +For example, suppose that: + +<programlisting> +genericLookup :: Ord a => Table a b -> a -> b +intLookup :: Table Int b -> Int -> b +</programlisting> + +where <function>intLookup</function> is an implementation of +<function>genericLookup</function> that works very fast for +keys of type <literal>Int</literal>. You might wish +to tell GHC to use <function>intLookup</function> instead of +<function>genericLookup</function> whenever the latter was called with +type <literal>Table Int b -> Int -> b</literal>. +It used to be possible to write + +<programlisting> +{-# SPECIALIZE genericLookup :: Table Int b -> Int -> b = intLookup #-} +</programlisting> + +This feature is no longer in GHC, but rewrite rules let you do the same thing: + +<programlisting> +{-# RULES "genericLookup/Int" genericLookup = intLookup #-} +</programlisting> + +This slightly odd-looking rule instructs GHC to replace +<function>genericLookup</function> by <function>intLookup</function> +<emphasis>whenever the types match</emphasis>. +What is more, this rule does not need to be in the same +file as <function>genericLookup</function>, unlike the +<literal>SPECIALIZE</literal> pragmas which currently do (so that they +have an original definition available to specialise). +</para> + +<para>It is <emphasis>Your Responsibility</emphasis> to make sure that +<function>intLookup</function> really behaves as a specialised version +of <function>genericLookup</function>!!!</para> + +<para>An example in which using <literal>RULES</literal> for +specialisation will Win Big: + +<programlisting> +toDouble :: Real a => a -> Double +toDouble = fromRational . toRational + +{-# RULES "toDouble/Int" toDouble = i2d #-} +i2d (I# i) = D# (int2Double# i) -- uses Glasgow prim-op directly +</programlisting> + +The <function>i2d</function> function is virtually one machine +instruction; the default conversion—via an intermediate +<literal>Rational</literal>—is obscenely expensive by +comparison. +</para> + +</sect2> + +<sect2> +<title>Controlling what's going on</title> + +<para> + +<itemizedlist> +<listitem> + +<para> + Use <option>-ddump-rules</option> to see what transformation rules GHC is using. +</para> +</listitem> +<listitem> + +<para> + Use <option>-ddump-simpl-stats</option> to see what rules are being fired. +If you add <option>-dppr-debug</option> you get a more detailed listing. +</para> +</listitem> +<listitem> + +<para> + The definition of (say) <function>build</function> in <filename>GHC/Base.lhs</filename> looks llike this: + +<programlisting> + build :: forall a. (forall b. (a -> b -> b) -> b -> b) -> [a] + {-# INLINE build #-} + build g = g (:) [] +</programlisting> + +Notice the <literal>INLINE</literal>! That prevents <literal>(:)</literal> from being inlined when compiling +<literal>PrelBase</literal>, so that an importing module will “see” the <literal>(:)</literal>, and can +match it on the LHS of a rule. <literal>INLINE</literal> prevents any inlining happening +in the RHS of the <literal>INLINE</literal> thing. I regret the delicacy of this. + +</para> +</listitem> +<listitem> + +<para> + In <filename>libraries/base/GHC/Base.lhs</filename> look at the rules for <function>map</function> to +see how to write rules that will do fusion and yet give an efficient +program even if fusion doesn't happen. More rules in <filename>GHC/List.lhs</filename>. +</para> +</listitem> + +</itemizedlist> + +</para> + +</sect2> + +<sect2 id="core-pragma"> + <title>CORE pragma</title> + + <indexterm><primary>CORE pragma</primary></indexterm> + <indexterm><primary>pragma, CORE</primary></indexterm> + <indexterm><primary>core, annotation</primary></indexterm> + +<para> + The external core format supports <quote>Note</quote> annotations; + the <literal>CORE</literal> pragma gives a way to specify what these + should be in your Haskell source code. Syntactically, core + annotations are attached to expressions and take a Haskell string + literal as an argument. The following function definition shows an + example: + +<programlisting> +f x = ({-# CORE "foo" #-} show) ({-# CORE "bar" #-} x) +</programlisting> + + Semantically, this is equivalent to: + +<programlisting> +g x = show x +</programlisting> +</para> + +<para> + However, when external for is generated (via + <option>-fext-core</option>), there will be Notes attached to the + expressions <function>show</function> and <varname>x</varname>. + The core function declaration for <function>f</function> is: +</para> + +<programlisting> + f :: %forall a . GHCziShow.ZCTShow a -> + a -> GHCziBase.ZMZN GHCziBase.Char = + \ @ a (zddShow::GHCziShow.ZCTShow a) (eta::a) -> + (%note "foo" + %case zddShow %of (tpl::GHCziShow.ZCTShow a) + {GHCziShow.ZCDShow + (tpl1::GHCziBase.Int -> + a -> + GHCziBase.ZMZN GHCziBase.Char -> GHCziBase.ZMZN GHCziBase.Cha +r) + (tpl2::a -> GHCziBase.ZMZN GHCziBase.Char) + (tpl3::GHCziBase.ZMZN a -> + GHCziBase.ZMZN GHCziBase.Char -> GHCziBase.ZMZN GHCziBase.Cha +r) -> + tpl2}) + (%note "foo" + eta); +</programlisting> + +<para> + Here, we can see that the function <function>show</function> (which + has been expanded out to a case expression over the Show dictionary) + has a <literal>%note</literal> attached to it, as does the + expression <varname>eta</varname> (which used to be called + <varname>x</varname>). +</para> + +</sect2> + +</sect1> + +<sect1 id="generic-classes"> +<title>Generic classes</title> + + <para>(Note: support for generic classes is currently broken in + GHC 5.02).</para> + +<para> +The ideas behind this extension are described in detail in "Derivable type classes", +Ralf Hinze and Simon Peyton Jones, Haskell Workshop, Montreal Sept 2000, pp94-105. +An example will give the idea: +</para> + +<programlisting> + import Generics + + class Bin a where + toBin :: a -> [Int] + fromBin :: [Int] -> (a, [Int]) + + toBin {| Unit |} Unit = [] + toBin {| a :+: b |} (Inl x) = 0 : toBin x + toBin {| a :+: b |} (Inr y) = 1 : toBin y + toBin {| a :*: b |} (x :*: y) = toBin x ++ toBin y + + fromBin {| Unit |} bs = (Unit, bs) + fromBin {| a :+: b |} (0:bs) = (Inl x, bs') where (x,bs') = fromBin bs + fromBin {| a :+: b |} (1:bs) = (Inr y, bs') where (y,bs') = fromBin bs + fromBin {| a :*: b |} bs = (x :*: y, bs'') where (x,bs' ) = fromBin bs + (y,bs'') = fromBin bs' +</programlisting> +<para> +This class declaration explains how <literal>toBin</literal> and <literal>fromBin</literal> +work for arbitrary data types. They do so by giving cases for unit, product, and sum, +which are defined thus in the library module <literal>Generics</literal>: +</para> +<programlisting> + data Unit = Unit + data a :+: b = Inl a | Inr b + data a :*: b = a :*: b +</programlisting> +<para> +Now you can make a data type into an instance of Bin like this: +<programlisting> + instance (Bin a, Bin b) => Bin (a,b) + instance Bin a => Bin [a] +</programlisting> +That is, just leave off the "where" clause. Of course, you can put in the +where clause and over-ride whichever methods you please. +</para> + + <sect2> + <title> Using generics </title> + <para>To use generics you need to</para> + <itemizedlist> + <listitem> + <para>Use the flags <option>-fglasgow-exts</option> (to enable the extra syntax), + <option>-fgenerics</option> (to generate extra per-data-type code), + and <option>-package lang</option> (to make the <literal>Generics</literal> library + available. </para> + </listitem> + <listitem> + <para>Import the module <literal>Generics</literal> from the + <literal>lang</literal> package. This import brings into + scope the data types <literal>Unit</literal>, + <literal>:*:</literal>, and <literal>:+:</literal>. (You + don't need this import if you don't mention these types + explicitly; for example, if you are simply giving instance + declarations.)</para> + </listitem> + </itemizedlist> + </sect2> + +<sect2> <title> Changes wrt the paper </title> +<para> +Note that the type constructors <literal>:+:</literal> and <literal>:*:</literal> +can be written infix (indeed, you can now use +any operator starting in a colon as an infix type constructor). Also note that +the type constructors are not exactly as in the paper (Unit instead of 1, etc). +Finally, note that the syntax of the type patterns in the class declaration +uses "<literal>{|</literal>" and "<literal>|}</literal>" brackets; curly braces +alone would ambiguous when they appear on right hand sides (an extension we +anticipate wanting). +</para> +</sect2> + +<sect2> <title>Terminology and restrictions</title> +<para> +Terminology. A "generic default method" in a class declaration +is one that is defined using type patterns as above. +A "polymorphic default method" is a default method defined as in Haskell 98. +A "generic class declaration" is a class declaration with at least one +generic default method. +</para> + +<para> +Restrictions: +<itemizedlist> +<listitem> +<para> +Alas, we do not yet implement the stuff about constructor names and +field labels. +</para> +</listitem> + +<listitem> +<para> +A generic class can have only one parameter; you can't have a generic +multi-parameter class. +</para> +</listitem> + +<listitem> +<para> +A default method must be defined entirely using type patterns, or entirely +without. So this is illegal: +<programlisting> + class Foo a where + op :: a -> (a, Bool) + op {| Unit |} Unit = (Unit, True) + op x = (x, False) +</programlisting> +However it is perfectly OK for some methods of a generic class to have +generic default methods and others to have polymorphic default methods. +</para> +</listitem> + +<listitem> +<para> +The type variable(s) in the type pattern for a generic method declaration +scope over the right hand side. So this is legal (note the use of the type variable ``p'' in a type signature on the right hand side: +<programlisting> + class Foo a where + op :: a -> Bool + op {| p :*: q |} (x :*: y) = op (x :: p) + ... +</programlisting> +</para> +</listitem> + +<listitem> +<para> +The type patterns in a generic default method must take one of the forms: +<programlisting> + a :+: b + a :*: b + Unit +</programlisting> +where "a" and "b" are type variables. Furthermore, all the type patterns for +a single type constructor (<literal>:*:</literal>, say) must be identical; they +must use the same type variables. So this is illegal: +<programlisting> + class Foo a where + op :: a -> Bool + op {| a :+: b |} (Inl x) = True + op {| p :+: q |} (Inr y) = False +</programlisting> +The type patterns must be identical, even in equations for different methods of the class. +So this too is illegal: +<programlisting> + class Foo a where + op1 :: a -> Bool + op1 {| a :*: b |} (x :*: y) = True + + op2 :: a -> Bool + op2 {| p :*: q |} (x :*: y) = False +</programlisting> +(The reason for this restriction is that we gather all the equations for a particular type consructor +into a single generic instance declaration.) +</para> +</listitem> + +<listitem> +<para> +A generic method declaration must give a case for each of the three type constructors. +</para> +</listitem> + +<listitem> +<para> +The type for a generic method can be built only from: + <itemizedlist> + <listitem> <para> Function arrows </para> </listitem> + <listitem> <para> Type variables </para> </listitem> + <listitem> <para> Tuples </para> </listitem> + <listitem> <para> Arbitrary types not involving type variables </para> </listitem> + </itemizedlist> +Here are some example type signatures for generic methods: +<programlisting> + op1 :: a -> Bool + op2 :: Bool -> (a,Bool) + op3 :: [Int] -> a -> a + op4 :: [a] -> Bool +</programlisting> +Here, op1, op2, op3 are OK, but op4 is rejected, because it has a type variable +inside a list. +</para> +<para> +This restriction is an implementation restriction: we just havn't got around to +implementing the necessary bidirectional maps over arbitrary type constructors. +It would be relatively easy to add specific type constructors, such as Maybe and list, +to the ones that are allowed.</para> +</listitem> + +<listitem> +<para> +In an instance declaration for a generic class, the idea is that the compiler +will fill in the methods for you, based on the generic templates. However it can only +do so if + <itemizedlist> + <listitem> + <para> + The instance type is simple (a type constructor applied to type variables, as in Haskell 98). + </para> + </listitem> + <listitem> + <para> + No constructor of the instance type has unboxed fields. + </para> + </listitem> + </itemizedlist> +(Of course, these things can only arise if you are already using GHC extensions.) +However, you can still give an instance declarations for types which break these rules, +provided you give explicit code to override any generic default methods. +</para> +</listitem> + +</itemizedlist> +</para> + +<para> +The option <option>-ddump-deriv</option> dumps incomprehensible stuff giving details of +what the compiler does with generic declarations. +</para> + +</sect2> + +<sect2> <title> Another example </title> +<para> +Just to finish with, here's another example I rather like: +<programlisting> + class Tag a where + nCons :: a -> Int + nCons {| Unit |} _ = 1 + nCons {| a :*: b |} _ = 1 + nCons {| a :+: b |} _ = nCons (bot::a) + nCons (bot::b) + + tag :: a -> Int + tag {| Unit |} _ = 1 + tag {| a :*: b |} _ = 1 + tag {| a :+: b |} (Inl x) = tag x + tag {| a :+: b |} (Inr y) = nCons (bot::a) + tag y +</programlisting> +</para> +</sect2> +</sect1> + + + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> + diff --git a/docs/users_guide/gone_wrong.xml b/docs/users_guide/gone_wrong.xml new file mode 100644 index 0000000000..d31087c164 --- /dev/null +++ b/docs/users_guide/gone_wrong.xml @@ -0,0 +1,213 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="wrong"> + <title>What to do when something goes wrong</title> + + <indexterm><primary>problems</primary></indexterm> + + <para>If you still have a problem after consulting this section, + then you may have found a <emphasis>bug</emphasis>—please + report it! See <xref linkend="bug-reporting"/> for details on how to + report a bug and a list of things we'd like to know about your bug. + If in doubt, send a report—we love mail from irate users + :-!</para> + + <para>(<xref linkend="vs-Haskell-defn"/>, which describes Glasgow + Haskell's shortcomings vs. the Haskell language definition, may + also be of interest.)</para> + + <sect1 id="wrong-compiler"> + <title>When the compiler “does the wrong thing”</title> + + <indexterm><primary>compiler problems</primary></indexterm> + <indexterm><primary>problems with the compiler</primary></indexterm> + + <variablelist> + <varlistentry> + <term>“Help! The compiler crashed (or `panic'd)!”</term> + <listitem> + <para>These events are <emphasis>always</emphasis> bugs in + the GHC system—please report them.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“This is a terrible error message.”</term> + <listitem> + <para>If you think that GHC could have produced a better + error message, please report it as a bug.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“What about this warning from the C + compiler?”</term> + <listitem> + <para>For example: “…warning: `Foo' declared + `static' but never defined.” Unsightly, but shouldn't + be a problem.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Sensitivity to <filename>.hi</filename> interface files:</term> + <listitem> + <para>GHC is very sensitive about interface files. For + example, if it picks up a non-standard + <filename>Prelude.hi</filename> file, pretty terrible things + will happen. If you turn on + <option>-fno-implicit-prelude</option><indexterm><primary>-fno-implicit-prelude + option</primary></indexterm>, the compiler will almost + surely die, unless you know what you are doing.</para> + + <para>Furthermore, as sketched below, you may have big + problems running programs compiled using unstable + interfaces.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“I think GHC is producing incorrect code”:</term> + <listitem> + <para>Unlikely :-) A useful be-more-paranoid option to give + to GHC is + <option>-dcore-lint</option><indexterm><primary>-dcore-lint + option</primary></indexterm>; this causes a + “lint” pass to check for errors (notably type + errors) after each Core-to-Core transformation pass. We run + with <option>-dcore-lint</option> on all the time; it costs + about 5% in compile time.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“Why did I get a link error?”</term> + <listitem> + <para>If the linker complains about not finding + <literal>_<something>_fast</literal>, + then something is inconsistent: you probably didn't compile + modules in the proper dependency order.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“Is this line number right?”</term> + <listitem> + <para>On this score, GHC usually does pretty well, + especially if you “allow” it to be off by one or + two. In the case of an instance or class declaration, the + line number may only point you to the declaration, not to a + specific method.</para> + + <para>Please report line-number errors that you find + particularly unhelpful.</para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + <sect1 id="wrong-compilee"> + <title>When your program “does the wrong thing”</title> + + <indexterm><primary>problems running your program</primary></indexterm> + + <para>(For advice about overly slow or memory-hungry Haskell + programs, please see <xref + linkend="sooner-faster-quicker"/>).</para> + + <variablelist> + + <varlistentry> + <term>“Help! My program crashed!”</term> + <listitem> + <para>(e.g., a `segmentation fault' or `core dumped') + <indexterm><primary>segmentation + fault</primary></indexterm></para> + + <para>If your program has no foreign calls in it, and no + calls to known-unsafe functions (such as + <literal>unsafePerformIO</literal>) then a crash is always a + BUG in the GHC system, except in one case: If your program + is made of several modules, each module must have been + compiled after any modules on which it depends (unless you + use <filename>.hi-boot</filename> files, in which case these + <emphasis>must</emphasis> be correct with respect to the + module source).</para> + + <para>For example, if an interface is lying about the type + of an imported value then GHC may well generate duff code + for the importing module. <emphasis>This applies to pragmas + inside interfaces too!</emphasis> If the pragma is lying + (e.g., about the “arity” of a value), then duff + code may result. Furthermore, arities may change even if + types do not.</para> + + <para>In short, if you compile a module and its interface + changes, then all the modules that import that interface + <emphasis>must</emphasis> be re-compiled.</para> + + <para>A useful option to alert you when interfaces change is + <option>-hi-diffs</option><indexterm><primary>-hi-diffs + option</primary></indexterm>. It will run + <command>diff</command> on the changed interface file, + before and after, when applicable.</para> + + <para>If you are using <command>make</command>, GHC can + automatically generate the dependencies required in order to + make sure that every module <emphasis>is</emphasis> + up-to-date with respect to its imported interfaces. Please + see <xref linkend="sec-makefile-dependencies"/>.</para> + + <para>If you are down to your + last-compile-before-a-bug-report, we would recommend that + you add a <option>-dcore-lint</option> option (for extra + checking) to your compilation options.</para> + + <para>So, before you report a bug because of a core dump, + you should probably:</para> + +<screen> +% rm *.o # scrub your object files +% make my_prog # re-make your program; use -hi-diffs to highlight changes; + # as mentioned above, use -dcore-lint to be more paranoid +% ./my_prog ... # retry... +</screen> + + <para>Of course, if you have foreign calls in your program + then all bets are off, because you can trash the heap, the + stack, or whatever.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“My program entered an `absent' argument.”</term> + <listitem> + <para>This is definitely caused by a bug in GHC. Please + report it (see <xref linkend="bug-reporting"/>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“What's with this `arithmetic (or `floating') + exception' ”?</term> + <listitem> + <para><literal>Int</literal>, <literal>Float</literal>, and + <literal>Double</literal> arithmetic is + <emphasis>unchecked</emphasis>. Overflows, underflows and + loss of precision are either silent or reported as an + exception by the operating system (depending on the + platform). Divide-by-zero <emphasis>may</emphasis> cause an + untrapped exception (please report it if it does).</para> + </listitem> + </varlistentry> + + </variablelist> + </sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/installing.xml b/docs/users_guide/installing.xml new file mode 100644 index 0000000000..9f8e4c9eb8 --- /dev/null +++ b/docs/users_guide/installing.xml @@ -0,0 +1,875 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="sec-installing-bin-distrib"> + <title>Installing GHC</title> +<indexterm><primary>binary installations</primary></indexterm> +<indexterm><primary>installation, of binaries</primary></indexterm> + +<para> +Installing from binary distributions is easiest, and recommended! +(Why binaries? Because GHC is a Haskell compiler written in Haskell, +so you've got to bootstrap it somehow. We provide machine-generated +C-files-from-Haskell for this purpose, but it's really quite a pain to +use them. If you must build GHC from its sources, using a +binary-distributed GHC to do so is a sensible way to proceed. For the +other <literal>fptools</literal> programs, many are written in +Haskell, so binary distributions allow you to install them without +having a Haskell compiler.) +</para> + +<para>This guide is in several parts:</para> + + <itemizedlist> + <listitem> + <para> Installing on Unix-a-likes (<xref + linkend="sec-unix-a-likes"/>). </para> + </listitem> + <listitem> + <para> Installing on Windows (<xref + linkend="sec-install-windows"/>). </para> + </listitem> + <listitem> + <para> The layout of installed files (<xref + linkend="sec-install-files"/>). You don't need to know this to + install GHC, but it's useful if you are changing the + implementation.</para> + </listitem> + </itemizedlist> + + <sect1 id="sec-unix-a-likes"><title>Installing on Unix-a-likes</title> + + <sect2> + <title>When a platform-specific package is available</title> + + <para>For certain platforms, we provide GHC binaries packaged + using the native package format for the platform. This is + likely to be by far the best way to install GHC for your + platform if one of these packages is available, since + dependencies will automatically be handled and the package + system normally provides a way to uninstall the package at a + later date.</para> + + <para>We generally provide the following packages:</para> + + <variablelist> + <varlistentry> + <term>RedHat or SuSE Linux/x86</term> + <listitem> + <para>RPM source & binary packages for RedHat and SuSE + Linux (x86 only) are available for most major + releases.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Debian Linux/x86</term> + <listitem> + <para>Debian packages for Linux (x86 only), also for most + major releases.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>FreeBSD/x86</term> + <listitem> + <para>On FreeBSD/x86, GHC can be installed using either + the ports tree (<literal>cd /usr/ports/lang/ghc && make + install</literal>) or from a pre-compiled package + available from your local FreeBSD mirror.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Other platform-specific packages may be available, check + the GHC download page for details.</para> + </sect2> + +<sect2> +<title>GHC binary distributions</title> + +<para> +<indexterm><primary>bundles of binary stuff</primary></indexterm> +</para> + +<para> +Binary distributions come in “bundles,” one bundle per file called +<literal><replaceable>bundle</replaceable>-<replaceable>platform</replaceable>.tar.gz</literal>. (See the building guide for the definition of a platform.) Suppose that you untar a binary-distribution bundle, thus: +</para> + +<para> + +<screen> +% cd /your/scratch/space +% gunzip < ghc-x.xx-sun-sparc-solaris2.tar.gz | tar xvf -</screen> + +</para> + +<para> +Then you should find a single directory, +<literal>ghc-<replaceable>version</replaceable></literal>, with the +following structure: +</para> + +<para> +<indexterm><primary>binary distribution, layout</primary></indexterm> +<indexterm><primary>directory layout (binary distributions)</primary></indexterm> +<variablelist> + +<varlistentry> +<term><literal>Makefile.in</literal></term> +<listitem> +<para> +the raw material from which the <literal>Makefile</literal> +will be made (<xref linkend="sec-install"/>). +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>configure</literal></term> +<listitem> +<para> +the configuration script (<xref linkend="sec-install"/>). +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>README</literal></term> +<listitem> +<para> +Contains this file summary. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>INSTALL</literal></term> +<listitem> +<para> +Contains this description of how to install +the bundle. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>ANNOUNCE</literal></term> +<listitem> +<para> +The announcement message for the bundle. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>NEWS</literal></term> +<listitem> +<para> +release notes for the bundle—a longer version +of <literal>ANNOUNCE</literal>. For GHC, the release notes are contained in the User +Guide and this file isn't present. +</para> +</listitem></varlistentry> +<varlistentry> + <term><literal>bin/<replaceable>platform</replaceable></literal></term> +<listitem> +<para> +contains platform-specific executable +files to be invoked directly by the user. These are the files that +must end up in your path. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>lib/<replaceable>platform</replaceable>/</literal></term> +<listitem> +<para> +contains platform-specific support +files for the installation. Typically there is a subdirectory for +each <literal>fptools</literal> project, whose name is the name of the project with its +version number. For example, for GHC there would be a sub-directory +<literal>ghc-x.xx</literal>/ where <literal>x.xx</literal> is the version number of GHC in the bundle. +</para> + +<para> +These sub-directories have the following general structure: +</para> + +<para> +<variablelist> + +<varlistentry> +<term><literal>libHSstd.a</literal> etc:</term> +<listitem> +<para> +supporting library archives. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>ghc-iface.prl</literal> etc:</term> +<listitem> +<para> +support scripts. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>import/</literal></term> +<listitem> +<para> +<indexterm><primary>Interface files</primary></indexterm> (<literal>.hi</literal>) for the prelude. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>include/</literal></term> +<listitem> +<para> +A few C <literal>#include</literal> files. +</para> +</listitem></varlistentry> +</variablelist> +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>share/</literal></term> +<listitem> +<para> +contains platform-independent support files +for the installation. Again, there is a sub-directory for each +<literal>fptools</literal> project. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>html/</literal></term> +<listitem> +<para> +contains HTML documentation files (one +sub-directory per project). +</para> +</listitem></varlistentry> +</variablelist> +</para> + +<sect3 id="sec-install"> +<title>Installing</title> + +<para> +OK, so let's assume that you have unpacked your chosen bundles. What +next? Well, you will at least need to run the +<literal>configure</literal><indexterm><primary>configure</primary></indexterm> +script by changing directory into the top-level directory for the +bundle and typing <literal>./configure</literal>. That should convert +<literal>Makefile.in</literal> to <literal>Makefile</literal>. +</para> + +<para> +<indexterm><primary>installing in-place</primary></indexterm> +<indexterm><primary>in-place installation</primary></indexterm> +You can now either start using the tools <emphasis>in-situ</emphasis> without going +through any installation process, just type <literal>make in-place</literal> to set the +tools up for this. You'll also want to add the path which <literal>make</literal> will +now echo to your <literal>PATH</literal> environment variable. This option is useful if +you simply want to try out the package and/or you don't have the +necessary privileges (or inclination) to properly install the tools +locally. Note that if you do decide to install the package `properly' +at a later date, you have to go through the installation steps that +follow. +</para> + +<para> +To install a package, you'll have to do the following: +</para> + +<para> + +<orderedlist> +<listitem> + +<para> + Edit the <literal>Makefile</literal> and check the settings of the following variables: + +<indexterm><primary>directories, installation</primary></indexterm> +<indexterm><primary>installation directories</primary></indexterm> + +<variablelist> + +<varlistentry> +<term><literal>platform</literal></term> +<listitem> +<para> +the platform you are going to install for. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>bindir</literal></term> +<listitem> +<para> +the directory in which to install user-invokable +binaries. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>libdir</literal></term> +<listitem> +<para> +the directory in which to install +platform-dependent support files. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>datadir</literal></term> +<listitem> +<para> +the directory in which to install +platform-independent support files. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>infodir</literal></term> +<listitem> +<para> +the directory in which to install Emacs info +files. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>htmldir</literal></term> +<listitem> +<para> +the directory in which to install HTML +documentation. +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>dvidir</literal></term> +<listitem> +<para> +the directory in which to install DVI +documentation. +</para> +</listitem></varlistentry> +</variablelist> + +The values for these variables can be set through invocation of the +<command>configure</command><indexterm><primary>configure</primary></indexterm> +script that comes with the distribution, but doing an optical diff to +see if the values match your expectations is always a Good Idea. +</para> + +<para> +<emphasis>Instead of running <command>configure</command>, it is +perfectly OK to copy <filename>Makefile.in</filename> to +<filename>Makefile</filename> and set all these variables directly +yourself. But do it right!</emphasis> +</para> + +</listitem> +<listitem> + +<para> +Run <literal>make install</literal>. This <emphasis> +should</emphasis> work with ordinary Unix +<literal>make</literal>—no need for fancy stuff like GNU +<literal>make</literal>. + +</para> +</listitem> +<listitem> + +<para> +<literal>rehash</literal> (t?csh or zsh users), so your shell will see the new +stuff in your bin directory. + +</para> +</listitem> +<listitem> + +<para> + Once done, test your “installation” as suggested in +<xref linkend="sec-GHC-test"/>. Be sure to use a <literal>-v</literal> +option, so you can see exactly what pathnames it's using. + +If things don't work as expected, check the list of known pitfalls in +the building guide. +</para> +</listitem> + +</orderedlist> + +</para> + +<para> +<indexterm><primary>link, installed as ghc</primary></indexterm> +When installing the user-invokable binaries, this installation +procedure will install GHC as <literal>ghc-x.xx</literal> where <literal>x.xx</literal> is the version +number of GHC. It will also make a link (in the binary installation +directory) from <literal>ghc</literal> to <literal>ghc-x.xx</literal>. If you install multiple versions +of GHC then the last one “wins”, and “<literal>ghc</literal>” will invoke the last +one installed. You can change this manually if you want. But +regardless, <literal>ghc-x.xx</literal> should always invoke GHC version <literal>x.xx</literal>. +</para> + +</sect3> + + +<sect3> +<title>What bundles there are</title> + +<para> +<indexterm><primary>bundles, binary</primary></indexterm> There are +plenty of “non-basic” GHC bundles. The files for them are +called +<literal>ghc-x.xx-<replaceable>bundle</replaceable>-<replaceable>platform</replaceable>.tar.gz</literal>, +where the <replaceable>platform</replaceable> is as above, and +<replaceable>bundle</replaceable> is one of these: +</para> + +<para> +<variablelist> + +<varlistentry> +<term><literal>prof</literal>:</term> +<listitem> +<para> +Profiling with cost-centres. You probably want this. +<indexterm><primary>profiling bundles</primary></indexterm> +<indexterm><primary>bundles, profiling</primary></indexterm> +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>par</literal>:</term> +<listitem> +<para> +Parallel Haskell features (sits on top of PVM). +You'll want this if you're into that kind of thing. +<indexterm><primary>parallel bundles</primary></indexterm> +<indexterm><primary>bundles, parallel</primary></indexterm> +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>gran</literal>:</term> +<listitem> +<para> +The “GranSim” parallel-Haskell simulator +(hmm… mainly for implementors). +<indexterm><primary>bundles, gransim</primary></indexterm> +<indexterm><primary>gransim bundles</primary></indexterm> +</para> +</listitem></varlistentry> +<varlistentry> +<term><literal>ticky</literal>:</term> +<listitem> +<para> +“Ticky-ticky” profiling; very detailed +information about “what happened when I ran this program”—really +for implementors. +<indexterm><primary>bundles, ticky-ticky</primary></indexterm> +<indexterm><primary>ticky-ticky bundles</primary></indexterm> +</para> +</listitem></varlistentry> +</variablelist> +</para> + +<para> +One likely scenario is that you will grab <emphasis>two</emphasis> +binary bundles—basic, and profiling. We don't usually make the +rest, although you can build them yourself from a source distribution. +</para> + +<para>The various GHC bundles are designed to be unpacked into the +same directory; then installing as per the directions above will +install the whole lot in one go. Note: you <emphasis>must</emphasis> +at least have the basic GHC binary distribution bundle, these extra +bundles won't install on their own.</para> + +</sect3> + +<sect3 id="sec-GHC-test"> +<title>Testing that GHC seems to be working +</title> + +<para> +<indexterm><primary>testing a new GHC</primary></indexterm> +</para> + +<para> +The way to do this is, of course, to compile and run <emphasis>this</emphasis> program +(in a file <literal>Main.hs</literal>): +</para> + +<para> + +<programlisting> +main = putStr "Hello, world!\n" +</programlisting> + +</para> + +<para> +Compile the program, using the <literal>-v</literal> (verbose) flag to verify that +libraries, etc., are being found properly: + +<screen> +% ghc -v -o hello Main.hs</screen> + +</para> + +<para> +Now run it: + +<screen> +% ./hello +Hello, world!</screen> + +</para> + +<para> +Some simple-but-profitable tests are to compile and run the notorious +<literal>nfib</literal><indexterm><primary>nfib</primary></indexterm> program, using different numeric types. Start with +<literal>nfib :: Int -> Int</literal>, and then try <literal>Integer</literal>, <literal>Float</literal>, <literal>Double</literal>, +<literal>Rational</literal> and perhaps the overloaded version. Code for this is +distributed in <literal>ghc/misc/examples/nfib/</literal> in a source distribution. +</para> + +<para>For more information on how to “drive” GHC, read +on...</para> + +</sect3> + +</sect2> + +</sect1> + + +<sect1 id="sec-install-windows"><title>Installing on Windows</title> + +<para> +Getting the Glasgow Haskell Compiler (post 5.02) to run on Windows platforms is +a snap: the Installshield does everything you need. +</para> + +<sect2><title>Installing GHC on Windows</title> + +<para> +To install GHC, use the following steps: +</para> +<itemizedlist> +<listitem><para>Download the Installshield <filename>setup.exe</filename> +from the GHC download page +<ulink +url="http://www.haskell.org/ghc">haskell.org</ulink>. +</para></listitem> + +<listitem><para>Run <filename>setup.exe</filename>. +On Windows, all of GHC's files are installed in a single directory. +If you choose ``Custom'' from the list of install options, you will be given a +choice about where this directory is; otherwise it will be installed +in <filename>c:/ghc/<replaceable>ghc-version</replaceable></filename>. +The executable binary for GHC will be installed in the <filename>bin/</filename> sub-directory +of the installation directory you choose. +</para> +<para>(If you have already installed the same version of GHC, Installshield will offer to "modify", +or "remove" GHC. Choose "remove"; then run <filename>setup.exe</filename> a +second time. This time it should offer to install.) +</para> +<para> +When installation is complete, you should find GHCi and the GHC documentation are +available in your Start menu under "Start/Programs/Glasgow Haskell Compiler". +</para> +</listitem> + +<listitem><para> +The final dialogue box from the install process reminds you where the GHC binary +has been installed (usually <filename>c:/ghc/<replaceable>ghc-version</replaceable>/bin/</filename>. +If you want to invoke GHC from a command line, add this +to your PATH environment variable. +</para></listitem> + +<listitem><para> +GHC needs a directory in which to create, and later delete, temporary files. +It uses the standard Windows procedure <literal>GetTempPath()</literal> to +find a suitable directory. This procedure returns: +<itemizedlist> +<listitem><para>The path in environment variable TMP, +if TMP is set.</para></listitem> +<listitem><para>Otherwise, the path in environment variable TEMP, +if TEMP is set.</para></listitem> +<listitem><para>Otherwise, there is a per-user default which varies +between versions of Windows. On NT and XP-ish versions, it might +be: +<filename>c:\Documents and Settings\<username>\Local Settings\Temp</filename> +</para></listitem> +</itemizedlist> +The main point is that if you don't do anything GHC will work fine; +but if you want to control where the directory is, you can do so by +setting TMP or TEMP. +</para></listitem> + +<listitem> +<para> +To test the fruits of your labour, try now to compile a simple +Haskell program: +</para> + +<screen> +bash$ cat main.hs +module Main(main) where + +main = putStrLn "Hello, world!" +bash$ ghc -o main main.hs +.. +bash$ ./main +Hello, world! +bash$ </screen> +</listitem> +</itemizedlist> + +<para> +You do <emphasis>not</emphasis> need the Cygwin toolchain, or anything +else, to install and run GHC. +</para> +<para> +An installation of GHC requires about 140M of disk space. +To run GHC comfortably, your machine should have at least +64M of memory. +</para> +</sect2> + +<sect2><title>Moving GHC around</title> +<para> +At the moment, GHC installs in a fixed place (<filename>c:/ghc/ghc-x.yy</filename>, +but once it is installed, you can freely move the entire GHC tree just by copying +the <filename>ghc-x.yy</filename> directory. (You may need to fix up +the links in "Start/Programs/Glasgow Haskell Compiler" if you do this.) +</para> +<para> +It is OK to put GHC tree in a directory whose path involves spaces. However, +don't do this if you use want to use GHC with the Cygwin tools, +because Cygwin can get confused when this happens. +We havn't quite got to the bottom of this, but so far as we know it's not +a problem with GHC itself. Nevertheless, just to keep life simple we usually +put GHC in a place with a space-free path. +</para> +</sect2> + +<sect2 id="winfaq"> +<title>Installing ghc-win32 FAQ</title> + + <variablelist> + <varlistentry> + <term>I'm having trouble with symlinks.</term> + <listitem> + <para>Symlinks only work under Cygwin (<xref linkend="sec-install" />), so binaries not linked to the Cygwin + DLL, in particular those built for Mingwin, will not work with + symlinks.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>I'm getting “permission denied” messages from the + <command>rm</command> or <command>mv</command>.</term> + <listitem> + <para>This can have various causes: trying to rename a directory + when an Explorer window is open on it tends to fail. Closing the + window generally cures the problem, but sometimes its cause is + more mysterious, and logging off and back on or rebooting may be + the quickest cure.</para> + </listitem> + </varlistentry> + </variablelist> + +<!-- doesn't add much value any longer; leave out [sof 7/2002]. +<para> +Further information on using GHC under Windows can be found in <ulink +url="http://www.dcs.gla.ac.uk/~sof/ghc-win32.html">Sigbjørn Finne's +pages</ulink>. Note: ignore the installation instructions, which are rather +out of date; the <emphasis>Miscellaneous</emphasis> section at the bottom of +the page is of most interest, covering topics beyond the scope of this +manual. +</para> +--> +</sect2> + +</sect1> + + +<sect1 id="sec-install-files"><title>The layout of installed files</title> + +<para> +This section describes what files get installed where. You don't need to know it +if you are simply installing GHC, but it is vital information if you are changing +the implementation. +</para> +<para> GHC is installed in two directory trees:</para> +<variablelist> +<varlistentry> +<term>Library directory,</term> +<listitem> <para> known as <filename>$(libdir)</filename>, holds all the +support files needed to run GHC. On Unix, this +directory is usually something like <filename>/usr/lib/ghc/ghc-5.02</filename>. </para> +</listitem> +</varlistentry> +<varlistentry> +<term>Binary directory</term> +<listitem> <para> known as <filename>$(bindir)</filename>, holds executables that +the user is expected to invoke. +Notably, it contains +<filename>ghc</filename> and <filename>ghci</filename>. On Unix, this directory +can be anywhere, but is typically something like <filename>/usr/local/bin</filename>. On Windows, +however, this directory <emphasis>must be</emphasis> <filename>$(libdir)/bin</filename>. +</para> +</listitem> +</varlistentry> +</variablelist> + +<para> +When GHC runs, it must know where its library directory is. +It finds this out in one of two ways: +</para> +<itemizedlist> +<listitem> +<para> +<filename>$(libdir)</filename> is passed to GHC using the <option>-B</option> flag. +On Unix (but not Windows), the installed <filename>ghc</filename> is just a one-line +shell script that invokes the real GHC, passing a suitable <option>-B</option> flag. +[All the user-supplied flags +follow, and a later <option>-B</option> flag overrides an earlier one, so a user-supplied +one wins.] +</para> +</listitem> +<listitem> +<para> On Windows (but not Unix), if no <option>-B</option> flag is given, GHC uses a system +call to find the directory in which the running GHC executable lives, and derives +<filename>$(libdir)</filename> from that. [Unix lacks such a system call.] +That is why <filename>$(bindir)</filename> must be <filename>$(libdir)/bin</filename>. +</para> +</listitem> +</itemizedlist> + +<sect2> <title>The binary directory</title> + +<para>The binary directory, <filename>$(bindir)</filename> contains user-visible +executables, notably <filename>ghc</filename> and <filename>ghci</filename>. +You should add it to your <literal>$PATH</literal> +</para> + +<para>On Unix, the user-invokable <filename>ghc</filename> invokes <filename>$(libdir)/ghc-<replaceable>version</replaceable></filename>, +passing a suitable <option>-B</option> flag to tell <filename>ghc-<replaceable>version</replaceable></filename> where +<filename>$(libdir)</filename> is. +Similarly <filename>ghci</filename>, except the extra flag <literal>--interactive</literal> is passed. +</para> + +<para>On Win32, the user-invokable <filename>ghc</filename> binary +is the Real Thing (no intervening +shell scripts or <filename>.bat</filename> files). +Reason: we sometimes invoke GHC with very long command lines, +and <filename>cmd.exe</filename> (which executes <filename>.bat</filename> files) +truncates them. Similarly <filename>ghci</filename> is a C wrapper program that invokes <filename>ghc --interactive</filename> +(passing on all other arguments), not a <filename>.bat</filename> file. +</para> + + +</sect2> + +<sect2> <title>The library directory</title> + +<para>The layout of the library directory, <filename>$(libdir)</filename> is almost identical on +Windows and Unix, as follows. Differences between Windows and Unix +are noted thus <literal>[Win32 only]</literal> and are commented below.</para> + +<programlisting> + $(libdir)/ + package.conf GHC package configuration + ghc-usage.txt Message displayed by ghc ––help + + bin/ [Win32 only] User-visible binaries + ghc.exe + ghci.exe + + unlit Remove literate markup + + touchy.exe [Win32 only] + perl.exe [Win32 only] + gcc.exe [Win32 only] + + ghc-x.xx GHC executable [Unix only] + + ghc-split Asm code splitter + ghc-asm Asm code mangler + + gcc-lib/ [Win32 only] Support files for gcc + specs gcc configuration + + cpp0.exe gcc support binaries + as.exe + ld.exe + + crt0.o Standard + ..etc.. binaries + + libmingw32.a Standard + ..etc.. libraries + + *.h Include files + + imports/ GHC interface files + std/*.hi 'std' library + lang/*.hi 'lang' library + ..etc.. + + include/ C header files + StgMacros.h GHC-specific + ..etc... header files + + mingw/*.h [Win32 only] Mingwin header files + + libHSrts.a GHC library archives + libHSstd.a + libHSlang.a + ..etc.. + + HSstd1.o GHC library linkables + HSstd2.o (used by ghci, which does + HSlang.o not grok .a files yet) +</programlisting> + +<para>Note that: +<itemizedlist> + + <listitem> + <para><filename>$(libdir)</filename> also contains support + binaries. These are <emphasis>not</emphasis> expected to be + on the user's <filename>PATH</filename>, but and are invoked + directly by GHC. In the Makefile system, this directory is + also called <filename>$(libexecdir)</filename>, but + <emphasis>you are not free to change it</emphasis>. It must + be the same as <filename>$(libdir)</filename>.</para> + </listitem> + +<listitem> +<para>We distribute <filename>gcc</filename> with the Win32 distribution of GHC, so that users +don't need to install <filename>gcc</filename>, nor need to care about which version it is. +All <filename>gcc</filename>'s support files are kept in <filename>$(libdir)/gcc-lib/</filename>. +</para> +</listitem> + +<listitem> +<para>Similarly, we distribute <filename>perl</filename> and a <filename>touch</filename> +replacement (<filename>touchy.exe</filename>) +with the Win32 distribution of GHC. </para> +</listitem> + + <listitem> + <para>The support programs <filename>ghc-split</filename> + and <filename>ghc-asm</filename> are Perl scripts. The + first line says <literal>#!/bin/perl</literal>; on Unix, the + script is indeed invoked as a shell script, which invokes + Perl; on Windows, GHC invokes + <filename>$(libdir)/perl.exe</filename> directly, which + treats the <literal>#!/bin/perl</literal> as a comment. + Reason: on Windows we want to invoke the Perl distributed + with GHC, rather than assume some installed one. </para> + </listitem> +</itemizedlist> +</para> + +</sect2> + +</sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/intro.xml b/docs/users_guide/intro.xml new file mode 100644 index 0000000000..d4b6a1241f --- /dev/null +++ b/docs/users_guide/intro.xml @@ -0,0 +1,409 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="introduction-GHC"> + <title>Introduction to GHC</title> + + <para>This is a guide to using the Glasgow Haskell Compiler (GHC): + an interactive and batch compilation system for the <ulink + url="http://www.haskell.org/">Haskell 98</ulink> + language.</para> + + <para>GHC has two main components: an interactive Haskell + interpreter (also known as GHCi), described in <xref + linkend="ghci"/>, and a batch compiler, described throughout <xref + linkend="using-ghc"/>. In fact, GHC consists of a single program + which is just run with different options to provide either the + interactive or the batch system.</para> + + <para>The batch compiler can be used alongside GHCi: compiled + modules can be loaded into an interactive session and used in the + same way as interpreted code, and in fact when using GHCi most of + the library code will be pre-compiled. This means you get the best + of both worlds: fast pre-compiled library code, and fast compile + turnaround for the parts of your program being actively + developed.</para> + + <para>GHC supports numerous language extensions, including + concurrency, a foreign function interface, exceptions, type system + extensions such as multi-parameter type classes, local universal and + existential quantification, functional dependencies, scoped type + variables and explicit unboxed types. These are all described in + <xref linkend="ghc-language-features"/>.</para> + + <para>GHC has a comprehensive optimiser, so when you want to Really + Go For It (and you've got time to spare) GHC can produce pretty fast + code. Alternatively, the default option is to compile as fast as + possible while not making too much effort to optimise the generated + code (although GHC probably isn't what you'd describe as a fast + compiler :-).</para> + + <para>GHC's profiling system supports “cost centre + stacks”: a way of seeing the profile of a Haskell program in a + call-graph like structure. See <xref linkend="profiling"/> for more + details.</para> + + <para>GHC comes with a large collection of libraries, with + everything from parser combinators to networking. The libraries are + described in separate documentation.</para> + + <sect1 id="mailing-lists-GHC"> + <title>Meta-information: Web sites, mailing lists, etc.</title> + + <indexterm><primary>mailing lists, Glasgow Haskell</primary></indexterm> + <indexterm><primary>Glasgow Haskell mailing lists</primary></indexterm> + + <para>On the World-Wide Web, there are several URLs of likely + interest:</para> + + <itemizedlist> + <listitem> + <para><ulink url="http://www.haskell.org/" >Haskell home + page</ulink></para> + </listitem> + + <listitem> + <para><ulink url="http://www.haskell.org/ghc/">GHC home + page</ulink></para> + </listitem> + + <listitem> + <para><ulink + url="http://www.cs.nott.ac.uk/~gmh/faq.html">comp.lang.functional + FAQ</ulink></para> + </listitem> + + </itemizedlist> + + <para>We run the following mailing lists about Glasgow Haskell. + We encourage you to join, as you feel is appropriate.</para> + + <variablelist> + <varlistentry> + <term>glasgow-haskell-users:</term> + <listitem> + <para>This list is for GHC users to chat among themselves. + If you have a specific question about GHC, please check the + <ulink + url="http://hackage.haskell.org/trac/ghc/wiki/FAQ">FAQ</ulink> + first.</para> + + <variablelist> + <varlistentry> + <term>list email address:</term> + <listitem> + <para><email>glasgow-haskell-users@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>subscribe at:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/mailman/listinfo/glasgow-haskell-users"><literal>http://www.haskell.org/mailman/listinfo/glasgow-haskell-users</literal></ulink>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>admin email address:</term> + <listitem> + <para><email>glasgow-haskell-users-admin@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>list archives:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/pipermail/glasgow-haskell-users/"><literal>http://www.haskell.org/pipermail/glasgow-haskell-users/</literal></ulink></para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term>glasgow-haskell-bugs:</term> + <listitem> + <para>Send bug reports for GHC to this address! The sad and + lonely people who subscribe to this list will muse upon + what's wrong and what you might do about it.</para> + + <variablelist> + <varlistentry> + <term>list email address:</term> + <listitem> + <para><email>glasgow-haskell-bugs@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>subscribe at:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs"><literal>http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs</literal></ulink>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>admin email address:</term> + <listitem> + <para><email>glasgow-haskell-bugs-admin@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>list archives:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/pipermail/glasgow-haskell-bugs/"><literal>http://www.haskell.org/pipermail/glasgow-haskell-bugs/</literal></ulink></para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term>cvs-ghc:</term> + <listitem> + <para>The hardcore GHC developers hang out here. This list + also gets commit message from the CVS repository. There are + several other similar lists for other parts of the CVS + repository (eg. <literal>cvs-hslibs</literal>, + <literal>cvs-happy</literal>, <literal>cvs-hdirect</literal> + etc.)</para> + + <variablelist> + <varlistentry> + <term>list email address:</term> + <listitem> + <para><email>cvs-ghc@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>subscribe at:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/mailman/listinfo/cvs-ghc"><literal>http://www.haskell.org/mailman/listinfo/cvs-ghc</literal></ulink>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>admin email address:</term> + <listitem> + <para><email>cvs-ghc-admin@haskell.org</email></para> + </listitem> + </varlistentry> + + <varlistentry> + <term>list archives:</term> + <listitem> + <para><ulink + url="http://www.haskell.org/pipermail/cvs-ghc/"><literal>http://www.haskell.org/pipermail/cvs-ghc/</literal></ulink></para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + </variablelist> + + <para>There are several other haskell and GHC-related mailing + lists served by <literal>www.haskell.org</literal>. Go to <ulink + url="http://www.haskell.org/mailman/listinfo/"><literal>http://www.haskell.org/mailman/listinfo/</literal></ulink> + for the full list.</para> + + <para>Some Haskell-related discussion also takes place in the + Usenet newsgroup <literal>comp.lang.functional</literal>.</para> + + </sect1> + + <sect1 id="bug-reporting"> + <title>Reporting bugs in GHC</title> + <indexterm><primary>bugs</primary><secondary>reporting</secondary> + </indexterm> + <indexterm><primary>reporting bugs</primary> + </indexterm> + + <para>Glasgow Haskell is a changing system so there are sure to be + bugs in it. </para> + + <para>To report a bug, either:</para> + + <itemizedlist> + <listitem> + <para>Preferred: <ulink + url="http://hackage.haskell.org/trac/ghc/newticket?type=bug">Create + a new bug</ulink>, and enter your bug report. You can also + search the bug database here to make sure your bug hasn't already + been reported (if it has, it might still help to add information + from your experience to the existing report).</para> + </listitem> + <listitem> + <para>Bug reports can also be emailed to + <email>glasgow-haskell-bugs@haskell.org</email>. </para> + </listitem> + </itemizedlist> + + <sect2> + <title>How do I tell if I should report my bug?</title> + + <para>Take a look at the <ulink + url="http://hackage.haskell.org/trac/ghc/wiki/FAQ">FAQ</ulink> and <xref + linkend="wrong"/>, which will give you some guidance as to + whether the behaviour you're seeing is really a bug or + not.</para> + + <para>If it is a bug, then it might have been reported before: + try searching on the <ulink + url="http://hackage.haskell.org/trac/ghc">bug tracker</ulink>, + and failing that, try <ulink + url="http://www.google.com">Google</ulink>.</para> + + <para>If in doubt, just report it.</para> + </sect2> + + <sect2> + <title>What to put in a bug report</title> + <indexterm><primary>bug reports</primary><secondary>contents</secondary></indexterm> + + <para>The name of the bug-reporting game is: facts, facts, + facts. Don't omit them because “Oh, they won't be + interested…”</para> + + <orderedlist> + <listitem> + <para>What kind of machine are you running on, and exactly + what version of the operating system are you using? (on a + Unix system, <command>uname -a</command> or <command>cat + /etc/motd</command> will show the desired information.) In + the bug tracker, this information can be given in the + “Architecture” and “Operating + system” fields.</para> + </listitem> + + <listitem> + <para>What version of GCC are you using? <command>gcc -v</command> will tell you.</para> + </listitem> + + <listitem> + <para>Run the sequence of compiles/runs that caused the + offending behaviour, cut-and-paste the whole session into + the bug report. We'd prefer to see the whole thing.</para> + </listitem> + + <listitem> + <para>Add the -v flag when running GHC, so we can see exactly + what was run, what versions of things you have, etc.</para> + </listitem> + + <listitem> + <para>What is the program behaviour that is wrong, in your + opinion?</para> + </listitem> + + <listitem> + <para>If practical, please attach or send enough source + files for us to duplicate the problem.</para> + </listitem> + + <listitem> + <para>If you are a Hero and track down the problem in the + compilation-system sources, please send us patches (either + <literal>darcs send</literal>, plain patches, or just whole + files if you prefer).</para> + </listitem> + </orderedlist> + </sect2> + </sect1> + + <sect1 id="version-numbering"> + <title>GHC version numbering policy</title> + <indexterm><primary>version, of ghc</primary></indexterm> + + <para>As of GHC version 6.0, we have adopted the following policy + for numbering GHC versions:</para> + + <variablelist> + <varlistentry> + <term>Stable Releases</term> + <listitem> + <para>These are numbered <literal><replaceable>x</replaceable>.<replaceable>y</replaceable>.<replaceable>z</replaceable></literal>, where + <replaceable>y</replaceable> is <emphasis>even</emphasis>, and + <replaceable>z</replaceable> is the patchlevel number (the trailing + <literal>.<replaceable>z</replaceable></literal> can be omitted if <replaceable>z</replaceable> + is zero). Patchlevels are bug-fix releases only, and never + change the programmer interface to any system-supplied code. + However, if you install a new patchlevel over an old one you + will need to recompile any code that was compiled against the + old libraries.</para> + + <para>The value of <literal>__GLASGOW_HASKELL__</literal> + (see <xref linkend="c-pre-processor"/>) for a major release + <literal><replaceable>x</replaceable>.<replaceable>y</replaceable>.<replaceable>z</replaceable></literal> + is the integer <replaceable>xyy</replaceable> (if + <replaceable>y</replaceable> is a single digit, then a leading zero + is added, so for example in version 6.2 of GHC, + <literal>__GLASGOW_HASKELL__==602</literal>).</para> + <indexterm> + <primary><literal>__GLASGOW_HASKELL__</literal></primary> + </indexterm> + </listitem> + </varlistentry> + + <varlistentry> + <term>Snapshots/unstable releases</term> + <listitem> + <para>We may make snapshot releases of the current + development sources from time to time, and the current + sources are always available via the CVS repository (see the + <ulink url="http://www.haskell.org/ghc/">GHC web + site</ulink> for details).</para> + + <para>Snapshot releases are named + <literal><replaceable>x</replaceable>.<replaceable>y</replaceable>.YYYYMMDD</literal> + where <literal>YYYYMMDD</literal> is the date of the sources + from which the snapshot was built. In theory, you can check + out the exact same sources from the CVS repository using + this date.</para> + + <para>If <replaceable>y</replaceable> is odd, then this is a + snapshot of the CVS HEAD (the main development branch). If + <replaceable>y</replaceable> is even, then it is a snapshot + of the stable branch between patchlevel releases. For + example, <literal>6.3.20040225</literal> would be a snapshot + of the HEAD, but <literal>6.2.20040225</literal> would be a + snapshot of the <literal>6.2</literal> branch.</para> + + <para>The value of <literal>__GLASGOW_HASKELL__</literal> + for a snapshot release is the integer + <replaceable>xyy</replaceable>. You should never write any + conditional code which tests for this value, however: since + interfaces change on a day-to-day basis, and we don't have + finer granularity in the values of + <literal>__GLASGOW_HASKELL__</literal>, you should only + conditionally compile using predicates which test whether + <literal>__GLASGOW_HASKELL__</literal> is equal to, later + than, or earlier than a given major release.</para> + <indexterm> + <primary><literal>__GLASGOW_HASKELL__</literal></primary> + </indexterm> + </listitem> + </varlistentry> + </variablelist> + + <para>The version number of your copy of GHC can be found by + invoking <literal>ghc</literal> with the + <literal>––version</literal> flag (see <xref + linkend="options-help"/>).</para> + </sect1> + + +&relnotes; + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/lang.xml b/docs/users_guide/lang.xml new file mode 100644 index 0000000000..7e9621ed8b --- /dev/null +++ b/docs/users_guide/lang.xml @@ -0,0 +1,15 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="ghc-language-features"> +<title>GHC Language Features</title> + +&glasgowexts; +∥ + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/license.xml b/docs/users_guide/license.xml new file mode 100644 index 0000000000..55e2395a7c --- /dev/null +++ b/docs/users_guide/license.xml @@ -0,0 +1,66 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<preface id="License"> +<title>The Glasgow Haskell Compiler License</title> + +<para> +Copyright 2002, The University Court of the University of Glasgow. +All rights reserved. +</para> + +<para> +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: +</para> + +<para> +<itemizedlist> + +<listitem> +<para> +Redistributions of source code must retain the above copyright notice, +this list of conditions and the following disclaimer. +</para> +</listitem> + +<listitem> +<para> +Redistributions in binary form must reproduce the above copyright notice, +this list of conditions and the following disclaimer in the documentation +and/or other materials provided with the distribution. +</para> +</listitem> + +<listitem> +<para> +Neither name of the University nor the names of its contributors may be +used to endorse or promote products derived from this software without +specific prior written permission. +</para> +</listitem> + +</itemizedlist> +</para> + +<para> +THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY COURT OF THE UNIVERSITY OF +GLASGOW AND THE CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, +INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +UNIVERSITY COURT OF THE UNIVERSITY OF GLASGOW OR THE CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH +DAMAGE. +</para> + +</preface> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "preface") *** + ;;; End: *** + --> diff --git a/docs/users_guide/packages.xml b/docs/users_guide/packages.xml new file mode 100644 index 0000000000..3bd65c66ce --- /dev/null +++ b/docs/users_guide/packages.xml @@ -0,0 +1,1193 @@ +<?xml version="1.0" encoding="iso-8859-1"?> + <sect1 id="packages"> + <title> +Packages + </title> + <indexterm><primary>packages</primary></indexterm> + + <para>A package is a library of Haskell modules known to the compiler. GHC + comes with several packages: see the accompanying + <ulink url="../libraries/index.html">library documentation</ulink>.</para> + + <para>Using a package couldn't be simpler: if you're using + <option>--make</option> or GHCi, then most of the installed packages will be + automatically available to your program without any further options. The + exceptions to this rule are covered below in <xref + linkend="using-packages" />.</para> + + <para>Building your own packages is also quite straightforward: we provide + the <ulink url="http://www.haskell.org/cabal/">Cabal</ulink> infrastructure which + automates the process of configuring, building, installing and distributing + a package. All you need to do is write a simple configuration file, put a + few files in the right places, and you have a package. See the + <ulink url="../Cabal/index.html">Cabal documentation</ulink> + for details, and also the Cabal libraries (<ulink url="../libraries/Cabal/Distribution-Simple.html">Distribution.Simple</ulink>, + for example).</para> + + <sect2 id="using-packages"> + <title>Using Packages + </title> + <indexterm><primary>packages</primary> + <secondary>using</secondary></indexterm> + + <para>To see which packages are installed, use the + <literal>ghc-pkg</literal> command:</para> + +<screen> +$ ghc-pkg list +/usr/lib/ghc-6.4/package.conf: + base-1.0, haskell98-1.0, template-haskell-1.0, mtl-1.0, unix-1.0, + Cabal-1.0, haskell-src-1.0, parsec-1.0, network-1.0, + QuickCheck-1.0, HUnit-1.1, fgl-1.0, X11-1.1, HGL-3.1, OpenGL-2.0, + GLUT-2.0, stm-1.0, readline-1.0, (lang-1.0), (concurrent-1.0), + (posix-1.0), (util-1.0), (data-1.0), (text-1.0), (net-1.0), + (hssource-1.0), rts-1.0 + </screen> + + <para>Packages are either exposed or hidden. Only + modules from exposed packages may be imported by your Haskell code; if + you try to import a module from a hidden package, GHC will emit an error + message.</para> + + <para>Each package has an exposed flag, which says whether it is exposed by + default or not. Packages hidden by default are listed in + parentheses (eg. <literal>(lang-1.0)</literal>) in the output from + <literal>ghc-pkg list</literal>. To expose a package which is hidden by + default, use the <option>-package</option> + flag (see below).</para> + + <para>To see which modules are exposed by a package:</para> + +<screen> +$ ghc-pkg field network exposed-modules +exposed-modules: Network.BSD, + Network.CGI, + Network.Socket, + Network.URI, + Network +</screen> + + <para>In general, packages containing hierarchical modules are usually + exposed by default. However, it is possible for two packages to contain + the same module: in this case, only one of the packages should be + exposed. It is an error to import a module that belongs to more than one + exposed package.</para> + + <para>The GHC command line options that control packages are:</para> + + <variablelist> + <varlistentry> + <term> + <option>-package <replaceable>P</replaceable></option> + <indexterm><primary><option>-package</option></primary></indexterm> + </term> + <listitem> + <para>This option causes package <replaceable>P</replaceable> to be + exposed. The package <replaceable>P</replaceable> can be specified + in full with its version number + (e.g. <literal>network-1.0</literal>) or the version number can be + omitted if there is only one version of the package + installed.</para> + + <para>If there are multiple versions of <replaceable>P</replaceable> + installed, then all other versions will become hidden.</para> + + <para>The <option>-package <replaceable>P</replaceable></option> + option also causes package <replaceable>P</replaceable> to be + linked into the resulting executable. In + <option>––make</option> mode and GHCi, the compiler + normally determines which packages are required by the current + Haskell modules, and links only those. In batch mode however, the + dependency information isn't available, and explicit + <option>-package</option> options must be given when linking.</para> + + <para>For example, to link a program consisting of objects + <filename>Foo.o</filename> and <filename>Main.o</filename>, where + we made use of the <literal>network</literal> package, we need to + give GHC the <literal>-package</literal> flag thus: + +<screen>$ ghc -o myprog Foo.o Main.o -package network</screen> + + The same flag is necessary even if we compiled the modules from + source, because GHC still reckons it's in batch mode: + +<screen>$ ghc -o myprog Foo.hs Main.hs -package network</screen> + + In <literal>--make</literal> and <literal>--interactive</literal> + modes (<xref linkend="modes" />), however, GHC figures out the + packages required for linking without further assistance.</para> + + <para>The one other time you might need to use + <option>-package</option> to force linking a package is when the + package does not contain any Haskell modules (it might contain a C + library only, for example). In that case, GHC + will never discover a dependency on it, so it has to be mentioned + explicitly.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-hide-all-packages</option> + <indexterm><primary><option>-hide-package</option></primary> + </indexterm></term> + <listitem> + <para>Ignore the exposed flag on installed packages, and hide them + all by default. If you use + this flag, then any packages you require (including + <literal>base</literal>) need to be explicitly exposed using + <option>-package</option> options.</para> + + <para>This is a good way to insulate your program from differences + in the globally exposed packages, and being explicit about package + dependencies is a Good Thing.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-hide-package</option> <replaceable>P</replaceable> + <indexterm><primary><option>-hide-package</option></primary> + </indexterm></term> + <listitem> + <para>This option does the opposite of <option>-package</option>: it + causes the specified package to be <firstterm>hidden</firstterm>, + which means that none of its modules will be available for import + by Haskell <literal>import</literal> directives.</para> + + <para>Note that the package might still end up being linked into the + final program, if it is a dependency (direct or indirect) of + another exposed package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-ignore-package</option> <replaceable>P</replaceable> + <indexterm><primary><option>-ignore-package</option></primary> + </indexterm></term> + <listitem> + <para>Causes the compiler to behave as if package + <replaceable>P</replaceable>, and any packages that depend on + <literal>P</literal>, are not installed at all.</para> + + <para>Saying <literal>-ignore-package P</literal> is the same as + giving <literal>-hide-package</literal> flags for + <literal>P</literal> and all the packages that depend on + <literal>P</literal>. Sometimes we don't know ahead of time which + packages will be installed that depend on <literal>P</literal>, + which is when the <literal>-ignore-package</literal> flag can be + useful.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="package-overlaps"> + <title>The module overlap restriction</title> + + <para>The module names in a Haskell program must be distinct. + This doesn't sound like a severe restriction, but in a Haskell program + using multiple packages with interdependencies, difficulties can start to + arise. You should be aware of what the module overlap + restriction means, and how to avoid it.</para> + + <para>GHC knows which packages are <emphasis>in use</emphasis> by your + program: a package is in use if you imported something from it, or if it + is a dependency of some other package in use. There must be no conflicts + between the packages in use; a conflict is when two packages contain + a module with the same name. If + GHC detects a conflict, it will issue a message stating which packages + are in conflict, and which modules are overlapping.</para> + + <para>For example, a conflict might arise if you use two packages, say P + and Q, which respectively depend on two different versions of another + package, say <literal>R-1.0</literal> and <literal>R-2.0</literal>. The + two versions of <literal>R</literal> are likely to contain at least some + of the same modules, so this situation would be a conflict.</para> + </sect2> + + <sect2 id="package-databases"> + <title>Package Databases</title> + + <para>A package database is a file, normally called + <literal>package.conf</literal> which contains descriptions of installed + packages. GHC usually knows about two package databases:</para> + + <itemizedlist> + <listitem> + <para>The global package database, which comes with your GHC + installation.</para> + </listitem> + <listitem> + <para>A package database private to each user. On Unix + systems this will be + <filename>$HOME/.ghc/<replaceable>arch</replaceable>-<replaceable>os</replaceable>-<replaceable>version</replaceable>/package.conf</filename>, and on + Windows it will be something like + <filename>C:\Documents And Settings\<replaceable>user</replaceable>\ghc</filename>. + The <literal>ghc-pkg</literal> tool knows where this file should be + located, and will create it if it doesn't exist (see <xref linkend="package-management" />).</para> + </listitem> + </itemizedlist> + + <para>When GHC starts up, it reads the contents of these two package + databases, and builds up a list of the packages it knows about. You can + see GHC's package table by running GHC with the <option>-v</option> + flag.</para> + + <para>Package databases may overlap: for example, packages in the user + database will override those of the same name in the global + database.</para> + + <para>You can control the loading of package databses using the following + GHC options:</para> + + <variablelist> + <varlistentry> + <term> + <option>-package-conf <replaceable>file</replaceable></option> + <indexterm><primary><option>-package-conf</option></primary></indexterm> + </term> + <listitem> + <para>Read in the package configuration file + <replaceable>file</replaceable> in addition to the system + default file and the user's local file. Packages in additional + files read this way will override those in the global and user + databases.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-no-user-package-conf</option> + <indexterm><primary><option>-no-user-package-conf</option></primary> + </indexterm> + </term> + <listitem> + <para>Prevent loading of the user's local package database.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>To create a new package database, just create + a new file and put the string + <quote><literal>[]</literal></quote> in it. Packages can be + added to the file using the + <literal>ghc-pkg</literal> tool, described in <xref + linkend="package-management"/>.</para> + + <sect3 id="ghc-package-path"> + <title>The <literal>GHC_PACKAGE_PATH</literal> environment variable</title> + <indexterm><primary>Environment variable</primary><secondary><literal>GHC_PACKAGE_PATH</literal></secondary> + </indexterm> + <indexterm><primary><literal>GHC_PACKAGE_PATH</literal></primary></indexterm> + <para>The <literal>GHC_PACKAGE_PATH</literal> environment variable may be + set to a <literal>:</literal>-separated (<literal>;</literal>-separated + on Windows) list of files containing package databases. This list of + package databases is used by GHC and ghc-pkg, with earlier databases in + the list overriding later ones. This order was chosen to match the + behaviour of the <literal>PATH</literal> environment variable; think of + it as a list of package databases that are searched left-to-right for + packages.</para> + + <para>If <literal>GHC_PACKAGE_PATH</literal> ends in a separator, then + the default user and system package databases are appended, in that + order. e.g. to augment the usual set of packages with a database of + your own, you could say (on Unix): +<screen> +$ export GHC_PACKAGE_PATH=$HOME/.my-ghc-packages.conf:</screen> + (use <literal>;</literal> instead of <literal>:</literal> on + Windows).</para> + + <para>To check whether your <literal>GHC_PACKAGE_PATH</literal> setting + is doing the right thing, <literal>ghc-pkg list</literal> will list all + the databases in use, in the reverse order they are searched.</para> + + </sect3> + </sect2> + + <sect2 id="building-packages"> + <title>Building a package from Haskell source</title> + <indexterm><primary>packages</primary> + <secondary>building</secondary></indexterm> + + <para>We don't recommend building packages the hard way. Instead, use the + <ulink url="../Cabal/index.html">Cabal</ulink> infrastructure + if possible. If your package is particularly complicated or requires a + lot of configuration, then you might have to fall back to the low-level + mechanisms, so a few hints for those brave souls follow.</para> + + <itemizedlist> + <listitem> + <para>You need to build an "installed package info" file for + passing to <literal>ghc-pkg</literal> when installing your + package. The contents of this file are described in <xref + linkend="installed-pkg-info" />.</para> + </listitem> + + <listitem> + <para>The Haskell code in a package may be built into one or + more archive libraries + (e.g. <filename>libHSfoo.a</filename>), or a single DLL on + Windows (e.g. <filename>HSfoo.dll</filename>). The + restriction to a single DLL on Windows is because the + package system is used to tell the compiler when it should + make an inter-DLL call rather than an intra-DLL call + (inter-DLL calls require an extra + indirection). <emphasis>Building packages as DLLs doesn't + work at the moment; see <xref linkend="win32-dlls-create"/> + for the gory details.</emphasis> + </para> + + <para>Building a static library is done by using the + <literal>ar</literal> tool, like so:</para> + +<screen>ar cqs libHSfoo.a A.o B.o C.o ...</screen> + + <para>where <filename>A.o</filename>, + <filename>B.o</filename> and so on are the compiled Haskell + modules, and <filename>libHSfoo.a</filename> is the library + you wish to create. The syntax may differ slightly on your + system, so check the documentation if you run into + difficulties.</para> + + <para>Versions of the Haskell libraries for use with GHCi + may also be included: GHCi cannot load <literal>.a</literal> + files directly, instead it will look for an object file + called <filename>HSfoo.o</filename> and load that. On some + systems, the <literal>ghc-pkg</literal> tool can + automatically build the GHCi version of each library, see + <xref linkend="package-management"/>. To build these + libraries by hand from the <literal>.a</literal> archive, it + is possible to use GNU <command>ld</command> as + follows:</para> + +<screen>ld -r ––whole-archive -o HSfoo.o libHSfoo.a</screen> + + <para>(replace + <literal>––--whole-archive</literal> with + <literal>–all_load</literal> on MacOS X)</para> + + <para>GHC does not maintain detailed cross-package + dependency information. It does remember which modules in + other packages the current module depends on, but not which + things within those imported things.</para> + </listitem> + </itemizedlist> + + <para>It is worth noting that on Windows, when each package + is built as a DLL, since a reference to a DLL costs an extra + indirection, intra-package references are cheaper than + inter-package references. Of course, this applies to the + <filename>Main</filename> package as well.</para> + </sect2> + + <sect2 id="package-management"> + <title>Package management (the <literal>ghc-pkg</literal> command)</title> + <indexterm><primary>packages</primary> + <secondary>management</secondary></indexterm> + + <para>The <literal>ghc-pkg</literal> tool allows packages to be + added or removed from a package database. By default, + the system-wide package database is modified, but alternatively + the user's local package database or another specified + file can be used.</para> + + <para>To see what package databases are in use, say + <literal>ghc-pkg list</literal>. The stack of databases that + <literal>ghc-pkg</literal> knows about can be modified using the + <literal>GHC_PACKAGE_PATH</literal> environment variable (see <xref + linkend="ghc-package-path" />, and using + <literal>--package-conf</literal> options on the + <literal>ghc-pkg</literal> command line.</para> + + <para>When asked to modify a database, <literal>ghc-pkg</literal> modifies + the global database by default. Specifying <option>--user</option> + causes it to act on the user database, or <option>--package-conf</option> + can be used to act on another database entirely. When multiple of these + options are given, the rightmost one is used as the database to act + upon.</para> + + <para>If the environment variable <literal>GHC_PACKAGE_PATH</literal> is + set, and its value does not end in a separator (<literal>:</literal> on + Unix, <literal>;</literal> on Windows), then the last database is + considered to be the global database, and will be modified by default by + <literal>ghc-pkg</literal>. The intention here is that + <literal>GHC_PACKAGE_PATH</literal> can be used to create a virtual + package environment into which Cabal packages can be installed without + setting anything other than <literal>GHC_PACKAGE_PATH</literal>.</para> + + <para>The <literal>ghc-pkg</literal> program may be run in the ways listed + below. Where a package name is required, the package can be named in + full including the version number + (e.g. <literal>network-1.0</literal>), or without the version number. + Naming a package without the version number matches all versions of the + package; the specified action will be applied to all the matching + packages. A package specifier that matches all version of the package + can also be written <replaceable>pkg</replaceable><literal>-*</literal>, + to make it clearer that multiple packages are being matched.</para> + + <variablelist> + <varlistentry> + <term><literal>ghc-pkg register <replaceable>file</replaceable></literal></term> + <listitem> + <para>Reads a package specification from + <replaceable>file</replaceable> (which may be “<literal>-</literal>” + to indicate standard input), + and adds it to the database of installed packages. The syntax of + <replaceable>file</replaceable> is given in <xref + linkend="installed-pkg-info" />.</para> + + <para>The package specification must be a package that isn't already + installed.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg update <replaceable>file</replaceable></literal></term> + <listitem> + <para>The same as <literal>register</literal>, except that if a + package of the same name is already installed, it is + replaced by the new one.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg unregister <replaceable>P</replaceable></literal></term> + <listitem> + <para>Remove the specified package from the database.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg expose <replaceable>P</replaceable></literal></term> + <listitem> + <para>Sets the <literal>exposed</literal> flag for package + <replaceable>P</replaceable> to <literal>True</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg hide <replaceable>P</replaceable></literal></term> + <listitem> + <para>Sets the <literal>exposed</literal> flag for package + <replaceable>P</replaceable> to <literal>False</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg list [<replaceable>P</replaceable>] [<option>--simple-output</option>]</literal></term> + <listitem> + <para>This option displays the currently installed + packages, for each of the databases known to + <literal>ghc-pkg</literal>. That includes the global database, the + user's local database, and any further files specified using the + <option>-f</option> option on the command line.</para> + + <para>Hidden packages (those for which the <literal>exposed</literal> + flag is <literal>False</literal>) are shown in parentheses in the + list of packages.</para> + + <para>If an optional package identifier <replaceable>P</replaceable> + is given, then only packages matching that identifier are + shown.</para> + + <para>If the option <option>--simple-output</option> is given, then + the packages are listed on a single line separated by spaces, and + the database names are not included. This is intended to make it + easier to parse the output of <literal>ghc-pkg list</literal> using + a script.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg latest <replaceable>P</replaceable></literal></term> + <listitem> + <para>Prints the latest available version of package + <replaceable>P</replaceable>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg describe <replaceable>P</replaceable></literal></term> + <listitem> + <para>Emit the full description of the specified package. The + description is in the form of an + <literal>InstalledPackageInfo</literal>, the same as the input file + format for <literal>ghc-pkg register</literal>. See <xref + linkend="installed-pkg-info" /> for details.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>ghc-pkg field <replaceable>P</replaceable> <replaceable>field</replaceable></literal></term> + <listitem> + <para>Show just a single field of the installed package description + for <literal>P</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Additionally, the following flags are accepted by + <literal>ghc-pkg</literal>:</para> + + <variablelist> + <varlistentry> + <term> + <option>––auto-ghci-libs</option><indexterm><primary><option>––auto-ghci-libs</option></primary> + </indexterm> + </term> + <listitem> + <para>Automatically generate the GHCi + <filename>.o</filename> version of each + <filename>.a</filename> Haskell library, using GNU ld (if + that is available). Without this option, + <literal>ghc-pkg</literal> will warn if GHCi versions of + any Haskell libraries in the package don't exist.</para> + + <para>GHCi <literal>.o</literal> libraries don't + necessarily have to live in the same directory as the + corresponding <literal>.a</literal> library. However, + this option will cause the GHCi library to be created in + the same directory as the <literal>.a</literal> + library.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-f</option> <replaceable>file</replaceable> + <indexterm><primary><option>-f</option></primary> + </indexterm> + </term> + <term> + <option>-package-conf</option> <replaceable>file</replaceable> + <indexterm><primary><option>-package-conf</option></primary> + </indexterm> + </term> + <listitem> + <para>Adds <replaceable>file</replaceable> to the stack of package + databases. Additionally, <replaceable>file</replaceable> will + also be the database modified by a <literal>register</literal>, + <literal>unregister</literal>, <literal>expose</literal> or + <literal>hide</literal> command, unless it is overriden by a later + <option>--package-conf</option>, <option>--user</option> or + <option>--global</option> option.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––force</option> + <indexterm><primary> + <option>––force</option> + </primary></indexterm> + </term> + <listitem> + <para>Causes <literal>ghc-pkg</literal> to ignore missing + dependencies, directories and libraries when registering a package, + and just go ahead and add it anyway. This might be useful if your + package installation system needs to add the package to + GHC before building and installing the files.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––global</option><indexterm><primary><option>––global</option></primary> + </indexterm> + </term> + <listitem> + <para>Operate on the global package database (this is the default). + This flag affects the <literal>register</literal>, + <literal>update</literal>, <literal>unregister</literal>, + <literal>expose</literal>, and <literal>hide</literal> + commands.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––help</option><indexterm><primary><option>––help</option></primary> + </indexterm> + </term> + <term> + <option>-?</option><indexterm><primary><option>-?</option></primary> + </indexterm> + </term> + <listitem> + <para>Outputs the command-line syntax.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––user</option><indexterm><primary><option>––user</option></primary> + </indexterm> + </term> + <listitem> + <para>Operate on the current user's local package database. + This flag affects the <literal>register</literal>, + <literal>update</literal>, <literal>unregister</literal>, + <literal>expose</literal>, and <literal>hide</literal> + commands.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-V</option><indexterm><primary><option>-V</option></primary> + </indexterm> + </term> + <term> + <option>––version</option><indexterm><primary><option>––version</option></primary> + </indexterm> + </term> + <listitem> + <para>Output the <literal>ghc-pkg</literal> version number.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>When modifying the package database + <replaceable>file</replaceable>, a copy of the original file is + saved in <replaceable>file</replaceable><literal>.old</literal>, + so in an emergency you can always restore the old settings by + copying the old file back again.</para> + + </sect2> + + <sect2 id="installed-pkg-info"> + <title> + <literal>InstalledPackageInfo</literal>: a package specification + </title> + + <para>A package specification is a Haskell record; in particular, it is the + record <ulink + url="../libraries/Cabal/Distribution-InstalledPackageInfo.html#%tInstalledPackageInfo">InstalledPackageInfo</ulink> in the module Distribution.InstalledPackageInfo, which is part of the Cabal package distributed with GHC.</para> + + <para>An <literal>InstalledPackageInfo</literal> has a human + readable/writable syntax. The functions + <literal>parseInstalledPackageInfo</literal> and + <literal>showInstalledPackageInfo</literal> read and write this syntax + respectively. Here's an example of the + <literal>InstalledPackageInfo</literal> for the <literal>unix</literal> package:</para> + +<screen> +$ ghc-pkg describe unix +name: unix +version: 1.0 +license: BSD3 +copyright: +maintainer: libraries@haskell.org +stability: +homepage: +package-url: +description: +category: +author: +exposed: True +exposed-modules: System.Posix, + System.Posix.DynamicLinker.Module, + System.Posix.DynamicLinker.Prim, + System.Posix.Directory, + System.Posix.DynamicLinker, + System.Posix.Env, + System.Posix.Error, + System.Posix.Files, + System.Posix.IO, + System.Posix.Process, + System.Posix.Resource, + System.Posix.Temp, + System.Posix.Terminal, + System.Posix.Time, + System.Posix.Unistd, + System.Posix.User, + System.Posix.Signals.Exts +import-dirs: /usr/lib/ghc-6.4/libraries/unix +library-dirs: /usr/lib/ghc-6.4/libraries/unix +hs-libraries: HSunix +extra-libraries: HSunix_cbits, dl +include-dirs: /usr/lib/ghc-6.4/libraries/unix/include +includes: HsUnix.h +depends: base-1.0 +</screen> + + <para>The full <ulink url="../Cabal/index.html">Cabal documentation</ulink> + is still in preparation (at time of writing), so in the meantime + here is a brief description of the syntax of this file:</para> + + <para>A package description consists of a number of field/value pairs. A + field starts with the field name in the left-hand column followed by a + “<literal>:</literal>”, and the value continues until the next line that begins in the + left-hand column, or the end of file.</para> + + <para>The syntax of the value depends on the field. The various field + types are:</para> + + <variablelist> + <varlistentry> + <term>freeform</term> + <listitem> + <para>Any arbitrary string, no interpretation or parsing is + done.</para> + </listitem> + </varlistentry> + <varlistentry> + <term>string</term> + <listitem> + <para>A sequence of non-space characters, or a sequence of arbitrary + characters surrounded by quotes <literal>"...."</literal>.</para> + </listitem> + </varlistentry> + <varlistentry> + <term>string list</term> + <listitem> + <para>A sequence of strings, separated by commas. The sequence may + be empty.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>In addition, there are some fields with special syntax (e.g. package + names, version, dependencies).</para> + + <para>The allowed fields, with their types, are:</para> + + <variablelist> + <varlistentry> + <term> + <literal>name</literal> + <indexterm><primary><literal>name</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>The package's name (without the version).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>version</literal> + <indexterm><primary><literal>version</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>The package's version, usually in the form + <literal>A.B</literal> (any number of components are allowed).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>license</literal> + <indexterm><primary><literal>auto</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string) The type of license under which this package is distributed. + This field is a value of the <ulink + url="../libraries/Cabal/Distribution-License.html#t:License"><literal>License</literal></ulink> type.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>license-file</literal> + <indexterm><primary><literal>license-file</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional string) The name of a file giving detailed license + information for this package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>copyright</literal> + <indexterm><primary><literal>copyright</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) The copyright string.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>maintainer</literal> + <indexterm><primary><literal>maintainer</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optinoal freeform) The email address of the package's maintainer.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>stability</literal> + <indexterm><primary><literal>stability</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) A string describing the stability of the package + (eg. stable, provisional or experimental).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>homepage</literal> + <indexterm><primary><literal>homepage</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) URL of the package's home page.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>package-url</literal> + <indexterm><primary><literal>package-url</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) URL of a downloadable distribution for this + package. The distribution should be a Cabal package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>description</literal> + <indexterm><primary><literal>description</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) Description of the package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>category</literal> + <indexterm><primary><literal>category</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optinoal freeform) Which category the package belongs to. This field + is for use in conjunction with a future centralised package + distribution framework, tentatively titled Hackage.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>author</literal> + <indexterm><primary><literal>author</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional freeform) Author of the package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>exposed</literal> + <indexterm><primary><literal>exposed</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(bool) Whether the package is exposed or not.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>exposed-modules</literal> + <indexterm><primary><literal>exposed-modules</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) modules exposed by this package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>hidden-modules</literal> + <indexterm><primary><literal>hidden-modules</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) modules provided by this package, + but not exposed to the programmer. These modules cannot be + imported, but they are still subject to the overlapping constraint: + no other package in the same program may provide a module of the + same name.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>import-dirs</literal> + <indexterm><primary><literal>import-dirs</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of directories containing interface files + (<literal>.hi</literal> files) for this package.</para> + + <para>If the package contains profiling libraries, then + the interface files for those library modules should have + the suffix <literal>.p_hi</literal>. So the package can + contain both normal and profiling versions of the same + library without conflict (see also + <literal>library_dirs</literal> below).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>library-dirs</literal> + <indexterm><primary><literal>library-dirs</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of directories containing libraries for this + package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>hs-libraries</literal> + <indexterm><primary><literal>hs-libraries</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of libraries containing Haskell code for this + package, with the <literal>.a</literal> or + <literal>.dll</literal> suffix omitted. When packages are + built as libraries, the + <literal>lib</literal> prefix is also omitted.</para> + + <para>For use with GHCi, each library should have an + object file too. The name of the object file does + <emphasis>not</emphasis> have a <literal>lib</literal> + prefix, and has the normal object suffix for your + platform.</para> + + <para>For example, if we specify a Haskell library as + <filename>HSfoo</filename> in the package spec, then the + various flavours of library that GHC actually uses will be + called:</para> + <variablelist> + <varlistentry> + <term><filename>libHSfoo.a</filename></term> + <listitem> + <para>The name of the library on Unix and Windows + (mingw) systems. Note that we don't support + building dynamic libraries of Haskell code on Unix + systems.</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>HSfoo.dll</filename></term> + <listitem> + <para>The name of the dynamic library on Windows + systems (optional).</para> + </listitem> + </varlistentry> + <varlistentry> + <term><filename>HSfoo.o</filename></term> + <term><filename>HSfoo.obj</filename></term> + <listitem> + <para>The object version of the library used by + GHCi.</para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>extra-libraries</literal> + <indexterm><primary><literal>extra-libraries</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of extra libraries for this package. The + difference between <literal>hs-libraries</literal> and + <literal>extra-libraries</literal> is that + <literal>hs-libraries</literal> normally have several + versions, to support profiling, parallel and other build + options. The various versions are given different + suffixes to distinguish them, for example the profiling + version of the standard prelude library is named + <filename>libHSbase_p.a</filename>, with the + <literal>_p</literal> indicating that this is a profiling + version. The suffix is added automatically by GHC for + <literal>hs-libraries</literal> only, no suffix is added + for libraries in + <literal>extra-libraries</literal>.</para> + + <para>The libraries listed in + <literal>extra-libraries</literal> may be any libraries + supported by your system's linker, including dynamic + libraries (<literal>.so</literal> on Unix, + <literal>.DLL</literal> on Windows).</para> + + <para>Also, <literal>extra-libraries</literal> are placed + on the linker command line after the + <literal>hs-libraries</literal> for the same package. If + your package has dependencies in the other direction (i.e. + <literal>extra-libraries</literal> depends on + <literal>hs-libraries</literal>), and the libraries are + static, you might need to make two separate + packages.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>include-dirs</literal> + <indexterm><primary><literal>include-dirs</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of directories containing C includes for this + package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>includes</literal> + <indexterm><primary><literal>includes</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of files to include for via-C compilations + using this package. Typically the include file(s) will + contain function prototypes for any C functions used in + the package, in case they end up being called as a result + of Haskell functions from the package being + inlined.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>depends</literal> + <indexterm><primary><literal>depends</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(package name list) Packages on which this package depends. This field contains + packages with explicit versions are required, except that when + submitting a package to <literal>ghc-pkg register</literal>, the + versions will be filled in if they are unambiguous.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>hugs-options</literal> + <indexterm><primary><literal>hugs-options</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) Options to pass to Hugs for this package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>cc-options</literal> + <indexterm><primary><literal>cc-options</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) Extra arguments to be added to the gcc command line + when this package is being used (only for via-C + compilations).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>ld-options</literal> + <indexterm><primary><literal>ld-options</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) Extra arguments to be added to the + <command>gcc</command> command line (for linking) when + this package is being used.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>framework-dirs</literal> + <indexterm><primary><literal>framework-dirs</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) On Darwin/MacOS X, a list of directories containing + frameworks for this package. This corresponds to the + <option>-framework-path</option> option. It is ignored on all other + platforms.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>frameworks</literal> + <indexterm><primary><literal>frameworks</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) On Darwin/MacOS X, a list of frameworks to link to. This + corresponds to the <option>-framework</option> option. Take a look + at Apple's developer documentation to find out what frameworks + actually are. This entry is ignored on all other platforms.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>haddock-interfaces</literal> + <indexterm><primary><literal>haddock-interfaces</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(string list) A list of filenames containing <ulink + url="http://www.haskell.org/haddock/">Haddock</ulink> interface + files (<literal>.haddock</literal> files) for this package.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <literal>haddock-html</literal> + <indexterm><primary><literal>haddock-html</literal></primary><secondary>package specification</secondary></indexterm> + </term> + <listitem> + <para>(optional string) The directory containing the Haddock-generated HTML + for this package.</para> + </listitem> + </varlistentry> + </variablelist> + +<!-- This isn't true any more. I'm not sure if we still need it -SDM + <para> + The <literal>ghc-pkg</literal> tool performs expansion of + environment variables occurring in input package specifications. + So, if the <literal>mypkg</literal> was added to the package + database as follows: + </para> +<screen> + $ installdir=/usr/local/lib ghc-pkg -a < mypkg.pkg +</screen> + + <para> + The occurrence of <literal>${installdir}</literal> is replaced + with <literal>/usr/local/lib</literal> in the package data that + is added for <literal>mypkg</literal>. + </para> + + <para> + This feature enables the distribution of package specification + files that can be easily configured when installing. + </para> + + <para>For examples of more package specifications, take a look + at the <literal>package.conf</literal> in your GHC + installation.</para> + +--> + + </sect2> + </sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/parallel.xml b/docs/users_guide/parallel.xml new file mode 100644 index 0000000000..11c2547898 --- /dev/null +++ b/docs/users_guide/parallel.xml @@ -0,0 +1,210 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="concurrent-and-parallel"> +<title>Concurrent and Parallel Haskell</title> + +<para> +<indexterm><primary>Concurrent Haskell</primary></indexterm> +<indexterm><primary>Parallel Haskell</primary></indexterm> +Concurrent and Parallel Haskell are Glasgow extensions to Haskell +which let you structure your program as a group of independent +`threads'. +</para> + +<para> +Concurrent and Parallel Haskell have very different purposes. +</para> + +<para> +Concurrent Haskell is for applications which have an inherent +structure of interacting, concurrent tasks (i.e. `threads'). Threads +in such programs may be <emphasis>required</emphasis>. For example, if a concurrent thread has been spawned to handle a mouse click, it isn't +optional—the user wants something done! +</para> + +<para> +A Concurrent Haskell program implies multiple `threads' running within +a single Unix process on a single processor. +</para> + +<para> +You will find at least one paper about Concurrent Haskell hanging off +of <ulink url="http://research.microsoft.com/~simonpj/">Simon Peyton +Jones's Web page</ulink>. +</para> + +<para> +Parallel Haskell is about <emphasis>speed</emphasis>—spawning +threads onto multiple processors so that your program will run faster. +The `threads' are always <emphasis>advisory</emphasis>—if the +runtime system thinks it can get the job done more quickly by +sequential execution, then fine. +</para> + +<para> +A Parallel Haskell program implies multiple processes running on +multiple processors, under a PVM (Parallel Virtual Machine) framework. +An MPI interface is under development but not fully functional, yet. +</para> + +<para> +Parallel Haskell is still relatively new; it is more about “research +fun” than about “speed.” That will change. +</para> + +<para> +Check the <ulink url="http://www.cee.hw.ac.uk/~dsg/gph/">GPH Page</ulink> +for more information on “GPH” (Haskell98 with extensions for +parallel execution), the latest version of “GUM” (the runtime +system to enable parallel executions) and papers on research issues. A +list of publications about GPH and about GUM is also available from Simon's +Web Page. +</para> + +<para> +Some details about Parallel Haskell follow. For more information +about concurrent Haskell, see the module +<literal>Control.Concurrent</literal> in the library documentation. +</para> + +<sect2> +<title>Features specific to Parallel Haskell +<indexterm><primary>Parallel Haskell—features</primary></indexterm></title> + +<sect3> +<title>The <literal>Parallel</literal> interface (recommended) +<indexterm><primary>Parallel interface</primary></indexterm></title> + +<para> +GHC provides two functions for controlling parallel execution, through +the <literal>Parallel</literal> interface: +</para> + +<para> + +<programlisting> +interface Parallel where +infixr 0 `par` +infixr 1 `seq` + +par :: a -> b -> b +seq :: a -> b -> b +</programlisting> + +</para> + +<para> +The expression <literal>(x `par` y)</literal> <emphasis>sparks</emphasis> the evaluation of <literal>x</literal> +(to weak head normal form) and returns <literal>y</literal>. Sparks are queued for +execution in FIFO order, but are not executed immediately. At the +next heap allocation, the currently executing thread will yield +control to the scheduler, and the scheduler will start a new thread +(until reaching the active thread limit) for each spark which has not +already been evaluated to WHNF. +</para> + +<para> +The expression <literal>(x `seq` y)</literal> evaluates <literal>x</literal> to weak head normal +form and then returns <literal>y</literal>. The <function>seq</function> primitive can be used to +force evaluation of an expression beyond WHNF, or to impose a desired +execution sequence for the evaluation of an expression. +</para> + +<para> +For example, consider the following parallel version of our old +nemesis, <function>nfib</function>: +</para> + +<para> + +<programlisting> +import Parallel + +nfib :: Int -> Int +nfib n | n <= 1 = 1 + | otherwise = par n1 (seq n2 (n1 + n2 + 1)) + where n1 = nfib (n-1) + n2 = nfib (n-2) +</programlisting> + +</para> + +<para> +For values of <varname>n</varname> greater than 1, we use <function>par</function> to spark a thread +to evaluate <literal>nfib (n-1)</literal>, and then we use <function>seq</function> to force the +parent thread to evaluate <literal>nfib (n-2)</literal> before going on to add +together these two subexpressions. In this divide-and-conquer +approach, we only spark a new thread for one branch of the computation +(leaving the parent to evaluate the other branch). Also, we must use +<function>seq</function> to ensure that the parent will evaluate <varname>n2</varname> <emphasis>before</emphasis> +<varname>n1</varname> in the expression <literal>(n1 + n2 + 1)</literal>. It is not sufficient to +reorder the expression as <literal>(n2 + n1 + 1)</literal>, because the compiler may +not generate code to evaluate the addends from left to right. +</para> + +</sect3> + +<sect3> +<title>Underlying functions and primitives +<indexterm><primary>parallelism primitives</primary></indexterm> +<indexterm><primary>primitives for parallelism</primary></indexterm></title> + +<para> +The functions <function>par</function> and <function>seq</function> are wired into GHC, and unfold +into uses of the <function>par#</function> and <function>seq#</function> primitives, respectively. If +you'd like to see this with your very own eyes, just run GHC with the +<option>-ddump-simpl</option> option. (Anything for a good time…) +</para> + +</sect3> + +<sect3> +<title>Scheduling policy for concurrent threads +<indexterm><primary>Scheduling—concurrent</primary></indexterm> +<indexterm><primary>Concurrent scheduling</primary></indexterm></title> + +<para> +Runnable threads are scheduled in round-robin fashion. Context +switches are signalled by the generation of new sparks or by the +expiry of a virtual timer (the timer interval is configurable with the +<option>-C[<num>]</option><indexterm><primary>-C<num> RTS option (concurrent, +parallel)</primary></indexterm> RTS option). However, a context switch doesn't +really happen until the current heap block is full. You can't get any +faster context switching than this. +</para> + +<para> +When a context switch occurs, pending sparks which have not already +been reduced to weak head normal form are turned into new threads. +However, there is a limit to the number of active threads (runnable or +blocked) which are allowed at any given time. This limit can be +adjusted with the <option>-t<num></option><indexterm><primary>-t <num> RTS option (concurrent, parallel)</primary></indexterm> +RTS option (the default is 32). Once the +thread limit is reached, any remaining sparks are deferred until some +of the currently active threads are completed. +</para> + +</sect3> + +<sect3> +<title>Scheduling policy for parallel threads +<indexterm><primary>Scheduling—parallel</primary></indexterm> +<indexterm><primary>Parallel scheduling</primary></indexterm></title> + +<para> +In GUM we use an unfair scheduler, which means that a thread continues to +perform graph reduction until it blocks on a closure under evaluation, on a +remote closure or until the thread finishes. +</para> + +</sect3> + +</sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/phases.xml b/docs/users_guide/phases.xml new file mode 100644 index 0000000000..fd034a305a --- /dev/null +++ b/docs/users_guide/phases.xml @@ -0,0 +1,874 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="options-phases"> + <title>Options related to a particular phase</title> + + <sect2 id="replacing-phases"> + <title>Replacing the program for one or more phases</title> + <indexterm><primary>phases, changing</primary></indexterm> + + <para>You may specify that a different program be used for one + of the phases of the compilation system, in place of whatever + the <command>ghc</command> has wired into it. For example, you + might want to try a different assembler. The following options + allow you to change the external program used for a given + compilation phase:</para> + + <variablelist> + <varlistentry> + <term> + <option>-pgmL</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmL</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the literate + pre-processor.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgmP</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmP</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the C + pre-processor (with <option>-cpp</option> only).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgmc</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmc</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the C + compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgma</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgma</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the + assembler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgml</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgml</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the + linker.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgmdll</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmdll</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the DLL + generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgmdep</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmdep</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the dependency + generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-pgmF</option> <replaceable>cmd</replaceable> + <indexterm><primary><option>-pgmF</option></primary></indexterm> + </term> + <listitem> + <para>Use <replaceable>cmd</replaceable> as the + pre-processor (with <option>-F</option> only).</para> + </listitem> + </varlistentry> + + + </variablelist> + </sect2> + + <sect2 id="forcing-options-through"> + <title>Forcing options to a particular phase</title> + <indexterm><primary>forcing GHC-phase options</primary></indexterm> + + <para>Options can be forced through to a particlar compilation + phase, using the following flags:</para> + + <variablelist> + <varlistentry> + <term> + <option>-optL</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optL</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the + literate pre-processor</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optP</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optP</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to CPP (makes + sense only if <option>-cpp</option> is also on).</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optF</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optF</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the + custom pre-processor (see <xref linkend="pre-processor"/>).</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optc</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optc</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the C compiler.</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-opta</option> <replaceable>option</replaceable> + <indexterm><primary><option>-opta</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the assembler.</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optl</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optl</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the linker.</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optdll</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optdll</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the DLL generator.</para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>-optdep</option> <replaceable>option</replaceable> + <indexterm><primary><option>-optdep</option></primary></indexterm> + </term> + <listitem> + <para>Pass <replaceable>option</replaceable> to the + dependency generator.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>So, for example, to force an <option>-Ewurble</option> + option to the assembler, you would tell the driver + <option>-opta-Ewurble</option> (the dash before the E is + required).</para> + + <para>GHC is itself a Haskell program, so if you need to pass + options directly to GHC's runtime system you can enclose them in + <literal>+RTS ... -RTS</literal> (see <xref + linkend="runtime-control"/>).</para> + + </sect2> + + <sect2 id="c-pre-processor"> + <title>Options affecting the C pre-processor</title> + + <indexterm><primary>pre-processing: cpp</primary></indexterm> + <indexterm><primary>C pre-processor options</primary></indexterm> + <indexterm><primary>cpp, pre-processing with</primary></indexterm> + + <variablelist> + + <varlistentry> + <term> + <option>-cpp</option> + <indexterm><primary><option>-cpp</option></primary></indexterm> + </term> + <listitem> + <para>The C pre-processor <command>cpp</command> is run + over your Haskell code only if the <option>-cpp</option> + option <indexterm><primary>-cpp + option</primary></indexterm> is given. Unless you are + building a large system with significant doses of + conditional compilation, you really shouldn't need + it.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-D</option><replaceable>symbol</replaceable><optional>=<replaceable>value</replaceable></optional> + <indexterm><primary><option>-D</option></primary></indexterm> + </term> + <listitem> + <para>Define macro <replaceable>symbol</replaceable> in the + usual way. NB: does <emphasis>not</emphasis> affect + <option>-D</option> macros passed to the C compiler + when compiling via C! For those, use the + <option>-optc-Dfoo</option> hack… (see <xref + linkend="forcing-options-through"/>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-U</option><replaceable>symbol</replaceable> + <indexterm><primary><option>-U</option></primary></indexterm> + </term> + <listitem> + <para> Undefine macro <replaceable>symbol</replaceable> in the + usual way.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-I</option><replaceable>dir</replaceable> + <indexterm><primary><option>-I</option></primary></indexterm> + </term> + <listitem> + <para> Specify a directory in which to look for + <literal>#include</literal> files, in the usual C + way.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>The GHC driver pre-defines several macros when processing + Haskell source code (<filename>.hs</filename> or + <filename>.lhs</filename> files).</para> + + <para>The symbols defined by GHC are listed below. To check which + symbols are defined by your local GHC installation, the following + trick is useful:</para> + +<screen>$ ghc -E -optP-dM -cpp foo.hs +$ cat foo.hspp</screen> + + <para>(you need a file <filename>foo.hs</filename>, but it isn't + actually used).</para> + + <variablelist> + <varlistentry> + <term> + <constant>__HASKELL98__</constant> + <indexterm><primary><literal>__HASKELL98__</literal></primary></indexterm> + </term> + <listitem> + <para>If defined, this means that GHC supports the + language defined by the Haskell 98 report.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant>__HASKELL__=98</constant> + <indexterm><primary><constant>__HASKELL__=98</constant></primary></indexterm> + </term> + <listitem> + <para>In GHC 4.04 and later, the + <constant>__HASKELL__</constant> + macro is defined as having the value + <constant>98</constant>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant>__HASKELL1__</constant> + <indexterm><primary><constant>__HASKELL1__</constant></primary></indexterm> + </term> + <listitem> + <para>If defined to <replaceable>n</replaceable>, that + means GHC supports the Haskell language defined in the + Haskell report version <emphasis>1.n</emphasis>. + Currently 5. This macro is deprecated, and will probably + disappear in future versions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant>__GLASGOW_HASKELL__</constant> + <indexterm><primary><constant>__GLASGOW_HASKELL__</constant></primary></indexterm> + </term> + <listitem> + <para>For version + <literal><replaceable>x</replaceable>.<replaceable>y</replaceable>.<replaceable>z</replaceable></literal> + of GHC, the value of + <constant>__GLASGOW_HASKELL__</constant> + is the integer <replaceable>xyy</replaceable> (if + <replaceable>y</replaceable> is a single digit, then a leading zero + is added, so for example in version 6.2 of GHC, + <literal>__GLASGOW_HASKELL__==602</literal>). More + information in <xref linkend="version-numbering"/>.</para> + + <para>With any luck, + <constant>__GLASGOW_HASKELL__</constant> + will be undefined in all other implementations that + support C-style pre-processing.</para> + + <para>(For reference: the comparable symbols for other + systems are: + <constant>__HUGS__</constant> + for Hugs, + <constant>__NHC__</constant> + for nhc98, and + <constant>__HBC__</constant> + for hbc.)</para> + + <para>NB. This macro is set when pre-processing both + Haskell source and C source, including the C source + generated from a Haskell module + (i.e. <filename>.hs</filename>, <filename>.lhs</filename>, + <filename>.c</filename> and <filename>.hc</filename> + files).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant>__CONCURRENT_HASKELL__</constant> + <indexterm><primary><constant>__CONCURRENT_HASKELL__</constant></primary></indexterm> + </term> + <listitem> + <para>This symbol is defined when pre-processing Haskell + (input) and pre-processing C (GHC output). Since GHC from + verion 4.00 now supports concurrent haskell by default, + this symbol is always defined.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant>__PARALLEL_HASKELL__</constant> + <indexterm><primary><constant>__PARALLEL_HASKELL__</constant></primary></indexterm> + </term> + <listitem> + <para>Only defined when <option>-parallel</option> is in + use! This symbol is defined when pre-processing Haskell + (input) and pre-processing C (GHC output).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant><replaceable>os</replaceable>_HOST_OS=1</constant> + </term> + <listitem> + <para>This define allows conditional compilation based on + the Operating System, where<replaceable>os</replaceable> is + the name of the current Operating System + (eg. <literal>linux</literal>, <literal>mingw32</literal> + for Windows, <literal>solaris</literal>, etc.).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <constant><replaceable>arch</replaceable>_HOST_ARCH=1</constant> + </term> + <listitem> + <para>This define allows conditional compilation based on + the host architecture, where<replaceable>arch</replaceable> + is the name of the current architecture + (eg. <literal>i386</literal>, <literal>x86_64</literal>, + <literal>powerpc</literal>, <literal>sparc</literal>, + etc.).</para> + </listitem> + </varlistentry> + </variablelist> + + <sect3 id="cpp-string-gaps"> + <title>CPP and string gaps</title> + + <para>A small word of warning: <option>-cpp</option> is not + friendly to “string gaps”.<indexterm><primary>-cpp + vs string gaps</primary></indexterm><indexterm><primary>string + gaps vs -cpp</primary></indexterm>. In other words, strings + such as the following:</para> + +<programlisting>strmod = "\ +\ p \ +\ "</programlisting> + + <para>don't work with <option>-cpp</option>; + <filename>/usr/bin/cpp</filename> elides the backslash-newline + pairs.</para> + + <para>However, it appears that if you add a space at the end + of the line, then <command>cpp</command> (at least GNU + <command>cpp</command> and possibly other + <command>cpp</command>s) leaves the backslash-space pairs + alone and the string gap works as expected.</para> + </sect3> + </sect2> + + <sect2 id="pre-processor"> + <title>Options affecting a Haskell pre-processor</title> + + <indexterm><primary>pre-processing: custom</primary></indexterm> + <indexterm><primary>Pre-processor options</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-F</option> + <indexterm><primary><option>-F</option></primary></indexterm> + </term> + <listitem> + <para>A custom pre-processor is run over your Haskell + source file only if the <option>-F</option> option + <indexterm><primary>-F</primary></indexterm> is + given.</para> + + <para>Running a custom pre-processor at compile-time is in + some settings appropriate and useful. The + <option>-F</option> option lets you run a pre-processor as + part of the overall GHC compilation pipeline, which has + the advantage over running a Haskell pre-processor + separately in that it works in interpreted mode and you + can continue to take reap the benefits of GHC's + recompilation checker.</para> + + <para>The pre-processor is run just before the Haskell + compiler proper processes the Haskell input, but after the + literate markup has been stripped away and (possibly) the + C pre-processor has washed the Haskell input.</para> + + <para>Use + <option>-pgmF <replaceable>cmd</replaceable></option> + to select the program to use as the preprocessor. When + invoked, the <replaceable>cmd</replaceable> pre-processor + is given at least three arguments on its command-line: the + first argument is the name of the original source file, + the second is the name of the file holding the input, and + the third is the name of the file where + <replaceable>cmd</replaceable> should write its output + to.</para> + + <para>Additional arguments to the pre-processor can be + passed in using the <option>-optF</option> option. These + are fed to <replaceable>cmd</replaceable> on the command + line after the three standard input and output + arguments.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="options-C-compiler"> + <title>Options affecting the C compiler (if applicable)</title> + + <indexterm><primary>include-file options</primary></indexterm> + <indexterm><primary>C compiler options</primary></indexterm> + <indexterm><primary>GCC options</primary></indexterm> + + <para>If you are compiling with lots of foreign calls, you may + need to tell the C compiler about some + <literal>#include</literal> files. The Right Way to do this is to + add an <literal>INCLUDE</literal> pragma to the top of your source file + (<xref linkend="include-pragma" />):</para> + +<programlisting>{-# INCLUDE <X/Xlib.h> #-}</programlisting> + + <para>Sometimes this isn't convenient. In those cases there's an + equivalent command-line option:</para> + +<screen>% ghc -c '-#include <X/Xlib.h>' Xstuff.lhs</screen> + + <indexterm><primary><option>-#include</option></primary> + </indexterm> + + </sect2> + + <sect2 id="options-codegen"> + <title>Options affecting code generation</title> + + <variablelist> + <varlistentry> + <term> + <option>-fasm</option> + <indexterm><primary><option>-fasm</option></primary></indexterm> + </term> + <listitem> + <para>Use GHC's native code generator rather than + compiling via C. This will compile faster (up to twice as + fast), but may produce code that is slightly slower than + compiling via C. <option>-fasm</option> is the default + when optimisation is off (see <xref + linkend="options-optimise"/>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fvia-C</option> + <indexterm><primary><option>-fvia-C</option></primary></indexterm> + </term> + <listitem> + <para>Compile via C instead of using the native code + generator. This is default for optimised compilations, + and on architectures for which GHC doesn't have a native + code generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-code</option> + <indexterm><primary><option>-fno-code</option></primary></indexterm> + </term> + <listitem> + <para>Omit code generation (and all later phases) + altogether. Might be of some use if you just want to see + dumps of the intermediate compilation phases.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fPIC</option> + <indexterm><primary><option>-fPIC</option></primary></indexterm> + </term> + <listitem> + <para>Generate position-independent code (code that can be put into + shared libraries). This currently works on Mac OS X; it works on + PowerPC Linux when using the native code generator (-fasm). + It is not quite ready to be used yet for x86 Linux. + On Windows, position-independent code is never used, + and on PowerPC64 Linux, position-independent code is always used, + so the flag is a no-op on those platforms.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dynamic</option> + </term> + <listitem> + <para>When generating code, assume that entities imported from a + different package will reside in a different shared library or + binary. This currently works on Mac OS X; it works on PowerPC Linux when + using the native code generator. As with <option>-fPIC</option>, + x86 Linux support is not quite ready yet. Windows is not supported, + and it is a no-op on PowerPC64 Linux.</para> + <para>Note that this option also causes GHC to use shared libraries + when linking.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="options-linker"> + <title>Options affecting linking</title> + + <indexterm><primary>linker options</primary></indexterm> + <indexterm><primary>ld options</primary></indexterm> + + + <para>GHC has to link your code with various libraries, possibly + including: user-supplied, GHC-supplied, and system-supplied + (<option>-lm</option> math library, for example).</para> + + <variablelist> + + <varlistentry> + <term> + <option>-l</option><replaceable>lib</replaceable> + <indexterm><primary><option>-l</option></primary></indexterm> + </term> + <listitem> + <para>Link in the <replaceable>lib</replaceable> library. + On Unix systems, this will be in a file called + <filename>lib<replaceable>lib</replaceable>.a</filename> + or + <filename>lib<replaceable>lib</replaceable>.so</filename> + which resides somewhere on the library directories path.</para> + + <para>Because of the sad state of most UNIX linkers, the + order of such options does matter. If library + <replaceable>foo</replaceable> requires library + <replaceable>bar</replaceable>, then in general + <option>-l</option><replaceable>foo</replaceable> should + come <emphasis>before</emphasis> + <option>-l</option><replaceable>bar</replaceable> on the + command line.</para> + + <para>There's one other gotcha to bear in mind when using + external libraries: if the library contains a + <literal>main()</literal> function, then this will be + linked in preference to GHC's own + <literal>main()</literal> function + (eg. <literal>libf2c</literal> and <literal>libl</literal> + have their own <literal>main()</literal>s). This is + because GHC's <literal>main()</literal> comes from the + <literal>HSrts</literal> library, which is normally + included <emphasis>after</emphasis> all the other + libraries on the linker's command line. To force GHC's + <literal>main()</literal> to be used in preference to any + other <literal>main()</literal>s from external libraries, + just add the option <option>-lHSrts</option> before any + other libraries on the command line.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-c</option> + <indexterm><primary><option>-c</option></primary></indexterm> + </term> + <listitem> + <para>Omits the link step. This option can be used with + <option>––make</option> to avoid the automatic linking + that takes place if the program contains a <literal>Main</literal> + module.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-package</option> <replaceable>name</replaceable> + <indexterm><primary><option>-package</option></primary></indexterm> + </term> + <listitem> + <para>If you are using a Haskell “package” + (see <xref linkend="packages"/>), don't forget to add the + relevant <option>-package</option> option when linking the + program too: it will cause the appropriate libraries to be + linked in with the program. Forgetting the + <option>-package</option> option will likely result in + several pages of link errors.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-framework</option> <replaceable>name</replaceable> + <indexterm><primary><option>-framework</option></primary></indexterm> + </term> + <listitem> + <para>On Darwin/MacOS X only, link in the framework <replaceable>name</replaceable>. + This option corresponds to the <option>-framework</option> option for Apple's Linker. + Please note that frameworks and packages are two different things - frameworks don't + contain any haskell code. Rather, they are Apple's way of packaging shared libraries. + To link to Apple's “Carbon” API, for example, you'd use + <option>-framework Carbon</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-L</option><replaceable>dir</replaceable> + <indexterm><primary><option>-L</option></primary></indexterm> + </term> + <listitem> + <para>Where to find user-supplied libraries… + Prepend the directory <replaceable>dir</replaceable> to + the library directories path.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-framework-path</option><replaceable>dir</replaceable> + <indexterm><primary><option>-framework-path</option></primary></indexterm> + </term> + <listitem> + <para>On Darwin/MacOS X only, prepend the directory <replaceable>dir</replaceable> to + the framework directories path. This option corresponds to the <option>-F</option> + option for Apple's Linker (<option>-F</option> already means something else for GHC).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-split-objs</option> + <indexterm><primary><option>-split-objs</option></primary></indexterm> + </term> + <listitem> + <para>Tell the linker to split the single object file that + would normally be generated into multiple object files, + one per top-level Haskell function or type in the module. + We use this feature for building GHC's libraries libraries + (warning: don't use it unless you know what you're + doing!).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-static</option> + <indexterm><primary><option>-static</option></primary></indexterm> + </term> + <listitem> + <para>Tell the linker to avoid shared Haskell libraries, + if possible. This is the default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-dynamic</option> + <indexterm><primary><option>-dynamic</option></primary></indexterm> + </term> + <listitem> + <para>Tell the linker to use shared Haskell libraries, if + available (this option is only supported on Mac OS X at the + moment, and also note that your distribution of GHC may + not have been supplied with shared libraries).</para> + <para>Note that this option also has an effect on + code generation (see above).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-main-is <replaceable>thing</replaceable></option> + <indexterm><primary><option>-main-is</option></primary></indexterm> + <indexterm><primary>specifying your own main function</primary></indexterm> + </term> + <listitem> + <para> The normal rule in Haskell is that your program must supply a <literal>main</literal> + function in module <literal>Main</literal>. When testing, it is often convenient + to change which function is the "main" one, and the <option>-main-is</option> flag + allows you to do so. The <replaceable>thing</replaceable> can be one of: + <itemizedlist> + <listitem><para>A lower-case identifier <literal>foo</literal>. GHC assumes that the main function is <literal>Main.foo</literal>.</para></listitem> + <listitem><para>An module name <literal>A</literal>. GHC assumes that the main function is <literal>A.main</literal>.</para></listitem> + <listitem><para>An qualified name <literal>A.foo</literal>. GHC assumes that the main function is <literal>A.foo</literal>.</para></listitem> + </itemizedlist> + Strictly speaking, <option>-main-is</option> is not a link-phase flag at all; it has no effect on the link step. + The flag must be specified when compiling the module containing the specified main function (e.g. module <literal>A</literal> + in the latter two items above). It has no effect for other modules, + and hence can safely be given to <literal>ghc --make</literal>. + However, if all the modules are otherwise up to date, you may need to force + recompilation both of the module where the new "main" is, and of the + module where the "main" function used to be; + <literal>ghc</literal> is not clever + enough to figure out that they both need recompiling. You can + force recompilation by removing the object file, or by using the + <option>-no-recomp</option> flag. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-no-hs-main</option> + <indexterm><primary><option>-no-hs-main</option></primary></indexterm> + <indexterm><primary>linking Haskell libraries with foreign code</primary></indexterm> + </term> + <listitem> + <para>In the event you want to include ghc-compiled code + as part of another (non-Haskell) program, the RTS will not + be supplying its definition of <function>main()</function> + at link-time, you will have to. To signal that to the + compiler when linking, use + <option>-no-hs-main</option>. See also <xref linkend="using-own-main"/>.</para> + + <para>Notice that since the command-line passed to the + linker is rather involved, you probably want to use + <command>ghc</command> to do the final link of your + `mixed-language' application. This is not a requirement + though, just try linking once with <option>-v</option> on + to see what options the driver passes through to the + linker.</para> + + <para>The <option>-no-hs-main</option> flag can also be + used to persuade the compiler to do the link step in + <option>--make</option> mode when there is no Haskell + <literal>Main</literal> module present (normally the + compiler will not attempt linking when there is no + <literal>Main</literal>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-debug</option> + <indexterm><primary><option>-debug</option></primary></indexterm> + </term> + <listitem> + <para>Link the program with a debugging version of the + runtime system. The debugging runtime turns on numerous + assertions and sanity checks, and provides extra options + for producing debugging output at runtime (run the program + with <literal>+RTS -?</literal> to see a list).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-threaded</option> + <indexterm><primary><option>-threaded</option></primary></indexterm> + </term> + <listitem> + <para>Link the program with the "threaded" runtime system. + This version of the runtime is designed to be used in + programs that use multiple operating-system threads. It + supports calls to foreign-exported functions from multiple + OS threads. Calls to foreign functions are made using the + same OS thread that created the Haskell thread (if it was + created by a call-in), or an arbitrary OS thread otherwise + (if the Haskell thread was created by + <literal>forkIO</literal>).</para> + + <para>More details on the use of "bound threads" in the + threaded runtime can be found in the <ulink + url="../libraries/base/Control.Concurrent.html"><literal>Control.Concurrent</literal></ulink> module.</para> + + <para>The threaded RTS does <emphasis>not</emphasis> + support using multiple CPUs to speed up execution of a + multi-threaded Haskell program. The GHC runtime platform + is still single-threaded, but using the + <option>-threaded</option> option it can be used safely in + a multi-threaded environment.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/primitives.xml b/docs/users_guide/primitives.xml new file mode 100644 index 0000000000..e41bb59ee1 --- /dev/null +++ b/docs/users_guide/primitives.xml @@ -0,0 +1,1215 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!-- UNBOXED TYPES AND PRIMITIVE OPERATIONS --> + +<sect1 id="primitives"> + <title>Unboxed types and primitive operations</title> + <indexterm><primary>GHC.Exts module</primary></indexterm> + + <para>This chapter defines all the types which are primitive in + Glasgow Haskell, and the operations provided for them. You bring + them into scope by importing module <literal>GHC.Exts</literal>.</para> + + <para>Note: while you really can use this stuff to write fast code, + we generally find it a lot less painful, and more satisfying in the + long run, to use higher-level language features and libraries. With + any luck, the code you write will be optimised to the efficient + unboxed version in any case. And if it isn't, we'd like to know + about it.</para> + +<sect2 id="glasgow-unboxed"> +<title>Unboxed types +</title> + +<para> +<indexterm><primary>Unboxed types (Glasgow extension)</primary></indexterm> +</para> + +<para>Most types in GHC are <firstterm>boxed</firstterm>, which means +that values of that type are represented by a pointer to a heap +object. The representation of a Haskell <literal>Int</literal>, for +example, is a two-word heap object. An <firstterm>unboxed</firstterm> +type, however, is represented by the value itself, no pointers or heap +allocation are involved. +</para> + +<para> +Unboxed types correspond to the “raw machine” types you +would use in C: <literal>Int#</literal> (long int), +<literal>Double#</literal> (double), <literal>Addr#</literal> +(void *), etc. The <emphasis>primitive operations</emphasis> +(PrimOps) on these types are what you might expect; e.g., +<literal>(+#)</literal> is addition on +<literal>Int#</literal>s, and is the machine-addition that we all +know and love—usually one instruction. +</para> + +<para> +Primitive (unboxed) types cannot be defined in Haskell, and are +therefore built into the language and compiler. Primitive types are +always unlifted; that is, a value of a primitive type cannot be +bottom. We use the convention that primitive types, values, and +operations have a <literal>#</literal> suffix. +</para> + +<para> +Primitive values are often represented by a simple bit-pattern, such +as <literal>Int#</literal>, <literal>Float#</literal>, +<literal>Double#</literal>. But this is not necessarily the case: +a primitive value might be represented by a pointer to a +heap-allocated object. Examples include +<literal>Array#</literal>, the type of primitive arrays. A +primitive array is heap-allocated because it is too big a value to fit +in a register, and would be too expensive to copy around; in a sense, +it is accidental that it is represented by a pointer. If a pointer +represents a primitive value, then it really does point to that value: +no unevaluated thunks, no indirections…nothing can be at the +other end of the pointer than the primitive value. +</para> + +<para> +There are some restrictions on the use of primitive types, the main +one being that you can't pass a primitive value to a polymorphic +function or store one in a polymorphic data type. This rules out +things like <literal>[Int#]</literal> (i.e. lists of primitive +integers). The reason for this restriction is that polymorphic +arguments and constructor fields are assumed to be pointers: if an +unboxed integer is stored in one of these, the garbage collector would +attempt to follow it, leading to unpredictable space leaks. Or a +<function>seq</function> operation on the polymorphic component may +attempt to dereference the pointer, with disastrous results. Even +worse, the unboxed value might be larger than a pointer +(<literal>Double#</literal> for instance). +</para> + +<para> +Nevertheless, A numerically-intensive program using unboxed types can +go a <emphasis>lot</emphasis> faster than its “standard” +counterpart—we saw a threefold speedup on one example. +</para> + +</sect2> + +<sect2 id="unboxed-tuples"> +<title>Unboxed Tuples +</title> + +<para> +Unboxed tuples aren't really exported by <literal>GHC.Exts</literal>, +they're available by default with <option>-fglasgow-exts</option>. An +unboxed tuple looks like this: +</para> + +<para> + +<programlisting> +(# e_1, ..., e_n #) +</programlisting> + +</para> + +<para> +where <literal>e_1..e_n</literal> are expressions of any +type (primitive or non-primitive). The type of an unboxed tuple looks +the same. +</para> + +<para> +Unboxed tuples are used for functions that need to return multiple +values, but they avoid the heap allocation normally associated with +using fully-fledged tuples. When an unboxed tuple is returned, the +components are put directly into registers or on the stack; the +unboxed tuple itself does not have a composite representation. Many +of the primitive operations listed in this section return unboxed +tuples. +</para> + +<para> +There are some pretty stringent restrictions on the use of unboxed tuples: +</para> + +<para> + +<itemizedlist> +<listitem> + +<para> + Unboxed tuple types are subject to the same restrictions as +other unboxed types; i.e. they may not be stored in polymorphic data +structures or passed to polymorphic functions. + +</para> +</listitem> +<listitem> + +<para> + Unboxed tuples may only be constructed as the direct result of +a function, and may only be deconstructed with a <literal>case</literal> expression. +eg. the following are valid: + + +<programlisting> +f x y = (# x+1, y-1 #) +g x = case f x x of { (# a, b #) -> a + b } +</programlisting> + + +but the following are invalid: + + +<programlisting> +f x y = g (# x, y #) +g (# x, y #) = x + y +</programlisting> + + +</para> +</listitem> +<listitem> + +<para> + No variable can have an unboxed tuple type. This is illegal: + + +<programlisting> +f :: (# Int, Int #) -> (# Int, Int #) +f x = x +</programlisting> + + +because <literal>x</literal> has an unboxed tuple type. + +</para> +</listitem> + +</itemizedlist> + +</para> + +<para> +Note: we may relax some of these restrictions in the future. +</para> + +<para> +The <literal>IO</literal> and <literal>ST</literal> monads use unboxed +tuples to avoid unnecessary allocation during sequences of operations. +</para> + +</sect2> + +<sect2> +<title>Character and numeric types</title> + +<indexterm><primary>character types, primitive</primary></indexterm> +<indexterm><primary>numeric types, primitive</primary></indexterm> +<indexterm><primary>integer types, primitive</primary></indexterm> +<indexterm><primary>floating point types, primitive</primary></indexterm> +<para> +There are the following obvious primitive types: +</para> + +<programlisting> +type Char# +type Int# +type Word# +type Addr# +type Float# +type Double# +type Int64# +type Word64# +</programlisting> + +<indexterm><primary><literal>Char#</literal></primary></indexterm> +<indexterm><primary><literal>Int#</literal></primary></indexterm> +<indexterm><primary><literal>Word#</literal></primary></indexterm> +<indexterm><primary><literal>Addr#</literal></primary></indexterm> +<indexterm><primary><literal>Float#</literal></primary></indexterm> +<indexterm><primary><literal>Double#</literal></primary></indexterm> +<indexterm><primary><literal>Int64#</literal></primary></indexterm> +<indexterm><primary><literal>Word64#</literal></primary></indexterm> + +<para> +If you really want to know their exact equivalents in C, see +<filename>ghc/includes/StgTypes.h</filename> in the GHC source tree. +</para> + +<para> +Literals for these types may be written as follows: +</para> + +<para> + +<programlisting> +1# an Int# +1.2# a Float# +1.34## a Double# +'a'# a Char#; for weird characters, use e.g. '\o<octal>'# +"a"# an Addr# (a `char *'); only characters '\0'..'\255' allowed +</programlisting> + +<indexterm><primary>literals, primitive</primary></indexterm> +<indexterm><primary>constants, primitive</primary></indexterm> +<indexterm><primary>numbers, primitive</primary></indexterm> +</para> + +</sect2> + +<sect2> +<title>Comparison operations</title> + +<para> +<indexterm><primary>comparisons, primitive</primary></indexterm> +<indexterm><primary>operators, comparison</primary></indexterm> +</para> + +<para> + +<programlisting> +{>,>=,==,/=,<,<=}# :: Int# -> Int# -> Bool + +{gt,ge,eq,ne,lt,le}Char# :: Char# -> Char# -> Bool + -- ditto for Word# and Addr# +</programlisting> + +<indexterm><primary><literal>>#</literal></primary></indexterm> +<indexterm><primary><literal>>=#</literal></primary></indexterm> +<indexterm><primary><literal>==#</literal></primary></indexterm> +<indexterm><primary><literal>/=#</literal></primary></indexterm> +<indexterm><primary><literal><#</literal></primary></indexterm> +<indexterm><primary><literal><=#</literal></primary></indexterm> +<indexterm><primary><literal>gt{Char,Word,Addr}#</literal></primary></indexterm> +<indexterm><primary><literal>ge{Char,Word,Addr}#</literal></primary></indexterm> +<indexterm><primary><literal>eq{Char,Word,Addr}#</literal></primary></indexterm> +<indexterm><primary><literal>ne{Char,Word,Addr}#</literal></primary></indexterm> +<indexterm><primary><literal>lt{Char,Word,Addr}#</literal></primary></indexterm> +<indexterm><primary><literal>le{Char,Word,Addr}#</literal></primary></indexterm> +</para> + +</sect2> + +<sect2> +<title>Primitive-character operations</title> + +<para> +<indexterm><primary>characters, primitive operations</primary></indexterm> +<indexterm><primary>operators, primitive character</primary></indexterm> +</para> + +<para> + +<programlisting> +ord# :: Char# -> Int# +chr# :: Int# -> Char# +</programlisting> + +<indexterm><primary><literal>ord#</literal></primary></indexterm> +<indexterm><primary><literal>chr#</literal></primary></indexterm> +</para> + +</sect2> + +<sect2> +<title>Primitive-<literal>Int</literal> operations</title> + +<para> +<indexterm><primary>integers, primitive operations</primary></indexterm> +<indexterm><primary>operators, primitive integer</primary></indexterm> +</para> + +<para> + +<programlisting> +{+,-,*,quotInt,remInt,gcdInt}# :: Int# -> Int# -> Int# +negateInt# :: Int# -> Int# + +iShiftL#, iShiftRA#, iShiftRL# :: Int# -> Int# -> Int# + -- shift left, right arithmetic, right logical + +addIntC#, subIntC#, mulIntC# :: Int# -> Int# -> (# Int#, Int# #) + -- add, subtract, multiply with carry +</programlisting> + +<indexterm><primary><literal>+#</literal></primary></indexterm> +<indexterm><primary><literal>-#</literal></primary></indexterm> +<indexterm><primary><literal>*#</literal></primary></indexterm> +<indexterm><primary><literal>quotInt#</literal></primary></indexterm> +<indexterm><primary><literal>remInt#</literal></primary></indexterm> +<indexterm><primary><literal>gcdInt#</literal></primary></indexterm> +<indexterm><primary><literal>iShiftL#</literal></primary></indexterm> +<indexterm><primary><literal>iShiftRA#</literal></primary></indexterm> +<indexterm><primary><literal>iShiftRL#</literal></primary></indexterm> +<indexterm><primary><literal>addIntC#</literal></primary></indexterm> +<indexterm><primary><literal>subIntC#</literal></primary></indexterm> +<indexterm><primary><literal>mulIntC#</literal></primary></indexterm> +<indexterm><primary>shift operations, integer</primary></indexterm> +</para> + +<para> +<emphasis>Note:</emphasis> No error/overflow checking! +</para> + +</sect2> + +<sect2> +<title>Primitive-<literal>Double</literal> and <literal>Float</literal> operations</title> + +<para> +<indexterm><primary>floating point numbers, primitive</primary></indexterm> +<indexterm><primary>operators, primitive floating point</primary></indexterm> +</para> + +<para> + +<programlisting> +{+,-,*,/}## :: Double# -> Double# -> Double# +{<,<=,==,/=,>=,>}## :: Double# -> Double# -> Bool +negateDouble# :: Double# -> Double# +double2Int# :: Double# -> Int# +int2Double# :: Int# -> Double# + +{plus,minus,times,divide}Float# :: Float# -> Float# -> Float# +{gt,ge,eq,ne,lt,le}Float# :: Float# -> Float# -> Bool +negateFloat# :: Float# -> Float# +float2Int# :: Float# -> Int# +int2Float# :: Int# -> Float# +</programlisting> + +</para> + +<para> +<indexterm><primary><literal>+##</literal></primary></indexterm> +<indexterm><primary><literal>-##</literal></primary></indexterm> +<indexterm><primary><literal>*##</literal></primary></indexterm> +<indexterm><primary><literal>/##</literal></primary></indexterm> +<indexterm><primary><literal><##</literal></primary></indexterm> +<indexterm><primary><literal><=##</literal></primary></indexterm> +<indexterm><primary><literal>==##</literal></primary></indexterm> +<indexterm><primary><literal>=/##</literal></primary></indexterm> +<indexterm><primary><literal>>=##</literal></primary></indexterm> +<indexterm><primary><literal>>##</literal></primary></indexterm> +<indexterm><primary><literal>negateDouble#</literal></primary></indexterm> +<indexterm><primary><literal>double2Int#</literal></primary></indexterm> +<indexterm><primary><literal>int2Double#</literal></primary></indexterm> +</para> + +<para> +<indexterm><primary><literal>plusFloat#</literal></primary></indexterm> +<indexterm><primary><literal>minusFloat#</literal></primary></indexterm> +<indexterm><primary><literal>timesFloat#</literal></primary></indexterm> +<indexterm><primary><literal>divideFloat#</literal></primary></indexterm> +<indexterm><primary><literal>gtFloat#</literal></primary></indexterm> +<indexterm><primary><literal>geFloat#</literal></primary></indexterm> +<indexterm><primary><literal>eqFloat#</literal></primary></indexterm> +<indexterm><primary><literal>neFloat#</literal></primary></indexterm> +<indexterm><primary><literal>ltFloat#</literal></primary></indexterm> +<indexterm><primary><literal>leFloat#</literal></primary></indexterm> +<indexterm><primary><literal>negateFloat#</literal></primary></indexterm> +<indexterm><primary><literal>float2Int#</literal></primary></indexterm> +<indexterm><primary><literal>int2Float#</literal></primary></indexterm> +</para> + +<para> +And a full complement of trigonometric functions: +</para> + +<para> + +<programlisting> +expDouble# :: Double# -> Double# +logDouble# :: Double# -> Double# +sqrtDouble# :: Double# -> Double# +sinDouble# :: Double# -> Double# +cosDouble# :: Double# -> Double# +tanDouble# :: Double# -> Double# +asinDouble# :: Double# -> Double# +acosDouble# :: Double# -> Double# +atanDouble# :: Double# -> Double# +sinhDouble# :: Double# -> Double# +coshDouble# :: Double# -> Double# +tanhDouble# :: Double# -> Double# +powerDouble# :: Double# -> Double# -> Double# +</programlisting> + +<indexterm><primary>trigonometric functions, primitive</primary></indexterm> +</para> + +<para> +similarly for <literal>Float#</literal>. +</para> + +<para> +There are two coercion functions for <literal>Float#</literal>/<literal>Double#</literal>: +</para> + +<para> + +<programlisting> +float2Double# :: Float# -> Double# +double2Float# :: Double# -> Float# +</programlisting> + +<indexterm><primary><literal>float2Double#</literal></primary></indexterm> +<indexterm><primary><literal>double2Float#</literal></primary></indexterm> +</para> + +<para> +The primitive version of <function>decodeDouble</function> +(<function>encodeDouble</function> is implemented as an external C +function): +</para> + +<para> + +<programlisting> +decodeDouble# :: Double# -> PrelNum.ReturnIntAndGMP +</programlisting> + +<indexterm><primary><literal>encodeDouble#</literal></primary></indexterm> +<indexterm><primary><literal>decodeDouble#</literal></primary></indexterm> +</para> + +<para> +(And the same for <literal>Float#</literal>s.) +</para> + +</sect2> + +<sect2 id="integer-operations"> +<title>Operations on/for <literal>Integers</literal> (interface to GMP) +</title> + +<para> +<indexterm><primary>arbitrary precision integers</primary></indexterm> +<indexterm><primary>Integer, operations on</primary></indexterm> +</para> + +<para> +We implement <literal>Integers</literal> (arbitrary-precision +integers) using the GNU multiple-precision (GMP) package (version +2.0.2). +</para> + +<para> +The data type for <literal>Integer</literal> is either a small +integer, represented by an <literal>Int</literal>, or a large integer +represented using the pieces required by GMP's +<literal>MP_INT</literal> in <filename>gmp.h</filename> (see +<filename>gmp.info</filename> in +<filename>ghc/includes/runtime/gmp</filename>). It comes out as: +</para> + +<para> + +<programlisting> +data Integer = S# Int# -- small integers + | J# Int# ByteArray# -- large integers +</programlisting> + +<indexterm><primary>Integer type</primary></indexterm> The primitive +ops to support large <literal>Integers</literal> use the +“pieces” of the representation, and are as follows: +</para> + +<para> + +<programlisting> +negateInteger# :: Int# -> ByteArray# -> Integer + +{plus,minus,times}Integer#, gcdInteger#, + quotInteger#, remInteger#, divExactInteger# + :: Int# -> ByteArray# + -> Int# -> ByteArray# + -> (# Int#, ByteArray# #) + +cmpInteger# + :: Int# -> ByteArray# + -> Int# -> ByteArray# + -> Int# -- -1 for <; 0 for ==; +1 for > + +cmpIntegerInt# + :: Int# -> ByteArray# + -> Int# + -> Int# -- -1 for <; 0 for ==; +1 for > + +gcdIntegerInt# :: + :: Int# -> ByteArray# + -> Int# + -> Int# + +divModInteger#, quotRemInteger# + :: Int# -> ByteArray# + -> Int# -> ByteArray# + -> (# Int#, ByteArray#, + Int#, ByteArray# #) + +integer2Int# :: Int# -> ByteArray# -> Int# + +int2Integer# :: Int# -> Integer -- NB: no error-checking on these two! +word2Integer# :: Word# -> Integer + +addr2Integer# :: Addr# -> Integer + -- the Addr# is taken to be a `char *' string + -- to be converted into an Integer. +</programlisting> + +<indexterm><primary><literal>negateInteger#</literal></primary></indexterm> +<indexterm><primary><literal>plusInteger#</literal></primary></indexterm> +<indexterm><primary><literal>minusInteger#</literal></primary></indexterm> +<indexterm><primary><literal>timesInteger#</literal></primary></indexterm> +<indexterm><primary><literal>quotInteger#</literal></primary></indexterm> +<indexterm><primary><literal>remInteger#</literal></primary></indexterm> +<indexterm><primary><literal>gcdInteger#</literal></primary></indexterm> +<indexterm><primary><literal>gcdIntegerInt#</literal></primary></indexterm> +<indexterm><primary><literal>divExactInteger#</literal></primary></indexterm> +<indexterm><primary><literal>cmpInteger#</literal></primary></indexterm> +<indexterm><primary><literal>divModInteger#</literal></primary></indexterm> +<indexterm><primary><literal>quotRemInteger#</literal></primary></indexterm> +<indexterm><primary><literal>integer2Int#</literal></primary></indexterm> +<indexterm><primary><literal>int2Integer#</literal></primary></indexterm> +<indexterm><primary><literal>word2Integer#</literal></primary></indexterm> +<indexterm><primary><literal>addr2Integer#</literal></primary></indexterm> +</para> + +</sect2> + +<sect2> +<title>Words and addresses</title> + +<para> +<indexterm><primary>word, primitive type</primary></indexterm> +<indexterm><primary>address, primitive type</primary></indexterm> +<indexterm><primary>unsigned integer, primitive type</primary></indexterm> +<indexterm><primary>pointer, primitive type</primary></indexterm> +</para> + +<para> +A <literal>Word#</literal> is used for bit-twiddling operations. +It is the same size as an <literal>Int#</literal>, but has no sign +nor any arithmetic operations. + +<programlisting> +type Word# -- Same size/etc as Int# but *unsigned* +type Addr# -- A pointer from outside the "Haskell world" (from C, probably); + -- described under "arrays" +</programlisting> + +<indexterm><primary><literal>Word#</literal></primary></indexterm> +<indexterm><primary><literal>Addr#</literal></primary></indexterm> +</para> + +<para> +<literal>Word#</literal>s and <literal>Addr#</literal>s have +the usual comparison operations. Other +unboxed-<literal>Word</literal> ops (bit-twiddling and coercions): +</para> + +<para> + +<programlisting> +{gt,ge,eq,ne,lt,le}Word# :: Word# -> Word# -> Bool + +and#, or#, xor# :: Word# -> Word# -> Word# + -- standard bit ops. + +quotWord#, remWord# :: Word# -> Word# -> Word# + -- word (i.e. unsigned) versions are different from int + -- versions, so we have to provide these explicitly. + +not# :: Word# -> Word# + +shiftL#, shiftRL# :: Word# -> Int# -> Word# + -- shift left, right logical + +int2Word# :: Int# -> Word# -- just a cast, really +word2Int# :: Word# -> Int# +</programlisting> + +<indexterm><primary>bit operations, Word and Addr</primary></indexterm> +<indexterm><primary><literal>gtWord#</literal></primary></indexterm> +<indexterm><primary><literal>geWord#</literal></primary></indexterm> +<indexterm><primary><literal>eqWord#</literal></primary></indexterm> +<indexterm><primary><literal>neWord#</literal></primary></indexterm> +<indexterm><primary><literal>ltWord#</literal></primary></indexterm> +<indexterm><primary><literal>leWord#</literal></primary></indexterm> +<indexterm><primary><literal>and#</literal></primary></indexterm> +<indexterm><primary><literal>or#</literal></primary></indexterm> +<indexterm><primary><literal>xor#</literal></primary></indexterm> +<indexterm><primary><literal>not#</literal></primary></indexterm> +<indexterm><primary><literal>quotWord#</literal></primary></indexterm> +<indexterm><primary><literal>remWord#</literal></primary></indexterm> +<indexterm><primary><literal>shiftL#</literal></primary></indexterm> +<indexterm><primary><literal>shiftRA#</literal></primary></indexterm> +<indexterm><primary><literal>shiftRL#</literal></primary></indexterm> +<indexterm><primary><literal>int2Word#</literal></primary></indexterm> +<indexterm><primary><literal>word2Int#</literal></primary></indexterm> +</para> + +<para> +Unboxed-<literal>Addr</literal> ops (C casts, really): + +<programlisting> +{gt,ge,eq,ne,lt,le}Addr# :: Addr# -> Addr# -> Bool + +int2Addr# :: Int# -> Addr# +addr2Int# :: Addr# -> Int# +addr2Integer# :: Addr# -> (# Int#, ByteArray# #) +</programlisting> + +<indexterm><primary><literal>gtAddr#</literal></primary></indexterm> +<indexterm><primary><literal>geAddr#</literal></primary></indexterm> +<indexterm><primary><literal>eqAddr#</literal></primary></indexterm> +<indexterm><primary><literal>neAddr#</literal></primary></indexterm> +<indexterm><primary><literal>ltAddr#</literal></primary></indexterm> +<indexterm><primary><literal>leAddr#</literal></primary></indexterm> +<indexterm><primary><literal>int2Addr#</literal></primary></indexterm> +<indexterm><primary><literal>addr2Int#</literal></primary></indexterm> +<indexterm><primary><literal>addr2Integer#</literal></primary></indexterm> +</para> + +<para> +The casts between <literal>Int#</literal>, +<literal>Word#</literal> and <literal>Addr#</literal> +correspond to null operations at the machine level, but are required +to keep the Haskell type checker happy. +</para> + +<para> +Operations for indexing off of C pointers +(<literal>Addr#</literal>s) to snatch values are listed under +“arrays”. +</para> + +</sect2> + +<sect2> +<title>Arrays</title> + +<para> +<indexterm><primary>arrays, primitive</primary></indexterm> +</para> + +<para> +The type <literal>Array# elt</literal> is the type of primitive, +unpointed arrays of values of type <literal>elt</literal>. +</para> + +<para> + +<programlisting> +type Array# elt +</programlisting> + +<indexterm><primary><literal>Array#</literal></primary></indexterm> +</para> + +<para> +<literal>Array#</literal> is more primitive than a Haskell +array—indeed, the Haskell <literal>Array</literal> interface is +implemented using <literal>Array#</literal>—in that an +<literal>Array#</literal> is indexed only by +<literal>Int#</literal>s, starting at zero. It is also more +primitive by virtue of being unboxed. That doesn't mean that it isn't +a heap-allocated object—of course, it is. Rather, being unboxed +means that it is represented by a pointer to the array itself, and not +to a thunk which will evaluate to the array (or to bottom). The +components of an <literal>Array#</literal> are themselves boxed. +</para> + +<para> +The type <literal>ByteArray#</literal> is similar to +<literal>Array#</literal>, except that it contains just a string +of (non-pointer) bytes. +</para> + +<para> + +<programlisting> +type ByteArray# +</programlisting> + +<indexterm><primary><literal>ByteArray#</literal></primary></indexterm> +</para> + +<para> +Arrays of these types are useful when a Haskell program wishes to +construct a value to pass to a C procedure. It is also possible to use +them to build (say) arrays of unboxed characters for internal use in a +Haskell program. Given these uses, <literal>ByteArray#</literal> +is deliberately a bit vague about the type of its components. +Operations are provided to extract values of type +<literal>Char#</literal>, <literal>Int#</literal>, +<literal>Float#</literal>, <literal>Double#</literal>, and +<literal>Addr#</literal> from arbitrary offsets within a +<literal>ByteArray#</literal>. (For type +<literal>Foo#</literal>, the $i$th offset gets you the $i$th +<literal>Foo#</literal>, not the <literal>Foo#</literal> at +byte-position $i$. Mumble.) (If you want a +<literal>Word#</literal>, grab an <literal>Int#</literal>, +then coerce it.) +</para> + +<para> +Lastly, we have static byte-arrays, of type +<literal>Addr#</literal> [mentioned previously]. (Remember +the duality between arrays and pointers in C.) Arrays of this types +are represented by a pointer to an array in the world outside Haskell, +so this pointer is not followed by the garbage collector. In other +respects they are just like <literal>ByteArray#</literal>. They +are only needed in order to pass values from C to Haskell. +</para> + +</sect2> + +<sect2> +<title>Reading and writing</title> + +<para> +Primitive arrays are linear, and indexed starting at zero. +</para> + +<para> +The size and indices of a <literal>ByteArray#</literal>, <literal>Addr#</literal>, and +<literal>MutableByteArray#</literal> are all in bytes. It's up to the program to +calculate the correct byte offset from the start of the array. This +allows a <literal>ByteArray#</literal> to contain a mixture of values of different +type, which is often needed when preparing data for and unpicking +results from C. (Umm…not true of indices…WDP 95/09) +</para> + +<para> +<emphasis>Should we provide some <literal>sizeOfDouble#</literal> constants?</emphasis> +</para> + +<para> +Out-of-range errors on indexing should be caught by the code which +uses the primitive operation; the primitive operations themselves do +<emphasis>not</emphasis> check for out-of-range indexes. The intention is that the +primitive ops compile to one machine instruction or thereabouts. +</para> + +<para> +We use the terms “reading” and “writing” to refer to accessing +<emphasis>mutable</emphasis> arrays (see <xref linkend="sect-mutable">), and +“indexing” to refer to reading a value from an <emphasis>immutable</emphasis> +array. +</para> + +<para> +Immutable byte arrays are straightforward to index (all indices are in +units of the size of the object being read): + +<programlisting> +indexCharArray# :: ByteArray# -> Int# -> Char# +indexIntArray# :: ByteArray# -> Int# -> Int# +indexAddrArray# :: ByteArray# -> Int# -> Addr# +indexFloatArray# :: ByteArray# -> Int# -> Float# +indexDoubleArray# :: ByteArray# -> Int# -> Double# + +indexCharOffAddr# :: Addr# -> Int# -> Char# +indexIntOffAddr# :: Addr# -> Int# -> Int# +indexFloatOffAddr# :: Addr# -> Int# -> Float# +indexDoubleOffAddr# :: Addr# -> Int# -> Double# +indexAddrOffAddr# :: Addr# -> Int# -> Addr# + -- Get an Addr# from an Addr# offset +</programlisting> + +<indexterm><primary><literal>indexCharArray#</literal></primary></indexterm> +<indexterm><primary><literal>indexIntArray#</literal></primary></indexterm> +<indexterm><primary><literal>indexAddrArray#</literal></primary></indexterm> +<indexterm><primary><literal>indexFloatArray#</literal></primary></indexterm> +<indexterm><primary><literal>indexDoubleArray#</literal></primary></indexterm> +<indexterm><primary><literal>indexCharOffAddr#</literal></primary></indexterm> +<indexterm><primary><literal>indexIntOffAddr#</literal></primary></indexterm> +<indexterm><primary><literal>indexFloatOffAddr#</literal></primary></indexterm> +<indexterm><primary><literal>indexDoubleOffAddr#</literal></primary></indexterm> +<indexterm><primary><literal>indexAddrOffAddr#</literal></primary></indexterm> +</para> + +<para> +The last of these, <function>indexAddrOffAddr#</function>, extracts an <literal>Addr#</literal> using an offset +from another <literal>Addr#</literal>, thereby providing the ability to follow a chain of +C pointers. +</para> + +<para> +Something a bit more interesting goes on when indexing arrays of boxed +objects, because the result is simply the boxed object. So presumably +it should be entered—we never usually return an unevaluated +object! This is a pain: primitive ops aren't supposed to do +complicated things like enter objects. The current solution is to +return a single element unboxed tuple (see <xref linkend="unboxed-tuples">). +</para> + +<para> + +<programlisting> +indexArray# :: Array# elt -> Int# -> (# elt #) +</programlisting> + +<indexterm><primary><literal>indexArray#</literal></primary></indexterm> +</para> + +</sect2> + +<sect2> +<title>The state type</title> + +<para> +<indexterm><primary><literal>state, primitive type</literal></primary></indexterm> +<indexterm><primary><literal>State#</literal></primary></indexterm> +</para> + +<para> +The primitive type <literal>State#</literal> represents the state of a state +transformer. It is parameterised on the desired type of state, which +serves to keep states from distinct threads distinct from one another. +But the <emphasis>only</emphasis> effect of this parameterisation is in the type +system: all values of type <literal>State#</literal> are represented in the same way. +Indeed, they are all represented by nothing at all! The code +generator “knows” to generate no code, and allocate no registers +etc, for primitive states. +</para> + +<para> + +<programlisting> +type State# s +</programlisting> + +</para> + +<para> +The type <literal>GHC.RealWorld</literal> is truly opaque: there are no values defined +of this type, and no operations over it. It is “primitive” in that +sense - but it is <emphasis>not unlifted!</emphasis> Its only role in life is to be +the type which distinguishes the <literal>IO</literal> state transformer. +</para> + +<para> + +<programlisting> +data RealWorld +</programlisting> + +</para> + +</sect2> + +<sect2> +<title>State of the world</title> + +<para> +A single, primitive, value of type <literal>State# RealWorld</literal> is provided. +</para> + +<para> + +<programlisting> +realWorld# :: State# RealWorld +</programlisting> + +<indexterm><primary>realWorld# state object</primary></indexterm> +</para> + +<para> +(Note: in the compiler, not a <literal>PrimOp</literal>; just a mucho magic +<literal>Id</literal>. Exported from <literal>GHC</literal>, though). +</para> + +</sect2> + +<sect2 id="sect-mutable"> +<title>Mutable arrays</title> + +<para> +<indexterm><primary>mutable arrays</primary></indexterm> +<indexterm><primary>arrays, mutable</primary></indexterm> +Corresponding to <literal>Array#</literal> and <literal>ByteArray#</literal>, we have the types of +mutable versions of each. In each case, the representation is a +pointer to a suitable block of (mutable) heap-allocated storage. +</para> + +<para> + +<programlisting> +type MutableArray# s elt +type MutableByteArray# s +</programlisting> + +<indexterm><primary><literal>MutableArray#</literal></primary></indexterm> +<indexterm><primary><literal>MutableByteArray#</literal></primary></indexterm> +</para> + +<sect3> +<title>Allocation</title> + +<para> +<indexterm><primary>mutable arrays, allocation</primary></indexterm> +<indexterm><primary>arrays, allocation</primary></indexterm> +<indexterm><primary>allocation, of mutable arrays</primary></indexterm> +</para> + +<para> +Mutable arrays can be allocated. Only pointer-arrays are initialised; +arrays of non-pointers are filled in by “user code” rather than by +the array-allocation primitive. Reason: only the pointer case has to +worry about GC striking with a partly-initialised array. +</para> + +<para> + +<programlisting> +newArray# :: Int# -> elt -> State# s -> (# State# s, MutableArray# s elt #) + +newCharArray# :: Int# -> State# s -> (# State# s, MutableByteArray# s elt #) +newIntArray# :: Int# -> State# s -> (# State# s, MutableByteArray# s elt #) +newAddrArray# :: Int# -> State# s -> (# State# s, MutableByteArray# s elt #) +newFloatArray# :: Int# -> State# s -> (# State# s, MutableByteArray# s elt #) +newDoubleArray# :: Int# -> State# s -> (# State# s, MutableByteArray# s elt #) +</programlisting> + +<indexterm><primary><literal>newArray#</literal></primary></indexterm> +<indexterm><primary><literal>newCharArray#</literal></primary></indexterm> +<indexterm><primary><literal>newIntArray#</literal></primary></indexterm> +<indexterm><primary><literal>newAddrArray#</literal></primary></indexterm> +<indexterm><primary><literal>newFloatArray#</literal></primary></indexterm> +<indexterm><primary><literal>newDoubleArray#</literal></primary></indexterm> +</para> + +<para> +The size of a <literal>ByteArray#</literal> is given in bytes. +</para> + +</sect3> + +<sect3> +<title>Reading and writing</title> + +<para> +<indexterm><primary>arrays, reading and writing</primary></indexterm> +</para> + +<para> + +<programlisting> +readArray# :: MutableArray# s elt -> Int# -> State# s -> (# State# s, elt #) +readCharArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, Char# #) +readIntArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, Int# #) +readAddrArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, Addr# #) +readFloatArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, Float# #) +readDoubleArray# :: MutableByteArray# s -> Int# -> State# s -> (# State# s, Double# #) + +writeArray# :: MutableArray# s elt -> Int# -> elt -> State# s -> State# s +writeCharArray# :: MutableByteArray# s -> Int# -> Char# -> State# s -> State# s +writeIntArray# :: MutableByteArray# s -> Int# -> Int# -> State# s -> State# s +writeAddrArray# :: MutableByteArray# s -> Int# -> Addr# -> State# s -> State# s +writeFloatArray# :: MutableByteArray# s -> Int# -> Float# -> State# s -> State# s +writeDoubleArray# :: MutableByteArray# s -> Int# -> Double# -> State# s -> State# s +</programlisting> + +<indexterm><primary><literal>readArray#</literal></primary></indexterm> +<indexterm><primary><literal>readCharArray#</literal></primary></indexterm> +<indexterm><primary><literal>readIntArray#</literal></primary></indexterm> +<indexterm><primary><literal>readAddrArray#</literal></primary></indexterm> +<indexterm><primary><literal>readFloatArray#</literal></primary></indexterm> +<indexterm><primary><literal>readDoubleArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeCharArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeIntArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeAddrArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeFloatArray#</literal></primary></indexterm> +<indexterm><primary><literal>writeDoubleArray#</literal></primary></indexterm> +</para> + +</sect3> + +<sect3> +<title>Equality</title> + +<para> +<indexterm><primary>arrays, testing for equality</primary></indexterm> +</para> + +<para> +One can take “equality” of mutable arrays. What is compared is the +<emphasis>name</emphasis> or reference to the mutable array, not its contents. +</para> + +<para> + +<programlisting> +sameMutableArray# :: MutableArray# s elt -> MutableArray# s elt -> Bool +sameMutableByteArray# :: MutableByteArray# s -> MutableByteArray# s -> Bool +</programlisting> + +<indexterm><primary><literal>sameMutableArray#</literal></primary></indexterm> +<indexterm><primary><literal>sameMutableByteArray#</literal></primary></indexterm> +</para> + +</sect3> + +<sect3> +<title>Freezing mutable arrays</title> + +<para> +<indexterm><primary>arrays, freezing mutable</primary></indexterm> +<indexterm><primary>freezing mutable arrays</primary></indexterm> +<indexterm><primary>mutable arrays, freezing</primary></indexterm> +</para> + +<para> +Only unsafe-freeze has a primitive. (Safe freeze is done directly in Haskell +by copying the array and then using <function>unsafeFreeze</function>.) +</para> + +<para> + +<programlisting> +unsafeFreezeArray# :: MutableArray# s elt -> State# s -> (# State# s, Array# s elt #) +unsafeFreezeByteArray# :: MutableByteArray# s -> State# s -> (# State# s, ByteArray# #) +</programlisting> + +<indexterm><primary><literal>unsafeFreezeArray#</literal></primary></indexterm> +<indexterm><primary><literal>unsafeFreezeByteArray#</literal></primary></indexterm> +</para> + +</sect3> + +</sect2> + +<sect2> +<title>Synchronizing variables (M-vars)</title> + +<para> +<indexterm><primary>synchronising variables (M-vars)</primary></indexterm> +<indexterm><primary>M-Vars</primary></indexterm> +</para> + +<para> +Synchronising variables are the primitive type used to implement +Concurrent Haskell's MVars (see the Concurrent Haskell paper for +the operational behaviour of these operations). +</para> + +<para> + +<programlisting> +type MVar# s elt -- primitive + +newMVar# :: State# s -> (# State# s, MVar# s elt #) +takeMVar# :: SynchVar# s elt -> State# s -> (# State# s, elt #) +putMVar# :: SynchVar# s elt -> State# s -> State# s +</programlisting> + +<indexterm><primary><literal>SynchVar#</literal></primary></indexterm> +<indexterm><primary><literal>newSynchVar#</literal></primary></indexterm> +<indexterm><primary><literal>takeMVar</literal></primary></indexterm> +<indexterm><primary><literal>putMVar</literal></primary></indexterm> +</para> + +</sect2> + +<sect2 id="glasgow-prim-arrays"> +<title>Primitive arrays, mutable and otherwise +</title> + +<para> +<indexterm><primary>primitive arrays (Glasgow extension)</primary></indexterm> +<indexterm><primary>arrays, primitive (Glasgow extension)</primary></indexterm> +</para> + +<para> +GHC knows about quite a few flavours of Large Swathes of Bytes. +</para> + +<para> +First, GHC distinguishes between primitive arrays of (boxed) Haskell +objects (type <literal>Array# obj</literal>) and primitive arrays of bytes (type +<literal>ByteArray#</literal>). +</para> + +<para> +Second, it distinguishes between… +<variablelist> + +<varlistentry> +<term>Immutable:</term> +<listitem> +<para> +Arrays that do not change (as with “standard” Haskell arrays); you +can only read from them. Obviously, they do not need the care and +attention of the state-transformer monad. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term>Mutable:</term> +<listitem> +<para> +Arrays that may be changed or “mutated.” All the operations on them +live within the state-transformer monad and the updates happen +<emphasis>in-place</emphasis>. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term>“Static” (in C land):</term> +<listitem> +<para> +A C routine may pass an <literal>Addr#</literal> pointer back into Haskell land. There +are then primitive operations with which you may merrily grab values +over in C land, by indexing off the “static” pointer. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term>“Stable” pointers:</term> +<listitem> +<para> +If, for some reason, you wish to hand a Haskell pointer (i.e., +<emphasis>not</emphasis> an unboxed value) to a C routine, you first make the +pointer “stable,” so that the garbage collector won't forget that it +exists. That is, GHC provides a safe way to pass Haskell pointers to +C. +</para> + +<para> +Please see the module <literal>Foreign.StablePtr</literal> in the +library documentation for more details. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term>“Foreign objects”:</term> +<listitem> +<para> +A “foreign object” is a safe way to pass an external object (a +C-allocated pointer, say) to Haskell and have Haskell do the Right +Thing when it no longer references the object. So, for example, C +could pass a large bitmap over to Haskell and say “please free this +memory when you're done with it.” +</para> + +<para> +Please see module <literal>Foreign.ForeignPtr</literal> in the library +documentatation for more details. +</para> +</listitem> +</varlistentry> +</variablelist> +</para> + +<para> +The libraries documentatation gives more details on all these +“primitive array” types and the operations on them. +</para> + +</sect2> + +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/profiling.xml b/docs/users_guide/profiling.xml new file mode 100644 index 0000000000..a88c8bbf4c --- /dev/null +++ b/docs/users_guide/profiling.xml @@ -0,0 +1,1440 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="profiling"> + <title>Profiling</title> + <indexterm><primary>profiling</primary> + </indexterm> + <indexterm><primary>cost-centre profiling</primary></indexterm> + + <para> Glasgow Haskell comes with a time and space profiling + system. Its purpose is to help you improve your understanding of + your program's execution behaviour, so you can improve it.</para> + + <para> Any comments, suggestions and/or improvements you have are + welcome. Recommended “profiling tricks” would be + especially cool! </para> + + <para>Profiling a program is a three-step process:</para> + + <orderedlist> + <listitem> + <para> Re-compile your program for profiling with the + <literal>-prof</literal> option, and probably one of the + <literal>-auto</literal> or <literal>-auto-all</literal> + options. These options are described in more detail in <xref + linkend="prof-compiler-options"/> </para> + <indexterm><primary><literal>-prof</literal></primary> + </indexterm> + <indexterm><primary><literal>-auto</literal></primary> + </indexterm> + <indexterm><primary><literal>-auto-all</literal></primary> + </indexterm> + </listitem> + + <listitem> + <para> Run your program with one of the profiling options, eg. + <literal>+RTS -p -RTS</literal>. This generates a file of + profiling information.</para> + <indexterm><primary><option>-p</option></primary><secondary>RTS + option</secondary></indexterm> + </listitem> + + <listitem> + <para> Examine the generated profiling information, using one of + GHC's profiling tools. The tool to use will depend on the kind + of profiling information generated.</para> + </listitem> + + </orderedlist> + + <sect1 id="cost-centres"> + <title>Cost centres and cost-centre stacks</title> + + <para>GHC's profiling system assigns <firstterm>costs</firstterm> + to <firstterm>cost centres</firstterm>. A cost is simply the time + or space required to evaluate an expression. Cost centres are + program annotations around expressions; all costs incurred by the + annotated expression are assigned to the enclosing cost centre. + Furthermore, GHC will remember the stack of enclosing cost centres + for any given expression at run-time and generate a call-graph of + cost attributions.</para> + + <para>Let's take a look at an example:</para> + + <programlisting> +main = print (nfib 25) +nfib n = if n < 2 then 1 else nfib (n-1) + nfib (n-2) +</programlisting> + + <para>Compile and run this program as follows:</para> + + <screen> +$ ghc -prof -auto-all -o Main Main.hs +$ ./Main +RTS -p +121393 +$ +</screen> + + <para>When a GHC-compiled program is run with the + <option>-p</option> RTS option, it generates a file called + <filename><prog>.prof</filename>. In this case, the file + will contain something like this:</para> + +<screen> + Fri May 12 14:06 2000 Time and Allocation Profiling Report (Final) + + Main +RTS -p -RTS + + total time = 0.14 secs (7 ticks @ 20 ms) + total alloc = 8,741,204 bytes (excludes profiling overheads) + +COST CENTRE MODULE %time %alloc + +nfib Main 100.0 100.0 + + + individual inherited +COST CENTRE MODULE entries %time %alloc %time %alloc + +MAIN MAIN 0 0.0 0.0 100.0 100.0 + main Main 0 0.0 0.0 0.0 0.0 + CAF PrelHandle 3 0.0 0.0 0.0 0.0 + CAF PrelAddr 1 0.0 0.0 0.0 0.0 + CAF Main 6 0.0 0.0 100.0 100.0 + main Main 1 0.0 0.0 100.0 100.0 + nfib Main 242785 100.0 100.0 100.0 100.0 +</screen> + + + <para>The first part of the file gives the program name and + options, and the total time and total memory allocation measured + during the run of the program (note that the total memory + allocation figure isn't the same as the amount of + <emphasis>live</emphasis> memory needed by the program at any one + time; the latter can be determined using heap profiling, which we + will describe shortly).</para> + + <para>The second part of the file is a break-down by cost centre + of the most costly functions in the program. In this case, there + was only one significant function in the program, namely + <function>nfib</function>, and it was responsible for 100% + of both the time and allocation costs of the program.</para> + + <para>The third and final section of the file gives a profile + break-down by cost-centre stack. This is roughly a call-graph + profile of the program. In the example above, it is clear that + the costly call to <function>nfib</function> came from + <function>main</function>.</para> + + <para>The time and allocation incurred by a given part of the + program is displayed in two ways: “individual”, which + are the costs incurred by the code covered by this cost centre + stack alone, and “inherited”, which includes the costs + incurred by all the children of this node.</para> + + <para>The usefulness of cost-centre stacks is better demonstrated + by modifying the example slightly:</para> + + <programlisting> +main = print (f 25 + g 25) +f n = nfib n +g n = nfib (n `div` 2) +nfib n = if n < 2 then 1 else nfib (n-1) + nfib (n-2) +</programlisting> + + <para>Compile and run this program as before, and take a look at + the new profiling results:</para> + +<screen> +COST CENTRE MODULE scc %time %alloc %time %alloc + +MAIN MAIN 0 0.0 0.0 100.0 100.0 + main Main 0 0.0 0.0 0.0 0.0 + CAF PrelHandle 3 0.0 0.0 0.0 0.0 + CAF PrelAddr 1 0.0 0.0 0.0 0.0 + CAF Main 9 0.0 0.0 100.0 100.0 + main Main 1 0.0 0.0 100.0 100.0 + g Main 1 0.0 0.0 0.0 0.2 + nfib Main 465 0.0 0.2 0.0 0.2 + f Main 1 0.0 0.0 100.0 99.8 + nfib Main 242785 100.0 99.8 100.0 99.8 +</screen> + + <para>Now although we had two calls to <function>nfib</function> + in the program, it is immediately clear that it was the call from + <function>f</function> which took all the time.</para> + + <para>The actual meaning of the various columns in the output is:</para> + + <variablelist> + <varlistentry> + <term>entries</term> + <listitem> + <para>The number of times this particular point in the call + graph was entered.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>individual %time</term> + <listitem> + <para>The percentage of the total run time of the program + spent at this point in the call graph.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>individual %alloc</term> + <listitem> + <para>The percentage of the total memory allocations + (excluding profiling overheads) of the program made by this + call.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>inherited %time</term> + <listitem> + <para>The percentage of the total run time of the program + spent below this point in the call graph.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>inherited %alloc</term> + <listitem> + <para>The percentage of the total memory allocations + (excluding profiling overheads) of the program made by this + call and all of its sub-calls.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>In addition you can use the <option>-P</option> RTS option + <indexterm><primary><option>-P</option></primary></indexterm> to + get the following additional information:</para> + + <variablelist> + <varlistentry> + <term><literal>ticks</literal></term> + <listitem> + <para>The raw number of time “ticks” which were + attributed to this cost-centre; from this, we get the + <literal>%time</literal> figure mentioned + above.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>bytes</literal></term> + <listitem> + <para>Number of bytes allocated in the heap while in this + cost-centre; again, this is the raw number from which we get + the <literal>%alloc</literal> figure mentioned + above.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>What about recursive functions, and mutually recursive + groups of functions? Where are the costs attributed? Well, + although GHC does keep information about which groups of functions + called each other recursively, this information isn't displayed in + the basic time and allocation profile, instead the call-graph is + flattened into a tree. The XML profiling tool (described in <xref + linkend="prof-xml-tool"/>) will be able to display real loops in + the call-graph.</para> + + <sect2><title>Inserting cost centres by hand</title> + + <para>Cost centres are just program annotations. When you say + <option>-auto-all</option> to the compiler, it automatically + inserts a cost centre annotation around every top-level function + in your program, but you are entirely free to add the cost + centre annotations yourself.</para> + + <para>The syntax of a cost centre annotation is</para> + + <programlisting> + {-# SCC "name" #-} <expression> +</programlisting> + + <para>where <literal>"name"</literal> is an arbitrary string, + that will become the name of your cost centre as it appears + in the profiling output, and + <literal><expression></literal> is any Haskell + expression. An <literal>SCC</literal> annotation extends as + far to the right as possible when parsing.</para> + + </sect2> + + <sect2 id="prof-rules"> + <title>Rules for attributing costs</title> + + <para>The cost of evaluating any expression in your program is + attributed to a cost-centre stack using the following rules:</para> + + <itemizedlist> + <listitem> + <para>If the expression is part of the + <firstterm>one-off</firstterm> costs of evaluating the + enclosing top-level definition, then costs are attributed to + the stack of lexically enclosing <literal>SCC</literal> + annotations on top of the special <literal>CAF</literal> + cost-centre. </para> + </listitem> + + <listitem> + <para>Otherwise, costs are attributed to the stack of + lexically-enclosing <literal>SCC</literal> annotations, + appended to the cost-centre stack in effect at the + <firstterm>call site</firstterm> of the current top-level + definition<footnote> <para>The call-site is just the place + in the source code which mentions the particular function or + variable.</para></footnote>. Notice that this is a recursive + definition.</para> + </listitem> + + <listitem> + <para>Time spent in foreign code (see <xref linkend="ffi"/>) + is always attributed to the cost centre in force at the + Haskell call-site of the foreign function.</para> + </listitem> + </itemizedlist> + + <para>What do we mean by one-off costs? Well, Haskell is a lazy + language, and certain expressions are only ever evaluated once. + For example, if we write:</para> + + <programlisting> +x = nfib 25 +</programlisting> + + <para>then <varname>x</varname> will only be evaluated once (if + at all), and subsequent demands for <varname>x</varname> will + immediately get to see the cached result. The definition + <varname>x</varname> is called a CAF (Constant Applicative + Form), because it has no arguments.</para> + + <para>For the purposes of profiling, we say that the expression + <literal>nfib 25</literal> belongs to the one-off costs of + evaluating <varname>x</varname>.</para> + + <para>Since one-off costs aren't strictly speaking part of the + call-graph of the program, they are attributed to a special + top-level cost centre, <literal>CAF</literal>. There may be one + <literal>CAF</literal> cost centre for each module (the + default), or one for each top-level definition with any one-off + costs (this behaviour can be selected by giving GHC the + <option>-caf-all</option> flag).</para> + + <indexterm><primary><literal>-caf-all</literal></primary> + </indexterm> + + <para>If you think you have a weird profile, or the call-graph + doesn't look like you expect it to, feel free to send it (and + your program) to us at + <email>glasgow-haskell-bugs@haskell.org</email>.</para> + </sect2> + </sect1> + + <sect1 id="prof-compiler-options"> + <title>Compiler options for profiling</title> + + <indexterm><primary>profiling</primary><secondary>options</secondary></indexterm> + <indexterm><primary>options</primary><secondary>for profiling</secondary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-prof</option>: + <indexterm><primary><option>-prof</option></primary></indexterm> + </term> + <listitem> + <para> To make use of the profiling system + <emphasis>all</emphasis> modules must be compiled and linked + with the <option>-prof</option> option. Any + <literal>SCC</literal> annotations you've put in your source + will spring to life.</para> + + <para> Without a <option>-prof</option> option, your + <literal>SCC</literal>s are ignored; so you can compile + <literal>SCC</literal>-laden code without changing + it.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>There are a few other profiling-related compilation options. + Use them <emphasis>in addition to</emphasis> + <option>-prof</option>. These do not have to be used consistently + for all modules in a program.</para> + + <variablelist> + <varlistentry> + <term> + <option>-auto</option>: + <indexterm><primary><option>-auto</option></primary></indexterm> + <indexterm><primary>cost centres</primary><secondary>automatically inserting</secondary></indexterm> + </term> + <listitem> + <para> GHC will automatically add + <function>_scc_</function> constructs for all + top-level, exported functions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-auto-all</option>: + <indexterm><primary><option>-auto-all</option></primary></indexterm> + </term> + <listitem> + <para> <emphasis>All</emphasis> top-level functions, + exported or not, will be automatically + <function>_scc_</function>'d.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-caf-all</option>: + <indexterm><primary><option>-caf-all</option></primary></indexterm> + </term> + <listitem> + <para> The costs of all CAFs in a module are usually + attributed to one “big” CAF cost-centre. With + this option, all CAFs get their own cost-centre. An + “if all else fails” option…</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ignore-scc</option>: + <indexterm><primary><option>-ignore-scc</option></primary></indexterm> + </term> + <listitem> + <para>Ignore any <function>_scc_</function> + constructs, so a module which already has + <function>_scc_</function>s can be compiled + for profiling with the annotations ignored.</para> + </listitem> + </varlistentry> + + </variablelist> + + </sect1> + + <sect1 id="prof-time-options"> + <title>Time and allocation profiling</title> + + <para>To generate a time and allocation profile, give one of the + following RTS options to the compiled program when you run it (RTS + options should be enclosed between <literal>+RTS...-RTS</literal> + as usual):</para> + + <variablelist> + <varlistentry> + <term> + <option>-p</option> or <option>-P</option>: + <indexterm><primary><option>-p</option></primary></indexterm> + <indexterm><primary><option>-P</option></primary></indexterm> + <indexterm><primary>time profile</primary></indexterm> + </term> + <listitem> + <para>The <option>-p</option> option produces a standard + <emphasis>time profile</emphasis> report. It is written + into the file + <filename><replaceable>program</replaceable>.prof</filename>.</para> + + <para>The <option>-P</option> option produces a more + detailed report containing the actual time and allocation + data as well. (Not used much.)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-px</option>: + <indexterm><primary><option>-px</option></primary></indexterm> + </term> + <listitem> + <para>The <option>-px</option> option generates profiling + information in the XML format understood by our new + profiling tool, see <xref linkend="prof-xml-tool"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-xc</option> + <indexterm><primary><option>-xc</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>This option makes use of the extra information + maintained by the cost-centre-stack profiler to provide + useful information about the location of runtime errors. + See <xref linkend="rts-options-debugging"/>.</para> + </listitem> + </varlistentry> + + </variablelist> + + </sect1> + + <sect1 id="prof-heap"> + <title>Profiling memory usage</title> + + <para>In addition to profiling the time and allocation behaviour + of your program, you can also generate a graph of its memory usage + over time. This is useful for detecting the causes of + <firstterm>space leaks</firstterm>, when your program holds on to + more memory at run-time that it needs to. Space leaks lead to + longer run-times due to heavy garbage collector activity, and may + even cause the program to run out of memory altogether.</para> + + <para>To generate a heap profile from your program:</para> + + <orderedlist> + <listitem> + <para>Compile the program for profiling (<xref + linkend="prof-compiler-options"/>).</para> + </listitem> + <listitem> + <para>Run it with one of the heap profiling options described + below (eg. <option>-hc</option> for a basic producer profile). + This generates the file + <filename><replaceable>prog</replaceable>.hp</filename>.</para> + </listitem> + <listitem> + <para>Run <command>hp2ps</command> to produce a Postscript + file, + <filename><replaceable>prog</replaceable>.ps</filename>. The + <command>hp2ps</command> utility is described in detail in + <xref linkend="hp2ps"/>.</para> + </listitem> + <listitem> + <para>Display the heap profile using a postscript viewer such + as <application>Ghostview</application>, or print it out on a + Postscript-capable printer.</para> + </listitem> + </orderedlist> + + <sect2 id="rts-options-heap-prof"> + <title>RTS options for heap profiling</title> + + <para>There are several different kinds of heap profile that can + be generated. All the different profile types yield a graph of + live heap against time, but they differ in how the live heap is + broken down into bands. The following RTS options select which + break-down to use:</para> + + <variablelist> + <varlistentry> + <term> + <option>-hc</option> + <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Breaks down the graph by the cost-centre stack which + produced the data.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hm</option> + <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Break down the live heap by the module containing + the code which produced the data.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hd</option> + <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Breaks down the graph by <firstterm>closure + description</firstterm>. For actual data, the description + is just the constructor name, for other closures it is a + compiler-generated string identifying the closure.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hy</option> + <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Breaks down the graph by + <firstterm>type</firstterm>. For closures which have + function type or unknown/polymorphic type, the string will + represent an approximation to the actual type.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hr</option> + <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Break down the graph by <firstterm>retainer + set</firstterm>. Retainer profiling is described in more + detail below (<xref linkend="retainer-prof"/>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hb</option> + <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Break down the graph by + <firstterm>biography</firstterm>. Biographical profiling + is described in more detail below (<xref + linkend="biography-prof"/>).</para> + </listitem> + </varlistentry> + </variablelist> + + <para>In addition, the profile can be restricted to heap data + which satisfies certain criteria - for example, you might want + to display a profile by type but only for data produced by a + certain module, or a profile by retainer for a certain type of + data. Restrictions are specified as follows:</para> + + <variablelist> + <varlistentry> + <term> + <option>-hc</option><replaceable>name</replaceable>,... + <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures produced by + cost-centre stacks with one of the specified cost centres + at the top.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hC</option><replaceable>name</replaceable>,... + <indexterm><primary><option>-hC</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures produced by + cost-centre stacks with one of the specified cost centres + anywhere in the stack.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hm</option><replaceable>module</replaceable>,... + <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures produced by the + specified modules.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hd</option><replaceable>desc</replaceable>,... + <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures with the specified + description strings.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hy</option><replaceable>type</replaceable>,... + <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures with the specified + types.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hr</option><replaceable>cc</replaceable>,... + <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures with retainer sets + containing cost-centre stacks with one of the specified + cost centres at the top.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hb</option><replaceable>bio</replaceable>,... + <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Restrict the profile to closures with one of the + specified biographies, where + <replaceable>bio</replaceable> is one of + <literal>lag</literal>, <literal>drag</literal>, + <literal>void</literal>, or <literal>use</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>For example, the following options will generate a + retainer profile restricted to <literal>Branch</literal> and + <literal>Leaf</literal> constructors:</para> + +<screen> +<replaceable>prog</replaceable> +RTS -hr -hdBranch,Leaf +</screen> + + <para>There can only be one "break-down" option + (eg. <option>-hr</option> in the example above), but there is no + limit on the number of further restrictions that may be applied. + All the options may be combined, with one exception: GHC doesn't + currently support mixing the <option>-hr</option> and + <option>-hb</option> options.</para> + + <para>There are two more options which relate to heap + profiling:</para> + + <variablelist> + <varlistentry> + <term> + <option>-i<replaceable>secs</replaceable></option>: + <indexterm><primary><option>-i</option></primary></indexterm> + </term> + <listitem> + <para>Set the profiling (sampling) interval to + <replaceable>secs</replaceable> seconds (the default is + 0.1 second). Fractions are allowed: for example + <option>-i0.2</option> will get 5 samples per second. + This only affects heap profiling; time profiles are always + sampled on a 1/50 second frequency.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-xt</option> + <indexterm><primary><option>-xt</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Include the memory occupied by threads in a heap + profile. Each thread takes up a small area for its thread + state in addition to the space allocated for its stack + (stacks normally start small and then grow as + necessary).</para> + + <para>This includes the main thread, so using + <option>-xt</option> is a good way to see how much stack + space the program is using.</para> + + <para>Memory occupied by threads and their stacks is + labelled as “TSO” when displaying the profile + by closure description or type description.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2 id="retainer-prof"> + <title>Retainer Profiling</title> + + <para>Retainer profiling is designed to help answer questions + like <quote>why is this data being retained?</quote>. We start + by defining what we mean by a retainer:</para> + + <blockquote> + <para>A retainer is either the system stack, or an unevaluated + closure (thunk).</para> + </blockquote> + + <para>In particular, constructors are <emphasis>not</emphasis> + retainers.</para> + + <para>An object B retains object A if (i) B is a retainer object and + (ii) object A can be reached by recursively following pointers + starting from object B, but not meeting any other retainer + objects on the way. Each live object is retained by one or more + retainer objects, collectively called its retainer set, or its + <firstterm>retainer set</firstterm>, or its + <firstterm>retainers</firstterm>.</para> + + <para>When retainer profiling is requested by giving the program + the <option>-hr</option> option, a graph is generated which is + broken down by retainer set. A retainer set is displayed as a + set of cost-centre stacks; because this is usually too large to + fit on the profile graph, each retainer set is numbered and + shown abbreviated on the graph along with its number, and the + full list of retainer sets is dumped into the file + <filename><replaceable>prog</replaceable>.prof</filename>.</para> + + <para>Retainer profiling requires multiple passes over the live + heap in order to discover the full retainer set for each + object, which can be quite slow. So we set a limit on the + maximum size of a retainer set, where all retainer sets larger + than the maximum retainer set size are replaced by the special + set <literal>MANY</literal>. The maximum set size defaults to 8 + and can be altered with the <option>-R</option> RTS + option:</para> + + <variablelist> + <varlistentry> + <term><option>-R</option><replaceable>size</replaceable></term> + <listitem> + <para>Restrict the number of elements in a retainer set to + <replaceable>size</replaceable> (default 8).</para> + </listitem> + </varlistentry> + </variablelist> + + <sect3> + <title>Hints for using retainer profiling</title> + + <para>The definition of retainers is designed to reflect a + common cause of space leaks: a large structure is retained by + an unevaluated computation, and will be released once the + computation is forced. A good example is looking up a value in + a finite map, where unless the lookup is forced in a timely + manner the unevaluated lookup will cause the whole mapping to + be retained. These kind of space leaks can often be + eliminated by forcing the relevant computations to be + performed eagerly, using <literal>seq</literal> or strictness + annotations on data constructor fields.</para> + + <para>Often a particular data structure is being retained by a + chain of unevaluated closures, only the nearest of which will + be reported by retainer profiling - for example A retains B, B + retains C, and C retains a large structure. There might be a + large number of Bs but only a single A, so A is really the one + we're interested in eliminating. However, retainer profiling + will in this case report B as the retainer of the large + structure. To move further up the chain of retainers, we can + ask for another retainer profile but this time restrict the + profile to B objects, so we get a profile of the retainers of + B:</para> + +<screen> +<replaceable>prog</replaceable> +RTS -hr -hcB +</screen> + + <para>This trick isn't foolproof, because there might be other + B closures in the heap which aren't the retainers we are + interested in, but we've found this to be a useful technique + in most cases.</para> + </sect3> + </sect2> + + <sect2 id="biography-prof"> + <title>Biographical Profiling</title> + + <para>A typical heap object may be in one of the following four + states at each point in its lifetime:</para> + + <itemizedlist> + <listitem> + <para>The <firstterm>lag</firstterm> stage, which is the + time between creation and the first use of the + object,</para> + </listitem> + <listitem> + <para>the <firstterm>use</firstterm> stage, which lasts from + the first use until the last use of the object, and</para> + </listitem> + <listitem> + <para>The <firstterm>drag</firstterm> stage, which lasts + from the final use until the last reference to the object + is dropped.</para> + </listitem> + <listitem> + <para>An object which is never used is said to be in the + <firstterm>void</firstterm> state for its whole + lifetime.</para> + </listitem> + </itemizedlist> + + <para>A biographical heap profile displays the portion of the + live heap in each of the four states listed above. Usually the + most interesting states are the void and drag states: live heap + in these states is more likely to be wasted space than heap in + the lag or use states.</para> + + <para>It is also possible to break down the heap in one or more + of these states by a different criteria, by restricting a + profile by biography. For example, to show the portion of the + heap in the drag or void state by producer: </para> + +<screen> +<replaceable>prog</replaceable> +RTS -hc -hbdrag,void +</screen> + + <para>Once you know the producer or the type of the heap in the + drag or void states, the next step is usually to find the + retainer(s):</para> + +<screen> +<replaceable>prog</replaceable> +RTS -hr -hc<replaceable>cc</replaceable>... +</screen> + + <para>NOTE: this two stage process is required because GHC + cannot currently profile using both biographical and retainer + information simultaneously.</para> + </sect2> + + <sect2 id="mem-residency"> + <title>Actual memory residency</title> + + <para>How does the heap residency reported by the heap profiler relate to + the actual memory residency of your program when you run it? You might + see a large discrepancy between the residency reported by the heap + profiler, and the residency reported by tools on your system + (eg. <literal>ps</literal> or <literal>top</literal> on Unix, or the + Task Manager on Windows). There are several reasons for this:</para> + + <itemizedlist> + <listitem> + <para>There is an overhead of profiling itself, which is subtracted + from the residency figures by the profiler. This overhead goes + away when compiling without profiling support, of course. The + space overhead is currently 2 extra + words per heap object, which probably results in + about a 30% overhead.</para> + </listitem> + + <listitem> + <para>Garbage collection requires more memory than the actual + residency. The factor depends on the kind of garbage collection + algorithm in use: a major GC in the standard + generation copying collector will usually require 3L bytes of + memory, where L is the amount of live data. This is because by + default (see the <option>+RTS -F</option> option) we allow the old + generation to grow to twice its size (2L) before collecting it, and + we require additionally L bytes to copy the live data into. When + using compacting collection (see the <option>+RTS -c</option> + option), this is reduced to 2L, and can further be reduced by + tweaking the <option>-F</option> option. Also add the size of the + allocation area (currently a fixed 512Kb).</para> + </listitem> + + <listitem> + <para>The stack isn't counted in the heap profile by default. See the + <option>+RTS -xt</option> option.</para> + </listitem> + + <listitem> + <para>The program text itself, the C stack, any non-heap data (eg. data + allocated by foreign libraries, and data allocated by the RTS), and + <literal>mmap()</literal>'d memory are not counted in the heap profile.</para> + </listitem> + </itemizedlist> + </sect2> + + </sect1> + + <sect1 id="prof-xml-tool"> + <title>Graphical time/allocation profile</title> + + <para>You can view the time and allocation profiling graph of your + program graphically, using <command>ghcprof</command>. This is a + new tool with GHC 4.08, and will eventually be the de-facto + standard way of viewing GHC profiles<footnote><para>Actually this + isn't true any more, we are working on a new tool for + displaying heap profiles using Gtk+HS, so + <command>ghcprof</command> may go away at some point in the future.</para> + </footnote></para> + + <para>To run <command>ghcprof</command>, you need + <productname>uDraw(Graph)</productname> installed, which can be + obtained from <ulink + url="http://www.informatik.uni-bremen.de/uDrawGraph/en/uDrawGraph/uDrawGraph.html"><citetitle>uDraw(Graph)</citetitle></ulink>. Install one of + the binary + distributions, and set your + <envar>UDG_HOME</envar> environment variable to point to the + installation directory.</para> + + <para><command>ghcprof</command> uses an XML-based profiling log + format, and you therefore need to run your program with a + different option: <option>-px</option>. The file generated is + still called <filename><prog>.prof</filename>. To see the + profile, run <command>ghcprof</command> like this:</para> + + <indexterm><primary><option>-px</option></primary></indexterm> + +<screen> +$ ghcprof <prog>.prof +</screen> + + <para>which should pop up a window showing the call-graph of your + program in glorious detail. More information on using + <command>ghcprof</command> can be found at <ulink + url="http://www.dcs.warwick.ac.uk/people/academic/Stephen.Jarvis/profiler/index.html"><citetitle>The + Cost-Centre Stack Profiling Tool for + GHC</citetitle></ulink>.</para> + + </sect1> + + <sect1 id="hp2ps"> + <title><command>hp2ps</command>––heap profile to PostScript</title> + + <indexterm><primary><command>hp2ps</command></primary></indexterm> + <indexterm><primary>heap profiles</primary></indexterm> + <indexterm><primary>postscript, from heap profiles</primary></indexterm> + <indexterm><primary><option>-h<break-down></option></primary></indexterm> + + <para>Usage:</para> + +<screen> +hp2ps [flags] [<file>[.hp]] +</screen> + + <para>The program + <command>hp2ps</command><indexterm><primary>hp2ps + program</primary></indexterm> converts a heap profile as produced + by the <option>-h<break-down></option> runtime option into a + PostScript graph of the heap profile. By convention, the file to + be processed by <command>hp2ps</command> has a + <filename>.hp</filename> extension. The PostScript output is + written to <filename><file>@.ps</filename>. If + <filename><file></filename> is omitted entirely, then the + program behaves as a filter.</para> + + <para><command>hp2ps</command> is distributed in + <filename>ghc/utils/hp2ps</filename> in a GHC source + distribution. It was originally developed by Dave Wakeling as part + of the HBC/LML heap profiler.</para> + + <para>The flags are:</para> + + <variablelist> + + <varlistentry> + <term><option>-d</option></term> + <listitem> + <para>In order to make graphs more readable, + <command>hp2ps</command> sorts the shaded bands for each + identifier. The default sort ordering is for the bands with + the largest area to be stacked on top of the smaller ones. + The <option>-d</option> option causes rougher bands (those + representing series of values with the largest standard + deviations) to be stacked on top of smoother ones.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-b</option></term> + <listitem> + <para>Normally, <command>hp2ps</command> puts the title of + the graph in a small box at the top of the page. However, if + the JOB string is too long to fit in a small box (more than + 35 characters), then <command>hp2ps</command> will choose to + use a big box instead. The <option>-b</option> option + forces <command>hp2ps</command> to use a big box.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-e<float>[in|mm|pt]</option></term> + <listitem> + <para>Generate encapsulated PostScript suitable for + inclusion in LaTeX documents. Usually, the PostScript graph + is drawn in landscape mode in an area 9 inches wide by 6 + inches high, and <command>hp2ps</command> arranges for this + area to be approximately centred on a sheet of a4 paper. + This format is convenient of studying the graph in detail, + but it is unsuitable for inclusion in LaTeX documents. The + <option>-e</option> option causes the graph to be drawn in + portrait mode, with float specifying the width in inches, + millimetres or points (the default). The resulting + PostScript file conforms to the Encapsulated PostScript + (EPS) convention, and it can be included in a LaTeX document + using Rokicki's dvi-to-PostScript converter + <command>dvips</command>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-g</option></term> + <listitem> + <para>Create output suitable for the <command>gs</command> + PostScript previewer (or similar). In this case the graph is + printed in portrait mode without scaling. The output is + unsuitable for a laser printer.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-l</option></term> + <listitem> + <para>Normally a profile is limited to 20 bands with + additional identifiers being grouped into an + <literal>OTHER</literal> band. The <option>-l</option> flag + removes this 20 band and limit, producing as many bands as + necessary. No key is produced as it won't fit!. It is useful + for creation time profiles with many bands.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-m<int></option></term> + <listitem> + <para>Normally a profile is limited to 20 bands with + additional identifiers being grouped into an + <literal>OTHER</literal> band. The <option>-m</option> flag + specifies an alternative band limit (the maximum is + 20).</para> + + <para><option>-m0</option> requests the band limit to be + removed. As many bands as necessary are produced. However no + key is produced as it won't fit! It is useful for displaying + creation time profiles with many bands.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-p</option></term> + <listitem> + <para>Use previous parameters. By default, the PostScript + graph is automatically scaled both horizontally and + vertically so that it fills the page. However, when + preparing a series of graphs for use in a presentation, it + is often useful to draw a new graph using the same scale, + shading and ordering as a previous one. The + <option>-p</option> flag causes the graph to be drawn using + the parameters determined by a previous run of + <command>hp2ps</command> on <filename>file</filename>. These + are extracted from <filename>file@.aux</filename>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-s</option></term> + <listitem> + <para>Use a small box for the title.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-t<float></option></term> + <listitem> + <para>Normally trace elements which sum to a total of less + than 1% of the profile are removed from the + profile. The <option>-t</option> option allows this + percentage to be modified (maximum 5%).</para> + + <para><option>-t0</option> requests no trace elements to be + removed from the profile, ensuring that all the data will be + displayed.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-c</option></term> + <listitem> + <para>Generate colour output.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-y</option></term> + <listitem> + <para>Ignore marks.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-?</option></term> + <listitem> + <para>Print out usage information.</para> + </listitem> + </varlistentry> + </variablelist> + + + <sect2 id="manipulating-hp"> + <title>Manipulating the hp file</title> + +<para>(Notes kindly offered by Jan-Willhem Maessen.)</para> + +<para> +The <filename>FOO.hp</filename> file produced when you ask for the +heap profile of a program <filename>FOO</filename> is a text file with a particularly +simple structure. Here's a representative example, with much of the +actual data omitted: +<screen> +JOB "FOO -hC" +DATE "Thu Dec 26 18:17 2002" +SAMPLE_UNIT "seconds" +VALUE_UNIT "bytes" +BEGIN_SAMPLE 0.00 +END_SAMPLE 0.00 +BEGIN_SAMPLE 15.07 + ... sample data ... +END_SAMPLE 15.07 +BEGIN_SAMPLE 30.23 + ... sample data ... +END_SAMPLE 30.23 +... etc. +BEGIN_SAMPLE 11695.47 +END_SAMPLE 11695.47 +</screen> +The first four lines (<literal>JOB</literal>, <literal>DATE</literal>, <literal>SAMPLE_UNIT</literal>, <literal>VALUE_UNIT</literal>) form a +header. Each block of lines starting with <literal>BEGIN_SAMPLE</literal> and ending +with <literal>END_SAMPLE</literal> forms a single sample (you can think of this as a +vertical slice of your heap profile). The hp2ps utility should accept +any input with a properly-formatted header followed by a series of +*complete* samples. +</para> +</sect2> + + <sect2> + <title>Zooming in on regions of your profile</title> + +<para> +You can look at particular regions of your profile simply by loading a +copy of the <filename>.hp</filename> file into a text editor and deleting the unwanted +samples. The resulting <filename>.hp</filename> file can be run through <command>hp2ps</command> and viewed +or printed. +</para> +</sect2> + + <sect2> + <title>Viewing the heap profile of a running program</title> + +<para> +The <filename>.hp</filename> file is generated incrementally as your +program runs. In principle, running <command>hp2ps</command> on the incomplete file +should produce a snapshot of your program's heap usage. However, the +last sample in the file may be incomplete, causing <command>hp2ps</command> to fail. If +you are using a machine with UNIX utilities installed, it's not too +hard to work around this problem (though the resulting command line +looks rather Byzantine): +<screen> + head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \ + | hp2ps > FOO.ps +</screen> + +The command <command>fgrep -n END_SAMPLE FOO.hp</command> finds the +end of every complete sample in <filename>FOO.hp</filename>, and labels each sample with +its ending line number. We then select the line number of the last +complete sample using <command>tail</command> and <command>cut</command>. This is used as a +parameter to <command>head</command>; the result is as if we deleted the final +incomplete sample from <filename>FOO.hp</filename>. This results in a properly-formatted +.hp file which we feed directly to <command>hp2ps</command>. +</para> +</sect2> + <sect2> + <title>Viewing a heap profile in real time</title> + +<para> +The <command>gv</command> and <command>ghostview</command> programs +have a "watch file" option can be used to view an up-to-date heap +profile of your program as it runs. Simply generate an incremental +heap profile as described in the previous section. Run <command>gv</command> on your +profile: +<screen> + gv -watch -seascape FOO.ps +</screen> +If you forget the <literal>-watch</literal> flag you can still select +"Watch file" from the "State" menu. Now each time you generate a new +profile <filename>FOO.ps</filename> the view will update automatically. +</para> + +<para> +This can all be encapsulated in a little script: +<screen> + #!/bin/sh + head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \ + | hp2ps > FOO.ps + gv -watch -seascape FOO.ps & + while [ 1 ] ; do + sleep 10 # We generate a new profile every 10 seconds. + head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \ + | hp2ps > FOO.ps + done +</screen> +Occasionally <command>gv</command> will choke as it tries to read an incomplete copy of +<filename>FOO.ps</filename> (because <command>hp2ps</command> is still running as an update +occurs). A slightly more complicated script works around this +problem, by using the fact that sending a SIGHUP to gv will cause it +to re-read its input file: +<screen> + #!/bin/sh + head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \ + | hp2ps > FOO.ps + gv FOO.ps & + gvpsnum=$! + while [ 1 ] ; do + sleep 10 + head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \ + | hp2ps > FOO.ps + kill -HUP $gvpsnum + done +</screen> +</para> +</sect2> + + + </sect1> + + <sect1 id="ticky-ticky"> + <title>Using “ticky-ticky” profiling (for implementors)</title> + <indexterm><primary>ticky-ticky profiling</primary></indexterm> + + <para>(ToDo: document properly.)</para> + + <para>It is possible to compile Glasgow Haskell programs so that + they will count lots and lots of interesting things, e.g., number + of updates, number of data constructors entered, etc., etc. We + call this “ticky-ticky” + profiling,<indexterm><primary>ticky-ticky + profiling</primary></indexterm> <indexterm><primary>profiling, + ticky-ticky</primary></indexterm> because that's the sound a Sun4 + makes when it is running up all those counters + (<emphasis>slowly</emphasis>).</para> + + <para>Ticky-ticky profiling is mainly intended for implementors; + it is quite separate from the main “cost-centre” + profiling system, intended for all users everywhere.</para> + + <para>To be able to use ticky-ticky profiling, you will need to + have built appropriate libraries and things when you made the + system. See “Customising what libraries to build,” in + the installation guide.</para> + + <para>To get your compiled program to spit out the ticky-ticky + numbers, use a <option>-r</option> RTS + option<indexterm><primary>-r RTS option</primary></indexterm>. + See <xref linkend="runtime-control"/>.</para> + + <para>Compiling your program with the <option>-ticky</option> + switch yields an executable that performs these counts. Here is a + sample ticky-ticky statistics file, generated by the invocation + <command>foo +RTS -rfoo.ticky</command>.</para> + +<screen> + foo +RTS -rfoo.ticky + + +ALLOCATIONS: 3964631 (11330900 words total: 3999476 admin, 6098829 goods, 1232595 slop) + total words: 2 3 4 5 6+ + 69647 ( 1.8%) function values 50.0 50.0 0.0 0.0 0.0 +2382937 ( 60.1%) thunks 0.0 83.9 16.1 0.0 0.0 +1477218 ( 37.3%) data values 66.8 33.2 0.0 0.0 0.0 + 0 ( 0.0%) big tuples + 2 ( 0.0%) black holes 0.0 100.0 0.0 0.0 0.0 + 0 ( 0.0%) prim things + 34825 ( 0.9%) partial applications 0.0 0.0 0.0 100.0 0.0 + 2 ( 0.0%) thread state objects 0.0 0.0 0.0 0.0 100.0 + +Total storage-manager allocations: 3647137 (11882004 words) + [551104 words lost to speculative heap-checks] + +STACK USAGE: + +ENTERS: 9400092 of which 2005772 (21.3%) direct to the entry code + [the rest indirected via Node's info ptr] +1860318 ( 19.8%) thunks +3733184 ( 39.7%) data values +3149544 ( 33.5%) function values + [of which 1999880 (63.5%) bypassed arg-satisfaction chk] + 348140 ( 3.7%) partial applications + 308906 ( 3.3%) normal indirections + 0 ( 0.0%) permanent indirections + +RETURNS: 5870443 +2137257 ( 36.4%) from entering a new constructor + [the rest from entering an existing constructor] +2349219 ( 40.0%) vectored [the rest unvectored] + +RET_NEW: 2137257: 32.5% 46.2% 21.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% +RET_OLD: 3733184: 2.8% 67.9% 29.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% +RET_UNBOXED_TUP: 2: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% + +RET_VEC_RETURN : 2349219: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% + +UPDATE FRAMES: 2241725 (0 omitted from thunks) +SEQ FRAMES: 1 +CATCH FRAMES: 1 +UPDATES: 2241725 + 0 ( 0.0%) data values + 34827 ( 1.6%) partial applications + [2 in place, 34825 allocated new space] +2206898 ( 98.4%) updates to existing heap objects (46 by squeezing) +UPD_CON_IN_NEW: 0: 0 0 0 0 0 0 0 0 0 +UPD_PAP_IN_NEW: 34825: 0 0 0 34825 0 0 0 0 0 + +NEW GEN UPDATES: 2274700 ( 99.9%) + +OLD GEN UPDATES: 1852 ( 0.1%) + +Total bytes copied during GC: 190096 + +************************************************** +3647137 ALLOC_HEAP_ctr +11882004 ALLOC_HEAP_tot + 69647 ALLOC_FUN_ctr + 69647 ALLOC_FUN_adm + 69644 ALLOC_FUN_gds + 34819 ALLOC_FUN_slp + 34831 ALLOC_FUN_hst_0 + 34816 ALLOC_FUN_hst_1 + 0 ALLOC_FUN_hst_2 + 0 ALLOC_FUN_hst_3 + 0 ALLOC_FUN_hst_4 +2382937 ALLOC_UP_THK_ctr + 0 ALLOC_SE_THK_ctr + 308906 ENT_IND_ctr + 0 E!NT_PERM_IND_ctr requires +RTS -Z +[... lots more info omitted ...] + 0 GC_SEL_ABANDONED_ctr + 0 GC_SEL_MINOR_ctr + 0 GC_SEL_MAJOR_ctr + 0 GC_FAILED_PROMOTION_ctr + 47524 GC_WORDS_COPIED_ctr +</screen> + + <para>The formatting of the information above the row of asterisks + is subject to change, but hopefully provides a useful + human-readable summary. Below the asterisks <emphasis>all + counters</emphasis> maintained by the ticky-ticky system are + dumped, in a format intended to be machine-readable: zero or more + spaces, an integer, a space, the counter name, and a newline.</para> + + <para>In fact, not <emphasis>all</emphasis> counters are + necessarily dumped; compile- or run-time flags can render certain + counters invalid. In this case, either the counter will simply + not appear, or it will appear with a modified counter name, + possibly along with an explanation for the omission (notice + <literal>ENT_PERM_IND_ctr</literal> appears + with an inserted <literal>!</literal> above). Software analysing + this output should always check that it has the counters it + expects. Also, beware: some of the counters can have + <emphasis>large</emphasis> values!</para> + + </sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/runtime_control.xml b/docs/users_guide/runtime_control.xml new file mode 100644 index 0000000000..daed07cee3 --- /dev/null +++ b/docs/users_guide/runtime_control.xml @@ -0,0 +1,622 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<sect1 id="runtime-control"> + <title>Running a compiled program</title> + + <indexterm><primary>runtime control of Haskell programs</primary></indexterm> + <indexterm><primary>running, compiled program</primary></indexterm> + <indexterm><primary>RTS options</primary></indexterm> + + <para>To make an executable program, the GHC system compiles your + code and then links it with a non-trivial runtime system (RTS), + which handles storage management, profiling, etc.</para> + + <para>You have some control over the behaviour of the RTS, by giving + special command-line arguments to your program.</para> + + <para>When your Haskell program starts up, its RTS extracts + command-line arguments bracketed between + <option>+RTS</option><indexterm><primary><option>+RTS</option></primary></indexterm> + and + <option>-RTS</option><indexterm><primary><option>-RTS</option></primary></indexterm> + as its own. For example:</para> + +<screen> +% ./a.out -f +RTS -p -S -RTS -h foo bar +</screen> + + <para>The RTS will snaffle <option>-p</option> <option>-S</option> + for itself, and the remaining arguments <literal>-f -h foo bar</literal> + will be handed to your program if/when it calls + <function>System.getArgs</function>.</para> + + <para>No <option>-RTS</option> option is required if the + runtime-system options extend to the end of the command line, as in + this example:</para> + +<screen> +% hls -ltr /usr/etc +RTS -A5m +</screen> + + <para>If you absolutely positively want all the rest of the options + in a command line to go to the program (and not the RTS), use a + <option>––RTS</option><indexterm><primary><option>--RTS</option></primary></indexterm>.</para> + + <para>As always, for RTS options that take + <replaceable>size</replaceable>s: If the last character of + <replaceable>size</replaceable> is a K or k, multiply by 1000; if an + M or m, by 1,000,000; if a G or G, by 1,000,000,000. (And any + wraparound in the counters is <emphasis>your</emphasis> + fault!)</para> + + <para>Giving a <literal>+RTS -f</literal> + <indexterm><primary><option>-f</option></primary><secondary>RTS option</secondary></indexterm> option + will print out the RTS options actually available in your program + (which vary, depending on how you compiled).</para> + + <para>NOTE: since GHC is itself compiled by GHC, you can change RTS + options in the compiler using the normal + <literal>+RTS ... -RTS</literal> + combination. eg. to increase the maximum heap + size for a compilation to 128M, you would add + <literal>+RTS -M128m -RTS</literal> + to the command line.</para> + + <sect2 id="rts-optinos-environment"> + <title>Setting global RTS options</title> + + <indexterm><primary>RTS options</primary><secondary>from the environment</secondary></indexterm> + <indexterm><primary>environment variable</primary><secondary>for + setting RTS options</secondary></indexterm> + + <para>RTS options are also taken from the environment variable + <envar>GHCRTS</envar><indexterm><primary><envar>GHCRTS</envar></primary> + </indexterm>. For example, to set the maximum heap size + to 128M for all GHC-compiled programs (using an + <literal>sh</literal>-like shell):</para> + +<screen> + GHCRTS='-M128m' + export GHCRTS +</screen> + + <para>RTS options taken from the <envar>GHCRTS</envar> environment + variable can be overridden by options given on the command + line.</para> + + </sect2> + + <sect2 id="rts-options-gc"> + <title>RTS options to control the garbage collector</title> + + <indexterm><primary>garbage collector</primary><secondary>options</secondary></indexterm> + <indexterm><primary>RTS options</primary><secondary>garbage collection</secondary></indexterm> + + <para>There are several options to give you precise control over + garbage collection. Hopefully, you won't need any of these in + normal operation, but there are several things that can be tweaked + for maximum performance.</para> + + <variablelist> + + <varlistentry> + <term> + <option>-A</option><replaceable>size</replaceable> + <indexterm><primary><option>-A</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>allocation area, size</primary></indexterm> + </term> + <listitem> + <para>[Default: 256k] Set the allocation area size + used by the garbage collector. The allocation area + (actually generation 0 step 0) is fixed and is never resized + (unless you use <option>-H</option>, below).</para> + + <para>Increasing the allocation area size may or may not + give better performance (a bigger allocation area means + worse cache behaviour but fewer garbage collections and less + promotion).</para> + + <para>With only 1 generation (<option>-G1</option>) the + <option>-A</option> option specifies the minimum allocation + area, since the actual size of the allocation area will be + resized according to the amount of data in the heap (see + <option>-F</option>, below).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-c</option> + <indexterm><primary><option>-c</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>garbage collection</primary><secondary>compacting</secondary></indexterm> + <indexterm><primary>compacting garbage collection</primary></indexterm> + </term> + <listitem> + <para>Use a compacting algorithm for collecting the oldest + generation. By default, the oldest generation is collected + using a copying algorithm; this option causes it to be + compacted in-place instead. The compaction algorithm is + slower than the copying algorithm, but the savings in memory + use can be considerable.</para> + + <para>For a given heap size (using the <option>-H</option> + option), compaction can in fact reduce the GC cost by + allowing fewer GCs to be performed. This is more likely + when the ratio of live data to heap size is high, say + >30%.</para> + + <para>NOTE: compaction doesn't currently work when a single + generation is requested using the <option>-G1</option> + option.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-c</option><replaceable>n</replaceable></term> + + <listitem> + <para>[Default: 30] Automatically enable + compacting collection when the live data exceeds + <replaceable>n</replaceable>% of the maximum heap size + (see the <option>-M</option> option). Note that the maximum + heap size is unlimited by default, so this option has no + effect unless the maximum heap size is set with + <option>-M</option><replaceable>size</replaceable>. </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-F</option><replaceable>factor</replaceable> + <indexterm><primary><option>-F</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>heap size, factor</primary></indexterm> + </term> + <listitem> + + <para>[Default: 2] This option controls the amount + of memory reserved for the older generations (and in the + case of a two space collector the size of the allocation + area) as a factor of the amount of live data. For example, + if there was 2M of live data in the oldest generation when + we last collected it, then by default we'll wait until it + grows to 4M before collecting it again.</para> + + <para>The default seems to work well here. If you have + plenty of memory, it is usually better to use + <option>-H</option><replaceable>size</replaceable> than to + increase + <option>-F</option><replaceable>factor</replaceable>.</para> + + <para>The <option>-F</option> setting will be automatically + reduced by the garbage collector when the maximum heap size + (the <option>-M</option><replaceable>size</replaceable> + setting) is approaching.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-G</option><replaceable>generations</replaceable> + <indexterm><primary><option>-G</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>generations, number of</primary></indexterm> + </term> + <listitem> + <para>[Default: 2] Set the number of generations + used by the garbage collector. The default of 2 seems to be + good, but the garbage collector can support any number of + generations. Anything larger than about 4 is probably not a + good idea unless your program runs for a + <emphasis>long</emphasis> time, because the oldest + generation will hardly ever get collected.</para> + + <para>Specifying 1 generation with <option>+RTS -G1</option> + gives you a simple 2-space collector, as you would expect. + In a 2-space collector, the <option>-A</option> option (see + above) specifies the <emphasis>minimum</emphasis> allocation + area size, since the allocation area will grow with the + amount of live data in the heap. In a multi-generational + collector the allocation area is a fixed size (unless you + use the <option>-H</option> option, see below).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-H</option><replaceable>size</replaceable> + <indexterm><primary><option>-H</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>heap size, suggested</primary></indexterm> + </term> + <listitem> + <para>[Default: 0] This option provides a + “suggested heap size” for the garbage collector. The + garbage collector will use about this much memory until the + program residency grows and the heap size needs to be + expanded to retain reasonable performance.</para> + + <para>By default, the heap will start small, and grow and + shrink as necessary. This can be bad for performance, so if + you have plenty of memory it's worthwhile supplying a big + <option>-H</option><replaceable>size</replaceable>. For + improving GC performance, using + <option>-H</option><replaceable>size</replaceable> is + usually a better bet than + <option>-A</option><replaceable>size</replaceable>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-I</option><replaceable>seconds</replaceable> + <indexterm><primary><option>-H</option></primary> + <secondary>RTS option</secondary> + </indexterm> + <indexterm><primary>idle GC</primary> + </indexterm> + </term> + <listitem> + <para>(default: 0.3) In the threaded and SMP versions of the RTS (see + <option>-threaded</option>, <xref linkend="options-linker" />), a + major GC is automatically performed if the runtime has been idle + (no Haskell computation has been running) for a period of time. + The amount of idle time which must pass before a GC is performed is + set by the <option>-I</option><replaceable>seconds</replaceable> + option. Specifying <option>-I0</option> disables the idle GC.</para> + + <para>For an interactive application, it is probably a good idea to + use the idle GC, because this will allow finalizers to run and + deadlocked threads to be detected in the idle time when no Haskell + computation is happening. Also, it will mean that a GC is less + likely to happen when the application is busy, and so + responsiveness may be improved. However, if the amount of live data in + the heap is particularly large, then the idle GC can cause a + significant delay, and too small an interval could adversely affect + interactive responsiveness.</para> + + <para>This is an experimental feature, please let us know if it + causes problems and/or could benefit from further tuning.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-k</option><replaceable>size</replaceable> + <indexterm><primary><option>-k</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>stack, minimum size</primary></indexterm> + </term> + <listitem> + <para>[Default: 1k] Set the initial stack size for + new threads. Thread stacks (including the main thread's + stack) live on the heap, and grow as required. The default + value is good for concurrent applications with lots of small + threads; if your program doesn't fit this model then + increasing this option may help performance.</para> + + <para>The main thread is normally started with a slightly + larger heap to cut down on unnecessary stack growth while + the program is starting up.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-K</option><replaceable>size</replaceable> + <indexterm><primary><option>-K</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>stack, maximum size</primary></indexterm> + </term> + <listitem> + <para>[Default: 8M] Set the maximum stack size for + an individual thread to <replaceable>size</replaceable> + bytes. This option is there purely to stop the program + eating up all the available memory in the machine if it gets + into an infinite loop.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-m</option><replaceable>n</replaceable> + <indexterm><primary><option>-m</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>heap, minimum free</primary></indexterm> + </term> + <listitem> + <para>Minimum % <replaceable>n</replaceable> of heap + which must be available for allocation. The default is + 3%.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-M</option><replaceable>size</replaceable> + <indexterm><primary><option>-M</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>heap size, maximum</primary></indexterm> + </term> + <listitem> + <para>[Default: unlimited] Set the maximum heap size to + <replaceable>size</replaceable> bytes. The heap normally + grows and shrinks according to the memory requirements of + the program. The only reason for having this option is to + stop the heap growing without bound and filling up all the + available swap space, which at the least will result in the + program being summarily killed by the operating + system.</para> + + <para>The maximum heap size also affects other garbage + collection parameters: when the amount of live data in the + heap exceeds a certain fraction of the maximum heap size, + compacting collection will be automatically enabled for the + oldest generation, and the <option>-F</option> parameter + will be reduced in order to avoid exceeding the maximum heap + size.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-s</option><replaceable>file</replaceable> + <indexterm><primary><option>-s</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <term> + <option>-S</option><replaceable>file</replaceable> + <indexterm><primary><option>-S</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Write modest (<option>-s</option>) or verbose + (<option>-S</option>) garbage-collector statistics into file + <replaceable>file</replaceable>. The default + <replaceable>file</replaceable> is + <filename><replaceable>program</replaceable>.stat</filename>. The + <replaceable>file</replaceable> <constant>stderr</constant> + is treated specially, with the output really being sent to + <constant>stderr</constant>.</para> + + <para>This option is useful for watching how the storage + manager adjusts the heap size based on the current amount of + live data.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-t<replaceable>file</replaceable></option> + <indexterm><primary><option>-t</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Write a one-line GC stats summary after running the + program. This output is in the same format as that produced + by the <option>-Rghc-timing</option> option.</para> + + <para>As with <option>-s</option>, the default + <replaceable>file</replaceable> is + <filename><replaceable>program</replaceable>.stat</filename>. The + <replaceable>file</replaceable> <constant>stderr</constant> + is treated specially, with the output really being sent to + <constant>stderr</constant>.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2> + <title>RTS options for profiling and Concurrent/Parallel Haskell</title> + + <para>The RTS options related to profiling are described in <xref + linkend="rts-options-heap-prof"/>; and those for concurrent/parallel + stuff, in <xref linkend="parallel-rts-opts"/>.</para> + </sect2> + + <sect2 id="rts-options-debugging"> + <title>RTS options for hackers, debuggers, and over-interested + souls</title> + + <indexterm><primary>RTS options, hacking/debugging</primary></indexterm> + + <para>These RTS options might be used (a) to avoid a GHC bug, + (b) to see “what's really happening”, or + (c) because you feel like it. Not recommended for everyday + use!</para> + + <variablelist> + + <varlistentry> + <term> + <option>-B</option> + <indexterm><primary><option>-B</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Sound the bell at the start of each (major) garbage + collection.</para> + + <para>Oddly enough, people really do use this option! Our + pal in Durham (England), Paul Callaghan, writes: “Some + people here use it for a variety of + purposes—honestly!—e.g., confirmation that the + code/machine is doing something, infinite loop detection, + gauging cost of recently added code. Certain people can even + tell what stage [the program] is in by the beep + pattern. But the major use is for annoying others in the + same office…”</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-D</option><replaceable>num</replaceable> + <indexterm><primary>-D</primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>An RTS debugging flag; varying quantities of output + depending on which bits are set in + <replaceable>num</replaceable>. Only works if the RTS was + compiled with the <option>DEBUG</option> option.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-r</option><replaceable>file</replaceable> + <indexterm><primary><option>-r</option></primary><secondary>RTS option</secondary></indexterm> + <indexterm><primary>ticky ticky profiling</primary></indexterm> + <indexterm><primary>profiling</primary><secondary>ticky ticky</secondary></indexterm> + </term> + <listitem> + <para>Produce “ticky-ticky” statistics at the + end of the program run. The <replaceable>file</replaceable> + business works just like on the <option>-S</option> RTS + option (above).</para> + + <para>“Ticky-ticky” statistics are counts of + various program actions (updates, enters, etc.) The program + must have been compiled using + <option>-ticky</option><indexterm><primary><option>-ticky</option></primary></indexterm> + (a.k.a. “ticky-ticky profiling”), and, for it to + be really useful, linked with suitable system libraries. + Not a trivial undertaking: consult the installation guide on + how to set things up for easy “ticky-ticky” + profiling. For more information, see <xref + linkend="ticky-ticky"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-xc</option> + <indexterm><primary><option>-xc</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>(Only available when the program is compiled for + profiling.) When an exception is raised in the program, + this option causes the current cost-centre-stack to be + dumped to <literal>stderr</literal>.</para> + + <para>This can be particularly useful for debugging: if your + program is complaining about a <literal>head []</literal> + error and you haven't got a clue which bit of code is + causing it, compiling with <literal>-prof + -auto-all</literal> and running with <literal>+RTS -xc + -RTS</literal> will tell you exactly the call stack at the + point the error was raised.</para> + + <para>The output contains one line for each exception raised + in the program (the program might raise and catch several + exceptions during its execution), where each line is of the + form:</para> + +<screen> +< cc<subscript>1</subscript>, ..., cc<subscript>n</subscript> > +</screen> + <para>each <literal>cc</literal><subscript>i</subscript> is + a cost centre in the program (see <xref + linkend="cost-centres"/>), and the sequence represents the + “call stack” at the point the exception was + raised. The leftmost item is the innermost function in the + call stack, and the rightmost item is the outermost + function.</para> + + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-Z</option> + <indexterm><primary><option>-Z</option></primary><secondary>RTS option</secondary></indexterm> + </term> + <listitem> + <para>Turn <emphasis>off</emphasis> “update-frame + squeezing” at garbage-collection time. (There's no + particularly good reason to turn it off, except to ensure + the accuracy of certain data collected regarding thunk entry + counts.)</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2 id="rts-hooks"> + <title>“Hooks” to change RTS behaviour</title> + + <indexterm><primary>hooks</primary><secondary>RTS</secondary></indexterm> + <indexterm><primary>RTS hooks</primary></indexterm> + <indexterm><primary>RTS behaviour, changing</primary></indexterm> + + <para>GHC lets you exercise rudimentary control over the RTS + settings for any given program, by compiling in a + “hook” that is called by the run-time system. The RTS + contains stub definitions for all these hooks, but by writing your + own version and linking it on the GHC command line, you can + override the defaults.</para> + + <para>Owing to the vagaries of DLL linking, these hooks don't work + under Windows when the program is built dynamically.</para> + + <para>The hook <literal>ghc_rts_opts</literal><indexterm><primary><literal>ghc_rts_opts</literal></primary> + </indexterm>lets you set RTS + options permanently for a given program. A common use for this is + to give your program a default heap and/or stack size that is + greater than the default. For example, to set <literal>-H128m + -K1m</literal>, place the following definition in a C source + file:</para> + +<programlisting> +char *ghc_rts_opts = "-H128m -K1m"; +</programlisting> + + <para>Compile the C file, and include the object file on the + command line when you link your Haskell program.</para> + + <para>These flags are interpreted first, before any RTS flags from + the <literal>GHCRTS</literal> environment variable and any flags + on the command line.</para> + + <para>You can also change the messages printed when the runtime + system “blows up,” e.g., on stack overflow. The hooks + for these are as follows:</para> + + <variablelist> + + <varlistentry> + <term> + <function>void OutOfHeapHook (unsigned long, unsigned long)</function> + <indexterm><primary><function>OutOfHeapHook</function></primary></indexterm> + </term> + <listitem> + <para>The heap-overflow message.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <function>void StackOverflowHook (long int)</function> + <indexterm><primary><function>StackOverflowHook</function></primary></indexterm> + </term> + <listitem> + <para>The stack-overflow message.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <function>void MallocFailHook (long int)</function> + <indexterm><primary><function>MallocFailHook</function></primary></indexterm> + </term> + <listitem> + <para>The message printed if <function>malloc</function> + fails.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>For examples of the use of these hooks, see GHC's own + versions in the file + <filename>ghc/compiler/parser/hschooks.c</filename> in a GHC + source tree.</para> + </sect2> +</sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/separate_compilation.xml b/docs/users_guide/separate_compilation.xml new file mode 100644 index 0000000000..c33ff2175b --- /dev/null +++ b/docs/users_guide/separate_compilation.xml @@ -0,0 +1,1213 @@ +<?xml version="1.0" encoding="iso-8859-1"?> + <sect1 id="separate-compilation"> + <title>Filenames and separate compilation</title> + + <indexterm><primary>separate compilation</primary></indexterm> + <indexterm><primary>recompilation checker</primary></indexterm> + <indexterm><primary>make and recompilation</primary></indexterm> + + <para>This section describes what files GHC expects to find, what + files it creates, where these files are stored, and what options + affect this behaviour.</para> + + <para>Note that this section is written with + <firstterm>hierarchical modules</firstterm> in mind (see <xref + linkend="hierarchical-modules"/>); hierarchical modules are an + extension to Haskell 98 which extends the lexical syntax of + module names to include a dot ‘.’. Non-hierarchical + modules are thus a special case in which none of the module names + contain dots.</para> + + <para>Pathname conventions vary from system to system. In + particular, the directory separator is + ‘<literal>/</literal>’ on Unix systems and + ‘<literal>\</literal>’ on Windows systems. In the + sections that follow, we shall consistently use + ‘<literal>/</literal>’ as the directory separator; + substitute this for the appropriate character for your + system.</para> + + <sect2 id="source-files"> + <title>Haskell source files</title> + + <indexterm><primary>filenames</primary></indexterm> + + <para>Each Haskell source module should be placed in a file on + its own.</para> + + <para>Usually, the file should be named after the module name, + replacing dots in the module name by directory separators. For + example, on a Unix system, the module <literal>A.B.C</literal> + should be placed in the file <literal>A/B/C.hs</literal>, + relative to some base directory. If the module is not going to + be imported by another module (<literal>Main</literal>, for + example), then you are free to use any filename for it.</para> + + <indexterm><primary>unicode</primary></indexterm> + + <para> GHC assumes that source files are + ASCII<indexterm><primary>ASCII</primary></indexterm> or + UTF-8<indexterm><primary>UTF-8</primary></indexterm> only, other + encodings<indexterm><primary>encoding</primary></indexterm> are + not recognised. However, invalid UTF-8 sequences will be + ignored in comments, so it is possible to use other encodings + such as + Latin-1<indexterm><primary>Latin-1</primary></indexterm>, as + long as the non-comment source code is ASCII only.</para> + </sect2> + + <sect2 id="output-files"> + <title>Output files</title> + + <indexterm><primary>interface files</primary></indexterm> + <indexterm><primary><literal>.hi</literal> files</primary></indexterm> + <indexterm><primary>object files</primary></indexterm> + <indexterm><primary><literal>.o</literal> files</primary></indexterm> + + <para>When asked to compile a source file, GHC normally + generates two files: an <firstterm>object file</firstterm>, and + an <firstterm>interface file</firstterm>. </para> + + <para>The object file, which normally ends in a + <literal>.o</literal> suffix, contains the compiled code for the + module.</para> + + <para>The interface file, + which normally ends in a <literal>.hi</literal> suffix, contains + the information that GHC needs in order to compile further + modules that depend on this module. It contains things like the + types of exported functions, definitions of data types, and so + on. It is stored in a binary format, so don't try to read one; + use the <option>--show-iface</option> option instead (see <xref + linkend="hi-options"/>).</para> + + <para>You should think of the object file and the interface file as a + pair, since the interface file is in a sense a compiler-readable + description of the contents of the object file. If the + interface file and object file get out of sync for any reason, + then the compiler may end up making assumptions about the object + file that aren't true; trouble will almost certainly follow. + For this reason, we recommend keeping object files and interface + files in the same place (GHC does this by default, but it is + possible to override the defaults as we'll explain + shortly).</para> + + <para>Every module has a <emphasis>module name</emphasis> + defined in its source code (<literal>module A.B.C where + ...</literal>).</para> + + <para>The name of the object file generated by GHC is derived + according to the following rules, where + <replaceable>osuf</replaceable> is the object-file suffix (this + can be changed with the <option>-osuf</option> option).</para> + + <itemizedlist> + <listitem> + <para>If there is no <option>-odir</option> option (the + default), then the object filename is derived from the + source filename (ignoring the module name) by replacing the + suffix with <replaceable>osuf</replaceable>.</para> + </listitem> + <listitem> + <para>If + <option>-odir</option> <replaceable>dir</replaceable> + has been specified, then the object filename is + <replaceable>dir</replaceable>/<replaceable>mod</replaceable>.<replaceable>osuf</replaceable>, + where <replaceable>mod</replaceable> is the module name with + dots replaced by slashes.</para> + </listitem> + </itemizedlist> + + <para>The name of the interface file is derived using the same + rules, except that the suffix is + <replaceable>hisuf</replaceable> (<literal>.hi</literal> by + default) instead of <replaceable>osuf</replaceable>, and the + relevant options are <option>-hidir</option> and + <option>-hisuf</option> instead of <option>-odir</option> and + <option>-osuf</option> respectively.</para> + + <para>For example, if GHC compiles the module + <literal>A.B.C</literal> in the file + <filename>src/A/B/C.hs</filename>, with no + <literal>-odir</literal> or <literal>-hidir</literal> flags, the + interface file will be put in <literal>src/A/B/C.hi</literal> + and the object file in <literal>src/A/B/C.o</literal>.</para> + + <para>For any module that is imported, GHC requires that the + name of the module in the import statement exactly matches the + name of the module in the interface file (or source file) found + using the strategy specified in <xref linkend="search-path"/>. + This means that for most modules, the source file name should + match the module name.</para> + + <para>However, note that it is reasonable to have a module + <literal>Main</literal> in a file named + <filename>foo.hs</filename>, but this only works because GHC + never needs to search for the interface for module + <literal>Main</literal> (because it is never imported). It is + therefore possible to have several <literal>Main</literal> + modules in separate source files in the same directory, and GHC + will not get confused.</para> + + <para>In batch compilation mode, the name of the object file can + also be overridden using the <option>-o</option> option, and the + name of the interface file can be specified directly using the + <option>-ohi</option> option.</para> + </sect2> + + <sect2 id="search-path"> + <title>The search path</title> + + <indexterm><primary>search path</primary> + </indexterm> + <indexterm><primary>interface files, finding them</primary></indexterm> + <indexterm><primary>finding interface files</primary></indexterm> + + <para>In your program, you import a module + <literal>Foo</literal> by saying <literal>import Foo</literal>. + In <option>--make</option> mode or GHCi, GHC will look for a + source file for <literal>Foo</literal> and arrange to compile it + first. Without <option>--make</option>, GHC will look for the + interface file for <literal>Foo</literal>, which should have + been created by an earlier compilation of + <literal>Foo</literal>. GHC uses the same strategy in each of + these cases for finding the appropriate file.</para> + + <para>This strategy is as follows: GHC keeps a list of + directories called the <firstterm>search path</firstterm>. For + each of these directories, it tries appending + <replaceable>basename</replaceable><literal>.</literal><replaceable>extension</replaceable> + to the directory, and checks whether the file exists. The value + of <replaceable>basename</replaceable> is the module name with + dots replaced by the directory separator ('/' or '\', depending + on the system), and <replaceable>extension</replaceable> is a + source extension (<literal>hs</literal>, <literal>lhs</literal>) + if we are in <option>--make</option> mode and GHCi, or + <replaceable>hisuf</replaceable> otherwise.</para> + + <para>For example, suppose the search path contains directories + <literal>d1</literal>, <literal>d2</literal>, and + <literal>d3</literal>, and we are in <literal>--make</literal> + mode looking for the source file for a module + <literal>A.B.C</literal>. GHC will look in + <literal>d1/A/B/C.hs</literal>, <literal>d1/A/B/C.lhs</literal>, + <literal>d2/A/B/C.hs</literal>, and so on.</para> + + <para>The search path by default contains a single directory: + <quote>.</quote> (i.e. the current directory). The following + options can be used to add to or change the contents of the + search path:</para> + + <variablelist> + <varlistentry> + <term><option>-i<replaceable>dirs</replaceable></option></term> + <listitem> + <para><indexterm><primary><option>-i<replaceable>dirs</replaceable></option> + </primary></indexterm>This flag appends a colon-separated + list of <filename>dirs</filename> to the search path.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-i</option></term> + <listitem> + <para>resets the search path back to nothing.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>This isn't the whole story: GHC also looks for modules in + pre-compiled libraries, known as packages. See the section on + packages (<xref linkend="packages"/>), for details.</para> + </sect2> + + <sect2 id="options-output"> + <title>Redirecting the compilation output(s)</title> + + <indexterm><primary>output-directing options</primary></indexterm> + <indexterm><primary>redirecting compilation output</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-o</option> <replaceable>file</replaceable> + <indexterm><primary><option>-o</option></primary></indexterm> + </term> + <listitem> + <para>GHC's compiled output normally goes into a + <filename>.hc</filename>, <filename>.o</filename>, etc., + file, depending on the last-run compilation phase. The + option <option>-o <replaceable>file</replaceable></option> + re-directs the output of that last-run phase to + <replaceable>file</replaceable>.</para> + + <para>Note: this “feature” can be + counterintuitive: <command>ghc -C -o foo.o + foo.hs</command> will put the intermediate C code in the + file <filename>foo.o</filename>, name + notwithstanding!</para> + + <para>This option is most often used when creating an + executable file, to set the filename of the executable. + For example: +<screen> ghc -o prog --make Main</screen> + + will compile the program starting with module + <literal>Main</literal> and put the executable in the + file <literal>prog</literal>.</para> + + <para>Note: on Windows, if the result is an executable + file, the extension "<filename>.exe</filename>" is added + if the specified filename does not already have an + extension. Thus +<programlisting> + ghc -o foo Main.hs +</programlisting> + will compile and link the module + <filename>Main.hs</filename>, and put the resulting + executable in <filename>foo.exe</filename> (not + <filename>foo</filename>).</para> + + <para>If you use <command>ghc --make</command> and you don't + use the <option>-o</option>, the name GHC will choose + for the executable will be based on the name of the file + containing the module <literal>Main</literal>. + Note that with GHC the <literal>Main</literal> module doesn't + have to be put in file <filename>Main.hs</filename>. + Thus both +<programlisting> + ghc --make Prog +</programlisting> + and +<programlisting> + ghc --make Prog.hs +</programlisting> + will produce <filename>Prog</filename> (or + <filename>Prog.exe</filename> if you are on Windows).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-odir</option> <replaceable>dir</replaceable> + <indexterm><primary><option>-odir</option></primary></indexterm> + </term> + <listitem> + <para>Redirects object files to directory + <replaceable>dir</replaceable>. For example:</para> + +<screen> +$ ghc -c parse/Foo.hs parse/Bar.hs gurgle/Bumble.hs -odir `arch` +</screen> + + <para>The object files, <filename>Foo.o</filename>, + <filename>Bar.o</filename>, and + <filename>Bumble.o</filename> would be put into a + subdirectory named after the architecture of the executing + machine (<filename>x86</filename>, + <filename>mips</filename>, etc).</para> + + <para>Note that the <option>-odir</option> option does + <emphasis>not</emphasis> affect where the interface files + are put; use the <option>-hidir</option> option for that. + In the above example, they would still be put in + <filename>parse/Foo.hi</filename>, + <filename>parse/Bar.hi</filename>, and + <filename>gurgle/Bumble.hi</filename>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ohi</option> <replaceable>file</replaceable> + <indexterm><primary><option>-ohi</option></primary></indexterm> + </term> + <listitem> + <para>The interface output may be directed to another file + <filename>bar2/Wurble.iface</filename> with the option + <option>-ohi bar2/Wurble.iface</option> (not + recommended).</para> + + <para>WARNING: if you redirect the interface file + somewhere that GHC can't find it, then the recompilation + checker may get confused (at the least, you won't get any + recompilation avoidance). We recommend using a + combination of <option>-hidir</option> and + <option>-hisuf</option> options instead, if + possible.</para> + + <para>To avoid generating an interface at all, you could + use this option to redirect the interface into the bit + bucket: <literal>-ohi /dev/null</literal>, for + example.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-hidir</option> <replaceable>dir</replaceable> + <indexterm><primary><option>-hidir</option></primary></indexterm> + </term> + <listitem> + <para>Redirects all generated interface files into + <replaceable>dir</replaceable>, instead of the + default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-stubdir</option> <replaceable>dir</replaceable> + <indexterm><primary><option>-stubdir</option></primary></indexterm> + </term> + <listitem> + <para>Redirects all generated FFI stub files into + <replaceable>dir</replaceable>. Stub files are generated when the + Haskell source contains a <literal>foreign export</literal> or + <literal>foreign import "&wrapper"</literal> declaration (see <xref + linkend="foreign-export-ghc" />). The <option>-stubdir</option> + option behaves in exactly the same way as <option>-odir</option> + and <option>-hidir</option> with respect to hierarchical + modules.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-osuf</option> <replaceable>suffix</replaceable> + <indexterm><primary><option>-osuf</option></primary></indexterm> + </term> + <term> + <option>-hisuf</option> <replaceable>suffix</replaceable> + <indexterm><primary><option>-hisuf</option></primary></indexterm> + </term> + <term> + <option>-hcsuf</option> <replaceable>suffix</replaceable> + <indexterm><primary><option>-hcsuf</option></primary></indexterm> + </term> + <listitem> + <para>The <option>-osuf</option> + <replaceable>suffix</replaceable> will change the + <literal>.o</literal> file suffix for object files to + whatever you specify. We use this when compiling + libraries, so that objects for the profiling versions of + the libraries don't clobber the normal ones.</para> + + <para>Similarly, the <option>-hisuf</option> + <replaceable>suffix</replaceable> will change the + <literal>.hi</literal> file suffix for non-system + interface files (see <xref linkend="hi-options"/>).</para> + + <para>Finally, the option <option>-hcsuf</option> + <replaceable>suffix</replaceable> will change the + <literal>.hc</literal> file suffix for compiler-generated + intermediate C files.</para> + + <para>The <option>-hisuf</option>/<option>-osuf</option> + game is particularly useful if you want to compile a + program both with and without profiling, in the same + directory. You can say: + <screen> + ghc ...</screen> + to get the ordinary version, and + <screen> + ghc ... -osuf prof.o -hisuf prof.hi -prof -auto-all</screen> + to get the profiled version.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="keeping-intermediates"> + <title>Keeping Intermediate Files</title> + <indexterm><primary>intermediate files, saving</primary> + </indexterm> + <indexterm><primary><literal>.hc</literal> files, saving</primary> + </indexterm> + <indexterm><primary><literal>.s</literal> files, saving</primary> + </indexterm> + + <para>The following options are useful for keeping certain + intermediate files around, when normally GHC would throw these + away after compilation:</para> + + <variablelist> + <varlistentry> + <term> + <option>-keep-hc-files</option> + <indexterm><primary><option>-keep-hc-files</option></primary></indexterm> + </term> + <listitem> + <para>Keep intermediate <literal>.hc</literal> files when + doing <literal>.hs</literal>-to-<literal>.o</literal> + compilations via C (NOTE: <literal>.hc</literal> files + aren't generated when using the native code generator, you + may need to use <option>-fvia-C</option> to force them + to be produced).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-keep-s-files</option> + <indexterm><primary><option>-keep-s-files</option></primary></indexterm> + </term> + <listitem> + <para>Keep intermediate <literal>.s</literal> files.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-keep-raw-s-files</option> + <indexterm><primary><option>-keep-raw-s-files</option></primary></indexterm> + </term> + <listitem> + <para>Keep intermediate <literal>.raw-s</literal> files. + These are the direct output from the C compiler, before + GHC does “assembly mangling” to produce the + <literal>.s</literal> file. Again, these are not produced + when using the native code generator.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-keep-tmp-files</option> + <indexterm><primary><option>-keep-tmp-files</option></primary></indexterm> + <indexterm><primary>temporary files</primary><secondary>keeping</secondary></indexterm> + </term> + <listitem> + <para>Instructs the GHC driver not to delete any of its + temporary files, which it normally keeps in + <literal>/tmp</literal> (or possibly elsewhere; see <xref + linkend="temp-files"/>). Running GHC with + <option>-v</option> will show you what temporary files + were generated along the way.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="temp-files"> + <title>Redirecting temporary files</title> + + <indexterm> + <primary>temporary files</primary> + <secondary>redirecting</secondary> + </indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-tmpdir</option> + <indexterm><primary><option>-tmpdir</option></primary></indexterm> + </term> + <listitem> + <para>If you have trouble because of running out of space + in <filename>/tmp</filename> (or wherever your + installation thinks temporary files should go), you may + use the <option>-tmpdir + <dir></option><indexterm><primary>-tmpdir + <dir> option</primary></indexterm> option to specify + an alternate directory. For example, <option>-tmpdir + .</option> says to put temporary files in the current + working directory.</para> + + <para>Alternatively, use your <constant>TMPDIR</constant> + environment variable.<indexterm><primary>TMPDIR + environment variable</primary></indexterm> Set it to the + name of the directory where temporary files should be put. + GCC and other programs will honour the + <constant>TMPDIR</constant> variable as well.</para> + + <para>Even better idea: Set the + <constant>DEFAULT_TMPDIR</constant> make variable when + building GHC, and never worry about + <constant>TMPDIR</constant> again. (see the build + documentation).</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="hi-options"> + <title>Other options related to interface files</title> + <indexterm><primary>interface files, options</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-ddump-hi</option> + <indexterm><primary><option>-ddump-hi</option></primary></indexterm> + </term> + <listitem> + <para>Dumps the new interface to standard output.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-hi-diffs</option> + <indexterm><primary><option>-ddump-hi-diffs</option></primary></indexterm> + </term> + <listitem> + <para>The compiler does not overwrite an existing + <filename>.hi</filename> interface file if the new one is + the same as the old one; this is friendly to + <command>make</command>. When an interface does change, + it is often enlightening to be informed. The + <option>-ddump-hi-diffs</option> option will make GHC run + <command>diff</command> on the old and new + <filename>.hi</filename> files.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-ddump-minimal-imports</option> + <indexterm><primary><option>-ddump-minimal-imports</option></primary></indexterm> + </term> + <listitem> + <para>Dump to the file "M.imports" (where M is the module + being compiled) a "minimal" set of import declarations. + You can safely replace all the import declarations in + "M.hs" with those found in "M.imports". Why would you + want to do that? Because the "minimal" imports (a) import + everything explicitly, by name, and (b) import nothing + that is not required. It can be quite painful to maintain + this property by hand, so this flag is intended to reduce + the labour.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>--show-iface</option> <replaceable>file</replaceable> + <indexterm><primary><option>--show-iface</option></primary></indexterm> + </term> + <listitem> + <para>Where <replaceable>file</replaceable> is the name of + an interface file, dumps the contents of that interface in + a human-readable (ish) format.</para> + </listitem> + </varlistentry> + </variablelist> + </sect2> + + <sect2 id="recomp"> + <title>The recompilation checker</title> + + <indexterm><primary>recompilation checker</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>-no-recomp</option> + <indexterm><primary><option>-recomp</option></primary></indexterm> + <indexterm><primary><option>-no-recomp</option></primary></indexterm> + </term> + <listitem> + <para>Turn off recompilation checking (which is on by + default). Recompilation checking normally stops + compilation early, leaving an existing + <filename>.o</filename> file in place, if it can be + determined that the module does not need to be + recompiled.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>In the olden days, GHC compared the newly-generated + <filename>.hi</filename> file with the previous version; if they + were identical, it left the old one alone and didn't change its + modification date. In consequence, importers of a module with + an unchanged output <filename>.hi</filename> file were not + recompiled.</para> + + <para>This doesn't work any more. Suppose module + <literal>C</literal> imports module <literal>B</literal>, and + <literal>B</literal> imports module <literal>A</literal>. So + changes to module <literal>A</literal> might require module + <literal>C</literal> to be recompiled, and hence when + <filename>A.hi</filename> changes we should check whether + <literal>C</literal> should be recompiled. However, the + dependencies of <literal>C</literal> will only list + <literal>B.hi</literal>, not <literal>A.hi</literal>, and some + changes to <literal>A</literal> (changing the definition of a + function that appears in an inlining of a function exported by + <literal>B</literal>, say) may conceivably not change + <filename>B.hi</filename> one jot. So now…</para> + + <para>GHC keeps a version number on each interface file, and on + each type signature within the interface file. It also keeps in + every interface file a list of the version numbers of everything + it used when it last compiled the file. If the source file's + modification date is earlier than the <filename>.o</filename> + file's date (i.e. the source hasn't changed since the file was + last compiled), and the recompilation checking is on, GHC will be + clever. It compares the version numbers on the things it needs + this time with the version numbers on the things it needed last + time (gleaned from the interface file of the module being + compiled); if they are all the same it stops compiling rather + early in the process saying “Compilation IS NOT + required”. What a beautiful sight!</para> + + <para>Patrick Sansom had a workshop paper about how all this is + done (though the details have changed quite a bit). <ulink + url="mailto:sansom@dcs.gla.ac.uk">Ask him</ulink> if you want a + copy.</para> + + </sect2> + + <sect2 id="mutual-recursion"> + <title>How to compile mutually recursive modules</title> + + <indexterm><primary>module system, recursion</primary></indexterm> + <indexterm><primary>recursion, between modules</primary></indexterm> + + <para>GHC supports the compilation of mutually recursive modules. + This section explains how.</para> + + <para>Every cycle in the module import graph must be broken by a <filename>hs-boot</filename> file. + Suppose that modules <filename>A.hs</filename> and <filename>B.hs</filename> are Haskell source files, + thus: +<programlisting> +module A where + import B( TB(..) ) + + newtype TA = MkTA Int + + f :: TB -> TA + f (MkTB x) = MkTA x + +module B where + import {-# SOURCE #-} A( TA(..) ) + + data TB = MkTB !Int + + g :: TA -> TB + g (MkTA x) = MkTB x +</programlisting> +<indexterm><primary><literal>hs-boot</literal> + files</primary></indexterm> <indexterm><primary>importing, + <literal>hi-boot</literal> files</primary></indexterm> +Here <filename>A</filename> imports <filename>B</filename>, but <filename>B</filename> imports +<filename>A</filename> with a <literal>{-# SOURCE #-}</literal> pragma, which breaks the +circular dependency. For every module <filename>A.hs</filename> that is <literal>{-# SOURCE #-}</literal>-imported +in this way there must exist a souce file <literal>A.hs-boot</literal>. This file contains an abbreviated +version of <filename>A.hs</filename>, thus: +<programlisting> +module A where + newtype TA = MkTA Int +</programlisting> +</para> +<para>To compile these three files, issue the following commands: +<programlisting> + ghc -c A.hs-boot -- Poduces A.hi-boot, A.o-boot + ghc -c B.hs -- Consumes A.hi-boot, produces B.hi, B.o + ghc -c A.hs -- Consumes B.hi, produces A.hi, A.o + ghc -o foo A.o B.o -- Linking the program +</programlisting> +</para> +<para>There are several points to note here: +<itemizedlist> +<listitem> + <para>The file <filename>A.hs-boot</filename> is a programmer-written source file. + It must live in the same directory as its parent source file <filename>A.hs</filename>. + Currently, if you use a literate source file <filename>A.lhs</filename> you must + also use a literate boot file, <filename>A.lhs-boot</filename>; and vice versa. + </para></listitem> + +<listitem><para> + A <filename>hs-boot</filename> file is compiled by GHC, just like a <filename>hs</filename> file: +<programlisting> + ghc -c A.hs-boot +</programlisting> +When a hs-boot file <filename>A.hs-boot</filename> + is compiled, it is checked for scope and type errors. + When its parent module <filename>A.hs</filename> is compiled, the two are compared, and + an error is reported if the two are inconsistent. + </para></listitem> + + <listitem> + <para> Just as compiling <filename>A.hs</filename> produces an + interface file <filename>A.hi</filename>, and an object file + <filename>A.o</filename>, so compiling + <filename>A.hs-boot</filename> produces an interface file + <filename>A.hi-boot</filename>, and an pseudo-object file + <filename>A.o-boot</filename>: </para> + + <itemizedlist> + <listitem> + <para>The pseudo-object file <filename>A.o-boot</filename> is + empty (don't link it!), but it is very useful when using a + Makefile, to record when the <filename>A.hi-boot</filename> was + last brought up to date (see <xref + linkend="using-make"/>).</para> + </listitem> + + <listitem> + <para>The <filename>hi-boot</filename> generated by compiling a + <filename>hs-boot</filename> file is in the same + machine-generated binary format as any other GHC-generated + interface file (e.g. <filename>B.hi</filename>). You can + display its contents with <command>ghc + --show-iface</command>. If you specify a directory for + interface files, the <option>-ohidir</option> flag, then that + affects <filename>hi-boot</filename> files + too.</para> + </listitem> + </itemizedlist> + </listitem> + + <listitem><para> If hs-boot files are considered distinct from their parent source + files, and if a <literal>{-# SOURCE #-}</literal> import is considered to refer to the + hs-boot file, then the module import graph must have no cycles. The command + <command>ghc -M</command> will report an error if a cycle is found. + </para></listitem> + + <listitem><para> A module <literal>M</literal> that is + <literal>{-# SOURCE #-}</literal>-imported in a program will usually also be + ordinarily imported elsewhere. If not, <command>ghc --make</command> + automatically adds <literal>M</literal> to the set of moudles it tries to + compile and link, to ensure that <literal>M</literal>'s implementation is included in + the final program. + </para></listitem> +</itemizedlist> +</para> +<para> +A hs-boot file need only contain the bare + minimum of information needed to get the bootstrapping process + started. For example, it doesn't need to contain declarations + for <emphasis>everything</emphasis> that module + <literal>A</literal> exports, only the things required by the + module(s) that import <literal>A</literal> recursively.</para> +<para>A hs-boot file is written in a subset of Haskell: +<itemizedlist> +<listitem><para> The module header (including the export list), and import statements, are exactly as in +Haskell, and so are the scoping rules. + Hence, to mention a non-Prelude type or class, you must import it.</para></listitem> + +<listitem><para> There must be no value declarations, but there can be type signatures for +values. For example: +<programlisting> + double :: Int -> Int +</programlisting> +</para></listitem> +<listitem><para> Fixity declarations are exactly as in Haskell.</para></listitem> +<listitem><para> Type synonym declarations are exactly as in Haskell.</para></listitem> +<listitem><para> A data type declaration can either be given in full, exactly as in Haskell, or it +can be given abstractly, by omitting the '=' sign and everything that follows. For example: +<programlisting> + data T a b +</programlisting> + In a <emphasis>source</emphasis> program + this would declare TA to have no constructors (a GHC extension: see <xref linkend="nullary-types"/>), + but in an hi-boot file it means "I don't know or care what the constructors are". + This is the most common form of data type declaration, because it's easy to get right. + You <emphasis>can</emphasis> also write out the constructors but, if you do so, you must write + it out precisely as in its real definition.</para> + <para> + If you do not write out the constructors, you may need to give a kind + annotation (<xref linkend="sec-kinding"/>), to tell + GHC the kind of the type variable, if it is not "*". (In source files, this is worked out + from the way the type variable is used in the constructors.) For example: +<programlisting> + data R (x :: * -> *) y +</programlisting> +</para></listitem> +<listitem><para> Class declarations is exactly as in Haskell, except that you may not put +default method declarations. You can also omit all the class methods entirely. +</para></listitem> +<listitem><para> Do not include instance declarations. There is a complication to do with +how the dictionary functions are named. It may well work, but it's not a well-tested feature. + </para></listitem> +</itemizedlist> +</para> + </sect2> + + + <sect2 id="using-make"> + <title>Using <command>make</command></title> + + <indexterm><primary><literal>make</literal></primary></indexterm> + + <para>It is reasonably straightforward to set up a + <filename>Makefile</filename> to use with GHC, assuming you name + your source files the same as your modules. Thus:</para> + +<programlisting> +HC = ghc +HC_OPTS = -cpp $(EXTRA_HC_OPTS) + +SRCS = Main.lhs Foo.lhs Bar.lhs +OBJS = Main.o Foo.o Bar.o + +.SUFFIXES : .o .hs .hi .lhs .hc .s + +cool_pgm : $(OBJS) + rm -f $@ + $(HC) -o $@ $(HC_OPTS) $(OBJS) + +# Standard suffix rules +.o.hi: + @: + +.lhs.o: + $(HC) -c $< $(HC_OPTS) + +.hs.o: + $(HC) -c $< $(HC_OPTS) + +.o-boot.hi-boot: + @: + +.lhs-boot.o-boot: + $(HC) -c $< $(HC_OPTS) + +.hs-boot.o-boot: + $(HC) -c $< $(HC_OPTS) + +# Inter-module dependencies +Foo.o Foo.hc Foo.s : Baz.hi # Foo imports Baz +Main.o Main.hc Main.s : Foo.hi Baz.hi # Main imports Foo and Baz +</programlisting> + + <para>(Sophisticated <command>make</command> variants may + achieve some of the above more elegantly. Notably, + <command>gmake</command>'s pattern rules let you write the more + comprehensible:</para> + +<programlisting> +%.o : %.lhs + $(HC) -c $< $(HC_OPTS) +</programlisting> + + <para>What we've shown should work with any + <command>make</command>.)</para> + + <para>Note the cheesy <literal>.o.hi</literal> rule: It records + the dependency of the interface (<filename>.hi</filename>) file + on the source. The rule says a <filename>.hi</filename> file + can be made from a <filename>.o</filename> file by + doing…nothing. Which is true.</para> + <para> Note that the suffix rules are all repeated twice, once + for normal Haskell source files, and once for <filename>hs-boot</filename> + files (see <xref linkend="mutual-recursion"/>).</para> + + <para>Note also the inter-module dependencies at the end of the + Makefile, which take the form + +<programlisting> +Foo.o Foo.hc Foo.s : Baz.hi # Foo imports Baz +</programlisting> + + They tell <command>make</command> that if any of + <literal>Foo.o</literal>, <literal>Foo.hc</literal> or + <literal>Foo.s</literal> have an earlier modification date than + <literal>Baz.hi</literal>, then the out-of-date file must be + brought up to date. To bring it up to date, + <literal>make</literal> looks for a rule to do so; one of the + preceding suffix rules does the job nicely. These dependencies + can be generated automatically by <command>ghc</command>; see + <xref linkend="sec-makefile-dependencies"/></para> + + </sect2> + + <sect2 id="sec-makefile-dependencies"> + <title>Dependency generation</title> + <indexterm><primary>dependencies in Makefiles</primary></indexterm> + <indexterm><primary>Makefile dependencies</primary></indexterm> + + <para>Putting inter-dependencies of the form <literal>Foo.o : + Bar.hi</literal> into your <filename>Makefile</filename> by + hand is rather error-prone. Don't worry, GHC has support for + automatically generating the required dependencies. Add the + following to your <filename>Makefile</filename>:</para> + +<programlisting> +depend : + ghc -M $(HC_OPTS) $(SRCS) +</programlisting> + + <para>Now, before you start compiling, and any time you change + the <literal>imports</literal> in your program, do + <command>make depend</command> before you do <command>make + cool_pgm</command>. The command <command>ghc -M</command> will + append the needed dependencies to your + <filename>Makefile</filename>.</para> + + <para>In general, <command>ghc -M Foo</command> does the following. + For each module <literal>M</literal> in the set + <literal>Foo</literal> plus all its imports (transitively), + it adds to the Makefile: + <itemizedlist> + <listitem><para>A line recording the dependence of the object file on the source file. +<programlisting> +M.o : M.hs +</programlisting> +(or <literal>M.lhs</literal> if that is the filename you used). + </para></listitem> + <listitem><para> For each import declaration <literal>import X</literal> in <literal>M</literal>, + a line recording the dependence of <literal>M</literal> on <literal>X</literal>: +<programlisting> +M.o : X.hi +</programlisting></para></listitem> + <listitem><para> For each import declaration <literal>import {-# SOURCE #-} X</literal> in <literal>M</literal>, + a line recording the dependence of <literal>M</literal> on <literal>X</literal>: +<programlisting> +M.o : X.hi-boot +</programlisting> + (See <xref linkend="mutual-recursion"/> for details of + <literal>hi-boot</literal> style interface files.) + </para></listitem> + </itemizedlist> + If <literal>M</literal> imports multiple modules, then there will + be multiple lines with <filename>M.o</filename> as the + target.</para> + <para>There is no need to list all of the source files as arguments to the <command>ghc -M</command> command; + <command>ghc</command> traces the dependencies, just like <command>ghc --make</command> + (a new feature in GHC 6.4).</para> + + <para>Note that <literal>ghc -M</literal> needs to find a <emphasis>source + file</emphasis> for each module in the dependency graph, so that it can + parse the import declarations and follow dependencies. Any pre-compiled + modules without source files must therefore belong to a + package<footnote><para>This is a change in behaviour relative to 6.2 and + earlier.</para> + </footnote>.</para> + + <para>By default, <command>ghc -M</command> generates all the + dependencies, and then concatenates them onto the end of + <filename>makefile</filename> (or + <filename>Makefile</filename> if <filename>makefile</filename> + doesn't exist) bracketed by the lines "<literal># DO NOT + DELETE: Beginning of Haskell dependencies</literal>" and + "<literal># DO NOT DELETE: End of Haskell + dependencies</literal>". If these lines already exist in the + <filename>makefile</filename>, then the old dependencies are + deleted first.</para> + + <para>Don't forget to use the same <option>-package</option> + options on the <literal>ghc -M</literal> command line as you + would when compiling; this enables the dependency generator to + locate any imported modules that come from packages. The + package modules won't be included in the dependencies + generated, though (but see the + <option>––include-pkg-deps</option> option below).</para> + + <para>The dependency generation phase of GHC can take some + additional options, which you may find useful. For historical + reasons, each option passed to the dependency generator from + the GHC command line must be preceded by + <literal>-optdep</literal>. For example, to pass <literal>-f + .depend</literal> to the dependency generator, you say + +<screen> +ghc -M -optdep-f -optdep.depend ... +</screen> + + The options which affect dependency generation are:</para> + + <variablelist> + <varlistentry> + <term><option>-w</option></term> + <listitem> + <para>Turn off warnings about interface file shadowing.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v2</option></term> + <listitem> + <para>Print a full list of the module depenencies to stdout. + (This is the standard verbosity flag, so the list will + also be displayed with <option>-v3</option> and + <option>-v4</option>; + <xref linkend ="options-help"/>.)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-f</option> <replaceable>file</replaceable></term> + <listitem> + <para>Use <replaceable>file</replaceable> as the makefile, + rather than <filename>makefile</filename> or + <filename>Makefile</filename>. If + <replaceable>file</replaceable> doesn't exist, + <command>mkdependHS</command> creates it. We often use + <option>-f .depend</option> to put the dependencies in + <filename>.depend</filename> and then + <command>include</command> the file + <filename>.depend</filename> into + <filename>Makefile</filename>.</para> + </listitem> + </varlistentry> + +<!-- Retired with the move away from 'mkdependHS'. + <varlistentry> + <term><option>-o <osuf></option></term> + <listitem> + <para>Use <filename>.<osuf></filename> as the + "target file" suffix ( default: <literal>o</literal>). + Multiple <option>-o</option> flags are permitted + (GHC2.05 onwards). Thus "<option>-o hc -o o</option>" + will generate dependencies for <filename>.hc</filename> + and <filename>.o</filename> files.</para> + </listitem> + </varlistentry> +--> + <varlistentry> + <term><option>-s <suf></option></term> + <listitem> + <para>Make extra dependencies that declare that files + with suffix + <filename>.<suf>_<osuf></filename> + depend on interface files with suffix + <filename>.<suf>_hi</filename>, or (for + <literal>{-# SOURCE #-}</literal> + imports) on <filename>.hi-boot</filename>. Multiple + <option>-s</option> flags are permitted. For example, + <option>-o hc -s a -s b</option> will make dependencies + for <filename>.hc</filename> on + <filename>.hi</filename>, + <filename>.a_hc</filename> on + <filename>.a_hi</filename>, and + <filename>.b_hc</filename> on + <filename>.b_hi</filename>. (Useful in + conjunction with NoFib "ways".)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>––exclude-module=<file></option></term> + <listitem> + <para>Regard <filename><file></filename> as + "stable"; i.e., exclude it from having dependencies on + it.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-x</option></term> + <listitem> + <para>same as <option>––exclude-module</option></para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>––exclude-directory=<dirs></option></term> + <listitem> + <para>Regard the colon-separated list of directories + <filename><dirs></filename> as containing stable, + don't generate any dependencies on modules + therein.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>––include-module=<file></option></term> + <listitem> + <para>Regard <filename><file></filename> as not + "stable"; i.e., generate dependencies on it (if + any). This option is normally used in conjunction with + the <option>––exclude-directory</option> option.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>––include-pkg-deps</option></term> + <listitem> + <para>Regard modules imported from packages as unstable, + i.e., generate dependencies on any imported package modules + (including <literal>Prelude</literal>, and all other + standard Haskell libraries). Dependencies are not traced + recursively into packages; dependencies are only generated for + home-package modules on external-package modules directly imported + by the home package module. + This option is normally + only used by the various system libraries.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2 id="orphan-modules"> + <title>Orphan modules and instance declarations</title> + +<para> Haskell specifies that when compiling module M, any instance +declaration in any module "below" M is visible. (Module A is "below" +M if A is imported directly by M, or if A is below a module that M imports directly.) +In principle, GHC must therefore read the interface files of every module below M, +just in case they contain an instance declaration that matters to M. This would +be a disaster in practice, so GHC tries to be clever. </para> + +<para>In particular, if an instance declaration is in the same module as the definition +of any type or class mentioned in the head of the instance declaration, then +GHC has to visit that interface file anyway. Example:</para> +<programlisting> + module A where + instance C a => D (T a) where ... + data T a = ... +</programlisting> +<para> The instance declaration is only relevant if the type T is in use, and if +so, GHC will have visited A's interface file to find T's definition. </para> + +<para> The only problem comes when a module contains an instance declaration +and GHC has no other reason for visiting the module. Example: +<programlisting> + module Orphan where + instance C a => D (T a) where ... + class C a where ... +</programlisting> +Here, neither D nor T is declared in module Orphan. +We call such modules ``orphan modules'', +defined thus:</para> +<itemizedlist> + <listitem> <para> An <emphasis>orphan module</emphasis> + <indexterm><primary>orphan module</primary></indexterm> + contains at least one <emphasis>orphan instance</emphasis> or at + least one <emphasis>orphan rule</emphasis>.</para> </listitem> + + <listitem><para> An instance declaration in a module M is an <emphasis>orphan instance</emphasis> if + <indexterm><primary>orphan instance</primary></indexterm> + none of the type constructors + or classes mentioned in the instance head (the part after the ``<literal>=></literal>'') are declared + in M.</para> + + <para> Only the instance head counts. In the example above, it is not good enough for C's declaration + to be in module A; it must be the declaration of D or T.</para> + </listitem> + + <listitem><para> A rewrite rule in a module M is an <emphasis>orphan rule</emphasis> + <indexterm><primary>orphan rule</primary></indexterm> + if none of the variables, type constructors, + or classes that are free in the left hand side of the rule are declared in M. + </para> </listitem> + </itemizedlist> + + +<para> GHC identifies orphan modules, and visits the interface file of +every orphan module below the module being compiled. This is usually +wasted work, but there is no avoiding it. You should therefore do +your best to have as few orphan modules as possible. + +</para> + +<para> You can identify an orphan module by looking in its interface +file, <filename>M.hi</filename>, using the +<option>--show-iface</option>. If there is a ``!'' on the first line, +GHC considers it an orphan module. +</para> +</sect2> + + </sect1> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter" "sect1") *** + ;;; End: *** + --> diff --git a/docs/users_guide/sooner.xml b/docs/users_guide/sooner.xml new file mode 100644 index 0000000000..1aba5d1af0 --- /dev/null +++ b/docs/users_guide/sooner.xml @@ -0,0 +1,602 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="sooner-faster-quicker"> +<title>Advice on: sooner, faster, smaller, thriftier</title> + +<para>Please advise us of other “helpful hints” that +should go here!</para> + +<sect1 id="sooner"> +<title>Sooner: producing a program more quickly +</title> + +<indexterm><primary>compiling faster</primary></indexterm> +<indexterm><primary>faster compiling</primary></indexterm> + + <variablelist> + <varlistentry> + <term>Don't use <option>-O</option> or (especially) <option>-O2</option>:</term> + <listitem> + <para>By using them, you are telling GHC that you are + willing to suffer longer compilation times for + better-quality code.</para> + + <para>GHC is surprisingly zippy for normal compilations + without <option>-O</option>!</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use more memory:</term> + <listitem> + <para>Within reason, more memory for heap space means less + garbage collection for GHC, which means less compilation + time. If you use the <option>-Rghc-timing</option> option, + you'll get a garbage-collector report. (Again, you can use + the cheap-and-nasty <option>+RTS -Sstderr -RTS</option> + option to send the GC stats straight to standard + error.)</para> + + <para>If it says you're using more than 20% of total + time in garbage collecting, then more memory would + help.</para> + + <para>If the heap size is approaching the maximum (64M by + default), and you have lots of memory, try increasing the + maximum with the + <option>-M<size></option><indexterm><primary>-M<size> + option</primary></indexterm> option, e.g.: <command>ghc -c + -O -M1024m Foo.hs</command>.</para> + + <para>Increasing the default allocation area size used by + the compiler's RTS might also help: use the + <option>-A<size></option><indexterm><primary>-A<size> + option</primary></indexterm> option.</para> + + <para>If GHC persists in being a bad memory citizen, please + report it as a bug.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Don't use too much memory!</term> + <listitem> + <para>As soon as GHC plus its “fellow citizens” + (other processes on your machine) start using more than the + <emphasis>real memory</emphasis> on your machine, and the + machine starts “thrashing,” <emphasis>the party + is over</emphasis>. Compile times will be worse than + terrible! Use something like the csh-builtin + <command>time</command> command to get a report on how many + page faults you're getting.</para> + + <para>If you don't know what virtual memory, thrashing, and + page faults are, or you don't know the memory configuration + of your machine, <emphasis>don't</emphasis> try to be clever + about memory use: you'll just make your life a misery (and + for other people, too, probably).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Try to use local disks when linking:</term> + <listitem> + <para>Because Haskell objects and libraries tend to be + large, it can take many real seconds to slurp the bits + to/from a remote filesystem.</para> + + <para>It would be quite sensible to + <emphasis>compile</emphasis> on a fast machine using + remotely-mounted disks; then <emphasis>link</emphasis> on a + slow machine that had your disks directly mounted.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Don't derive/use <function>Read</function> unnecessarily:</term> + <listitem> + <para>It's ugly and slow.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GHC compiles some program constructs slowly:</term> + <listitem> + <para>Deeply-nested list comprehensions seem to be one such; + in the past, very large constant tables were bad, + too.</para> + + <para>We'd rather you reported such behaviour as a bug, so + that we can try to correct it.</para> + + <para>The part of the compiler that is occasionally prone to + wandering off for a long time is the strictness analyser. + You can turn this off individually with + <option>-fno-strictness</option>. + <indexterm><primary>-fno-strictness + anti-option</primary></indexterm></para> + + <para>To figure out which part of the compiler is badly + behaved, the + <option>-v2</option><indexterm><primary><option>-v</option></primary> + </indexterm> option is your friend.</para> + + <para>If your module has big wads of constant data, GHC may + produce a huge basic block that will cause the native-code + generator's register allocator to founder. Bring on + <option>-fvia-C</option><indexterm><primary>-fvia-C + option</primary></indexterm> (not that GCC will be that + quick about it, either).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Explicit <literal>import</literal> declarations:</term> + <listitem> + <para>Instead of saying <literal>import Foo</literal>, say + <literal>import Foo (...stuff I want...)</literal> You can + get GHC to tell you the minimal set of required imports by + using the <option>-ddump-minimal-imports</option> option + (see <xref linkend="hi-options"/>).</para> + + <para>Truthfully, the reduction on compilation time will be + very small. However, judicious use of + <literal>import</literal> declarations can make a program + easier to understand, so it may be a good idea + anyway.</para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + <sect1 id="faster"> + <title>Faster: producing a program that runs quicker</title> + + <indexterm><primary>faster programs, how to produce</primary></indexterm> + + <para>The key tool to use in making your Haskell program run + faster are GHC's profiling facilities, described separately in + <xref linkend="profiling"/>. There is <emphasis>no + substitute</emphasis> for finding where your program's time/space + is <emphasis>really</emphasis> going, as opposed to where you + imagine it is going.</para> + + <para>Another point to bear in mind: By far the best way to + improve a program's performance <emphasis>dramatically</emphasis> + is to use better algorithms. Once profiling has thrown the + spotlight on the guilty time-consumer(s), it may be better to + re-think your program than to try all the tweaks listed below.</para> + + <para>Another extremely efficient way to make your program snappy + is to use library code that has been Seriously Tuned By Someone + Else. You <emphasis>might</emphasis> be able to write a better + quicksort than the one in <literal>Data.List</literal>, but it + will take you much longer than typing <literal>import + Data.List</literal>.</para> + + <para>Please report any overly-slow GHC-compiled programs. Since + GHC doesn't have any credible competition in the performance + department these days it's hard to say what overly-slow means, so + just use your judgement! Of course, if a GHC compiled program + runs slower than the same program compiled with NHC or Hugs, then + it's definitely a bug.</para> + + <variablelist> + <varlistentry> + <term>Optimise, using <option>-O</option> or <option>-O2</option>:</term> + <listitem> + <para>This is the most basic way to make your program go + faster. Compilation time will be slower, especially with + <option>-O2</option>.</para> + + <para>At present, <option>-O2</option> is nearly + indistinguishable from <option>-O</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Compile via C and crank up GCC:</term> + <listitem> + <para>The native code-generator is designed to be quick, not + mind-bogglingly clever. Better to let GCC have a go, as it + tries much harder on register allocation, etc.</para> + + <para>At the moment, if you turn on <option>-O</option> you + get GCC instead. This may change in the future.</para> + + <para>So, when we want very fast code, we use: <option>-O + -fvia-C</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Overloaded functions are not your friend:</term> + <listitem> + <para>Haskell's overloading (using type classes) is elegant, + neat, etc., etc., but it is death to performance if left to + linger in an inner loop. How can you squash it?</para> + + <variablelist> + <varlistentry> + <term>Give explicit type signatures:</term> + <listitem> + <para>Signatures are the basic trick; putting them on + exported, top-level functions is good + software-engineering practice, anyway. (Tip: using + <option>-fwarn-missing-signatures</option><indexterm><primary>-fwarn-missing-signatures + option</primary></indexterm> can help enforce good + signature-practice).</para> + + <para>The automatic specialisation of overloaded + functions (with <option>-O</option>) should take care + of overloaded local and/or unexported functions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use <literal>SPECIALIZE</literal> pragmas:</term> + <listitem> + <indexterm><primary>SPECIALIZE pragma</primary></indexterm> + <indexterm><primary>overloading, death to</primary></indexterm> + + <para>Specialize the overloading on key functions in + your program. See <xref linkend="specialize-pragma"/> + and <xref linkend="specialize-instance-pragma"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“But how do I know where overloading is creeping in?”:</term> + <listitem> + <para>A low-tech way: grep (search) your interface + files for overloaded type signatures. You can view + interface files using the + <option>--show-iface</option> option (see <xref + linkend="hi-options"/>). + +<programlisting> +% ghc --show-iface Foo.hi | egrep '^[a-z].*::.*=>' +</programlisting> +</para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term>Strict functions are your dear friends:</term> + <listitem> + <para>and, among other things, lazy pattern-matching is your + enemy.</para> + + <para>(If you don't know what a “strict + function” is, please consult a functional-programming + textbook. A sentence or two of explanation here probably + would not do much good.)</para> + + <para>Consider these two code fragments: + +<programlisting> +f (Wibble x y) = ... # strict + +f arg = let { (Wibble x y) = arg } in ... # lazy +</programlisting> + + The former will result in far better code.</para> + + <para>A less contrived example shows the use of + <literal>cases</literal> instead of <literal>lets</literal> + to get stricter code (a good thing): + +<programlisting> +f (Wibble x y) # beautiful but slow + = let + (a1, b1, c1) = unpackFoo x + (a2, b2, c2) = unpackFoo y + in ... + +f (Wibble x y) # ugly, and proud of it + = case (unpackFoo x) of { (a1, b1, c1) -> + case (unpackFoo y) of { (a2, b2, c2) -> + ... + }} +</programlisting> + + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term>GHC loves single-constructor data-types:</term> + <listitem> + <para>It's all the better if a function is strict in a + single-constructor type (a type with only one + data-constructor; for example, tuples are single-constructor + types).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Newtypes are better than datatypes:</term> + <listitem> + <para>If your datatype has a single constructor with a + single field, use a <literal>newtype</literal> declaration + instead of a <literal>data</literal> declaration. The + <literal>newtype</literal> will be optimised away in most + cases.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>“How do I find out a function's strictness?”</term> + <listitem> + <para>Don't guess—look it up.</para> + + <para>Look for your function in the interface file, then for + the third field in the pragma; it should say + <literal>__S <string></literal>. The + <literal><string></literal> gives the strictness of + the function's arguments. <function>L</function> is lazy + (bad), <function>S</function> and <function>E</function> are + strict (good), <function>P</function> is + “primitive” (good), <function>U(...)</function> + is strict and “unpackable” (very good), and + <function>A</function> is absent (very good).</para> + + <para>For an “unpackable” + <function>U(...)</function> argument, the info inside tells + the strictness of its components. So, if the argument is a + pair, and it says <function>U(AU(LSS))</function>, that + means “the first component of the pair isn't used; the + second component is itself unpackable, with three components + (lazy in the first, strict in the second \& + third).”</para> + + <para>If the function isn't exported, just compile with the + extra flag <option>-ddump-simpl</option>; next to the + signature for any binder, it will print the self-same + pragmatic information as would be put in an interface file. + (Besides, Core syntax is fun to look at!)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Force key functions to be <literal>INLINE</literal>d (esp. monads):</term> + <listitem> + <para>Placing <literal>INLINE</literal> pragmas on certain + functions that are used a lot can have a dramatic effect. + See <xref linkend="inline-pragma"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Explicit <literal>export</literal> list:</term> + <listitem> + <para>If you do not have an explicit export list in a + module, GHC must assume that everything in that module will + be exported. This has various pessimising effects. For + example, if a bit of code is actually + <emphasis>unused</emphasis> (perhaps because of unfolding + effects), GHC will not be able to throw it away, because it + is exported and some other module may be relying on its + existence.</para> + + <para>GHC can be quite a bit more aggressive with pieces of + code if it knows they are not exported.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Look at the Core syntax!</term> + <listitem> + <para>(The form in which GHC manipulates your code.) Just + run your compilation with <option>-ddump-simpl</option> + (don't forget the <option>-O</option>).</para> + + <para>If profiling has pointed the finger at particular + functions, look at their Core code. <literal>lets</literal> + are bad, <literal>cases</literal> are good, dictionaries + (<literal>d.<Class>.<Unique></literal>) [or + anything overloading-ish] are bad, nested lambdas are + bad, explicit data constructors are good, primitive + operations (e.g., <literal>eqInt#</literal>) are + good,…</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use strictness annotations:</term> + <listitem> + <para>Putting a strictness annotation ('!') on a constructor + field helps in two ways: it adds strictness to the program, + which gives the strictness analyser more to work with, and + it might help to reduce space leaks.</para> + + <para>It can also help in a third way: when used with + <option>-funbox-strict-fields</option> (see <xref + linkend="options-f"/>), a strict field can be unpacked or + unboxed in the constructor, and one or more levels of + indirection may be removed. Unpacking only happens for + single-constructor datatypes (<literal>Int</literal> is a + good candidate, for example).</para> + + <para>Using <option>-funbox-strict-fields</option> is only + really a good idea in conjunction with <option>-O</option>, + because otherwise the extra packing and unpacking won't be + optimised away. In fact, it is possible that + <option>-funbox-strict-fields</option> may worsen + performance even <emphasis>with</emphasis> + <option>-O</option>, but this is unlikely (let us know if it + happens to you).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use unboxed types (a GHC extension):</term> + <listitem> + <para>When you are <emphasis>really</emphasis> desperate for + speed, and you want to get right down to the “raw + bits.” Please see <xref linkend="glasgow-unboxed"/> for + some information about using unboxed types.</para> + + <para>Before resorting to explicit unboxed types, try using + strict constructor fields and + <option>-funbox-strict-fields</option> first (see above). + That way, your code stays portable.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use <literal>foreign import</literal> (a GHC extension) to plug into fast libraries:</term> + <listitem> + <para>This may take real work, but… There exist piles + of massively-tuned library code, and the best thing is not + to compete with it, but link with it.</para> + + <para><xref linkend="ffi"/> describes the foreign function + interface.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Don't use <literal>Float</literal>s:</term> + <listitem> + <para>If you're using <literal>Complex</literal>, definitely + use <literal>Complex Double</literal> rather than + <literal>Complex Float</literal> (the former is specialised + heavily, but the latter isn't).</para> + + <para><literal>Floats</literal> (probably 32-bits) are + almost always a bad idea, anyway, unless you Really Know + What You Are Doing. Use <literal>Double</literal>s. + There's rarely a speed disadvantage—modern machines + will use the same floating-point unit for both. With + <literal>Double</literal>s, you are much less likely to hang + yourself with numerical errors.</para> + + <para>One time when <literal>Float</literal> might be a good + idea is if you have a <emphasis>lot</emphasis> of them, say + a giant array of <literal>Float</literal>s. They take up + half the space in the heap compared to + <literal>Doubles</literal>. However, this isn't true on a + 64-bit machine.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use unboxed arrays (<literal>UArray</literal>)</term> + <listitem> + <para>GHC supports arrays of unboxed elements, for several + basic arithmetic element types including + <literal>Int</literal> and <literal>Char</literal>: see the + <literal>Data.Array.Unboxed</literal> library for details. + These arrays are likely to be much faster than using + standard Haskell 98 arrays from the + <literal>Data.Array</literal> library.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term>Use a bigger heap!</term> + <listitem> + <para>If your program's GC stats + (<option>-S</option><indexterm><primary>-S RTS + option</primary></indexterm> RTS option) indicate that it's + doing lots of garbage-collection (say, more than 20% + of execution time), more memory might help—with the + <option>-M<size></option><indexterm><primary>-M<size> + RTS option</primary></indexterm> or + <option>-A<size></option><indexterm><primary>-A<size> + RTS option</primary></indexterm> RTS options (see <xref + linkend="rts-options-gc"/>).</para> + + <para>This is especially important if your program uses a + lot of mutable arrays of pointers or mutable variables + (i.e. <literal>STArray</literal>, + <literal>IOArray</literal>, <literal>STRef</literal> and + <literal>IORef</literal>, but not <literal>UArray</literal>, + <literal>STUArray</literal> or <literal>IOUArray</literal>). + GHC's garbage collector currently scans these objects on + every collection, so your program won't benefit from + generational GC in the normal way if you use lots of + these. Increasing the heap size to reduce the number of + collections will probably help.</para> + </listitem> + </varlistentry> + </variablelist> + +</sect1> + +<sect1 id="smaller"> +<title>Smaller: producing a program that is smaller +</title> + +<para> +<indexterm><primary>smaller programs, how to produce</primary></indexterm> +</para> + +<para> +Decrease the “go-for-it” threshold for unfolding smallish +expressions. Give a +<option>-funfolding-use-threshold0</option><indexterm><primary>-funfolding-use-threshold0 +option</primary></indexterm> option for the extreme case. (“Only unfoldings with +zero cost should proceed.”) Warning: except in certain specialised +cases (like Happy parsers) this is likely to actually +<emphasis>increase</emphasis> the size of your program, because unfolding +generally enables extra simplifying optimisations to be performed. +</para> + +<para> +Avoid <function>Read</function>. +</para> + +<para> +Use <literal>strip</literal> on your executables. +</para> + +</sect1> + +<sect1 id="thriftier"> +<title>Thriftier: producing a program that gobbles less heap space +</title> + +<para> +<indexterm><primary>memory, using less heap</primary></indexterm> +<indexterm><primary>space-leaks, avoiding</primary></indexterm> +<indexterm><primary>heap space, using less</primary></indexterm> +</para> + +<para> +“I think I have a space leak…” Re-run your program +with <option>+RTS -Sstderr</option>, and remove all doubt! (You'll +see the heap usage get bigger and bigger…) +[Hmmm…this might be even easier with the +<option>-G1</option> RTS option; so… <command>./a.out +RTS +-Sstderr -G1</command>...] +<indexterm><primary>-G RTS option</primary></indexterm> +<indexterm><primary>-Sstderr RTS option</primary></indexterm> +</para> + +<para> +Once again, the profiling facilities (<xref linkend="profiling"/>) are +the basic tool for demystifying the space behaviour of your program. +</para> + +<para> +Strict functions are good for space usage, as they are for time, as +discussed in the previous section. Strict functions get right down to +business, rather than filling up the heap with closures (the system's +notes to itself about how to evaluate something, should it eventually +be required). +</para> + +</sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/ug-book.xml.in b/docs/users_guide/ug-book.xml.in new file mode 100644 index 0000000000..c5710f1d77 --- /dev/null +++ b/docs/users_guide/ug-book.xml.in @@ -0,0 +1,30 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<bookinfo> +<title>@ProjectName@ User's Guide, Version @ProjectVersion@</title> +<author><othername>The GHC Team</othername></author> +<address> +<email>glasgow-haskell-{bugs,users}-request@haskell.org</email> +</address> +</bookinfo> + +&license; +&intro; +&installing; +&ghci; +&using; +&prof; +&sooner; +&lang-features; +&ffi-chap; +&wrong; +&utils; +&win32-dll; +&bugs; + +<index/> +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book") *** + ;;; End: *** + --> diff --git a/docs/users_guide/ug-ent.xml b/docs/users_guide/ug-ent.xml new file mode 100644 index 0000000000..cad75ab499 --- /dev/null +++ b/docs/users_guide/ug-ent.xml @@ -0,0 +1,23 @@ +<!ENTITY ghci SYSTEM "ghci.xml"> +<!ENTITY flags SYSTEM "flags.xml"> +<!ENTITY license SYSTEM "license.xml"> +<!ENTITY intro SYSTEM "intro.xml" > +<!ENTITY relnotes SYSTEM "6.0-notes.xml" > +<!ENTITY installing SYSTEM "installing.xml" > +<!ENTITY using SYSTEM "using.xml" > +<!ENTITY runtime SYSTEM "runtime_control.xml" > +<!ENTITY prof SYSTEM "profiling.xml" > +<!ENTITY debug SYSTEM "debugging.xml" > +<!ENTITY sooner SYSTEM "sooner.xml" > +<!ENTITY lang-features SYSTEM "lang.xml" > +<!ENTITY glasgowexts SYSTEM "glasgow_exts.xml" > +<!ENTITY packages SYSTEM "packages.xml" > +<!ENTITY parallel SYSTEM "parallel.xml" > +<!ENTITY phases SYSTEM "phases.xml" > +<!ENTITY primitives SYSTEM "primitives.xml" > +<!ENTITY separate SYSTEM "separate_compilation.xml" > +<!ENTITY bugs SYSTEM "bugs.xml" > +<!ENTITY wrong SYSTEM "gone_wrong.xml" > +<!ENTITY utils SYSTEM "utils.xml" > +<!ENTITY win32-dll SYSTEM "win32-dlls.xml"> +<!ENTITY ffi-chap SYSTEM "ffi-chap.xml"> diff --git a/docs/users_guide/users_guide.xml b/docs/users_guide/users_guide.xml new file mode 100644 index 0000000000..740e729b72 --- /dev/null +++ b/docs/users_guide/users_guide.xml @@ -0,0 +1,11 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" + "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [ +<!ENTITY % ug-ent SYSTEM "ug-ent.xml"> +%ug-ent; +<!ENTITY ug-book SYSTEM "ug-book.xml"> +]> + +<book id="users-guide"> +&ug-book; +</book> diff --git a/docs/users_guide/using.xml b/docs/users_guide/using.xml new file mode 100644 index 0000000000..8cbcd35fca --- /dev/null +++ b/docs/users_guide/using.xml @@ -0,0 +1,1976 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="using-ghc"> + <title>Using GHC</title> + + <indexterm><primary>GHC, using</primary></indexterm> + <indexterm><primary>using GHC</primary></indexterm> + + <sect1> + <title>Options overview</title> + + <para>GHC's behaviour is controlled by + <firstterm>options</firstterm>, which for historical reasons are + also sometimes referred to as command-line flags or arguments. + Options can be specified in three ways:</para> + + <sect2> + <title>command-line arguments</title> + + <indexterm><primary>structure, command-line</primary></indexterm> + <indexterm><primary>command-line</primary><secondary>arguments</secondary></indexterm> + <indexterm><primary>arguments</primary><secondary>command-line</secondary></indexterm> + + <para>An invocation of GHC takes the following form:</para> + +<screen> +ghc [argument...] +</screen> + + <para>command-line arguments are either options or file names.</para> + + <para>command-line options begin with <literal>-</literal>. + They may <emphasis>not</emphasis> be grouped: + <option>-vO</option> is different from <option>-v -O</option>. + Options need not precede filenames: e.g., <literal>ghc *.o -o + foo</literal>. All options are processed and then applied to + all files; you cannot, for example, invoke <literal>ghc -c -O1 + Foo.hs -O2 Bar.hs</literal> to apply different optimisation + levels to the files <filename>Foo.hs</filename> and + <filename>Bar.hs</filename>.</para> + </sect2> + + <sect2 id="source-file-options"> + <title>command line options in source files</title> + + <indexterm><primary>source-file options</primary></indexterm> + + <para>Sometimes it is useful to make the connection between a + source file and the command-line options it requires quite + tight. For instance, if a Haskell source file uses GHC + extensions, it will always need to be compiled with the + <option>-fglasgow-exts</option> option. Rather than maintaining + the list of per-file options in a <filename>Makefile</filename>, + it is possible to do this directly in the source file using the + <literal>OPTIONS_GHC</literal> pragma <indexterm><primary>OPTIONS_GHC + pragma</primary></indexterm>:</para> + +<programlisting> +{-# OPTIONS_GHC -fglasgow-exts #-} +module X where +... +</programlisting> + + <para><literal>OPTIONS_GHC</literal> pragmas are only looked for at + the top of your source files, upto the first + (non-literate,non-empty) line not containing + <literal>OPTIONS_GHC</literal>. Multiple <literal>OPTIONS_GHC</literal> + pragmas are recognised. Do not put comments before, or on the same line + as, the <literal>OPTIONS_GHC</literal> pragma.</para> + + <para>Note that your command shell does not + get to the source file options, they are just included literally + in the array of command-line arguments the compiler + maintains internally, so you'll be desperately disappointed if + you try to glob etc. inside <literal>OPTIONS_GHC</literal>.</para> + + <para>NOTE: the contents of OPTIONS_GHC are prepended to the + command-line options, so you <emphasis>do</emphasis> have the + ability to override OPTIONS_GHC settings via the command + line.</para> + + <para>It is not recommended to move all the contents of your + Makefiles into your source files, but in some circumstances, the + <literal>OPTIONS_GHC</literal> pragma is the Right Thing. (If you + use <option>-keep-hc-file-too</option> and have OPTION flags in + your module, the OPTIONS_GHC will get put into the generated .hc + file).</para> + </sect2> + + <sect2> + <title>Setting options in GHCi</title> + + <para>Options may also be modified from within GHCi, using the + <literal>:set</literal> command. See <xref linkend="ghci-set"/> + for more details.</para> + </sect2> + </sect1> + + <sect1 id="static-dynamic-flags"> + <title>Static, Dynamic, and Mode options</title> + <indexterm><primary>static</primary><secondary>options</secondary> + </indexterm> + <indexterm><primary>dynamic</primary><secondary>options</secondary> + </indexterm> + <indexterm><primary>mode</primary><secondary>options</secondary> + </indexterm> + + <para>Each of GHC's command line options is classified as either + <firstterm>static</firstterm> or <firstterm>dynamic</firstterm> or + <firstterm>mode</firstterm>:</para> + + <variablelist> + <varlistentry> + <term>Mode flags</term> + <listitem> + <para>For example, <option>--make</option> or <option>-E</option>. + There may be only a single mode flag on the command line. The + available modes are listed in <xref linkend="modes"/>.</para> + </listitem> + </varlistentry> + <varlistentry> + <term>Dynamic Flags</term> + <listitem> + <para>Most non-mode flags fall into this category. A dynamic flag + may be used on the command line, in a + <literal>GHC_OPTIONS</literal> pragma in a source file, or set + using <literal>:set</literal> in GHCi.</para> + </listitem> + </varlistentry> + <varlistentry> + <term>Static Flags</term> + <listitem> + <para>A few flags are "static", which means they can only be used on + the command-line, and remain in force over the entire GHC/GHCi + run.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>The flag reference tables (<xref + linkend="flag-reference"/>) lists the status of each flag.</para> + </sect1> + + <sect1 id="file-suffixes"> + <title>Meaningful file suffixes</title> + + <indexterm><primary>suffixes, file</primary></indexterm> + <indexterm><primary>file suffixes for GHC</primary></indexterm> + + <para>File names with “meaningful” suffixes (e.g., + <filename>.lhs</filename> or <filename>.o</filename>) cause the + “right thing” to happen to those files.</para> + + <variablelist> + + <varlistentry> + <term><filename>.hs</filename></term> + <listitem> + <para>A Haskell module.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <filename>.lhs</filename> + <indexterm><primary><literal>lhs</literal> suffix</primary></indexterm> + </term> + <listitem> + <para>A “literate Haskell” module.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>.hi</filename></term> + <listitem> + <para>A Haskell interface file, probably + compiler-generated.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>.hc</filename></term> + <listitem> + <para>Intermediate C file produced by the Haskell + compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>.c</filename></term> + <listitem> + <para>A C file not produced by the Haskell + compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>.s</filename></term> + <listitem> + <para>An assembly-language source file, usually produced by + the compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><filename>.o</filename></term> + <listitem> + <para>An object file, produced by an assembler.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>Files with other suffixes (or without suffixes) are passed + straight to the linker.</para> + + </sect1> + + <sect1 id="modes"> + <title>Modes of operation</title> + + <para>GHC's behaviour is firstly controlled by a mode flag. Only + one of these flags may be given, but it does not necessarily need + to be the first option on the command-line. The available modes + are:</para> + + <variablelist> + <varlistentry> + <term> + <cmdsynopsis><command>ghc</command> + <arg choice='plain'>––interactive</arg> + </cmdsynopsis> + <indexterm><primary>interactive mode</primary></indexterm> + <indexterm><primary>ghci</primary></indexterm> + </term> + <listitem> + <para>Interactive mode, which is also available as + <command>ghci</command>. Interactive mode is described in + more detail in <xref linkend="ghci"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <cmdsynopsis><command>ghc</command> + <arg choice='plain'>––make</arg> + </cmdsynopsis> + <indexterm><primary>make mode</primary></indexterm> + <indexterm><primary><option>––make</option></primary></indexterm> + </term> + <listitem> + <para>In this mode, GHC will build a multi-module Haskell + program automatically, figuring out dependencies for itself. + If you have a straightforward Haskell program, this is + likely to be much easier, and faster, than using + <command>make</command>. Make mode is described in <xref + linkend="make-mode"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <cmdsynopsis><command>ghc</command> + <arg choice='plain'>–e</arg> <arg choice='plain'><replaceable>expr</replaceable></arg> + </cmdsynopsis> + <indexterm><primary>eval mode</primary></indexterm> + </term> + <listitem> + <para>Expression-evaluation mode. This is very similar to + interactive mode, except that there is a single expression + to evaluate (<replaceable>expr</replaceable>) which is given + on the command line. See <xref linkend="eval-mode"/> for + more details.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <cmdsynopsis> + <command>ghc</command> + <group> + <arg>-E</arg> + <arg>-C</arg> + <arg>-S</arg> + <arg>-c</arg> + </group> + </cmdsynopsis> + <indexterm><primary><option>-E</option></primary></indexterm> + <indexterm><primary><option>-C</option></primary></indexterm> + <indexterm><primary><option>-S</option></primary></indexterm> + <indexterm><primary><option>-c</option></primary></indexterm> + </term> + <listitem> + <para>This is the traditional batch-compiler mode, in which + GHC can compile source files one at a time, or link objects + together into an executable. This mode also applies if + there is no other mode flag specified on the command line, + in which case it means that the specified files should be + compiled and then linked to form a program. See <xref + linkend="options-order"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <cmdsynopsis> + <command>ghc</command> + <arg choice='plain'>–M</arg> + </cmdsynopsis> + <indexterm><primary>dependency-generation mode</primary></indexterm> + </term> + <listitem> + <para>Dependency-generation mode. In this mode, GHC can be + used to generate dependency information suitable for use in + a <literal>Makefile</literal>. See <xref + linkend="sec-makefile-dependencies"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <cmdsynopsis> + <command>ghc</command> + <arg choice='plain'>––mk-dll</arg> + </cmdsynopsis> + <indexterm><primary>dependency-generation mode</primary></indexterm> + </term> + <listitem> + <para>DLL-creation mode (Windows only). See <xref + linkend="win32-dlls-create"/>.</para> + </listitem> + </varlistentry> + </variablelist> + + <sect2 id="make-mode"> + <title>Using <command>ghc</command> <option>––make</option></title> + <indexterm><primary><option>––make</option></primary></indexterm> + <indexterm><primary>separate compilation</primary></indexterm> + + <para>When given the <option>––make</option> option, + GHC will build a multi-module Haskell program by following + dependencies from a single root module (usually + <literal>Main</literal>). For example, if your + <literal>Main</literal> module is in a file called + <filename>Main.hs</filename>, you could compile and link the + program like this:</para> + +<screen> +ghc ––make Main.hs +</screen> + + <para>The command line may contain any number of source file + names or module names; GHC will figure out all the modules in + the program by following the imports from these initial modules. + It will then attempt to compile each module which is out of + date, and finally if there is a <literal>Main</literal> module, + the program will also be linked into an executable.</para> + + <para>The main advantages to using <literal>ghc + ––make</literal> over traditional + <literal>Makefile</literal>s are:</para> + + <itemizedlist> + <listitem> + <para>GHC doesn't have to be restarted for each compilation, + which means it can cache information between compilations. + Compiling a multi-module program with <literal>ghc + ––make</literal> can be up to twice as fast as + running <literal>ghc</literal> individually on each source + file.</para> + </listitem> + <listitem> + <para>You don't have to write a <literal>Makefile</literal>.</para> + <indexterm><primary><literal>Makefile</literal>s</primary><secondary>avoiding</secondary></indexterm> + </listitem> + <listitem> + <para>GHC re-calculates the dependencies each time it is + invoked, so the dependencies never get out of sync with the + source.</para> + </listitem> + </itemizedlist> + + <para>Any of the command-line options described in the rest of + this chapter can be used with + <option>––make</option>, but note that any options + you give on the command line will apply to all the source files + compiled, so if you want any options to apply to a single source + file only, you'll need to use an <literal>OPTIONS_GHC</literal> + pragma (see <xref linkend="source-file-options"/>).</para> + + <para>If the program needs to be linked with additional objects + (say, some auxiliary C code), then the object files can be + given on the command line and GHC will include them when linking + the executable.</para> + + <para>Note that GHC can only follow dependencies if it has the + source file available, so if your program includes a module for + which there is no source file, even if you have an object and an + interface file for the module, then GHC will complain. The + exception to this rule is for package modules, which may or may + not have source files.</para> + + <para>The source files for the program don't all need to be in + the same directory; the <option>-i</option> option can be used + to add directories to the search path (see <xref + linkend="search-path"/>).</para> + </sect2> + + <sect2 id="eval-mode"> + <title>Expression evaluation mode</title> + + <para>This mode is very similar to interactive mode, except that + there is a single expression to evaluate which is specified on + the command line as an argument to the <option>-e</option> + option:</para> + +<screen> +ghc -e <replaceable>expr</replaceable> +</screen> + + <para>Haskell source files may be named on the command line, and + they will be loaded exactly as in interactive mode. The + expression is evaluated in the context of the loaded + modules.</para> + + <para>For example, to load and run a Haskell program containing + a module <literal>Main</literal>, we might say</para> + +<screen> +ghc -e Main.main Main.hs +</screen> + + <para>or we can just use this mode to evaluate expressions in + the context of the <literal>Prelude</literal>:</para> + +<screen> +$ ghc -e "interact (unlines.map reverse.lines)" +hello +olleh +</screen> + </sect2> + + <sect2 id="options-order"> + <title>Batch compiler mode</title> + + <para>In <emphasis>batch mode</emphasis>, GHC will compile one or more source files + given on the command line.</para> + + <para>The first phase to run is determined by each input-file + suffix, and the last phase is determined by a flag. If no + relevant flag is present, then go all the way through linking. + This table summarises:</para> + + <informaltable> + <tgroup cols="4"> + <colspec align="left"/> + <colspec align="left"/> + <colspec align="left"/> + <colspec align="left"/> + + <thead> + <row> + <entry>Phase of the compilation system</entry> + <entry>Suffix saying “start here”</entry> + <entry>Flag saying “stop after”</entry> + <entry>(suffix of) output file</entry> + </row> + </thead> + <tbody> + <row> + <entry>literate pre-processor</entry> + <entry><literal>.lhs</literal></entry> + <entry>-</entry> + <entry><literal>.hs</literal></entry> + </row> + + <row> + <entry>C pre-processor (opt.) </entry> + <entry><literal>.hs</literal> (with + <option>-cpp</option>)</entry> + <entry><option>-E</option></entry> + <entry><literal>.hspp</literal></entry> + </row> + + <row> + <entry>Haskell compiler</entry> + <entry><literal>.hs</literal></entry> + <entry><option>-C</option>, <option>-S</option></entry> + <entry><literal>.hc</literal>, <literal>.s</literal></entry> + </row> + + <row> + <entry>C compiler (opt.)</entry> + <entry><literal>.hc</literal> or <literal>.c</literal></entry> + <entry><option>-S</option></entry> + <entry><literal>.s</literal></entry> + </row> + + <row> + <entry>assembler</entry> + <entry><literal>.s</literal></entry> + <entry><option>-c</option></entry> + <entry><literal>.o</literal></entry> + </row> + + <row> + <entry>linker</entry> + <entry><replaceable>other</replaceable></entry> + <entry>-</entry> + <entry><filename>a.out</filename></entry> + </row> + </tbody> + </tgroup> + </informaltable> + + <indexterm><primary><option>-C</option></primary></indexterm> + <indexterm><primary><option>-E</option></primary></indexterm> + <indexterm><primary><option>-S</option></primary></indexterm> + <indexterm><primary><option>-c</option></primary></indexterm> + + <para>Thus, a common invocation would be: </para> + +<screen> +ghc -c Foo.hs</screen> + + <para>to compile the Haskell source file + <filename>Foo.hs</filename> to an object file + <filename>Foo.o</filename>.</para> + + <para>Note: What the Haskell compiler proper produces depends on + whether a native-code generator<indexterm><primary>native-code + generator</primary></indexterm> is used (producing assembly + language) or not (producing C). See <xref + linkend="options-codegen"/> for more details.</para> + + <para>Note: C pre-processing is optional, the + <option>-cpp</option><indexterm><primary><option>-cpp</option></primary></indexterm> + flag turns it on. See <xref linkend="c-pre-processor"/> for more + details.</para> + + <para>Note: The option <option>-E</option><indexterm><primary>-E + option</primary></indexterm> runs just the pre-processing passes + of the compiler, dumping the result in a file. Note that this + differs from the previous behaviour of dumping the file to + standard output.</para> + + <sect3 id="overriding-suffixes"> + <title>Overriding the default behaviour for a file</title> + + <para>As described above, the way in which a file is processed by GHC + depends on its suffix. This behaviour can be overriden using the + <option>-x</option> option:</para> + + <variablelist> + <varlistentry> + <term><option>-x</option> <replaceable>suffix</replaceable> + <indexterm><primary><option>-x</option></primary> + </indexterm></term> + <listitem> + <para>Causes all files following this option on the command + line to be processed as if they had the suffix + <replaceable>suffix</replaceable>. For example, to compile a + Haskell module in the file <literal>M.my-hs</literal>, + use <literal>ghc -c -x hs M.my-hs</literal>.</para> + </listitem> + </varlistentry> + </variablelist> + </sect3> + + </sect2> + </sect1> + + <sect1 id="options-help"> + <title>Help and verbosity options</title> + + <indexterm><primary>help options</primary></indexterm> + <indexterm><primary>verbosity options</primary></indexterm> + + <variablelist> + <varlistentry> + <term> + <option>––help</option> + <indexterm><primary><option>––help</option></primary></indexterm> + </term> + <term> + <option>-?</option> + <indexterm><primary><option>-?</option></primary></indexterm> + </term> + <listitem> + <para>Cause GHC to spew a long usage message to standard + output and then exit.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-v</option> + <indexterm><primary><option>-v</option></primary></indexterm> + </term> + <listitem> + <para>The <option>-v</option> option makes GHC + <emphasis>verbose</emphasis>: it reports its version number + and shows (on stderr) exactly how it invokes each phase of + the compilation system. Moreover, it passes the + <option>-v</option> flag to most phases; each reports its + version number (and possibly some other information).</para> + + <para>Please, oh please, use the <option>-v</option> option + when reporting bugs! Knowing that you ran the right bits in + the right order is always the first thing we want to + verify.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-v</option><replaceable>n</replaceable> + <indexterm><primary><option>-v</option></primary></indexterm> + </term> + <listitem> + <para>To provide more control over the compiler's verbosity, + the <option>-v</option> flag takes an optional numeric + argument. Specifying <option>-v</option> on its own is + equivalent to <option>-v3</option>, and the other levels + have the following meanings:</para> + + <variablelist> + <varlistentry> + <term><option>-v0</option></term> + <listitem> + <para>Disable all non-essential messages (this is the + default).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v1</option></term> + <listitem> + <para>Minimal verbosity: print one line per + compilation (this is the default when + <option>––make</option> or + <option>––interactive</option> is on).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v2</option></term> + <listitem> + <para>Print the name of each compilation phase as it + is executed. (equivalent to + <option>-dshow-passes</option>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v3</option></term> + <listitem> + <para>The same as <option>-v2</option>, except that in + addition the full command line (if appropriate) for + each compilation phase is also printed.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-v4</option></term> + <listitem> + <para>The same as <option>-v3</option> except that the + intermediate program representation after each + compilation phase is also printed (excluding + preprocessed and C/assembly files).</para> + </listitem> + </varlistentry> + </variablelist> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-V</option> + <indexterm><primary><option>-V</option></primary></indexterm> + </term> + <term> + <option>––version</option> + <indexterm><primary><option>––version</option></primary></indexterm> + </term> + <listitem> + <para>Print a one-line string including GHC's version number.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––numeric-version</option> + <indexterm><primary><option>––numeric-version</option></primary></indexterm> + </term> + <listitem> + <para>Print GHC's numeric version number only.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>––print-libdir</option> + <indexterm><primary><option>––print-libdir</option></primary></indexterm> + </term> + <listitem> + <para>Print the path to GHC's library directory. This is + the top of the directory tree containing GHC's libraries, + interfaces, and include files (usually something like + <literal>/usr/local/lib/ghc-5.04</literal> on Unix). This + is the value of + <literal>$libdir</literal><indexterm><primary><literal>libdir</literal></primary> + </indexterm>in the package configuration file (see <xref + linkend="packages"/>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-ferror-spans</option> + <indexterm><primary><option>-ferror-spans</option></primary> + </indexterm> + </term> + <listitem> + <para>Causes GHC to emit the full source span of the + syntactic entity relating to an error message. Normally, GHC + emits the source location of the start of the syntactic + entity only.</para> + + <para>For example:</para> + +<screen>test.hs:3:6: parse error on input `where'</screen> + + <para>becomes:</para> + +<screen>test296.hs:3:6-10: parse error on input `where'</screen> + + <para>And multi-line spans are possible too:</para> + +<screen>test.hs:(5,4)-(6,7): + Conflicting definitions for `a' + Bound at: test.hs:5:4 + test.hs:6:7 + In the binding group for: a, b, a</screen> + + <para>Note that line numbers start counting at one, but + column numbers start at zero. This choice was made to + follow existing convention (i.e. this is how Emacs does + it).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-Rghc-timing</option> + <indexterm><primary><option>-Rghc-timing</option></primary></indexterm> + </term> + <listitem> + <para>Prints a one-line summary of timing statistics for the + GHC run. This option is equivalent to + <literal>+RTS -tstderr</literal>, see <xref + linkend="rts-options-gc" />. + </para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + + &separate; + + <sect1 id="options-sanity"> + <title>Warnings and sanity-checking</title> + + <indexterm><primary>sanity-checking options</primary></indexterm> + <indexterm><primary>warnings</primary></indexterm> + + + <para>GHC has a number of options that select which types of + non-fatal error messages, otherwise known as warnings, can be + generated during compilation. By default, you get a standard set + of warnings which are generally likely to indicate bugs in your + program. These are: + <option>-fwarn-overlapping-patterns</option>, + <option>-fwarn-deprecations</option>, + <option>-fwarn-duplicate-exports</option>, + <option>-fwarn-missing-fields</option>, and + <option>-fwarn-missing-methods</option>. The following flags are + simple ways to select standard “packages” of warnings: + </para> + + <variablelist> + + <varlistentry> + <term><option>-W</option>:</term> + <listitem> + <indexterm><primary>-W option</primary></indexterm> + <para>Provides the standard warnings plus + <option>-fwarn-incomplete-patterns</option>, + <option>-fwarn-unused-matches</option>, + <option>-fwarn-unused-imports</option>, + <option>-fwarn-misc</option>, and + <option>-fwarn-unused-binds</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-w</option>:</term> + <listitem> + <indexterm><primary><option>-w</option></primary></indexterm> + <para>Turns off all warnings, including the standard ones.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-Wall</option>:</term> + <listitem> + <indexterm><primary><option>-Wall</option></primary></indexterm> + <para>Turns on all warning options.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-Werror</option>:</term> + <listitem> + <indexterm><primary><option>-Werror</option></primary></indexterm> + <para>Makes any warning into a fatal error. Useful so that you don't + miss warnings when doing batch compilation. </para> + </listitem> + </varlistentry> + + </variablelist> + + <para>The full set of warning options is described below. To turn + off any warning, simply give the corresponding + <option>-fno-warn-...</option> option on the command line.</para> + + <variablelist> + + <varlistentry> + <term><option>-fwarn-deprecations</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-deprecations</option></primary> + </indexterm> + <indexterm><primary>deprecations</primary></indexterm> + <para>Causes a warning to be emitted when a deprecated + function or type is used. Entities can be marked as + deprecated using a pragma, see <xref + linkend="deprecated-pragma"/>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-duplicate-exports</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-duplicate-exports</option></primary></indexterm> + <indexterm><primary>duplicate exports, warning</primary></indexterm> + <indexterm><primary>export lists, duplicates</primary></indexterm> + + <para>Have the compiler warn about duplicate entries in + export lists. This is useful information if you maintain + large export lists, and want to avoid the continued export + of a definition after you've deleted (one) mention of it in + the export list.</para> + + <para>This option is on by default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-hi-shadowing</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-hi-shadowing</option></primary></indexterm> + <indexterm><primary>shadowing</primary> + <secondary>interface files</secondary></indexterm> + + <para>Causes the compiler to emit a warning when a module or + interface file in the current directory is shadowing one + with the same module name in a library or other + directory.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-incomplete-patterns</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-incomplete-patterns</option></primary></indexterm> + <indexterm><primary>incomplete patterns, warning</primary></indexterm> + <indexterm><primary>patterns, incomplete</primary></indexterm> + + <para>Similarly for incomplete patterns, the function + <function>g</function> below will fail when applied to + non-empty lists, so the compiler will emit a warning about + this when <option>-fwarn-incomplete-patterns</option> is + enabled.</para> + +<programlisting> +g [] = 2 +</programlisting> + + <para>This option isn't enabled be default because it can be + a bit noisy, and it doesn't always indicate a bug in the + program. However, it's generally considered good practice + to cover all the cases in your functions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-incomplete-record-updates</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-incomplete-record-updates</option></primary></indexterm> + <indexterm><primary>incomplete record updates, warning</primary></indexterm> + <indexterm><primary>record updates, incomplete</primary></indexterm> + + <para>The function + <function>f</function> below will fail when applied to + <literal>Bar</literal>, so the compiler will emit a warning about + this when <option>-fwarn-incomplete-record-updates</option> is + enabled.</para> + +<programlisting> +data Foo = Foo { x :: Int } + | Bar + +f :: Foo -> Foo +f foo = foo { x = 6 } +</programlisting> + + <para>This option isn't enabled be default because it can be + very noisy, and it often doesn't indicate a bug in the + program.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fwarn-misc</option>: + <indexterm><primary><option>-fwarn-misc</option></primary></indexterm> + </term> + <listitem> + <para>Turns on warnings for various harmless but untidy + things. This currently includes: importing a type with + <literal>(..)</literal> when the export is abstract, and + listing duplicate class assertions in a qualified type.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fwarn-missing-fields</option>: + <indexterm><primary><option>-fwarn-missing-fields</option></primary></indexterm> + <indexterm><primary>missing fields, warning</primary></indexterm> + <indexterm><primary>fields, missing</primary></indexterm> + </term> + <listitem> + + <para>This option is on by default, and warns you whenever + the construction of a labelled field constructor isn't + complete, missing initializers for one or more fields. While + not an error (the missing fields are initialised with + bottoms), it is often an indication of a programmer error.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-missing-methods</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-missing-methods</option></primary></indexterm> + <indexterm><primary>missing methods, warning</primary></indexterm> + <indexterm><primary>methods, missing</primary></indexterm> + + <para>This option is on by default, and warns you whenever + an instance declaration is missing one or more methods, and + the corresponding class declaration has no default + declaration for them.</para> + <para>The warning is suppressed if the method name + begins with an underscore. Here's an example where this is useful: + <programlisting> + class C a where + _simpleFn :: a -> String + complexFn :: a -> a -> String + complexFn x y = ... _simpleFn ... + </programlisting> + The idea is that: (a) users of the class will only call <literal>complexFn</literal>; + never <literal>_simpleFn</literal>; and (b) + instance declarations can define either <literal>complexFn</literal> or <literal>_simpleFn</literal>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-missing-signatures</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-missing-signatures</option></primary></indexterm> + <indexterm><primary>type signatures, missing</primary></indexterm> + + <para>If you would like GHC to check that every top-level + function/value has a type signature, use the + <option>-fwarn-missing-signatures</option> option. This + option is off by default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-name-shadowing</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-name-shadowing</option></primary></indexterm> + <indexterm><primary>shadowing, warning</primary></indexterm> + + <para>This option causes a warning to be emitted whenever an + inner-scope value has the same name as an outer-scope value, + i.e. the inner value shadows the outer one. This can catch + typographical errors that turn into hard-to-find bugs, e.g., + in the inadvertent cyclic definition <literal>let x = ... x + ... in</literal>.</para> + + <para>Consequently, this option does + <emphasis>will</emphasis> complain about cyclic recursive + definitions.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-orphans</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-orphans</option></primary></indexterm> + <indexterm><primary>orphan instances, warning</primary></indexterm> + <indexterm><primary>orphan rules, warning</primary></indexterm> + + <para>This option causes a warning to be emitted whenever the + module contains an "orphan" instance declaration or rewrite rule. + An instance declartion is an orphan if it appears in a module in + which neither the class nor the type being instanced are declared + in the same module. A rule is an orphan if it is a rule for a + function declared in another module. A module containing any + orphans is called an orphan module.</para> + <para>The trouble with orphans is that GHC must pro-actively read the interface + files for all orphan modules, just in case their instances or rules + play a role, whether or not the module's interface would otherwise + be of any use. Other things being equal, avoid orphan modules.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fwarn-overlapping-patterns</option>: + <indexterm><primary><option>-fwarn-overlapping-patterns</option></primary></indexterm> + <indexterm><primary>overlapping patterns, warning</primary></indexterm> + <indexterm><primary>patterns, overlapping</primary></indexterm> + </term> + <listitem> + <para>By default, the compiler will warn you if a set of + patterns are overlapping, i.e.,</para> + +<programlisting> +f :: String -> Int +f [] = 0 +f (_:xs) = 1 +f "2" = 2 +</programlisting> + + <para>where the last pattern match in <function>f</function> + won't ever be reached, as the second pattern overlaps + it. More often than not, redundant patterns is a programmer + mistake/error, so this option is enabled by default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-simple-patterns</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-simple-patterns</option></primary> + </indexterm> + <para>Causes the compiler to warn about lambda-bound + patterns that can fail, eg. <literal>\(x:xs)->...</literal>. + Normally, these aren't treated as incomplete patterns by + <option>-fwarn-incomplete-patterns</option>.</para> + <para>``Lambda-bound patterns'' includes all places where there is a single pattern, + including list comprehensions and do-notation. In these cases, a pattern-match + failure is quite legitimate, and triggers filtering (list comprehensions) or + the monad <literal>fail</literal> operation (monads). For example: + <programlisting> + f :: [Maybe a] -> [a] + f xs = [y | Just y <- xs] + </programlisting> + Switching on <option>-fwarn-simple-patterns</option> will elicit warnings about + these probably-innocent cases, which is why the flag is off by default. </para> + <para> The <literal>deriving( Read )</literal> mechanism produces monadic code with + pattern matches, so you will also get misleading warnings about the compiler-generated + code. (This is arguably a Bad Thing, but it's awkward to fix.)</para> + + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-type-defaults</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-type-defaults</option></primary></indexterm> + <indexterm><primary>defaulting mechanism, warning</primary></indexterm> + <para>Have the compiler warn/inform you where in your source + the Haskell defaulting mechanism for numeric types kicks + in. This is useful information when converting code from a + context that assumed one default into one with another, + e.g., the `default default' for Haskell 1.4 caused the + otherwise unconstrained value <constant>1</constant> to be + given the type <literal>Int</literal>, whereas Haskell 98 + defaults it to <literal>Integer</literal>. This may lead to + differences in performance and behaviour, hence the + usefulness of being non-silent about this.</para> + + <para>This warning is off by default.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-unused-binds</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-unused-binds</option></primary></indexterm> + <indexterm><primary>unused binds, warning</primary></indexterm> + <indexterm><primary>binds, unused</primary></indexterm> + <para>Report any function definitions (and local bindings) + which are unused. For top-level functions, the warning is + only given if the binding is not exported.</para> + <para>A definition is regarded as "used" if (a) it is exported, or (b) it is + mentioned in the right hand side of another definition that is used, or (c) the + function it defines begins with an underscore. The last case provides a + way to suppress unused-binding warnings selectively. </para> + <para> Notice that a variable + is reported as unused even if it appears in the right-hand side of another + unused binding. </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-unused-imports</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-unused-imports</option></primary></indexterm> + <indexterm><primary>unused imports, warning</primary></indexterm> + <indexterm><primary>imports, unused</primary></indexterm> + + <para>Report any modules that are explicitly imported but + never used. However, the form <literal>import M()</literal> is + never reported as an unused import, because it is a useful idiom + for importing instance declarations, which are anonymous in Haskell.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fwarn-unused-matches</option>:</term> + <listitem> + <indexterm><primary><option>-fwarn-unused-matches</option></primary></indexterm> + <indexterm><primary>unused matches, warning</primary></indexterm> + <indexterm><primary>matches, unused</primary></indexterm> + + <para>Report all unused variables which arise from pattern + matches, including patterns consisting of a single variable. + For instance <literal>f x y = []</literal> would report + <varname>x</varname> and <varname>y</varname> as unused. The + warning is suppressed if the variable name begins with an underscore, thus: + <programlisting> + f _x = True + </programlisting> + </para> + </listitem> + </varlistentry> + + </variablelist> + + <para>If you're feeling really paranoid, the + <option>-dcore-lint</option> + option<indexterm><primary><option>-dcore-lint</option></primary></indexterm> + is a good choice. It turns on heavyweight intra-pass + sanity-checking within GHC. (It checks GHC's sanity, not + yours.)</para> + + </sect1> + + &packages; + + <sect1 id="options-optimise"> + <title>Optimisation (code improvement)</title> + + <indexterm><primary>optimisation</primary></indexterm> + <indexterm><primary>improvement, code</primary></indexterm> + + <para>The <option>-O*</option> options specify convenient + “packages” of optimisation flags; the + <option>-f*</option> options described later on specify + <emphasis>individual</emphasis> optimisations to be turned on/off; + the <option>-m*</option> options specify + <emphasis>machine-specific</emphasis> optimisations to be turned + on/off.</para> + + <sect2 id="optimise-pkgs"> + <title><option>-O*</option>: convenient “packages” of optimisation flags.</title> + + <para>There are <emphasis>many</emphasis> options that affect + the quality of code produced by GHC. Most people only have a + general goal, something like “Compile quickly” or + “Make my program run like greased lightning.” The + following “packages” of optimisations (or lack + thereof) should suffice.</para> + + <para>Note that higher optimisation levels cause more + cross-module optimisation to be performed, which can have an + impact on how much of your program needs to be recompiled when + you change something. This is one reaosn to stick to + no-optimisation when developing code.</para> + + <variablelist> + + <varlistentry> + <term> + No <option>-O*</option>-type option specified: + <indexterm><primary>-O* not specified</primary></indexterm> + </term> + <listitem> + <para>This is taken to mean: “Please compile + quickly; I'm not over-bothered about compiled-code + quality.” So, for example: <command>ghc -c + Foo.hs</command></para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-O0</option>: + <indexterm><primary><option>-O0</option></primary></indexterm> + </term> + <listitem> + <para>Means “turn off all optimisation”, + reverting to the same settings as if no + <option>-O</option> options had been specified. Saying + <option>-O0</option> can be useful if + eg. <command>make</command> has inserted a + <option>-O</option> on the command line already.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-O</option> or <option>-O1</option>: + <indexterm><primary>-O option</primary></indexterm> + <indexterm><primary>-O1 option</primary></indexterm> + <indexterm><primary>optimise</primary><secondary>normally</secondary></indexterm> + </term> + <listitem> + <para>Means: “Generate good-quality code without + taking too long about it.” Thus, for example: + <command>ghc -c -O Main.lhs</command></para> + + <para><option>-O</option> currently also implies + <option>-fvia-C</option>. This may change in the + future.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-O2</option>: + <indexterm><primary>-O2 option</primary></indexterm> + <indexterm><primary>optimise</primary><secondary>aggressively</secondary></indexterm> + </term> + <listitem> + <para>Means: “Apply every non-dangerous + optimisation, even if it means significantly longer + compile times.”</para> + + <para>The avoided “dangerous” optimisations + are those that can make runtime or space + <emphasis>worse</emphasis> if you're unlucky. They are + normally turned on or off individually.</para> + + <para>At the moment, <option>-O2</option> is + <emphasis>unlikely</emphasis> to produce better code than + <option>-O</option>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-Ofile <file></option>: + <indexterm><primary>-Ofile <file> option</primary></indexterm> + <indexterm><primary>optimising, customised</primary></indexterm> + </term> + <listitem> + <para>(NOTE: not supported since GHC 4.x. Please ask if + you're interested in this.)</para> + + <para>For those who need <emphasis>absolute</emphasis> + control over <emphasis>exactly</emphasis> what options are + used (e.g., compiler writers, sometimes :-), a list of + options can be put in a file and then slurped in with + <option>-Ofile</option>.</para> + + <para>In that file, comments are of the + <literal>#</literal>-to-end-of-line variety; blank + lines and most whitespace is ignored.</para> + + <para>Please ask if you are baffled and would like an + example of <option>-Ofile</option>!</para> + </listitem> + </varlistentry> + </variablelist> + + <para>We don't use a <option>-O*</option> flag for day-to-day + work. We use <option>-O</option> to get respectable speed; + e.g., when we want to measure something. When we want to go for + broke, we tend to use <option>-O2 -fvia-C</option> (and we go for + lots of coffee breaks).</para> + + <para>The easiest way to see what <option>-O</option> (etc.) + “really mean” is to run with <option>-v</option>, + then stand back in amazement.</para> + </sect2> + + <sect2 id="options-f"> + <title><option>-f*</option>: platform-independent flags</title> + + <indexterm><primary>-f* options (GHC)</primary></indexterm> + <indexterm><primary>-fno-* options (GHC)</primary></indexterm> + + <para>These flags turn on and off individual optimisations. + They are normally set via the <option>-O</option> options + described above, and as such, you shouldn't need to set any of + them explicitly (indeed, doing so could lead to unexpected + results). However, there are one or two that may be of + interest:</para> + + <variablelist> + <varlistentry> + <term><option>-fexcess-precision</option>:</term> + <listitem> + <indexterm><primary><option>-fexcess-precision</option></primary></indexterm> + <para>When this option is given, intermediate floating + point values can have a <emphasis>greater</emphasis> + precision/range than the final type. Generally this is a + good thing, but some programs may rely on the exact + precision/range of + <literal>Float</literal>/<literal>Double</literal> values + and should not use this option for their compilation.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-fignore-asserts</option>:</term> + <listitem> + <indexterm><primary><option>-fignore-asserts</option></primary></indexterm> + <para>Causes GHC to ignore uses of the function + <literal>Exception.assert</literal> in source code (in + other words, rewriting <literal>Exception.assert p + e</literal> to <literal>e</literal> (see <xref + linkend="sec-assertions"/>). This flag is turned on by + <option>-O</option>. + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-cse</option> + <indexterm><primary><option>-fno-cse</option></primary></indexterm> + </term> + <listitem> + <para>Turns off the common-sub-expression elimination optimisation. + Can be useful if you have some <literal>unsafePerformIO</literal> + expressions that you don't want commoned-up.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-strictness</option> + <indexterm><primary><option>-fno-strictness</option></primary></indexterm> + </term> + <listitem> + <para>Turns off the strictness analyser; sometimes it eats + too many cycles.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-full-laziness</option> + <indexterm><primary><option>-fno-full-laziness</option></primary></indexterm> + </term> + <listitem> + <para>Turns off the full laziness optimisation (also known as + let-floating). Full laziness increases sharing, which can lead + to increased memory residency.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-fno-state-hack</option> + <indexterm><primary><option>-fno-state-hack</option></primary></indexterm> + </term> + <listitem> + <para>Turn off the "state hack" whereby any lambda with a + <literal>State#</literal> token as argument is considered to be + single-entry, hence it is considered OK to inline things inside + it. This can improve performance of IO and ST monad code, but it + runs the risk of reducing sharing.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-funbox-strict-fields</option>: + <indexterm><primary><option>-funbox-strict-fields</option></primary></indexterm> + <indexterm><primary>strict constructor fields</primary></indexterm> + <indexterm><primary>constructor fields, strict</primary></indexterm> + </term> + <listitem> + <para>This option causes all constructor fields which are + marked strict (i.e. “!”) to be unboxed or + unpacked if possible. It is equivalent to adding an + <literal>UNPACK</literal> pragma to every strict + constructor field (see <xref + linkend="unpack-pragma"/>).</para> + + <para>This option is a bit of a sledgehammer: it might + sometimes make things worse. Selectively unboxing fields + by using <literal>UNPACK</literal> pragmas might be + better.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-funfolding-update-in-place<n></option> + <indexterm><primary><option>-funfolding-update-in-place</option></primary></indexterm> + </term> + <listitem> + <para>Switches on an experimental "optimisation". + Switching it on makes the compiler a little keener to + inline a function that returns a constructor, if the + context is that of a thunk. +<programlisting> + x = plusInt a b +</programlisting> + If we inlined plusInt we might get an opportunity to use + update-in-place for the thunk 'x'.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <option>-funfolding-creation-threshold<n></option>: + <indexterm><primary><option>-funfolding-creation-threshold</option></primary></indexterm> + <indexterm><primary>inlining, controlling</primary></indexterm> + <indexterm><primary>unfolding, controlling</primary></indexterm> + </term> + <listitem> + <para>(Default: 45) Governs the maximum size that GHC will + allow a function unfolding to be. (An unfolding has a + “size” that reflects the cost in terms of + “code bloat” of expanding that unfolding at + at a call site. A bigger function would be assigned a + bigger cost.) </para> + + <para> Consequences: (a) nothing larger than this will be + inlined (unless it has an INLINE pragma); (b) nothing + larger than this will be spewed into an interface + file. </para> + + + <para> Increasing this figure is more likely to result in longer + compile times than faster code. The next option is more + useful:</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-funfolding-use-threshold<n></option>:</term> + <listitem> + <indexterm><primary><option>-funfolding-use-threshold</option></primary></indexterm> + <indexterm><primary>inlining, controlling</primary></indexterm> + <indexterm><primary>unfolding, controlling</primary></indexterm> + + <para>(Default: 8) This is the magic cut-off figure for + unfolding: below this size, a function definition will be + unfolded at the call-site, any bigger and it won't. The + size computed for a function depends on two things: the + actual size of the expression minus any discounts that + apply (see <option>-funfolding-con-discount</option>).</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + </sect1> + + &phases; + + <sect1 id="sec-using-concurrent"> + <title>Using Concurrent Haskell</title> + <indexterm><primary>Concurrent Haskell</primary><secondary>using</secondary></indexterm> + + <para>GHC supports Concurrent Haskell by default, without requiring a + special option or libraries compiled in a certain way. To get access to + the support libraries for Concurrent Haskell, just import + <ulink + url="../libraries/base/Control-Concurrent.html"><literal>Control.Concurrent</literal></ulink>. More information on Concurrent Haskell is provided in the documentation for that module.</para> + + <para>The following RTS option(s) affect the behaviour of Concurrent + Haskell programs:<indexterm><primary>RTS options, concurrent</primary></indexterm></para> + + <variablelist> + <varlistentry> + <term><option>-C<replaceable>s</replaceable></option></term> + <listitem> + <para><indexterm><primary><option>-C<replaceable>s</replaceable></option></primary><secondary>RTS option</secondary></indexterm> + Sets the context switch interval to <replaceable>s</replaceable> + seconds. A context switch will occur at the next heap block + allocation after the timer expires (a heap block allocation occurs + every 4k of allocation). With <option>-C0</option> or + <option>-C</option>, context switches will occur as often as + possible (at every heap block allocation). By default, context + switches occur every 20ms. Note that GHC's internal timer ticks + every 20ms, and the context switch timer is always a multiple of + this timer, so 20ms is the maximum granularity available for timed + context switches.</para> + </listitem> + </varlistentry> + </variablelist> + </sect1> + +<sect1 id="sec-using-parallel"> +<title>Using parallel Haskell</title> + +<para> +<indexterm><primary>Parallel Haskell</primary><secondary>using</secondary></indexterm> +[NOTE: GHC does not support Parallel Haskell by default, you need to + obtain a special version of GHC from the <ulink + url="http://www.cee.hw.ac.uk/~dsg/gph/">GPH</ulink> site. Also, +you won't be able to execute parallel Haskell programs unless PVM3 +(parallel Virtual Machine, version 3) is installed at your site.] +</para> + +<para> +To compile a Haskell program for parallel execution under PVM, use the +<option>-parallel</option> option,<indexterm><primary>-parallel +option</primary></indexterm> both when compiling <emphasis>and +linking</emphasis>. You will probably want to <literal>import +Control.Parallel</literal> into your Haskell modules. +</para> + +<para> +To run your parallel program, once PVM is going, just invoke it +“as normal”. The main extra RTS option is +<option>-qp<n></option>, to say how many PVM +“processors” your program to run on. (For more details of +all relevant RTS options, please see <xref +linkend="parallel-rts-opts"/>.) +</para> + +<para> +In truth, running parallel Haskell programs and getting information +out of them (e.g., parallelism profiles) is a battle with the vagaries of +PVM, detailed in the following sections. +</para> + +<sect2 id="pvm-dummies"> +<title>Dummy's guide to using PVM</title> + +<para> +<indexterm><primary>PVM, how to use</primary></indexterm> +<indexterm><primary>parallel Haskell—PVM use</primary></indexterm> +Before you can run a parallel program under PVM, you must set the +required environment variables (PVM's idea, not ours); something like, +probably in your <filename>.cshrc</filename> or equivalent: + +<programlisting> +setenv PVM_ROOT /wherever/you/put/it +setenv PVM_ARCH `$PVM_ROOT/lib/pvmgetarch` +setenv PVM_DPATH $PVM_ROOT/lib/pvmd +</programlisting> + +</para> + +<para> +Creating and/or controlling your “parallel machine” is a purely-PVM +business; nothing specific to parallel Haskell. The following paragraphs +describe how to configure your parallel machine interactively. +</para> + +<para> +If you use parallel Haskell regularly on the same machine configuration it +is a good idea to maintain a file with all machine names and to make the +environment variable PVM_HOST_FILE point to this file. Then you can avoid +the interactive operations described below by just saying +</para> + +<programlisting> +pvm $PVM_HOST_FILE +</programlisting> + +<para> +You use the <command>pvm</command><indexterm><primary>pvm command</primary></indexterm> command to start PVM on your +machine. You can then do various things to control/monitor your +“parallel machine;” the most useful being: +</para> + +<para> +<informaltable> +<tgroup cols="2"> +<colspec align="left"/> +<tbody> + +<row> +<entry><keycombo><keycap>Control</keycap><keycap>D</keycap></keycombo></entry> +<entry>exit <command>pvm</command>, leaving it running</entry> +</row> + +<row> +<entry><command>halt</command></entry> +<entry>kill off this “parallel machine” & exit</entry> +</row> + +<row> +<entry><command>add <host></command></entry> +<entry>add <command><host></command> as a processor</entry> +</row> + +<row> +<entry><command>delete <host></command></entry> +<entry>delete <command><host></command></entry> +</row> + +<row> +<entry><command>reset</command></entry> +<entry>kill what's going, but leave PVM up</entry> +</row> + +<row> +<entry><command>conf</command></entry> +<entry>list the current configuration</entry> +</row> + +<row> +<entry><command>ps</command></entry> +<entry>report processes' status</entry> +</row> + +<row> +<entry><command>pstat <pid></command></entry> +<entry>status of a particular process</entry> +</row> + +</tbody> +</tgroup> +</informaltable> +</para> + +<para> +The PVM documentation can tell you much, much more about <command>pvm</command>! +</para> + +</sect2> + +<sect2 id="par-profiles"> +<title>parallelism profiles</title> + +<para> +<indexterm><primary>parallelism profiles</primary></indexterm> +<indexterm><primary>profiles, parallelism</primary></indexterm> +<indexterm><primary>visualisation tools</primary></indexterm> +</para> + +<para> +With parallel Haskell programs, we usually don't care about the +results—only with “how parallel” it was! We want pretty pictures. +</para> + +<para> +parallelism profiles (à la <command>hbcpp</command>) can be generated with the +<option>-qP</option><indexterm><primary>-qP RTS option</primary></indexterm> RTS option. The +per-processor profiling info is dumped into files named +<filename><full-path><program>.gr</filename>. These are then munged into a PostScript picture, +which you can then display. For example, to run your program +<filename>a.out</filename> on 8 processors, then view the parallelism profile, do: +</para> + +<para> + +<screen> +<prompt>$</prompt> ./a.out +RTS -qP -qp8 +<prompt>$</prompt> grs2gr *.???.gr > temp.gr # combine the 8 .gr files into one +<prompt>$</prompt> gr2ps -O temp.gr # cvt to .ps; output in temp.ps +<prompt>$</prompt> ghostview -seascape temp.ps # look at it! +</screen> + +</para> + +<para> +The scripts for processing the parallelism profiles are distributed +in <filename>ghc/utils/parallel/</filename>. +</para> + +</sect2> + +<sect2> +<title>Other useful info about running parallel programs</title> + +<para> +The “garbage-collection statistics” RTS options can be useful for +seeing what parallel programs are doing. If you do either +<option>+RTS -Sstderr</option><indexterm><primary>-Sstderr RTS option</primary></indexterm> or <option>+RTS -sstderr</option>, then +you'll get mutator, garbage-collection, etc., times on standard +error. The standard error of all PE's other than the `main thread' +appears in <filename>/tmp/pvml.nnn</filename>, courtesy of PVM. +</para> + +<para> +Whether doing <option>+RTS -Sstderr</option> or not, a handy way to watch +what's happening overall is: <command>tail -f /tmp/pvml.nnn</command>. +</para> + +</sect2> + +<sect2 id="parallel-rts-opts"> +<title>RTS options for Parallel Haskell +</title> + +<para> +<indexterm><primary>RTS options, parallel</primary></indexterm> +<indexterm><primary>parallel Haskell—RTS options</primary></indexterm> +</para> + +<para> +Besides the usual runtime system (RTS) options +(<xref linkend="runtime-control"/>), there are a few options particularly +for parallel execution. +</para> + +<para> +<variablelist> + +<varlistentry> +<term><option>-qp<N></option>:</term> +<listitem> +<para> +<indexterm><primary>-qp<N> RTS option</primary></indexterm> +(paraLLEL ONLY) Use <literal><N></literal> PVM processors to run this program; +the default is 2. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-C[<s>]</option>:</term> +<listitem> +<para> +<indexterm><primary>-C<s> RTS option</primary></indexterm> Sets +the context switch interval to <literal><s></literal> seconds. +A context switch will occur at the next heap block allocation after +the timer expires (a heap block allocation occurs every 4k of +allocation). With <option>-C0</option> or <option>-C</option>, +context switches will occur as often as possible (at every heap block +allocation). By default, context switches occur every 20ms. Note that GHC's internal timer ticks every 20ms, and +the context switch timer is always a multiple of this timer, so 20ms +is the maximum granularity available for timed context switches. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-q[v]</option>:</term> +<listitem> +<para> +<indexterm><primary>-q RTS option</primary></indexterm> +(paraLLEL ONLY) Produce a quasi-parallel profile of thread activity, +in the file <filename><program>.qp</filename>. In the style of <command>hbcpp</command>, this profile +records the movement of threads between the green (runnable) and red +(blocked) queues. If you specify the verbose suboption (<option>-qv</option>), the +green queue is split into green (for the currently running thread +only) and amber (for other runnable threads). We do not recommend +that you use the verbose suboption if you are planning to use the +<command>hbcpp</command> profiling tools or if you are context switching at every heap +check (with <option>-C</option>). +--> +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-qt<num></option>:</term> +<listitem> +<para> +<indexterm><primary>-qt<num> RTS option</primary></indexterm> +(paraLLEL ONLY) Limit the thread pool size, i.e. the number of +threads per processor to <literal><num></literal>. The default is +32. Each thread requires slightly over 1K <emphasis>words</emphasis> in +the heap for thread state and stack objects. (For 32-bit machines, this +translates to 4K bytes, and for 64-bit machines, 8K bytes.) +</para> +</listitem> +</varlistentry> +<!-- no more -HWL +<varlistentry> +<term><option>-d</option>:</term> +<listitem> +<para> +<indexterm><primary>-d RTS option (parallel)</primary></indexterm> +(paraLLEL ONLY) Turn on debugging. It pops up one xterm (or GDB, or +something…) per PVM processor. We use the standard <command>debugger</command> +script that comes with PVM3, but we sometimes meddle with the +<command>debugger2</command> script. We include ours in the GHC distribution, +in <filename>ghc/utils/pvm/</filename>. +</para> +</listitem> +</varlistentry> +--> +<varlistentry> +<term><option>-qe<num></option>:</term> +<listitem> +<para> +<indexterm><primary>-qe<num> RTS option +(parallel)</primary></indexterm> (paraLLEL ONLY) Limit the spark pool size +i.e. the number of pending sparks per processor to +<literal><num></literal>. The default is 100. A larger number may be +appropriate if your program generates large amounts of parallelism +initially. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-qQ<num></option>:</term> +<listitem> +<para> +<indexterm><primary>-qQ<num> RTS option (parallel)</primary></indexterm> +(paraLLEL ONLY) Set the size of packets transmitted between processors +to <literal><num></literal>. The default is 1024 words. A larger number may be +appropriate if your machine has a high communication cost relative to +computation speed. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-qh<num></option>:</term> +<listitem> +<para> +<indexterm><primary>-qh<num> RTS option (parallel)</primary></indexterm> +(paraLLEL ONLY) Select a packing scheme. Set the number of non-root thunks to pack in one packet to +<num>-1 (0 means infinity). By default GUM uses full-subgraph +packing, i.e. the entire subgraph with the requested closure as root is +transmitted (provided it fits into one packet). Choosing a smaller value +reduces the amount of pre-fetching of work done in GUM. This can be +advantageous for improving data locality but it can also worsen the balance +of the load in the system. +</para> +</listitem> +</varlistentry> +<varlistentry> +<term><option>-qg<num></option>:</term> +<listitem> +<para> +<indexterm><primary>-qg<num> RTS option +(parallel)</primary></indexterm> (paraLLEL ONLY) Select a globalisation +scheme. This option affects the +generation of global addresses when transferring data. Global addresses are +globally unique identifiers required to maintain sharing in the distributed +graph structure. Currently this is a binary option. With <num>=0 full globalisation is used +(default). This means a global address is generated for every closure that +is transmitted. With <num>=1 a thunk-only globalisation scheme is +used, which generated global address only for thunks. The latter case may +lose sharing of data but has a reduced overhead in packing graph structures +and maintaining internal tables of global addresses. +</para> +</listitem> +</varlistentry> +</variablelist> +</para> + +</sect2> + +</sect1> + + <sect1 id="options-platform"> + <title>Platform-specific Flags</title> + + <indexterm><primary>-m* options</primary></indexterm> + <indexterm><primary>platform-specific options</primary></indexterm> + <indexterm><primary>machine-specific options</primary></indexterm> + + <para>Some flags only make sense for particular target + platforms.</para> + + <variablelist> + + <varlistentry> + <term><option>-mv8</option>:</term> + <listitem> + <para>(SPARC machines)<indexterm><primary>-mv8 option (SPARC + only)</primary></indexterm> Means to pass the like-named + option to GCC; it says to use the Version 8 SPARC + instructions, notably integer multiply and divide. The + similar <option>-m*</option> GCC options for SPARC also + work, actually.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><option>-monly-[32]-regs</option>:</term> + <listitem> + <para>(iX86 machines)<indexterm><primary>-monly-N-regs + option (iX86 only)</primary></indexterm> GHC tries to + “steal” four registers from GCC, for performance + reasons; it almost always works. However, when GCC is + compiling some modules with four stolen registers, it will + crash, probably saying: + +<screen> +Foo.hc:533: fixed or forbidden register was spilled. +This may be due to a compiler bug or to impossible asm +statements or clauses. +</screen> + + Just give some registers back with + <option>-monly-N-regs</option>. Try `3' first, then `2'. + If `2' doesn't work, please report the bug to us.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect1> + +&runtime; + +<sect1 id="ext-core"> + <title>Generating and compiling External Core Files</title> + + <indexterm><primary>intermediate code generation</primary></indexterm> + + <para>GHC can dump its optimized intermediate code (said to be in “Core” format) + to a file as a side-effect of compilation. Core files, which are given the suffix + <filename>.hcr</filename>, can be read and processed by non-GHC back-end + tools. The Core format is formally described in <ulink url="http://www.haskell.org/ghc/docs/papers/core.ps.gz"> + <citetitle>An External Representation for the GHC Core Language</citetitle></ulink>, + and sample tools (in Haskell) + for manipulating Core files are available in the GHC source distribution + directory <literal>/fptools/ghc/utils/ext-core</literal>. + Note that the format of <literal>.hcr</literal> + files is <emphasis>different</emphasis> (though similar) to the Core output format generated + for debugging purposes (<xref linkend="options-debugging"/>).</para> + + <para>The Core format natively supports notes which you can add to + your source code using the <literal>CORE</literal> pragma (see <xref + linkend="pragmas"/>).</para> + + <variablelist> + + <varlistentry> + <term> + <option>-fext-core</option> + <indexterm><primary><option>-fext-core</option></primary></indexterm> + </term> + <listitem> + <para>Generate <literal>.hcr</literal> files.</para> + </listitem> + </varlistentry> + + </variablelist> + +<para>GHC can also read in External Core files as source; just give the <literal>.hcr</literal> file on +the command line, instead of the <literal>.hs</literal> or <literal>.lhs</literal> Haskell source. +A current infelicity is that you need to give the <literal>-fglasgow-exts</literal> flag too, because +ordinary Haskell 98, when translated to External Core, uses things like rank-2 types.</para> +</sect1> + +&debug; +&flags; + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/utils.xml b/docs/users_guide/utils.xml new file mode 100644 index 0000000000..6c82f6b38a --- /dev/null +++ b/docs/users_guide/utils.xml @@ -0,0 +1,564 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="utils"> + <title>Other Haskell utility programs</title> + <indexterm><primary>utilities, Haskell</primary></indexterm> + + <para>This section describes other program(s) which we distribute, + that help with the Great Haskell Programming Task.</para> + +<!-- comment: hasktags documentation loosely based on that for hstags --> + +<sect1 id ="hasktags"> + <title>Ctags and Etags for Haskell: <command>hasktags</command></title> + <indexterm><primary><command>hasktags</command></primary></indexterm> + <indexterm><primary>CTAGS for Haskell</primary></indexterm> + + <para><command>hasktags</command> is a very simple Haskell program that produces ctags "tags" and etags "TAGS" files for Haskell programs.</para> + + <para>When loaded into an editor such an NEdit, Vim, or Emacs, this allows one to easily navigate around a multi-file program, finding definitions of functions, types, and constructors.</para> + + <para>Invocation Syntax:</para> + +<screen> +hasktags files +</screen> + +<para>This will read all the files listed in <option>files</option> and produce a ctags "tags" file and an etags "TAGS" file in the current directory.</para> + + <para>Example usage</para> + +<screen> +find -name \*.\*hs | xargs hasktags +</screen> + +<para>This will find all haskell source files in the current directory and below, and create tags files indexing them in the current directory.</para> + + <para><command>hasktags</command> is a simple program that uses simple + parsing rules to find definitions of functions, constructors, and types. It isn't guaranteed to find everything, and will sometimes create false index entries, but it usually gets the job done fairly well. In particular, at present, functions are only indexed if a type signature is given for them.</para> + + <para>Before hasktags, there used to be <command>fptags</command> and <command>hstags</command>, which did essentially the same job, however neither of these seem to be maintained any more.</para> + +<sect2> +<title>Using tags with your editor</title> + +<para>With NEdit, load the "tags" file using "File/Load Tags File". Use "Ctrl-D" to search for a tag.</para> + +<para>With XEmacs, load the "TAGS" file using "visit-tags-table". Use "M-." to search for a tag.</para> + + +</sect2> + +</sect1> + +<!-- comment: hstags doesn't work anymore + + <sect1 id="hstags"> + <title>Emacs `TAGS' for Haskell: <command>hstags</command></title> + <indexterm><primary><command>hstags</command></primary></indexterm> + <indexterm><primary>TAGS for Haskell</primary></indexterm> + + <para>`Tags' is a facility for indexing the definitions of + programming-language things in a multi-file program, and then + using that index to jump around among these definitions.</para> + + <para>Rather than scratch your head, saying “Now where did + we define `foo'?”, you just do (in Emacs) <Literal>M-. foo + RET</Literal>, and You're There! Some people go wild over this + stuff…</para> + + <para>GHC comes with a program <command>hstags</command>, which + build Emacs-able TAGS files. The invocation syntax is:</para> + +<screen> +hstags [GHC-options] file [files...] +</screen> + + <para>The best thing is just to feed it your GHC command-line + flags. A good Makefile entry might be:</para> + +<programlisting> +tags: + $(RM) TAGS + hstags $(GHC_FLAGS) *.lhs +</programlisting> + + <para>The only flags of its own are: <Option>-v</Option> to be + verbose; <Option>-a</Option> to <Emphasis>APPEND</Emphasis> to the + TAGS file, rather than write to it.</para> + + <para>Shortcomings: (1) Instance declarations don't get into + the TAGS file (but the definitions inside them do); as instances + aren't named, this is probably just as well. + (2) Data-constructor definitions don't get in. Go for the + corresponding type constructor instead.</para> + + <para>Actually, GHC also comes with <command>etags</command> + [for C], and <command>perltags</command> [for You + Know What]. And—I cannot tell a lie—there is + Denis Howe's <command>fptags</command> [for Haskell, + etc.] in the <Filename>ghc/CONTRIB</Filename> + section…)</para> + + </sect1> +--> + + <sect1 id="happy"> + <title>“Yacc for Haskell”: <command>happy</command></title> + + <indexterm><primary>Happy</primary></indexterm> + <indexterm><primary>Yacc for Haskell</primary></indexterm> + <indexterm><primary>parser generator for Haskell</primary></indexterm> + + <para>Andy Gill and Simon Marlow have written a parser-generator + for Haskell, called + <command>happy</command>.<indexterm><primary>happy parser + generator</primary></indexterm> <command>Happy</command> is to + Haskell what <command>Yacc</command> is to C.</para> + + <para>You can get <command>happy</command> from <ulink + url="http://www.haskell.org/happy/">the Happy + Homepage</ulink>.</para> + + <para><command>Happy</command> is at its shining best when + compiled by GHC.</para> + + </sect1> + +<!-- we don't distribute this anymore + <sect1 id="pphs"> + <title>Pretty-printing Haskell: <command>pphs</command></title> + <indexterm><primary>pphs</primary></indexterm> + <indexterm><primary>pretty-printing Haskell code</primary></indexterm> + + <para>Andrew Preece has written + <command>pphs</command>,<indexterm><primary>pphs</primary></indexterm><indexterm><primary>pretty-printing + Haskell</primary></indexterm> a utility to pretty-print Haskell + code in LaTeX documents. Keywords in bolds, variables in + italics—that sort of thing. It is good at lining up program + clauses and equals signs, things that are very tiresome to do by + hand.</para> + + <para>The code is distributed with GHC in + <Filename>ghc/CONTRIB/pphs</Filename>.</para> + </sect1> +--> + + <sect1 id="hsc2hs"> + <title>Writing Haskell interfaces to C code: + <command>hsc2hs</command></title> + <indexterm><primary><command>hsc2hs</command></primary> + </indexterm> + + <para>The <command>hsc2hs</command> command can be used to automate + some parts of the process of writing Haskell bindings to C code. + It reads an almost-Haskell source with embedded special + constructs, and outputs a real Haskell file with these constructs + processed, based on information taken from some C headers. The + extra constructs deal with accessing C data from Haskell.</para> + + <para>It may also output a C file which contains additional C + functions to be linked into the program, together with a C header + that gets included into the C code to which the Haskell module + will be compiled (when compiled via C) and into the C file. These + two files are created when the <literal>#def</literal> construct + is used (see below).</para> + + <para>Actually <command>hsc2hs</command> does not output the Haskell + file directly. It creates a C program that includes the headers, + gets automatically compiled and run. That program outputs the + Haskell code.</para> + + <para>In the following, “Haskell file” is the main + output (usually a <literal>.hs</literal> file), “compiled + Haskell file” is the Haskell file after + <command>ghc</command> has compiled it to C (i.e. a + <literal>.hc</literal> file), “C program” is the + program that outputs the Haskell file, “C file” is the + optionally generated C file, and “C header” is its + header file.</para> + + <sect2> + <title>command line syntax</title> + + <para><command>hsc2hs</command> takes input files as arguments, + and flags that modify its behavior:</para> + + <variablelist> + <varlistentry> + <term><literal>-o FILE</literal> or + <literal>––output=FILE</literal></term> + <listitem> + <para>Name of the Haskell file.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-t FILE</literal> or + <literal>––template=FILE</literal></term> + <listitem> + <para>The template file (see below).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-c PROG</literal> or + <literal>––cc=PROG</literal></term> + <listitem> + <para>The C compiler to use (default: + <command>ghc</command>)</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-l PROG</literal> or + <literal>––ld=PROG</literal></term> + <listitem> + <para>The linker to use (default: + <command>gcc</command>).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-C FLAG</literal> or + <literal>––cflag=FLAG</literal></term> + <listitem> + <para>An extra flag to pass to the C compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-I DIR</literal></term> + <listitem> + <para>Passed to the C compiler.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-L FLAG</literal> or + <literal>––lflag=FLAG</literal></term> + <listitem> + <para>An extra flag to pass to the linker.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-i FILE</literal> or + <literal>––include=FILE</literal></term> + <listitem> + <para>As if the appropriate <literal>#include</literal> + directive was placed in the source.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-D NAME[=VALUE]</literal> or + <literal>––define=NAME[=VALUE]</literal></term> + <listitem> + <para>As if the appropriate <literal>#define</literal> + directive was placed in the source.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>––no-compile</literal></term> + <listitem> + <para>Stop after writing out the intermediate C program to disk. + The file name for the intermediate C program is the input file name + with <literal>.hsc</literal> replaced with <literal>_hsc_make.c</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-?</literal> or <literal>––help</literal></term> + <listitem> + <para>Display a summary of the available flags and exit successfully.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>-V</literal> or <literal>––version</literal></term> + <listitem> + <para>Output version information and exit successfully.</para> + </listitem> + </varlistentry> + </variablelist> + + <para>The input file should end with .hsc (it should be plain + Haskell source only; literate Haskell is not supported at the + moment). Output files by default get names with the + <literal>.hsc</literal> suffix replaced:</para> + + <informaltable> + <tgroup cols="2"> + <tbody> + <row> + <entry><literal>.hs</literal></entry> + <entry>Haskell file</entry> + </row> + <row> + <entry><literal>_hsc.h</literal></entry> + <entry>C header</entry> + </row> + <row> + <entry><literal>_hsc.c</literal></entry> + <entry>C file</entry> + </row> + </tbody> + </tgroup> + </informaltable> + + <para>The C program is compiled using the Haskell compiler. This + provides the include path to <filename>HsFFI.h</filename> which + is automatically included into the C program.</para> + + </sect2> + <sect2><title>Input syntax</title> + + <para>All special processing is triggered by + the <literal>#</literal> operator. To output + a literal <literal>#</literal>, write it twice: + <literal>##</literal>. Inside string literals and comments + <literal>#</literal> characters are not processed.</para> + + <para>A <literal>#</literal> is followed by optional + spaces and tabs, an alphanumeric keyword that describes + the kind of processing, and its arguments. Arguments look + like C expressions separated by commas (they are not + written inside parens). They extend up to the nearest + unmatched <literal>)</literal>, <literal>]</literal> or + <literal>}</literal>, or to the end of line if it occurs outside + any <literal>() [] {} '' "" /**/</literal> and is not preceded + by a backslash. Backslash-newline pairs are stripped.</para> + + <para>In addition <literal>#{stuff}</literal> is equivalent + to <literal>#stuff</literal> except that it's self-delimited + and thus needs not to be placed at the end of line or in some + brackets.</para> + + <para>Meanings of specific keywords:</para> + + <variablelist> + + <varlistentry> + <term><literal>#include <file.h></literal></term> + <term><literal>#include "file.h"</literal></term> + <listitem> + <para>The specified file gets included into the C program, + the compiled Haskell file, and the C header. + <literal><HsFFI.h></literal> is included + automatically.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#define name</literal></term> + <term><literal>#define name value</literal></term> + <term><literal>#undef name</literal></term> + <listitem> + <para>Similar to <literal>#include</literal>. Note that + <literal>#includes</literal> and + <literal>#defines</literal> may be put in the same file + twice so they should not assume otherwise.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#let name parameters = "definition"</literal></term> + <listitem> + <para>Defines a macro to be applied to the Haskell + source. Parameter names are comma-separated, not + inside parens. Such macro is invoked as other + <literal>#</literal>-constructs, starting with + <literal>#name</literal>. The definition will be + put in the C program inside parens as arguments of + <literal>printf</literal>. To refer to a parameter, + close the quote, put a parameter name and open the + quote again, to let C string literals concatenate. + Or use <literal>printf</literal>'s format directives. + Values of arguments must be given as strings, unless the + macro stringifies them itself using the C preprocessor's + <literal>#parameter</literal> syntax.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#def C_definition</literal></term> + <listitem> + <para>The definition (of a function, variable, struct or + typedef) is written to the C file, and its prototype or + extern declaration to the C header. Inline functions are + handled correctly. struct definitions and typedefs are + written to the C program too. The + <literal>inline</literal>, <literal>struct</literal> or + <literal>typedef</literal> keyword must come just after + <literal>def</literal>.</para> + + <note><para>A <literal>foreign import</literal> of a + C function may be inlined across a module boundary, + in which case you must arrange for the importing + module to <literal>#include</literal> the C header + file generated by <command>hsc2hs</command> (see + <xref linkend="glasgow-foreign-headers"/>). + For this reason we avoid using <literal>#def</literal> + in the libraries.</para></note> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#if condition</literal></term> + <term><literal>#ifdef name</literal></term> + <term><literal>#ifndef name</literal></term> + <term><literal>#elif condition</literal></term> + <term><literal>#else</literal></term> + <term><literal>#endif</literal></term> + <term><literal>#error message</literal></term> + <term><literal>#warning message</literal></term> + <listitem> + <para>Conditional compilation directives are passed + unmodified to the C program, C file, and C header. Putting + them in the C program means that appropriate parts of the + Haskell file will be skipped.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#const C_expression</literal></term> + <listitem> + <para>The expression must be convertible to + <literal>long</literal> or <literal>unsigned + long</literal>. Its value (literal or negated literal) + will be output.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#const_str C_expression</literal></term> + <listitem> + <para>The expression must be convertible to const char + pointer. Its value (string literal) will be output.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#type C_type</literal></term> + <listitem> + <para>A Haskell equivalent of the C numeric type will be + output. It will be one of + <literal>{Int,Word}{8,16,32,64}</literal>, + <literal>Float</literal>, <literal>Double</literal>, + <literal>LDouble</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#peek struct_type, field</literal></term> + <listitem> + <para>A function that peeks a field of a C struct will be + output. It will have the type + <literal>Storable b => Ptr a -> IO b</literal>. + + The intention is that <literal>#peek</literal> and + <literal>#poke</literal> can be used for implementing the + operations of class <literal>Storable</literal> for a + given C struct (see the + <literal>Foreign.Storable</literal> module in the library + documentation).</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#poke struct_type, field</literal></term> + <listitem> + <para>Similarly for poke. It will have the type + <literal>Storable b => Ptr a -> b -> IO ()</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#ptr struct_type, field</literal></term> + <listitem> + <para>Makes a pointer to a field struct. It will have the type + <literal>Ptr a -> Ptr b</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#offset struct_type, field</literal></term> + <listitem> + <para>Computes the offset, in bytes, of + <literal>field</literal> in + <literal>struct_type</literal>. It will have type + <literal>Int</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#size struct_type</literal></term> + <listitem> + <para>Computes the size, in bytes, of + <literal>struct_type</literal>. It will have type + <literal>Int</literal>.</para> + </listitem> + </varlistentry> + + <varlistentry> + <term><literal>#enum type, constructor, value, value, ...</literal></term> + <listitem> + <para>A shortcut for multiple definitions which use + <literal>#const</literal>. Each <literal>value</literal> + is a name of a C integer constant, e.g. enumeration value. + The name will be translated to Haskell by making each + letter following an underscore uppercase, making all the rest + lowercase, and removing underscores. You can supply a different + translation by writing <literal>hs_name = c_value</literal> + instead of a <literal>value</literal>, in which case + <literal>c_value</literal> may be an arbitrary expression. + The <literal>hs_name</literal> will be defined as having the + specified <literal>type</literal>. Its definition is the specified + <literal>constructor</literal> (which in fact may be an expression + or be empty) applied to the appropriate integer value. You can + have multiple <literal>#enum</literal> definitions with the same + <literal>type</literal>; this construct does not emit the type + definition itself.</para> + </listitem> + </varlistentry> + </variablelist> + + </sect2> + + <sect2> + <title>Custom constructs</title> + + <para><literal>#const</literal>, <literal>#type</literal>, + <literal>#peek</literal>, <literal>#poke</literal> and + <literal>#ptr</literal> are not hardwired into the + <command>hsc2hs</command>, but are defined in a C template that is + included in the C program: <filename>template-hsc.h</filename>. + Custom constructs and templates can be used too. Any + <literal>#</literal>-construct with unknown key is expected to + be handled by a C template.</para> + + <para>A C template should define a macro or function with name + prefixed by <literal>hsc_</literal> that handles the construct + by emitting the expansion to stdout. See + <filename>template-hsc.h</filename> for examples.</para> + + <para>Such macros can also be defined directly in the + source. They are useful for making a <literal>#let</literal>-like + macro whose expansion uses other <literal>#let</literal> macros. + Plain <literal>#let</literal> prepends <literal>hsc_</literal> + to the macro name and wraps the definition in a + <literal>printf</literal> call.</para> + + </sect2> + + </sect1> + +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/users_guide/win32-dlls.xml b/docs/users_guide/win32-dlls.xml new file mode 100644 index 0000000000..959f7ce1b6 --- /dev/null +++ b/docs/users_guide/win32-dlls.xml @@ -0,0 +1,493 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<chapter id="win32"> +<title>Running GHC on Win32 systems</title> + +<sect1> +<title> +Starting GHC on Win32 platforms</title> + +<para> +The installer that installs GHC on Win32 also sets up the file-suffix associations +for ".hs" and ".lhs" files so that double-clicking them starts <command>ghci</command>. +</para> +<para> +Be aware of that <command>ghc</command> and <command>ghci</command> do +require filenames containing spaces to be escaped using quotes: +<programlisting> + c:\ghc\bin\ghci "c:\\Program Files\\Haskell\\Project.hs" +</programlisting> +If the quotes are left off in the above command, <command>ghci</command> will +interpret the filename as two, "c:\\Program" and "Files\\Haskell\\Project.hs". +</para> + +<!-- not clear whether there are current editions of Win32 OSes that + doesn't do this by default. + +<para> Solution: don't use "Open With...", avoid spaces in file names, +or fiddle with the appropriate registry setting: +<programlisting> + HKEY_CLASSES_ROOT\Unknown\shell\openas\command +</programlisting> +Notice how the "%1" argument is quoted (or not). +</para> +<para> This problem doesn't occur when double-clicking. +</para> +--> + +</sect1> + +<sect1> +<title> +Interacting with the terminal</title> + +<para>By default GHC builds applications that open a console window when they start. +If you want to build a GUI-only application, with no console window, use the flag +<literal>-optl-mwindows</literal> in the link step. +</para> + +<para> <emphasis>Warning:</emphasis> Windows GUI-only programs have no + stdin, stdout or stderr so using the ordinary Haskell + input/output functions will cause your program to fail with an + IO exception, such as: +<screen> + Fail: <stdout>: hPutChar: failed (Bad file descriptor) +</screen> + However using Debug.Trace.trace is alright because it uses + Windows debugging output support rather than stderr.</para> + +<para>For some reason, Mingw ships with the <literal>readline</literal> library, +but not with the <literal>readline</literal> headers. As a result, GHC (like Hugs) does not +use <literal>readline</literal> for interactive input on Windows. +You can get a close simulation by using an emacs shell buffer! +</para> + +</sect1> + +<sect1> +<title> +Differences in library behaviour </title> + +<para> +Some of the standard Haskell libraries behave slightly differently on Windows. + +<itemizedlist> +<listitem> <para> +On Windows, the '<literal>^Z</literal>' character is interpreted as an +end-of-file character, so if you read a file containing this character +the file will appear to end just before it. To avoid this, +use <literal>IOExts.openFileEx</literal> to open a file in binary +(untranslated) mode or change an already opened file handle into +binary mode using <literal>IOExts.hSetBinaryMode</literal>. The +<literal>IOExts</literal> module is part of the +<literal>lang</literal> package. +</para> +</listitem> +</itemizedlist> +</para> +</sect1> + +<sect1> +<title> +Using GHC (and other GHC-compiled executables) with cygwin</title> + +<sect2> +<title>Background</title> <para>The cygwin tools aim to provide a +unix-style API on top of the windows libraries, to facilitate ports of +unix software to windows. To this end, they introduce a unix-style +directory hierarchy under some root directory (typically +<filename>/</filename> is <filename>C:\cygwin\</filename>). Moreover, +everything built against the cygwin API (including the cygwin tools +and programs compiled with cygwin's ghc) will see / as the root of +their file system, happily pretending to work in a typical unix +environment, and finding things like <filename>/bin</filename> and <filename>/usr/include</filename> without +ever explicitly bothering with their actual location on the windows +system (probably <filename>C:\cygwin\bin</filename> and <filename>C:\cygwin\usr\include</filename>). +</para> +</sect2> + +<sect2><title>The problem</title> +<para>GHC, by default, no longer depends on cygwin, but is a native +windows program. It is built using mingw, and it uses mingw's ghc +while compiling your Haskell sources (even if you call it from +cygwin's bash), but what matters here is that - just like any other +normal windows program - neither GHC nor the executables it produces +are aware of cygwin's pretended unix hierarchy. GHC will happily +accept either '/' or '\' as path separators, but it won't know where +to find <filename>/home/joe/Main.hs</filename> or <filename>/bin/bash</filename> +or the like. This causes all +kinds of fun when GHC is used from within cygwin's bash, or in +make-sessions running under cygwin. +</para> +</sect2> + +<sect2><title>Things to do</title> +<itemizedlist> +<listitem> +<para> Don't use absolute paths in make, configure & co if there is any chance + that those might be passed to GHC (or to GHC-compiled programs). Relative + paths are fine because cygwin tools are happy with them and GHC accepts + '/' as path-separator. And relative paths don't depend on where cygwin's + root directory is located, or on which partition or network drive your source + tree happens to reside, as long as you 'cd' there first. +</para></listitem> + +<listitem> +<para> If you have to use absolute paths (beware of the innocent-looking + <literal>ROOT=`pwd`</literal> in makefile hierarchies or configure scripts), cygwin provides + a tool called <command>cygpath</command> that can convert cygwin's unix-style paths to their + actual windows-style counterparts. Many cygwin tools actually accept + absolute windows-style paths (remember, though, that you either need + to escape '\' or convert '\' to '/'), so you should be fine just using those + everywhere. If you need to use tools that do some kind of path-mangling + that depends on unix-style paths (one fun example is trying to interpret ':' + as a separator in path lists..), you can still try to convert paths using + <command>cygpath</command> just before they are passed to GHC and friends. +</para></listitem> + +<listitem> +<para> If you don't have <command>cygpath</command>, you probably don't have cygwin and hence + no problems with it... unless you want to write one build process for several + platforms. Again, relative paths are your friend, but if you have to use + absolute paths, and don't want to use different tools on different platforms, + you can simply write a short Haskell program to print the current directory + (thanks to George Russell for this idea): compiled with GHC, this will give + you the view of the file system that GHC depends on (which will differ + depending on whether GHC is compiled with cygwin's gcc or mingw's + gcc or on a real unix system..) - that little program can also deal with + escaping '\' in paths. Apart from the banner and the startup time, + something like this would also do: +<programlisting> + $ echo "Directory.getCurrentDirectory >>= putStrLn . init . tail . show " | ghci +</programlisting> +</para></listitem> +</itemizedlist> +</sect2> +</sect1> + + +<sect1 id="win32-dlls"> +<title>Building and using Win32 DLLs +</title> + +<para> +<emphasis>Making Haskell libraries into DLLs doesn't work on Windows at the +moment; however, all the machinery is +still there. If you're interested, contact the GHC team. Note that +building an entire Haskell application as a single DLL is still supported: it's + just multi-DLL Haskell programs that don't work. The Windows + distribution of GHC contains static libraries only.</emphasis></para> + +<!-- +<para> +<indexterm><primary>Dynamic link libraries, Win32</primary></indexterm> +<indexterm><primary>DLLs, Win32</primary></indexterm> +On Win32 platforms, the compiler is capable of both producing and using +dynamic link libraries (DLLs) containing ghc-compiled code. This +section shows you how to make use of this facility. +</para> + +<para> +Until recently, <command>strip</command> didn't work reliably on DLLs, so you +should test your version with care, or make sure you have the latest +binutils. Unfortunately, we don't know exactly which version of binutils +cured the problem (it was supposedly fixed some years ago). +</para> + + +<sect2 id="win32-dlls-link"> +<title>Linking with DLLs</title> + +<para> +The default on Win32 platforms is to link applications in such a way +that the executables will use the Prelude and system libraries DLLs, +rather than contain (large chunks of) them. This is transparent at the +command-line, so +</para> + +<para> +<screen> +sh$ cat main.hs +module Main where +main = putStrLn "hello, world!" +sh$ ghc -o main main.hs +ghc: module version changed to 1; reason: no old .hi file +sh$ strip main.exe +sh$ ls -l main.exe +-rwxr-xr-x 1 544 everyone 4608 May 3 17:11 main.exe* +sh$ ./main +hello, world! +sh$ +</screen> +</para> + +<para> +will give you a binary as before, but the <filename>main.exe</filename> +generated will use the Prelude and RTS DLLs instead of linking them in +statically. +</para> + +<para> +4K for a <literal>"hello, world"</literal> application—not bad, huh? :-) +</para> + +</sect2> + +<sect2 id="win32-dlls-linking-static"> +<title>Not linking with DLLs +<indexterm><primary>-static option (Win32)</primary></indexterm></title> + +<para> +If you want to build an executable that doesn't depend on any +ghc-compiled DLLs, use the <option>-static</option> option to link in +the code statically. +</para> + +<para> +Notice that you cannot mix code that has been compiled with +<option>-static</option> and not, so you have to use the <option>-static</option> +option on all the Haskell modules that make up your application. +</para> + +</sect2> +--> + +<sect2 id="win32-dlls-create"> +<title>Creating a DLL</title> + +<para> +<indexterm><primary>Creating a Win32 DLL</primary></indexterm> +<indexterm><primary>––mk-dll</primary></indexterm> +Sealing up your Haskell library inside a DLL is straightforward; +compile up the object files that make up the library, and then build +the DLL by issuing a command of the form: +</para> + +<para> +<screen> +ghc ––mk-dll -o foo.dll bar.o baz.o wibble.a -lfooble +</screen> +</para> + +<para> +By feeding the ghc compiler driver the option <option>––mk-dll</option>, it +will build a DLL rather than produce an executable. The DLL will +consist of all the object files and archives given on the command +line. +</para> + +<!-- +<para> +To create a `static' DLL, i.e. one that does not depend on the GHC DLLs, +use the <option>-static</option> when compiling up your Haskell code and +building the DLL. +</para> +--> + +<para> +A couple of things to notice: +</para> + +<para> + +<itemizedlist> +<!-- +<listitem> +<para> +Since DLLs correspond to packages (see <xref linkend="packages"/>) you need +to use <option>-package-name dll-name</option> when compiling modules that +belong to a DLL if you're going to call them from Haskell. Otherwise, Haskell +code that calls entry points in that DLL will do so incorrectly, and crash. +For similar reasons, you can only compile a single module tree into a DLL, +as <function>startupHaskell</function> needs to be able to call its +initialisation function, and only takes one such argument (see <xref +linkend="win32-dlls-foreign"/>). Hence the modules +you compile into a DLL must have a common root. +</para> +</listitem> +--> + +<listitem> +<para> +By default, the entry points of all the object files will be exported from +the DLL when using <option>––mk-dll</option>. Should you want to constrain +this, you can specify the <emphasis>module definition file</emphasis> to use +on the command line as follows: + +<screen> +ghc ––mk-dll -o .... -optdll––def -optdllMyDef.def +</screen> + +See Microsoft documentation for details, but a module definition file +simply lists what entry points you want to export. Here's one that's +suitable when building a Haskell COM server DLL: + +<programlisting> +EXPORTS + DllCanUnloadNow = DllCanUnloadNow@0 + DllGetClassObject = DllGetClassObject@12 + DllRegisterServer = DllRegisterServer@0 + DllUnregisterServer = DllUnregisterServer@0 +</programlisting> +</para> +</listitem> + +<listitem> +<para> +In addition to creating a DLL, the <option>––mk-dll</option> option also +creates an import library. The import library name is derived from the +name of the DLL, as follows: + +<programlisting> +DLL: HScool.dll ==> import lib: libHScool_imp.a +</programlisting> + +The naming scheme may look a bit weird, but it has the purpose of allowing +the co-existence of import libraries with ordinary static libraries (e.g., +<filename>libHSfoo.a</filename> and +<filename>libHSfoo_imp.a</filename>. + +Additionally, when the compiler driver is linking in non-static mode, it +will rewrite occurrence of <option>-lHSfoo</option> on the command line to +<option>-lHSfoo_imp</option>. By doing this for you, switching from +non-static to static linking is simply a question of adding +<option>-static</option> to your command line. + +</para> +</listitem> +</itemizedlist> +</para> + +</sect2> + + +<sect2 id="win32-dlls-foreign"> +<title>Making DLLs to be called from other languages</title> + +<para> + +If you want to package up Haskell code to be called from other languages, +such as Visual Basic or C++, there are some extra things it is useful to +know. The dirty details are in the <emphasis>Foreign Function +Interface</emphasis> definition, but it can be tricky to work out how to +combine this with DLL building, so here's an example: + +</para> + +<itemizedlist> + +<listitem> +<para> +Use <literal>foreign export</literal> declarations to export the Haskell +functions you want to call from the outside. For example, + +<programlisting> +module Adder where + +adder :: Int -> Int -> IO Int –– gratuitous use of IO +adder x y = return (x+y) + +foreign export stdcall adder :: Int -> Int -> IO Int +</programlisting> +</para> +</listitem> + +<listitem> +<para> +Compile it up: + +<screen> +ghc -c adder.hs -fglasgow-exts +</screen> + +This will produce two files, adder.o and adder_stub.o +</para> +</listitem> + +<listitem> +<para> +compile up a <function>DllMain()</function> that starts up the Haskell +RTS-––a possible implementation is: + +<programlisting> +#include <windows.h> +#include <Rts.h> + +extern void__stginit_Adder(void); + +static char* args[] = { "ghcDll", NULL }; + /* N.B. argv arrays must end with NULL */ +BOOL +STDCALL +DllMain + ( HANDLE hModule + , DWORD reason + , void* reserved + ) +{ + if (reason == DLL_PROCESS_ATTACH) { + /* By now, the RTS DLL should have been hoisted in, but we need to start it up. */ + startupHaskell(1, args, __stginit_Adder); + return TRUE; + } + return TRUE; +} +</programlisting> + +Here, <literal>Adder</literal> is the name of the root module in the module +tree (as mentioned above, there must be a single root module, and hence a +single module tree in the DLL). + +Compile this up: + +<screen> +ghc -c dllMain.c +</screen> +</para> +</listitem> + +<listitem> +<para> +Construct the DLL: + +<screen> +ghc ––mk-dll -o adder.dll adder.o adder_stub.o dllMain.o +</screen> + +</para> +</listitem> + +<listitem> +<para> +Start using <function>adder</function> from VBA-––here's how I would +<constant>Declare</constant> it: + +<programlisting> +Private Declare Function adder Lib "adder.dll" Alias "adder@8" + (ByVal x As Long, ByVal y As Long) As Long +</programlisting> + +Since this Haskell DLL depends on a couple of the DLLs that come with GHC, +make sure that they are in scope/visible. +</para> + +<para> +Building statically linked DLLs is the same as in the previous section: it +suffices to add <option>-static</option> to the commands used to compile up +the Haskell source and build the DLL. +</para> + +</listitem> + +</itemizedlist> + +</sect2> + +</sect1> +</chapter> + +<!-- Emacs stuff: + ;;; Local Variables: *** + ;;; mode: xml *** + ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") *** + ;;; End: *** + --> diff --git a/docs/vh/Makefile b/docs/vh/Makefile new file mode 100644 index 0000000000..4410e4953d --- /dev/null +++ b/docs/vh/Makefile @@ -0,0 +1,7 @@ +TOP = ../.. +include $(TOP)/mk/boilerplate.mk + +XML_DOC = vh +INSTALL_XML_DOC = vh + +include $(TOP)/mk/target.mk diff --git a/docs/vh/vh.xml b/docs/vh/vh.xml new file mode 100644 index 0000000000..f7d636a71f --- /dev/null +++ b/docs/vh/vh.xml @@ -0,0 +1,319 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" + "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [ +]> + + <article id="visual-haskell"> + + <articleinfo> + + <title>Visual Haskell User's Guide</title> + <author> + <firstname>Simon</firstname> + <surname>Marlow</surname> + <email>simonmar@microsoft.com</email> + </author> + <author> + <firstname>Krasimir</firstname> + <surname>Angelov</surname> + <email>kr.angelov@gmail.com</email> + </author> + +<!-- + <abstract> + <para></para> + </abstract> +--> + + </articleinfo> + + <section id="sec-introduction"> + <title>Introduction</title> + + <para>Visual Haskell is a plugin for Microsoft's Visual Studio + development environment to support development of Haskell software. + Like the other Visual languages, Visual Haskell integrates with the + Visual Studio editor to provide interactive features to aid Haskell + development, and it enables the construction of projects consisting of + multiple Haskell modules.</para> + + <section id="sec-obtaining"> + <title>Installing Visual Haskell</title> + + <para>In order to use Visual Haskell, you need <ulink url="http://msdn.microsoft.com/vstudio/productinfo/">Visual Studio .NET + 2003</ulink>. Right now, this is the only supported version of Visual + Studio - unfortunately we haven't yet added support for the 2005 + Beta. The Express languages (Visual C++ Express etc.) also will not + work, because they don't have support for plugins.</para> + + <para>You don't need to install GHC separately: Visual Haskell + is bundled with a complete GHC distribution, and various other tools + (Happy, Alex, Haddock).</para> + + <para>The latest Visual Haskell installer can be obtained from + here:</para> + + <para><ulink + url="http://www.haskell.org/visualhaskell/"><literal>http://www.haskell.org/visualhaskell/</literal></ulink></para> + </section> + + <section id="release-notes"> + <title>Release Notes</title> + + <section> + <title>Version 0.0, first release</title> + + <para>This release is a technology preview, and should be considered + alpha quality. It works for us, but you are fairly likely to + encounter problems. If you're willing to try it out and report + bugs, we'd be grateful for the feedback.</para> + + <itemizedlist> + <listitem> + <para>This release of Visual Haskell is bundled with a + development snapshot of GHC, version 6.5 from around 14 + September 2005. This version of GHC is used to provide the + interactive editing features, and will be used to compile all + code inside Visual Haskell. It is possible that in future + releases we may be able to relax this tight coupling between + Visual Haskell and the bundled GHC.</para> + + <para>Please note that future releases of Visual + Haskell will update the compiler, and hence the + packages, and so may break your code. Also note that because + the bundled GHC is not a released version, it may have bugs and + quirks itself: please report them as usual to + <email>glasgow-haskell-bugs@haskell.org</email>.</para> + </listitem> + + <listitem> + <para>We're not making source code for the plugin generally + available at this time, due to licensing restrictions on the + Visual Studio APIs that the plugin uses (for more + information see <ulink + url="http://msdn.microsoft.com/vstudio/extend/">Visual Studio + Extensibility Center</ulink>). If you're interested in + contributing to Visual Haskell, please get in touch with the + authors.</para> + </listitem> + </itemizedlist> + </section> + </section> + + <section id="sec-bugs"> + <title>Getting support, reporting bugs</title> + <para>Please report bugs to + <email>glasgow-haskell-bugs@haskell.org</email> (subscribe <ulink url="http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs">here</ulink>), clearly indicating + that your bug report relates to Visual Haskell, and giving as much + information as possible so that we can reproduce the bug. Even if + you can't reproduce the bug reliably, it is still useful to report + what you've seen.</para> + + <para>For help and support, use the + <email>glasgow-haskell-users@haskell.org</email> (subscribe <ulink + url="http://www.haskell.org/mailman/listinfo/glasgow-haskell-users">here</ulink>) mailing list.</para> + </section> + + <section id="sec-license"> + <title>License</title> + + <blockquote> + <para>Copyright © Microsoft Corporation. All rights reserved.</para> + <para>Copyright © The University of Glasgow. All rights reserved.</para> + <para>Copyright © Krasimir Angelov. All rights reserved.</para> + + <para>Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met:</para> + + <itemizedlist> + <listitem> + <para>Redistributions of source code must retain the above + copyright notice, this list of conditions and the following + disclaimer.</para> + </listitem> + + <listitem> + <para>Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution.</para> + </listitem> + + <listitem> + <para>The names of the copyright holders may not be used to endorse + or promote products derived from this software without specific + prior written permission.</para> + </listitem> + </itemizedlist> + + <para>THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + POSSIBILITY OF SUCH DAMAGE.</para> + </blockquote> + </section> + + </section> + + <section id="sec-using"> + <title>Using Visual Haskell</title> + + <section> + <title>Overview of features</title> + + <para>The following features are provided in the Visual Studio editor + when editing Haskell code:</para> + + <itemizedlist> + <listitem> + <para>Automatic checking of code as you type, and visual indication + of parse errors, scoping errors and type errors.</para> + </listitem> + + <listitem> + <para>Quick info: hovering the mouse over an identifier pops up + an information box, including the type of the identifier.</para> + </listitem> + + <listitem> + <para>A drop-down bar at the top of the editing window lists the + top-level declarations in the module, and allows quick navigation + to a declaration.</para> + </listitem> + + <listitem> + <para>Name completion for identifiers in scope: press Ctrl+Space + after a partial identifier to see the completions.</para> + </listitem> + + <listitem> + <para>Go to declaration: right clicking on an identifier and + selecting "Go to declaration" will jump the cursor to the + declaration of the identifier. This works for locally-defined + identifiers and those defined in another module of the project; it + does not work for library functions currently.</para> + </listitem> + </itemizedlist> + </section> + + <para>The following features are provided by the project system for + constructing Haskell projects:</para> + + <itemizedlist> + <listitem> + <para>Multi-module Haskell projects are fully supported, based on the + <ulink url="http://www.haskell.org/cabal">Cabal</ulink> + infrastructure. A project in Visual Haskell <emphasis>is</emphasis> + a Cabal package, and vice-versa. A Visual Studio project can be + taken to a machine without Visual Haskell and built/installed as a + normal Cabal package, and an existing Cabal package can be edited + directly in Visual Haskell<footnote><para>This works as long as the + Cabal package is using Cabal's simple build system; Cabal + packages using their own build systems cannot be edited in Visual + Haskell.</para> + </footnote>.</para> + </listitem> + + <listitem> + <para>Editing of most of the package meta-data is supported through + the project property pages.</para> + </listitem> + + <listitem> + <para>The interactive editing features work across multiple modules in + a project. When one module is edited, changes are automatically + propagated to dependent modules, even if the edited module has not yet + been saved.</para> + </listitem> + + <listitem> + <para>Building is supported through the Cabal build system, and build + errors are communicated back to the editor and placed in the task + list. Use any of the Visual Studio build commands (e.g. Build + Project from the context menu on the project, or Ctrl-Shift-B to + build the whole solution).</para> + </listitem> + + </itemizedlist> + + <para>Additionally, Visual Haskell is bundled with a large collection of + documentation: the GHC manual, the hierarchical libraries reference, and + other material all of which can be browsed within Visual Studio + itself.</para> + + <section> + <title>Getting Started</title> + + <para>After installing Visual Haskell, start up Visual Studio as you + would normally, and observe that on the splash screen where it lists + the supported languages you should now see an icon for Visual + Haskell (if you don't see this, something has gone wrong... please let + us know).</para> + + <para>Firstly, take a look at the bundled documentation. Go to + Help->Contents, and you should see the “Visual Haskell Help + Collection”, which contains a large collection of GHC and + Haskell-related documentaiton, including this document.</para> + + <para>To start using Visual Haskell right away, create a new + project (File->New->Project...). Select one of the Haskell + project types (Console Application or Library Package), and hit Ok. + The project will be created for you, and an example module + added: <literal>Main.hs</literal> for an application, or + <literal>Module1.hs</literal> for a library.</para> + + <para>You can now start adding code to + <literal>Main.hs</literal>, or adding new modules. To add a new + module, right-click on the <literal>src</literal> directory, and + select Add->New Item. Visual Haskell supports hierarchical + modules too: you can add new folders using the same Add menu to + create new nodes in the hierarchy.</para> + + <para>If you have any errors in your code, they will be underlined with + a red squiggly line. Select the Tasks window (usually a tab near the + bottom of the Visual Studio window) to see the error messages, and + click on an error message to jump to it in the editor.</para> + + <para>To build the program, hit Ctrl-Shift-B, or select one of the + options from the Build menu.</para> + </section> + + <section> + <title>Editing Haskell code</title> + + <para>(ToDo: more detail here)</para> + + <para>Your module must be plain Haskell (<literal>.hs</literal>) for the interactive features to + fully work. If your module is pre-processed with CPP or Literate + Haskell, then Visual Haskell will only check the module when it is + saved; between saves the source will not be checked for errors and + the type information will not be updated. If the source file is + pre-processed with Happy or another pre-processor, then you may have + to build the project before the type information will be updated + (because the pre-processor is only run as part of the build + process). Pre-processed source files work fine in a multi-module + setting; you can have modules which depend on a pre-processed module + and full interactive checking will still be available in those + modules.</para> + + <para>Because Visual Haskell is using GHC as a backend for its + interactive editing features, it supports the full GHC language, + including all extensions.</para> + </section> + + <section> + <title>Using Projects</title> + <para>(ToDo: more detail here)</para> + </section> + + </section> + </article> |