adds instructions on coverage testingdsilvers/coverage-testing

1 files changed, 42 insertions, 2 deletions

diff --git a/TESTING b/TESTING
index c50744c..f9a9b5b 100644
--- a/TESTING
+++ b/TESTING
@@ -104,7 +104,7 @@ The fundamental parts of the `testing/` directory are
 * `admin-patches/`: This directory contains patches to the gitano-admin.git
   which can be applied during tests.  If you change the skeleton ruleset (or
   otherwise) for Gitano, you may need to refresh these patches.
-			 
+
 * `library.yarn`: This contains the implementations for the test suite yarn
   statements.
 
@@ -113,7 +113,7 @@ The fundamental parts of the `testing/` directory are
 * `02_*.yarn`: These tests are focussed around validating the commands
    specifically.  The test might not use only the command in question, but that
    is their goal.
-						 
+
 * `03_*.yarn` These tests are more integration tests which look to simulate
   more complex user use-cases.
 
@@ -274,3 +274,43 @@ This is not compatible with running yarns in parallel
 and depends on the test suite coverage to produce sufficient results.
 
 Making use of this also requires the penlight library to be installed.
+
+Obtaining coverage statistics
+=============================
+
+When writing tests, one often wishes to know that ones test has improved the
+coverage of the codebase.  To that end, the build system for Gitano has support
+for using luacov to generate and report coverage statistics.
+
+In order to use this, you must first ensure that you have the submodules
+present:
+
+    $ git submodule init
+    $ git submodule update
+
+Next, to gather coverage data run:
+
+    $ make COVERAGE=yes test
+
+You can specify `SCENARIO=blahfoo`, or `YARN_ARGS=...` etc, on the command line
+if you wish.  If you do not specify `COVERAGE=yes` then the tests will be run
+without augmenting the coverage data.
+
+Coverage data is aggregated across multiple runs.  This allows you to run only
+specific scenarios, and still see aggregated coverage across the codebase.
+
+To generate the coverage report, run:
+
+    $ make coverage-report
+
+This will merge all the aggregated coverage data into a single file, and then
+will run the luacov tool to generate `luacov.report.out` which contains the
+report.  Finally it will display a summary of the coverage for the relevant
+parts of the codebase.  Some of the codebase, as well as anything in
+`/usr/share` will be ignored by the report generator.
+
+If you find that something which should be included is not, or something which
+should be excluded is not, then adjust the `.luacov` file which is the
+configuration for the coverage tool.  This is also loaded by the internal
+in-tree-only module `lib/gitano/coverage.lua` which is generated at build time
+from `lib/gitano/coverage.lua.in`.