1 files changed, 190 insertions, 0 deletions
diff --git a/yarns.webapp/060-validation.yarn b/yarns.webapp/060-validation.yarn
new file mode 100644
index 0000000..989c80b
--- /dev/null
+++ b/yarns.webapp/060-validation.yarn
@@ -0,0 +1,190 @@
+Validation of CONFGIT
+=====================
+
+The CONFGIT repository contains two types of files we should validate:
+the `lorry-controller.conf` file, and the local Lorry files (specified
+by the former file in `lorries` sections).
+
+Validate `lorry-controller.conf`
+--------------------------------
+
+We'll start by validating the `lorry-controller.conf` file. There's
+several aspects here that need to be tested:
+
+* JSON syntax correctness: if the file doesn't parse as JSON, the
+  WEBAPP should cope and shouldn't change STATEDB in any way.
+* Semantic correctness: the file should contain a list of dicts, and
+  each dict should have the right fields with the right kind of
+  values. See the `README` for details. Other fields are also allowed,
+  though ignored. Again, if there's an error, WEBAPP should cope, and
+  probably shouldn't update STATEDB if there are any problems.
+
+The approach for testing this is to set up an empty STATEDB, then get
+WEBAPP to read a `lorry-controller.conf` with various kinds of
+brokenness, and after each read verify that STATEDB is still empty.
+This doesn't test that if the STATEDB wasn't empty it doesn't change
+existing data, but it seems like a reasonable assumption that an
+update happens regardless of previous contents of STATEDB, given how
+SQL transactions work.
+
+In summary:
+
+* Start WEBAPP without a STATEDB, and have it read its config. Verify
+  STATEDB is empty.
+* Add a `lorry-controller.conf` that is broken in some specific way.
+* Tell WEBAPP to re-read its config.
+* Verify that WEBAPP gives an error message.
+* Verify that STATEDB is still empty.
+
+Repeat this for each type of brokenness we want to ensure WEBAPP
+validates for.
+
+    SCENARIO validate lorry-controller.conf
+    GIVEN a new git repository in CONFGIT
+    AND WEBAPP uses CONFGIT as its configuration directory
+    AND a running WEBAPP
+
+First of all, have WEBAPP read CONFGIT. This should succeed even if
+the `lorry-controller.conf` file doesn't actually exist.
+
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "Configuration has been updated"
+    AND STATEDB is empty
+
+Add an empty configuration file. This is different from a file
+containing an empty JSON list. It should be treated as an error.
+
+    GIVEN a lorry-controller.conf in CONFGIT containing ""
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "ERROR"
+    AND STATEDB is empty
+
+Add a syntactically invalid JSON file.
+
+    GIVEN a lorry-controller.conf in CONFGIT containing "blah blah blah"
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "ERROR"
+    AND STATEDB is empty
+
+Replace the bad JSON file with one that has an unknown section (no
+`type` field). Please excuse the non-escaping of double quotes: it's
+an artifact of how yarn steps are implemented and is OK.
+
+    GIVEN a lorry-controller.conf in CONFGIT containing "[{"foo": "bar"}]"
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "ERROR"
+    AND STATEDB is empty
+
+What about a section that has a `type` field, but it's set to a
+non-sensical value?
+
+    GIVEN a lorry-controller.conf in CONFGIT containing "[{"type": "BACKUPS!"}]"
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "ERROR"
+    AND STATEDB is empty
+
+Now we're getting to real sections. A `troves` section must have
+`trovehost`, `interval`, `ls-interval`, and `prefixmap` set, and may
+optionally have `ignore` set. The `trovehost` field can't really be
+checked, and `interval` and `ls-interval` don't need much checking: if
+they don't parse as sensible intervals, Lorry Controller will just use
+a default value.
+
+`prefixmap`, however, can have a reasonable check: it shouldn't map
+something to be under the Trove ID of the local Trove, otherwise Lorry
+won't be able to push the repositories. However, at this time, we do
+not have a reasonable way to get the Trove ID of the local Trove, so
+we're skipping implementing that test for now. (FIXME: fix this lack
+of testing.)
+
+Clean up at the end.
+
+    FINALLY WEBAPP terminates
+
+
+Validate local Lorry files
+--------------------------
+
+Lorry files (`.lorry`) are consumed by the Lorry program itself, but
+also by Lorry Controller. In fact, the ones that are in CONFGIT are
+only consumed by Lorry Controller: it reads them in, parses them,
+extracts the relevant information, puts that into STATEDB, and then
+generates a whole new (temporary) file for each Lorry run.
+
+Lorry Controller doesn't validate the Lorry files much, only
+enough that it can extract each separate Lorry specification and feed
+them to Lorry one by one. In other words:
+
+* The `.lorry` file must be valid JSON.
+* It must be a dict.
+* Each key must map to another dict.
+* Each inner dict must have a key `type`, which maps to a string.
+
+Everything else is left for Lorry itself. Lorry Controller only needs
+to handle Lorry not working, and it already does that.
+
+Firstly, some setup.
+
+    SCENARIO validate .lorry files
+    GIVEN a new git repository in CONFGIT
+    AND an empty lorry-controller.conf in CONFGIT
+    AND lorry-controller.conf in CONFGIT adds lorries *.lorry using prefix upstream
+    AND WEBAPP uses CONFGIT as its configuration directory
+    AND a running WEBAPP
+
+Make sure WEBAPP handles there not being any `.lorry` files.
+
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    AND STATEDB is empty
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to []
+
+Add a `.lorry` file that contains broken JSON.
+
+    GIVEN Lorry file CONFGIT/notjson.lorry with THIS IS NOT JSON
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    AND STATEDB is empty
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to []
+
+Add a `.lorry` file that is valid JSON, but is not a dict.
+
+    GIVEN Lorry file CONFGIT/notadict.lorry with [1,2,3]
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    AND STATEDB is empty
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to []
+
+Add a `.lorry` that is a dict, but doesn't map keys to dicts.
+
+    GIVEN Lorry file CONFGIT/notadictofdicts.lorry with { "foo": 1 }
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    AND STATEDB is empty
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to []
+
+Add a `.lorry` whose inner dict does not have a `type` field.
+
+    GIVEN Lorry file CONFGIT/notype.lorry with { "foo": { "bar": "yo" }}
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    AND STATEDB is empty
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to []
+
+Add a `.lorry` that is A-OK. This should work even when there are some
+broken ones too.
+
+    GIVEN Lorry file CONFGIT/a-ok.lorry with { "foo": { "type": "git", "url": "git://example.com/foo" }}
+    WHEN admin makes request POST /1.0/read-configuration with dummy=value
+    THEN response matches "has been updated"
+    WHEN admin makes request GET /1.0/list-queue
+    THEN response has queue set to ["upstream/foo"]
+
+Clean up at the end.
+
+    FINALLY WEBAPP terminates