path: root/yarns.webapp/060-validation.yarn

Validation of CONFGIT
=====================

The CONFGIT repository contains two types of files we should validate:
the `lorry-controller.conf` file, and the local Lorry files (specified
by the former file in `lorries` sections).

Validate `lorry-controller.conf`
--------------------------------

We'll start by validating the `lorry-controller.conf` file. There's
several aspects here that need to be tested:

* JSON syntax correctness: if the file doesn't parse as JSON, the
  WEBAPP should cope and shouldn't change STATEDB in any way.
* Semantic correctness: the file should contain a list of dicts, and
  each dict should have the right fields with the right kind of
  values. See the `README` for details. Other fields are also allowed,
  though ignored. Again, if there's an error, WEBAPP should cope, and
  probably shouldn't update STATEDB if there are any problems.

The approach for testing this is to set up an empty STATEDB, then get
WEBAPP to read a `lorry-controller.conf` with various kinds of
brokenness, and after each read verify that STATEDB is still empty.
This doesn't test that if the STATEDB wasn't empty it doesn't change
existing data, but it seems like a reasonable assumption that an
update happens regardless of previous contents of STATEDB, given how
SQL transactions work.

In summary:

* Start WEBAPP without a STATEDB, and have it read its config. Verify
  STATEDB is empty.
* Add a `lorry-controller.conf` that is broken in some specific way.
* Tell WEBAPP to re-read its config.
* Verify that WEBAPP gives an error message.
* Verify that STATEDB is still empty.

Repeat this for each type of brokenness we want to ensure WEBAPP
validates for.

    SCENARIO validate lorry-controller.conf
    GIVEN a new git repository in CONFGIT
    AND WEBAPP uses CONFGIT as its configuration directory
    AND a running WEBAPP

First of all, have WEBAPP read CONFGIT. This should succeed even if
the `lorry-controller.conf` file doesn't actually exist.

    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "Configuration has been updated"
    AND STATEDB is empty

Add an empty configuration file. This is different from a file
containing an empty JSON list. It should be treated as an error.

    GIVEN a lorry-controller.conf in CONFGIT containing ""
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

Add a syntactically invalid JSON file.

    GIVEN a lorry-controller.conf in CONFGIT containing "blah blah blah"
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

Replace the bad JSON file with one that has an unknown section (no
`type` field). Please excuse the non-escaping of double quotes: it's
an artifact of how yarn steps are implemented and is OK.

    GIVEN a lorry-controller.conf in CONFGIT containing "[{"foo": "bar"}]"
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

What about a section that has a `type` field, but it's set to a
non-sensical value?

    GIVEN a lorry-controller.conf in CONFGIT containing "[{"type": "BACKUPS!"}]"
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

Now we're getting to real sections. A `troves` section must have
`trovehost`, `interval`, `ls-interval`, and `prefixmap` set, and may
optionally have `ignore` set. The `trovehost` field can't really be
checked, and `interval` and `ls-interval` don't need much checking: if
they don't parse as sensible intervals, Lorry Controller will just use
a default value.

`prefixmap`, however, can have a reasonable check: it shouldn't map
something to be under the Trove ID of the local Trove, otherwise Lorry
won't be able to push the repositories. However, at this time, we do
not have a reasonable way to get the Trove ID of the local Trove, so
we're skipping implementing that test for now. (FIXME: fix this lack
of testing.)

The `ignore` field in a `trove` section and the `globs` field in a
`lorry` section are expected to be lists of strings, test a non-list
value.

    GIVEN an empty lorry-controller.conf in CONFGIT
    AND lorry-controller.conf in CONFGIT adds trove example-trove
    AND lorry-controller.conf in CONFGIT sets ignore to "foo" for trove example-trove
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

And a non-string list.

    GIVEN lorry-controller.conf in CONFGIT sets ignore to "[1, 2]" for trove example-trove
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "ERROR"
    AND STATEDB is empty

`ignore` is is not a mandatory option, verify this is the case.

    GIVEN lorry-controller.conf in CONFGIT removes field ignore from trove example-trove
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "Configuration has been updated"

Clean up at the end.

    FINALLY WEBAPP terminates


Validate local Lorry files
--------------------------

Lorry files (`.lorry`) are consumed by the Lorry program itself, but
also by Lorry Controller. In fact, the ones that are in CONFGIT are
only consumed by Lorry Controller: it reads them in, parses them,
extracts the relevant information, puts that into STATEDB, and then
generates a whole new (temporary) file for each Lorry run.

Lorry Controller doesn't validate the Lorry files much, only
enough that it can extract each separate Lorry specification and feed
them to Lorry one by one. In other words:

* The `.lorry` file must be valid JSON.
* It must be a dict.
* Each key must map to another dict.
* Each inner dict must have a key `type`, which maps to a string.

Everything else is left for Lorry itself. Lorry Controller only needs
to handle Lorry not working, and it already does that.

Firstly, some setup.

    SCENARIO validate .lorry files
    GIVEN a new git repository in CONFGIT
    AND an empty lorry-controller.conf in CONFGIT
    AND lorry-controller.conf in CONFGIT adds lorries *.lorry using prefix upstream
    AND WEBAPP uses CONFGIT as its configuration directory
    AND a running WEBAPP

Make sure WEBAPP handles there not being any `.lorry` files.

    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    AND STATEDB is empty
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to []

Add a `.lorry` file that contains broken JSON.

    GIVEN Lorry file CONFGIT/notjson.lorry with { THIS IS NOT JSON ]
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    AND STATEDB is empty
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to []

Add a `.lorry` file that is valid JSON, but is not a dict.

    GIVEN Lorry file CONFGIT/notadict.lorry with [1,2,3]
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    AND STATEDB is empty
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to []

Add a `.lorry` that is a dict, but doesn't map keys to dicts.

    GIVEN Lorry file CONFGIT/notadictofdicts.lorry with { "foo": 1 }
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    AND STATEDB is empty
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to []

Add a `.lorry` whose inner dict does not have a `type` field.

    GIVEN Lorry file CONFGIT/notype.lorry with { "foo": { "bar": "yo" }}
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    AND STATEDB is empty
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to []

Add a `.lorry` that is A-OK. This should work even when there are some
broken ones too.

    GIVEN Lorry file CONFGIT/a-ok.lorry with { "foo": { "type": "git", "url": "git://example.com/foo" }}
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"
    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to ["upstream/foo"]

Clean up at the end.

    FINALLY WEBAPP terminates


Load broken Lorry files
-----------------------

This scenario checks that if STATEDB already contains a valid Lorry
file, and we're trying to load an invalid one, then the valid one
doesn't get overwritten or discarded.

First, some setup.

    SCENARIO load broken .lorry files
    GIVEN a new git repository in CONFGIT
    AND an empty lorry-controller.conf in CONFGIT
    AND lorry-controller.conf in CONFGIT adds lorries *.lorry using prefix upstream
    AND WEBAPP uses CONFGIT as its configuration directory
    AND a running WEBAPP

Add a `.lorry` file that contains valid JSON.

    GIVEN Lorry file CONFGIT/a-ok.lorry with { "foo": { "type": "git", "url": "git://example.com/foo" }}
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"

Make sure it's the one we loaded.

    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to ["upstream/foo"]
    WHEN admin makes request GET /1.0/lorry/upstream/foo
    THEN response matches "git://example.com/foo"

Now, add a broken file and try to load that.

    GIVEN Lorry file CONFGIT/notjson.lorry with THIS IS NOT JSON
    WHEN admin makes request POST /1.0/read-configuration
    THEN response matches "has been updated"

Now, make sure we still have what we originally had.

    WHEN admin makes request GET /1.0/list-queue
    THEN response has queue set to ["upstream/foo"]
    WHEN admin makes request GET /1.0/lorry/upstream/foo
    THEN response matches "git://example.com/foo"

Clean up at the end.

    FINALLY WEBAPP terminates