README for lorry-controller
===========================

Lorry Controller mirrors Troves and mirrors or converts upstream
projects into git repositories on the local Trove. Lorry Controller
reads a configuration file (see below) and runs the Lorry program
against the intended targets at suitable intervals.

Lorry Controller configuration
------------------------------

Lorry Controller reads a configuration file of the following format.

* The file uses JSON syntax.
* The file is a list.
* Each item in the list is a mapping specifying a Trove or a set of
  `.lorry` files.
* A Trove spec specifies another Trove to mirror completely.
* A Lorry spec specifies a set of Lorry specification (`.lorry` files)
  for individual project repositories to convert or mirror to git on
  the local Trove.

Each spec (mapping) has a number of key/value pairs. The following are
shared between Trove and Lorry specs:

* `type` is the type of the spec; value MUST be either `trove` or
  `lorries`.
* `interval` specifies how often Lorry Controller should mirror the
  repositories in the spec. See below for INTERVAL.
* `protocol`: specifies how Lorry Controller (and Lorry) should talk
  to remove Troves. Allowed values are `ssh`, `https`, `http`. This
  field is mandatory.
* `auth`: Specifies how to authenticate to the remote Trove over
  https. The is an optional field. If present, it should be a
  dictionary with the fields `username` and `password`.
* Additionally, the following seem to be supported by an old version
  of Lorry Controller, but are ignored by the new Lorry Controller:
  `uuid`, `serial`, `create`, `destroy`, `stagger`, `tarball`.

Trove specs have the following keys:

* `trovehost` is the other Trove to mirror; a domain name or IP
  address. It is mandatory.
* `ls-interval` determines how often should Lorry Controller query the
  other Trove for a list of repositories it may mirror. See below for
  INTERVAL. `ls-interval` is mandatory.
* `prefixmap` maps repository path prefixes from the other Trove to
  the local Trove. It is mandatory in a Trove spec. If the remote
  prefix is `foo`, and the local prefix is `bar`, then remote
  repository `foo/baserock/yeehaa` gets mirrored to local repository
  `bar/baserock/yeehaa`. If the remote Trove has a repository that
  does not match a prefix, that repository gets ignored.
* `ignore` is a list of git repositories from the other Trove that
  should NOT be mirrored. Each list element is a path to the git
  repository (not including leading slash). `ignore` is optional.

An INTERVAL value (for `interval` or `ls-interval`) is number and a
unit to indicate a time interval. Allowed units are minutes (`m`),
hours (`h`), and days (`d`), expressed as single-letter codes in upper
or lower case.

Lorry specs have the following keys:

* `prefix` is a path prefix to be prepended to all repositories
  created from the `.lorry` files from this spec. It is mandatory.
* `globs` is a list of globs (as strings) for `.lorry` files to use.
  The glob is matched in the directory containing the configuration
  file in which this spec is. It is OK for the globs to not match
  anything. A `globs` entry is mandatory, however.

A fairly minimal example for mirroring `git.baserock.org` and using
local `.lorry` files.

    [
        {
            "type": "trove",
            "trovehost": "git.baserock.org",
            "ls-interval": "4d",
            "interval": "2h",
            "prefixmap": {
                "baserock": "baserock",
                "delta": "delta"
            }
        },
        {
            "type": "lorries",
            "interval": "1h",
            "prefix": "delta",
            "globs": [
                "open-source-lorries/*.lorry"
            ]
        }
    ]

The syntax of `.lorry` files is specified by the Lorry program; see
its documentation for details.