1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
README for lorry-controller
===========================
Overview
--------
Lorry Controller, or LC for short, manages the importing of source
code from external sources into git repositories on a Trove. A Trove
is a component in the Baserock system for hosting source code, and LC
runs inside a Trove.
LC uses the Lorry tool to do the actual import. Lorry can read code
from several different version control systems, and convert them to
git. External repositories can be specfied individually, as Lorry
`.lorry` specification files. In addition, LC can be told to mirror
all the git repositories on another Trove.
LC runs Lorry for the right external repositories, and takes care of
running a suitable number of Lorry instances concurrently, and
recovering from any problems. LC has a web based administration
interface, and an HTTP API for reporting and controlling its state.
This README file documents the LC configuration file and general use.
For the architecture of LC and the HTTP API, see the `ARCH` file.
Lorry Controller configuration
------------------------------
Lorry Controller reads a configuration file of the following format.
* The file uses JSON syntax.
* The file is a list.
* Each item in the list is a mapping specifying a Trove or a set of
`.lorry` files.
* A Trove spec specifies another Trove to mirror completely.
* A Lorry spec specifies a set of Lorry specification (`.lorry` files)
for individual project repositories to convert or mirror to git on
the local Trove.
Each spec (mapping) has a number of key/value pairs. The following are
shared between Trove and Lorry specs:
* `type` is the type of the spec; value MUST be either `trove` or
`lorries`.
* `interval` specifies how often Lorry Controller should mirror the
repositories in the spec. See below for INTERVAL.
* `protocol`: specifies how Lorry Controller (and Lorry) should talk
to remove Troves. Allowed values are `ssh`, `https`, `http`. This
field is mandatory.
* `auth`: Specifies how to authenticate to the remote Trove over
https. The is an optional field. If present, it should be a
dictionary with the fields `username` and `password`.
* Additionally, the following seem to be supported by an old version
of Lorry Controller, but are ignored by the new Lorry Controller:
`uuid`, `serial`, `create`, `destroy`, `stagger`, `tarball`.
Trove specs have the following keys:
* `trovehost` is the other Trove to mirror; a domain name or IP
address. It is mandatory.
* `ls-interval` determines how often should Lorry Controller query the
other Trove for a list of repositories it may mirror. See below for
INTERVAL. `ls-interval` is mandatory.
* `prefixmap` maps repository path prefixes from the other Trove to
the local Trove. It is mandatory in a Trove spec. If the remote
prefix is `foo`, and the local prefix is `bar`, then remote
repository `foo/baserock/yeehaa` gets mirrored to local repository
`bar/baserock/yeehaa`. If the remote Trove has a repository that
does not match a prefix, that repository gets ignored.
* `ignore` is a list of git repositories from the other Trove that
should NOT be mirrored. Each list element is a path to the git
repository (not including leading slash). `ignore` is optional.
An INTERVAL value (for `interval` or `ls-interval`) is number and a
unit to indicate a time interval. Allowed units are minutes (`m`),
hours (`h`), and days (`d`), expressed as single-letter codes in upper
or lower case.
Lorry specs have the following keys:
* `prefix` is a path prefix to be prepended to all repositories
created from the `.lorry` files from this spec. It is mandatory.
* `globs` is a list of globs (as strings) for `.lorry` files to use.
The glob is matched in the directory containing the configuration
file in which this spec is. It is OK for the globs to not match
anything. A `globs` entry is mandatory, however.
A fairly minimal example for mirroring `git.baserock.org` and using
local `.lorry` files.
[
{
"type": "trove",
"trovehost": "git.baserock.org",
"ls-interval": "4d",
"interval": "2h",
"prefixmap": {
"baserock": "baserock",
"delta": "delta"
}
},
{
"type": "lorries",
"interval": "1h",
"prefix": "delta",
"globs": [
"open-source-lorries/*.lorry"
]
}
]
The syntax of `.lorry` files is specified by the Lorry program; see
its documentation for details.
HTTP proxy configuration
------------------------
Lorry Controller will look for a file called `proxy.conf` in the same
directory as the `lorry-controller.conf` configuration file.
It is in JSON format, with the following key/value pairs:
* `hostname` -- the hostname of the HTTP proxy
* `username` -- username for authenticating to the proxy
* `password` -- a **cleartext** password for authenticating to the
proxy
* `port` -- port number for connecting to the proxy
Lorry Controller will use this information for both HTTP and HTTPS
proxying.
Do note that the **password is stored in cleartext** and that access
to the configuration file (and the git repository where it is stored)
must be controlled appropriately.
|
