From 4c6ab89d46a3ef95dfb2b7f36ecb98d0ed771154 Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Sun, 12 Oct 2014 18:38:30 +0300 Subject: Add a chapter on code structure --- ARCH | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/ARCH b/ARCH index 15ce872..a815c99 100644 --- a/ARCH +++ b/ARCH @@ -392,3 +392,87 @@ WEBAPP knows about. It has the following columns: Lorry when a job is run. * `generated` is set to 0 or 1, depending on if the Lorry came from an actual `.lorry` file or was generated by Lorry Controller. + + +Code structure +============== + +The Lorry Controller code base is laid out as follows: + +* `lorry-controller-webapp` is the main program of WEBAPP. It sets up + the bottle.py framework. All the implementations for the various + HTTP requests are in classes in the `lorrycontroller` Python + package, as subclasses of the `LorryControllerRoute` class. The main + program uses introspection ("magic") to find the subclasses + automatically and sets up the bottle.py routes correctly. This makes + it possible to spread the code into simple classes; bottle's normal + way (with the `@app.route` decorator) seemed to make that harder and + require everything in the same class. + +* `lorrycontroller` is a Python package with the HTTP request + handlers, management of STATEDB, plus some helpful utilities. + +* `lorry-controller-minion` is the entirety of the MINION, except that + it uses the `lorrycontroller.setup_proxy` function. + The MINION is kept very simple on purpose: all the interesting logic + is in the WEBAPP instead. + +* `static` has static content to be served over HTTP. Primarily, the + CSS file for the HTML interfaces. When LC is integrated within the + Trove, the web server gets configured to serve these files directly. + The `static` directory will be accessible over plain HTTP on port + 80, and on port 12765 via the WEBAPP, to allow HTML pages to refer + to it via a simple path. + +* `templates` contains bottle.py HTML templates for various pages. + +* `etc` contains files to be installed in `/etc` when LC is installed + on a Baserock system. Primarily this is the web server (lighttpd) + configuration to invoke WEBAPP. + +* `units` contains various systemd units that start services and run + time-based jobs. + +* `yarns.webapp` contains an integration test suite for WEBAPP. + This is run by the `./check` script. The `./test-wait-for-port` + script is used by the yarns. + +Example +------- + +As an example, to modify how the `/1.0/status-html` request works, you +would look at its implementation in `lorrycontroller/status.py`, and +perhaps also the HTML templates in `templates/*.tpl`. + +STATEDB +------- + +The persistent state of WEBAPP is stored in an Sqlite3 database. All +access to STATEDB within WEBAPP is via the +`lorrycontroller/statedb.py` code module. That means there are no SQL +statements outside `statedb.py` at all, nor is it OK to add any. If +the interface provided by the `StateDB` class isn't sufficient, then +modify the class suitably, but do not add any new SQL outside it. + +All access from outside of WEBAPP happens via WEBAPP's HTTP API. +Only the WEBAPP is allowed to touch STATEDB in any way. + +The bottle.py framework runs multiple threads of WEBAPP code. The +threads communicate only via STATEDB. There is no shared state in +memory. SQL's locking is used for mutual exclusion. + +The `StateDB` class acts as a context manager for Python's `with` +statements to provide locking. To access STATEDB with locking, use +code such as this: + + with self.open_statedb() as statedb: + troves = statedb.get_troves() + for trove in troves: + statedb.remove_trove(troves) + +The code executed by the `with` statement is run under lock, and the +lock gets released automatically even if there is an exception. + +(You could manage locks manually. It's a good way to build character +and learn why using the context manager is really simple and leads to +more correct code.) -- cgit v1.2.1