summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* WIP: supply a single repository name to all raw-file mirrors instead of ↵jonathanmaw/raw-file-mirrorJonathan Maw2021-10-141-1/+12
| | | | using the lorry's path
* Merge branch 'bwh/add-failures-page' into 'master'Ben Brown2020-10-064-6/+72
|\ | | | | | | | | | | | | Add a page listing failing lorries Closes #20 See merge request CodethinkLabs/lorry/lorry-controller!25
| * Add a page listing failing lorriesBen Hutchings2020-10-064-6/+72
|/ | | | | | | | | | | | | Provide a status page that lists only the lorries that failed to update on the last attempt. This will make it easier to spot problems that require an update to the lorry configuration. Reuse some of the existing status page logic and render the run queue with a different template. Add a link to this page from the status page. Closes #20.
* Merge branch 'bwh/fix-remove-old-jobs' into 'master'Ben Brown2020-09-162-27/+31
|\ | | | | | | | | | | | | Reduce work and log output from lorry-controller-remove-old-jobs Closes #18 See merge request CodethinkLabs/lorry/lorry-controller!24
| * lorry-controller-remove-old-jobs: Filter out jobs started too recentlyBen Hutchings2020-09-151-1/+13
| | | | | | | | | | | | | | | | | | Job IDs are assigned sequentially, and jobs cannot finish before they start. Therefore we can iterate over jobs in order of ID and stop when we find a job that started more recently than max-age-seconds ago. Related to #18.
| * lorry-controller-remove-old-jobs: Add start time to JobInfo classBen Hutchings2020-09-151-4/+7
| | | | | | | | | | | | | | In order to filter jobs more efficiently, we need to look at the start time (job_started in the API) as well as the finish time. Related to #18.
| * lorry-controller-remove-old-jobs: Use a single loop in process_argsBen Hutchings2020-09-151-10/+5
| | | | | | | | | | | | | | In order to filter jobs earlier, we need to have a single loop over jobs in process_args(). Related to #18.
| * lorry-controller-remove-old-jobs: Inline methods called by process_args()Ben Hutchings2020-09-151-21/+15
| | | | | | | | | | | | | | | | In order to filter jobs earlier, we need to have a single loop over jobs in process_args() instead of in multiple functions that it calls. Prepare for that by inlining the methods that it calls directly. Related to #18.
| * units: Increase delay for lorry-controller-remove-old-jobsBen Hutchings2020-09-151-1/+1
|/ | | | | | | | | lorry-controller-remove-old-jobs is currently run at very short intervals, with a delay of 1 minute between each run. Its work isn't time-critical, and this results in a lot of unnecessary work and log spam. Increase the delay between runs to 1 hour. Related to #18.
* Merge branch 'bwh/include-gpl' into 'master'Ben Brown2020-08-131-3/+341
|\ | | | | | | | | LICENSE: Include full text of GPLv2 See merge request CodethinkLabs/lorry/lorry-controller!23
| * LICENSE: Include full text of GPLv2Ben Hutchings2020-08-131-3/+341
|/ | | | | Since Lorry Controller is now separate from Baserock we shouldn't assume that it's accompanied by any license texts.
* Merge branch 'bwh/pyflakes' into 'master'Ben Brown2020-08-1214-29/+56
|\ | | | | | | | | Fix pyflakes errors and warnings; add it to CI configuration See merge request CodethinkLabs/lorry/lorry-controller!22
| * gitlab-ci: Run pyflakes3 over all Python sourcesBen Hutchings2020-08-101-0/+17
| |
| * lorrycontroller: Explicitly list all names to be re-exportedBen Hutchings2020-08-101-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | pyflakes isn't able to recognise that: __all__ = locals() effectively uses all the imported names. Replace this with an explicit list of names. Running pyflakes in CI should ensure that the list doesn't get out of sync.
| * Delete unused module importsBen Hutchings2020-08-107-15/+0
| | | | | | | | | | | | Various modules are imported and not used. Caught by pyflakes.
| * Delete assignments to local variables that are not used laterBen Hutchings2020-08-104-9/+3
| | | | | | | | | | | | | | pyflakes found various assignments to local variables which are not used again. In some cases we still need to evaluate the expression that's assigned. In most places we can delete the assignment entirely.
| * showjob: Use local variable instead of re-evaluating the same expressionBen Hutchings2020-08-101-1/+1
| | | | | | | | | | | | | | | | JobShower.get_job_as_json assigns to a local 'path' variable, but doesn't use it. Use it instead of re-evaluating the same expression assigned to it. Caught by pyflakes.
| * readconf: Fix handling of invalid time values in configurationBen Hutchings2020-08-101-1/+1
| | | | | | | | | | | | | | | | | | ReadConfiguration.fix_up_interval was returning default_value when the string value did not match the expected format. This variable is not defined, but default_interval is. This bug has been present since the beginning of LC development, and was caught by pyflakes.
| * givemejob: Fix upstream host metadata lookupBen Hutchings2020-08-101-2/+2
|/ | | | | | | | Commit bc7f80d "givemejob: Move upstream host metadata lookup out of get_repo_metadata" failed to pass through the statedb variable into the new function, so it will never work. Caught by pyflakes.
* Merge branch 'robjh/debug-yaml-error' into 'master'Ben Brown2020-08-031-0/+1
|\ | | | | | | | | Rewind lorry FDs before parsing as JSON. See merge request CodethinkLabs/lorry/lorry-controller!21
| * Rewind lorry FDs before parsing as JSON.Robert J. Heywood2020-08-031-0/+1
|/ | | | | | | | | First, the lorry files are parsed as YAML. If the parser fails, then the lorry files are parsed as JSON. If the YAML parser has advanced the Lorry file's file-descriptor, this can lead to JSON parser errors. This change resets the file descriptor to the start of the file before calling the JSON parser.
* Merge branch 'bwh/single-lorry-metadata' into 'master'Ben Brown2020-08-033-25/+124
|\ | | | | | | | | | | | | Add metadata for single repositories Closes #15 See merge request CodethinkLabs/lorry/lorry-controller!17
| * gitlab: Ignore failure to set the default branchBen Hutchings2020-07-301-3/+12
| | | | | | | | | | | | | | | | | | | | In testing the change to define the default branch for single repositories, I found that GitLab will reject a change of default branch if the branch does not yet exist. (It doesn't seem to do this when creating a repository.) Ignore failure to change the default branch, as we will fix it on the next run after the branch has been created.
| * givemejob: Add metadata for single repositoriesBen Hutchings2020-07-302-3/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When mirroring a repository found through a 'lorries' configuration section (instead of 'trove' or 'gitlab'), we currently don't set a description or default branch. * Set the description to the upstream repository path, but allow this to be overridden by a description field in the .lorry file. Prepend the host-name, just as we do when mirroring an Upstream Host. * Set the default branch to: - Bazaar: 'trunk' - Git: upstream default branch, found using 'git ls-remote' - others: 'master' Closes #15.
| * givemejob: Move upstream host metadata lookup out of get_repo_metadataBen Hutchings2020-07-151-10/+18
| | | | | | | | | | | | In preparation for adding metadata for single repositories, move the code for looking up metadata through the Upstream Host connector into a separate function.
| * givemejob: Stop synthesising host information when it's missingBen Hutchings2020-07-151-10/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Any lorry in the database should have been either: 1. Generated based on a 'trove' or 'gitlab' section, with 'from_host' set to the Upstream Host name. 2. Read from a lorry referred to in a 'lorries' section, with 'from_host' set to an empty string. So this error case indicates a dangling reference in the database. The current schema does not use foreign keys so this could happen, but it's reasonable to just give up and return empty metadata.
* | Merge branch 'bwh/local-empty-description' into 'master'Ben Brown2020-07-304-2/+8
|\ \ | | | | | | | | | | | | | | | | | | local: Create repos with empty description and enabled for export Closes #16 See merge request CodethinkLabs/lorry/lorry-controller!20
| * | local: Create repos with empty description and enabled for exportBen Hutchings2020-07-294-2/+8
|/ / | | | | | | | | | | | | | | | | | | | | | | 'git init' copies files into the repository from a template directory, which contains a 'replace me' description that we don't want. Provide and use our own template directory instead. While we're at it, also add the git-daemon-export-ok flag file that git-daemon and git-http-backend may check for (depending on their configuration). Closes #16.
* | Merge branch 'bwh/minion-multiple-lorry-configs' into 'master'Ben Hutchings2020-07-221-3/+3
|\ \ | | | | | | | | | | | | MINION: Allow specifying multiple configuration files for Lorry See merge request CodethinkLabs/lorry/lorry-controller!19
| * | MINION: Allow specifying multiple configuration files for LorryBen Hutchings2020-07-211-3/+3
|/ / | | | | | | | | Change the 'lorry-config' setting from string to string-list type. This will allow separating generic and site configuration files.
* | Merge branch 'bwh/sensible-disk-space' into 'master'Ben Brown2020-07-172-5/+11
|\ \ | |/ |/| | | | | | | | | Improve disk space formatting Closes #14 See merge request CodethinkLabs/lorry/lorry-controller!18
| * status: Show free disk space in MiB when it's < 1 GiBBen Hutchings2020-07-161-1/+7
| | | | | | | | | | | | Showing free disk space as 0 when it drops below 1 GiB may be unncessarily alarming, so show it in units of MiB if that happens. The status dictionary already includes it in both units.
| * status: Use integer division where appropriateBen Hutchings2020-07-161-4/+4
|/ | | | | | | | | | | | | | | In Python 2, the / operator between two integer-typed values produces another integer. In Python 3, it produces a float. We need to use the // operator to get the old behaviour. The change to floats in get_free_disk_space resulted in unnecessarily precise values for free disk space. The change in formats_secs_nicely doesn't seem to make a difference since we format the quotients with '%d', but I'm making it use // so it's clearer what we want. Closes #14.
* Merge branch 'bwh/doc-names' into 'master'Ben Brown2020-07-154-4/+18
|\ | | | | | | | | Use document names expected by GitLab See merge request CodethinkLabs/lorry/lorry-controller!16
| * Add copyright and license statements to a LICENSE fileBen Hutchings2020-07-151-0/+14
| | | | | | | | This is what git hosting services like GitLab expect.
| * Add .md extension to Markdown documentsBen Hutchings2020-07-153-4/+4
|/ | | | | This will cause them to be rendered on GitLab and other git hosts' web interfaces.
* Merge branch 'bwh/update-docs' into 'master'Ben Hutchings2020-06-142-0/+278
|\ | | | | | | | | | | | | Update documentation of dependencies and installation Closes #2 and #1 See merge request CodethinkLabs/lorry-controller!15
| * INSTALL: Document how to install Lorry and Lorry Controller themselvesBen Hutchings2020-06-091-0/+37
| | | | | | | | | | | | | | Installation is mostly covered by setup.py. However the configuration files, many of the directories, and the systemd commands are not. Related to #2.
| * INSTALL: Briefly document creation of CONFGITBen Hutchings2020-06-091-0/+10
| | | | | | | | | | | | | | This is mostly covered in README, but should probably be mentioned here too. Refer to README for the contents of the repository. Related to #2.
| * INSTALL: Document how to configure a front-end web serverBen Hutchings2020-06-091-0/+22
| | | | | | | | | | | | | | This is mostly a generalisation of the example lighttpd configuration. I haven't yet tried to configure any other web server. Relates to #2.
| * INSTALL: Document how to integrate Lorry/LC with each Downstream Host typeBen Hutchings2020-06-091-0/+140
| | | | | | | | | | | | | | | | | | Document how to configure Gerrit, Gitea, and GitLab for use with Lorry and Lorry Controller, and vice versa. For Gitano/Trove, refer to the Baserock documentation. Relates to #2.
| * INSTALL: Document creation of a user account for Lorry and LCBen Hutchings2020-06-081-0/+8
| | | | | | | | Related to #2.
| * INSTALL: List all the known direct dependencies and how to install themBen Hutchings2020-06-082-0/+61
| | | | | | | | | | | | | | | | Add a new documentation file with a full list of dependencies, stating which of them are optional, how to install each of them, and which versions have been tested. Refer to this from README. Closes #1.
* | Merge branch 'bwh/statedb-no-json1' into 'master'Ben Brown2020-06-141-1/+1
|\ \ | | | | | | | | | | | | STATEDB: Avoid depending on SQLite json1 extension See merge request CodethinkLabs/lorry-controller!13
| * | STATEDB: Avoid depending on SQLite json1 extensionBen Hutchings2020-06-081-1/+1
| |/ | | | | | | | | | | | | | | | | | | The migration to a generalised hosts table uses json_object(), which is part of the optional json1 extension to SQLite. This is not enabled by default, and isn't enabled in the Freedesktop-SDK build of SQLite. Since GitLab tokens shouldn't include double-quotes, we can avoid this by using standard SQL concatenation instead.
* | Merge branch 'bwh/update-copyright' into 'master'Ben Brown2020-06-1411-11/+11
|\ \ | |/ |/| | | | | Update copyright dates for files I recently changed See merge request CodethinkLabs/lorry-controller!14
| * Update copyright dates for files I recently changedBen Hutchings2020-06-0811-11/+11
|/
* Merge branch 'bwh/gitea-support' into 'master'Ben Hutchings2020-06-084-5/+146
|\ | | | | | | | | | | | | Add gitea Downstream Host connector Closes #9 See merge request CodethinkLabs/lorry-controller!12
| * Merge branch 'master' into 'bwh/gitea-support'Ben Hutchings2020-06-082-13/+31
| |\ | |/ |/| | | # Conflicts: # lorry-controller-webapp
* | Merge branch 'bwh/gitlab-config' into 'master'Ben Hutchings2020-06-084-10/+44
|\ \ | | | | | | | | | | | | | | | | | | Improve repo/group configuration on GitLab downstream Closes #6 See merge request CodethinkLabs/lorry-controller!10
View Lorry Controller Status