summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Add `bst source-checkout` commandchandan/source-checkoutChandan Singh2018-10-267-27/+284
| | | | | | | | | | | | As discussed in https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html, add `bst source-checkout` command. This will allow users to checkout sources for a given target. * _frontend/cli.py: Add source-checkout command * _pipeline.py: Add assert_sources_cached() method * _stream.py: Add source_checkout method, abstract out __check_location_writable() method that used to part of checkout()
* buildstream/_pipeline.py: Simplify error message for assert_consistent()Chandan Singh2018-10-261-1/+1
| | | | | | | | | | | | | | | Currently, `Pipeline.assert_consistent()` prints an error message like the one below when certain elements have inconsistent sources: ``` ... Element: hello.bst is inconsistent Source tar source at hello.bst [line 16 column 2] is missing ref ... ``` Drop the word "Source" from the beginning of the message as "Source <source-kind> source ..." is kind of awkward to read.
* Merge branch 'danielsilverstone-ct/yaml-sentinel-rework' into 'master'James Ennis2018-10-264-18/+7
|\ | | | | | | | | Move _sentinel from utils.py to _yaml.py See merge request BuildStream/buildstream!903
| * various: Move _sentinel from utils.py to _yaml.pyDaniel Silverstone2018-10-264-18/+7
|/ | | | | | | | | The `_sentinel` in `utils.py` was used only for yaml node tracking. As such, simplify matters by removing it from `utils.py` and move it to `_yaml.py` which means that we no longer need to try and avoid a circular import issue by means of runtime importing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'jennis/another_docs_patch' into 'master'James Ennis2018-10-261-3/+42
|\ | | | | | | | | More information regarding the use of systemd to manage the cache See merge request BuildStream/buildstream!904
| * docs: Add more information to "managing the cache with systemmd" sectionJames Ennis2018-10-261-3/+36
| |
| * using_configuring_artifact_server.rst: Fix lines that are too longJames Ennis2018-10-261-3/+9
|/
* Merge branch 'danielsilverstone-ct/use-enumerate' into 'master'Phil Dawson2018-10-261-5/+3
|\ | | | | | | | | _loader/loadelement.py: Use enumerate() in dependency extraction See merge request BuildStream/buildstream!901
| * _loader/loadelement.py: Use enumerate() in dependency extractiondanielsilverstone-ct/use-enumerateDaniel Silverstone2018-10-261-5/+3
|/ | | | | | | | | In cases where elements end up with large numbers of dependencies, the n-squared performance in `_extract_depends_from_node` can produce significant delays during project load. This patch corrects that. In addition we were double-retrieving provenance which was unnecessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'bschubert/fix-command-sandbox' into 'master'Benjamin Schubert2018-10-263-8/+12
|\ | | | | | | | | | | | | Check is command is a str and replace by list before checking existence Closes #728 See merge request BuildStream/buildstream!898
| * Check if command is a str and replace by list before checking existenceBenjamin Schubert2018-10-263-8/+12
|/ | | | | | | | Currently we would make sure the sandbox had a command before converting it to a list if it was given as a string. That meant that a string command would never exist and the check be invalid. This also adds the same logic in the dummy sandbox for consistency.
* Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'Jonathan Maw2018-10-261-3/+2
|\ | | | | | | | | _yaml.py: Do not insert into cache if retrieved from cache See merge request BuildStream/buildstream!902
| * _yaml.py: Do not insert into cache if retrieved from cacheDaniel Silverstone2018-10-261-3/+2
|/ | | | | | | | We were unconditionally inserting into the yaml cache anything we had retrieved from the yaml cache. This had the side-effect of being expensive on re-runs of commands where yaml hadn't changed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'chandan/source-doc-directory' into 'master'Chandan Singh2018-10-2523-34/+63
|\ | | | | | | | | | | | | plugins: Don't redundantly document common configurations Closes #713 See merge request BuildStream/buildstream!896
| * build elements: Add reference to BuildElement's built-in configchandan/source-doc-directoryChandan Singh2018-10-2512-0/+35
| | | | | | | | | | | | | | | | | | | | | | For each element plugin derived from BuildElement, add reference to the built-in functionality documentation of BuildElement. * buildelement.py: Add anchor for built-in functionality. * plugins/elements/*.py: Add refence to BuildElement's built-in functionality for each build element. Related to #713.
| * sources: Don't redundantly document directory configurationChandan Singh2018-10-2511-34/+28
|/ | | | | | | | | | | | Instead of redundantly documenting "directory" configuration option in each source plugin, refer to the built-in functionality documentation of the Source class. * source.py: Add anchor for built-in functionality. * sources/plugins/*.py: Remove directory config option, and add reference to Source's built-in functionality. Fixes #713.
* Merge branch 'danielsilverstone-ct/fix-lint-issues' into 'master'Chandan Singh2018-10-2530-91/+91
|\ | | | | | | | | Fix lint issues See merge request BuildStream/buildstream!892
| * .gitlab-ci.yml: Change how coverage is recoveredDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | We need to alter how coverage data is recovered in order to work with newer coverage tooling. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * MANIFEST: Include a lot more missing stuff in the manifestDaniel Silverstone2018-10-252-7/+16
| | | | | | | | | | | | | | | | | | Versioneer needs to be in the MANIFEST.in and its .pyc needs to be ignored in the .gitignore. Also much docs were not being included, nor conftest.py. Much of the test suite data files needed including, so changed that to a raw include of everything in the tests/ tree. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * Bump CI image tagsTristan Maat2018-10-251-6/+6
| | | | | | | | We need to do this so that the new linting errors pop up
| * _artifactcache/cascache.py: Cleanup lint warning about indexingDaniel Silverstone2018-10-251-2/+3
| | | | | | | | | | | | | | In order to quieten the linter, and to make a more correct comparison, we switch to using the `Code.OK` enumeration from `google.rpc.Code` Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Prepare attributes in __init__Daniel Silverstone2018-10-251-0/+4
| | | | | | | | | | | | | | Python linters prefer that attributes are defined in __init__ rather than left to later routines. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Silence len(SEQUENCE) as condition warningsDaniel Silverstone2018-10-251-3/+3
| | | | | | | | | | | | | | The lint tool dislikes the use of `if len(SEQ) == 0` type statements because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Refactor slightly for indentationDaniel Silverstone2018-10-251-3/+4
| | | | | | | | | | | | | | Correct some multi-line conditional indentation by slight refactor if necessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | Sometimes `dict.get()` is preferable to an `if` statement, but this time it's clearer if we keep the structure. As such, silence the warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about preexec_fnDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | We are super-careful to not use threads in places where we might use Popen and as such this warning is save to quash. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about unused variableDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The variable was unused, remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxbwrap.py: Disable lint about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | Sometimes `dict.get()` is preferable, in this instance it's less clear so we'll disable that lint Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_mounter.py: Remove useless inheritance on `object`Daniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | In Python 3, inheriting from `object` explicitly is pointless. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_mount.py: Do not use dict literals in argument defaultsDaniel Silverstone2018-10-251-3/+6
| | | | | | | | | | | | | | The use of dictionary literals in argument defaults is disrecommended due to the way that they are static and thus potentially very confusing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * plugins/source/git.py: Rationalised unused return valuesDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | The `assert_ref_in_track()` method's return value was never used so rationalise it to never return a value. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/queue/trackqueue.py: Remove unused importDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The `SourceError` import was never used Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/queue/pullqueue.py: Remove unused return valueDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | The `done()` method does not return anything anywhere else. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/jobs/job.py: Correct lint issue around abstract methodDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The linter cannot detect abstract methods the way we use them, so this silences the warning which ensues. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/linux.py: Simplify return statementDaniel Silverstone2018-10-251-5/+1
| | | | | | | | | | | | | | The linter dislikes an if/return/else/return where it can be simplified to a return of the if condition. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/linux.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/darwin.py: Remove unused importsDaniel Silverstone2018-10-251-2/+0
| | | | | | | | | | | | | | The import of `resource` was unused. The import of `PlatformError` was unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/darwin.py: Remove useless __init__()Daniel Silverstone2018-10-251-4/+0
| | | | | | | | | | | | | | An otherwise empty __init__ passing no arguments to super.__init__ is worthless. Remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _options/optionbool.py: Refactor conditionsDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | The pylint tool prefers that conditions of the form "var == foo or var == bar" be refactored into "var in (foo, bar)". Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _fuse/mount.py: Deal with abstract method lint issueDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | | | Unfortunately pylint can't tell this is an abstract method which will only ever end up invoked from a class which concretises it. As such we need to silence this warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaultsDaniel Silverstone2018-10-252-3/+5
| | | | | | | | | | | | | | | | Since default values in arguments to functions and methods are created once at the compilation of the code, it is dangerous to include list and dict literals in them. This changes the use of {} to None. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _frontend/status.py: Reorder importsDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The `curses` import is "standard" whereas `click` is not. As such, despite alphabetical order normally being the rule, `curses` must come before `click`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _frontend/app.py: Remove unused import of `resource`Daniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The `resource` import was unused. Removed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/artifactcache.py: Silence lint issuesDaniel Silverstone2018-10-251-3/+3
| | | | | | | | | | | | | | | | | | The pylint tool is unable to understand that the abstract methods in the artifact cache will only be called when there's concrete implementations behind them. Silence the lint errors for these specific calls. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/artifactcache.py: Remove superfluous parensDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The pylint report dislikes the superfluous parens which used to be here in this assert. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * utils.py: Group stat importsDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The imports of `stat` and `S_ISDIR` from `stat` should be grouped to satisfy pylint. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * utils.py: Disable lint warning about popenDaniel Silverstone2018-10-251-1/+2
| | | | | | | | | | | | | | | | | | | | We use `preexec_fn` in our use of Popen. Since there are threads in the codebase, this is not necessarily safe. However we go to great lengths to ensure that the main process doesn't spawn threads (they're used in gRPC a lot) and so it should be safe for our use. As such, we disable the lint here. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * source.py: Remove unused variableDaniel Silverstone2018-10-251-2/+0
| | | | | | | | | | | | Remove the `context` variable which was unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * source.py: Remove unused local variableDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | Remove `source_kind` as it was unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>