summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/frontend/buildcheckout.py: Enable test_build_checkout_tarball_stdoutdanielsilverstone-ct/preserve-stdoutbschubert/fix-binary-output-captureDaniel Silverstone2018-10-261-2/+1
| | | | | | | With the new support for saving the stdout FDs cleanly, and binary mode capture in runcli, enable the tarball-to-stdout test. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* tests/testutils/runcli.py: Support binary-mode capture of stdoutDaniel Silverstone2018-10-261-6/+7
| | | | | | | | In order to test things which write tarballs to stdout correctly, we need to capture the binary output cleanly. This ensures we're not potentially mismatching encodings in and out. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _stream.py: Preserve stdout FD across checkout-to-stdoutDaniel Silverstone2018-10-261-3/+10
| | | | | | | | In order to support things which might need to use the stdout FD after checkout has completed writing a tarball to it, preserve it by means of `os.dup()` and `os.dup2()` Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'valentindavid/ruamel_warnings' into 'master'Valentin David2018-10-261-2/+2
|\ | | | | | | | | Fix some ruamel warnings during tests See merge request BuildStream/buildstream!889
| * Fix some ruamel warnings during testsValentin David2018-10-261-2/+2
|/
* Merge branch 'danielsilverstone-ct/yaml-sentinel-rework' into 'master'James Ennis2018-10-264-18/+7
|\ | | | | | | | | Move _sentinel from utils.py to _yaml.py See merge request BuildStream/buildstream!903
| * various: Move _sentinel from utils.py to _yaml.pyDaniel Silverstone2018-10-264-18/+7
|/ | | | | | | | | The `_sentinel` in `utils.py` was used only for yaml node tracking. As such, simplify matters by removing it from `utils.py` and move it to `_yaml.py` which means that we no longer need to try and avoid a circular import issue by means of runtime importing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'jennis/another_docs_patch' into 'master'James Ennis2018-10-261-3/+42
|\ | | | | | | | | More information regarding the use of systemd to manage the cache See merge request BuildStream/buildstream!904
| * docs: Add more information to "managing the cache with systemmd" sectionJames Ennis2018-10-261-3/+36
| |
| * using_configuring_artifact_server.rst: Fix lines that are too longJames Ennis2018-10-261-3/+9
|/
* Merge branch 'danielsilverstone-ct/use-enumerate' into 'master'Phil Dawson2018-10-261-5/+3
|\ | | | | | | | | _loader/loadelement.py: Use enumerate() in dependency extraction See merge request BuildStream/buildstream!901
| * _loader/loadelement.py: Use enumerate() in dependency extractiondanielsilverstone-ct/use-enumerateDaniel Silverstone2018-10-261-5/+3
|/ | | | | | | | | In cases where elements end up with large numbers of dependencies, the n-squared performance in `_extract_depends_from_node` can produce significant delays during project load. This patch corrects that. In addition we were double-retrieving provenance which was unnecessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'bschubert/fix-command-sandbox' into 'master'Benjamin Schubert2018-10-263-8/+12
|\ | | | | | | | | | | | | Check is command is a str and replace by list before checking existence Closes #728 See merge request BuildStream/buildstream!898
| * Check if command is a str and replace by list before checking existenceBenjamin Schubert2018-10-263-8/+12
|/ | | | | | | | Currently we would make sure the sandbox had a command before converting it to a list if it was given as a string. That meant that a string command would never exist and the check be invalid. This also adds the same logic in the dummy sandbox for consistency.
* Merge branch 'danielsilverstone-ct/do-not-reinsert-yaml' into 'master'Jonathan Maw2018-10-261-3/+2
|\ | | | | | | | | _yaml.py: Do not insert into cache if retrieved from cache See merge request BuildStream/buildstream!902
| * _yaml.py: Do not insert into cache if retrieved from cacheDaniel Silverstone2018-10-261-3/+2
|/ | | | | | | | We were unconditionally inserting into the yaml cache anything we had retrieved from the yaml cache. This had the side-effect of being expensive on re-runs of commands where yaml hadn't changed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'chandan/source-doc-directory' into 'master'Chandan Singh2018-10-2523-34/+63
|\ | | | | | | | | | | | | plugins: Don't redundantly document common configurations Closes #713 See merge request BuildStream/buildstream!896
| * build elements: Add reference to BuildElement's built-in configchandan/source-doc-directoryChandan Singh2018-10-2512-0/+35
| | | | | | | | | | | | | | | | | | | | | | For each element plugin derived from BuildElement, add reference to the built-in functionality documentation of BuildElement. * buildelement.py: Add anchor for built-in functionality. * plugins/elements/*.py: Add refence to BuildElement's built-in functionality for each build element. Related to #713.
| * sources: Don't redundantly document directory configurationChandan Singh2018-10-2511-34/+28
|/ | | | | | | | | | | | Instead of redundantly documenting "directory" configuration option in each source plugin, refer to the built-in functionality documentation of the Source class. * source.py: Add anchor for built-in functionality. * sources/plugins/*.py: Remove directory config option, and add reference to Source's built-in functionality. Fixes #713.
* Merge branch 'danielsilverstone-ct/fix-lint-issues' into 'master'Chandan Singh2018-10-2530-91/+91
|\ | | | | | | | | Fix lint issues See merge request BuildStream/buildstream!892
| * .gitlab-ci.yml: Change how coverage is recoveredDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | We need to alter how coverage data is recovered in order to work with newer coverage tooling. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * MANIFEST: Include a lot more missing stuff in the manifestDaniel Silverstone2018-10-252-7/+16
| | | | | | | | | | | | | | | | | | Versioneer needs to be in the MANIFEST.in and its .pyc needs to be ignored in the .gitignore. Also much docs were not being included, nor conftest.py. Much of the test suite data files needed including, so changed that to a raw include of everything in the tests/ tree. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * Bump CI image tagsTristan Maat2018-10-251-6/+6
| | | | | | | | We need to do this so that the new linting errors pop up
| * _artifactcache/cascache.py: Cleanup lint warning about indexingDaniel Silverstone2018-10-251-2/+3
| | | | | | | | | | | | | | In order to quieten the linter, and to make a more correct comparison, we switch to using the `Code.OK` enumeration from `google.rpc.Code` Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Prepare attributes in __init__Daniel Silverstone2018-10-251-0/+4
| | | | | | | | | | | | | | Python linters prefer that attributes are defined in __init__ rather than left to later routines. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Silence len(SEQUENCE) as condition warningsDaniel Silverstone2018-10-251-3/+3
| | | | | | | | | | | | | | The lint tool dislikes the use of `if len(SEQ) == 0` type statements because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/cascache.py: Refactor slightly for indentationDaniel Silverstone2018-10-251-3/+4
| | | | | | | | | | | | | | Correct some multi-line conditional indentation by slight refactor if necessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | Sometimes `dict.get()` is preferable to an `if` statement, but this time it's clearer if we keep the structure. As such, silence the warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about preexec_fnDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | We are super-careful to not use threads in places where we might use Popen and as such this warning is save to quash. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxchroot.py: Silence warning about unused variableDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The variable was unused, remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_sandboxbwrap.py: Disable lint about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | Sometimes `dict.get()` is preferable, in this instance it's less clear so we'll disable that lint Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_mounter.py: Remove useless inheritance on `object`Daniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | In Python 3, inheriting from `object` explicitly is pointless. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * sandbox/_mount.py: Do not use dict literals in argument defaultsDaniel Silverstone2018-10-251-3/+6
| | | | | | | | | | | | | | The use of dictionary literals in argument defaults is disrecommended due to the way that they are static and thus potentially very confusing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * plugins/source/git.py: Rationalised unused return valuesDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | The `assert_ref_in_track()` method's return value was never used so rationalise it to never return a value. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/queue/trackqueue.py: Remove unused importDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The `SourceError` import was never used Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/queue/pullqueue.py: Remove unused return valueDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | The `done()` method does not return anything anywhere else. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _scheduler/jobs/job.py: Correct lint issue around abstract methodDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The linter cannot detect abstract methods the way we use them, so this silences the warning which ensues. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/linux.py: Simplify return statementDaniel Silverstone2018-10-251-5/+1
| | | | | | | | | | | | | | The linter dislikes an if/return/else/return where it can be simplified to a return of the if condition. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/linux.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/darwin.py: Remove unused importsDaniel Silverstone2018-10-251-2/+0
| | | | | | | | | | | | | | The import of `resource` was unused. The import of `PlatformError` was unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _platform/darwin.py: Remove useless __init__()Daniel Silverstone2018-10-251-4/+0
| | | | | | | | | | | | | | An otherwise empty __init__ passing no arguments to super.__init__ is worthless. Remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _options/optionbool.py: Refactor conditionsDaniel Silverstone2018-10-251-2/+2
| | | | | | | | | | | | | | The pylint tool prefers that conditions of the form "var == foo or var == bar" be refactored into "var in (foo, bar)". Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _fuse/mount.py: Deal with abstract method lint issueDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | | | Unfortunately pylint can't tell this is an abstract method which will only ever end up invoked from a class which concretises it. As such we need to silence this warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaultsDaniel Silverstone2018-10-252-3/+5
| | | | | | | | | | | | | | | | Since default values in arguments to functions and methods are created once at the compilation of the code, it is dangerous to include list and dict literals in them. This changes the use of {} to None. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _frontend/status.py: Reorder importsDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The `curses` import is "standard" whereas `click` is not. As such, despite alphabetical order normally being the rule, `curses` must come before `click`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _frontend/app.py: Remove unused import of `resource`Daniel Silverstone2018-10-251-1/+0
| | | | | | | | | | | | The `resource` import was unused. Removed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/artifactcache.py: Silence lint issuesDaniel Silverstone2018-10-251-3/+3
| | | | | | | | | | | | | | | | | | The pylint tool is unable to understand that the abstract methods in the artifact cache will only be called when there's concrete implementations behind them. Silence the lint errors for these specific calls. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _artifactcache/artifactcache.py: Remove superfluous parensDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The pylint report dislikes the superfluous parens which used to be here in this assert. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * utils.py: Group stat importsDaniel Silverstone2018-10-251-1/+1
| | | | | | | | | | | | | | The imports of `stat` and `S_ISDIR` from `stat` should be grouped to satisfy pylint. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>