summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* element.py: Only store the direct reverse dependencies on an element.tristan/bschubert-pipelineTristan Van Berkom2019-03-081-1/+1
| | | | | | | Ensures that `Element.__update_state_recursively()` works as intended and only updates the states on reverse dependencies when a source cache key has changed, stop descending into reverse dependencies as soon as a cache key is unchanged.
* tests/frontend/workspaces.py: Extended test that all elements get built in ↵Tristan Van Berkom2019-03-089-6/+41
| | | | | | | | the same session This extends the existing test which catches failures only in build-only dependency chains, with a similar test for runtime-only dependency chains.
* tests/frontend/workspace.py: Test that all elements build with workspace in playTristan Van Berkom2019-03-079-0/+74
| | | | | | | Tests that the target is still built even when a workspace is open on a runtime dependency of a build-only dependency. This adds the regression test provided by Matthew Yates for issue #919
* element.py: Add _fetch_done method and use this to update the stateBenjamin Schubert2019-03-072-1/+9
| | | | This is to be consistent will all other queues that follow this setup.
* Don't update state before summary. It should be consistentBenjamin Schubert2019-03-071-12/+1
| | | | | Since we know when an object's state can change, we don't need to update the state of all objects for consistency.
* Stop updating state in queue.status()Benjamin Schubert2019-03-074-11/+20
| | | | | | | | | | | | | | | Statuses of an element can be changed when: 1) It is pulled 2) It is fetched 3) It is workspaced and it finished building 4) One of its dependencies is tracked 5) One of its dependencies is workspaced and finished building We can therefore update the statuses at those moments and we don't need to check all the time. This reduces considerably the calls to update_states that are done
* Merge branch 'bschubert/tests-behind-proxy' into 'master'bst-marge-bot2019-03-061-0/+7
|\ | | | | | | | | test: Pass proxy and cert env variable to tox environment See merge request BuildStream/buildstream!1207
| * test: Pass proxy and cert env variable to tox environmentbschubert/tests-behind-proxyBenjamin Schubert2019-03-061-0/+7
|/ | | | | | In order to be able to run integration tests behind a proxy, we need to let proxy related environment variable cascade to the tox environment.
* Merge branch 'jennis/compose_backwards' into 'master'bst-marge-bot2019-03-062-6/+13
|\ | | | | | | | | cleanup: Move YAML handling logic in _include.py to _yaml.py See merge request BuildStream/buildstream!1205
| * _include.py: Move yaml related logic to _yaml.pyjennis/compose_backwardsJames Ennis2019-03-062-6/+13
|/
* Merge branch 'juerg/fast-import' into 'master'bst-marge-bot2019-03-0612-103/+183
|\ | | | | | | | | Improve import performance See merge request BuildStream/buildstream!1190
| * filter.py: Set BST_RUN_COMMANDS to FalseJürg Billeter2019-03-061-0/+3
| | | | | | | | Filter elements do not run any commands. This will allow faster staging.
| * filter.py: Set BST_VIRTUAL_DIRECTORYJürg Billeter2019-03-061-0/+3
| | | | | | | | The filter plugin doesn't use Sandbox.get_directory().
| * import.py: Set BST_RUN_COMMANDS to FalseJürg Billeter2019-03-061-0/+3
| | | | | | | | Import elements do not run any commands. This will allow faster staging.
| * element.py: Disable Sandbox.run() for elements that don't need itJürg Billeter2019-03-061-0/+7
| | | | | | | | This will allow faster staging.
| * element.py: Add BST_RUN_COMMANDS flagJürg Billeter2019-03-061-0/+6
| | | | | | | | | | | | This flag indicates whether the plugin may run commands using `Sandbox.run()` and defaults to `True`. This will allow faster staging for plugins that do not need to run any commands in the sandbox.
| * sandbox.py: Add _disable_run() methodJürg Billeter2019-03-061-0/+18
| | | | | | | | | | This enables use of CasBasedDirectory for faster staging when command execution is not required.
| * sandbox.py: Add _use_cas_based_directory() methodJürg Billeter2019-03-061-2/+14
| | | | | | | | | | Allow Sandbox implementations to decide whether to use CasBasedDirectory.
| * _casbaseddirectory.py: Deduplicate os.path.join() calls in _add_file()Jürg Billeter2019-03-061-2/+3
| |
| * _casbaseddirectory.py: Don't set relative_pathname in _check_replacementJürg Billeter2019-03-061-5/+4
| | | | | | | | | | We already have relative_pathname in the caller. Reuse that instead of calling os.path.join().
| * storage: Use variable-length argument list for Directory.descend()Jürg Billeter2019-03-0610-84/+70
| | | | | | | | | | This provides an API in line with, e.g., os.path.join(), and eliminates isinstance checks.
| * _casbaseddirectory.py: Add fast path for CAS-to-CAS importJürg Billeter2019-03-061-10/+52
|/ | | | | | If the destination directory does not exist yet and there is no filter, we can import the whole source directory by digest instead of importing each directory entry individually.
* Merge branch 'tristan/optional-coverage-fixup' into 'master'bst-marge-bot2019-03-052-4/+9
|\ | | | | | | | | Fixes for coverage optionality See merge request BuildStream/buildstream!1206
| * tests/testutils/artifactshare.py: Support optional coverageTristan Van Berkom2019-03-051-2/+7
| | | | | | | | | | | | | | This scaffolding needs to manually tell coverage to cleanup when sigterm happens in the process in order to collect the coverage report, but we need to do this conditionally in case we are running tests without coverage support.
| * requirements/Makefile: Do not depend on the "force" packageTristan Van Berkom2019-03-051-2/+2
|/ | | | Chandan, you silly pony.
* Merge branch 'chandan/always-regen-requirements' into 'master'bst-marge-bot2019-03-041-2/+4
|\ | | | | | | | | requirements/Makefile: Always rebuild requirements files See merge request BuildStream/buildstream!1204
| * requirements/Makefile: Always rebuild requirements filesChandan Singh2019-03-041-2/+4
|/ | | | | | | We always want to update all the requirements files together such that the versions of the common dependencies do not drift between them. Do so by adding a dummy phony target `FORCE` and depend on it from all the `.txt` targets.
* Merge branch 'tristan/optional-coverage' into 'master'bst-marge-bot2019-03-047-22/+43
|\ | | | | | | | | | | | | Make coverage optional Closes #916 See merge request BuildStream/buildstream!1189
| * .gitlab-ci.yml: Use recent toxTristan Van Berkom2019-03-041-2/+2
| | | | | | | | | | | | | | There appears to be multiple versions of tox in the WSL test environment, prepending /root/.local/bin to the PATH while running tests ensures that we are using a recent enough tox to execute the tests.
| * .gitlab-ci.yml: Adding tox --version output to test diagnosticsTristan Van Berkom2019-03-041-0/+2
| | | | | | | | | | This is generally useful to know, since the tox version is not enforced by BuildStream.
| * tox.ini: Support running tests without coverage present at allTristan Van Berkom2019-03-041-7/+13
| | | | | | | | | | | | | | | | | | | | | | This change simply makes it possible to run the tests without coverage and without any coverage related dependencies present in the tox venvs. To run tests without coverage, simply choose the python version and append "-nocover" to the env name, e.g.: tox -e py36-nocover
| * setup.cfg, tox.ini: Moved coverage support to the tox.ini levelTristan Van Berkom2019-03-042-2/+2
| | | | | | | | | | | | | | This disables any attempt to use coverage when running tests directly against installed dependencies with setup.py. This fixes #916
| * tox.ini / requirements: Separated coverage requirementsTristan Van Berkom2019-03-045-14/+27
|/ | | | Since coverage is optional, lets make it really optional.
* Merge branch 'jmac/make_marked_directories' into 'master'bst-marge-bot2019-03-021-0/+8
|\ | | | | | | | | | | | | _sandboxremote.py: Create marked directories before running Closes #851 See merge request BuildStream/buildstream!1056
| * _sandboxremote.py: Create marked directories before runningjmac/make_marked_directoriesJim MacArthur2019-03-021-0/+8
|/ | | | | | | The other two sandboxes (BWrap and chroot) both do this because marked directories may be mount points. _cache_artifact requires install-root to be present, and existing BuildStream projects may rely on these too.
* Merge branch 'bschubert/no-subprocess-decode' into 'master'bst-marge-bot2019-03-016-25/+28
|\ | | | | | | | | refactor: remove the need to decode output from subprocess See merge request BuildStream/buildstream!1199
| * Let subprocess decode stdout based on localebschubert/no-subprocess-decodeBenjamin Schubert2019-03-016-25/+28
|/ | | | | | | | | Subprocesses can return decoded strings if we give them a "universal_newlines=True" argument. We can therefore offload that to them, and not explicitly decode output ourselves. This also fixes multiple bugs where we would not be respecting the locale used by the user, and in some cases force it to "ascii".
* Merge branch 'chandan/remove-redundant-pytest-mark' into 'master'bst-marge-bot2019-03-017-15/+0
|\ | | | | | | | | tests: Remove redundant integration pytest markers See merge request BuildStream/buildstream!1202
| * tests: Remove redundant integration pytest markersChandan Singh2019-03-017-15/+0
|/ | | | | | | | | Remove `pytest.mark.integration` for individual tests that already include it in `pytestmarker` at the top. FWIW the list of such files was found using: git grep --name-only 'pytestmark = pytest.mark.integration' | xargs grep '@pytest.mark.integration'
* Merge branch 'remove-dead-code' into 'master'bst-marge-bot2019-03-0185-361/+265
|\ | | | | | | | | Remove dead code See merge request BuildStream/buildstream!1195
| * tests: Remove unused importsremove-dead-codeBenjamin Schubert2019-03-0143-69/+15
| |
| * tests: Remove unused parameters in functionsBenjamin Schubert2019-03-0154-148/+152
| | | | | | | | | | For parameters that are required as part of an API, prefix them by "_" to make it clear they are unused
| * tests: Remove unused variablesBenjamin Schubert2019-03-0138-144/+98
|/
* Merge branch 'bschubert/more-pythonic-list-concat' into 'master'bst-marge-bot2019-03-0118-34/+37
|\ | | | | | | | | refactor: Use [a, b, *c] instead of [a, b] + c when building list See merge request BuildStream/buildstream!1200
| * Use [a, b, *c] instead of [a, b] + c when building listbschubert/more-pythonic-list-concatBenjamin Schubert2019-03-0118-34/+37
|/ | | | | This pattern is available from python3.5 and provides a simpler understanding of what is going on
* Merge branch 'aevri/unused_color' into 'master'bst-marge-bot2019-03-011-3/+3
|\ | | | | | | | | runcli: rm unused 'color' and '**extra' params See merge request BuildStream/buildstream!1198
| * runcli: rm unused 'color' and '**extra' paramsAngelos Evripiotis2019-03-011-3/+3
|/ | | | | | | | | | | The only customer for 'invoke' is the 'run' method on the same class. Make it private and simplify it's interface. Avoid disappointment by removing unused 'color' and 'extra' parameters from 'invoke'. These appear to have been here to mirror the interface of click.testing.CliRunner.invoke(), which this class replaced.
* Merge branch 'aevri/nonecach2' into 'master'bst-marge-bot2019-03-011-2/+3
|\ | | | | | | | | | | | | cascache: limit 'infinite' cache to volume size Closes #906 See merge request BuildStream/buildstream!1196
| * cascache: limit 'infinite' cache to volume sizeAngelos Evripiotis2019-03-011-2/+3
|/ | | | | | | | | | | Make sure we don't exceed the size of the volume when calculating an 'unlimited' quota. Note that at present, cache_size is an estimate that will be the same as the cache_size or larger. In my own case, I have a volume dedicated to the cache, so it was easy for the cache_quota to exceed this. This prevented me from running BuildStream, although I had plenty of free space.
* Merge branch 'aevri/unused_commandname' into 'master'bst-marge-bot2019-03-011-3/+3
|\ | | | | | | | | buildelement: rm unused 'cmd_name' argument See merge request BuildStream/buildstream!1197