summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Remove "tar" pluginjjardon/tar_plugin_moveJavier Jardón2020-01-2745-843/+49
| | | | Use the one from bst-plugins-experimental instead
* Merge branch 'coldtom/testing-api' into 'master'Javier Jardón2020-01-2494-326/+373
|\ | | | | | | | | | | | | Stop plugin tests using private API Closes #1258 See merge request BuildStream/buildstream!1806
| * testing: Add functions to generate yaml filesThomas Coldrick2020-01-2410-136/+116
| | | | | | | | | | | | | | Adds functions to the `buildstream.testing` package to allow plugins to dump elements and projects on the fly. Before this plugins were just accessing the private yaml API for tests and loading/dumping directly. I also allow access to just `_yaml.load()` from testing.
| * exceptions: Expose ErrorDomain, ErrorLoadReasonThomas Coldrick2020-01-2390-190/+257
|/ | | | | | | Plugin tests are already accessing this API, but using imports from private modules. For motivation for this to be exposed publicly, note that ErrorDomain is an argument for most things in runcli.py, and LoadErrorReason may be another.
* Merge branch 'tpollard/shellbuildtree' into 'master'Tom Pollard2020-01-212-30/+89
|\ | | | | | | | | | | | | _frontend/cli.py: Make show() --use-buildtree respect pull semantics Closes #1151 See merge request BuildStream/buildstream!1767
| * _frontend/cli.py: Make show() --use-buildtree respect pull semanticsTom Pollard2020-01-212-30/+89
|/ | | | | | | Ensure that if a buildtree isn't cached locally, it's only fetched if --pull and pull-buildtrees config are set. Also, only attempt to fetch if it's plausible that it could be pulled, with appropriate messaging based on local cached state.
* Merge branch 'juerg/assemble_done' into 'master'Jürg Billeter2020-01-183-9/+24
|\ | | | | | | | | element.py: Optimize assemble_done() See merge request BuildStream/buildstream!1802
| * element.py: Optimize assemble_done()Jürg Billeter2020-01-183-9/+24
|/ | | | | After a successful build we know that the artifact is cached. Avoid querying buildbox-casd and the filesystem.
* Merge branch 'chandan/1.93.0' into 'master'1.93.0Chandan Singh2020-01-171-0/+5
|\ | | | | | | | | NEWS: Add mark for 1.93.0 See merge request BuildStream/buildstream!1804
| * NEWS: Add mark for 1.93.0chandan/1.93.0Chandan Singh2020-01-171-0/+5
|/ | | | Prepare for next release.
* Merge branch 'chandan/plugin-namespace-fix' into 'master'Chandan Singh2020-01-176-8/+19
|\ | | | | | | | | Plugins: Update entrypoint group for pip origin See merge request BuildStream/buildstream!1803
| * doc/sample_plugin: Update name of entry point groupChandan Singh2020-01-171-4/+1
| | | | | | | | | | | | | | | | Update sample plugin documentation based on the recent change to entry point group for plugins. While I'm here, also remove the unnecessary dependency on `setuptools` from the sample plugin.
| * NEWS: Announce new entry point groups for pluginsChandan Singh2020-01-171-0/+4
| |
| * Plugins: Update entrypoint group for pip originChandan Singh2020-01-174-4/+14
|/ | | | | | | | | | | | | | | As discussed in https://mail.gnome.org/archives/buildstream-list/2020-January/msg00001.html, separate the entry point groups for elements and sources. Previously, both used `buildstream.plugins` which meant that it wasn't possible to have an element and a source plugin with the same name, when using `pip` origin. This a breaking change for all plugins, but will only affect projects using the `pip` plugin origin. As part of this, also bump the version of bst-plugins-experimental used in our tests to avoid CI breakages.
* Merge branch 'jjardon/move_deb_source' into 'master'Javier Jardón2020-01-1733-304/+9
|\ | | | | | | | | Remove deb surce plugin, it has beem moved to bst-plugins-experimental See merge request BuildStream/buildstream!1729
| * Remove "deb" surce plugin, it has beem moved to bst-plugins-experimentaljjardon/move_deb_sourceJavier Jardón2020-01-1733-304/+9
|/
* Merge branch 'bschubert/optimize-consistency' into 'master'Benjamin Schubert2020-01-1635-285/+288
|\ | | | | | | | | | | | | Optimize consistency and state handling Closes #1126 See merge request BuildStream/buildstream!1739
| * NEWS: Add news information about Consistency breakagebschubert/optimize-consistencyBenjamin Schubert2020-01-161-0/+21
| |
| * element.py: Stop asserting we are cached after fetch and mark us as suchBenjamin Schubert2020-01-161-9/+4
| | | | | | | | | | | | This assert would only be run in development mode and would actually not mark the source as downloaded. Therefore forcefully mark them as fetched and remove the assert. Our test suite will catch if there is a problem.
| * types.py: Completely remove 'Consistency'Benjamin Schubert2020-01-162-43/+13
| | | | | | | | | | 'Consistency' is not needed anymore, now that we have everything in place to not use it
| * element.py: Optimize _should_fetch conditionBenjamin Schubert2020-01-161-6/+3
| | | | | | | | | | By looking at the flag first, we can avoid expensive checks on whether the element is cached or not.
| * element.py: Compute whether an element is cached only on `is_cached`Benjamin Schubert2020-01-163-23/+20
| | | | | | | | | | | | | | | | | | | | | | This removes the early call to get whether sources are locally cached in `_update_source_state` by delegating it to the call of `is_cached`. Once it is cached, the element is assumed to stay that way for the duration of the pipeline, we can therefore cache the result once it is true. Also remove `Consistency.IS_CACHED`, which is not used anywhere else.
| * source.py: Remove 'get_consistency' completelyBenjamin Schubert2020-01-1622-163/+32
| | | | | | | | | | This is not needed now that we have 'is_resolved' and 'is_cached'. We can therefore drop all calling places and implementations of it.
| * source.py: Remove the reliance on consistency to get whether a source is cachedBenjamin Schubert2020-01-1618-12/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes the need to use consistency in Sources, by asking explicitely whether the source is cached or not. This introduces a new public method on source: `is_cached` that needs implementation and that should return whether the source has a local copy or not. - On fetch, also reset whether the source was cached or set if as cached when we know it was. - Validate the cache's source after fetching it This doesn't need to be run in the scheduler's process and can be offloaded to the child, which will allow better multiprocessing
| * source.py: Add a new 'is_resolved' to get whether a source is resolved.Benjamin Schubert2020-01-1612-16/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `get_consistency` is coarse grained and hard to optimize, in addition to being un-userfriendly. This adds a new `is_resolved` that has for default implementation `get_ref() is not None`, which is true for most sources in BuildStream. Sources for which this is not true can override the method to give a more accurate description. Checking for this before looking whether the source is cached can reduce the amount of work necessary in some pipeline and opens the door for more optimizations and the removal of the source state check.
| * element.py: Remove _get_consistency and introduce explicit methodsBenjamin Schubert2020-01-165-30/+33
| | | | | | | | | | | | | | This replaces the _get_consistency method by two methods: `_has_all_sources_resolved` and `_has_all_sources_cached` which allows a more fine grained control on what information is needed.
| * element.py: Rename '_source_cached' to '_has_all_sources_in_source_cache'Benjamin Schubert2020-01-169-27/+27
| | | | | | | | | | | | '_source_cached' is not explicit enough as it doesn't distinguishes between sources in their respective caches and sources in the global sourcecache.
| * source.py: Introduce methods to query state instead of get_consistencyBenjamin Schubert2020-01-163-9/+19
| | | | | | | | | | | | | | | | `get_consistency` doesn't allow being fine grained and asking only for a specific bit of information. This introduces methods `is_cached` and `is_resolved` which will be more flexible for refactoring.
| * tox.ini: Bump version of bst-plugins-experimental to 0.14.0Benjamin Schubert2020-01-161-2/+2
|/
* Merge branch 'chandan/remove-bst-experimental-unused' into 'master'Chandan Singh2020-01-161-6/+0
|\ | | | | | | | | tests/integration/project.conf: Remove unused external plugins See merge request BuildStream/buildstream!1799
| * tests/integration/project.conf: Remove unused external pluginschandan/remove-bst-experimental-unusedChandan Singh2020-01-151-6/+0
|/ | | | | There are no tests that rely on these plugins anymore, so remove the corresponding declarations from project.conf.
* Merge branch 'tpollard/removeassert' into 'master'Tom Pollard2020-01-151-10/+0
|\ | | | | | | | | | | | | element.py: Remove unused __staged_sources_directory & assert Closes #1249 See merge request BuildStream/buildstream!1798
| * element.py: Remove unused __staged_sources_directory & asserttpollard/removeassertTom Pollard2020-01-151-10/+0
|/
* Merge branch 'tpollard/messagerate' into 'master'Tom Pollard2020-01-154-13/+62
|\ | | | | | | | | Make message & status rendering be tick driven by default See merge request BuildStream/buildstream!1745
| * userconfig.yaml: Add bool for disabling console output throttlingtpollard/messagerateTom Pollard2020-01-154-3/+33
| | | | | | | | Also highlight the change & configuration in NEWS
| * _frontend/app.py: Make message printing tick drivenTom Pollard2020-01-151-12/+31
|/
* Merge branch 'chandan/gitlab-ci-toxenvs' into 'master'Chandan Singh2020-01-141-5/+2
|\ | | | | | | | | .gitlab-ci.yml: Simplify test environment declaration See merge request BuildStream/buildstream!1795
| * .gitlab-ci.yml: Simplify test environment declarationChandan Singh2020-01-141-5/+2
|/ | | | | | | Defining `EXTERNAL_TESTS_COMMAND` as an extra command makes it easy to miss it, especially when adding new environments (as I was recently bitten by it). Instead of that, we can control the environments using `TOXENV` variable.
* Merge branch 'bschubert/optimize-loading-multiple-targets' into 'master'Benjamin Schubert2020-01-142-3/+16
|\ | | | | | | | | loader.py: Optimize sorting of elements when they are multiple targets See merge request BuildStream/buildstream!1794
| * loader.py: Optimize sorting of elements when they are multiple targetsbschubert/optimize-loading-multiple-targetsBenjamin Schubert2020-01-142-3/+16
|/ | | | | | | Currently, with multiple targets, we would go through all the elements in the target and sort them, even though they might already be sorted. This ensures we sort every element only once.
* Merge branch 'bschubert/standardized-tests' into 'master'Benjamin Schubert2020-01-1327-351/+62
|\ | | | | | | | | introduce cross-repo standardized source tests See merge request BuildStream/buildstream!1757
| * tox.ini: Add a external plugins environment test and run it in CIBenjamin Schubert2020-01-134-9/+57
| | | | | | | | | | | | | | | | | | | | | | This runs two versions of the plugins: - The latest stable is not allowed failures and is run on every platform - The master version is allowed failure, and only runs on a single architecture This also adds a new entrypoint to register source tests to run against BuildStream.
| * tox.ini: Move coverage handling in commands_post.Benjamin Schubert2020-01-131-2/+3
| | | | | | | | | | This is not part of the test running itself, but more handling afterwards
| * tox.ini: Remove external plugins testsBenjamin Schubert2020-01-133-91/+3
| | | | | | | | | | These tests don't do anything, as they have no plugins registered at all.
| * tests: Remove some ostree stuffThomas Coldrick2020-01-132-2/+0
| |
| * Tests: Remove bst-plugins-experimental dependencyThomas Coldrick2020-01-1314-99/+2
| | | | | | | | This stops the tests from being interlinked for BuildStream itself.
| * Remove integration tests for moved pluginsThomas Coldrick2020-01-137-150/+0
| |
| * tox.ini: Remove duplicated move of the coverage fileBenjamin Schubert2020-01-131-1/+0
|/
* Merge branch 'juerg/batch-command-logging' into 'master'Jürg Billeter2020-01-135-7/+17
|\ | | | | | | | | _sandboxreapi.py: Improve batch command logging See merge request BuildStream/buildstream!1793
| * tests/integration/workspace.py: Relax assert for buildbox-runjuerg/batch-command-loggingJürg Billeter2020-01-131-2/+2
| | | | | | | | | | When commands are batched, the failure message can't include the command group label with buildbox-run.