summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _messenger.py: Make `timed_suspendable` public and use it in job.pybschubert/timed-suspendableBenjamin Schubert2020-08-222-50/+39
| | | | This reduces the amount of code duplication
* Merge branch 'tristan/build-root-cache-key' into 'master'Tristan Van Berkom2020-08-2131-30/+42
|\ | | | | | | | | | | | | Consider %{build-root} in the cache key unconditionally Closes #1386 See merge request BuildStream/buildstream!2036
| * buildstream/element.py: Consider "build-root" variable in the cache key ↵tristan/build-root-cache-keyTristan van Berkom2020-08-2029-28/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | unconditionally Not all elements use the "build-root" variable, but it is the standard variable to use for the build directory, and the build directory must be considered in the cache key. Handling this unilaterally in the core is safer than delegating this to element implementations, as we have less chance of plugin authors missing this detail and possibly introducing binary variance for artifacts where only the build directory differs (something which happens when the project name or element names change). This commit also updates the hard coded cache keys in the cache key test, so as to ensure every commit passes it's own tests. This fixes #1386.
| * tests/cachekey/cachekey.py: Use the same project name for cache key testsTristan van Berkom2020-08-201-1/+1
| | | | | | | | | | | | | | | | In the test_cache_key_fatal_warnings() test, use the same project name in both generated project directories in order to pass the tests in the case that elements are guaranteed to have differing cache keys for differeing element/project names (which is the case when we consider the "build-root" in the cache key unconditionally).
| * testing/_cachekeys.py: Fix formatting in cache key failure assertionTristan van Berkom2020-08-201-1/+1
|/
* Merge branch 'chandan/1.93.5' into 'master'1.93.5Chandan Singh2020-08-191-0/+8
|\ | | | | | | | | NEWS: Prepare for next development snapshot - 1.93.5 See merge request BuildStream/buildstream!2029
| * NEWS: Prepare for next development snapshot - 1.93.5chandan/1.93.5Chandan Singh2020-08-191-0/+8
|/ | | | | Among other things, this will allow BuildStream plugins that rely on our `testing` module to update to more recent pytest versions.
* Merge branch 'chandan/upgrade-ruamel' into 'master'bst-marge-bot2020-08-193-1/+37
|\ | | | | | | | | | | | | Bump minimum version requirement for ruamel.yaml Closes #1265 See merge request BuildStream/buildstream!2028
| * Bump minimum version requirement for ruamel.yamlchandan/upgrade-ruamelChandan Singh2020-08-183-1/+37
|/ | | | | | | | | | | | ruamel.yaml <= 0.16.6 suffers from a bug where ruamel's yaml dumper crashes when used on a sequence that has comments before it. In BuildStream, this manifests in form of issues like #1265. See upstream issue at https://sourceforge.net/p/ruamel-yaml/tickets/335. Also, add a regression test for it. Fixes #1265.
* Merge branch 'tristan/docs-fixes' into 'master'Tristan Van Berkom2020-08-172-5/+8
|\ | | | | | | | | Minor documentation fixes See merge request BuildStream/buildstream!2034
| * plugin.py: Fix documentation reference to MappingNode.validate_keys()tristan/docs-fixesTristan van Berkom2020-08-171-1/+1
| |
| * doc/source/format_declaring.rst: Documenting the ``all`` dependency typeTristan van Berkom2020-08-171-4/+7
|/ | | | | | The loader has supported dependencies explicitly being ``all`` for the dependency type (instead of only "build", "runtime" and unspecified) for a long time, but this was not properly documented.
* Merge branch 'chandan/cli-track-deps' into 'master'bst-marge-bot2020-08-1435-29/+96
|\ | | | | | | | | _frontend/cli: Add --deps `build` & `run` values for `source track --deps` See merge request BuildStream/buildstream!2027
| * NEWS: Add entry for `source track --deps build|run`chandan/cli-track-depsChandan Singh2020-08-141-0/+1
| |
| * Re-generate man pagesChandan Singh2020-08-1425-26/+26
| |
| * _frontend/cli: Add --deps `build` & `run` values for `source track --deps`Chandan Singh2020-08-149-3/+69
|/ | | | | | | This is part of #1349. This patch will conclude the first part of that issue, i.e. ensuring that the possible options for `--deps` are consistent across all commands (with the exception of `--deps plan` that we will handle separately).
* Merge branch 'dwinship/test_for_dot_bst_artifact_checkout_bug' into 'master'bst-marge-bot2020-08-141-0/+24
|\ | | | | | | | | regression-test for ".bst" artifact checkout bug See merge request BuildStream/buildstream!2014
| * regression-test for ".bst" artifact checkout bugdwinship/test_for_dot_bst_artifact_checkout_bugDouglas Winship2020-08-131-0/+24
|/
* Merge branch 'juerg/remote-asset' into 'master'bst-marge-bot2020-08-1334-2519/+3952
|\ | | | | | | | | | | | | Use Remote Asset API for artifact and source caches Closes #1273 and #1272 See merge request BuildStream/buildstream!1978
| * NEWS: Add note about the switch to the Remote Asset APIJürg Billeter2020-08-131-0/+8
| |
| * doc: Update for Remote Asset APIJürg Billeter2020-08-133-11/+10
| |
| * casremote.py: Migrate to GetInstanceNameForRemotes()Jürg Billeter2020-08-131-7/+17
| | | | | | | | GetInstanceNameForRemote() has been deprecated.
| * _protos: Update local_cas.proto from buildbox-commonJürg Billeter2020-08-133-8/+257
| |
| * _protos: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-139-809/+5
| | | | | | | | Replaced by Remote Asset API Fetch and Push services.
| * casserver.py: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-132-325/+0
| | | | | | | | Replaced by Remote Asset API Fetch and Push services.
| * _assetcache.py: Drop index_remote_class and storage_remote_classJürg Billeter2020-08-133-11/+6
| | | | | | | | They no longer differ among subclasses.
| * _sourcecache.py: Use AssetRemoteJürg Billeter2020-08-132-113/+48
| | | | | | | | | | This migrates the source cache from the BuildStream Source protocol to the Remote Asset API.
| * _artifactcache.py: Use AssetRemoteJürg Billeter2020-08-132-187/+117
| | | | | | | | | | | | | | This migrates the artifact cache from the BuildStream Artifact protocol to the Remote Asset API. Co-authored-by: Sander Striker <s.striker@striker.nl>
| * _assetcache.py: Add fetch_directory() and push_directory() methodsJürg Billeter2020-08-131-0/+78
| |
| * _assetcache.py: Add AssetRemote for Remote Asset APIJürg Billeter2020-08-131-3/+147
| | | | | | | | Co-authored-by: Sander Striker <s.striker@striker.nl>
| * tests/frontend/push.py: Pull complete artifactJürg Billeter2020-08-131-7/+8
| | | | | | | | | | | | | | | | Use a separate Cli instance with a separate local cache for the second pull in `test_recently_pulled_artifact_does_not_expire()` to ensure the complete artifact is pulled. If only a part of the artifact is pulled, there is no guarantee that the other blobs of that artifact won't expire.
| * Rename CacheError to AssetCacheErrorJürg Billeter2020-08-133-8/+8
| |
| * Rename BaseCache to AssetCacheJürg Billeter2020-08-133-7/+7
| |
| * Remove unused BaseCache._spec_errorJürg Billeter2020-08-133-3/+0
| |
| * casserver.py: Add proxy for Remote Asset APIJürg Billeter2020-08-131-0/+50
| |
| * casdprocessmanager.py: Add stubs for the Remote Asset APIJürg Billeter2020-08-131-0/+25
| |
| * Add generated python protobuffer files Remote Asset APISander Striker2020-08-134-0/+990
| |
| * Add Remote Asset API protoSander Striker2020-08-131-0/+445
| |
| * _protos: Update generated filesJürg Billeter2020-08-137-1251/+1957
|/ | | | Use grpcio-tools 1.28.1, matching grpcio from requirements.txt
* Merge branch 'tristan/meta-element-death' into 'master'bst-marge-bot2020-08-1315-2297/+2171
|\ | | | | | | | | Kill MetaElement See merge request BuildStream/buildstream!2021
| * doc: Updating architecture overview to remove mentions of MetaElementtristan/meta-element-deathTristan van Berkom2020-08-132-1964/+1959
| |
| * Completely remove MetaElementTristan van Berkom2020-08-1311-325/+206
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This dramatically affects the load process and removes one hoop we had to jump through, which is the creation of the extra intermediate MetaElement objects. This allows us to more easily carry state discovered by the Loader over to the Element constructor, as we need not add additional state to the intermediate MetaElement for this. Instead we have the Element initializer understand the LoadElement directly. Summary of changes: * _loader/metaelement.py: Removed * _loader/loadelement.py: Added some attributes previously required on MetaElement * _loader/loader.py: Removed _collect_element() and collect_element_no_deps(), removing the process of Loader.load() which translates LoadElements into MetaElements completely. * _loader/init.py: Export LoadElement, Dependency and Symbol types, stop exporting MetaElement * _loader/metasource.py: Now take the 'first_pass' parameter as an argument * _artifactelement.py: Use a virtual LoadElement instead of a virtual MetaElement to instantiate the ArtifactElement objects. * _pluginfactory/elementfactory.py: Adjust to now take a LoadElement * _project.py: Adjust Project.create_element() to now take a LoadElement, and call the new Element._new_from_load_element() instead of the old Element._new_from_meta() function * element.py: - Now export Element._new_from_load_element() instead of Element._new_from_meta() - Adjust the constructor to do the LoadElement toplevel node parsing instead of expecting members on the MetaElement object - Added __load_sources() which parses out and creates MetaSource objects for the sake of instantiating the element's Source objects. Consequently this simplifies the scenario where workspaces are involved. * source.py: Adjusted to use the new `first_pass` parameter to MetaSource when creating a duplicate clone.
| * element.py: Early return in _initialize_state()Tristan van Berkom2020-08-133-8/+6
|/ | | | | | | | | | | | | | | | | | | | Instead of having an assertion here, lets just have an early return and make the __resolved_initial_state variable internal private (with two leading underscores). We also stop checking for it in _pipeline.py before resolving state. Some background: * We only defer _initialize_state() to a later stage because it can be a resource intensive task which interrogates the disk or the local CAS, thus we have the Pipeline iterate over the instantiated elements and resolve them separately for better user feedback. * Some "first pass" elements must have their state initialized earlier, i.e. the "link" and "junction" elements need to be usable during the load sequence.
* Merge branch 'juerg/ci' into 'master'bst-marge-bot2020-08-122-35/+24
|\ | | | | | | | | CI: Update BuildBox to 0.0.14, add Fedora 32 See merge request BuildStream/buildstream!2025
| * .gitlab-ci.yml: Drop tests-python-3.8-buster jobJürg Billeter2020-08-121-13/+0
| | | | | | | | | | Python 3.8 is now covered by Fedora 32. There is no longer a need to use a special Debian image for Python 3.8.
| * .gitlab-ci.yml: Add Fedora 32Jürg Billeter2020-08-121-10/+12
| | | | | | | | Test coverage is not available yet with Python 3.8 (#1173).
| * doc/source/main_install.rst: Update BuildBox components to 0.0.14Jürg Billeter2020-08-121-8/+8
| |
| * .gitlab-ci.yml: Update BuildBox components to 0.0.14Jürg Billeter2020-08-121-5/+5
|/
* Merge branch 'juerg/ci' into 'master'bst-marge-bot2020-08-123-3/+7
|\ | | | | | | | | CI fixes for Python 3.8 and Breezy See merge request BuildStream/buildstream!2024
| * .gitlab-ci.yml: Replace py38 with py38-nocover in TOXENVJürg Billeter2020-08-121-1/+1
| | | | | | | | Coverage is still broken with Python 3.8 (#1173).
View Lorry Controller Status