summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* regression-test for ".bst" artifact checkout bugdwinship/test_for_dot_bst_artifact_checkout_bugDouglas Winship2020-08-131-0/+24
|
* Merge branch 'juerg/remote-asset' into 'master'bst-marge-bot2020-08-1334-2519/+3952
|\ | | | | | | | | | | | | Use Remote Asset API for artifact and source caches Closes #1273 and #1272 See merge request BuildStream/buildstream!1978
| * NEWS: Add note about the switch to the Remote Asset APIJürg Billeter2020-08-131-0/+8
| |
| * doc: Update for Remote Asset APIJürg Billeter2020-08-133-11/+10
| |
| * casremote.py: Migrate to GetInstanceNameForRemotes()Jürg Billeter2020-08-131-7/+17
| | | | | | | | GetInstanceNameForRemote() has been deprecated.
| * _protos: Update local_cas.proto from buildbox-commonJürg Billeter2020-08-133-8/+257
| |
| * _protos: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-139-809/+5
| | | | | | | | Replaced by Remote Asset API Fetch and Push services.
| * casserver.py: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-132-325/+0
| | | | | | | | Replaced by Remote Asset API Fetch and Push services.
| * _assetcache.py: Drop index_remote_class and storage_remote_classJürg Billeter2020-08-133-11/+6
| | | | | | | | They no longer differ among subclasses.
| * _sourcecache.py: Use AssetRemoteJürg Billeter2020-08-132-113/+48
| | | | | | | | | | This migrates the source cache from the BuildStream Source protocol to the Remote Asset API.
| * _artifactcache.py: Use AssetRemoteJürg Billeter2020-08-132-187/+117
| | | | | | | | | | | | | | This migrates the artifact cache from the BuildStream Artifact protocol to the Remote Asset API. Co-authored-by: Sander Striker <s.striker@striker.nl>
| * _assetcache.py: Add fetch_directory() and push_directory() methodsJürg Billeter2020-08-131-0/+78
| |
| * _assetcache.py: Add AssetRemote for Remote Asset APIJürg Billeter2020-08-131-3/+147
| | | | | | | | Co-authored-by: Sander Striker <s.striker@striker.nl>
| * tests/frontend/push.py: Pull complete artifactJürg Billeter2020-08-131-7/+8
| | | | | | | | | | | | | | | | Use a separate Cli instance with a separate local cache for the second pull in `test_recently_pulled_artifact_does_not_expire()` to ensure the complete artifact is pulled. If only a part of the artifact is pulled, there is no guarantee that the other blobs of that artifact won't expire.
| * Rename CacheError to AssetCacheErrorJürg Billeter2020-08-133-8/+8
| |
| * Rename BaseCache to AssetCacheJürg Billeter2020-08-133-7/+7
| |
| * Remove unused BaseCache._spec_errorJürg Billeter2020-08-133-3/+0
| |
| * casserver.py: Add proxy for Remote Asset APIJürg Billeter2020-08-131-0/+50
| |
| * casdprocessmanager.py: Add stubs for the Remote Asset APIJürg Billeter2020-08-131-0/+25
| |
| * Add generated python protobuffer files Remote Asset APISander Striker2020-08-134-0/+990
| |
| * Add Remote Asset API protoSander Striker2020-08-131-0/+445
| |
| * _protos: Update generated filesJürg Billeter2020-08-137-1251/+1957
|/ | | | Use grpcio-tools 1.28.1, matching grpcio from requirements.txt
* Merge branch 'tristan/meta-element-death' into 'master'bst-marge-bot2020-08-1315-2297/+2171
|\ | | | | | | | | Kill MetaElement See merge request BuildStream/buildstream!2021
| * doc: Updating architecture overview to remove mentions of MetaElementtristan/meta-element-deathTristan van Berkom2020-08-132-1964/+1959
| |
| * Completely remove MetaElementTristan van Berkom2020-08-1311-325/+206
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This dramatically affects the load process and removes one hoop we had to jump through, which is the creation of the extra intermediate MetaElement objects. This allows us to more easily carry state discovered by the Loader over to the Element constructor, as we need not add additional state to the intermediate MetaElement for this. Instead we have the Element initializer understand the LoadElement directly. Summary of changes: * _loader/metaelement.py: Removed * _loader/loadelement.py: Added some attributes previously required on MetaElement * _loader/loader.py: Removed _collect_element() and collect_element_no_deps(), removing the process of Loader.load() which translates LoadElements into MetaElements completely. * _loader/init.py: Export LoadElement, Dependency and Symbol types, stop exporting MetaElement * _loader/metasource.py: Now take the 'first_pass' parameter as an argument * _artifactelement.py: Use a virtual LoadElement instead of a virtual MetaElement to instantiate the ArtifactElement objects. * _pluginfactory/elementfactory.py: Adjust to now take a LoadElement * _project.py: Adjust Project.create_element() to now take a LoadElement, and call the new Element._new_from_load_element() instead of the old Element._new_from_meta() function * element.py: - Now export Element._new_from_load_element() instead of Element._new_from_meta() - Adjust the constructor to do the LoadElement toplevel node parsing instead of expecting members on the MetaElement object - Added __load_sources() which parses out and creates MetaSource objects for the sake of instantiating the element's Source objects. Consequently this simplifies the scenario where workspaces are involved. * source.py: Adjusted to use the new `first_pass` parameter to MetaSource when creating a duplicate clone.
| * element.py: Early return in _initialize_state()Tristan van Berkom2020-08-133-8/+6
|/ | | | | | | | | | | | | | | | | | | | Instead of having an assertion here, lets just have an early return and make the __resolved_initial_state variable internal private (with two leading underscores). We also stop checking for it in _pipeline.py before resolving state. Some background: * We only defer _initialize_state() to a later stage because it can be a resource intensive task which interrogates the disk or the local CAS, thus we have the Pipeline iterate over the instantiated elements and resolve them separately for better user feedback. * Some "first pass" elements must have their state initialized earlier, i.e. the "link" and "junction" elements need to be usable during the load sequence.
* Merge branch 'juerg/ci' into 'master'bst-marge-bot2020-08-122-35/+24
|\ | | | | | | | | CI: Update BuildBox to 0.0.14, add Fedora 32 See merge request BuildStream/buildstream!2025
| * .gitlab-ci.yml: Drop tests-python-3.8-buster jobJürg Billeter2020-08-121-13/+0
| | | | | | | | | | Python 3.8 is now covered by Fedora 32. There is no longer a need to use a special Debian image for Python 3.8.
| * .gitlab-ci.yml: Add Fedora 32Jürg Billeter2020-08-121-10/+12
| | | | | | | | Test coverage is not available yet with Python 3.8 (#1173).
| * doc/source/main_install.rst: Update BuildBox components to 0.0.14Jürg Billeter2020-08-121-8/+8
| |
| * .gitlab-ci.yml: Update BuildBox components to 0.0.14Jürg Billeter2020-08-121-5/+5
|/
* Merge branch 'juerg/ci' into 'master'bst-marge-bot2020-08-123-3/+7
|\ | | | | | | | | CI fixes for Python 3.8 and Breezy See merge request BuildStream/buildstream!2024
| * .gitlab-ci.yml: Replace py38 with py38-nocover in TOXENVJürg Billeter2020-08-121-1/+1
| | | | | | | | Coverage is still broken with Python 3.8 (#1173).
| * tox.ini: Replace py38 with py38-nocover in default envlistJürg Billeter2020-08-121-1/+1
| | | | | | | | | | Coverage is still broken with Python 3.8 (#1173). Don't enable it by default.
| * testing/_utils/site.py: Add BRZ_EMAIL to support Breezy 3.0juerg/fedora-32Jürg Billeter2020-08-121-1/+5
|/ | | | Fedora 32 has replaced Bazaar with Breezy.
* Merge branch 'juerg/fedora-30' into 'master'bst-marge-bot2020-08-121-5/+0
|\ | | | | | | | | .gitlab-ci.yml: Drop tests-fedora-30 job See merge request BuildStream/buildstream!2022
| * .gitlab-ci.yml: Drop tests-fedora-30 jobjuerg/fedora-30Jürg Billeter2020-08-121-5/+0
|/ | | | | | Fedora 30 is end of life since 2020-05-26. https://fedoraproject.org/wiki/End_of_life
* Merge branch 'tcanabrava/cache-server-test' into 'master'bst-marge-bot2020-08-1116-8/+254
|\ | | | | | | | | | | | | CI For the Cache Server Closes #1362 See merge request BuildStream/buildstream!1997
| * Use a template for docker-variablestcanabrava/cache-server-testTomaz Canabrava2020-08-111-8/+8
| |
| * Create template for remote testsWilliam Salmon2020-08-111-13/+6
| | | | | | | | | | This moves boiler plate for remote tests in to a template to make maintenance easier.
| * Test buildstream with a reference cache serverWilliam Salmon2020-08-113-1/+59
| | | | | | | | | | | | | | | | This test defines a docker compose for a buildstream cache server and uses it. This stage could be altered or duplicated to use other cache servers so we can check our continued compatibility.
| * Add tests for checking the working of a remote cacheWilliam Salmon2020-08-1112-0/+195
| | | | | | | | | | These basic tests are ment to be run with a remote cache and can be used to check bst conpatiblity with a remote cache server.
| * Artifacts configuration for test can also pushTomaz Canabrava2020-08-111-1/+1
|/
* Merge branch 'chandan/fix-artifact-checkout' into 'master'Chandan Singh2020-08-112-8/+20
|\ | | | | | | | | | | | | _frontend/cli.py: Fix `artifact checkout` with no arguments Closes #1367 See merge request BuildStream/buildstream!2017
| * _frontend/cli.py: Fix `artifact checkout` with no argumentsChandan Singh2020-08-112-8/+20
|/ | | | | | | | | | | | | | | When no arguments are passed to `artifact checkout` command, it currently crashes. We actually have a check of this, but we crash before that becasue this method tries to read the value of `target` before we've had time to check for it. That can only be done correctly after the app has been initialized. So, refactor that bit of the method to run after we've checked that we are working with a non-empty target. Also, add a regression test for it. Fixes #1367.
* Merge branch 'tristan/loader-dependency-refactor' into 'master'tristan/loader-dependency-refactorbst-marge-bot2020-08-105-234/+249
|\ | | | | | | | | _loader: Use only one Dependency() class See merge request BuildStream/buildstream!2019
| * _loader/loadelement.pyx: Removed an unused line of codeTristan van Berkom2020-08-101-1/+0
| |
| * _loader/loadelement.pyx: Dependency now implements `provenance` as a propertyTristan van Berkom2020-08-101-3/+12
| | | | | | | | | | | | Since ProvenanceInformation are created on the demand, it's better to just hold on to the originating Node and create the provenance only when needed.
| * _loader: Use only one Dependency() classTristan van Berkom2020-08-105-233/+240
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This moves the Dependency() implementation from the _loader.types module into the _loader.loadelement module, replacing the duplicate Dependency() definition there. Instead of creating _loader.loadelement.Dependency objects on the fly during the load process, we simply fill in the new LoadElement member on the original Dependency objects. This refactor should make the code easier to work with, since any originally loaded state from the YAML is preserved in the intermediate LoadElement data model and does not need to be manually handed over to a separate object. Summary of changes: * _loader/types.pyx: Removed Dependency() implementation from here, and renamed to _loader/types.py * _loader/loadelement.py: Replaced the second Dependency object implementation with the original one from _loader/types.py, adding a new LoadElement member which is resolved during the load process via the new Dependency.set_element() API * _loader/loader.py: Instead of creating a second kind of Dependency object on the fly during the load process, simply resolve the existing dependencies with Dependency.set_element() * setup.py: Fixed up cython build instructions, now that _loader/types.py no longer contains any cython code.
* Merge branch 'tristan/fix-redundant-session-headings' into 'master'bst-marge-bot2020-08-1029-158/+338
|\ | | | | | | | | | | | | Refactor session headings in the frontend Closes #1369 See merge request BuildStream/buildstream!2009