summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _basecache.py: Raise appropriate error if 'artifacts' is not dict/list of dictsshared/yaml-reworkJames Ennis2019-03-271-9/+10
|
* _cachekey.py _artifactelement.py: Create new is_key() and use itDaniel Silverstone2019-03-272-2/+27
| | | | | | | | To more cleanly check if a given string could be a cache key, create a new _cachekey.is_key() function and use that in _artifactelement's verify_artifact_ref() function. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* The new YAML World OrderDaniel Silverstone2019-03-2754-1027/+1774
| | | | | | | | | Replace YAML internals with a new Node type, and refactor everything to use it cleanly. This work was also by James Ennis <james.ennis@codethink.co.uk> Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* YAML Cache: Remove the YAML CacheDaniel Silverstone2019-03-275-572/+21
| | | | | | | The new YAML world order doesn't need the YAML cache, so remove it before we begin the change. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'juerg/partial-cas-remote' into 'master'bst-marge-bot2019-03-275-220/+177
|\ | | | | | | | | Initial partial CAS support for remote execution See merge request BuildStream/buildstream!1232
| * cascache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-17/+0
| |
| * _artifactcache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-24/+0
| |
| * tests/artifactcache/push.py: Remove push_directory() testJürg Billeter2019-03-271-108/+0
| | | | | | | | push_directory() is unused and will be removed.
| * _sandboxremote.py: Fetch missing blobs from artifact serversJürg Billeter2019-03-271-4/+28
| | | | | | | | | | | | | | | | | | | | If any blobs required for the action are missing both on the remote execution CAS server and in the local cache, attempt to fetch these blobs from the configured artifact servers. Artifacts are currently still guaranteed to be complete in the local cache. However, this is in preparation for the planned partial CAS support.
| * _artifactcache.py: Add fetch_missing_blobs() methodJürg Billeter2019-03-271-0/+21
| |
| * cascache.py: Make send_blobs() publicJürg Billeter2019-03-271-7/+15
| |
| * cascache.py: Do not fetch files in _fetch_directory()Jürg Billeter2019-03-271-18/+21
| | | | | | | | | | Separate fetching of directory objects and fetching of files in preparation of partial CAS support.
| * cascache.py: Add fetch_blobs() methodJürg Billeter2019-03-271-2/+42
| |
| * cascache.py: Add local_missing_blobs() methodJürg Billeter2019-03-271-0/+17
| |
| * cascache.py: Add remote_missing_blobs_for_directory() methodJürg Billeter2019-03-271-19/+33
| | | | | | | | Extracted from _send_directory().
| * casremote.py: Add missing_blobs parameter to _CASBatchRead.send()Jürg Billeter2019-03-271-3/+7
| | | | | | | | | | This optionally returns a list of missing blobs instead of raising an error.
| * casremote.py: Remove unused verify_digest_on_remote() methodJürg Billeter2019-03-271-22/+0
| |
| * _sandboxremote.py: Remove call to verify_digest_on_remote()Jürg Billeter2019-03-271-3/+0
|/ | | | | | If upload is not successful, there should be an error. Skip the extra round trip verifying the upload of the top-level directory object was successful.
* Merge branch 'aevri/copyright' into 'master'bst-marge-bot2019-03-271-0/+19
|\ | | | | | | | | testutils/yaml.py: add copyright notice See merge request BuildStream/buildstream!1263
| * testutils/yaml.py: add copyright noticeAngelos Evripiotis2019-03-271-0/+19
|/
* Merge branch 'aevri/junction_load_provenance' into 'master'bst-marge-bot2019-03-2610-19/+116
|\ | | | | | | | | loader: provenance in LoadErrors from _get_loader See merge request BuildStream/buildstream!1237
| * tests: Add test_invalid_junctiondep_not_a_junctionAngelos Evripiotis2019-03-262-0/+17
| | | | | | | | Test that we error correctly when we junction-depend on a non-junction.
| * tests: ensure provenance for _get_loader errorsAngelos Evripiotis2019-03-265-9/+52
| | | | | | | | | | | | | | | | | | | | | | | | Make the tests that currently cover _get_loader ensure that we are getting the expected provenance. Note that for some tests, we must use yaml_file_get_provenance, as the generated yaml is not stable across versions of ruamel. In later work we may replace all instances of provenance string tests with yaml_file_get_provenance, as it will be more robust to future changes.
| * loader: provenance in LoadErrors from _get_loaderAngelos Evripiotis2019-03-261-9/+17
| | | | | | | | | | | | | | | | | | | | | | | | Give better error messages to the user by including provenance. Pass 'provenance' through to places where we raise exceptions in _get_loader. For example, here you can see the provenance information now included (the bit before the ':'): callHello.bst [line 8 column 4]: Could not find the project.conf file in the project referred to by junction element 'hello-junction.bst'.
| * test: Add yaml_file_get_provenanceAngelos Evripiotis2019-03-262-0/+29
| | | | | | | | | | | | Add a new helper - testutils.yaml_file_get_provenance, this will let us make less fragile assertions about the presence of provenance in BuildStream output.
| * buildstream/_yaml: Provenance, fix type in doc strAngelos Evripiotis2019-03-261-1/+1
|/
* Merge branch 'mablanch/629-remote-execution-test' into 'master'Jürg Billeter2019-03-2512-7/+371
|\ | | | | | | | | Add remote execution tests to the CI pipeline See merge request BuildStream/buildstream!1239
| * Generate and publish HTML coverage reportMartin Blanchard2019-03-252-3/+9
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * .gitlab-ci.yml: Add a CI test job for remote executionMartin Blanchard2019-03-253-0/+99
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * tests: Add basic remote-execution tests (autotools)Martin Blanchard2019-03-256-0/+111
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * runcli.py: Add a CLI test fixture for remote-executionMartin Blanchard2019-03-253-2/+135
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * tests: Add a --remote-execution CLI optionsMartin Blanchard2019-03-251-2/+17
|/ | | | | | | | Unlike the --integration option that activates additional tests marked with 'integration', this new --remote-execution option deactivates all the tests except those marked with 'remoteexecution'. https://gitlab.com/BuildStream/buildstream/issues/629
* Merge branch 'raoul/440-source-cache-remotes' into 'master'bst-marge-bot2019-03-2526-78/+956
|\ | | | | | | | | | | | | Remote source cache Closes #440 See merge request BuildStream/buildstream!1214
| * PushQueue -> ArtifactPushQueueRaoul Hidalgo Charman2019-03-253-5/+5
| | | | | | | | | | | | Rename to be consistent with SourcePushQueue Part of #440
| * tests: check source push for workspaced elementsRaoul Hidalgo Charman2019-03-251-0/+40
| | | | | | | | | | | | | | Workspaced elements should not push their sources as consistency is marked as cached even if the sources are not necessarily available. Part of #440
| * tests: check sources are pushed even if build failsRaoul Hidalgo Charman2019-03-254-0/+100
| | | | | | | | | | | | | | Adds a plugin build element that always fails. The test tries to build an element of this kind. Part of #440
| * tests: source and artifact cache testRaoul Hidalgo Charman2019-03-251-0/+91
| | | | | | | | | | | | Ensures that remote source cache and artifact caches work fine together. Part of #440
| * using_config.rst: Add source cache documentationRaoul Hidalgo Charman2019-03-251-0/+46
| | | | | | | | Part of #440
| * format_project.rst: Add source cache documentationRaoul Hidalgo Charman2019-03-251-0/+27
| | | | | | | | Part of #440
| * tests: Add source cache push failure testRaoul Hidalgo Charman2019-03-251-0/+42
| | | | | | | | Part of #440
| * tests: Add source cache pull after a push testRaoul Hidalgo Charman2019-03-251-0/+45
| | | | | | | | Part of #440
| * tests: Add source cache push testRaoul Hidalgo Charman2019-03-251-0/+96
| | | | | | | | Part of #440
| * _stream.py: Schedule source push queueRaoul Hidalgo Charman2019-03-251-1/+7
| | | | | | | | Part of #440
| * sourcepushqueue.py: Add queue for pushing sourcesRaoul Hidalgo Charman2019-03-256-6/+49
| | | | | | | | | | | | | | Updates other queues and widget to have consistent naming and display, and tests have been updated to reflect this. Part of #440
| * element.py: Add source push methods to ElementRaoul Hidalgo Charman2019-03-251-0/+19
| | | | | | | | Part of #440
| * _sourcecache.py: Add push methodRaoul Hidalgo Charman2019-03-252-1/+36
| | | | | | | | Part of #440
| * tests: Add source cache fetch failure testRaoul Hidalgo Charman2019-03-251-0/+52
| | | | | | | | Part of #440
| * tests: Add source cache test for fallbackRaoul Hidalgo Charman2019-03-251-0/+59
| | | | | | | | | | | | | | Checks buildstream calls fetch method when remote source cache does not have a source. Part of #440
| * tests: Add source cache fetch testRaoul Hidalgo Charman2019-03-251-0/+111
| | | | | | | | Part of #440
| * element.py: Pull from sourcecache in _fetch()Raoul Hidalgo Charman2019-03-252-9/+17
| | | | | | | | Part of #440
View Lorry Controller Status