summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* lint: Fix or silence 'cyclic-import' errors and enable pylint for itbschubert/lint/cyclic-importBenjamin Schubert2019-03-2920-36/+38
| | | | | | | | Cyclic imports can be confusing because the order in which we import dependencies can make the import fail or not. We should not rely on ordering of imports for our code. This fixes everywhere possible the imports and silence explicitely some which are not convenient or would require big refactors
* Merge branch 'tpollard/945' into 'master'bst-marge-bot2019-03-297-55/+129
|\ | | | | | | | | | | | | Add initial TestArtifact() abstraction class to testutils Closes #945 See merge request BuildStream/buildstream!1252
| * plugintestutils/runcli.py: Add TestArtifact() 'abstraction' classTom Pollard2019-03-295-48/+123
| | | | | | | | | | | | | | | | | | This adds the initial Artifact 'abstractions', taking existing test methods into a dedicated Artifact class. The class is accessed via an instance attribute of the generic Cli() fixture. This is in preperation for AaaP, to ensure existing tests pass whilst creating an entry point for further abstractions when necessary.
| * _cas/cascache.py: Add opt check_exists arg to _reachable_refs_dir()Tom Pollard2019-03-292-7/+6
|/ | | | | | | | | | | By default the reachable set() includes all file digests from a given reachable directory/tree digest, however this does not guarantee the file objects are actually reachable. testutils/artifactshare.py has_artifact() ascertained if a given artifact is available by ensuring any 'reachable' digests are actually on the filesystem, this is made redudant with the optional check_exists objpath check during the reachable set creation.
* Merge branch 'tpollard/pin-sphinx' into 'master'bst-marge-bot2019-03-291-1/+1
|\ | | | | | | | | tox.ini: pin [testenv:docs] sphinx dependency to 1.8.5 See merge request BuildStream/buildstream!1267
| * tox.ini: pin [testenv:docs] sphinx dependency to 1.8.5Tom Pollard2019-03-291-1/+1
|/ | | | | | In sphinx 2.0.0 sphinx.apidoc has moved to sphinx.ext.apidoc, which causes a build import error. Pin the version to 1.8.5 to allow docs to be built until issues can be fixed.
* Merge branch 'jjardon/overnigth_tests_fix' into 'master'Javier Jardón2019-03-281-1/+1
|\ | | | | | | | | | | | | .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth tests Closes #976 See merge request BuildStream/buildstream!1265
| * .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth testsJavier Jardón2019-03-281-1/+1
|/
* Merge branch 'jennis/track_is_overworking' into 'master'bst-marge-bot2019-03-281-0/+40
|\ | | | | | | | | Write test to ensure that track does not needlessly rewrite See merge request BuildStream/buildstream!1235
| * tests: Add a track test to ensure that we do not needlessly overwrite filesjennis/track_is_overworkingJames Ennis2019-03-281-0/+40
|/
* Merge branch 'shared/yaml-rework' into 'master'bst-marge-bot2019-03-2857-1599/+1821
|\ | | | | | | | | | | | | YAML New World Order Closes #591 See merge request BuildStream/buildstream!1257
| * _basecache.py: Raise appropriate error if 'artifacts' is not dict/list of dictsshared/yaml-reworkJames Ennis2019-03-271-9/+10
| |
| * _cachekey.py _artifactelement.py: Create new is_key() and use itDaniel Silverstone2019-03-272-2/+27
| | | | | | | | | | | | | | | | To more cleanly check if a given string could be a cache key, create a new _cachekey.is_key() function and use that in _artifactelement's verify_artifact_ref() function. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * The new YAML World OrderDaniel Silverstone2019-03-2754-1027/+1774
| | | | | | | | | | | | | | | | | | Replace YAML internals with a new Node type, and refactor everything to use it cleanly. This work was also by James Ennis <james.ennis@codethink.co.uk> Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * YAML Cache: Remove the YAML CacheDaniel Silverstone2019-03-275-572/+21
|/ | | | | | | The new YAML world order doesn't need the YAML cache, so remove it before we begin the change. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'juerg/partial-cas-remote' into 'master'bst-marge-bot2019-03-275-220/+177
|\ | | | | | | | | Initial partial CAS support for remote execution See merge request BuildStream/buildstream!1232
| * cascache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-17/+0
| |
| * _artifactcache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-24/+0
| |
| * tests/artifactcache/push.py: Remove push_directory() testJürg Billeter2019-03-271-108/+0
| | | | | | | | push_directory() is unused and will be removed.
| * _sandboxremote.py: Fetch missing blobs from artifact serversJürg Billeter2019-03-271-4/+28
| | | | | | | | | | | | | | | | | | | | If any blobs required for the action are missing both on the remote execution CAS server and in the local cache, attempt to fetch these blobs from the configured artifact servers. Artifacts are currently still guaranteed to be complete in the local cache. However, this is in preparation for the planned partial CAS support.
| * _artifactcache.py: Add fetch_missing_blobs() methodJürg Billeter2019-03-271-0/+21
| |
| * cascache.py: Make send_blobs() publicJürg Billeter2019-03-271-7/+15
| |
| * cascache.py: Do not fetch files in _fetch_directory()Jürg Billeter2019-03-271-18/+21
| | | | | | | | | | Separate fetching of directory objects and fetching of files in preparation of partial CAS support.
| * cascache.py: Add fetch_blobs() methodJürg Billeter2019-03-271-2/+42
| |
| * cascache.py: Add local_missing_blobs() methodJürg Billeter2019-03-271-0/+17
| |
| * cascache.py: Add remote_missing_blobs_for_directory() methodJürg Billeter2019-03-271-19/+33
| | | | | | | | Extracted from _send_directory().
| * casremote.py: Add missing_blobs parameter to _CASBatchRead.send()Jürg Billeter2019-03-271-3/+7
| | | | | | | | | | This optionally returns a list of missing blobs instead of raising an error.
| * casremote.py: Remove unused verify_digest_on_remote() methodJürg Billeter2019-03-271-22/+0
| |
| * _sandboxremote.py: Remove call to verify_digest_on_remote()Jürg Billeter2019-03-271-3/+0
|/ | | | | | If upload is not successful, there should be an error. Skip the extra round trip verifying the upload of the top-level directory object was successful.
* Merge branch 'aevri/copyright' into 'master'bst-marge-bot2019-03-271-0/+19
|\ | | | | | | | | testutils/yaml.py: add copyright notice See merge request BuildStream/buildstream!1263
| * testutils/yaml.py: add copyright noticeAngelos Evripiotis2019-03-271-0/+19
|/
* Merge branch 'aevri/junction_load_provenance' into 'master'bst-marge-bot2019-03-2610-19/+116
|\ | | | | | | | | loader: provenance in LoadErrors from _get_loader See merge request BuildStream/buildstream!1237
| * tests: Add test_invalid_junctiondep_not_a_junctionAngelos Evripiotis2019-03-262-0/+17
| | | | | | | | Test that we error correctly when we junction-depend on a non-junction.
| * tests: ensure provenance for _get_loader errorsAngelos Evripiotis2019-03-265-9/+52
| | | | | | | | | | | | | | | | | | | | | | | | Make the tests that currently cover _get_loader ensure that we are getting the expected provenance. Note that for some tests, we must use yaml_file_get_provenance, as the generated yaml is not stable across versions of ruamel. In later work we may replace all instances of provenance string tests with yaml_file_get_provenance, as it will be more robust to future changes.
| * loader: provenance in LoadErrors from _get_loaderAngelos Evripiotis2019-03-261-9/+17
| | | | | | | | | | | | | | | | | | | | | | | | Give better error messages to the user by including provenance. Pass 'provenance' through to places where we raise exceptions in _get_loader. For example, here you can see the provenance information now included (the bit before the ':'): callHello.bst [line 8 column 4]: Could not find the project.conf file in the project referred to by junction element 'hello-junction.bst'.
| * test: Add yaml_file_get_provenanceAngelos Evripiotis2019-03-262-0/+29
| | | | | | | | | | | | Add a new helper - testutils.yaml_file_get_provenance, this will let us make less fragile assertions about the presence of provenance in BuildStream output.
| * buildstream/_yaml: Provenance, fix type in doc strAngelos Evripiotis2019-03-261-1/+1
|/
* Merge branch 'mablanch/629-remote-execution-test' into 'master'Jürg Billeter2019-03-2512-7/+371
|\ | | | | | | | | Add remote execution tests to the CI pipeline See merge request BuildStream/buildstream!1239
| * Generate and publish HTML coverage reportMartin Blanchard2019-03-252-3/+9
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * .gitlab-ci.yml: Add a CI test job for remote executionMartin Blanchard2019-03-253-0/+99
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * tests: Add basic remote-execution tests (autotools)Martin Blanchard2019-03-256-0/+111
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * runcli.py: Add a CLI test fixture for remote-executionMartin Blanchard2019-03-253-2/+135
| | | | | | | | https://gitlab.com/BuildStream/buildstream/issues/629
| * tests: Add a --remote-execution CLI optionsMartin Blanchard2019-03-251-2/+17
|/ | | | | | | | Unlike the --integration option that activates additional tests marked with 'integration', this new --remote-execution option deactivates all the tests except those marked with 'remoteexecution'. https://gitlab.com/BuildStream/buildstream/issues/629
* Merge branch 'raoul/440-source-cache-remotes' into 'master'bst-marge-bot2019-03-2526-78/+956
|\ | | | | | | | | | | | | Remote source cache Closes #440 See merge request BuildStream/buildstream!1214
| * PushQueue -> ArtifactPushQueueRaoul Hidalgo Charman2019-03-253-5/+5
| | | | | | | | | | | | Rename to be consistent with SourcePushQueue Part of #440
| * tests: check source push for workspaced elementsRaoul Hidalgo Charman2019-03-251-0/+40
| | | | | | | | | | | | | | Workspaced elements should not push their sources as consistency is marked as cached even if the sources are not necessarily available. Part of #440
| * tests: check sources are pushed even if build failsRaoul Hidalgo Charman2019-03-254-0/+100
| | | | | | | | | | | | | | Adds a plugin build element that always fails. The test tries to build an element of this kind. Part of #440
| * tests: source and artifact cache testRaoul Hidalgo Charman2019-03-251-0/+91
| | | | | | | | | | | | Ensures that remote source cache and artifact caches work fine together. Part of #440
| * using_config.rst: Add source cache documentationRaoul Hidalgo Charman2019-03-251-0/+46
| | | | | | | | Part of #440
| * format_project.rst: Add source cache documentationRaoul Hidalgo Charman2019-03-251-0/+27
| | | | | | | | Part of #440
View Lorry Controller Status