summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/format/variables.py: Adding new test for lazy variable resolutiontristan/partial-variablesTristan van Berkom2020-07-226-0/+30
| | | | | | Check that lazy variable resolution allows using variables in junction definitions which would not successfully resolve if we needed to resolve synchronously.
* _variables.pyx, element.py: Lazy variable resolutionTristan van Berkom2020-07-222-5/+2
| | | | | | | | Variables are resolved on demand until Variables.check() is explicitly called, allowing junctions to use variables without requiring that they be completely resolved. Approach is courtesy of Valentin David.
* tests/frontend/overlaps.py: Test undefined variablesTristan van Berkom2020-07-222-1/+27
| | | | | Ensure that we get the expected provenance when expanding a variable included in an overlap whitelist entry.
* tests/format/variables.py: Added some new testsTristan van Berkom2020-07-225-12/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Test scenarios where a junction needs to resolve variables in order to configure a subproject, but where some other variables may be derived from the same subproject. In this scenario we allow partial resolution of variables for junction elements. * Enhanced the undefined variables and circular reference tests to also check for the expected provenances. * Test for deep variable resolution Test variable indirection with 50, 500 and 5000 variables: * 50 - tests generally large indirections in the recursive algorithm, which is limited to 200 recursions * 500 - tests that the non-recursive algorithm works for successful outcomes and not only for error resolution * 5000 - tests that the iterative algorithm works and ensures it is not discarded, as a recursive algorithm cannot be implemented to support this depth with python (which limits itself to merely 1000 stack frames).
* _variables.pyx: Rewrite Variables code.Tristan van Berkom2020-07-224-190/+603
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Main enhancements here include: * Support for deeply nested variable declarations, removing the limitations of the recursive variable resolution algorithm. We were unable to achieve equal performance with the iterative resolution algorithm, so we now have the recursive approach as the fast path and only support 200 recursions with this approach before falling back on the iterative code path, which will support deep variable resolution and take care of error reporting. * Better error reporting for undefined variables. Variables.subst() now requires a ScalarNode and not a simple `str`, making it more difficult for the core to substitute an undefined variable without providing the provenance of where that value expression was declared. Code changes: * _variables.pyx: Complete rewrite * exceptions.py: Added new LoadErrorReason.CIRCULAR_REFERENCE_VARIABLE * element.py: Pass ScalarNode to Variable.subst() when substituting overlap whitelists.
* Merge branch 'abderrahim/timed' into 'master'bst-marge-bot2020-07-181-13/+11
|\ | | | | | | | | element.py: don't use timed activity for staging every artifact See merge request BuildStream/buildstream!1968
| * element.py: don't use timed activity for staging every artifactabderrahim/timedAbderrahim Kitouni2020-07-171-13/+11
|/ | | | | Staging now takes very little time, so we have two lines per artifact with no benefit. Use a STATUS line instead.
* Merge branch 'sotk/update-artifact-server-docs' into 'master'bst-marge-bot2020-07-171-3/+3
|\ | | | | | | | | Remove the suggestion that BuildGrid can be used as an artifact server See merge request BuildStream/buildstream!1998
| * Remove the suggestion that BuildGrid can be used as an artifact serversotk/update-artifact-server-docsAdam Coldrick2020-07-171-3/+3
|/ | | | | | | | | | BuildGrid's CAS can be used as artifact storage, but BuildGrid no longer supports being used as an Artifact Cache, since it doesn't implement the new proto-based approach to artifacts. This commit makes this current situation clear in the docs, rather than incorrectly suggesting the use of BuildGrid for the `index` and `both` cache types.
* Merge branch 'chandan/spring-clean' into 'master'bst-marge-bot2020-07-171-3/+1
|\ | | | | | | | | .gitlab-ci.yml: Minor spring cleaning See merge request BuildStream/buildstream!1996
| * .gitlab-ci.yml: Run missing-deps as a normal userChandan Singh2020-07-171-1/+1
| | | | | | | | The buildstream user was previously created but was not used.
| * .gitlab-ci.yml: Remove undefined PLUGINS_TESTS_COMMANDChandan Singh2020-07-171-2/+0
|/ | | | | This variable is undefined, and was probably missed during some previous cleanup. At present, this acts as a no-op.
* Merge branch 'dwinship/remove-Aarch64-from-overnights' into 'master'bst-marge-bot2020-07-171-11/+0
|\ | | | | | | | | Remove Aarch64 from overnights See merge request BuildStream/buildstream!1994
| * Remove Aarch64 from overnightsWilliam Salmon2020-07-171-11/+0
|/ | | | This stage has been removed as we dont have docker images for it any more.
* Merge branch 'abderrahim/loader-workspace' into 'master'bst-marge-bot2020-07-162-3/+6
|\ | | | | | | | | loader.py: use the full name of the element when looking up a workspace See merge request BuildStream/buildstream!1971
| * loader.py: use the full name of the element when looking up a workspaceAbderrahim Kitouni2020-07-162-3/+6
|/ | | | | This fixes a bug where buildstream would ignore the opened workspace on a cross-junction element.
* Merge branch 'chandan/remove-outdated-comment' into 'master'bst-marge-bot2020-07-151-1/+0
|\ | | | | | | | | _platform/platform.py: Remove outdated comment See merge request BuildStream/buildstream!1995
| * _platform/platform.py: Remove outdated commentChandan Singh2020-07-151-1/+0
|/ | | | | | | | "Preferred" sandboxes used to be a thing when we had sandbox implementations in BuildStream Core. This should probably have been removed at the same time as dropping the notion of preferred sandboxes, but that was likely an oversight.
* Merge branch 'tristan/fix-juncle-include-link' into 'master'Tristan Van Berkom2020-07-1518-3/+79
|\ | | | | | | | | | | | | Fix including of files across linked junction boundaries Closes #1359 See merge request BuildStream/buildstream!1993
| * tests/format/link.py: Added test for including files across linked junction ↵tristan/fix-juncle-include-linkTristan van Berkom2020-07-1515-0/+62
| | | | | | | | boundaries
| * _loader: Treat `link` elements like junctionsTristan van Berkom2020-07-152-3/+3
| | | | | | | | | | | | | | link elements need to be treated the same way as junctions in the load process. This fixes issue #1359
| * _project.py: Documenting why ensure_fully_loaded() is reentrant.Tristan van Berkom2020-07-151-0/+14
|/
* Merge branch 'dwinship/overnight-freedesktop-sdk-buildstream2' into 'master'bst-marge-bot2020-07-151-1/+1
|\ | | | | | | | | | | | | Test freedesktop-sdk against buildstream2 in CI Closes celduin/bsps/bst-boardconsumption#21 and #1334 See merge request BuildStream/buildstream!1987
| * Update overnight tests for freedesktop-sdkdwinship/overnight-freedesktop-sdk-buildstream2Douglas Winship2020-07-141-1/+1
|/ | | | | | Use the current 'main' bst2 branch of freedesktop-sdk, and use tag 1.93.4 of bst-plugins-experimental (contains an important fix for flatpak_image.py)
* Merge branch 'chandan/minor-formatting-fix' into 'master'bst-marge-bot2020-07-141-2/+2
|\ | | | | | | | | doc/main_glossary.rst: Fix formatting of .bst file reference See merge request BuildStream/buildstream!1974
| * doc/main_glossary.rst: Fix formatting of .bst file referencechandan/minor-formatting-fixChandan Singh2020-07-141-2/+2
|/ | | | | One of the references to the "`.bst` file" term was formatted incorrectly, and was missing a period.
* Merge branch 'bschubert/update-plugins' into 'master'Chandan Singh2020-07-142-5/+5
|\ | | | | | | | | Update bst-plugins-experimental to 1.93.4 See merge request BuildStream/buildstream!1991
| * Update bst-plugins-experimental to 1.93.4bschubert/update-pluginsBenjamin Schubert2020-07-142-5/+5
|/ | | | This removes the 'ostree' extra and some build dependencies
* Merge branch 'dwinship/fix-tab-completion' into 'master'bst-marge-bot2020-07-141-13/+1
|\ | | | | | | | | fix tab-completion: hardcode default min version See merge request BuildStream/buildstream!1989
| * fix tab-completion: hardcode default min versiondwinship/fix-tab-completionDouglas Winship2020-07-141-13/+1
|/ | | | | | | Hardcodes the default min version as "2.0", instead of using a function. (The function was causing an ImportError during tab completion, because it attempts to import "__version__", which isn't available at tab completion time.)
* Merge branch 'dwinship/dont-strip-bst-from-user-supplied-checkout-dir' into ↵bst-marge-bot2020-07-101-2/+2
|\ | | | | | | | | | | | | | | | | 'master' Don't strip '.bst' from user-supplied checkout dir Closes #1361 See merge request BuildStream/buildstream!1992
| * Don't strip '.bst' from user-supplied checkout dirDouglas Winship2020-07-101-2/+2
|/ | | | | | | | Allows user to supply a directory name ending in '.bst' for "bst artifact checkout", without buildstream removing '.bst' from the end. (It's only supposed to remove '.bst' if the user didn't supply a directory name, and buildstream is auto-generating a name based on the target name.)
* Merge branch 'bschubert/mypy-docs' into 'master'bst-marge-bot2020-07-091-11/+21
|\ | | | | | | | | coding_guidelines.rst: Update documentation around type annotation See merge request BuildStream/buildstream!1988
| * coding_guidelines.rst: Update documentation around type annotationbschubert/mypy-docsBenjamin Schubert2020-07-091-11/+21
|/ | | | | | | | After https://lists.apache.org/thread.html/r99119ddf038b3baa63afa3edf82ca28104f02550e6424dc5f14cb360%40%3Cdev.buildstream.apache.org%3E, we now allow mypy in any part of the codebase. - Also update previous documentation about typing public API
* Merge branch 'bschubert/tar-filter-nondev' into 'master'bst-marge-bot2020-07-092-2/+13
|\ | | | | | | | | tar.py: Don't import 'dev' nodes See merge request BuildStream/buildstream!1990
| * tar.py: Don't import 'dev' nodesbschubert/tar-filter-nondevBenjamin Schubert2020-07-082-2/+13
|/ | | | | This filters out all the 'dev' nodes when extracting tar files since they are not supported by any of our backends anyways
* Merge branch 'bschubert/simplify-stream-interactions' into 'master'bst-marge-bot2020-07-068-217/+102
|\ | | | | | | | | simplify stream interactions (Remove Notifications) See merge request BuildStream/buildstream!1985
| * scheduler.py: Remove all usage of notificationsBenjamin Schubert2020-07-063-97/+20
| | | | | | | | Call directly the relevant methods from the stream to the scheduler
| * scheduler.py: Remove notifications from scheduler to streamBenjamin Schubert2020-07-062-27/+7
| | | | | | | | | | This removes all notifications left coming from the scheduler, and replaces them by callbacks
| * _stream.py: Stop using a 'RUNNING' event to know the state of the schedulerBenjamin Schubert2020-07-062-11/+4
| | | | | | | | | | The stream is itself calling the `run` method on the scheduler, we don't need another indirection
| * _stream.py: Stop using a 'TERMINATED' event to know the state of the schedulerBenjamin Schubert2020-07-062-6/+3
| | | | | | | | | | We are calling the scheduler, and it returning correctly already tells us this.
| * _stream.py: Stop using a 'SUSPENDED' event to know the state of the schedulerBenjamin Schubert2020-07-062-7/+4
| | | | | | | | | | We are calling the scheduler, and it returning correctly already tells us this.
| * scheduler.py: Pass all 'retry' operations through the stateBenjamin Schubert2020-07-063-8/+37
| | | | | | | | | | Stop using 'Notifications' for retries, the state is the one handling the callbacks required for every change in status of elements
| * _state.py: Only use a single place of truth for the start timeBenjamin Schubert2020-07-065-19/+19
| | | | | | | | | | This moves all implementations of 'start_time' into a single place for easier handling and removing roundtrips of notifications
| * scheduler.py: Remove task-based notifications and use the stateBenjamin Schubert2020-07-062-26/+5
| | | | | | | | | | The State is the interface between both, there is no need to do multiple round-trips to handle such notifications
| * scheduler.py: Remove 'Message' notification type, use the messengerBenjamin Schubert2020-07-064-22/+9
|/ | | | | The messenger should be the one receiving messages directly, we don't need this indirection
* Merge branch 'bschubert/cleanup' into 'master'bst-marge-bot2020-07-062-3/+2
|\ | | | | | | | | app.py: Remove unnecessary operation in _message() See merge request BuildStream/buildstream!1984
| * sandbox.py: Remove unused importBenjamin Schubert2020-07-061-1/+1
| |
| * app.py: Remove unnecessary operation in _message()Benjamin Schubert2020-07-061-2/+1
|/ | | | | This copies a dictionary before it gets expanded, which is not necessary since the expansion will not keep the dictionary as is
* Merge branch 'juerg/source-push' into 'master'bst-marge-bot2020-07-0633-41/+219
|\ | | | | | | | | Add `bst source push` command See merge request BuildStream/buildstream!1977