summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* NEWS: Updated for breaking change of junction target removaltristan/link-elementTristan van Berkom2020-06-011-3/+4
|
* junctions: Remove all traces of the `target` optionTristan van Berkom2020-06-0115-167/+13
| | | | | | | | | | | | | | | This removes: * The `target` feature from the junction plugin * Special case code in the loader for the `target` feature * The `target` related cases in tests/format/junctions.py This also adjusts the `target` related documentation in the `junction` element to suggest using a `link` element for the purpose of using a subproject junction configuration to access a common sub-subproject.
* doc: Adding new `link` plugin to the core plugin indexTristan van Berkom2020-06-011-0/+1
|
* tests/format/link.py: Adding tests for the link elementTristan van Berkom2020-06-0160-0/+407
|
* link element: Adding support for new link elementTristan van Berkom2020-06-014-4/+160
| | | | | | This element acts as a symbolic link, it has no other configurations other than to specify the element (or junction) to which it refers to, either in the local project or in a subproject.
* _loader/loader.py: Make collect_element_no_deps() publicTristan van Berkom2020-06-011-75/+78
| | | | | Within the _loader module, the LoadElement will need to use this in order to resolve links.
* Merge branch 'tristan/loader-fixes' into 'master'bst-marge-bot2020-05-301-2/+1
|\ | | | | | | | | _loader/loader.py: Remove useless line of code. See merge request BuildStream/buildstream!1946
| * _loader/loader.py: Remove useless line of code.Tristan van Berkom2020-05-301-2/+1
|/ | | | | | | | | We were redundantly calling Loader._load_file() directly before calling Loader.get_loader(), even though Loader.get_loader() implicitly loads the file anyway. To pass tests, we needed to fix get_loader() to pass provenance along to Loader._load_file().
* Merge branch 'tristan/remove-loader-no-progress-object' into 'master'bst-marge-bot2020-05-302-18/+12
|\ | | | | | | | | loader: removing the NO_PROGRESS object See merge request BuildStream/buildstream!1947
| * loader: removing the NO_PROGRESS objecttristan/remove-loader-no-progress-objectTristan van Berkom2020-05-302-18/+12
|/ | | | | | | | | | | While adding a mock task object to track progress in some loader tests in commit 17144d84c2b63daf6e3aa9b42c6c773f134e8660, a new `_NO_PROGRESS` object was added as an explicit marker to denote that progress information should not be reported. This complicates the code, None should be a sufficient value for not reporting progress, while still permitting mock task objects to capture progress if desired.
* Merge branch 'tristan/version-specific-config' into 'master'bst-marge-bot2020-05-293-3/+50
|\ | | | | | | | | Support version specific configuration files See merge request BuildStream/buildstream!1941
| * doc/source/using_config.rst: Document support for version specific config files.Tristan van Berkom2020-05-291-0/+11
| |
| * tests/internals/context.py: Test correct config file is chosen.Tristan van Berkom2020-05-291-0/+28
| |
| * _context.py: Support parallel installations of BuildStreamTristan van Berkom2020-05-291-3/+11
|/ | | | | | | | | | Allow users to keep separate buildstream.conf files in their XDG_CONFIG_HOME for separate BuildStream versions, in case they have BuildStream 1 and 2 installed. Like other major version safeguards and warnings, this new addition also supports future major versions if they should appear.
* Merge branch 'bschubert/resolve-public-variables' into 'master'bst-marge-bot2020-05-295-14/+48
|\ | | | | | | | | | | | | element.py: Always expand public data's variables Closes #1310 See merge request BuildStream/buildstream!1943
| * element.py: Always expand public data's variablesBenjamin Schubert2020-05-295-14/+48
|/ | | | | | | | | | | | | | | | | This fixes a bug introduced by d7d18c1a2e454c507afd9e1d3f1358639dd43871, where public data integration commands and others would never get their data expanded and would thus fail to run. The previous however revelead a previous bug, where variables values in public data of elements would not be part of the cache key of an element or it's reverse dependencies, and thus, on variable change, rebuilds would not happen when they would have been needed. This ensures that all the public data is always resolved and part of the element's cache key. This however will bring a rebuild of an element whenever its integration commands variables change, which is simpler to handle than to try to push that responsibility on reverse dependencies, since public data can contain plugin-defined values.
* Merge branch 'juerg/cas-send-directory' into 'master'bst-marge-bot2020-05-281-3/+5
|\ | | | | | | | | cascache.py: Drop unnecessary FindMissingBlobs from _send_directory() See merge request BuildStream/buildstream!1944
| * cascache.py: Drop unnecessary FindMissingBlobs from _send_directory()juerg/cas-send-directoryJürg Billeter2020-05-281-3/+5
|/ | | | | | buildbox-casd `UploadMissingBlobs` will call `FindMissingBlobs` before the actual upload. Having BuildStream call `FindMissingBlobs` as well adds unnecessary overhead.
* Merge branch 'juerg/sast' into 'master'bst-marge-bot2020-05-281-0/+1
|\ | | | | | | | | .gitlab-ci.yml: Configure SAST analyzers See merge request BuildStream/buildstream!1939
| * .gitlab-ci.yml: Configure SAST analyzersJürg Billeter2020-05-281-0/+1
|/ | | | | Disable flawfinder and eslint jobs as this repository contains no C or JavaScript code to be analyzed.
* Merge branch 'juerg/expandsandbox' into 'master'bst-marge-bot2020-05-285-31/+52
|\ | | | | | | | | | | | | element.py: Expand variables in sandbox config Closes #1303 See merge request BuildStream/buildstream!1938
| * tests/integration/sandbox.py: Test variable expansion in sandbox configjuerg/expandsandboxJürg Billeter2020-05-283-0/+25
| |
| * element.py: Expand variables in sandbox configJürg Billeter2020-05-281-3/+5
| |
| * Move sandbox YAML processing from Element to SandboxConfigJürg Billeter2020-05-282-29/+23
|/
* Merge branch 'tristan/junction-docs-touchup' into 'master'bst-marge-bot2020-05-281-4/+4
|\ | | | | | | | | doc/source/format_declaring.rst: Fixing spelling mistake See merge request BuildStream/buildstream!1940
| * doc/source/format_declaring.rst: Fixing spelling mistakeTristan van Berkom2020-05-281-4/+4
|/ | | | | Making some examples more consistent with regards to indentation of YAML, and fixing "recursice -> recursive" spelling mistake.
* Merge branch 'tristan/junction-plugin-origin' into 'master'bst-marge-bot2020-05-2817-237/+774
|\ | | | | | | | | Introduce new `junction` plugin origin See merge request BuildStream/buildstream!1935
| * doc/source/format_project.rst: Documenting the junction plugin origin.tristan/junction-plugin-originTristan van Berkom2020-05-281-0/+63
| |
| * tests/plugins/loading.py: Adding tests for junction plugin originTristan van Berkom2020-05-285-0/+193
| |
| * _pluginfactory/pluginoriginjunction.py: Add support for junction plugin origin.Tristan van Berkom2020-05-283-0/+89
| |
| * _pluginfactory: Delegating the work of locating plugins to the PluginOriginTristan van Berkom2020-05-2810-237/+429
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | This way we split up the logic of how to load plugins from different origins into their respective classes. This commit also: o Introduces PluginType (which is currently either SOURCE or ELEMENT) o Reduces the complexity of the PluginFactory constructor o Kills the loaded_dependencies list and the all_loaded_plugins API, and replaces both of these with a new list_plugins() API. Consequently the jobpickler.py from the scheduler, and the widget.py from the frontend, are updated to use list_plugins(). o Split up the PluginOrigin implementations into separate files Instead of having all PluginOrigin classes in pluginorigin.py, split it up into one base class and separate files for each implementation, which is more inline with BuildStream coding style. This has the unfortunate side effect of adding load_plugin_origin() into the __init__.py file, because keeping new_from_node() as a PluginOrigin class method cannot be done without introducing a cyclic dependency with PluginOrigin and it's implementations.
* Merge branch 'juerg/dynamic-plan' into 'master'bst-marge-bot2020-05-275-14/+139
|\ | | | | | | | | Restore dynamic build plan feature See merge request BuildStream/buildstream!1926
| * tests/frontend/pull.py: Add test for dynamic build planJürg Billeter2020-05-271-0/+47
| | | | | | | | Make sure build-only dependencies are pulled only when necessary.
| * Revert "Simplify queue management"Jürg Billeter2020-05-272-3/+14
| | | | | | | | This reverts commit aa25f6fcf49f0015fae34dfd79b4626a816bf886.
| * Revert "Schedule elements instead of "requiring" them"Jürg Billeter2020-05-274-11/+70
| | | | | | | | This reverts commit 14e32a34f67df754d9146efafe9686bfe6c91e50.
| * Revert "element: remove _is_required()/__required"Jürg Billeter2020-05-271-0/+8
|/ | | | This reverts commit 750aa60ab3809167ff14c479068a328dc477de3e.
* Merge branch 'juerg/node-properties' into 'master'bst-marge-bot2020-05-2715-264/+386
|\ | | | | | | | | Field-based node properties See merge request BuildStream/buildstream!1918
| * .gitlab-ci.yml: Update BuildBox components to 0.0.8juerg/node-propertiesJürg Billeter2020-05-271-5/+5
| |
| * Update node property support to match proto changesJürg Billeter2020-05-279-59/+87
| |
| * _protos: Update local_cas.protoJürg Billeter2020-05-252-54/+53
| |
| * _protos: Update remote_execution.protoJürg Billeter2020-05-253-146/+241
|/
* Merge branch 'tristan/loader-api-fixup' into 'master'bst-marge-bot2020-05-242-197/+197
|\ | | | | | | | | _loader/loader.py: Reoganized public/private methods See merge request BuildStream/buildstream!1936
| * _loader/loader.py: Reoganized public/private methodstristan/loader-api-fixupTristan van Berkom2020-05-242-197/+197
|/ | | | | | | | | | | | | * This file had `clean_caches()` documented as public but the function was actually private `_clean_caches()`: Moved this to the end of the class in the private section. * The `_get_loader()` was marked as private but is in fact public, and used by the project to load cross junction include files. This patch also updates `_includes.py` to call the public `get_loader()` method instead of sneaking in and calling the private `_get_loader()` method (while also removing one redundant line of code from the same function).
* Merge branch 'tristan/improve-frontend-crash-handling' into 'master'bst-marge-bot2020-05-192-6/+29
|\ | | | | | | | | | | | | Improve handling of frontend crashes Closes #1312 See merge request BuildStream/buildstream!1933
| * _scheduler/scheduler.py: Reset the schedule handler at the beginning of ↵Tristan Van Berkom2020-05-191-3/+15
| | | | | | | | | | | | | | | | | | | | real_schedule() In case queuing jobs results in jobs completing, we need to reset the schedule handler at the beginning of the function and not after queueing the jobs. This fixes the failure to exit the main loop in #1312
| * _scheduler: Fix order of launching jobs and sending notifications.Tristan Van Berkom2020-05-192-3/+14
|/ | | | | | | | | | | | | | | | | | | | | | | Sending notifications causes potentially large bodies of code to run in the abstracted frontend codebase, we are not allowed to have knowledge of the frontend from this code. Previously, we were adding the job to the active jobs, sending the notification, and then starting the job. This means that if a BuildStream frontend implementation crashes, we handle the excepting in an inconsistent state and try to kill jobs which are not running. In addition to making sure that active_jobs list adjustment and job starting does not have any code body run in the danger window in between these, this patch also adds some fault tolerance and assertions around job termination so that: o Job.terminate() and Job.kill() do not crash with None _process o Job.start() raises an assertion if started after being terminated This fixes the infinite looping aspects of frontend crashes at job_start() time described in #1312.
* Merge branch 'bschubert/ensure-composite-works-with-variables' into 'master'bst-marge-bot2020-05-196-2/+42
|\ | | | | | | | | node.pyx: Deep clone ScalarNode too See merge request BuildStream/buildstream!1929
| * node.pyx: Deep clone ScalarNode toobschubert/ensure-composite-works-with-variablesBenjamin Schubert2020-05-196-2/+42
|/ | | | | This ensures that when resolving variables, we do not overwrite the values for a different element in the case we were using composition.
* Merge branch 'chandan/fix-subst-non-greedy' into 'master'bst-marge-bot2020-05-182-1/+21
|\ | | | | | | | | | | | | _frontend/profile: Use non-greedy search to substitute variables Closes #1307 See merge request BuildStream/buildstream!1932
| * _frontend/profile: Use non-greedy search to substitute variablesChandan Singh2020-05-182-1/+21
|/ | | | | | | | Use non-greedy search to ensure that we stop at the next closing brace. Otherwise, for a string like `%{variable} {variable}`, the second variable will also get substituted. Fixes #1307.