summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* NEWS: Document breaking change, keys which can only be specified in project.conftristan/fix-redundant-session-headingsTristan van Berkom2020-08-101-0/+10
|
* doc/source/format_project.rst: Documenting keys which must be in project.confTristan van Berkom2020-08-101-3/+21
| | | | | | Clarify the few keys which must absolutely be declared in project.conf if declared at all, these keys cannot be declared in a separate include file.
* _frontend/widget.py: Refactor the Widget.print_heading() functionTristan van Berkom2020-08-101-34/+87
| | | | | | | | | | | | | This function now prints information about each loaded project in the session: * Information about each loaded project - Junction path, if any - Provenance of how the project was initially loaded, if any - Shows which projects declare the loaded project as a duplicate, if any - Shows which projects declare the loaded project as internal, if any * Plugins loaded in each project, including how they were loaded * Project option values resolved for each loaded project
* _pluginfactory: Make list_plugins() report new display informationTristan van Berkom2020-08-106-18/+31
| | | | | | | | Now the PluginFactory.list_plugins() API also reports a human readable explanation of where the plugin was loaded from for each plugin. This is internally supported by an extension of the abstract PluginOrigin.get_plugin_paths() APIs.
* _project.py, _loader/loadcontext.py: Added Project.loaded_projects()Tristan van Berkom2020-08-103-0/+62
| | | | | | | | | This adds a new _ProjectInformation type to types.py which is returned by the new Project.loaded_projects() API, supported by a new internal LoaderContext.loaded_projects() function. This allows the frontend some helpful information to print about the loaded projects in the session heading.
* _project.py: Only one set of plugin factoriesTristan van Berkom2020-08-104-35/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit makes it mandatory to specify the `plugins` configuration directly in the `project.conf`, and at the same time removes any ambiguity as to which plugins are loaded and how. It thus becomes impossible for the first pass to load the same plugins differently than in the second pass. Additionally, some additional project.conf keys are asserted to be specified in the toplevel project.conf and not included as a matter of policy and consistency. Summary of changes: * _project.py - Only one set of plugin factories, loaded only in the first pass. - Assert the following keys are only specified in project.conf and not included from other files: name, element-path, min-version, plugins - The create_element() and create_source() methods no longer need to take a `first_pass` argument * element.py: Updated to not specify `first_pass` to create_element() or create_source() * _frontend/widget.py: Print the only set of plugins which are loaded, no more ambiguity about first or second pass. * _pluginfactory/pluginoriginjunction.py: Chain load plugins from the single element/source factory, instead of accessing the Project.config member which no longer has the factories.
* _yaml.pyx: Make shortname a mandatory argument to _yaml.load()Tristan van Berkom2020-08-1017-62/+67
| | | | | | | | | | | | | This ensures that important calls to this function do give some thought to providing a reasonable shortname, which will be used as a display name in errors. This continues to support `None` as a shortname, which is used in various tests which don't need to provide a reasonable user facing error. The buildstream.testing module now exports a `load_yaml` function which only takes a filename and no shortname.
* _stream.py: Conditionally call the session startup frontend callback.Tristan van Berkom2020-08-101-12/+16
| | | | | | | Avoid redundantly announcing the session heading in the frontend and only selectively announce it once for the main session. Fixes #1369
* Merge branch 'chandan/contrib-runtime-deps' into 'master'bst-marge-bot2020-08-101-0/+10
|\ | | | | | | | | | | | | doc/using_the_testsuite: Link to instructions for runtime dependencies Closes #1115 See merge request BuildStream/buildstream!2016
| * doc/using_the_testsuite: Link to instructions for runtime dependenciesChandan Singh2020-08-101-0/+10
|/ | | | | | | | | Running the test suite requires one to install BuildStream's runtime dependencies as well, out of which BuildBox is the tricky one. We already have instructions on how to do that, so we just need to point to it from the hacking docs. Closes #1115.
* Merge branch 'tristan/cleanup-pickle' into 'master'bst-marge-bot2020-08-102-2/+0
|\ | | | | | | | | element/source: Remove pointless extra variable See merge request BuildStream/buildstream!2020
| * element/source: Remove pointless extra variabletristan/cleanup-pickleTristan van Berkom2020-08-102-2/+0
|/ | | | | These __meta_kind members are documented as relevant for pickling, which was removed a short while ago.
* Merge branch 'juerg/element-sources' into 'master'bst-marge-bot2020-08-0610-284/+447
|\ | | | | | | | | Extract ElementSources class See merge request BuildStream/buildstream!2007
| * Extract ElementSources classjuerg/element-sourcesJürg Billeter2020-08-066-215/+398
| | | | | | | | An ElementSources object represents the combined sources of an element.
| * element.py: Push all sourcesJürg Billeter2020-08-061-4/+5
| | | | | | | | Do not exit the loop if a source is already in the remote source cache.
| * element.py: Drop `__has_all_sources_cached` instance variableJürg Billeter2020-08-061-6/+1
| | | | | | | | | | This is already cached in each `Source` object. As this method is not called frequently, this extra caching layer is not needed.
| * element.py: Drop cache checks from `_tracking_done()`Jürg Billeter2020-08-062-10/+2
| | | | | | | | | | | | This was a workaround for inconsistent error handling in the frontend. As the error handling is now fixed, drop these cache checks after tracking.
| * source.py: Validate cache when it's used, not in `_is_cached()`Jürg Billeter2020-08-062-40/+23
| | | | | | | | | | | | | | `_is_cached()` is indirectly called by the frontend, which is not optimal for handling per-plugin errors. Instead, call `validate_cache()` right before the cache is used: in fetch jobs and when opening a workspace.
| * _frontend/widget.py: Add context to errors in `show_pipeline()`Jürg Billeter2020-08-061-15/+20
| | | | | | | | | | Provide context with the element name if an error is raised when trying to determine element state.
| * _frontend/app.py: Catch exceptions in `_print_summary()`Jürg Billeter2020-08-061-1/+5
|/ | | | | | | | `_print_summary()` is called outside a `try` block in `initialized()` to print a summary at the end of a session. Add a `try` block to properly handle plugin errors from `show_pipeline()`. This provides error handling consistency with `print_heading()`, which is called at the start of a session.
* Merge branch 'bschubert/update-to-pytest-6' into 'master'bst-marge-bot2020-08-056-34/+33
|\ | | | | | | | | | | | | requirements.txt: Update dependencies Closes #1377 See merge request BuildStream/buildstream!2010
| * requirements.txt: Update dependenciesbschubert/update-to-pytest-6Benjamin Schubert2020-08-056-34/+33
|/ | | | | | This now pulls in pytest > 6.0.1, which has a few breaking changes Fix #1377
* Merge branch 'chandan/publish-types' into 'master'bst-marge-bot2020-08-042-1/+8
|\ | | | | | | | | Publish our type annotations See merge request BuildStream/buildstream!2012
| * Publish our type annotationschandan/publish-typesChandan Singh2020-08-042-1/+8
|/ | | | | | | | | | | As defined by PEP 561 (https://www.python.org/dev/peps/pep-0561), add a `py.typed` file to the BuildStream package. This allows any downstream packages (currently plugins, maybe other use cases in future) to run type checkers against BuildStream as well. Although we don't have type hints for the private API surface, downstream packages should only be using public API anyway, so they should mostly be fine.
* Merge branch 'jjardon/doc_fuse' into 'master'bst-marge-bot2020-08-041-15/+4
|\ | | | | | | | | doc/source/main_install.rst: BuildStream doesn't directly depend on fuse / bubblewrap See merge request BuildStream/buildstream!2015
| * doc/source/main_install.rst: BuildStream doesn't directly depend on bubblewrapjjardon/doc_fuseJavier Jardón2020-08-041-14/+4
| |
| * doc/source/main_install.rst: BuildStream doesn't directly depend on fuseJavier Jardón2020-08-041-5/+4
|/
* Merge branch 'chandan/fix-utils-495' into 'master'bst-marge-bot2020-08-041-0/+15
|\ | | | | | | | | | | | | data/userconfig.yaml: Document valid symbols for format strings Closes #510 See merge request BuildStream/buildstream!2013
| * data/userconfig.yaml: Document valid symbols for format stringschandan/fix-utils-495Chandan Singh2020-08-041-0/+15
|/ | | | | | | | Document the valid symbols that can be used as part of the `message-format`. And, in case of `element-format`, redirect to `bst show --help` to avoid duplicating the help text in two places. Fixes #510.
* Merge branch 'dwinship/reintroduce-install-instructions-to-docs' into 'master'bst-marge-bot2020-08-042-0/+341
|\ | | | | | | | | Reintroduce install instructions to docs See merge request BuildStream/buildstream!1999
| * Remove install instructions to ostree and deb pluginsdwinship/reintroduce-install-instructions-to-docsJavier Jardón2020-08-031-49/+5
| | | | | | | | They are not part of core anymore
| * Reintroduce install instructions to docsDouglas Winship2020-08-032-0/+385
|/ | | | | | | | | | This is mostly a copy of the install from source instructions from https://buildstream.build/install.html. Edits have been made to focus more on Buildstream 2. Sections for tarball installation and package installation have been removed, as they cannot currently be used for BuildStream 2. Docker section has been moved to the docker-image repository. Docs have been reduced down to a single page.
* Merge branch 'tristan/increase-timeout-variables-test' into 'master'bst-marge-bot2020-08-031-1/+1
|\ | | | | | | | | tests/format/variables.py: Increase timeout See merge request BuildStream/buildstream!2008
| * tests/format/variables.py: Increase timeoutTristan van Berkom2020-08-031-1/+1
|/ | | | | | | With sometimes very slow runners, this test has been found to timeout more often after the recent refactoring. Double the timeout to avoid erronously failing CI.
* Merge branch 'abderrahim/cached-failure' into 'master'bst-marge-bot2020-07-296-54/+45
|\ | | | | | | | | | | | | Rework handling of cached failures Closes #967 See merge request BuildStream/buildstream!1970
| * tests/integration/interactive_build.py: make a new test for retrying failed ↵abderrahim/cached-failureAbderrahim Kitouni2020-07-291-1/+16
| | | | | | | | | | | | builds The old one tested that retrying the failed build doesn't actually retry
| * element.py: allow re-scheduling failed buildsAbderrahim Kitouni2020-07-291-2/+1
| |
| * Rework handling of cached failuresAbderrahim Kitouni2020-07-294-41/+23
| |
| * element.py: move printing the build environment from elementjob.pyAbderrahim Kitouni2020-07-292-10/+5
|/
* Merge branch 'tristan/options-order' into 'master'bst-marge-bot2020-07-2927-5/+192
|\ | | | | | | | | Append lists when merging conditional statements See merge request BuildStream/buildstream!2002
| * tests/format/include.py: Test preservation of conditionals at include timetristan/options-orderTristan van Berkom2020-07-2926-0/+179
| | | | | | | | | | | | Added tests to ensure that conditional statements don't get overwritten when performing composition of one dictionary on top of another due to include processing.
| * node.pyx: Support merging of conditional sequencesTristan van Berkom2020-07-291-5/+13
|/ | | | | | | | | | | | | | | | | | | | | | | | | | During composition of project.conf, it can happen that we are including and composing node hierarchies before options are resolved and before we are ready to evaluate conditional statements. Without this patch, conditional statements are treated like regular sequences, causing existing conditional statements to be overwritten by subsequently composited conditional statements without ever having been resolved. This patch introduces a special case for composing sequences, when composing a sequence that is a conditional statement, the sequence is appended instead of overwriting the underlying list. Note that precedence of declarations in including vs included yaml fragments did raise some concern, and a test in the following commit is added to cover this case. Composition of conditional lists on top of other conditional lists are unconditionally appended, however the semantics of includes ensures that the including fragment's conditionals is always composited *on top* over the included fragment, ensuring that we have the correct pecedence even when compositing yet to be resolved conditional directives.
* Merge branch 'willsalmon/overnightcache' into 'master'bst-marge-bot2020-07-271-0/+14
|\ | | | | | | | | | | | | Add a remote cache to overnight builds Closes #897 See merge request BuildStream/buildstream!2004
| * Add a remote cache to overnight buildswillsalmon/overnightcacheWilliam Salmon2020-07-241-0/+14
|/ | | | | This means the over nights will also push and pull when they did not in the past.
* Adding CODEOWNERSTristan van Berkom2020-07-242-38/+4
| | | | | | | | | As per this announcement regarding formalizing committers: https://mail.gnome.org/archives/buildstream-list/2020-May/msg00023.html We are now using a CODEOWNERS file to ensure only maintainers can greenlight patches for protected branches in gitlab.
* Merge branch 'tristan/partial-variables' into 'master'bst-marge-bot2020-07-2315-202/+755
|\ | | | | | | | | Variables non-recursive and lazy resolution refactor See merge request BuildStream/buildstream!1973
| * tests/format/variables.py: Adding new test for lazy variable resolutiontristan/partial-variablesTristan van Berkom2020-07-226-0/+30
| | | | | | | | | | | | Check that lazy variable resolution allows using variables in junction definitions which would not successfully resolve if we needed to resolve synchronously.
| * _variables.pyx, element.py: Lazy variable resolutionTristan van Berkom2020-07-222-5/+2
| | | | | | | | | | | | | | | | Variables are resolved on demand until Variables.check() is explicitly called, allowing junctions to use variables without requiring that they be completely resolved. Approach is courtesy of Valentin David.
| * tests/frontend/overlaps.py: Test undefined variablesTristan van Berkom2020-07-222-1/+27
| | | | | | | | | | Ensure that we get the expected provenance when expanding a variable included in an overlap whitelist entry.
| * tests/format/variables.py: Added some new testsTristan van Berkom2020-07-225-12/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Test scenarios where a junction needs to resolve variables in order to configure a subproject, but where some other variables may be derived from the same subproject. In this scenario we allow partial resolution of variables for junction elements. * Enhanced the undefined variables and circular reference tests to also check for the expected provenances. * Test for deep variable resolution Test variable indirection with 50, 500 and 5000 variables: * 50 - tests generally large indirections in the recursive algorithm, which is limited to 200 recursions * 500 - tests that the non-recursive algorithm works for successful outcomes and not only for error resolution * 5000 - tests that the iterative algorithm works and ensures it is not discarded, as a recursive algorithm cannot be implemented to support this depth with python (which limits itself to merely 1000 stack frames).
View Lorry Controller Status