summaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* testutils/platform: Refactor to be compatible with Python 3.9chandan/py39-platformChandan Singh2020-10-291-6/+12
| | | | | | | | | | | Starting from Python 3.9, it seems like the `_replace()` method no longer works on `platform.uname_result` objects, that are returned by `platform.uname()`. This causes some of our tests to fail on Python 3.9. See https://bugs.python.org/issue42163 for upstream issue. Fix it by slightly changing the way we override the values of the `platform.uname()` function, such that it works on both Python 3.9 and 3.8 (and below).
* tests/frontend/logging.py: Test log lines for correct element names/keysTristan van Berkom2020-10-278-4/+174
| | | | | | | | | | | | When printing log lines to the master log, we ensure that log lines are printed with the element name and cache key which are related to the task from which the messages are being issued. When printing log lines to task specific log lines, we prefer to print the element names and cache keys which pertain to the element from which the log line was actually issued. This new tests asserts this behavior.
* Restore task element name / element name distinction in UITristan van Berkom2020-10-272-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This behavior has regressed a while back when introducing the messenger object in 0026e379 from merge request !1500. Main behavior change: - Messages in the master log always appear with the task element's element name and cache key, even if the element or plugin issuing the log line is not the primary task element. - Messages logged in the task specific log, retain the context of the element names and cache keys which are issuing the log lines. Changes include: * _message.py: Added the task element name & key members * _messenger.py: Log the element key as well if it is provided * _widget.py: Prefer the task name & key when logging, we fallback to the element name & key in case messages are being logged outside of any ongoing task (main process/context) * job.py: Unconditionally stamp messages with the task name & key Also removed some unused parameters here, clearing up an XXX comment * plugin.py: Add new `_message_kwargs` instance property, it is the responsibility of the core base class to maintain the base keyword arguments which are to be used as kwargs for Message() instances created on behalf of the issuing plugin. Use this method to construct messages in Plugin.__message() and to pass kwargs along to Messenger.timed_activity(). * element.py: Update the `_message_kwargs` when the cache key is updated * tests/frontend/logging.py: Fix test to expect the cache key in the logline * tests/frontend/artifact_log.py: Fix test to expect the cache key in the logline Fixes #1393
* _frontend/cli.py: Drop support for bst shell --use-buildtree=tryJürg Billeter2020-10-274-85/+18
| | | | | `--use-buildtree` is now a boolean option for `bst shell`. If no buildtree is available, an error is raised.
* _stream.py: Simplify code in shell()Jürg Billeter2020-10-271-15/+7
| | | | | | Use single scheduler run for pulling dependencies and buildtree. Deduplicate cache checks and corresponding warning/error messages with and without pulling enabled.
* Move shell element loading logic from Cli to StreamJürg Billeter2020-10-271-3/+3
|
* tests/frontend/track.py: Conditionally skip a testtristan/conditional-subsecond-precisionTristan van Berkom2020-10-251-0/+6
| | | | | | Skip a test which relies on mtimes differing within a short timespan, this will fail if it happens fast enough (which it should) on systems which do not support subsecond precision mtimes.
* tests/internals/utils_move_atomic.py: Conditionally skip a testTristan van Berkom2020-10-251-0/+7
| | | | | Skip the mtime related test if the underlying filesystem does not support subsecond precision mtime.
* tests/artifactcache/expiry.py: Conditionally skip some testsTristan van Berkom2020-10-251-0/+45
| | | | | Skip some of the artifact expiry tests in the case we don't have subsecond mtime precision.
* tests/internals/storage_vdir_import.py: Conditionally skipTristan van Berkom2020-10-251-0/+7
| | | | Conditionally skip tests which require subsecond mtime precision.
* tests/format/junctions.py: Test multiple layers of link indirection in overridesTristan van Berkom2020-10-065-0/+26
|
* tests/format/junctions.py: Test that links work for override keys in nested ↵Tristan van Berkom2020-10-063-0/+18
| | | | projects
* tests/format/junctions.py: Test that we can use a link to address which ↵Tristan van Berkom2020-10-064-1/+28
| | | | subproject to override
* _stream.py: Pull missing artifacts in push()juerg/pushJürg Billeter2020-09-292-12/+2
| | | | | | | | | | | As per #819, BuildStream should pull missing artifacts by default. The previous behavior was to only pull missing buildtrees. A top-level `--no-pull` option can easily be supported in the future. This change makes it possible to use a single scheduler session (with concurrent pull and push jobs). This commit also simplifies the code as it removes the `sched_error_action` emulation, using the regular frontend code path instead.
* tests/format/dependencies.py: Test bad filename configurationTristan van Berkom2020-09-282-0/+24
| | | | Test a dictionary instead of a string when given to the filename list.
* tests/format/dependencies.py: Testing shorthand dependency configurationsTristan van Berkom2020-09-289-0/+51
|
* tests/format/dependencies3/elements/supported2.bst: Adding commentTristan van Berkom2020-09-281-0/+6
| | | | Adding some clarifications about an existing test
* tests/integration/filter.py: Add test_filter_pass_integration_uncachedJürg Billeter2020-09-241-0/+32
| | | | | This tests that built filter artifacts don't depend on build dependencies for integration.
* tests/frontend/track.py: Test behavior of SKIP messagesTristan van Berkom2020-09-241-0/+79
| | | | | | | | This test checks that: * We get SKIP messages for tracking local sources * We get SKIP messages for tracking workspaced elements * We go no messages at all for elemenents which have no sources
* tests/integration/manual.py: Test staging files in custom directoriesTristan van Berkom2020-09-183-0/+37
|
* tests/cachekey: Added new cachekey test for new BuildElement configurationTristan van Berkom2020-09-184-1/+16
|
* buildelement.py: Remove legacy command stepsTristan van Berkom2020-09-183-3/+3
| | | | | | | | | | | This was actually deadcode, since node.validate_keys() was called on the configure dictionary without the legacy command steps. If any element was using the legacy commands, they would have been met with a load time error anyway. This commit also updates the cache key test, since removing these legacy commands affects BuildElement internally in such a way as to affect the cache keys.
* ScriptElement: Porting to new APITristan van Berkom2020-09-188-44/+20
| | | | | | | | | | | | | | | | | | | | | | | | Instead of relying on Element.search(), use Element.configure_dependencies() to configure the layout. Summary of changes: * scriptelement.py: Change ScriptElement.layout_add() API to take an Element instead of an element name, this is now not optional (one cannot specify a `None` element). This is an API breaking change * plugins/elements/script.py: Implement Element.configure_dependencies() in order to call ScriptElement.layout_add(). This is a breaking YAML format change. * tests/integration: Script integration tests updated to use the new YAML format * tests/cachekey: Updated for `script` element changes
* tests/frontend/overlaps.py: Test multiple calls to ↵Tristan van Berkom2020-09-188-2/+192
| | | | | | | | | | | | | | | | | | | | Element.stage_dependency_artifacts() This patch adds a new test plugin which implements Element.configure_dependencies() in order to offer better flexibility for testing overlaps. Newly added tests: * Test overlap warnings and errors when staging elsewhere than in the sandbox root. * Test unstaged files failure modes when staging elsewhere than in the sandbox root. * Test various overlap behaviors of OverlapAction, when different calls to Element.stage_dependency_artifacts() cause overlaps to occur after staging files into separate directories.
* tests/format/dependencies.py: Adding tests for dependency `config` attributesTristan van Berkom2020-09-189-0/+119
|
* tests/format/dependencies.py: Test merging of multiple dependenciesTristan van Berkom2020-09-183-0/+34
| | | | | | Test that when the same dependency is added as a build and runtime dependency separately, they end up being the same dependency which is both a build & runtime dependency in the loaded build graph.
* tests/format/dependencies.py: Remove tests for ↵Tristan van Berkom2020-09-189-97/+0
| | | | LoadErrorReason.DUPLICATE_DEPENDENCY
* tests/frontend/overlaps.py: Test CoreWarnings.UNSTAGED_FILESTristan van Berkom2020-09-186-2/+38
|
* tests/frontend/overlaps.py: Minor cleanup and refactoringTristan van Berkom2020-09-181-37/+32
| | | | | | | | | | | | | | | | | This commit: * Removes testing of the deprecated `fail-on-overlap` project configuration option, this is going away soon and unneeded. * Tests that warnings are issued whenever they should be (some tests were happy to see a successful run but failed to check for an expected warning). * Test error/warning more evenly across tests, some were missing the warning mode. * Use `bst show` instead of `bst build` for the undefined_variable test, it should fail without needing a build.
* tests/elements/filter/basic/element_plugins/dynamic.py: Stage in Element.stage()Tristan van Berkom2020-09-181-5/+2
| | | | | This test element was also staging artifacts in Element.assemble(), which is now illegal.
* tests/frontend/rebuild.py: Add test_modify_and_revertJürg Billeter2020-09-161-0/+41
| | | | | | This tests that, in non-strict mode, a cached artifact matching the strict cache key is preferred to a more recent artifact matching only the weak cache key.
* tests/frontend/rebuild.py: Add assertion for built elementsJürg Billeter2020-09-161-0/+8
|
* tests/frontend/push.py: Test artifact push with globJürg Billeter2020-09-101-0/+30
|
* tests/frontend/buildcheckout.py: Add non-strict cache key testjuerg/cache-keyJürg Billeter2020-09-081-0/+34
| | | | | This is a regression test for the previously broken dependency cache key check in non-strict mode.
* element.py: Hide dependencies which are irrelevant to the ElementTristan van Berkom2020-09-042-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is a large breaking change, a summary of the changes are that: * The Scope type is now private, since Element plugins do not have the choice to view any other scopes. * Element.dependencies() API change Now it accepts a "selection" (sequence) of dependency elements, so that Element.dependencies() can iterate over a collection of dependencies, ensuring that we iterate over every element only once even when we need to iterate over multiple element's dependencies. The old API is moved to Element._dependencies() and still used internally. * Element.stage_dependency_artifacts() API change This gets the same treatment as Element.dependencies(), and the old API is also preserved as Element._stage_dependency_artifacts(), so that the CLI can stage things for `bst artifact checkout` and such. * Element.search() API change The Scope argument is removed, and the old API is preserved as Element._search() temporarily, until we can remove this completely.
* Element.dependencies() now yields ElementProxy wrappers by default.Tristan van Berkom2020-09-041-1/+1
| | | | | | | | | | | | | | | This prepares the ground for policing the dependencies which are visible to an Element plugin, such that plugins are only allowed to see the elements in their Scope.BUILD scope, even if they call Element.dependencies() on a dependency. This commit does the following: * Element.dependencies() is now a user facing frontend which yields ElementProxy elements instead of Elements. * Various core codepaths have been updated to call the internal Element._dependencies() codepath which still returns Elements.
* Move handling of the source `directory` configuration to ElementSourcesjuerg/element-source-cacheJürg Billeter2020-09-0328-28/+28
| | | | | | | | | | | | | | | | | The `directory` value determines where a source is staged within the build root of an element, however, it does not directly affect individual sources. With this change the sources will individually be cached in CAS independent of the value of `directory`. `ElementSources` will use the value of `directory` when staging all element sources into the build root. This results in a cache key change as the `directory` value is moved from the unique key of individual sources to the unique key of `ElementSources`. This is in preparation for #1274.
* Add ElementSourcesCacheJürg Billeter2020-09-034-20/+44
| | | | | | | | | | | | | | | | | | Sources have been cached in CAS individually, except for sources that transform other sources, which have been cached combined with all previous sources of the element. This caching structure may be confusing as sources are specified in the element as a list and this is not a good fit for #1274 where we want to support caching individual sources in a Remote Asset server with a BuildStream-independent URI (especially the `directory` configuration would be problematic). This replaces the combined caching of 'previous' sources with an element-level source cache, which caches all sources of an element staged together. Sources that don't depend on previous sources are still cached individually. This also makes it possible to add a list of all element sources to the source proto used by the element-level source cache.
* tests/sourcecache: Do not inspect uninitialized elementsJürg Billeter2020-09-033-0/+10
|
* tests/sourcecache/push.py: Add test_push_missing_source_after_build()Jürg Billeter2020-09-011-0/+30
| | | | | Test that source push succeeds if the source needs to be fetched even if the artifact of the corresponding element is already cached.
* runcli.py: Configurable index/storage artifact typesJosh Smith2020-08-261-0/+8
| | | | | This enables for seperate index/storage artifact servers to be configured by environment variables passed through tox.
* _elementsources.py: Fix source names in unique keyjuerg/source-cache-keyJürg Billeter2020-08-2528-28/+28
| | | | | | | | | | | | | | The goal was to include the source plugin kind in the element cache key as the unique key of a source may not be unique across different source plugins. This is the source equivalent of the `element-plugin-name` value in the element cache key. However, `Source._get_source_name()` was the wrong method for this as that also includes the key itself, which may not even be set yet. This results in a cache key change. Fixes: 3953bcc6 ("element.py: clobber sources with workspace")
* _fixtures.py: Only get the normal number of threads at the start of sessionBenjamin Schubert2020-08-231-1/+5
| | | | | This ensures that we also have our tests correctly shutting down background threads and not interferring with each other
* buildstream/element.py: Consider "build-root" variable in the cache key ↵tristan/build-root-cache-keyTristan van Berkom2020-08-2028-28/+28
| | | | | | | | | | | | | | | | | | | unconditionally Not all elements use the "build-root" variable, but it is the standard variable to use for the build directory, and the build directory must be considered in the cache key. Handling this unilaterally in the core is safer than delegating this to element implementations, as we have less chance of plugin authors missing this detail and possibly introducing binary variance for artifacts where only the build directory differs (something which happens when the project name or element names change). This commit also updates the hard coded cache keys in the cache key test, so as to ensure every commit passes it's own tests. This fixes #1386.
* tests/cachekey/cachekey.py: Use the same project name for cache key testsTristan van Berkom2020-08-201-1/+1
| | | | | | | | In the test_cache_key_fatal_warnings() test, use the same project name in both generated project directories in order to pass the tests in the case that elements are guaranteed to have differing cache keys for differeing element/project names (which is the case when we consider the "build-root" in the cache key unconditionally).
* Bump minimum version requirement for ruamel.yamlchandan/upgrade-ruamelChandan Singh2020-08-182-0/+36
| | | | | | | | | | | | ruamel.yaml <= 0.16.6 suffers from a bug where ruamel's yaml dumper crashes when used on a sequence that has comments before it. In BuildStream, this manifests in form of issues like #1265. See upstream issue at https://sourceforge.net/p/ruamel-yaml/tickets/335. Also, add a regression test for it. Fixes #1265.
* _frontend/cli: Add --deps `build` & `run` values for `source track --deps`Chandan Singh2020-08-148-2/+65
| | | | | | | This is part of #1349. This patch will conclude the first part of that issue, i.e. ensuring that the possible options for `--deps` are consistent across all commands (with the exception of `--deps plan` that we will handle separately).
* regression-test for ".bst" artifact checkout bugdwinship/test_for_dot_bst_artifact_checkout_bugDouglas Winship2020-08-131-0/+24
|
* casserver.py: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-131-107/+0
| | | | Replaced by Remote Asset API Fetch and Push services.
* _sourcecache.py: Use AssetRemoteJürg Billeter2020-08-131-11/+23
| | | | | This migrates the source cache from the BuildStream Source protocol to the Remote Asset API.
View Lorry Controller Status