summaryrefslogtreecommitdiff
path: root/tests/frontend
Commit message (Collapse)AuthorAgeFilesLines
* _stream.py: Added internal _track_cross_junction_filter()Tristan van Berkom2020-12-231-1/+1
| | | | | | | | | | Replaces Pipeline method `track_cross_junction_filter()`. This changes the error domain for invalid cross junction tracking, so updating the following two test cases: * testing/_sourcetests/track.py * tests/frontend/track.py
* tests/frontend/push.py: Fix fallout from recent teststristan/fix-broken-testTristan van Berkom2020-12-221-3/+1
| | | | | | | | | | | This test was broken from the following commits: 98c807002cf3beb2110695083450a42fe8feefd0 9a124386a0ba8995e7cfd92e2c7d8fb23854f7e4 As of these commits, a Cli() instance requires it's directory to not be created in advance, this was not caught in CI because we skip the tests in CI where the runners lack proper subsecond mtime precision.
* plugins/elements/stack.py: Require all dependencies be build & run.Tristan van Berkom2020-12-105-24/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stack elements cannot be build-only dependencies, as this would defeat the purpose of using stack elements in order to directly build-depend on them. Stack element dependencies must all be built in order to build depend on them, and as such we gain no build parallelism by allowing runtime-only dependencies on stack elements. Declaring a runtime-only dependency on a stack element as a whole might still be useful, but still requires the entire stack to be built at the time we need that stack. Instead, it is more useful to ensure that a stack element is a logical group of all dependencies, including runtime dependencies, such that we can guarantee cache key alignment with all stack dependencies. This allows for stronger reliability in commands such as `bst artifact checkout`, which can now reliably download and checkout a fully built stack as a result, without any uncertainty about possible runtime-only dependencies which might exist in the project where that artifact was created. This consequently closes #1075 This also fixes the following tests such that the no longer require build-depends or runtime-depends to work in stack elements: * tests/frontend/default_target.py: Was not necessary to check results of show, these stacks were set to runtime-depends so that they would have the same buildable state as their dependencies when shown. * tests/format/dependencies.py: tests/frontend/pull.py, test/frontend/show.py, tests/integration/compose.py: These tests were using specific build/runtime dependencies in stacks, but for no particular reason.
* Refactor ArtifactElement instantiationTristan van Berkom2020-12-073-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When instantiating an ArtifactElement, use an ArtifactProject to ensure that the Element does not accidentally have access to any incidentally existing project loaded from the current working directory. Also pass along the Artifact to the Element's initializer directly, and conditionally instantiate the element based on it's artifact instead of based on loading YAML configuration. Fixes #1410 Summary of changes: * _artifactelement.py: - Now load the Artifact and pass it along to the Element constructor - Now use an ArtifactProject for the element's project - Remove overrides of Element methods, instead of behaving differently, now we just fill in all the blanks for an Element to behave more naturally when loaded from an artifact. - Avoid redundantly loading the same artifact twice, if the artifact was cached then we will load only one artifact. * element.py: - Conditionally instantiate from the passed Artifact instead of considering any YAML loading. - Error out early in _prepare_sandbox() in case that we are trying to instantiate a sandbox for an uncached artifact, in which case we don't have any SandboxConfig at hand to do so. * _stream.py: - Clear the ArtifactProject cache after loading artifacts - Ensure we load a list of unique artifacts without any duplicates * tests/frontend/buildcheckout.py: Expect a different error when trying to checkout an uncached artifact * tests/frontend/push.py, tests/frontend/artifact_show.py: No longer expect duplicates to show up with wild card statements which would capture multiple versions of the same artifact (this changes because of #1410 being fixed)
* tests/frontend/push.py: Skip expiry test without subsecond mtimeJürg Billeter2020-11-241-0/+10
| | | | | Skip an artifact expiry test in the case we don't have subsecond mtime precision.
* tests/frontend/show.py: Added test for glob pattern matches.tristan/fix-glob-handlingTristan van Berkom2020-11-191-0/+46
|
* tests/frontend/artifact_show.py: Added tests for glob pattern matches.Tristan van Berkom2020-11-199-0/+121
| | | | | | This tests a few glob patterns through `bst artifact show` and also asserts that globs which match both elements and artifacts will produce an error.
* _stream.py: Use consistent glob handling with utils.glob()Tristan van Berkom2020-11-191-1/+1
| | | | | | | | | | | | | We should not have a different globbing behavior than split rules for the command line. This should also make artifact globbing slightly more performant, as the regular expression under the hood need not be recompiled for each file being checked. This commit also updates tests/frontend/artifact_list_contents.py to use a double star `**` (globstar syntax) in order to match path separators as well as all other characters in the list contents command.
* _stream.py: Only optionally try to glob for artifact namesTristan van Berkom2020-11-193-7/+3
| | | | | | | | | | | | | | | | | | | | | | | | | The patch does the following things: * Ensure that we only ever try to match artifacts to user provided glob patterns if we are performing a command which tries to load artifacts. * Stops being selective about glob patterns, if the user provides a pattern which does not end in ".bst", we still try to match it against elements. * Provide a warning when the provided globs did not match anything, previously this code only provided this warning if artifacts were not matched to globs, but not elements. * tests/frontend/artifact_delete.py, tests/frontend/push.py, tests/frontend/buildcheckout.py: Fixed tests to to not try to determine success by examining the wording of a user facing message, use the machine readable errors instead. Fixes #959
* tests/frontend/logging.py: Test log lines for correct element names/keysTristan van Berkom2020-10-278-4/+174
| | | | | | | | | | | | When printing log lines to the master log, we ensure that log lines are printed with the element name and cache key which are related to the task from which the messages are being issued. When printing log lines to task specific log lines, we prefer to print the element names and cache keys which pertain to the element from which the log line was actually issued. This new tests asserts this behavior.
* Restore task element name / element name distinction in UITristan van Berkom2020-10-272-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This behavior has regressed a while back when introducing the messenger object in 0026e379 from merge request !1500. Main behavior change: - Messages in the master log always appear with the task element's element name and cache key, even if the element or plugin issuing the log line is not the primary task element. - Messages logged in the task specific log, retain the context of the element names and cache keys which are issuing the log lines. Changes include: * _message.py: Added the task element name & key members * _messenger.py: Log the element key as well if it is provided * _widget.py: Prefer the task name & key when logging, we fallback to the element name & key in case messages are being logged outside of any ongoing task (main process/context) * job.py: Unconditionally stamp messages with the task name & key Also removed some unused parameters here, clearing up an XXX comment * plugin.py: Add new `_message_kwargs` instance property, it is the responsibility of the core base class to maintain the base keyword arguments which are to be used as kwargs for Message() instances created on behalf of the issuing plugin. Use this method to construct messages in Plugin.__message() and to pass kwargs along to Messenger.timed_activity(). * element.py: Update the `_message_kwargs` when the cache key is updated * tests/frontend/logging.py: Fix test to expect the cache key in the logline * tests/frontend/artifact_log.py: Fix test to expect the cache key in the logline Fixes #1393
* tests/frontend/track.py: Conditionally skip a testtristan/conditional-subsecond-precisionTristan van Berkom2020-10-251-0/+6
| | | | | | Skip a test which relies on mtimes differing within a short timespan, this will fail if it happens fast enough (which it should) on systems which do not support subsecond precision mtimes.
* _stream.py: Pull missing artifacts in push()juerg/pushJürg Billeter2020-09-291-10/+2
| | | | | | | | | | | As per #819, BuildStream should pull missing artifacts by default. The previous behavior was to only pull missing buildtrees. A top-level `--no-pull` option can easily be supported in the future. This change makes it possible to use a single scheduler session (with concurrent pull and push jobs). This commit also simplifies the code as it removes the `sched_error_action` emulation, using the regular frontend code path instead.
* tests/frontend/track.py: Test behavior of SKIP messagesTristan van Berkom2020-09-241-0/+79
| | | | | | | | This test checks that: * We get SKIP messages for tracking local sources * We get SKIP messages for tracking workspaced elements * We go no messages at all for elemenents which have no sources
* tests/frontend/overlaps.py: Test multiple calls to ↵Tristan van Berkom2020-09-188-2/+192
| | | | | | | | | | | | | | | | | | | | Element.stage_dependency_artifacts() This patch adds a new test plugin which implements Element.configure_dependencies() in order to offer better flexibility for testing overlaps. Newly added tests: * Test overlap warnings and errors when staging elsewhere than in the sandbox root. * Test unstaged files failure modes when staging elsewhere than in the sandbox root. * Test various overlap behaviors of OverlapAction, when different calls to Element.stage_dependency_artifacts() cause overlaps to occur after staging files into separate directories.
* tests/frontend/overlaps.py: Test CoreWarnings.UNSTAGED_FILESTristan van Berkom2020-09-186-2/+38
|
* tests/frontend/overlaps.py: Minor cleanup and refactoringTristan van Berkom2020-09-181-37/+32
| | | | | | | | | | | | | | | | | This commit: * Removes testing of the deprecated `fail-on-overlap` project configuration option, this is going away soon and unneeded. * Tests that warnings are issued whenever they should be (some tests were happy to see a successful run but failed to check for an expected warning). * Test error/warning more evenly across tests, some were missing the warning mode. * Use `bst show` instead of `bst build` for the undefined_variable test, it should fail without needing a build.
* tests/frontend/rebuild.py: Add test_modify_and_revertJürg Billeter2020-09-161-0/+41
| | | | | | This tests that, in non-strict mode, a cached artifact matching the strict cache key is preferred to a more recent artifact matching only the weak cache key.
* tests/frontend/rebuild.py: Add assertion for built elementsJürg Billeter2020-09-161-0/+8
|
* tests/frontend/push.py: Test artifact push with globJürg Billeter2020-09-101-0/+30
|
* tests/frontend/buildcheckout.py: Add non-strict cache key testjuerg/cache-keyJürg Billeter2020-09-081-0/+34
| | | | | This is a regression test for the previously broken dependency cache key check in non-strict mode.
* Bump minimum version requirement for ruamel.yamlchandan/upgrade-ruamelChandan Singh2020-08-182-0/+36
| | | | | | | | | | | | ruamel.yaml <= 0.16.6 suffers from a bug where ruamel's yaml dumper crashes when used on a sequence that has comments before it. In BuildStream, this manifests in form of issues like #1265. See upstream issue at https://sourceforge.net/p/ruamel-yaml/tickets/335. Also, add a regression test for it. Fixes #1265.
* _frontend/cli: Add --deps `build` & `run` values for `source track --deps`Chandan Singh2020-08-148-2/+65
| | | | | | | This is part of #1349. This patch will conclude the first part of that issue, i.e. ensuring that the possible options for `--deps` are consistent across all commands (with the exception of `--deps plan` that we will handle separately).
* regression-test for ".bst" artifact checkout bugdwinship/test_for_dot_bst_artifact_checkout_bugDouglas Winship2020-08-131-0/+24
|
* tests/frontend/push.py: Pull complete artifactJürg Billeter2020-08-131-7/+8
| | | | | | | | Use a separate Cli instance with a separate local cache for the second pull in `test_recently_pulled_artifact_does_not_expire()` to ensure the complete artifact is pulled. If only a part of the artifact is pulled, there is no guarantee that the other blobs of that artifact won't expire.
* _frontend/cli.py: Fix `artifact checkout` with no argumentsChandan Singh2020-08-111-0/+12
| | | | | | | | | | | | | | | When no arguments are passed to `artifact checkout` command, it currently crashes. We actually have a check of this, but we crash before that becasue this method tries to read the value of `target` before we've had time to check for it. That can only be done correctly after the app has been initialized. So, refactor that bit of the method to run after we've checked that we are working with a non-empty target. Also, add a regression test for it. Fixes #1367.
* _yaml.pyx: Make shortname a mandatory argument to _yaml.load()Tristan van Berkom2020-08-103-13/+13
| | | | | | | | | | | | | This ensures that important calls to this function do give some thought to providing a reasonable shortname, which will be used as a display name in errors. This continues to support `None` as a shortname, which is used in various tests which don't need to provide a reasonable user facing error. The buildstream.testing module now exports a `load_yaml` function which only takes a filename and no shortname.
* element.py: Drop cache checks from `_tracking_done()`Jürg Billeter2020-08-061-3/+2
| | | | | | This was a workaround for inconsistent error handling in the frontend. As the error handling is now fixed, drop these cache checks after tracking.
* tests/frontend/overlaps.py: Test undefined variablesTristan van Berkom2020-07-222-1/+27
| | | | | Ensure that we get the expected provenance when expanding a variable included in an overlap whitelist entry.
* loader.py: use the full name of the element when looking up a workspaceAbderrahim Kitouni2020-07-161-2/+5
| | | | | This fixes a bug where buildstream would ignore the opened workspace on a cross-junction element.
* tests/frontend/remote-caches.py: Split test_source_artifact_cachesjuerg/source-pushJürg Billeter2020-07-061-4/+26
| | | | | Use `bst source push` in a new test instead of manually deleting parts of a remote cache.
* Add `bst source push` commandJürg Billeter2020-07-061-0/+1
|
* tests/frontend/push.py: Add test_push_update_after_rebuildJürg Billeter2020-06-301-0/+40
| | | | | This is a regression test to skip push only if server has identical artifact.
* tests/frontend: Renaming some projects to avoid new conflictsTristan van Berkom2020-06-192-2/+2
|
* tests/frontend/workspace.py: Removing testsTristan van Berkom2020-06-111-30/+0
| | | | | | Remove tests which check for a user message to be issued upon closing a workspace who's metadata was used to launch BuildStream and find the BuildStream project directory.
* cascache.py: Fix file modes in checkout()Jürg Billeter2020-06-101-6/+8
| | | | | Do not copy file mode from casd object file. Do not change non-executable mode bits if file is executable.
* tests/frontend/pull.py: Add test for dynamic build planJürg Billeter2020-05-271-0/+47
| | | | Make sure build-only dependencies are pulled only when necessary.
* Ensure there are no duplicates in Elements.dependencies()Chandan Singh2020-05-182-4/+21
| | | | | | | | | | | | | | When we are not recursing, `Element.dependencies()` uses a much more light weight codepath since it just needs to print the direct dependencies. However, this simple codepath was not accounting for duplicates, in case something is both a build time and run time dependency. One way this manifested itself was in `bst show --format %{deps}`, but it would also affect anything that was using this method to iterate on the dependencies. Fixes #1308.
* _frontend/cli: Support "build" and "run" values for `artifact push --deps`Chandan Singh2020-05-131-12/+23
|
* _frontend/cli: Support "build" and "run" values for `artifact pull --deps`Chandan Singh2020-05-131-8/+20
|
* _frontend/cli: Support "build" and "run" values for `source fetch --deps`Chandan Singh2020-05-137-0/+68
|
* _frontend/cli: Support "build" value for `bst build --deps`Chandan Singh2020-05-131-0/+16
| | | | | | This can be quite handy when one is preparing to open a build shell. This originally came up in https://gitlab.com/BuildStream/buildstream/-/issues/685#note_105460896.
* Update all packages requirementsBenjamin Schubert2020-05-113-5/+3
| | | | Also fix linting errors coming with new version of pylint
* plugin.py/pluginfactory.py: Implementing BST_MIN_VERSIONTristan Van Berkom2020-04-297-0/+21
| | | | | | | | | | | | | | The BST_MIN_VERSION guards assert that the BuildStream core which loaded the plugin is compatible with the plugin itself. This commit adds BST_MIN_VERSION to the base plugin.py with documentation informing Plugin authors how to set the minimum version, and also adds the assertions at plugin loading time in pluginfactory.py. This commit also: * Adds the BST_MIN_VERSION specification to all current core plugins * Adds the BST_MIN_VERSION specification to plugins used in test cases
* Plugin loading refactor, removing all versioningTristan Van Berkom2020-04-295-9/+7
| | | | | | | | | | | | | | | | Plugin format versioning was decided to be removed for local plugins and any plugins for which we do not load an explicitly provided plugin. For pip, this will be handled with a standard distutils/setuptools approach, allowing users to specify pip style version boundaries in the plugin origins. This patch refactors plugin loading so that all related code goes into the private _pluginfactory module, a new small PluginOrigin type was added to better manipulate loaded origins. Test cases have been removed and will be readded in a following commit, adjusted to new expectations.
* element.py: Drop BST_VIRTUAL_DIRECTORYJürg Billeter2020-04-281-2/+0
| | | | `Sandbox.get_directory()` no longer exists.
* Replace format-version with min-versionTristan Van Berkom2020-04-2531-41/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * "min-version" is specified as a <major>.<minor> point version and uses the installed BuildStream version instead of having a separate versioning number for the format. * The presence of "format-version" is now used to indicate that we might be loading a BuildStream 1 project. * For now, where parsing the version at startup is concerned, and also where `bst init` is concerned, we artificially bump the detected BuildStream version to 2.0 if we detect a version < 2.0, these exceptions can be removed once 2.0 is tagged and released. Summary of changes: _project.py: Now parse "min-version" and detect "format-version" to warn about loading a BuildStream 1 project _versions.py: Remove obsolete BST_FORMAT_VERSION numbers from here data/projectconfig.yaml: Remove old "format-version" from defaults utils.py: Added new private _parse_version() helper function, and another _get_bst_api_version() to get an adjusted API version. frontend/app.py, frontend/cli.py: Updated `bst init` implementation testing (buildstream.testing): Updated testing utilities to generate and use projects with min-version instead of format-version. tests and examples: Updated to use min-version across the board.
* tests: Correctly handle resources when opening files in testsbschubert/no-warningsBenjamin Schubert2020-04-231-39/+41
| | | | | | | This ensures we do not leak resources during our tests and that we close everything nicely. This removes multiple warnings that is thrown by our code.
* tests: Stop using tmpdir_factory.mkdtemp("")bschubert/fix-mktemp-usageBenjamin Schubert2020-04-232-14/+15
| | | | | Using "" as a directory is invalid starting from pytest 5.4.0, let's use the __file__ name instead
* tests/frontend/push.py: Add test_push_after_rebuildjuerg/artifact-blob-not-foundJürg Billeter2020-04-203-1/+74
|
View Lorry Controller Status