| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactored this to remove unneeded complexity in the code base,
as described here:
https://lists.apache.org/thread.html/r4b9517742433f07c79379ba5b67932cfe997c1e64965a9f1a2b613fc%40%3Cdev.buildstream.apache.org%3E
Changes:
* source.py: Added private Source._cache_directory() context manager
We also move the assertion about nodes which are safe to write to
a bit lower in Source._set_ref(), as this was unnecessarily early.
When tracking a workspace, the ref will be none and will turn out
to be none afterwards, it is not a problem that a workspace's node
is a synthetic one, as tracking will never affect it.
* local plugin: Implement get_unique_key() and stage() using
the new context manager in order to optimize staging and
cache key calculations here.
* workspace plugin: Implement get_unique_key() and stage() using
the new context manager in order to optimize staging and
cache key calculations here.
* trackqueue.py: No special casing with Source._is_trackable()
|
|
|
|
|
| |
We don't allow importing symbols from sub-packages of BuildStream, so
any public API must find it's way to the toplevel __init__.py.
|
|\
| |
| |
| |
| | |
.gitlab-ci.yml: Use ported plugins and fdsdk for overnight tests
See merge request BuildStream/buildstream!2068
|
| | |
|
| |
| |
| |
| |
| | |
This was a dead codepath, not used by any sandbox implementation
for any reason, and used to be called inconsistently by some elements.
|
| |
| |
| |
| | |
That API is useless fluff and does not have any affect on anything.
|
| | |
|
|/
|
|
| |
This was missing in the initial implementation.
|
|\
| |
| |
| |
| | |
element.py: Clarify weak and strict key descriptions in __update_cache_keys()
See merge request BuildStream/buildstream!2069
|
|/ |
|
|\
| |
| |
| |
| | |
Implement Element.configure_dependencies()
See merge request BuildStream/buildstream!2032
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
This addresses the feature request to stage dependencies in sysroots
from a couple years back:
https://mail.gnome.org/archives/buildstream-list/2018-August/msg00009.html
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This was actually deadcode, since node.validate_keys() was called
on the configure dictionary without the legacy command steps. If any
element was using the legacy commands, they would have been met with
a load time error anyway.
This commit also updates the cache key test, since removing these
legacy commands affects BuildElement internally in such a way as
to affect the cache keys.
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Instead of relying on Element.search(), use Element.configure_dependencies() to
configure the layout.
Summary of changes:
* scriptelement.py:
Change ScriptElement.layout_add() API to take an Element instead of an element name,
this is now not optional (one cannot specify a `None` element).
This is an API breaking change
* plugins/elements/script.py:
Implement Element.configure_dependencies() in order to call ScriptElement.layout_add().
This is a breaking YAML format change.
* tests/integration: Script integration tests updated to use the new YAML format
* tests/cachekey: Updated for `script` element changes
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Element.stage_dependency_artifacts()
This patch adds a new test plugin which implements
Element.configure_dependencies() in order to offer better flexibility for
testing overlaps.
Newly added tests:
* Test overlap warnings and errors when staging elsewhere than
in the sandbox root.
* Test unstaged files failure modes when staging elsewhere than
in the sandbox root.
* Test various overlap behaviors of OverlapAction, when different
calls to Element.stage_dependency_artifacts() cause overlaps to
occur after staging files into separate directories.
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch implements the essentials of the proposal to extend the
dependency attributes:
https://lists.apache.org/thread.html/r850aeb270200daade44261f16dbad403bf762e553b89dcafa0848fe7%40%3Cdev.buildstream.apache.org%3E
And essentially this will obsolete issue #931 by providing a more
suitable replacement for Element.search().
Summary of changes:
* _loader/loadelement.pyx: The Dependency object now loads the `config` node,
and raises an error if the `config` node is specified on a runtime-only
dependency.
* element.py: Created the new Element.configure_dependencies() virtual method.
If elements implement this method, then a list of all dependencies are
collected at element construction time and given to the implementing
(depending) element.
If a build dependency has more than one `config` specified, then
it will be given to the Element twice, and if there is no `config`
specified, then the tuple will still be given to the element with
a Null `config`.
Elements are provided via a new DependencyConfiguration type.
|
| |
| |
| |
| |
| | |
This helps simplify the following Element.configure_dependencies()
implementing patch.
|
| |
| |
| |
| |
| |
| | |
Test that when the same dependency is added as a build and runtime
dependency separately, they end up being the same dependency which
is both a build & runtime dependency in the loaded build graph.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When the same element is specified multiple times as a direct dependency,
merge and accumulate the results into the already loaded dependency.
* If the same element is a runtime and build dependency separately, it
will be a single dependency of both runtime and build.
* If either of the dependencies are `strict`, it will be a strict
dependency.
The build graph retains the invariant that an element only ever depends
on another element once directly, only the YAML can express the same
dependency differently more than once, and the results are accumulated.
This consequently remoces LoadErrorReason.DUPLICATE_DEPENDENCY as this is
no longer relevant.
|
| |
| |
| |
| | |
LoadErrorReason.DUPLICATE_DEPENDENCY
|
| |
| |
| |
| |
| |
| | |
This is a bit nicer than relying on strings in the Symbol enumeration,
and allows for some bitwise operations so we can test for BUILD or
RUNTIME.
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit:
* Removes testing of the deprecated `fail-on-overlap` project configuration
option, this is going away soon and unneeded.
* Tests that warnings are issued whenever they should be (some tests
were happy to see a successful run but failed to check for an
expected warning).
* Test error/warning more evenly across tests, some were missing the
warning mode.
* Use `bst show` instead of `bst build` for the undefined_variable
test, it should fail without needing a build.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Setup the OverlapCollector in Element.stage() routines, and ensure we
call OverlapCollector.start_session() and OverlapCollector.end_session()
in the right places.
This adds the OverlapAction `action` parameter to the Element.stage_artifact()
and Element.stage_dependency_artifacts() APIs so that Elements can choose
how to behave when multiple artifact staging calls overlap with files staged
by previous artifact staging calls.
|
| | |
|
| |
| |
| |
| |
| | |
Makes the warning fatal if we fail to stage a file because it would have
otherwise overwritten a non-empty directory.
|
| |
| |
| |
| |
| | |
Used to define the behavior of multiple calls to Element.stage_artifact()
and Element.stage_dependency_artifacts()
|
| |
| |
| |
| |
| | |
This test element was also staging artifacts in Element.assemble(), which is
now illegal.
|
| |
| |
| |
| | |
Staging artifacts at Element.assemble() time is now illegal
|
|/
|
|
|
| |
It will now be illegal to call Element.stage_dependency_artifacts() outside
of the Element.stage() abstract method.
|
|\
| |
| |
| |
| |
| |
| | |
Fix overnight tests
Closes #1365
See merge request BuildStream/buildstream!2066
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Point the FDSDK branch to the newly branched:
freedesktop-sdk-20.08beta.1-buildstream2-overnight-tests
We used to use freedesktop-sdk-20.08beta.1-buildstream2, but this
branch is apparently used by unknown parties for unknown reasons.
Use a branch that has the specific purpose of BuildStream overnight
tests, so we can safely update the branch as needed.
* Point to latest commit sha on bst-plugins-experimental which works
with the latest BuildStream APIs.
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
This was failing because we are trying to run the py38-randomized
tox environment in the test, which is not supported.
Use py38-randomized-nocover instead.
|
| |
| |
| |
| |
| |
| |
| |
| | |
In december, Ben removed the "external" tox environment marker,
in commit: 2d2bd16a1f931b4aece069baf94257df22e869cc
The overnight tests have still been trying to run tests with
environments such as `py36-randomized-external` which is invalid.
|
|/
|
|
|
|
| |
This allows plugins to keep making statements such as `element in dependencies`
or `elementA is elementB`, which was currently broken due to creating proxies
on demand.
|
|\
| |
| |
| |
| | |
Improve element cache state handling
See merge request BuildStream/buildstream!2067
|
| |
| |
| |
| |
| |
| | |
This tests that, in non-strict mode, a cached artifact matching the
strict cache key is preferred to a more recent artifact matching only
the weak cache key.
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
In non-strict mode, `Element._pull_pending()` checked whether the strict
artifact is already in the local cache to determine whether to attempt
pulling the strict artifact from a remote cache. However, when staging a
cached element, BuildStream always used the weak cache key. The weak
cache key is not guaranteed to point to the same artifact as the strict
cache key even if the strict artifact is cached.
This removes the `Element.__strict_artifact` instance member to keep
strict artifact handling contained in `__update_artifact_state()`.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We can always calculate the weak cache key if we can calculate the
strict cache key and having the weak cache key available without the
strict cache key doesn't provide any benefits.
With this change each element either has both of these cache keys
calculated or neither of them. This reduces the number of states an
element can be in, reducing the risk of state handling bugs.
|
| |
| |
| |
| |
| |
| | |
As `bst build --track` and unstable workspace cache keys have been
removed, dynamic cache key updates across the element graph are no
longer needed.
|
|/
|
|
|
|
| |
As `bst build --track` and unstable workspace cache keys have been
removed, dynamic cache key updates across the element graph are no
longer needed.
|