| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Artifacts can be cached explicitly with an empty `build tree` when
built via the cli main options or user config for all or only
successful build artifacts. Default behaviour is to still create
and cache all expected buildtrees.
element.py: _cache_artifact() Check if context for cache_buildtrees
has been set to always or failure with a corresponding build
result, if not skip attempting to export the build-root. Element
types without a build-root are cached with an empty buildtree
regardless. Update _stage_sources_at() to warn the user that the
buildtree import is empty.
tests/integration: Add test to artifact.py for the optional caching
of buildtree content from bst build. Rename build-tree.py to
shellbuildtrees.py to reflect included test cases, add test for
empty buildtree warning and failure option.
NEWS: Add entry for new option.
|
| |
|
|
|
|
| |
not _cached_sucess() could resolve to true if the element wasn't
cached at all. switch to _cached_failure() to ensure condition
reflects expected artifact state
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
_context.py: Add cache_buildtrees global user context, the default
of which is set to by default to 'always' via the addition of
cache-buildtrees to userconfig.yaml cache group. 'failure' & 'never'
can be given as valid options.
app.py & cli.py: Add --cache-buildtrees as a bst main option, which
when passed with a valid option can override the default or user
defined context for cache_buildtrees.
tests/completions/completions.py: Update for the added flag.
|
| |\
| |
| |
| |
| | |
Return all directories in list_relative_paths()
See merge request BuildStream/buildstream!1139
|
| | |
| |
| |
| | |
This matches the change in utils.list_relative_paths().
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Returning only empty directories leads to inconsistencies when computing
a manifest by combining results from multiple list_relative_paths()
calls as done by the compose plugin.
I.e., the same directory may be empty in one dependency and non-empty in
another dependency. The merged file list will still contain that
directory even though it's no longer empty.
This inconsistency causes problems when calculating differences between
manifests. Returning all directories fixes these inconsistencies.
This is a change in API behavior.
|
| | |
| |
| |
| |
| | |
This is required to fix tests with the following commit that changes
list_relative_paths() to return all directories.
|
| |/
|
|
|
|
|
|
|
| |
Most split rules already included the relevant directories themselves in
addition to the directory contents. Add the missing bin, sbin, and
libexec directories.
This is required to fix tests with the following commit that changes
list_relative_paths() to return all directories.
|
| |\
| |
| |
| |
| | |
Refactor artifact log command
See merge request BuildStream/buildstream!1101
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
_classify_artifacts() no longer filters glob expressions by first
obtaining a list of ALL refs locally cached. We now only obtain refs
specified by the glob expression.
Furthermore, the Project.element_path is used to start searching for
globbed elements as opposed to the Project.directory.
|
| | |
| |
| |
| |
| | |
This commit ensures that CASCache.list_refs(), and
ArtifactCache.list_artifacts(), can both handle glob expressions.
|
| | |
| |
| |
| | |
This method has been moved to Stream, where is it used there exclusively.
|
| | |
| |
| |
| |
| | |
The loading of elements and the handling of artifacts does not belong
in this module. Such logic should be invoked using the Stream API
|
| | |
| |
| |
| |
| |
| |
| | |
A CasBasedDirectory object of an artifacts logs can be obtained
with ArtifactCache.get_artifacts_log(). This ultimately calls
CASCache.get_top_level_dir() to obtain a CasBasedDirectory
of an artifact's subdirectory (or subdirectories).
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
* There is no need for this method to use a cas object.
* Search for artifact globs in the project's element path
* An artifact key is always 64 chars long
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
This object should be used when we want to handle artifact refs directly
from the command line.
An ArtifactElementError has also been added to _exceptions.py
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
| |
This commit removes the method ArtifactCache.get_artifact_fullname()
and replaces it with Element.get_artifact_name()
Given a key, we are now able to construct the full name of any of an
element's artifacts.
|
| |\
| |
| |
| |
| | |
buildstream/_cas/cascache.py: Set 0644 rights to pulled files
See merge request BuildStream/buildstream!1144
|
| |/
|
|
| |
This was broken by 5ef19a0b31df84caed1e41719ef7ea5c6bd8a8bc.
|
| |\
| |
| |
| |
| |
| |
| | |
Stage junctions into .bst instead of a tmpdir
Closes #895
See merge request BuildStream/buildstream!1134
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since junction check-outs are now persistent, there is no need to clean
up temporary directories once finished.
Now, junction checkouts are stored within the top-level project dir and
will be cleaned-up when that directory is removed.
This involves changes in:
* loader.py: Remove cleanup logic and passing-around of tmpdir.
* _project.py: Remove passing around tmpdir and calling loader's
cleanup.
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| | |
Further optimisations
See merge request BuildStream/buildstream!1131
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
While get_strict() doesn't look expensive per-se, it is called so many
times that it is valuable to cache the result once computed. Since I
don't think it can change once it is computable, cache it immediately
that becomes possible and we save 20s in my test case.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
By re-using the isinstance replacements from earlier commits and
using a tuple of the string constants for checking for composition
markers, we reduce the cost of node_final_assertions by two thirds
in basic testing.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| | |
A non-trivial proportion of time pre-scheduler in `bst build` is spent
copying (or chain-copying) nodes. Approximately a quarter of the time
spent in that effort is in `isinstance()`. This removes that CPU load.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We know that nodes are typically one of:
str, list, dict, bool, tuple, NoneType or our ChainMap
Of these, dict and ChainMap are Mapping, only list is list
and the rest are returned unchanged. We can reduce/defer our use
of isinstance here, dramatically, improving performance.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |/
|
|
|
|
|
|
|
|
| |
We were indiscriminately retrieving the node's provenance data in the
`node_get()` function which was accounting for approximately a third of
the total runtime of `node_get()` which dominates pre-scheduler time in
`bst build`. This change ameliorates that situation by only retrieving
the provenance data when it's actually needed.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Test with current fedora release: 29
See merge request BuildStream/buildstream!1137
|
| | |
| |
| |
| | |
Fedora 29 includes python 3.7 so this is not needed anymore
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| | |
_frontend: Allow printing dependencies using `bst show`
Closes #890
See merge request BuildStream/buildstream!1121
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At present, there isn't an easy way to print anything from `bst show`
that would give the users an idea of what the dependency graph looks
like. One could use things like `--deps build`, but that will just print
a list, without any information about the dependency edges.
Add `%{deps}`, `%{build-deps}` and `%{runtime-deps}` format strings to
`bst show` that would simply print the list of all dependencies, build
dependencies and runtime dependencies respectively.
Summary of changes:
* buildstream/_frontend/cli.py: Add help for new format symbols.
* buildstream/_frontend/widget.py: Add support for new format symbols
for dependencies.
* tests/frontend/show.py: Add tests for new format symbols.
|
| |\
| |
| |
| |
| | |
Increase read buffer size to improve performance
See merge request BuildStream/buildstream!1142
|
| | |
| |
| |
| |
| | |
Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
factor 4, according to a very simple benchmark.
|
| |/
|
|
|
| |
Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
factor 4, according to a very simple benchmark.
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out
See merge request BuildStream/buildstream!1145
|
| |/
|
|
| |
are sorted out
|
| |\
| |
| |
| |
| | |
More user-friendly reporting on include errors
See merge request BuildStream/buildstream!891
|
| | |
| |
| |
| |
| | |
Don't create and remove temp dirs unnecessarily when they are not used,
looks like these were just copy-pastes without intended side-effects.
|
| | |
| |
| |
| | |
Avoid an unnecessary call to os.path.join().
|
