| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
This fix is applied to plugins bzr, git, patch.
Fixes #543 #544 #555.
|
| |
|
|
|
| |
magic_timestamp is moved into file scope so other classes
can use it.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This address issue #491.
When attempting to run buildstream with a configuration specifying
a cache quota larger than your available disk space, buildstream
will alert the user and exit.
Note:
This takes into consideration your current cache usage and
therefore restricts the overall size of your artifact cache folder.
|
| |
|
|
| |
The separator is useful in source files other than _project.py
|
| | |
|
| | |
|
| |
|
|
| |
Fixes #424
|
| |
|
|
| |
Fixes issue #380
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Using setuptools_scm had a couple of bad problems:
o Unexpected versioning semantics, setuptools_scm would
increment the micro version by itself in the case that
we derive a version number from something that is not a tag,
making the assumption that we are "leading up to" the next
micro version.
People mostly dont expect this.
o When installing in developer mode, i.e. with `pip3 install --user -e .`,
then we were always picking the generated version at install time
and never again dynamically resolving it.
Many of our users install this way and update through git, so it's
important that we report more precise versions all the time.
This commit needs to make a series of changes at the same time:
o Adds versioneer.py to the toplevel, this is used by setup.py
for various activities.
This is modified only to inform the linter to skip
o Adds buildstream/_version.py, which is generated by versioneer
and gives us the machinery to automatically derive the correct version
This is modified only to inform the linter to skip
o Adds a .gitattributes file which informs git to substitute
the buildstream/_version.py file, this is just to ensure that
the versioning output would work if ever we used `git archive`
to release a tarball.
o Modifies setup.py and setup.cfg for versioneer
o Modifies utils.py and _frontend/cli.py such as to avoid importing
the derived version when running bash completion mode, we dont
derive the version at completion time because this can result
in running a subprocess (when running in developer install mode)
and is an undesirable overhead.
o Updates tests/frontend/version.py to expect changed version output
|
| | |
|
| |
|
|
|
| |
A helper function to identify if we are running in the main process
or not.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The os.path.realpath() function is expensive and we call it many times,
to the point that os.path.realpath() calls make up around 40% of the
total time spent in Element.stage_artifact().
The cleanest way to fix this is with a `functools.lru_cache()` wrapper
that caches recently used values. None of the code in question can be
removed (as the tests added in the previous commit will demonstrate).
I tested this by running `bst shell base/base-system.bst true` in
the GNOME modulesets project.
o Without this patch there are 240,019 calls os.path.realpath()
o With this patch there are 10,379 calls to os.path.realpath()
o If we increase the cache size to 128 items, there are 10,359 calls
to os.path.realpath().
o If we reduce the cache size to 32 items, there are 10,426 calls.
o In all cases the number of *unique* calls is 10,327.
This fixes issue #174.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This removes most of the usages of pkg_resources from the codebase, helping
the goal of getting rid of that completely.
With this change, version number is generated during install and embedded into
`__version__` which is then imported by root level `__init__`. From there,
it is used by other parts of the codebase when needed.
Generated `__version__` file is ignored and not tracked by git to prevent
unnecessary 'changes' messages and accidental commits of that file.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This extends the 'artifacts' configuration block such that a list of
`url` mappings can be given instead of a single entry. For example:
artifacts:
- url: http://example.com/artifacts1
- url: ssh://ostree@example.com/artifacts2
The OSTreeCache class is updated to set up multiple remotes and query
remote refs from all of them.
There are no automated tests for this yet.
Empty URLs ('') now raise an exception. They cause breakages internally
if we allow them through, and they can only occur if the user or our
tests are misconfiguring things somehow.
We report failure to fetch from the cache by printing a message to
stderr for now. This is because BuildStream's actual logging
functionality can't be used during frontend init -- see issue #168.
|
| |
|
|
|
|
|
|
|
|
|
| |
This is needed because plugins make use of this function to
generate a cache key which must be stable.
In addition to the above, this patch also mitigates the performance
hit of sorting, and allows _process_list() to function to iterate
through the generator when additional sorting is not needed.
This patch is an enhanced version of Sam Thursfield's patch on MR !216
|
| |
|
|
|
| |
This avoids a circular dependency between the 'utils' and '_yaml'
modules.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This is a context manager that can be used to divert file writes into a
temporary file, which is then renamed into place once writing is
complete. It is primarily intended for use by source plugins which
download files, so they can ensure that their downloads appear atomic
and there is no risk of leaving half-downloaded files in the cache.
So far this is not used in the core, but it is needed by the Docker
source plugin that is proposed for the bst-external plugins repo. See:
https://gitlab.com/BuildStream/bst-external/merge_requests/9
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Outline of changes to exceptions:
o Now BstError base class has a `domain` and `reason` member,
reason members are optional.
o All derived error classes now specify their `domain`
o For LoadError, LoadErrorReason defines the error's `reason`
o Now the scheduler `job` class takes care of sending the error
reason and domain back to the main process where the last
exception which caused a child task to fail can be discretely stored.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Tested by inflating the chance of hitting the race. First, insert a
sleep between opening the terminator context and starting the process,
then:
python3 -c 'import buildstream.utils; import os; \
buildstream.utils._call(["echo", "hello"], True); \
print(os.getpid())' & sleep 1; kill $!
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Report UtilError instead of OSError and similar python errors.
Also ensure we catch system errors and raise UtilError with
descriptive text instead; for the user experience; this is the
difference between:
o A FAILURE message with a description as to
what went wrong (exception handled with UtilError)
o A BUG message with the unhandled system error printed
with a stack trace (exception left unhandled)
Also, UtilsError and ProgramNotFoundError are now public exceptions
declared in utils.py, where they will appear in the documentation.
|
| | |
|
| | |
|
| |
|
|
|
| |
These are not used anywhere internally and are not really desirable
to use, as usually you need only care about moving a toplevel directory.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
The 'result' field is optional, and currently used only by the internals
of {copy,move,link}_files. I think it makes sense to mandate that it's
called as a keyword arg in future.
|
| |
|
|
| |
Hide all of buildstream's internal exceptions from the API surface.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Profiling suggested that the cumulative time spent in os.path.relpath()
was the dominant cost of utils.list_relative_paths().
Try to call this only once per directory walked.
In Python 3.5, we can optimise further by using os.scandir() and
maintaining relative paths as we go.
|
| |
|
|
|
| |
utils._force_rmtree() was overwriting it's path directory, and
then only recursively removing the last entry which was walked.
|
| | |
|
| | |
|
| |
|
|
|
| |
This little bit of code was being repeated a bunch of times
already throughout buildstream, looks sane enough to offer a utility.
|
| | |
|
| |
|
|
| |
Fetches the numeric major/minor version of the BuildStream package.
|
| | |
|
| |
|
|
|
|
|
| |
Support for prefix=None and suffix=None was added in python 3.5.
The default for suffix happens to be "", and the prefix happens to be
"tmp" (though that's an implementation detail), but it's sufficient,
and simpler than writing conditionals that omit certain args.
|
| |
|
|
|
| |
A temporary directory context manager which cleans itself out
on sigterm.
|
