| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
| |
Simplify the custom 'handle_message' / 'send_message' protocol by not
requiring a message_type. These message types share a namespace with the
base Job implementation, which could cause trouble.
Introduce a new private '_send_message' to implement the old
functionality.
Subclasses are free to pack a message type into their own messages, this
isn't necessary at present and simplifies existing subclass code.
|
| | |
|
| |\
| |
| |
| |
| | |
Ensure pylint runs in some tests paths
See merge request BuildStream/buildstream!1378
|
| | |
| |
| |
| |
| | |
tests/plugins was missing an __init__.py, which meant pylint
was never run there.
|
| | |
| |
| |
| |
| | |
tests/remoteexecution missing an __init__.py, which meant pylint
was never run there.
|
| |/
|
|
|
| |
tests/sandboxes was missing an __init__.py, which meant pylint
was never run there.
|
| |\
| |
| |
| |
| | |
_loader/loader: cythonize valid_chars_name
See merge request BuildStream/buildstream!1371
|
| |/
|
|
|
|
|
|
|
| |
- Create a new _loader/utils.pyx cython module for functions cythonized
in the loader module.
- Move valid_chars_name from loader to utils and cythonize.
This function is called extensively, and easy to extract
|
| |\
| |
| |
| |
| |
| |
| | |
Improved handling of legacy remotes
Closes #1025
See merge request BuildStream/buildstream!1366
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This creates a new ArtifactRemote class, derived from CASRemote that
extends initialisation to check for an artifact service. This drops the
remote early rather than raising an error on method not found each time
it tries to use it.
Fixes #1025
|
| | |
| |
| |
| |
| |
| | |
And update relavant grpc files.
Part of #1025
|
| |/ |
|
| |\
| |
| |
| |
| | |
Split ChildJob out from Job class
See merge request BuildStream/buildstream!1334
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Make it clearer what happens in which process by splitting out a
'ChildJob', which encapsulates the work that happens in the child
process.
This also makes it possible to control what is transferred to the
child process. This is very useful for adding support for the 'spawn'
method of creating child processes as opposed to the 'fork' method.
|
| | |
| |
| |
| | |
This just does the default behaviour, clearer to remove it.
|
| |/ |
|
| |\
| |
| |
| |
| | |
Optimize _yaml.node_get
See merge request BuildStream/buildstream!1370
|
| | |
| |
| |
| |
| | |
This reduces considerably the number of nodes created and thus speeds
up the calls to node_get.
|
| |/
|
|
|
|
|
|
|
|
|
| |
Calls to `isinstance` can be particularily costly. Using type() is
much faster. The only known case where the `isinstance` was useful
is for dictionnaries where we would ask for a 'Mapping' instead.
Disallowing 'Mapping' for expected_type considerably speeds up the
calls to this functions.
Also add into NEWS
|
| |\
| |
| |
| |
| | |
setup.py: Be more restrictive with BST_CYTHON_TRACE values
See merge request BuildStream/buildstream!1368
|
| |/
|
|
|
|
| |
"0" evaluates to 'True' in python, which incorrectly switched on
the BST_CYTHON_TRACE. Forcing an int for this environment variable
allows us to ensure we are correct.
|
| |\
| |
| |
| |
| | |
Setup.py: Disable linetrace by default and only enable when using coverage
See merge request BuildStream/buildstream!1367
|
| |/
|
|
|
|
|
|
|
| |
It turns out that enabling 'linetrace', does have a runtime cost even
if not enabled in distutils.
Therefore disabling it by default. In order to run coverage, we
therefore need to retranspile the .pyx files with ENABLE_CYTHON_TRACE
set.
|
| |\
| |
| |
| |
| |
| |
| | |
Update docs regarding artifact and source caches
Closes #1024
See merge request BuildStream/buildstream!1362
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Now that we have both artifact and source caches the documentation
is updated to reflect that. Some sections headings/links etc. have been
expanded and changed.
Part of #1025
|
| | |
| |
| |
| | |
Part of #1024
|
| |/
|
|
| |
Part of #1024
|
| |\
| |
| |
| |
| | |
Make more parts of the loader iterative
See merge request BuildStream/buildstream!1365
|
| | |
| |
| |
| |
| |
| |
| | |
When multiple top level elements are specified we need to keep track
of whether we have completed the iterative collection process.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| | |
To reduce stack usage during load, make the LoadElement to MetaElement
conversion be iterative.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |/
|
|
|
|
|
|
| |
The _sort_dependencies() method does not need to be an instance method,
nor does it need to be recursive. This fixes both of those things which
can get us closer to being able to cythonize the loader.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |\
| |
| |
| |
| | |
Rewrite `Loader._check_circular_deps()` to be iterative
See merge request BuildStream/buildstream!1364
|
| | |
| |
| |
| |
| |
| |
| |
| | |
In an effort to reduce the places where the stack might be a problem
as we Cythonize things, rewrite the circular dependency checker to
be iterative.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |/
|
|
|
|
|
|
| |
This did not need to be an instance method, making it static might
improve performance and definitely makes it clear that it's not
actually bound to the loader instances.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |\
| |
| |
| |
| | |
Introduce Cython for better performances
See merge request BuildStream/buildstream!1350
|
| | |
| |
| |
| |
| | |
This allows for a quicker comparison while keeping a good readability
of the code
|
| | |
| |
| |
| |
| |
| |
| | |
The `SYNTHETIC_COUNTER` is an iterator that is called a lot in
_yaml, one for each synthetic node. Cython is not able to optimize
`itertools.counter` well enough. Providing a custom C function allows
to reduce the amount of python code called in this critical codepath
|
| | |
| |
| |
| |
| | |
This requires the addition of a definition file (.pxd), to list
symbols exported.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Providing c definitions for functions allows us to not have to
go back to the python interpreter when inside the module. We
therefore gain more performance.
One gotcha is that keyword only arguments are not valid in cpdef
functions.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
We used to have a tuple to keep file information.
This makes it hard to read, accessing attributes by index.
With an extension class FileInfo, we get better readability,
without sacrificing performance
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`Representer` is the main interface with the `ruamel` library to
parse the yaml files. Rewriting it with Cython introduces significant
performance gains.
Since `Representer` is not a python class anymore, we can't call
`getattr` on it, and therefore have to do a manual switch on the types
of events. While this is harder to read, it is also much more
performant.
Finally, sotp using `yaml.parse`, but call the parser manually, in order
to avoid going in and out of the python code. This part could be made
even better in the future when `ruamel` becomes stable and if they
expose cython definitions, as they are coded in Cython.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
_yaml is a bottleneck in a normal BuildStream run. Typing the
external API allows us to reduce this bottleneck.
Since we type the input variables, we can also remove some asserts
that are checking if the parameters are of the correct type as
Cython will throw TypeError if called incorrectly.
|
| | |
| |
| |
| |
| |
| | |
Internal functions are not used outside the module, and are
therefore easy to refactor and only call from C, leading us to
significant performance gains.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Node used to be a NamedTuple that we used to access by index
for speed reasons. Moving to an extension class allows us to
access attributes by name, making the code easier to read and
less error prone. Moreover, we do gain speed and memory by
doing this move.
Also fix a few places where we would not have an entire `Node`
object but were instead just returning a tuple, missing some entries.
|
| | | |
|
| | | |
|
