summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * element.py: Remove update_state_recursively()James Ennis2019-07-162-78/+4
| | | | | | | | | | | | Now that __update_ready_for_runtime() and __update_strict_cache_key_of_rdeps() have been introduced, we no longer need to recursively update the state of elements
| * element.py: Introduce __update_ready_for_runtime()James Ennis2019-07-161-1/+50
| | | | | | | | | | | | | | | | | | | | | | An element becomes ready for runtime once it has a cache key and all of its transitive runtime dependencies have cache keys. This patch introduces the method __update_ready_for_runtime() which is called once the cache key of an Element is calculated. When called, we immediately notify reverse dependencies and, if appropriate, try to calculate their cache keys.
| * element.py: Introduce __update_strict_cache_key_of_rdeps()James Ennis2019-07-163-3/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Once an Element's strict cache key is determined, we should attempt to update the strict cache key of it's reverse dependencies. The state of a reverse dependency will be updated once all of its dependencies have strict cache keys This patch introduces the potential for a RecursionError because _update_state() can now trigger further _update_state calls (on reverse dependencies). Therefore, the maximum recursion limit for our "test_max_recursion_depth" test has been lowered. If this becomes a problem, we can always consider setting a larger recursion limit, for now, this change has been tested with the Debian stack and works as expected.
| * element.py: Remove needless function overheadJames Ennis2019-07-161-34/+13
| | | | | | | | | | | | | | | | This patch inlines the __on_{build,runtime}_dependency_ready_for_runtime_and_cached() functions. Their logic now exists within _update_ready_for_runtime_and_cached, thus the overhead of the extra functions have been removed.
| * element.py: Rename remaining_*_deps_uncached to *_deps_uncachedJames Ennis2019-07-161-11/+11
| | | | | | | | | | | | | | The __remaining_runtime_deps_uncached and __remaining_build_deps_uncached variables are rather verbose and do not really need to specify that they are "remaining", this is implicit. This patch renames these variables
| * element.py: Remove misleading and incorrect commentsJames Ennis2019-07-161-3/+2
| | | | | | | | These comments should have been removed/fixed in 176a3057
| * element.py: Introduce __cache_keys_unstable for workspace handlingJames Ennis2019-07-161-13/+36
| | | | | | | | | | | | | | | | This patch marks the cache keys of workspaced elements as initially unstable. If the cache keys are considered unstable, we don't recursively update reverse dependencies. Only when a workspaced element is confirmed to be cached do we mark the cache keys as stable.
| * _pipeline.py, loader.py: Move Element._preflight() to new_from_meta()James Ennis2019-07-164-50/+45
|/ | | | | | | It's essential to call preflight() when loading/resolving the Elements. This patch moves the preflight call to new_from_meta, so that it is called as soon as the Element is created. This avoids the need for having multiple callsites.
* Merge branch 'tristan/config-max-jobs' into 'master'bst-marge-bot2019-07-1629-70/+155
|\ | | | | | | | | | | | | Add max-jobs configuration and command line option Closes #1033 See merge request BuildStream/buildstream!1473
| * man: Updated man pages for new `--max-jobs` CLI optionTristan Van Berkom2019-07-1622-67/+70
| |
| * tests/frontend/show.py: Test proper resolution of max-jobsTristan Van Berkom2019-07-161-0/+51
| | | | | | | | | | | | | | | | This tests that the resolution of the `max-jobs` automatic variable is properly controlled by the new user configuration and command line option, including the default automatic '0' value. Regression test for #1033
| * _project.py: Use the user specified max-jobsTristan Van Berkom2019-07-161-2/+7
| | | | | | | | | | | | | | Only use automatic max-jobs by default if the user has not requested a specific value for max-jobs. This is a part of #1033
| * _frontend: Added toplevel `--max-jobs` configurationTristan Van Berkom2019-07-163-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While this is currently only relevant for `bst build`, it is a current implementation detail that user configuration which gets overridden by command line options must use toplevel options. This patch allows invocations such as the following to override the max-jobs: bst --max-jobs 16 build target.bst This also updates the completions test to expect the new toplevel option. This is a part of #1033
| * _context.py: Added max-jobs configurationTristan Van Berkom2019-07-162-1/+23
|/ | | | | | | | This loads the configuration of the max-jobs variable from the user configuration, where the default is 0 (meaning use the maximum number of cores with a limit of 8). This is a part of #1033
* Merge branch 'bschubert/new-node-api' into 'master'bst-marge-bot2019-07-15143-2603/+2894
|\ | | | | | | | | Rewrite of the Node API See merge request BuildStream/buildstream!1472
| * NEWS: update for new Node APIBenjamin Schubert2019-07-151-14/+7
| |
| * plugins: Update public documentation to be correct with the new NodesBenjamin Schubert2019-07-153-9/+9
| | | | | | | | | | We need to update every place where we were passing a yaml 'dict' to now pass a 'MappingNode'
| * node: Document buildstream-private API and remove unused parameterBenjamin Schubert2019-07-153-3/+108
| | | | | | | | | | _create_new_file did not use the 'toplevel' parameter, we can therefore remove it.
| * node: document public APIBenjamin Schubert2019-07-152-28/+462
| |
| * node: Make error messages more user-friendlyBenjamin Schubert2019-07-151-4/+12
| | | | | | | | | | Users should not have to know anything about nodes, and should be greeted by commonly defined yaml types.
| * node: Rename 'copy' to 'clone'Benjamin Schubert2019-07-158-30/+30
| | | | | | | | | | | | | | | | A 'clone' operation has an implicit understanding that it is expensive, which is not the case of a 'copy' operation, which is more usually a shallow copy. Therefore renaming to 'clone'
| * node: use 'pop' instead of 'del' for safe_delBenjamin Schubert2019-07-151-4/+1
| | | | | | | | | | This reduces slightly the amount of C code needed and makes the run marginally faster
| * node: Mark module-private functions and classes with '__'Benjamin Schubert2019-07-151-56/+56
| | | | | | | | | | This way, it is easier to know what is BuildStream-private from what is module-private
| * node: Mark `MappingNode._get` as privateBenjamin Schubert2019-07-152-18/+18
| |
| * node: make 'assert_symbol_name' privateBenjamin Schubert2019-07-155-25/+25
| | | | | | | | This is not needed for plugins and should therefore not be public
| * _yaml: Split Node-related parts into 'node.pyx'Benjamin Schubert2019-07-1523-1041/+1118
| | | | | | | | | | This makes the 'Node' API public, and available for use directly for plugins.
| * Reorganize methods in Node classesBenjamin Schubert2019-07-152-196/+260
| |
| * _yaml: Mark 'strip_node_info' as buildstream-privateBenjamin Schubert2019-07-159-18/+18
| |
| * _yaml: Mark new_syntethic_file as BuildStream privateBenjamin Schubert2019-07-152-2/+2
| |
| * _yaml: Set 'MappingNode' public-private APIBenjamin Schubert2019-07-1510-42/+42
| | | | | | | | | | | | | | - _composite -> __composite (internal) - composite -> _composite (BuildStream private) - composite_under -> _composite_under (BuildStream private) - get -> _get (internal)
| * _yaml: Move '__contains__' to MappingNodeBenjamin Schubert2019-07-151-6/+3
| | | | | | | | | | This method is only ever used in this context and should not be on the base Node anyways
| * _yaml: Mark attributes in ProvenanceInformation as Buildstream-privateBenjamin Schubert2019-07-156-44/+47
| | | | | | | | | | Users should not need to get access to any of those, and should only need access to the ProvenanceInformation to print it.
| * _yaml: Remove code duplication on '_new_node_from_*'Benjamin Schubert2019-07-151-17/+5
| |
| * plugin: remove 'node_get_provenance', use the Node API directlyBenjamin Schubert2019-07-154-24/+9
| |
| * _yaml: Remove 'node_get_provenance' and add 'Node.get_provenance'Benjamin Schubert2019-07-1524-121/+115
| | | | | | | | | | | | | | | | This replaces the helper method by adding a 'get_provenance' on the node directly - Adapt all call sites - Delay getting provenance wherever possible without major refactor
| * _yaml: rework 'assert_symbol_names' to not require provenanceBenjamin Schubert2019-07-155-11/+11
| | | | | | | | | | Now that we get scalar Nodes, it is easier to just give the node and extract the provenance as needed.
| * _project: remove 'key' argument from 'get_path_from_node'Benjamin Schubert2019-07-154-14/+18
| | | | | | | | | | | | | | Now that we have scalar nodes, we can directly give the scalar to the method. - Adapt the plugin method to also not access via the key
| * _yaml: Remove 'indices' from 'node_get_provenance'Benjamin Schubert2019-07-155-29/+29
| | | | | | | | | | | | | | | | | | With nodes being propagated better, we now don't need to extract provenance beforehand. Therefore removing the ability to provide indiced when getting provenance. - Also add 'scalar_at' on 'SequenceNode', in order to fetch a scalar node at a specific indice
| * types: rework dependency to always get a 'Node' as dep and remove provenanceBenjamin Schubert2019-07-151-9/+4
| | | | | | | | | | The provenance can be constructed afterwards, so we don't need to pre-calculate it.
| * _yaml: Use __cinit__ and __new__ to create node more effectivelyBenjamin Schubert2019-07-151-27/+34
| | | | | | | | | | | | | | Cython can bypass the normal python mechanism to create new objects, which makes their instantation faster. See https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#fast-instantiation for more details.
| * _yaml: Move 'value' of Node in each sub nodeBenjamin Schubert2019-07-153-39/+71
| | | | | | | | | | | | | | | | This allows us to type its value more strictly, allowing for more efficient access. - Also implement 'node_at' on 'SequenceNode', to allow some tests to test some internals generically.
| * _yaml: Mark attributes on 'Node' as private (only c-accessible)Benjamin Schubert2019-07-152-4/+6
| | | | | | | | | | This ensures that we are not breaking our encapsulation and that the modules accessing 'Node' from C are correctly typed.
| * _yaml: Remove 'is_node', which is not used in the codebase anymoreBenjamin Schubert2019-07-151-20/+0
| | | | | | | | | | A direct replacement would be isinstance(obj, MappingNode) if someone really needs it.
| * _yaml: Remove 'node_validate' and replace by 'MappingNode.validate_keys'Benjamin Schubert2019-07-1530-103/+71
| | | | | | | | - adapt all call sites to use the new API
| * _yaml: Create 'from_dict' on Node and remove node creation methodsBenjamin Schubert2019-07-159-53/+17
| | | | | | | | | | | | | | Using 'Node.from_dict({})' can replace new_empty_node, and the rest is not needed anymore. - Adapt all call sites
| * _yaml: Remove 'node_extend_list' and add 'SequenceNode.append'Benjamin Schubert2019-07-154-80/+16
| | | | | | | | | | | | | | There was a single place using 'node_extend_list', which we can replace more easily with 'SequenceNode.append'. Also rewrite _projectrefs.py:_lookup to use the new API more effectively
| * _yaml: Refer new synthetic nodes to their parentsBenjamin Schubert2019-07-151-19/+22
| | | | | | | | | | | | | | | | | | This changes how we were handling synthetic nodes. Previously, we would have them as coming from a 'SYNTHETIC_FILE'. This is not correct when we need the position in the real file when adding new information to a file we will end up writing to disk (like project_ref). This is a prerequisite for further work on cleaning how it is done.
| * _yaml: Add a 'from_dict' on Node to create new nodes from dictsBenjamin Schubert2019-07-154-18/+22
| | | | | | | | | | | | | | This new methods is here to replace the previous 'new_node_from_dict' that will be moved to a private method. Adapt all call sites to use the new 'from_dict' method.
| * _yaml: Move 'node_composite' to a method on 'MappingNode'Benjamin Schubert2019-07-1510-68/+64
| | | | | | | | | | - Also take care of node_composite_move in the same way. - Adapt all calling places
| * _yaml: move 'composite_dict' to 'MappingNode' as '_composite'Benjamin Schubert2019-07-152-18/+14
| |
View Lorry Controller Status