summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* docs: Updating documentation regarding element names and addressing.tristan/element-full-pathsTristan van Berkom2020-06-073-61/+80
| | | | | | | | | | | | | | | | | | | | | | | | | This commit does the following: * Move the section about rules on element naming to the beginning of the document instead of the end. Element names are probably the first thing you need to know about elements, not the last. * Expand this element naming section to also speak about element addressing, and how to express element paths and how these can be used across the boundaries of multiple junctions. It helps to have a single place to discuss this and link to in the documentation, instead of repetitively explaining in multiple places that one must specify a filename relative to a project's element path. * Update documentation about dependencies to refer to the element naming and addressing documentation. * Update the documentation in the `link` and `junction` elements to link back to the element name documentation.
* tests/format/include.py: Test including of files using full pathsTristan van Berkom2020-06-0511-0/+54
|
* tests/format/link.py: Testing full path link targetsTristan van Berkom2020-06-055-1/+23
| | | | | | | | * Test that we succeed to load links with full path targets * Test that we get correct provenance information in errors when linking to non-existing elements in existing subprojects, using full paths.
* tests/plugins/loading.py: Testing junction plugins with full pathsTristan van Berkom2020-06-051-1/+109
| | | | | | | | | | Added tests to cover: * Successfully loading plugins with full paths in junction origins * Failing to load missing plugins in sub-junctions which are successfully loaded * Failing to load missing plugins in sub-junctions which are invalid and couldnt load
* tests/format/junctions.py: Test loading dependencies and targets with full pathsTristan van Berkom2020-06-055-0/+66
|
* _includes.py: Use rsplit to split included yaml instead of splitTristan van Berkom2020-06-051-1/+1
| | | | | Need to consider only the last element of a path as the filename while the rest of the path is a junction path.
* _pluginfactory/pluginoriginjunction.py: Call get_loader() with provenance.Tristan van Berkom2020-06-051-1/+1
| | | | | This is important for reporting errors due to missing files while loading a junction.
* _loader.py: Support full path dependencies and targetsTristan van Berkom2020-06-052-191/+217
| | | | | | | | | | | | | | | | | | | | | | This commit does the following: * Moves the `get_loader()` implementation to a private `_get_loader()` implementation again, without changing it. * Replaces the public `get_loader()` method with a simple one which parses the passed junction name and walks the `:` splits until the ultimate loader is found (by calling _get_loader() internally). * Removes an assertion in the Dependency constructor about having multiple `:` in the element name, which is now allowed, and adjusts the Dependency constructor to properly rsplit() the passed dependency target from the right, storing the full junction path in dep.junction and the target only in the dep.target. * Adds missing `provenance` argument to Loader._parse_name(), ensuring that the provenance of where a target name was found is propagated through `get_loader()` and `_load_file()` preserving the original provenance of a loaded element name.
* plugin.py: Load full element names recursively.Tristan Van Berkom2020-06-051-1/+1
| | | | | | | | | | | When opening up the possibility of loading the same junction twice, it becomes possible to have multiple elements with the same name following the current implementation logic which enforces that you must have a toplevel junction to coalesce junctions and synchronize their configuration. Since we no longer coalesce junction names, we need to allow elements to be reported in the UI with their full depth.
* _loader/loadelement.pyx: Load full element names recursively.Tristan Van Berkom2020-06-051-1/+1
| | | | | | | | | | | When opening up the possibility of loading the same junction twice, it becomes possible to have multiple elements with the same name following the current implementation logic which enforces that you must have a toplevel junction to coalesce junctions and synchronize their configuration. Since we no longer coalesce junction names, we need to allow elements to be reported in the UI with their full depth.
* Merge branch 'juerg/sandbox-drop-bwrap' into 'master'bst-marge-bot2020-06-0330-2753/+75
|\ | | | | | | | | Drop bwrap sandboxing backend See merge request BuildStream/buildstream!1952
| * Drop Platform subclasses and BST_FORCE_BACKENDjuerg/sandbox-drop-bwrapJürg Billeter2020-06-037-184/+18
| | | | | | | | | | With the use of buildbox-run on all platforms, there is no longer any platform-specific code in the Platform subclasses.
| * Drop sandbox selection and BST_FORCE_SANDBOXJürg Billeter2020-06-037-112/+22
| | | | | | | | | | | | | | buildbox-run is the only local sandbox and there are no plans to add other sandboxing backends in the future. New platforms can be supported by new buildbox-run implementations without requiring any changes in BuildStream.
| * NEWS: Add note about dropping bwrap sandboxing backendJürg Billeter2020-06-031-0/+7
| |
| * doc/source/arch_sandboxing.rst: Update for buildbox-runJürg Billeter2020-06-031-33/+24
| |
| * linux.py: Remove unused can_crossbuild() and _uid/_gidJürg Billeter2020-06-031-26/+0
| |
| * Drop unused _fuse packageJürg Billeter2020-06-036-1460/+1
| |
| * sandbox: Remove unused root and scratch directoriesJürg Billeter2020-06-031-22/+0
| |
| * sandbox: Drop unused _mount.pyJürg Billeter2020-06-031-148/+0
| |
| * sandbox: Drop _use_cas_based_directory()Jürg Billeter2020-06-032-26/+2
| | | | | | | | It is always true.
| * Drop SandboxBwrapJürg Billeter2020-06-031-511/+0
| | | | | | | | Replaced by buildbox-run.
| * _platform: Drop maximize_open_file_limit()Jürg Billeter2020-06-033-64/+0
| | | | | | | | | | Without the SafeHardlinks FUSE filesystem there is no longer a need to increase the open file limit.
| * darwin.py: Update error messageJürg Billeter2020-06-031-3/+1
| | | | | | | | FUSE is no longer a blocker.
| * linux.py: Drop bwrap supportJürg Billeter2020-06-031-26/+0
| |
| * tests: Drop bwrap-specific testsJürg Billeter2020-06-038-132/+1
| |
| * .gitlab-ci.yml: Drop bwrap jobJürg Billeter2020-06-031-7/+0
| | | | | | | | The bwrap sandbox will be removed.
| * tests/integration/cachedfail.py: Do not skip test with buildbox-runJürg Billeter2020-06-031-1/+1
|/
* Merge branch 'valentindavid/optimize-variables' into 'master'bst-marge-bot2020-06-035-65/+109
|\ | | | | | | | | Optimize variable flattening See merge request BuildStream/buildstream!1949
| * Optimize variable flatteningValentin David2020-06-035-65/+109
|/ | | | | | | | | | | | | | | | | Verifying for variables (all used variables defined and no cycle) is much faster than flattening them. It also uses much less memory. Only `bst show -f "%{vars}"` requires to flatten all variables. For other cases we can improve performance by only checking rather than flattening. Also flattening very nested variables could lead to very long lists of empty strings. The memory usage for flattening could be a lot bigger than the resulting value. Force flattening and caching every variable definitions can improve intermediate memory consumption and only consume memory of the size of the result. Finally, an optimization allowed inifinite cycles leading to segmentation fault. This was solved by simplifying the code.
* Merge branch 'traveltissues/update-overnight' into 'master'bst-marge-bot2020-06-031-2/+2
|\ | | | | | | | | update FD overnights See merge request BuildStream/buildstream!1953
| * Update target branch in FD overnightstraveltissues/update-overnightDarius Makovsky2020-06-021-2/+2
|/ | | | bst2 efforts are now concentrated on the `valentindavid/bst2` branch
* Merge branch 'tristan/link-element' into 'master'bst-marge-bot2020-06-0271-216/+630
|\ | | | | | | | | Add new core `link` element kind See merge request BuildStream/buildstream!1948
| * NEWS: Updated for breaking change of junction target removaltristan/link-elementTristan van Berkom2020-06-011-3/+4
| |
| * junctions: Remove all traces of the `target` optionTristan van Berkom2020-06-0115-167/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes: * The `target` feature from the junction plugin * Special case code in the loader for the `target` feature * The `target` related cases in tests/format/junctions.py This also adjusts the `target` related documentation in the `junction` element to suggest using a `link` element for the purpose of using a subproject junction configuration to access a common sub-subproject.
| * doc: Adding new `link` plugin to the core plugin indexTristan van Berkom2020-06-011-0/+1
| |
| * tests/format/link.py: Adding tests for the link elementTristan van Berkom2020-06-0160-0/+407
| |
| * link element: Adding support for new link elementTristan van Berkom2020-06-014-4/+160
| | | | | | | | | | | | This element acts as a symbolic link, it has no other configurations other than to specify the element (or junction) to which it refers to, either in the local project or in a subproject.
| * _loader/loader.py: Make collect_element_no_deps() publicTristan van Berkom2020-06-011-75/+78
|/ | | | | Within the _loader module, the LoadElement will need to use this in order to resolve links.
* Merge branch 'tristan/loader-fixes' into 'master'bst-marge-bot2020-05-301-2/+1
|\ | | | | | | | | _loader/loader.py: Remove useless line of code. See merge request BuildStream/buildstream!1946
| * _loader/loader.py: Remove useless line of code.Tristan van Berkom2020-05-301-2/+1
|/ | | | | | | | | We were redundantly calling Loader._load_file() directly before calling Loader.get_loader(), even though Loader.get_loader() implicitly loads the file anyway. To pass tests, we needed to fix get_loader() to pass provenance along to Loader._load_file().
* Merge branch 'tristan/remove-loader-no-progress-object' into 'master'bst-marge-bot2020-05-302-18/+12
|\ | | | | | | | | loader: removing the NO_PROGRESS object See merge request BuildStream/buildstream!1947
| * loader: removing the NO_PROGRESS objecttristan/remove-loader-no-progress-objectTristan van Berkom2020-05-302-18/+12
|/ | | | | | | | | | | While adding a mock task object to track progress in some loader tests in commit 17144d84c2b63daf6e3aa9b42c6c773f134e8660, a new `_NO_PROGRESS` object was added as an explicit marker to denote that progress information should not be reported. This complicates the code, None should be a sufficient value for not reporting progress, while still permitting mock task objects to capture progress if desired.
* Merge branch 'tristan/version-specific-config' into 'master'bst-marge-bot2020-05-293-3/+50
|\ | | | | | | | | Support version specific configuration files See merge request BuildStream/buildstream!1941
| * doc/source/using_config.rst: Document support for version specific config files.Tristan van Berkom2020-05-291-0/+11
| |
| * tests/internals/context.py: Test correct config file is chosen.Tristan van Berkom2020-05-291-0/+28
| |
| * _context.py: Support parallel installations of BuildStreamTristan van Berkom2020-05-291-3/+11
|/ | | | | | | | | | Allow users to keep separate buildstream.conf files in their XDG_CONFIG_HOME for separate BuildStream versions, in case they have BuildStream 1 and 2 installed. Like other major version safeguards and warnings, this new addition also supports future major versions if they should appear.
* Merge branch 'bschubert/resolve-public-variables' into 'master'bst-marge-bot2020-05-295-14/+48
|\ | | | | | | | | | | | | element.py: Always expand public data's variables Closes #1310 See merge request BuildStream/buildstream!1943
| * element.py: Always expand public data's variablesBenjamin Schubert2020-05-295-14/+48
|/ | | | | | | | | | | | | | | | | This fixes a bug introduced by d7d18c1a2e454c507afd9e1d3f1358639dd43871, where public data integration commands and others would never get their data expanded and would thus fail to run. The previous however revelead a previous bug, where variables values in public data of elements would not be part of the cache key of an element or it's reverse dependencies, and thus, on variable change, rebuilds would not happen when they would have been needed. This ensures that all the public data is always resolved and part of the element's cache key. This however will bring a rebuild of an element whenever its integration commands variables change, which is simpler to handle than to try to push that responsibility on reverse dependencies, since public data can contain plugin-defined values.
* Merge branch 'juerg/cas-send-directory' into 'master'bst-marge-bot2020-05-281-3/+5
|\ | | | | | | | | cascache.py: Drop unnecessary FindMissingBlobs from _send_directory() See merge request BuildStream/buildstream!1944
| * cascache.py: Drop unnecessary FindMissingBlobs from _send_directory()juerg/cas-send-directoryJürg Billeter2020-05-281-3/+5
|/ | | | | | buildbox-casd `UploadMissingBlobs` will call `FindMissingBlobs` before the actual upload. Having BuildStream call `FindMissingBlobs` as well adds unnecessary overhead.