summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Replace strip-binaries by a NOOP scriptvalentindavid/remove-strip-binariesValentin David2018-12-0710-70/+24
|
* Merge branch 'tristan/yaml-optionally-allow-none' into 'master'Tristan Van Berkom2018-12-075-5/+18
|\ | | | | | | | | | | | | Only optionally allow None values in user provided YAML Closes #803 See merge request BuildStream/buildstream!999
| * tests/format/project.py: Added new regression test for empty dependency ↵Tristan Van Berkom2018-12-073-0/+11
| | | | | | | | | | | | dictionaries This adds a regression test for issue #803
| * plugin.py: Added new `allow_none` parameter to Plugin.node_get_member(), ↵Tristan Van Berkom2018-12-071-2/+3
| | | | | | | | | | | | | | | | | | defaulting to False. Allow plugins to conveniently leverage the new `allow_none` parameter, and have Plugin.node_get_member() raise a LoadError in the usual case where None is not an acceptable value to be explicitly set in the user provided YAML.
| * _yaml.py: Added `allow_none` parameter to _yaml.node_get()Tristan Van Berkom2018-12-071-3/+4
|/ | | | | | | | | | | | | | | | | | | | | | | | | | This allows specifying whether or not the code believes a None value is acceptable if explicitly expressed to be None in the YAML. A previous patch 3ba544b80f9f268be8ffe62fc8589b30212ec4a2 changed the behavior to accept None for all values across the board, along with allowing explicitly setting the `default_value` to `None` which caused the code to be much more readable (this was the main motivation of the patch, but it had the side effect of allowing None for everything). In the majority of cases we load YAML however, it either has a value or it is not specified in the YAML, and None is hardly ever acceptable to be explicitly specified (it may be in the case you want to override something with nothing in the YAML, but it is the edge case). This had the side effect of causing unexpected crashes where the YAML specifies an empty dictionary for instance. Instead of forcing the caller to handle a possible None value, give the choice to the caller if they really want to accept a None value, and raise the LoadError otherwise. This fixes issue #803, which is a crash due to receival of an unexpected None value through _yaml.node_get()
* Merge branch 'jonathan/source-checkout-workspace' into 'master'Jonathan Maw2018-12-062-2/+9
|\ | | | | | | | | | | | | Fix bst source-checkout not working with open workspaces Closes #807 See merge request BuildStream/buildstream!997
| * Fix bst source-checkout not working with open workspacesJonathan Maw2018-12-062-2/+9
|/
* Merge branch 'tristan/submodule-warnings' into 'master'Tristan Van Berkom2018-12-065-51/+404
|\ | | | | | | | | Implement submodule warnings See merge request BuildStream/buildstream!996
| * tests/sources/git.py: Test invalid submodules warning appearing after trackTristan Van Berkom2018-12-061-0/+61
| |
| * tests/sources/git.py: Test unlisted submodules warning appearing after trackTristan Van Berkom2018-12-061-0/+63
| |
| * tests/sources/git.py: Testing the git:invalid-submodule warningTristan Van Berkom2018-12-061-0/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | o Test that it is not triggered in show before fetch, because we don't know the submodules yet so we cannot know if they are valid or not. o Test that it is triggered by a fetch command o Test that it is triggered by `show` after having completed a fetch command, since now we have the repository and know which specified submodules are invalid o Test all of this under warning or error conditions (parameterized for fatal-warnings)
| * tests/sources/git.py: Testing the git:unlisted-submodule warningTristan Van Berkom2018-12-061-0/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | o Test that it is not triggered in show before fetch, because we don't know about the unlisted submodules yet o Test that it is triggered by a fetch command o Test that it is triggered by `show` after having completed a fetch command, since now we have the repository and know about the unlisted submodule o Test all of this under warning or error conditions (parameterized for fatal-warnings)
| * tests/testutils/repo/git.py: Adding remove_path() helperTristan Van Berkom2018-12-061-0/+6
| | | | | | | | | | A function for removing paths from a git repo, can also be used to remove submodules.
| * git source plugin: Emmit the ref-not-in-track warning from validate_cache()Tristan Van Berkom2018-12-061-37/+30
| | | | | | | | | | | | | | | | | | | | Now that we have Source.validate_cache(), this is a better place to emmit the ref-not-in-track warning, since it will be emmitted at the earliest opportunity and not only at Source.stage() or Source.init_workspace(). This also allows us to remove the `self.tracked` local state, and cleanup some convoluted calling paths, removing some unnecessary parameters from the usual codepaths and making the plugin overall more readable.
| * git source plugin: Implementing submodule warningsTristan Van Berkom2018-12-062-13/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | o Unlisted submodule warning Now the git plugin will issue a configurable warning if a submodule exists and is used (checking out the submodule is not disabled), but is not specified in the source configuration. o Invalid submodule warning Now the git source plugin will issue a warning if the configuration specified a submodule which does not exist in the underlying git repository. As a side effect, this patch also changes the flow control of the git plugin such that submodules which are explicitly set to not be checked out, are also not fetched but instead ignored completely.
| * source.py: Add new delegate method validate_cache()Tristan Van Berkom2018-12-061-1/+24
|/ | | | | | | This is guaranteed to be called only once for a given session once the sources are known to be Consistency.CACHED, if source tracking is enabled in the session for this source, then this will only be called if the sources become cached after tracking completes.
* Merge branch 'tristan/misc-cleanup' into 'master'Tristan Van Berkom2018-12-0610-97/+77
|\ | | | | | | | | Misc cleanups See merge request BuildStream/buildstream!993
| * git source plugin: Rename symbol for consistency and readabilityTristan Van Berkom2018-12-061-2/+3
| | | | | | | | | | Change 'INCONSISTENT_SUBMODULE' for 'WARN_INCONSISTENT_SUBMODULE' to make things a bit more readable and comprehensive.
| * git source plugin: Fixing documentation linking and typoTristan Van Berkom2018-12-061-5/+6
| | | | | | | | | | | | | | | | | | | | Fixes the configurable warning part of this to properly link to the project.conf documentation describing what configurable warnings actually are. Also fix the plugin raise the actual configurable warning which it advertizes that it raises; which is 'inconsistent-submodule', not 'inconsistent-submodules'.
| * Source: Fix typo in documentation, and make it a link to `bst track`Tristan Van Berkom2018-12-061-1/+2
| | | | | | | | | | | | | | Now the commands have documentation linkability so we can link to them when referring to them anywhere in the docs. This reference was previously referring to a bogus non-existent `build-stream track`, maybe this dates way back to the days before we named the frontend `bst`.
| * types.py: Moving CoreWarnings to the types.py moduleTristan Van Berkom2018-12-068-49/+50
| | | | | | | | | | | | This also ensures it is exposed via the main buildstream __init__.py file, fixes the imports of CoreWarnings, and adjusts the documentation links to point to the right place.
| * tests/sources/git.py: Refactor ref-not-in-track test to use parameterizationTristan Van Berkom2018-12-061-41/+17
|/ | | | | Instead of duplicating the whole test body, run it twice while checking for a warning or an error depending on the parameter.
* Merge branch 'raoul/628-RE-flow-optimisation' into 'master'Raoul Hidalgo Charman2018-12-054-74/+152
|\ | | | | | | | | | | | | Remote-execution client flow optimisation Closes #628 See merge request BuildStream/buildstream!982
| * format_project.rst: Add docs for action-cache-serviceRaoul Hidalgo Charman2018-12-051-0/+7
| | | | | | | | Part of #628
| * _sandboxremote.py: Add action-cache-service endpointRaoul Hidalgo Charman2018-12-051-6/+37
| | | | | | | | | | | | The endpoint may not necessarily be the same as execution-service endpoint. Part of #628
| * _sandboxremote.py: Add checks to action cache before attempting to push.Raoul Hidalgo Charman2018-12-051-67/+93
| | | | | | | | | | | | | | Stops unneccesary pushing of builds that have already been built, just checks the action cache to begin with. Fixes #628
| * utils.py: Add message digest functionRaoul Hidalgo Charman2018-12-052-4/+18
|/ | | | Get cascache.py to use this
* Merge branch 'raoul/775-execution-environment-reqs' into 'master'Jürg Billeter2018-12-0561-94/+389
|\ | | | | | | | | | | | | Execution environment reqs Closes #775 See merge request BuildStream/buildstream!969
| * format_project.rst: Add docs for OptionOSRaoul Hidalgo Charman2018-12-051-0/+24
| |
| * format_declaring.rst: Update documentation for sandbox optionsRaoul Hidalgo Charman2018-12-051-2/+22
| |
| * format_project.rst: Update documentation for changes to OptionArchRaoul Hidalgo Charman2018-12-051-6/+19
| |
| * Sandbox: use linux32 for x86-32 builds on x86-64 machinesRaoul Hidalgo Charman2018-12-052-5/+30
| | | | | | | | | | | | | | | | o _platform/linux.py: Add linux32 flag to send to sandbox bwrap when the build arch is x86-32 and the machines arch is x86-64 or similarly with aarch32 and aarch64. o sandbox/_sandboxbwrap.py: Use flag passed to start bwrap command with linux32 if set.
| * _platform: Add checks for os and architectureRaoul Hidalgo Charman2018-12-052-4/+22
| |
| * optionos.py: Add tests for OptionOSRaoul Hidalgo Charman2018-12-053-0/+99
| | | | | | | | Same format as OptionArch tests, but with OS's.
| * optionos.py: Add new option for OS configurationRaoul Hidalgo Charman2018-12-052-0/+43
| |
| * _sandboxremote.py: Add os and arch to command uploadedRaoul Hidalgo Charman2018-12-051-1/+8
| | | | | | | | These are contained within the platform field.
| * optionarch.py: update to use same arch names as SandboxConfigRaoul Hidalgo Charman2018-12-0518-43/+43
| | | | | | | | Also update tests to be consistent with this
| * _config.py: Use os and architecture settingsRaoul Hidalgo Charman2018-12-0531-37/+45
| | | | | | | | | | | | In element, platform asks for host os and architecture to default to when SandboxConfig is initialised. This changes element cache keys so those have been updated in the tests.
| * platform.py: Add mapping of uname to OS-independent valueslachlanmackenzie/debug_benchmark_failureRaoul Hidalgo Charman2018-12-051-0/+38
|/
* Merge branch 'valentindavid/git_describe_tracking' into 'master'Tristan Van Berkom2018-12-059-15/+390
|\ | | | | | | | | | | | | Track of git tags and save them to reproduce minimum shallow repository Closes #487 See merge request BuildStream/buildstream!906
| * NEWS: Add news about broken 'git describe' by default.Valentin David2018-12-051-0/+4
| |
| * git source plugin: Track git tags and save them to reproduce a minimum ↵Valentin David2018-12-058-15/+386
|/ | | | | | | | | | | | | | | shallow repository Instead of tag information being fetched which can change with time, they are tracked and saved in the projects.refs/.bst. Then we re-tag automatically the closest tag so that `git describe` works and is reproducible. This new feature is opt-in with the new `track-tags` configuration, and must be used to fix modules which are broken by our new policy of omitting the `.git/` repository when staging git sources. This fixes issue #487
* Merge branch 'tristan/refactor-queues-update-state' into 'master'Tristan Van Berkom2018-12-052-6/+11
|\ | | | | | | | | _scheduler/queues/queue.py: Don't call update state outside of error handling harness See merge request BuildStream/buildstream!988
| * _scheduler/queues: Don't call update state outside of error handling harnessTristan Van Berkom2018-12-052-6/+11
|/ | | | | | | | | | | | | | | | | | | | | | | | | Commit 3fa79d8da, part of an initiative for caching of the failed builds, introduced a call to Element._update_state() after a job completes and before entering the error handling harness intended for handling plugin raised errors. Element._update_state() can result in triggering plugin code to run, so this is incorrect, and causes raised errors to crash BuildStream if they happen here. After analyzing the code, it appears that this additional call to Element._update_state() is unneeded, and was only added because the state needs to be updated for a failure as well as a success. Instead, we now have the BuildQueue call Element._assemble_done() unconditionally, regardless of whether the build was successful or not, which has the same effect and also reads better. In addition, added a FIXME comment that we are still conditionally updating the artifact cache size from BuildQueue.done() only if the build is successful, which is incorrect because failed builds also increase the local artifact cache size - to fix this we need to communicate the added artifact size through Element._assemble() regardless of whether the build succeeded or failed.
* Merge branch 'jonathan/fix-identical-element' into 'master'Jürg Billeter2018-12-0318-6/+99
|\ | | | | | | | | | | | | _yamlcache.py: Use a project's junction name if present Closes #795 See merge request BuildStream/buildstream!980
| * tests/loader/junctions.py: Test that we can build a junction that is ↵jonathan/fix-identical-elementJames Ennis2018-12-0317-0/+69
| | | | | | | | | | | | | | referred differently, twice. Test required to close #795 - We should be able to build a project which refers to the same junction twice, but with two different filenames.
| * _yamlcache.py: Use a project's junction name if presentJonathan Maw2018-12-031-6/+30
|/ | | | | | | | | This is required because if there are projects with the same name loaded, that have elements in the same path, with the same contents, they are considered the same, despite actually belonging to different Projects. This fixes buildstream issue #795
* Merge branch 'jjardon/doc_git_submodules' into 'master'Tristan Van Berkom2018-12-031-0/+5
|\ | | | | | | | | git: warn about checkout submodules by default See merge request BuildStream/buildstream!983
| * buildstream/plugins/sources/git.py: warn about checkout by defaultJavier Jardón2018-12-031-0/+5
|/ | | | See #783
* Merge branch 'chandan/bst-checkout-build' into 'master'Tristan Van Berkom2018-12-035-28/+47
|\ | | | | | | | | | | | | Add `--deps build` option to `bst checkout` Closes #670 See merge request BuildStream/buildstream!819