summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Revert "_site.py: Add check_bwrap_version() function"tristan/revert-bwrap-checksTristan Van Berkom2018-05-111-49/+0
| | | | | | | | This reverts commit 03823d124f61c18346dd2d8282055ab25d6f9aa6. For some reason, the changes introduced here cause issue #395 to occur, without these changes we are not hitting the spurrious errors described in #395.
* Revert "Replace bwrap checks with calls to check_bwrap_version"Tristan Van Berkom2018-05-111-2/+18
| | | | | | | | This reverts commit f8952d6b8a775026d8a566969dd2570badf838fe. For some reason, the changes introduced here cause issue #395 to occur, without these changes we are not hitting the spurrious errors described in #395.
* _workspaces.py: Use file mtime for workspace cache keys instead of checksummingTristan Van Berkom2018-05-111-9/+6
| | | | | | | | | | | | | | | | | | | The rationale for this is basically: o A workspace is completely local o A cache key for a workspace is used to determine if the workspace has changed or not o Within reason, one should not expect buildstream to detect a change without a file's mtime having been modified On my machine, it takes about 1 minute to checksum 11GB of WebKit checkout, that is annoyingly a long time. With this change, it takes only 5 seconds. Not perfect, but it's a start. This is related to issues #294, #295 and #392
* _stream.py: Do not force `PipelineSelection.ALL` for build with trackingJürg Billeter2018-05-112-2/+2
| | | | | | With the dynamic plan using `_set_required()` we can avoid pulling/fetching/building unneeded elements even if elements are being tracked.
* Do not pull/fetch/build elements that are not requiredJürg Billeter2018-05-115-4/+62
| | | | | For `bst build --deps plan`, do not process elements in pull/fetch/build queues until they are requested by a reverse dependency.
* _ostree.py: Remove unused function list_remote_refs()Jürg Billeter2018-05-111-20/+0
|
* doc/source/artifacts.rst: Drop section about summary file updatesJürg Billeter2018-05-111-17/+0
| | | | Summary files are no longer used.
* _artifactcache/ostreecache.py: Do not use summary fileJürg Billeter2018-05-111-87/+18
|
* _artifactcache/artifactcache.py: Remove unused methodsJürg Billeter2018-05-111-35/+0
|
* element.py: Do not query remote artifact caches ahead of pull/pushJürg Billeter2018-05-114-74/+54
|
* _artifactcache: Add element paramater to has_fetch_remotes()Jürg Billeter2018-05-112-3/+15
|
* _artifactcache: Add bool return value to pull()Jürg Billeter2018-05-113-6/+11
| | | | | Calling pull() for a potentially unavailable artifact is no longer considered an error.
* _stream.py: Remove unused downloadable parameter from load_selection()Jürg Billeter2018-05-111-4/+1
|
* _frontend/cli.py: Remove bst show --downloadable optionJürg Billeter2018-05-111-5/+2
| | | | This is in preparation for moving away from summary files.
* tests: Do not rely on 'downloadable' statusJürg Billeter2018-05-114-21/+25
| | | | This is in preparation for moving away from summary files.
* _pipeline.py: Allow pulling strict artifacts in non-strict modeJürg Billeter2018-05-111-3/+7
| | | | | | | | | | | Remotely cached artifacts matching the strict cache key take precedence over locally cached artifacts matching only the weak cache key. However, locally cached artifacts were excluded from the build plan, which means that BuildStream never even checked whether the strict artifact is available in the remote artifact cache. This changes planning to keep cached elements in the build plan in non-strict mode if a remote artifact cache is used.
* _pipeline.py: Remove unused ignore_cache parameter from _PlannerJürg Billeter2018-05-111-7/+7
|
* _artifactcache/ostreecache.py: Fix global remotesJürg Billeter2018-05-111-2/+2
| | | | | | Counter-intuitively, Python list += mutates the list. Use explicit copy() and extend() instead of += to avoid adding project-specific remotes to the global remote list.
* _project.py: Remove unused internal function _extract_plugin_paths()Chandan Singh2018-05-111-8/+0
| | | | | | For reference, this method was added in commit c41f1093d83eb32606f81005227542699f756b0a, but since commit 4912ed5f8922b47c1a16c3752eb92a3f1dea76b2 it has been unused.
* Add soft reset functionality for workspacesTristan Van Berkom2018-05-102-4/+13
| | | | | | | | | | | Add `--soft` option to `bst workspace reset` which would allow uses to reset workspace-related state without affecting its contents. This will be useful in case when an user wants to re-run configure-commands for a workspaced element. Patch originally by Chandan Singh, rebased against recent refactor. Fixes #375.
* buildstream/plugins/sources/_downloadablefilesource.py: Store etag along ↵Valentin David2018-05-101-23/+42
| | | | | | with cache. Fixes #377.
* _frontend/app.py: Fixed ordering of initialization sequence.Tristan Van Berkom2018-05-101-14/+14
| | | | | We need the logging handler setup before creating the Stream(), as the Stream() will expect logging to be already functional.
* _frontend/cli.py: Fetch all elements when tracking a build plantristan/pipeline-refactorTristan Van Berkom2018-05-081-0/+7
| | | | | | | | | | | | | | | | It makes not sense to type `bst fetch --track --deps all <targets>`, because tracking will inevitably modify the build plan. Stream initialization will not cope with this either, instead of silently doing something which does not make any sense, we add an assertion that this should not happen. Unfortunately since `plan` is the default deps type for `bst fetch`, this is likely to happen so it's important to correct. This patch adds a warning in the case tracking of the build plan elements is requested, and converts the request to track all elements instead.
* _stream.py, _pipeline.py: Refactoring of the pipeline itselfTristan Van Berkom2018-05-087-606/+685
| | | | | | | | | | | | | | | | | | | Here the pipeline becomes essentially stateless, some dangling state remains to be factored out because of frontend accesses which will be changed in a later commit. Essentially, the Pipeline.load() method no longer has any knowledge of the specific purposes of the loaded targets, and now takes a list of target groups and returns a corresponding list of element groups. The Stream() business logic methods now use other pipeline helper methods to create and filter lists from the loaded target elements. The Stream() also finally absorbs the Scheduler frontend facing APIs. However Queues are still exposed on the Stream object for logging purposes and through callbacks such that the frontend can retry elements.
* _artifactcache: Added ArtifactCache.setup_remotes()Tristan Van Berkom2018-05-087-103/+131
| | | | | | | This removes some additional initialization code from Pipeline(). Some symbols have changed here, the initialization is now called from Stream(), and a test case was also adjusted for this.
* _frontend/app.py: Remove partially_initialized()Tristan Van Berkom2018-05-081-67/+42
| | | | There is no more need for this distinction.
* Moving 'fetch_subprojects' configuration to stream <--> loaderTristan Van Berkom2018-05-086-44/+53
| | | | | | | | | This was previously decided in CLI, but knowledge of what to initialize has been moved to Stream(). Now there is no more point to store this configuration in the Context, we just have the Stream() decide it when asking the Pipeline() to invoke the Loader().
* _stream.py, _frontend: Call Stream() APIs with CLI arguments directly.Tristan Van Berkom2018-05-085-320/+553
| | | | | | This shifts the whole responsibility of interpreting command line targets etc to the Stream() object itself. With this commit, the Pipeline() truly becomes slaved to the Stream().
* _frontend: Early logging initializationTristan Van Berkom2018-05-083-147/+120
| | | | | | | | o This makes logging independent from the Pipeline() o Removed size_request Widget() method, add context to Widget() initializer o Make the Status() widget derive anything it needs through the Context()
* _stream.py: Absorb workspace functionality from App.Tristan Van Berkom2018-05-083-157/+214
|
* _stream.py: Add StreamError exceptionTristan Van Berkom2018-05-0815-53/+64
| | | | Use Stream error for Stream errors.
* _stream.py: New Stream object, main calling interface for BuildStream coreTristan Van Berkom2018-05-086-380/+427
| | | | | | This is the first part of the pipeline refactor, at this stage all calling interfaces remain the same, except that invocation of the scheduler has been moved from Pipline to Stream.
* tests/format/variables.py: Removed testing of stderrTristan Van Berkom2018-05-081-4/+0
| | | | | | We have machine readable errors for this purpose, and the strings happen to change causing tests to break if we test the specific UI (reported error strings are UI).
* Replace bwrap checks with calls to check_bwrap_versionmilloni2018-05-071-18/+2
| | | | See: #373
* _site.py: Add check_bwrap_version() functionmilloni2018-05-071-0/+49
| | | | | | | Lazily parse the version of bwrap the first time the function is called. On subsequent calls, used cached version info. See: #373
* .gitlab-ci.yml: Remove redundant declaration of dependencies:Javier Jardón2018-05-071-6/+0
| | | | | "artifacts from all previous stages are passed by default." https://docs.gitlab.com/ee/ci/yaml/#dependencies
* bst-here: Allow users to specify additional volumes to mountChandan Singh2018-05-071-2/+9
| | | | | | | | | One may want to mount additional volumes to preserve certain directories or to share some data between the host and the container. Allow users to do so by providing a `-v` option that passes its arguments to corresponding `-v`/`--volume` option for `docker run` command. Part of #378.
* bst-here: Allow users to specify Docker imageChandan Singh2018-05-071-12/+17
| | | | | | | | | | | | | | | | | Currently, the name of the Docker image is hardcoded in bst-here script. This makes it harder for users to override it, which may be needed for various reasons (custom plugins, private Docker registries etc.) Add `-i` option to allow users to specify the base image. When `-i` is not provided, default to `BST_HERE_IMAGE` environment variable if it's set and otherwise to the current image - `buildstream/buildstream-fedora`. Also, re-order the command-line options in help text and source code in alphabetical order to maintain sanity as the number of options is slowly growing. Part of #378.
* bst-here: Fix typo in help textChandan Singh2018-05-071-1/+1
|
* app.py: Ensure an elements sub-dir is created upon project initialisationJames Ennis2018-05-071-0/+8
|
* tests/format/variables.py: Add a test for undefined variable error messageValentin David2018-05-063-0/+24
|
* Fix provenance in error message for undefined variables.Valentin David2018-05-062-4/+19
|
* commands.rst: Split up commands to easily allow for further descriptionJames Ennis2018-05-041-1/+119
| | | | An introduction for this section was also added
* element.py: Fix buildable check in non-strict modejuerg/non-strict-buildableJürg Billeter2018-05-031-1/+6
| | | | | | | | | | | | | | | Ensure that the strong cache key of each build dependency is available before an element is built. Otherwise the strong cache key of the element cannot be calculated and caching the artifact produces an AssertionError. In non-strict mode an element's strong cache key may not be available yet even though an artifact is available in the local cache. This can happen if the pull job is still pending as the remote cache may have an artifact that matches the strict cache key, which is preferred over a locally cached artifact with a weak cache key match. Fixes #383.
* _artifactcache/pushreceive.py: Modified incorrect error messagePhillip Smyth2018-05-011-1/+1
| | | | | | | changed encode_message push exception from 'Command must by GLib.Variant' to 'Command must be PushCommand'
* tests/testutils/runcli.py: Improve detection of pulled/pushed elementsJürg Billeter2018-04-301-2/+2
|
* element.py: Print info message when artifact was actually pushedJürg Billeter2018-04-301-1/+8
| | | | This matches the pull code path.
* _artifactcache/pushreceive.py: Do not return True for skipped pushJürg Billeter2018-04-301-1/+1
|
* _artifactcache/pushreceive.py: Fix commit object check in send_info()Jürg Billeter2018-04-301-1/+1
| | | | Fixes #325.
* README.rst: Fixed formatting1.1.3Tristan Van Berkom2018-04-261-5/+5
| | | | No empty lines after a section title, 2 empty lines before a section title.