summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* .gitlab-ci.yml: Add a test with job spawning onaevri/spawnAngelos Evripiotis2019-08-151-0/+6
| | | | Add a GitLab test with the 'spawn' method of starting processes forced.
* _frontend/cli: support 'BST_FORCE_START_METHOD'Angelos Evripiotis2019-08-151-0/+13
| | | | | | Allow users to force the start method via a 'BST_FORCE_START_METHOD' environment variable. This enables testing of the 'spawn' and 'forkserver' methods on platforms that don't require them.
* Support pickling jobs if the platform requires itAngelos Evripiotis2019-08-133-8/+106
| | | | | | | | Add support for using `multiprocessing.Manager` and the associated queues. Downgrade the queue event callback guarantees accordingly. In later work we may be able to support callbacks in all scenarios. Pickle and unpickle the child job if the platform requires it.
* Abstract mp Queue usage, prep to spawn processesAngelos Evripiotis2019-08-134-25/+89
| | | | | | | Pave the way to supporting starting processes by the 'spawn' method, by abstracting our usage of `multiprocessing.Queue`. This means we can easily switch to using a multiprocessing.Manager() and associated queues instead when necessary.
* utils: uppercase global _MAIN_PID, move to topAngelos Evripiotis2019-08-091-6/+5
| | | | This is consistent with the other global vars in the module.
* Merge branch 'coldtom/move-meson' into 'master'bst-marge-bot2019-08-084-152/+1
|\ | | | | | | | | Move meson element to bst-plugins-experimental See merge request BuildStream/buildstream!1532
| * Move meson element to bst-plugins-experimentalThomas Coldrick2019-08-084-152/+1
|/
* Merge branch 'danielsilverstone-ct/load-reject-duplicates' into 'master'bst-marge-bot2019-08-0811-4/+121
|\ | | | | | | | | | | | | Reject elements which have duplicated dependencies Closes #1077 See merge request BuildStream/buildstream!1528
| * tests: Add test for DUPLICATE_DEPENDENCY verificationDaniel Silverstone2019-08-089-0/+94
| | | | | | | | Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _loader: Detect and reject duplicated dependenciesDaniel Silverstone2019-08-081-4/+24
| | | | | | | | | | | | | | | | | | | | When loading an element, if it has duplicated dependencies of any kind then we reject the element with a LoadError(DUPLICATE_DEPENDENCY). This means that an element `foo.bst` can only appear once in any given dependency kind, allowing it to be both `runtime` and `build`, but not either of those twice, nor either plus `all`, nor `all` twice. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _exception: Add DUPLICATE_DEPENDENCY to LoadErrorReasonDaniel Silverstone2019-08-081-0/+3
|/ | | | Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'tpollard/messageobject' into 'master'bst-marge-bot2019-08-0821-162/+194
|\ | | | | | | | | Message rework See merge request BuildStream/buildstream!1500
| * _message.py: Use element_name & element_key instead of unique_idtpollard/messageobjectTom Pollard2019-08-0820-156/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | Adding the element full name and display key into all element related messages removes the need to look up the plugintable via a plugin unique_id just to retrieve the same values for logging and widget frontend display. Relying on plugintable state is also incompatible if the frontend will be running in a different process, as it will exist in multiple states. The element full name is now displayed instead of the unique_id, such as in the debugging widget. It is also displayed in place of 'name' (i.e including any junction prepend) to be more informative.
| * plugin.py: cache full_name member in __init__Tom Pollard2019-08-081-7/+22
| | | | | | | | | | | | Once project & type are resolved, the full_name can be computed and cached for efficiency. The accessor for getting the private member should also be moved to the correct section.
| * element.py: Remove redundant second call to _get_cache_key()Tom Pollard2019-08-081-1/+1
|/
* Merge branch 'juerg/artifact-blobs' into 'master'bst-marge-bot2019-08-086-27/+42
|\ | | | | | | | | Artifact blob handling See merge request BuildStream/buildstream!1527
| * cli.py: Require artifact file contents to be present for `bst show`Jürg Billeter2019-08-081-3/+0
| | | | | | | | | | With blob-based expiry in buildbox-casd, the presence of the artifact proto will no longer be sufficient.
| * _artifact.py: Update blob mtimes in cached()Jürg Billeter2019-08-081-1/+1
| | | | | | | | This is in preparation for blob-based expiry as used by buildbox-casd.
| * cascache.py: Add update_mtime parameter to contains_directory()Jürg Billeter2019-08-081-4/+13
| |
| * cascache.py: Fix check_exists in _reachable_refs_dir()Jürg Billeter2019-08-081-0/+3
| | | | | | | | | | If `check_exists` is set, missing directories should raise a FileNotFoundError.
| * _sandboxremote.py: Fetch blobs on artifact creation, not after commandJürg Billeter2019-08-083-19/+23
| | | | | | | | | | | | | | | | | | We need to ensure artifact files are available locally also for elements that don't run any commands (e.g., compose elements). It's also not necessary to fetch file blobs after every command. This moves file blob fetching from SandboxRemote.process_job_output() (part of Sandbox.run()) to a separate fetch_missing_blobs() method, which is invoked by Element._cache_artifact().
| * cascache.py: Add missing remote.init() to fetch_blobs()Jürg Billeter2019-08-081-0/+2
|/
* doc/source/core_plugins.rst: bst-external -> bst-plugins-experimentalJavier Jardón2019-08-081-1/+1
|
* Merge branch 'coldtom/move-cmake' into 'master'1089-reporting-of-unexpected-deaths-of-worker-processes-could-be-betterbst-marge-bot2019-08-087-147/+17
|\ | | | | | | | | Move cmake plugin to bst-plugins-experimental See merge request BuildStream/buildstream!1520
| * Move cmake plugin to bst-plugins-experimentalcoldtom/move-cmakeThomas Coldrick2019-08-087-147/+17
|/ | | | | | | | | | | | It was agreed on the mailing list to move all plugins to a single repository, before moving them into domain-specific repositories. As a result it seems reasonable to move everything to the bst-plugins-experimental repo as this stepping stone, rather than creating a whole new repo. This commit starts the process of moving things over by moving only the cmake plugin to bst-plugins-experimental, and altering the tests to reflect the new location.
* Merge branch 'juerg/buildbox' into 'master'bst-marge-bot2019-08-0724-123/+3
|\ | | | | | | | | Enable integration tests for buildbox sandboxing backend See merge request BuildStream/buildstream!1525
| * tests/integration/shell.py: Remove buildbox xfail for passing testsJürg Billeter2019-08-071-5/+0
| |
| * tests/integration/make.py: Remove buildbox xfail for passing testsJürg Billeter2019-08-071-2/+0
| |
| * Revert "Mark xfail test that only fail in CI"Jürg Billeter2019-08-0722-113/+3
| | | | | | | | This reverts commit b54c8cb07ca257be79940ffa70853bf75d2c287c.
| * .gitlab-ci.yml: Do not remove bubblewrap in tests-buildboxJürg Billeter2019-08-071-3/+0
|/ | | | buildbox-fuse uses bubblewrap for sandboxing.
* Merge branch 'jennis/junctions_and_remotes' into 'master'bst-marge-bot2019-08-067-10/+148
|\ | | | | | | | | Revert default junction behaviour and add new junction config options See merge request BuildStream/buildstream!1403
| * NEWS: Added a news entry announcing the two new junction config optionsJames Ennis2019-08-061-0/+8
| |
| * plugins/elements/junction.py: Add 'ignore-junction-remotes' optionJames Ennis2019-08-063-4/+77
| | | | | | | | | | | | | | | | This option allows us to completely ignore remotes defined in the subproject. As a user, we may not wish to interact with subproject remotes as we may not be able to trust them, for example. An appropriate test and some documentation has also been added.
| * plugins/elements/junction.py: Add 'cache-junction-elements' optionJames Ennis2019-08-064-2/+61
| | | | | | | | | | | | | | | | | | The 'cache-junction-elements' configuration option for junction's is a boolean which enables elements from within the junction to interact with the parent project's remote(s). A test has been added to enforce this behaviour and the BST_FORMAT_VERSION has been bumped.
| * _project.py: Revert back to not caching junctioned elementsJames Ennis2019-08-063-11/+7
| | | | | | | | | | | | | | | | | | | | | | Originally, the intended default behaviour of BuildStream was to only cache project elements in project specific remotes. This behaviour was broken by: 24c0de16faec2b8b9bd6a03504ce951dc49afbe2. Introduced in !1113. This patch reverts the behaviour and asserts that junctioned elements are not pushed to their parent's project-specific remote.
| * format_project.rst: Add required push flag to documentationJames Ennis2019-08-061-0/+2
|/ | | | | When declaring artifact/source servers which we want to push to, we must set a "push" boolean
* Merge branch 'traveltissues/setstability' into 'master'bst-marge-bot2019-08-061-10/+20
|\ | | | | | | | | | | | | Add Element.__update_cache_keys_stability method Closes #1087 See merge request BuildStream/buildstream!1492
| * Add Element.__update_cache_keys_stability()Darius Makovsky2019-08-061-10/+20
|/ | | | | | This method updates the value of Element.__cache_keys_stable which is initially set in the Element.__update_cache_keys() method.
* Merge branch 'jjardon/code_quality' into 'master'bst-marge-bot2019-08-051-24/+3
|\ | | | | | | | | | | | | .gitlab-ci.yml: Use new way to define code_quality job Closes #1099 See merge request BuildStream/buildstream!1522
| * .gitlab-ci.yml: Use new way to define code_quality jobJavier Jardón2019-08-051-24/+3
|/ | | | See https://docs.gitlab.com/ee/ci/examples/code_quality.html
* .gitlab-ci.yml: Use latest fdsdk release for overnght testsJavier Jardón2019-08-051-1/+1
|
* Merge branch 'update_committers' into 'master'bst-marge-bot2019-08-022-5/+66
|\ | | | | | | | | Update COMMITTERS.rst file See merge request BuildStream/buildstream!1510
| * Update COMMITTERS.rstupdate_committersJürg Billeter2019-08-011-0/+60
| |
| * contrib/update_committers.py: Move COMMITTERS.rst to top-level directoryJürg Billeter2019-08-011-4/+5
| |
| * contrib/update_committers.py: Fix typo in template file nameJürg Billeter2019-08-011-1/+1
|/
* Merge branch 'alexfazakas/fetch-committers' into 'master'bst-marge-bot2019-07-314-7/+178
|\ | | | | | | | | | | | | Automatically build a list of committers Closes #1071 See merge request BuildStream/buildstream!1505
| * Remove MAINTAINERS fileAlexandru Fazakas2019-07-311-7/+0
| |
| * contrib: Update COMMITTERS automaticallyAlexandru Fazakas2019-07-312-0/+129
| | | | | | | | | | | | | | | | | | | | Since we want the COMMITTERS list to be sync'd with the GitLab permissions, use a script for fetching all the information and building a table with it automatically. The script uses a template located in the contrib/ directory. Closes https://gitlab.com/BuildStream/buildstream/issues/1071
| * Update CONTRIBUTING.rstLaurence Urhegyi2019-07-311-0/+49
|/ | | | | Adds some guidelines on how committers are chosen,granted access and a note about non-code contributions.
* Merge branch 'bschubert/small-fixes' into 'master'bst-marge-bot2019-07-311-1/+1
|\ | | | | | | | | job: fix exception caught from enum translation See merge request BuildStream/buildstream!1509