summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* buildstream/plugins/sources/git.py: warn about checkout by defaultjjardon/doc_git_submodulesJavier Jardón2018-12-021-0/+5
| | | | See #783
* Merge branch 'mandatory_suffix' into 'master'Jürg Billeter2018-11-3012-10/+149
|\ | | | | | | | | Mandatory .bst suffix See merge request BuildStream/buildstream!967
| * NEWS: Added update about the Mandatory .bst changePhillip Smyth2018-11-301-0/+4
| |
| * Added tests for .bst suffix and completionsPhillip Smyth2018-11-307-6/+91
| | | | | | | | | | | | | | | | buildcheckout.py: Add mandatory .bst suffix tests completions.py: Add test for fail on invalid suffix Added required files for testing integration/source-determinism.py: renamed test elements to end with .bst loader/__init__.py: initialised context properly with load and a message handler
| * cli.py: no autocomplete on files without .bstPhillip Smyth2018-11-301-1/+5
| |
| * loader: Added file suffix check and helper functionPhillip Smyth2018-11-301-0/+40
| | | | | | | | _loader/loader.py: Added Suffix check and warning for named elements and add helper function
| * _project.py: Check for fatal_warnings earlierPhillip Smyth2018-11-301-3/+3
| |
| * plugin.py: Add BAD_ELEMENT_SUFFIX warningPhillip Smyth2018-11-301-0/+6
|/
* Merge branch 'tpollard/774' into 'master'Jürg Billeter2018-11-304-26/+97
|\ | | | | | | | | | | | | _stream.py: Ability to pull missing buildtrees outside of pull/build Closes #774 See merge request BuildStream/buildstream!978
| * _stream.py: Ability to pull missing buildtrees outside of pull/buildTom Pollard2018-11-304-26/+97
|/ | | | | | | | | | | | | | | | | | Adds helper function _buildtree_pull_required() to determine if a pullqueue should be constructed, for commands outside of bst pull and build where it is determined that an element's buildtree artifact is to be required given the respective semantics and config. Utilised in push() to attempt to mitigate skipping the push of partial elements without the user having to have preceded it with an explicit pull. cli.py: Add new behaviour to push command description element.py: Move _cached_buildtree() to be non local private method, use _KeyStrength types to reduce duplication. tests/integration/pullbuildtrees.py also updated to cover this use-case.
* Merge branch 'aevri/update_man' into 'master'Jürg Billeter2018-11-3019-105/+158
|\ | | | | | | | | man/: update with changes since Apr 2018 See merge request BuildStream/buildstream!981
| * man/: update with changes since Apr 2018Angelos Evripiotis2018-11-3019-105/+158
|/ | | | | | | | | | | | | | | | It seems that the regenerated man pages haven't been committed for some time. Add them all in one lump now. Had to manually strip the version number from the generated files. Had to manually fix the NAME field of bst.1 to not be 'bst - None'. Oddly had to invoke click-man directly in order to get the changes to be applied, the usual setup.py route mentioned in contributing.rst didn't work. Eyeballed the result of invoking man on each of the commands, seems good.
* Merge branch 'valentindavid/netrc' into 'master'Valentin David2018-11-299-3/+431
|\ | | | | | | | | | | | | Add support for .netrc in remote/tar/zip plugins Closes #723 See merge request BuildStream/buildstream!908
| * .gitlab-ci.yml: Update image versions where forgottenvalentindavid/netrcValentin David2018-11-291-2/+2
| |
| * Add support for .netrc in remote/tar/zip pluginsValentin David2018-11-298-1/+429
|/ | | | Fixes #723.
* Merge branch 'jmac/no-verify-digests' into 'master'Jim MacArthur2018-11-291-8/+1
|\ | | | | | | | | | | | | _sandboxremote.py: Remove unnecessary tests. Closes #786 See merge request BuildStream/buildstream!976
| * _sandboxremote.py: Remove unnecessary tests.Jim MacArthur2018-11-291-8/+1
|/ | | | | | | push_message and push_directory will both raise assertion failures if they fail to send the digest to the server. Checking the digest returned by each only tests that the content was hashed locally, which cannot reasonably fail.
* Merge branch 'valentindavid/cache_server_fill_up' into 'master'Valentin David2018-11-294-96/+254
|\ | | | | | | | | | | | | Fix cleanup of cache in server when disk is full Closes #678 See merge request BuildStream/buildstream!830
| * Lock cache cleanup in cas servervalentindavid/cache_server_fill_upValentin David2018-11-281-46/+56
| | | | | | | | Cleaning up in parallel might slow down the cleaning process
| * Cleanup cache in cas server more agressivelyValentin David2018-11-283-67/+107
| | | | | | | | | | When there is less than 2GB left, it cleans up have 10GB available. These values are configurable.
| * Move cas server from ref-based to object-based garbage collection.Valentin David2018-11-282-3/+46
| |
| * Update mtimes of objects for requested references.Valentin David2018-11-283-11/+37
| | | | | | | | | | This also remove references when some objects are missing. This is in preparation for the move from reference to object garbage collection.
| * "Fallocate" object disk space to avoid getting NOSPC error when writingValentin David2018-11-281-6/+24
| | | | | | | | | | This locks the temporary object file so that cleanup does not need to be done for every write.
| * Avoid copying temporary file when adding object to CAS in server.Valentin David2018-11-282-18/+26
| | | | | | | | | | | | | | The file is already a temporary file and does not need copy. ENOSPC is thrown during that copy in issue #609. Fixes #678.
| * Fix type error in RPC messagesValentin David2018-11-281-2/+3
| |
| * Make cache clients not fail when a blob is not available.Valentin David2018-11-281-0/+12
| | | | | | | | | | | | | | We plan to make cache incomplete. That is some blobs are missing. For most of cases we will delete references when requested if they are incomplete. But there will be corner cases where objects are removed after the reference is requested.
| * Use f_bavail to query available space. Not f_bfree.Valentin David2018-11-281-1/+1
|/ | | | | | f_bfree space might not be usable. In practice we see failures in big disks because f_bfree is over 2GB and f_bavail is 0. We get ENOSPC if writing on disk then.
* Merge branch 'valentindavid/ruamel_0_15_41' into 'master'Valentin David2018-11-281-1/+8
|\ | | | | | | | | setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52 See merge request BuildStream/buildstream!975
| * setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52valentindavid/ruamel_0_15_41Valentin David2018-11-281-1/+8
|/
* Merge branch 'jmac/remote_execution_split' into 'master'Jim MacArthur2018-11-2712-116/+288
|\ | | | | | | | | | | | | Split remote execution from artifact cache Closes #750 See merge request BuildStream/buildstream!946
| * Docs: Correct URLs in artifact cache exampleJim MacArthur2018-11-271-2/+2
| | | | | | | | | | The port number must come after the hostname and before the path. The path is ignored by CASCache anyway, so including it is misleading.
| * artifactcache.py: Remove unused verify_digest_pushedJim MacArthur2018-11-271-28/+0
| |
| * tests/sandboxes/remote-exec-config.py: New test.Jim MacArthur2018-11-274-0/+102
| | | | | | | | | | Tests that we get a sensible error message when we specify incomplete or wrong information in the configuration for remote execution.
| * Documentation: Update to show new config optionsJim MacArthur2018-11-271-3/+17
| |
| * _sandboxremote: Remove references to artifact cache.Jim MacArthur2018-11-274-37/+109
| | | | | | | | | | Uses the new config options introduced earlier to specify remote exec storage and execution service.
| * artifactcache.py, _context.py: Move CASCache object into contextJim MacArthur2018-11-272-2/+9
| | | | | | | | | | | | Since the artifact cache and remote execution share the same local CAS store, they should share the same CASCache object. Moving this into context allows us to do this.
| * artifactcache: Move ArtifactCacheSpec code into CASRemoteSpec.Jim MacArthur2018-11-272-45/+50
|/ | | | | | There is nothing in ArtifactCacheSpec that's actually specific to artifacts, so I've made it a CAS class so we can use the same spec for remote execution.
* Merge branch 'juerg/command-batching' into 'master'Jürg Billeter2018-11-2714-108/+498
|\ | | | | | | | | | | | | Command batching Closes #675 See merge request BuildStream/buildstream!915
| * NEWS: Add entry for command batchingJürg Billeter2018-11-271-0/+3
| |
| * sandbox/_sandboxremote.py: Implement command batchingJürg Billeter2018-11-271-1/+69
| | | | | | | | Execute batched commands in a single shell script.
| * compose.py: Support batching for integration commandsJürg Billeter2018-11-271-2/+3
| |
| * scriptelement.py: Support batching for integration and build commandsJürg Billeter2018-11-271-25/+26
| |
| * buildelement.py: Support batching for integration and build commandsJürg Billeter2018-11-272-13/+12
| |
| * element.py: Add batch_prepare_assemble() methodJürg Billeter2018-11-271-6/+47
| | | | | | | | | | This allows batching of all sandbox commands aross prepare() and assemble().
| * element.py: Support command batching for integration commandsJürg Billeter2018-11-271-8/+7
| |
| * element.py: Defer workspace.prepared setting in case of pending commandsJürg Billeter2018-11-271-1/+6
| |
| * element.py: Cache artifact also on SandboxCommandErrorJürg Billeter2018-11-271-2/+2
| |
| * element.py: Pass element object to sandboxJürg Billeter2018-11-271-0/+2
| | | | | | | | This allows the sandbox to output messages using the plugin id.
| * sandbox/sandbox.py: Add command batching APIJürg Billeter2018-11-273-6/+251
| | | | | | | | This adds the batch() context manager.
| * scriptelement.py: Use SandboxFlags.NONE instead of 0Jürg Billeter2018-11-271-2/+5
| |