summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/frontend/push.py: Add test_push_after_rebuildjuerg/artifact-blob-not-foundJürg Billeter2020-04-203-1/+74
|
* _artifact.py: Don't cache the proto of incomplete artifactsJürg Billeter2020-04-201-10/+14
| | | | | | | We need to make sure that we read the new proto after a fresh build. And there is no need to keep the proto around for an incomplete artifact. This fixes `FileNotFoundError` of CAS blob on push after rebuild.
* _casbaseddirectory.py: Fix temp file permissions in open_file()Jürg Billeter2020-04-201-0/+2
| | | | This is required when buildbox-casd is running as separate user.
* _cas/cascache.py: Drop unused contains_file()Jürg Billeter2020-04-201-12/+0
|
* _artifact.py: Don't consider an artifact cached if logs are missingJürg Billeter2020-04-201-12/+7
| | | | | | | | | Artifact push and pull operations currently fail if logs are missing. We don't currently have a config option to control how long artifact logs should be kept in the cache. Until this changes, we should be conservative and consider logs to be an essential part of artifacts, keeping them from getting expired before the rest of the artifact.
* _cas/cascache.py: Add contains_files() methodJürg Billeter2020-04-201-5/+17
| | | | Like `contains_file()` but checks multiple files in a single request.
* tests/frontend/pull.py: Add test_pull_missing_blob_split_shareJürg Billeter2020-04-201-29/+63
| | | | | This is identical to `test_pull_missing_blob` except that it uses separate servers for artifact proto index and CAS.
* cascache.py: Add allow_partial parameter to fetch_blobs()Jürg Billeter2020-04-204-15/+9
| | | | This fixes handling of missing blobs in `ArtifactCache.pull()`.
* _artifactcache.py: Handle BlobNotFound error in pull()Jürg Billeter2020-04-201-0/+5
|
* _artifactcache.py: Fix misleading log message when pulling artifactsJürg Billeter2020-04-201-2/+1
| | | | | Do not print message "Pulled artifact" when pulling the artifact proto without data. This matches the log messages in `SourceCache`.
* Merge branch 'juerg/tar-hardlinks' into 'master'Jürg Billeter2020-04-202-3/+8
|\ | | | | | | | | tests/sources/tar.py: Fix flaky test_out_of_basedir_hardlinks See merge request BuildStream/buildstream!1870
| * tests/internals/cascache.py: Increase sleep duration to fix flaky testJürg Billeter2020-04-201-2/+2
| | | | | | | | | | Increase sleep duration in test_casd_redirects_stderr_to_file_and_rotate from 0.05s to 0.5s to reduce risk of test failure.
| * tests/sources/tar.py: Fix flaky test_out_of_basedir_hardlinksjuerg/tar-hardlinksJürg Billeter2020-04-201-1/+6
|/ | | | | | Don't recursively add `contents` to the tar file as the order is not guaranteed. We need to add `elsewhere` before `to_extract` as the latter references the former in `linkname`.
* Merge branch 'tristan/user-guide-refactor' into 'master'Tristan Van Berkom2020-04-18147-806/+4781
|\ | | | | | | | | User guide enhancements See merge request BuildStream/buildstream!1864
| * doc/source/sessions-stored: Refreshing built session filestristan/user-guide-refactorTristan Van Berkom2020-04-1753-683/+2180
| | | | | | | | | | | | | | | | | | This refreshes the stored sessions and adds the new ones generated by recent additions to the user guide. The stored sessions exist to allow contributors to build the docs without having to build the sessions, which requires an internet connection and takes longer to build.
| * doc/source/conf.py: Update copyright yearTristan Van Berkom2020-04-171-1/+1
| |
| * doc/examples/overlaps: Added new user guide entry for overlapping filesTristan Van Berkom2020-04-1716-0/+344
| |
| * doc/examples/filtering: Added a new user guide entry about filter elementsTristan Van Berkom2020-04-1715-0/+433
| |
| * doc/examples/composition: Adding user guide about split-rules and compose ↵Tristan Van Berkom2020-04-1717-1/+410
| | | | | | | | elements
| * doc/examples/strict-mode: Adding a user guide for using non-strict build plansTristan Van Berkom2020-04-1717-0/+507
| |
| * doc/examples/directives: Add user guide for using project options and directivesTristan Van Berkom2020-04-1710-0/+318
| |
| * doc/examples/junction-includes: Adding new example about includes with junctionsTristan Van Berkom2020-04-1711-0/+376
| |
| * doc: Refactoring junction documentationTristan Van Berkom2020-04-1711-97/+177
| | | | | | | | | | | | | | | | | | | | | | | | Renamed the "Advanced topics" section to "Combining projects", and splitting up the junctions section into two separate documents, one which describes the simplest junction example and another which explains how to use workspaces in conjunction with junctions. This commit also makes some minor changes the examples, correcting some grammatical errors, improving links and formatting, also rewording a `note::` in order to clarify that it would be unusual to junction a project that is in a subdirectory.
| * doc/source/using_developing.rst: Renamed to "Developing"Tristan Van Berkom2020-04-171-5/+5
| | | | | | | | | | | | | | | | Calling this "Developing using BuildStream" is just redundant, seeing that this is a part of a BuildStream user guide already. Also, rephrased the introduction text (which appears on a page which is rarely frequented anyway).
| * doc/source/using_tutorial.rst: Renamed to "Getting started"Tristan Van Berkom2020-04-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | This section of the user manual describes the basics of creating your first BuildStream project, while we've discussed this in terms of a "Getting started" guide while developing it, we ended up naming this a "Tutorial" because of it's walkthrough nature. Due to it's name as a Tutorial, developers (our target audience) have a tendency to avoid it and look for more terse and advanced material, while this material could be more suitable for getting started.
| * doc/source/format_intro.rst: Clarifications around directivesTristan Van Berkom2020-04-171-15/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Clarification of (@) include documentation The "important" annotation here was very confusing to read, rewrote this section to clarify that files included across a junction cannot be used to inform the declaration of a junction, as this can present a circular dependency. Clarification around conditionals and includes Clarify that conditional statements are always resolved in the context of the project where the conditional statement was declared.
| * doc/source/main_glossary.rst: rename 'Core Plugin' -> 'Core plugin'Tristan Van Berkom2020-04-171-2/+2
|/ | | | Fix an error in documentation build.
* Merge branch 'juerg/cache-key' into 'master'Jürg Billeter2020-04-172-2/+24
|\ | | | | | | | | element.py: Fix strong cache key calculation in non-strict mode See merge request BuildStream/buildstream!1865
| * tests/frontend/buildcheckout.py: Add non-strict testJürg Billeter2020-04-161-0/+22
| | | | | | | | | | Test that a clean build in non-strict mode produces an artifact matching the strict cache key.
| * element.py: Tweak cache key calculation for `BST_STRICT_REBUILD`Jürg Billeter2020-04-161-1/+1
| | | | | | | | | | Include the project name and element name of dependencies in all cache key variants as the dependency names are stored in the artifact proto.
| * element.py: Fix strong cache key calculation in non-strict modeJürg Billeter2020-04-161-1/+1
|/ | | | | The algorithm for calculating the strong cache key should be identical in strict and non-strict mode.
* Merge branch 'valentindavid/include-options-from-junction' into 'master'Tristan Van Berkom2020-04-1621-12/+217
|\ | | | | | | | | Process options in includes files with the options of their junction See merge request BuildStream/buildstream!1851
| * Process options in includes files with the options of their junctionvalentindavid/include-options-from-junctionValentin David2020-04-1521-12/+217
|/ | | | | | | | | Unfortunately the options from main project cannot always be processed in the include processing since project configuration might load option declarations from a separate file. For that reason the result of `Include.process` should still be passed through the option processor. But all options files included from junctioned are already evaluated.
* Merge branch 'juerg/platform' into 'master'bst-marge-bot2020-04-1413-66/+66
|\ | | | | | | | | Improve sandbox configuration handling See merge request BuildStream/buildstream!1845
| * _sandboxbuildboxrun.py: Detect platforms supported by buildbox-runjuerg/platformJürg Billeter2020-04-142-8/+22
| | | | | | | | | | | | This allows builds where the host OS or architecture doesn't match the build OS or architecture, if the buildbox-run implementation supports it. E.g., this allows x86-32 builds on x86-64 systems.
| * sandbox: Reinitialize class when pickle is usedJürg Billeter2020-04-142-0/+8
| | | | | | | | | | | | Class data is not pickled and thus, class variables set by `check_available()` are not set when `check_sandbox_config()` is called by a job subprocess.
| * element.py: Move sandbox configuration check to __sandbox()Jürg Billeter2020-04-141-4/+2
| | | | | | | | | | This allows `bst show` and other commands that don't require a sandbox to work with elements that are not supported by the current sandbox.
| * Drop concept of partially supported sandbox configurationJürg Billeter2020-04-148-23/+7
| | | | | | | | | | | | | | | | | | | | | | | | Allowing builds without affecting the cache key but disallowing push, when the sandbox configuration is not fully compatible, results in an inconsistent user experience and may lead to unexpected build issues. Especially as push is allowed or disallowed based on the sandbox available at the time of push, not at the time of build. The previous commit making build-uid and build-gid configuration optional allows each project to decide whether the sandbox is required to support UID/GID configuration or not.
| * sandbox: Make build-uid and build-gid configuration optionalJürg Billeter2020-04-146-28/+22
| | | | | | | | | | This allows use of sandbox implementations that don't support configuring sandbox UID/GID such as buildbox-run-userchroot.
| * node.pyx: MappingNode.get_int(): Support `None` as defaultJürg Billeter2020-04-142-4/+6
|/
* Merge branch 'tristan/fix-docs-build-error' into 'master'Tristan Van Berkom2020-04-111-2/+2
|\ | | | | | | | | doc/source/main_glossary.rst: Fix docs build errors See merge request BuildStream/buildstream!1863
| * doc/source/main_glossary.rst: Fix docs build errorstristan/fix-docs-build-errorTristan Van Berkom2020-04-111-2/+2
|/ | | | | Some of the referenced terms within the glossary were not using the capitalization for which the term was declared.
* Merge branch 'juerg/job-sigterm' into 'master'bst-marge-bot2020-04-093-11/+13
|\ | | | | | | | | Fix SIGTERM handling in job processes See merge request BuildStream/buildstream!1861
| * _sandboxbuildboxrun.py: Reduce SIGTERM timeout to 15sJürg Billeter2020-04-091-1/+1
| | | | | | | | | | | | | | The scheduler timeout for `SIGTERM` of job processes is 20s. The timeout in the sandbox backend should be shorter to allow graceful termination of the job process even when `buildbox-run` fails to terminate gracefully.
| * job.py: Use `_signals.terminator()` to handle `SIGTERM`Jürg Billeter2020-04-091-9/+7
| | | | | | | | | | | | `Sandbox` subclasses use `_signals.terminator()` to gracefully terminate the running command and cleanup the sandbox. Setting a `SIGTERM` handler in `job.py` breaks this.
| * _signals.py: Allow SIGTERM handler to call `os.exit()`Jürg Billeter2020-04-091-1/+5
|/ | | | Use exit code from the `SystemExit` exception raised by `os.exit()`.
* Merge branch 'jjardon/wsl' into 'master'bst-marge-bot2020-04-081-3/+0
|\ | | | | | | | | | | | | Revert ".gitlab-ci.yml: Allow WSL test to fail" Closes #1277 See merge request BuildStream/buildstream!1860
| * Revert ".gitlab-ci.yml: Allow WSL test to fail"jjardon/wslJavier Jardón2020-04-081-3/+0
|/ | | | This reverts commit edcd52f8f0d9e062bb9a4d089799b47625c37ab6.
* Merge branch 'abderrahim/public-data' into 'master'bst-marge-bot2020-04-082-1/+12
|\ | | | | | | | | | | | | Fix expiry of public data of artifact Closes #1264 See merge request BuildStream/buildstream!1837
| * cascache.py: use buildbox-casd when checking whether a file is cachedabderrahim/public-dataAbderrahim Kitouni2020-04-081-1/+7
| | | | | | | | this lets buildbox-casd know it is needed and not delete it when pruning