summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests: Add filter element tests214-we-need-a-way-to-make-elements-depend-on-a-subset-of-an-element-s-outputJonathan Maw2018-03-0114-0/+168
|
* Add Filter elementJonathan Maw2018-03-012-0/+133
|
* Pipeline: Give more helpful errors when opening a workspace for an element ↵Jonathan Maw2018-03-012-3/+7
| | | | | | | | | that has no sources When trying to look at the source code for an element, elements that modify artifacts (e.g. script and compose elements) don't have sources, so we suggest some other elements that they might try opening workspaces for.
* scriptelement: Use built-in checks on runtime depends and sourcesJonathan Maw2018-03-011-10/+8
|
* compose: Use built-in checks on runtime depends and sourcesJonathan Maw2018-03-011-11/+9
|
* Allow plugins to forbid runtime dependencies and sourcesJonathan Maw2018-03-014-2/+36
| | | | | | The flags BST_FORBID_RDEPENDS and BST_FORBID_SOURCES can be set in the plugin's class declaration, and exceptions will be raised during pre-flight checks
* Update expected cache keys in testsJames Ennis2018-02-286-6/+6
|
* compose.py: key dictionary is now correctly initialisedJames Ennis2018-02-281-4/+3
|
* Modify how the INTEGRATION_CACHE environment var is interpretedSam Thursfield2018-02-282-3/+3
| | | | | | | | When specifying a location for the integration tests cache directory, we would previously append '/integration-cache' to whatever path we were given. This is regarded as confusing. See: https://gitlab.com/BuildStream/buildstream/issues/267
* Store integration tests cache inside the current directory by defaultSam Thursfield2018-02-281-6/+6
| | | | | | | | Previously the code would default to a directory in `/tmp`, but this is often unsuitable as the Linux 'tmpfs' filesystem doesn't support extended file attributes and thus cannot store OSTree repositories. See: https://gitlab.com/BuildStream/buildstream/issues/267
* doc/source/projectconf.rst: Documenting `host-files` shell configurationTristan Van Berkom2018-02-281-0/+42
|
* tests/integration/shell.py: Added tests for host-filesTristan Van Berkom2018-02-282-0/+98
| | | | | | | | | | | | | | | | | Newly added tests: o Test that bind mounting a file works o Test that bind mounting a file into a non-existing directory works o Test that bind mounting is disabled in an isolated shell o Test that the shell still works, but a warning is printed and the mount is refused in case the specified file on the host was found to be a directory o Test that the shell still works, but a warning is printed in the case that the file specified on the host does not exist
* sandbox/_mount.py: Ensure file existence for bind mountsTristan Van Berkom2018-02-281-1/+17
| | | | | | | This logic existed but is now a bit more complex with the addition of allowing explicit bind mounts to be defined for the shell; the change is that now files can also be mounted into the sandbox instead of just directories.
* element.py: Inform the sandbox of host-files when running a shellTristan Van Berkom2018-02-281-0/+10
| | | | Does not apply to isolated shells.
* _project.py: Added 'host-files' shell configurationTristan Van Berkom2018-02-281-2/+16
| | | | | | | | | | | A shell configuration allowing one to bind mount files into the shell sandbox. This bumps the BST_FORMAT_VERSION to 2. This is related to #223, and solves #241 (name resolution problems) by making it possible to specify a bind mount for `/etc/resolv.conf` in the project configuration.
* tests/integration/shell.py: Generate project.conf on the flyTristan Van Berkom2018-02-282-7/+28
|
* tests/frontend/workspace.py: Add cache key checks to test_buildJürg Billeter2018-02-271-0/+2
| | | | Regression test for #273
* Determine as early as possible whether a build is pendingJürg Billeter2018-02-272-4/+41
| | | | | | | | | Workspaced sources are considered unstable if a build is pending as the build will modify the contents of the workspace. Determine as early as possible if a build is pending to be able to discard unstable cache keys. Fixes #273
* tests/frontend/pull.py: Do not use a workspace for the non-strict testJürg Billeter2018-02-271-2/+3
| | | | | The cache status of reverse dependencies of workspaced elements cannot always be determined upfront.
* element.py: Remove unused _get_strict_cache_key() methodJürg Billeter2018-02-271-12/+0
| | | | Cache key handling is now contained within the Element class.
* element.py: Add _can_query_cache() methodJürg Billeter2018-02-273-4/+13
| | | | | Keep the knowledge about which cache key is required for cache queries within the Element class.
* Remove unused _KeyStrength importsJürg Billeter2018-02-273-5/+0
|
* _artifactcache: Add key parameter to pull() methodJürg Billeter2018-02-273-34/+61
| | | | Contain cache key logic in Element class.
* _artifactcache: Add keys parameter to push() methodJürg Billeter2018-02-273-14/+16
| | | | Contain cache key logic in Element class.
* _artifactcache: Add key parameter to push_needed() methodJürg Billeter2018-02-273-7/+10
| | | | Contain cache key logic in Element class.
* _artifactcache: Add keys parameter to commit() methodJürg Billeter2018-02-274-26/+24
| | | | Contain cache key logic in Element class.
* _ostree.py: Use list of refs as commit parameterJürg Billeter2018-02-272-11/+7
|
* _artifactcache: Add key parameter to extract() methodJürg Billeter2018-02-274-22/+22
| | | | Contain cache key logic in Element class.
* _artifactcache: Add key parameter to contains() methodJürg Billeter2018-02-274-43/+14
| | | | Contain cache key logic in Element class.
* _artifactcache/artifactcache.py: Add pull() to abstract base classJürg Billeter2018-02-271-0/+12
|
* _artifactcache/artifactcache.py: Add push() to abstract base classJürg Billeter2018-02-271-0/+17
|
* _artifactcache/tarcache.py: Remove unused remove() methodJürg Billeter2018-02-271-14/+0
|
* element.py: Remove unused strength parameter from _cached()Jürg Billeter2018-02-271-22/+4
| | | | | | Remove unused strength parameter from _cached() and _remotely_cached(), which makes them simple accessor functions as `__cached` is equivalent to `__strong_cached` with a strict build plan.
* Integration tests: Removing some dead codeTristan Van Berkom2018-02-278-29/+4
| | | | | | | | | | o Removing format_files() helper in testutils o Removing imports of format_files() o Removing imports of walk_dir() where it's not used o Removing `element_path` variables that are unused
* tests/frontend/push.py: Test that we don't push stuff that we just pulledSam Thursfield2018-02-262-0/+67
|
* tests/frontend: Summarize the existing push and pull testsSam Thursfield2018-02-262-0/+27
| | | | | These tests are too long to understand at a glance so a summary is needed of each.
* Don't push artifacts to remote caches that already contain themSam Thursfield2018-02-263-8/+64
| | | | | | | | | | | | | | | | | | | | | This adds two separate checks to avoid pushing artifacts unnecessarily. First, in the OSTreeCache.push() method we now first obtain the list of remotes that contain the refs that we are about to push. We then avoid pushing to any remote that already contains that ref. Note that the internal ref map is read once on process startup, so if multiple `bst` processes are pushing to a cache they might still both push the same artifact. There is an existing issue for this: https://gitlab.com/BuildStream/buildstream/issues/179 Secondly the Element._skip_push() method now checks if all remote caches configured for pushing already have a given artifact, and will skip the push job altogether if they do. The first check would already mean that no pushes would actually happen, but without the second check the user would still see Push jobs being created for every artifact which would be quite misleading.
* _artifactcache/ostreecache.py: Rework so that all remotes for a ref are trackedSam Thursfield2018-02-261-58/+81
| | | | | | | | | | This commit adds a couple of simple classes in order to reduce the number of different dictionaries tracking the state of the remotes within the OSTreeCache object. It also extends the internal ref map to remember all the remotes that store a given ref, not just the highest priority ref, and modifies the remote_contains_key() method to expose that data.
* _artifactcache/artifactcache.py: Fix name mismatch in doc commentSam Thursfield2018-02-261-1/+1
|
* doc/source/format.rst: Document format version for junction dependency attributeTristan Van Berkom2018-02-261-0/+4
| | | | | | The junction attribute was merged fairly recently in the 1.1 series without bumping the project format version, this documents the junctions addition to be added since the closest version bump where support exists.
* _downloadablefilesource.py: Add ETag supportjuerg/etagJürg Billeter2018-02-251-7/+24
| | | | | | This avoids unnecessary downloads when tracking `tar` and `zip` sources. Fixes #62
* MANIFEST.in: remove files that are no longer in this repoChandan Singh2018-02-241-2/+0
| | | | | We don't have `Dockerfile` and `Dockerfile-build.sh` anymore in this repository so remove them from `MANIFEST.in`.
* tests/integration/shell.py: Added tests for inheriting environment variablesTristan Van Berkom2018-02-252-3/+44
|
* tests/integration/shell.py: Dont use shlex, use command vectorsTristan Van Berkom2018-02-251-9/+15
| | | | | | | | | Using shlex messes with the argv we want to pass to a shell, this was not working for the newly added test which passes "${var}" strings as arguments to the shell. Also, removed a redundant line to explicitly build something in the 'no_shell' test; that is taken care of by the helper.
* testutils/runcli.py: Print something helpful if BuildStream fails to exit ↵Tristan Van Berkom2018-02-251-0/+1
| | | | | | | | properly When testing a recent patch, I had missed a return of the exit code in `bst shell` resulting in the CLI exiting with `None`; this patch makes the fixture more helpful, where previously it just printed `None`
* doc/source/projectconf.rst: Added documentation about customizing the shellTristan Van Berkom2018-02-252-0/+57
|
* element.py: Use project shell configuration when launching shells.Tristan Van Berkom2018-02-251-9/+10
| | | | | | | Use the toplevel project configuration to decide: o What command to run for an interactive shell o Which environment variables to inherit from host environment when not running an isolated shell
* _context.py: Added _get_toplevel_project() convenience accessorTristan Van Berkom2018-02-251-0/+11
|
* _project.py: Parse the new 'shell' sectionTristan Van Berkom2018-02-251-3/+15
| | | | | | | | | | | | | | | | | | | | | This new section informs BuildStream how to launch interactive shells for the specific project. These new options are part of the effort to make applications work better inside a `bst shell` environment, issue #223 Initial options include: o Default shell command to run for an interactive shell, this is used for interactive debugging sessions, and also if no command was specified in `bst shell` o List of environment variables to inherit from user environment, this is useful to propagate some host stuff into the environment for debugging; and only considered when it is not an "isolated" shell. These changes also bump the BST_FORMAT_VERSION
* _frontend: Added App.shell() to call instead of directly invoking the ↵Tristan Van Berkom2018-02-242-2/+20
| | | | | | | | Element._shell() And use this place to format a custom prompt for PS1, here we have click and we use the ANSI colors in PS1 only if colors are enabled.