summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/integration/shell.py: TESTING TESTING ONE TWO THREEEEEEproject-shell-customizeTristan Van Berkom2018-02-241-0/+15
|
* tests/integration/shell.py: Added tests for inheriting environment variablesTristan Van Berkom2018-02-242-0/+27
|
* tests/integration/shell.py: Dont use shlex, use command vectorsTristan Van Berkom2018-02-241-9/+15
| | | | | | | | | Using shlex messes with the argv we want to pass to a shell, this was not working for the newly added test which passes "${var}" strings as arguments to the shell. Also, removed a redundant line to explicitly build something in the 'no_shell' test; that is taken care of by the helper.
* testutils/runcli.py: Print something helpful if BuildStream fails to exit ↵Tristan Van Berkom2018-02-241-0/+1
| | | | | | | | properly When testing a recent patch, I had missed a return of the exit code in `bst shell` resulting in the CLI exiting with `None`; this patch makes the fixture more helpful, where previously it just printed `None`
* doc/source/projectconf.rst: Added documentation about customizing the shellTristan Van Berkom2018-02-232-0/+51
|
* element.py: Use project shell configuration when launching shells.Tristan Van Berkom2018-02-231-9/+10
| | | | | | | Use the toplevel project configuration to decide: o What command to run for an interactive shell o Which environment variables to inherit from host environment when not running an isolated shell
* _context.py: Added _get_toplevel_project() convenience accessorTristan Van Berkom2018-02-231-0/+11
|
* _project.py: Parse the new 'shell' sectionTristan Van Berkom2018-02-231-1/+13
| | | | | | | | | | | | | | | | This section informs BuildStream how to launch interactive shells for the specific project. Initial options include: o Default shell command to run for an interactive shell, this is used for interactive debugging sessions, and also if no command was specified in `bst shell` o List of environment variables to inherit from user environment, this is useful to propagate some host stuff into the environment for debugging; and only considered when it is not an "isolated" shell.
* _frontend: Added App.shell() to call instead of directly invoking the ↵Tristan Van Berkom2018-02-232-2/+20
| | | | | | | | Element._shell() And use this place to format a custom prompt for PS1, here we have click and we use the ANSI colors in PS1 only if colors are enabled.
* element.py: Element._shell() now allows a custom prompt for PS1Tristan Van Berkom2018-02-231-2/+6
|
* bst shell: Added --isolate option, disabled by default.tristan/isolate-shellTristan Van Berkom2018-02-233-17/+19
| | | | | | | | | | | | | | | | | | This changes a few things: o How the shell environment and sandbox modes are setup is not decided by whether a `--build` sandbox was staged or not o In interactive build debugging mode, an isolated build sandbox is used o The bst shell command now has the option to control it o All invocations of bst shell now use a read-only rootfs, avoiding FUSE layer and file descriptor limits (this can be made optional later if needed) This is a part of issue #223
* Inherit user id and group id for bst shellJürg Billeter2018-02-223-2/+13
| | | | | | This allows D-Bus access. Fixes #227
* _ostree.py: Disable OSTree's minimum disk space check in our repossam/disable-ostree-free-space-checkSam Thursfield2018-02-221-0/+7
| | | | | | | | | | | | | | | | This interacts badly with our builds, usually if the check does fire it's at the point that we are trying to commit a completed build (which may have taken a long time to complete) into the cache. Even though there might be enough space available to store the build, the error causes BuildStream to consider it "failed" and forces the user to start again. This aims to fix https://gitlab.com/BuildStream/buildstream/issues/216 although there is some evidence that it does not completely do so. BuildStream currently has no minimum disk space check, and no way to automatically clean up the cache. There is an open issue for this: https://gitlab.com/BuildStream/buildstream/issues/135
* doc/source/projectconf.rst: Moving docs for `fail-on-overlaps`Tristan Van Berkom2018-02-211-17/+19
| | | | | | This is not a part of the documentation on how external plugins are loaded and referenced; lets put it in the main general section instead.
* tests/integration/base/generate-base.sh: Typo fix in commentSam Thursfield2018-02-211-1/+1
|
* tests/integration: Use a minimal custom base sysrootSam Thursfield2018-02-219-29/+83
| | | | | | | | | | | | | | | We have been using the Freedesktop SDK binaries to provide a base system to run the integration tests. This works OK but it weighs in at 985MB of content, and there is no simple way to customize it so that we only download the bits we actually need. This commit changes the tests to use a custom sysroot based on the Alpine Linux distribution. The sysroot is 155MB unpacked, and packs down to a 27MB .tar.xz. This speeds up the integration tests significantly as we greatly reduce the amount of network traffic required and the amount of data that gets copied around when creating the staging area.
* tests/integration: Do not use C++ compiler in workspace-mount testJürg Billeter2018-02-213-8/+8
| | | | | | | This means that the integration test suite no longer requires a C++ compiler, and in practice this can allow us to avoid depending on the GNU C/C++ compilers altogether and instead use the vastly smaller Tiny C Compiler.
* tests/integration: Use a simpler CMake test projectSam Thursfield2018-02-215-31/+17
| | | | | | | | | | | The 'step7' tutorial was fine, but it requires a C++ compiler to build. I have replaced it with a simpler testcase which is based on the existing amhello testcase. This means that the integration test suite no longer requires a C++ compiler, and in practice this can allow us to avoid depending on the GNU C/C++ compilers altogether and instead use the vastly smaller Tiny C Compiler.
* doc: Add 'sandboxing' sectionSam Thursfield2018-02-203-0/+204
| | | | | This attempts to document and specify our current sandboxing functionality.
* Create workspace directory after checking for potential issuesChandan Singh2018-02-191-14/+15
| | | | | | | | | | At present, in case that `bst workspace open` for reasons such as sources not being tracked or if tracking fails, BuildStream will leave behind an empty directort corresponding to the workspace. Restructure code so that workspace directory is only created after other checks have passed. Fixes #200.
* doc: Update information on how plugin configuration is composedJonathan Maw2018-02-191-19/+34
| | | | | | | | It previously neglected to describe how sources were composed, and we've changed how sources are composed, recently. In addition, it was slightly confusing to have the project conf be the title of two different stages of composition.
* tests: Add a git test that project overrides workJonathan Maw2018-02-194-0/+52
|
* Add a 'sources' field to project.conf to override defaultsJonathan Maw2018-02-193-3/+56
| | | | | Equivalent to the 'elements' field, but slightly different because sources don't have accompanying yaml.
* tests: Add tests to verify git sources obey submodule checkout configJonathan Maw2018-02-192-0/+208
|
* testutils: Add support for git source controlling submodule checkoutJonathan Maw2018-02-191-6/+11
|
* tests: fix erroneous comment in git testJonathan Maw2018-02-191-1/+1
|
* git.py: Add the ability to set whether to checkout submodulesJonathan Maw2018-02-191-4/+35
| | | | | | | | | | This can be set as broad as "No submodule in this element should be checked out" and as narrow as "only this submodule should be checked out" Note that this will change the cache-keys of every element that uses a git source. I could have structured the code to prevent the cache key changing, but that adds extra complexity in the long-term.
* Regenerating man pages for 1.1.0 dev release1.1.0Tristan Van Berkom2018-02-1916-123/+105
|
* Updating NEWS for 1.1.0 releaseTristan Van Berkom2018-02-191-0/+3
|
* bst-here: add usage instructionsChandan Singh2018-02-171-1/+25
|
* bst-here: fix path of bst binaryChandan Singh2018-02-171-1/+1
| | | | | | | | | | | As shown in the snippet below, the `bst` binary is installed at `/usr/local/bin/bst` in the current Docker image `buildstream/buildstream-fedora`, ``` $ docker run --rm buildstream/buildstream-fedora type -a bst bst is /usr/local/bin/bst ```
* element.py: Create real build shell for `bst shell --build`proper-build-shellsTristan Van Berkom2018-02-161-11/+18
| | | | | | | | Now we infer that a build shell is desired if `bst shell --build` is specified, before there was no real build shell (only staged sources in the build style shell). Fixes issue #232
* _yaml.py: Preserve quotes when run through yaml parsingPhillip Smyth2018-02-151-1/+1
| | | | This preserves the quoting style for string values in round trips.
* _yaml.py: Prevent ruamel from removing underscoresPhillip Smyth2018-02-157-20/+52
| | | | | | | | | | The ruamel parser interprets values such as 1_2_3 as numbers, ignoring the underscores. However, for `track` we need the value as a verbatim string. This change configures ruamel to parse all values as strings, leaving potential conversions to node_get() with the `expected_type` parameter. This fixes #166 and adds tests
* _plugincontext.py: Fix unhandled exception if plugin is not foundsam/plugin-errorSam Thursfield2018-02-141-0/+5
| | | | | | | | | | | | | | | | | | This fixes an AttributeError triggered if we try to import a plugin from a package which is installed but does not contain that plugin. Before: [--:--:--] START Resolving pipeline Traceback (most recent call last): ... AttributeError: 'NoneType' object has no attribute 'dist' After: [--:--:--] START Resolving pipeline [00:00:00] FAILURE Resolving pipeline Error loading pipeline: Pip package buildstream-external does not contain a plugin named 'nonexistant-plugin'
* .gitlab-ci.yml: Support test execution in other repositoriesJürg Billeter2018-02-131-2/+5
| | | | | | | Do not make any assumptions about the working directory as that varies across repositories. Fixes #251
* _frontend/cli.py: Check for invalid --track-except or --track-savePhillip Smyth2018-02-131-0/+8
| | | | | | | Report error when --track-except or --track-save is specified without --track or --track-all. Fixes #181
* _frontend/cli.py: Improve closing of non-existing workspacePhillip Smyth2018-02-131-0/+5
| | | | Fixes #182
* source.py: fix comment typoAngelos Evripiotis2018-02-131-1/+1
|
* doc: consistent source staging dir docAngelos Evripiotis2018-02-133-0/+9
| | | | | All source plugins respond to the 'directory' key, so add the doc for it to those that didn't have it.
* source.py: Fix typo in workspace warningJürg Billeter2018-02-131-1/+1
| | | | Fixes #250
* scheduler.py: Do not prematurely terminate loop after skipping jobsjuerg/schedulerJürg Billeter2018-02-121-1/+7
| | | | | | | | It is possible that Queue.process_ready() skips jobs without starting a job. Pull the skipped jobs forward through the queues and process them instead of prematurely terminating the loop due to lack of active jobs. Fixes #236
* .gitlab-ci.yml: Fix INTEGRATION_CACHEJürg Billeter2018-02-121-5/+2
| | | | | The working directory of the runner is /builds/BuildStream/buildstream, not /builds/BuildStream.
* test: Add tests that bzr workspace initialization acts as expectedJonathan Maw2018-02-121-0/+19
|
* bzr: Implement init_workspaceJonathan Maw2018-02-121-0/+15
| | | | | This will checkout a branch from the cache, and switch its parent branch to the url and ref that the source came from
* utils.py: Wrap calls to os.path.realpath() in an LRU cacheSam Thursfield2018-02-121-9/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | The os.path.realpath() function is expensive and we call it many times, to the point that os.path.realpath() calls make up around 40% of the total time spent in Element.stage_artifact(). The cleanest way to fix this is with a `functools.lru_cache()` wrapper that caches recently used values. None of the code in question can be removed (as the tests added in the previous commit will demonstrate). I tested this by running `bst shell base/base-system.bst true` in the GNOME modulesets project. o Without this patch there are 240,019 calls os.path.realpath() o With this patch there are 10,379 calls to os.path.realpath() o If we increase the cache size to 128 items, there are 10,359 calls to os.path.realpath(). o If we reduce the cache size to 32 items, there are 10,426 calls. o In all cases the number of *unique* calls is 10,327. This fixes issue #174.
* Add integration tests for edge cases involving symlinks and overlapsSam Thursfield2018-02-125-0/+152
|
* projectconf.rst: clarify 'type option'options_typeAngelos Evripiotis2018-02-111-2/+2
| | | | | | | Reword 'Flags type options' slightly for more clarity. We use 'option type' to refer to these thingies elsewhere, adjust 'arch' to be the same way.
* Refactor CLI importscompletion-optimizationsTristan Maat2018-02-085-700/+721
| | | | | This reduces the number of unconditional imports, mainly to speed up bash completion.
* Add a 'path' config option to junction elementsSam Thursfield2018-02-082-1/+16
| | | | | | | | | | | | | | | | | | | This makes it possible to depend on a project which is in a subdirectory of a Git repository. The error message given when the expected project.conf file is not found has also been improved. Previously the error would look like this: Error loading pipeline: Could not find file at /home/sam/.cache/buildstream/build/freedesktop-sdk-junction-rvmn17s2/project.conf This is giving the path of an internal temporary directory where the foreign project's source is checked out. The new error gives more information: Error loading pipeline: Could not find the project.conf file for junction element at freedesktop-sdk-junction.bst [line 1 column 0]. Expecting a project at path '.' within git source at freedesktop-sdk-junction.bst [line 4 column 2]