summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/workspaces: Test edge case when linking files to RO directoryshashwatdalal/docker-source-patchShahwat Dalal2019-09-052-4/+18
| | | | | | | When we open a workspace, we are linking the sources to the directory specified. As such testing that we are able to open a workspace on a read-only directory should show that we are able to link files to a RO directory when we are staging complex sources (eg. Docker source).
* tests/tar: Add test for unreadable and unwritable content tar sourceShahwat Dalal2019-09-052-2/+4
| | | | | This test was added to consider the edge case when a unreadable file is being added to CAS.
* sources/tar.py: Ensure tar source files are readableShahwat Dalal2019-09-051-2/+19
| | | | | | When non-readable files were being added to CAS, `add_object` was failing. Thus all sourced files must be readable if they are to be cached. The tar source has been modified to ensure this precondition.
* utils.py: Ensure dest copy directory is writableShahwat Dalal2019-09-051-1/+11
| | | | | | | | | | | | | In the case we are copying something to a read-only directory, `_copy_directories` fails. Copying should work regardless. Now when we are copying something to a read-only directory, we reset the permissions of the directory to writable, and then revert to the directory's original permissions after copying is completed. An example where this is nessesary is when extracting Docker layers. If one layer adds a read-only directory `/a` and the following layers adds `/a/foo`, we must be able to copy `foo` into the otherwise read-only directory, whilst still ensuring `/a` is read-only once the layer is staged.
* Merge branch 'jennis/update_push_pull' into 'master'bst-marge-bot2019-09-056-19/+161
|\ | | | | | | | | Update artifact push and pull so that they can handle artifact refs See merge request BuildStream/buildstream!1568
| * cli.py: Allow pull to handle artifact refsJames Ennis2019-09-052-5/+47
| | | | | | | | | | This patch adds support for the handling of artifact refs in bst artifact pull. A test for this has also been added.
| * _project.py: Don't only load cached artifactsJames Ennis2019-09-053-8/+5
| | | | | | | | | | | | | | | | Previously, before loading artifact refs (ArtifactElements) we were checking that they were cached before loading them. This obviously becomes a problem when we want to try and pull an artifact. This patch removes the check for cached artifacts as we should still be able to contruct an ArtifactElement regardless of whether it is cached or not.
| * cli.py: Allow push to handle artifact refsJames Ennis2019-09-053-6/+109
|/ | | | | | | | This patch extends support for bst artifact push so that is now able to handle artifact refs. If --deps all is selected and a ref is given, BuildStream will error.
* Merge branch 'juerg/cache-usage' into 'master'bst-marge-bot2019-09-056-23/+291
|\ | | | | | | | | Display cache usage in status bar See merge request BuildStream/buildstream!1564
| * _frontend/status.py: Readd cache usage to status barJürg Billeter2019-09-051-2/+33
| | | | | | | | | | This was removed with the switch to buildbox-casd. Readd it now that cache usage tracking is supported again.
| * cascache.py: Add cache usage monitoring via buildbox-casdJürg Billeter2019-09-051-0/+118
| | | | | | | | | | This spawns a subprocess to periodically retrieve current cache usage information from buildbox-casd via gRPC.
| * cascache.py: Remove unused CACHE_SIZE_FILEJürg Billeter2019-09-051-3/+0
| |
| * _protos: Update local_cas.protoJürg Billeter2019-09-053-3/+125
| |
| * .gitlab-ci.yml: Update docker images for buildbox-casdJürg Billeter2019-09-051-17/+17
|/ | | | | The updated version of buildbox-casd includes support for disk usage monitoring.
* Merge branch 'jennis/tasks' into 'master'bst-marge-bot2019-09-044-10/+30
|\ | | | | | | | | Improve long-running task reporting See merge request BuildStream/buildstream!1573
| * _pipeline.py: Add task progress to check_remote()jennis/tasksJames Ennis2019-09-041-1/+5
| |
| * _pipeline.py: Add task progress to resolve_elements()James Ennis2019-09-041-1/+8
| | | | | | | | | | | | | | Resolving the initial cached state of Elements could potentially take a while, especially for large projects or local caches which are on a latent filesystem. We should report progress.
| * _loader.py: Cache how many Elements we loadJames Ennis2019-09-042-5/+7
| |
| * _messenger.py: Don't display subtask information before 3 secondsJames Ennis2019-09-041-3/+10
|/ | | | | | | Prior to this patch, subtask information was always displayed, regardless of how long it took. Now, I've implemented a _DISPLAY_LIMIT of 3 seconds. If a task takes longer than 3 seconds, the subtask information will be printed.
* Merge branch 'traveltissues/1118' into 'master'bst-marge-bot2019-09-041-2/+2
|\ | | | | | | | | | | | | Fix typo in node attribute Closes #1118 See merge request BuildStream/buildstream!1582
| * Fix typo in node attributeDarius Makovsky2019-09-041-2/+2
|/ | | | closes #1118
* Merge branch 'juerg/synthetic-file-index' into 'master'bst-marge-bot2019-09-044-16/+33
|\ | | | | | | | | node.pyx: _SYNTHETIC_FILE_INDEX must not be module-private See merge request BuildStream/buildstream!1576
| * tests: Add autouse fixture to reset global state in node.pyxjuerg/synthetic-file-indexJürg Billeter2019-09-043-3/+8
| |
| * node.pyx: Add _reset_global_state() methodJürg Billeter2019-09-041-0/+12
| | | | | | | | | | This will be used by the test suite to improve isolation between tests running in the same process.
| * node.pyx: _SYNTHETIC_FILE_INDEX must not be module-privateJürg Billeter2019-09-041-13/+13
|/ | | | | | | | | | | | | | | | | | | | It's declared in node.pxd and used by _yaml.pyx. This fixes the following error when running individual tests: src/buildstream/testing/runcli.py:563: in run_project_config base_config = _yaml.load_data(config) src/buildstream/_yaml.pyx:293: in buildstream._yaml.load_data cpdef MappingNode load_data(str data, int file_index=node._SYNTHETIC_FILE_INDEX, str file_name=None, bint copy_tree=False): src/buildstream/_yaml.pyx:324: in buildstream._yaml.load_data node._set_root_node_for_file(file_index, contents) > f_info = <__FileInfo> __FILE_LIST[file_index] E IndexError: list index out of range src/buildstream/node.pyx:1550: IndexError Fixes: 97b8ab7d ("node: Mark module-private functions and classes...")
* Merge branch 'juerg/await-threads' into 'master'bst-marge-bot2019-09-041-3/+15
|\ | | | | | | | | utils.py: Wait for threads to exit in _is_single_threaded() See merge request BuildStream/buildstream!1583
| * utils.py: Wait for threads to exit in _is_single_threaded()Jürg Billeter2019-09-041-3/+15
|/ | | | gRPC threads are not joined when shut down.
* Merge branch 'juerg/http-test-server' into 'master'bst-marge-bot2019-09-031-10/+1
|\ | | | | | | | | tests/testutils/http_server.py: Drop queue to avoid lingering thread See merge request BuildStream/buildstream!1581
| * tests/testutils/http_server.py: Drop queue to avoid lingering threadjuerg/http-test-serverJürg Billeter2019-09-031-10/+1
|/ | | | | | | | | | Some CI jobs still sporadically encounter a thread that wasn't cleaned up in tests that use the HTTP server, despite calling Queue.close() and Queue.join_thread(). As a simple SIGTERM signal should suffice to properly terminate the HTTP server child process, this simply removes the queue and also the extra thread for serving requests.
* Merge branch 'jjardon/badge' into 'master'bst-marge-bot2019-09-031-1/+1
|\ | | | | | | | | README.rst: Point badge to bst-1 branch See merge request BuildStream/buildstream!1580
| * README.rst: Point badge to latest bst 1.x releasejjardon/badgeJavier Jardón2019-09-031-1/+1
|/
* Merge branch 'juerg/fork' into 'master'bst-marge-bot2019-09-0316-150/+268
|\ | | | | | | | | Replace safeguard for fork with multiple threads See merge request BuildStream/buildstream!1577
| * tests: Catch tests that don't shut down background threadsjuerg/forkJürg Billeter2019-09-033-0/+49
| |
| * testing/_forked.py: Update _MAIN_PID for tests running in subprocessesJürg Billeter2019-09-031-0/+6
| | | | | | | | | | This reduces the difference between regular execution and the test environment.
| * cascache.py: Remove fork_disabled mechanismJürg Billeter2019-09-031-12/+0
| | | | | | | | The fork safeguard is now handled by Context.is_fork_allowed().
| * casserver.py: Remove notify_fork_disabled() callJürg Billeter2019-09-031-4/+0
| | | | | | | | The fork safeguard is now handled by Context.is_fork_allowed().
| * _context.py: Remove unused disable_fork() methodJürg Billeter2019-09-031-12/+0
| |
| * _stream.py: Remove disable_fork() callsJürg Billeter2019-09-031-5/+0
| | | | | | | | The fork safeguard is now handled by Context.is_fork_allowed().
| * scheduler.py: Use Context.is_fork_allowed()Jürg Billeter2019-09-031-1/+1
| |
| * _context.py: Add is_fork_allowed() methodJürg Billeter2019-09-031-0/+17
| |
| * _basecache.py: Add has_open_grpc_channels() methodJürg Billeter2019-09-031-0/+12
| |
| * cascache.py: Add has_open_grpc_channels() methodJürg Billeter2019-09-031-0/+8
| |
| * utils.py: Add _is_single_threaded() methodJürg Billeter2019-09-031-0/+20
| | | | | | | | | | This will be used to safeguard against fork issues with multiple threads.
| * tests/testutils/http_server.py: Close queue to stop background threadJürg Billeter2019-09-031-0/+2
| |
| * tests/artifactcache: Use context manager for gRPC channelsJürg Billeter2019-09-031-50/+50
| |
| * _sandboxremote.py: Use context manager for CASRemote instancesJürg Billeter2019-09-031-43/+45
| |
| * _sandboxremote.py: Use context manager for gRPC channelsJürg Billeter2019-09-031-28/+30
| |
| * _context.py: Release artifactcache and sourcecache resourcesJürg Billeter2019-09-031-0/+6
| |
| * _basecache.py: Add release_resources() methodJürg Billeter2019-09-031-0/+10
| |
| * casremote.py: Add close() method to close gRPC channelJürg Billeter2019-09-031-0/+12
| | | | | | | | Also support use as context manager.
View Lorry Controller Status