summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* cli.py: Change artifact checkout to pull implicitlyjennis/pull_implicitly_in_checkoutJames Ennis2019-09-113-15/+8
| | | | This makes it consistent with source checkout
* _stream.py: Load the appropriate PipelineSelection in checkoutJames Ennis2019-09-112-13/+10
| | | | | | | This patch ensures checkout behaves like the rest of our commands which support --deps options. That is, we carry the "deps" string through to the Stream and load the corresponding PipelineSelection.
* Merge branch 'juerg/cache-usage' into 'master'bst-marge-bot2019-09-102-2/+36
|\ | | | | | | | | | | | | cascache.py: Fix cache usage monitor on Python older than 3.7 Closes #1131 See merge request BuildStream/buildstream!1597
| * tests/artifactcache/expiry.py: Add test for cache usage monitorjuerg/cache-usageJürg Billeter2019-09-101-0/+33
| |
| * cascache.py: Fix cache usage monitor on Python older than 3.7Jürg Billeter2019-09-101-2/+3
|/ | | | | | | Older Python versions don't support the typecode 'q', see https://bugs.python.org/issue19896 Fixes #1131.
* Merge branch 'traveltissues/none-childdata' into 'master'bst-marge-bot2019-09-101-3/+4
|\ | | | | | | | | Check result is not None in tracking queue done() See merge request BuildStream/buildstream!1595
| * Check result is not None in tracking queue done()Darius Makovsky2019-09-101-3/+4
|/ | | | | | The tracking queue is the only queue implementing `done()` which requires the result to be iterable. If there is no result from the job this will hit a TypeError.
* Merge branch 'tpollard/sandboxmessage' into 'master'bst-marge-bot2019-09-101-2/+2
|\ | | | | | | | | _message.py: Use bool for sandbox Message() parameter See merge request BuildStream/buildstream!1592
| * _message.py: Use bool for sandbox Message() parameterTom Pollard2019-09-101-2/+2
|/ | | | | The sandbox member of BstError is a bool, which is then used as an arg for the message.
* Merge branch 'traveltissues/typo' into 'master'bst-marge-bot2019-09-101-1/+1
|\ | | | | | | | | Fix typo in pipeline msg detail See merge request BuildStream/buildstream!1594
| * Fix typo in pipeline msg detailDarius Makovsky2019-09-101-1/+1
|/
* Merge branch 'tpollard/notificationhandler' into 'master'bst-marge-bot2019-09-107-79/+234
|\ | | | | | | | | Stream - Scheduler notification handler See merge request BuildStream/buildstream!1550
| * scheduler.py: Notification for Message() propagationTom Pollard2019-09-104-7/+22
| | | | | | | | | | | | | | Add a notification for MESSAGE. Instead of scheduler's Queues and Jobs directly calling the message handler that App has assigned to Context, the Message() is now sent over the notification handler where it is then given to Messenger's handler.
| * scheduler.py: Notification for interactive failure retryTom Pollard2019-09-103-18/+21
| | | | | | | | | | | | | | | | | | Add a notifcation for RETRY. This moves the retry handling into scheduler, which will be running in the process which has been suspended for interactivity and as such will be able to load the relevant Element. Note a failed job via the scheduler should never not have a related queue, so the try except when matching the queue via the action name should not be needed.
| * scheduler.py: Notification for sched suspended statusTom Pollard2019-09-102-1/+9
| | | | | | | | | | | | Add a notification for SUSPENDED. Stored as a bool in Stream which is flipped when the scheduler notifies that the event loop has changed state.
| * scheduler.py: Notifications for sched suspendingTom Pollard2019-09-102-7/+19
| | | | | | | | | | | | Add a notification for SUSPEND & UNSUSPEND. This splits the jobs_suspended context manager into two methods, which are triggered by suspend() context manager yielding in Stream().
| * scheduler.py: Notification for sched terminated statusTom Pollard2019-09-102-1/+9
| | | | | | | | | | | | Add a notification for TERMINATED, which is notified as True in Stream when the scheduler terminates the event loop via user interactivity or SIG handling.
| * scheduler.py: Notification for sched loop running statusTom Pollard2019-09-102-3/+13
| | | | | | | | | | | | Add a notification for RUNNING, which is stored as a bool in Stream which is flipped when the scheduler notifies that the event loop has changed state.
| * scheduler.py: Move elapsed_time() to shared State()Tom Pollard2019-09-103-22/+32
| | | | | | | | | | | | | | | | The starttime relative to the scheduler, used to track deltas for time spent not suspended across Stream--Scheduler is now synced via a notification. This removes the need to call into the scheduler from the 'frontend' to determine the time difference for scheduler relative queries.
| * scheduler.py: Introduce a notification queue for stream interactionTom Pollard2019-09-103-28/+117
|/ | | | | | | | | | | | | | For the 'backend' of BuildStream to run in a subprocess certain interactions between the 'frontend' need to be augmented over a queue between the processes, as well as adaptions for the differences in state between them. This commit implements the core of these interactions as notification objects over a deque with notifying callbacks on either side, which should be switched to a multiprocessing queue once there's process separation at which point 'both ends' can execute an event loop. This also removes the checks for the ticker & interrupt callbacks to be optional, as currently they're always implemented.
* Merge branch 'juerg/casd' into 'master'bst-marge-bot2019-09-104-120/+138
|\ | | | | | | | | | | | | casremote.py: Limit request size for batch download and upload Closes #1129 See merge request BuildStream/buildstream!1591
| * Add tests/frontend/large_directory.pyJürg Billeter2019-09-102-0/+90
| |
| * casremote.py: Limit request size for batch download and uploadJürg Billeter2019-09-101-28/+47
| | | | | | | | Fixes #1129.
| * cascache.py: Remove unused pull() and push() methodsJürg Billeter2019-09-091-92/+1
|/ | | | These methods used the old ReferenceStorage service.
* Merge branch 'jennis/update_python_deps' into 'master'bst-marge-bot2019-09-093-29/+38
|\ | | | | | | | | Update all Python dependencies See merge request BuildStream/buildstream!1587
| * Update all python dependenciesJames Ennis2019-09-093-29/+38
|/ | | | | | | | | When running the testsuite, we see a DeprecationWarning for one of our dependencies, pluginbase and a UserWarning for /usr/lib64/python3.x/distutils/dist.py:274. This patch updates all Python dependencies, which as well as updating things, stops these warnings.
* Merge branch 'coldtom/fix-status-job-render' into 'master'bst-marge-bot2019-09-091-1/+1
|\ | | | | | | | | _frontend/status.py: Fix non-formatted terminator See merge request BuildStream/buildstream!1589
| * _frontend/status.py: Fix non-formatted terminatorcoldtom/fix-status-job-renderThomas Coldrick2019-09-091-1/+1
|/ | | | | | | | Currently we don't format the terminator to the displayed widget for a StatusJob, which means that we have blue brackets apart from the terminating one, which is white. This is a trivial patch to fix this.
* Merge branch 'traveltissues/1110' into 'master'bst-marge-bot2019-09-092-45/+43
|\ | | | | | | | | | | | | use show_default Closes #1110 See merge request BuildStream/buildstream!1588
| * cli.py: remove superfluous kwargs for flagsDarius Makovsky2019-09-091-38/+38
| | | | | | | | cleanup help msgs
| * casserver.py: Use show_default for optionsDarius Makovsky2019-09-091-7/+5
|/ | | | is_flag is always False by default (you can't specify an absent flag)
* Merge branch 'tlater/cache-endpoints' into 'master'bst-marge-bot2019-09-0625-563/+1329
|\ | | | | | | | | | | | | Support separate end points for artifact caches Closes #1041 See merge request BuildStream/buildstream!1540
| * Change unfitting function nameTristan Maat2019-09-0610-20/+20
| |
| * Update documentation to explain split cachesTristan Maat2019-09-062-0/+59
| |
| * Allow splitting artifact cachesTristan Maat2019-09-0614-492/+986
| | | | | | | | | | | | | | This is now split into storage/index remotes, where the former is expected to be a CASRemote and the latter a BuildStream-specific remote with the extensions required to store BuildStream artifact protos.
| * _basecache.py: Refactor the BaseCache a littleTristan Maat2019-09-061-30/+67
| | | | | | | | | | This is in preparation for split artifact caches. The code is hairy as-is, let's not make it worse :)
| * Remove cache-specifc spec classesTristan Maat2019-09-068-97/+32
| | | | | | | | This was almost entirely just historical code duplication.
| * Create BaseRemote base classTristan Maat2019-09-061-0/+241
| |
| * _basecache.py: Remove references to 'artifacts'Tristan Maat2019-09-061-3/+3
|/
* Merge branch 'jennis/pylint_no_else_raise' into 'master'bst-marge-bot2019-09-0614-60/+63
|\ | | | | | | | | Address pylint 2.3.x's no-else-raise warning See merge request BuildStream/buildstream!1586
| * Address no-else-raise pylint warningJames Ennis2019-09-0614-60/+63
|/ | | | | | | | | | | In the latest released version of pylint (2.3.x), the "no-else-raise" (R1720) warning has been introduced. See: http://pylint.pycqa.org/en/latest/whatsnew/changelog.html#what-s-new-in-pylint-2-3-0 There are many instances of this in our codebase, which this patch addresses.
* Merge branch 'jennis/update_checkout' into 'master'bst-marge-bot2019-09-054-39/+81
|\ | | | | | | | | Update artifact checkout to handle artifact refs See merge request BuildStream/buildstream!1565
| * buildcheckout.py: Add tests which checkout artifact depsjennis/update_checkoutJames Ennis2019-09-051-0/+37
| | | | | | | | | | | | These tests ensure that we can checkout the build dependencies of an artifact and that if we try to checkout the runtime dependencies the command will fail.
| * cli.py: Allow checkout to handle artifact refsJames Ennis2019-09-053-9/+13
| |
| * cli.py: Clean up dependency scope handling in checkoutJames Ennis2019-09-051-8/+2
| |
| * _stream.py: Remove separate handling of ArtifactElementsJames Ennis2019-09-053-22/+29
|/ | | | | | | | | | | ArtifactElement inherits Element, both have an Artifact object as a member, thus we should not need to handle these separately. This change has resulted in introducing a configure_sandbox() method in ArtifactElement. The method is similar to BuildElement.configure_sandbox() but does not configure the sandbox to actually be used for building.
* Merge branch 'shashwatdalal/docker-source-patch' into 'master'bst-marge-bot2019-09-055-6/+49
|\ | | | | | | | | Add default permissions when adding to CAS and when linking files See merge request BuildStream/buildstream!1458
| * tests/workspaces: Test edge case when linking files to RO directoryShahwat Dalal2019-09-051-0/+14
| | | | | | | | | | | | | | When we open a workspace, we are linking the sources to the directory specified. As such testing that we are able to open a workspace on a read-only directory should show that we are able to link files to a RO directory when we are staging complex sources (eg. Docker source).
| * tests/tar: Add test for unreadable and unwritable content tar sourceShahwat Dalal2019-09-052-3/+5
| | | | | | | | | | This test was added to consider the edge case when a unreadable file is being added to CAS.
| * sources/tar.py: Ensure tar source files are readableShahwat Dalal2019-09-051-2/+19
| | | | | | | | | | | | When non-readable files were being added to CAS, `add_object` was failing. Thus all sourced files must be readable if they are to be cached. The tar source has been modified to ensure this precondition.
View Lorry Controller Status