summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/artifactcache: Add pull unit-testsjmac/remote_execution_clientMartin Blanchard2018-09-071-0/+320
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* tests/artifactcache: Add push unit-testsMartin Blanchard2018-09-072-0/+330
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* tests/artifactcache: Add a simple test projectMartin Blanchard2018-09-077-0/+48
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* qmake.py: Mark this as a BST_VIRTUAL_DIRECTORY‧pluginMartin Blanchard2018-09-071-1/+2
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* meson.py: Mark this as a BST_VIRTUAL_DIRECTORY‧pluginMartin Blanchard2018-09-071-1/+2
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* make.py: Mark this as a BST_VIRTUAL_DIRECTORY‧pluginMartin Blanchard2018-09-071-1/+2
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* cmake.py: Mark this as a BST_VIRTUAL_DIRECTORY pluginMartin Blanchard2018-09-071-1/+2
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* autotools.py: Mark this as a BST_VIRTUAL_DIRECTORY pluginJim MacArthur2018-09-071-1/+2
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* element.py: Switch to SandboxRemote if config option is setJim MacArthur2018-09-071-1/+27
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* element.py: Get the updated virtual directory after runningJim MacArthur2018-09-071-0/+2
| | | | | | | | | Executing run() on a sandbox can now replace the virtual directory, since remote execution returns a potentially different directory rather than an update to the existing one. Call get_virtual_directory() again after running to accout for this. https://gitlab.com/BuildStream/buildstream/issues/454
* sandbox/__init__.py: Add SandboxRemoteJim MacArthur2018-09-071-0/+1
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* _sandboxremote.py: Implement the REAPI clientJim MacArthur2018-09-071-0/+226
| | | | | | | | The remote execution client is implemented as a remote sandbox that sends sources and build commands to a REAPI server and fetches results once remotely executed. New file. https://gitlab.com/BuildStream/buildstream/issues/454
* format_project.rst: Document remote-execution optionJim MacArthur2018-09-071-0/+18
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* Add "remote-execution" project configuration optionJim MacArthur2018-09-073-1/+16
| | | | | | This just adds one option, "remote-execution/url". Affects multiple files. https://gitlab.com/BuildStream/buildstream/issues/454
* sandbox.py: Allow setting the virtual directoryJim MacArthur2018-09-072-3/+23
| | | | | | | | This is for use after remote execution has finished, since remote execution produces a new output directory rather than modifying the initial directory. https://gitlab.com/BuildStream/buildstream/issues/454
* _casbaseddirectory.py: Add a method for hash recalculationMartin Blanchard2018-09-071-0/+9
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* cascache.py: Introduce new pull helpersMartin Blanchard2018-09-071-0/+51
| | | | | | Add a pull_tree() helper. https://gitlab.com/BuildStream/buildstream/issues/454
* cascache.py: Introduce new push helpersMartin Blanchard2018-09-071-0/+59
| | | | | | | Add push_directory() and push_message() helpers along with a verify_digest_pushed(). https://gitlab.com/BuildStream/buildstream/issues/454
* cascache.py: Preparation for remote executionJim MacArthur2018-09-071-90/+109
| | | | | | | Refactor the push() and pull() implementations so that API additions needed for remote-execution is made easier. https://gitlab.com/BuildStream/buildstream/issues/454
* Merge branch 'danielsilverstone-ct/maybe-reduce-fd-leaks' into 'master'Tristan Maat2018-09-041-1/+6
|\ | | | | | | | | jobs.py: Reduce FD leaks from queues and process objects See merge request BuildStream/buildstream!778
| * jobs.py: Reduce FD leaks from queues and process objectsdanielsilverstone-ct/maybe-reduce-fd-leaksDaniel Silverstone2018-09-041-1/+6
|/ | | | | | | | | | | | The garbage collector can take too long to get around to cleaning up the Queue and Process instances in completed Job instances. As such, FDs tend to leak and in very large projects this can result in running out of FDs before a build, fetch, track, or other process can complete. This patch reduces the chance of that by only creating the queue when it's needed, and forcing the queue and process instances to be deleted when the parent is finished with them. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'benbrewer/install-artifacts-doc-improvements' into 'master'Javier Jardón2018-09-041-5/+8
|\ | | | | | | | | Improve documentation for artifact cache installation See merge request BuildStream/buildstream!777
| * Improve documentation for artifact cache installationBen Brewer2018-09-041-5/+8
|/ | | | | | | Remove ambiguity about systemd service files being separate. Add URL for more information about systemd service files. Add note about public keys being mandatory for self-signed certs. Make cert/key file naming consistent throughout document.
* Merge branch 'tpollard/483' into 'master'Tom Pollard2018-09-032-4/+109
|\ | | | | | | | | plugins/git.py: Warn if ref is not in given track See merge request BuildStream/buildstream!564
| * tests/sources/git.py: Add tests for REF_NOT_IN_TRACKtpollard/483Tom Pollard2018-09-031-0/+68
| | | | | | | | | | Add tests that cover assert_ref_in_track & the configurable CoreWarnings REF_NOT_IN_TRACK warnings token.
| * plugins/git.py: Warn if ref is not in given trackTom Pollard2018-09-031-4/+41
|/ | | | | | | | | | | Add a helper function assert_ref_in_track to git.py GitMirror() which is used when staging & initing the source workspace. It checks the element's ref exists in the track (branch/tag) if it has been specified, raising a warning if necessary. The warning makes use of the warning token 'REF_NOT_IN_TRACK' from the configurable CoreWarnings. If the element has been tracked with bst, it is assumed that the value of ref exists in the track as it was generated from it & as such is not asserted.
* Merge branch 'tristan/source-fetcher-changes' into 'master'Tristan Van Berkom2018-09-035-32/+149
|\ | | | | | | | | Source fetcher changes See merge request BuildStream/buildstream!772
| * source.py: Track marked URLs and assert they are marked during ↵tristan/source-fetcher-changesTristan Van Berkom2018-09-021-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Plugin.configure() This cannot test for unaliased URLs, as those can be discovered later on outside of user provided element configuration; at least we assert that if an alias was used, we have seen it at load time. This will cause a BUG to occur for a plugin which marks an aliased URL (or attempts to translate one) outside of `Plugin.configure()`, if that URL was not previously seen. This is a part of #620
| * source.py: Added `primary` argument to URL marking APIsTristan Van Berkom2018-09-023-19/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | The Source must now mention whether the marked or translated URL is "primary" or not. Even when a Source may have multiple URLs, the auxilliary URLs are derived from the primary one, not only is this true for git, but it is mandated by our tracking API which assumes there is a primary URL. This adjusts the `git` source and the test `fetch_source.py` source to behave properly and advertize it's primary URL properly. This is a part of #620
| * plugin.py: Added _configure() and _get_configuring() private APIsTristan Van Berkom2018-09-023-2/+29
| | | | | | | | | | | | | | | | | | Keeps track of whether the plugin is currently being configured. Adjusted Element and Source classes to call _configure() in place of calling configure() directly. This is a part of #620
| * source.py: Documenting that marking download URLs is mandatoryTristan Van Berkom2018-09-021-6/+34
| | | | | | | | | | | | | | A download URL must be interpreted by the core at `Plugin.configure()` time, even if only employed later on. This is a part of #620
| * source.py: Fixing docs link formatting to be consistent.Tristan Van Berkom2018-09-021-8/+10
| |
| * source.py: Document Source.get_source_fetchers() to return an iterableTristan Van Berkom2018-09-021-2/+9
|/ | | | | | | | Also highlight the fact that the plugin can rely on the fetcher's fetch() method getting called before consuming the next item in the list, which is the magick behavior that the git plugin relies on. This is a part of #620
* Merge branch 'Qinusty/cache-size-directory' into 'master'Tristan Van Berkom2018-09-011-1/+1
|\ | | | | | | | | Move cache_size.pid.log files into a subdirectory of logs See merge request BuildStream/buildstream!769
| * scheduler.py: Move cache_size logs into folderQinusty/cache-size-directoryJosh Smith2018-08-311-1/+1
|/ | | | | This prevents the cache_size.pid.log files from cluttering the root log directory.
* Merge branch 'Qinusty/retries-should-fail' into 'master'Tristan Van Berkom2018-08-314-10/+6
|\ | | | | | | | | Retries log as failures See merge request BuildStream/buildstream!766
| * tests: Modify tests to ensure retry FAILUREJosh Smith2018-08-313-3/+3
| |
| * job.py: Modify retry messages to be FAILJosh Smith2018-08-311-7/+3
|/ | | | | | | | This adjusts the message handler for the child processes to no longer override the message type. This also removes the ability for unhandled non BstError exceptions to retry.
* _project.py: Adding FIXME comment to address #591 properlyTristan Van Berkom2018-08-311-1/+7
|
* Merge branch 'tpollard/591' into 'master'Tiago Gomes2018-08-301-0/+5
|\ | | | | | | | | buildstream/_project.py: Report if project.conf is missing name See merge request BuildStream/buildstream!680
| * buildstream/_project.py: Report if project.conf is missing nameTom Pollard2018-08-301-0/+5
|/ | | | | | | Explicitly check that project.conf contains a name. This resolves the issue of the provenance check from _yaml.py incorrectly reporting the offending file as the default_config_node projectconfig.yaml when no name key exists in the pre_config_node dict.
* Merge branch 'bzr_fix' into 'master'Phillip Smyth2018-08-301-5/+7
|\ | | | | | | | | Replacing string 'bzr' with value from host tools See merge request BuildStream/buildstream!764
| * Replacing string 'bzr' with value from host toolsbzr_fixknownexus2018-08-301-5/+7
|/
* Merge branch 'tristan/atomic-cache-size-file' into 'master'Tristan Van Berkom2018-08-301-1/+1
|\ | | | | | | | | _artifactcache/artifactcache.py: Write the cache_size file atomically See merge request BuildStream/buildstream!762
| * _artifactcache/artifactcache.py: Write the cache_size file atomicallytristan/atomic-cache-size-fileTristan Van Berkom2018-08-301-1/+1
|/ | | | | | | | This is causing issues while the size file is being read and written simultaneously. The proper fix will be to read/add/save the file atomically and that will require locking, but this fix is a good stop gap to existing crashes.
* Merge branch 'valentindavid/post_tracking_errors' into 'master'Tristan Van Berkom2018-08-302-12/+37
|\ | | | | | | | | | | | | Report processing errors from tracking Closes #533 See merge request BuildStream/buildstream!747
| * Report processing errors from trackingValentin David2018-08-302-12/+37
|/ | | | | | Failures to write files when tracking were not reported. Fixes #533.
* Merge branch 'tristan/source-mirroring-changes' into 'master'Tristan Van Berkom2018-08-302-17/+31
|\ | | | | | | | | Minor code changes revolving around source mirroring See merge request BuildStream/buildstream!758
| * plugins/source/git.py: Fix formatting of url in trackingTristan Van Berkom2018-08-301-1/+3
| | | | | | | | | | This was displaying the aliased URL which is pretty useless, use the translated URL for the timed activity.
| * source.py: Move Source.mark_download_url() to the public methods.Tristan Van Berkom2018-08-301-13/+13
| | | | | | | | | | | | This was sitting in the section for abstract methods, but this is most definitely not an abstract method to be implemented by Sources.
View Lorry Controller Status