summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* cascache: add_object, assert path if link_directlyaevri/cascache_nitsAngelos Evripiotis2019-06-271-0/+3
|
* cascache: refactor, rm some unused exception varsAngelos Evripiotis2019-06-271-2/+2
|
* Merge branch 'bschubert/remove-useless-sanitize' into 'master'bst-marge-bot2019-06-2512-24/+23
|\ | | | | | | | | _yaml: Remove useless calls to '_yaml.node_sanitize' See merge request BuildStream/buildstream!1412
| * _yaml: Remove useless calls to '_yaml.node_sanitize'Benjamin Schubert2019-06-2512-24/+23
|/ | | | | | Calling '_yaml.dump' will itself call '_yaml.node_sanitize', therefore we can remove all calls to it in places where we directly after call dump.
* Merge branch 'juerg/fetch-subprojects' into 'master'bst-marge-bot2019-06-259-122/+98
|\ | | | | | | | | Improve subproject fetching See merge request BuildStream/buildstream!1414
| * _stream.py: Use scheduler to fetch subprojectsJürg Billeter2019-06-251-2/+8
| |
| * Handle subproject fetching in the Stream classJürg Billeter2019-06-254-8/+23
| |
| * app.py: Create stream before loading projectJürg Billeter2019-06-252-26/+39
| | | | | | | | | | | | This is in preparation for moving subproject fetching to the Stream class. Project loading may require subproject fetching as part of processing include directives.
| * Always fetch subprojects as neededJürg Billeter2019-06-258-90/+32
|/ | | | | | Treat junction element sources the same as sources of any other element and always fetch subprojects as needed. Do not ask the user to manually fetch subprojects.
* Merge branch 'jennis/notify_reverse_deps' into 'master'bst-marge-bot2019-06-242-24/+93
|\ | | | | | | | | Notify direct reverse dependencies when an Element becomes ready for runtime and cached See merge request BuildStream/buildstream!1406
| * element.py: Notify reverse deps when ready for runtime and cachedjennis/notify_reverse_depsJames Ennis2019-06-242-23/+86
| | | | | | | | | | | | | | | | | | | | | | | | An Element becomes ready for runtime and cached when: 1. It has a strong cache key 2. It is cached 3. Its runtime dependencies are ready for runtime and cached (this ensures the runtimes of runtimes are also cached). This patch introduces the method attempt_to_notify_reverse_dependencies which will notify all direct reverse dependencies of an Element once said Element becomes ready for runtime and cached.
| * element.py: cache key should be set in __update_cache_keysJames Ennis2019-06-241-1/+6
| |
| * element.py: Set can_query_cache_callback to None when calledJames Ennis2019-06-241-0/+1
|/ | | | | Each other callback callsite immediately sets the callback to None once invoked. We should be consistent.
* Merge branch 'raoul/1044-blobs-on-demand' into 'master'bst-marge-bot2019-06-248-27/+229
|\ | | | | | | | | | | | | CLI options for on demand blob fetching Closes #1044 See merge request BuildStream/buildstream!1400
| * tests: Add case for using pull and use buildtreesRaoul Hidalgo Charman2019-06-241-0/+29
| | | | | | | | Part of #1044
| * tests: Add shell test with pull optionRaoul Hidalgo Charman2019-06-241-0/+50
| | | | | | | | Part of #1044
| * _stream/cli: Add pull option to shellRaoul Hidalgo Charman2019-06-242-15/+30
| | | | | | | | | | | | This will fetch artifacts if they're not local or are incomplete. Part of #1044
| * tests: Add tests to buildcheckout for pullingRaoul Hidalgo Charman2019-06-241-1/+63
| | | | | | | | | | | | | | | | One test to check that artifacts are pulled and a checkout is successful when the --pull option is specified. Another to check that it attempts to pull but fails gracefully if the artifact isn't present. Part of #1044
| * cli.py/_stream.py: Add pull option to checkoutRaoul Hidalgo Charman2019-06-242-6/+24
| | | | | | | | | | | | | | This will set up a pull queue and pull relevant artifacts if not present or incomplete. Part of #1044
| * tests/partial: Add test to cover find_missing_blobsRaoul Hidalgo Charman2019-06-211-0/+25
| | | | | | | | Add test that ensures method is actually covered now and it works.
| * _artifactcache.py: fix find_missing_blobs methodRaoul Hidalgo Charman2019-06-212-5/+8
|/ | | | | | | This never will have worked because digests aren't hashable, and this method wasn't being covered in any tests. Also changed `remote_missing_blobs` to ensure the blobs list is an iterator and updated its docs.
* .gitlab-ci.yml: Use up-to-date bst-plugins-experimentalJavier Jardón2019-06-201-1/+1
| | | | Fixes #1046
* Merge branch 'tpollard/shellbuildtree' into 'master'bst-marge-bot2019-06-193-17/+29
|\ | | | | | | | | _frontend/cli.py: Tweak 'try' & 'always' bst shell buildtree handling See merge request BuildStream/buildstream!1392
| * _frontend/cli.py: Tweak 'try' & 'always' bst shell buildtree handlingTom Pollard2019-06-193-17/+29
|/ | | | | | | | | | | If the cached Artifact wasn't originally generated with a buildtree, then there's no need to attempt to find it in remotes by entering Stream(). In interactive mode we only present the user the option to attempt the pull with the above assumption on buildtree_exists, so apply the same to applicable non interactive choices. This also includes some tweaks in integration/shellbuildtrees.py to reflect changes and cover cases that were missing.
* Merge branch 'juerg/grpc' into 'master'Jürg Billeter2019-06-192-0/+6
|\ | | | | | | | | Guard against gRPC channels in the main process See merge request BuildStream/buildstream!1405
| * _sandboxremote.py: Guard against gRPC channels in the main processjuerg/grpcJürg Billeter2019-06-191-0/+3
| |
| * casremote.py: Guard against gRPC channels in the main processJürg Billeter2019-06-191-0/+3
|/
* Merge branch 'aevri/smallerjobs' into 'master'bst-marge-bot2019-06-191-6/+6
|\ | | | | | | | | jobs/job: send ChildJob the context, not scheduler See merge request BuildStream/buildstream!1407
| * jobs/job: send ChildJob the context, not schedulerAngelos Evripiotis2019-06-191-6/+6
|/ | | | | | | Instead of passing the whole scheduler to the ChildJob, only pass the part that is used - the context. Reducing the amount of shared state makes it easier to follow what's going on, and will make it more economical to move to away from the 'fork' model later.
* Merge branch 'aevri/pylint_argsdiff' into 'master'bst-marge-bot2019-06-1822-87/+132
|\ | | | | | | | | enable PyLint "arguments-differ" error, and fixup complaints See merge request BuildStream/buildstream!1390
| * _fuse/hardlinks: don't return result of os.unlinkAngelos Evripiotis2019-06-181-1/+1
| | | | | | | | | | | | Don't return, instead of returning 'the result' of os.unlink - it also doesn't return anything (and neither does the semantically identical os.remove).
| * pylint: enable arguments-differAngelos Evripiotis2019-06-181-3/+0
| |
| * tests/frontend: fix or ignore pylint args-differAngelos Evripiotis2019-06-183-3/+3
| | | | | | | | | | Ignore the warning for `_init_project_interactive`, as we really don't care what the arguments are.
| * _options: match methods to base classAngelos Evripiotis2019-06-185-5/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update OptionEnum and OptionFlags to match the Option base-class. Introduce 'load_special' to allow explicit usage of extended functionality, which wouldn't necessarily be supported by different subclasses. By making the signatures of overridden methods deliberately match, we can use PyLint to ensure the signatures don't accidentally differ. Also replace `super(cls, self)` with the equivalent and less error-prone `super()`, wherever we've touched it. In separate work we may want to replace this across the project. We must introduce __init__() functions for OptionEnum and OptionFlags to get a clean pass from PyLint. It seems that it's unable to see that load_special() is also transitively called from __init__().
| * testing/_utils: match git methods to base classAngelos Evripiotis2019-06-184-9/+15
| | | | | | | | | | | | | | | | | | | | | | | | Update 'source_config' of Git and _SimpleGit to match the Repo base class. This ensures that they are fully substitutable Repos. Introduce 'source_config_extra' to allow explicit usage of extended functionality, which wouldn't necessarily be supported by a different subclass. By making the signatures deliberately match, we can use PyLint to ensure the signatures don't accidentally differ.
| * CliIntegration: match methods to base classAngelos Evripiotis2019-06-183-6/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update 'run' to match the base class. This ensures that it is a fully substitutable sub-class. Introduce 'run_project_config' to allow explicit usage of extended functionality, which wouldn't necessarily be supported by a different subclass. By making the signatures deliberately match, we can use PyLint to ensure the signatures don't accidentally differ. Since we're already relying on `kwargs['project']` succeeding, enforce that run_project_config() only takes keyword arguments.
| * _filebaseddirectory: match methods to base classAngelos Evripiotis2019-06-181-10/+10
| | | | | | | | | | | | | | | | | | Update import_single_file and export_to_tar to match the Directory base class. This ensures that FileBasedDirectory is a fully substitutable Directory. By making the signatures deliberately match, we can use PyLint to ensure the signatures don't accidentally differ.
| * _casbaseddirectory: match methods to base classAngelos Evripiotis2019-06-181-13/+28
| | | | | | | | | | | | | | | | | | Update import_single_file and list_relative_paths to match the Directory base class. This ensures that CasBasedDirectory is a fully substitutable Directory. By making the signatures deliberately match, we can use PyLint to ensure the signatures don't accidentally differ.
| * _fuse: match SafeHardlinkOps to base classAngelos Evripiotis2019-06-181-7/+7
| | | | | | | | | | | | | | Update signatures of SafeHardlinkOps methods to match the fuse.Operations base class. This helps ensure there's no confusion about what the arguments are for, and that SafeHardlinkOps can actually be substituted for any other Operations subclass.
| * _fuse/fuse: make create() match SafeHardlinkOpsAngelos Evripiotis2019-06-181-17/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Make the base-class fuse.Operations class match the subclass SafeHardlinkOps. Remove the complication of worrying about raw_fi mode - we already rely on it being inactive. Remove ambiguity from FUSE.create() - we only support the non-raw_fi case. In separate work, we may want to remove raw_fi completely. Update Operations.create() to use a 'flags' param explicitly, rather than the more confusing dual-meaning of the 'fi' param.
| * _fuse: converge args for link() and symlink()Angelos Evripiotis2019-06-181-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Get consistency between the Operations base class and the SafeHardlinkOps subclass. Use the 'target, source' convention as it appears in Operations. This avoids the confusion of 'target' changing meaning in the 'name, target' convention introduced by SafeHardlinkOps. The naming is also consistent with 'man ln', and more consistent with Python's equivalent functions. Unfortunately FUSE.link() and Operations.link() swap the order of 'target' and 'source', but the meaning is the same.
| * _gitsourcebase: ignore or fix arguments-differ errAngelos Evripiotis2019-06-181-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | PyLint helpfully points out that _GitMirror.fetch() does not match SourceFetcher.fetch() - it is missing the *kwargs argument. The same for _GitSourceBase.track(). Move towards being able to enable the 'arguments-differ' error faster by side-stepping the class design debate and ignoring the error for now. Change _GitSourceBase.set_ref() argument names to match Source.set_ref().
| * _frontend/widget: match LogFile.render to base clsAngelos Evripiotis2019-06-181-3/+5
|/ | | | | | | | | Make the signature of LogFile.render() match Widget.render() by making a new LogFile.render_abbrev() method with the extra functionality. Fix up the single call site to use that instead. This means that we won't get linter warnings that helpful inform us that Widgets are not really substitutable.
* Merge branch 'aevri/psutil_affinity' into 'master'bst-marge-bot2019-06-181-1/+3
|\ | | | | | | | | _platform.get_cpu_count: use psutil instead of os See merge request BuildStream/buildstream!1394
| * _platform.get_cpu_count: use psutil instead of osaevri/psutil_affinityAngelos Evripiotis2019-06-181-1/+3
|/ | | | | | | | Be Windows-friendly by using the portable `psutil.Process().cpu_affinity()` instead of `os.sched_getaffinity`. Note that unfortunately this still doesn't work on Mac, so we keep that platform-specific override.
* Merge branch 'jennis/fix_typo' into 'master'bst-marge-bot2019-06-171-1/+1
|\ | | | | | | | | tests/frontend/pull.py: Fix typo See merge request BuildStream/buildstream!1404
| * tests/frontend/pull.py: Fix typojennis/fix_typoJames Ennis2019-06-171-1/+1
|/
* Merge branch 'raoul/unique-error-domains' into 'master'bst-marge-bot2019-06-142-3/+4
|\ | | | | | | | | Make ErrorDomain a unique enumeration See merge request BuildStream/buildstream!1399
| * _exceptions.py: Make ErrorDomain uniqueraoul/unique-error-domainsRaoul Hidalgo Charman2019-06-132-3/+4
|/ | | | | | | | For some reason PROG_NOT_FOUND and APP were both under 12 which could be confusing. Changed PROG_NOT_FOUND to 16 and added a unique decorator to prevent this from happening again. Updated shellbuildtrees test to check correct error domain.
* Merge branch 'tpollard/elapsedtime' into 'master'bst-marge-bot2019-06-133-7/+4
|\ | | | | | | | | _scheduler/scheduler.py: Remove unused elapsed_time() calls See merge request BuildStream/buildstream!1398
View Lorry Controller Status