summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* casserver.py: Create artifact proto file atomicallyjuerg/atomic-artifactJürg Billeter2019-05-231-1/+2
|
* _artifactcache.py: Create artifact proto file atomicallyJürg Billeter2019-05-231-1/+1
|
* _artifact.py: Create artifact proto file atomicallyJürg Billeter2019-05-231-1/+1
|
* Merge branch 'jonathan/reset_cached_when_needed' into 'master'Jonathan Maw2019-05-232-14/+12
|\ | | | | | | | | | | | | element.py: Only reset Artifact's cachedness when it might change Closes #1028 See merge request BuildStream/buildstream!1352
| * element.py: Only reset Artifact's cachedness when it might changejonathan/reset_cached_when_neededJonathan Maw2019-05-222-14/+12
|/ | | | | | | | i.e. after calculating the original cached state (as soon as weak/strict cache keys are available), cachedness needs to be reset if the artifact may have appeared on the filesystem. We expect that to happen after _pull_done() or _assemble_done()
* Merge branch 'raoul/983-cas-source-staging' into 'master'bst-marge-bot2019-05-227-65/+126
|\ | | | | | | | | | | | | Staging sources directly into CAS Closes #983 See merge request BuildStream/buildstream!1340
| * utils.py: remove misleading documentationraoul/983-cas-source-stagingRaoul Hidalgo Charman2019-05-221-2/+2
| | | | | | | | | | copy_files and link_files suggested that a single file path is a valid argument when it isn't.
| * local.py: Support staging directly into CASRaoul Hidalgo Charman2019-05-221-33/+15
| | | | | | | | Part of #983
| * _casbaseddirectory: Add support for can_linkRaoul Hidalgo Charman2019-05-221-9/+17
| | | | | | | | | | | | | | Without this, there's not much benefit to using the virtual directories as we still copy files back into the CAS. Part of #983
| * Directory: add `import_single_file` methodRaoul Hidalgo Charman2019-05-223-0/+26
| | | | | | | | | | | | | | This a new method which deals with importing a single file. Implemented for both FileBasedDirectory and CasBasedDirectory. Part of #983
| * Add BST_STAGE_VIRTUAL_DIRECTORY optionRaoul Hidalgo Charman2019-05-222-21/+66
|/ | | | | | | | Add flag to indicate whether sources can stage directly to a virtual directory. Adds `__stage_previous_sources` method which stages previous sources taking into account whether they use virtual directories or not. Part of #983
* Merge branch 'aevri/mv_job_parent_above_child' into 'master'bst-marge-bot2019-05-221-131/+131
|\ | | | | | | | | _scheduler/jobs/job: mv _parent* above _child* See merge request BuildStream/buildstream!1349
| * _scheduler/jobs/job: mv _parent* above _child*Angelos Evripiotis2019-05-221-131/+131
|/ | | | | | | | Move the parent-specific methods above the child-specific methods. This makes slightly more sense chronologically, as the parent creates the child. It will also make diffs cleaner when splitting parent and child into separate classes.
* Merge branch 'aevri/fix_logging_regex_test' into 'master'Angelos Evripiotis2019-05-221-2/+13
|\ | | | | | | | | tests/frontend/logging.py: fix error message regex See merge request BuildStream/buildstream!1348
| * tests/frontend/logging.py: fix error message regexAngelos Evripiotis2019-05-221-2/+13
|/ | | | | | | | | | | This regex was not matching, and therefore the assert was not doing anything. Fix the regex to match BuildStream's current output. Add a check to make sure that we get exactly the number of expected matches. This means we get an obvious error if the regex stops matching again.
* Merge branch 'chandan/src-directory' into 'master'Chandan Singh2019-05-21250-51/+52
|\ | | | | | | | | | | | | Move source from 'buildstream' to 'src/buildstream' Closes #1009 See merge request BuildStream/buildstream!1322
| * CONTRIBUTING.rst: Update path to BuildStream source filesChandan Singh2019-05-211-2/+2
| |
| * .gitlab-ci.yml: Fix source path for analysis jobsChandan Singh2019-05-211-6/+6
| | | | | | | | | | Since the `buildstream` module now lives inside `src` directory, also update the paths that we give to `radon`.
| * .gitattributes: Update path to buildstream/_version.pyChandan Singh2019-05-211-1/+1
| | | | | | | | | | This file has been moved under `src` so update the `.gitattributes` entry accordingly.
| * .coveragerc: Fix coverage reporting after move to src directorychandan/src-directoryChandan Singh2019-05-211-3/+1
| | | | | | | | | | | | | | | | | | * `include` directive is unnecessary as all of `buildstream` is already included, so remove it. * Having `buildstream/buildstream` in `paths` confuses `coverage combine` as it tries to insert `src/buildstream` in the path twice in that case.
| * tests: Fix import order of buildstream and tests modulesChandan Singh2019-05-214-7/+9
| | | | | | | | | | | | | | | | Now that the `buildstream` module is inside `src` directory, `pylint` treats `buildstream` as a third party module inside `tests`. As such, it wants the imports from `buildstream` to be placed before imports from `tests`. Adopt this style to fix the lint errors, which is probably also good for readability.
| * Move source from 'buildstream' to 'src/buildstream'Chandan Singh2019-05-21243-33/+34
|/ | | | | | This was discussed in #1008. Fixes #1009.
* Merge branch 'chandan/coverage-doesnt-need-deps' into 'master'Chandan Singh2019-05-211-2/+1
|\ | | | | | | | | tox.ini: Coverage does not need module installed See merge request BuildStream/buildstream!1347
| * tox.ini: Coverage does not need module installedChandan Singh2019-05-211-2/+1
|/ | | | | Coverage does not need the module, or its dependencies, to be installed. Save time by not installing them extraneously.
* Merge branch 'jonathan/cached-to-artifact' into 'master'Jonathan Maw2019-05-212-32/+41
|\ | | | | | | | | | | | | Move Element.__*cached variable to Artifact class Closes #1015 See merge request BuildStream/buildstream!1325
| * Delegate storage of cached state to the Artifact classJonathan Maw2019-05-212-32/+41
|/ | | | | This commit also removes Element.__is_cached because it doesn't seem to serve a purpose.
* Merge branch 'danielsilverstone-ct/classmethod-inits' into 'master'Tom Pollard2019-05-203-62/+77
|\ | | | | | | | | Make various Element/Source init helpers @classmethod to clarify data flow See merge request BuildStream/buildstream!1346
| * {meta,}element.py: Move is_junction into MetaElementdanielsilverstone-ct/classmethod-initsDaniel Silverstone2019-05-172-10/+10
| | | | | | | | | | | | | | | | To reduce the `kind == "junction"` stuff all over Element's init helpers, move the `is_junction` logic into `MetaElement` to clarify things. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * source.py: Make initialisation more classyDaniel Silverstone2019-05-171-8/+9
| | | | | | | | | | | | | | | | Construction of the plugin defaults and the config construction don't actually need access to the source instance, so make them into classmethods ready for extraction into the loader. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Make sandbox configuration extraction classyDaniel Silverstone2019-05-171-9/+8
| | | | | | | | | | | | | | To clarify the flow of data and dependencies, make sandbox extraction classy. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Make config extraction classyDaniel Silverstone2019-05-171-2/+3
| | | | | | | | | | | | | | To clarify data flow, configuration extraction can be made to be a classmethod. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Make public data extraction classyDaniel Silverstone2019-05-171-6/+15
| | | | | | | | | | | | | | | | To clarify data flow, make the extraction of public data into a classmethod, and add an expand method to deal with the instance variable expansion. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Extract environment as classmethodDaniel Silverstone2019-05-171-10/+16
| | | | | | | | | | | | | | | | To clarify the data flow for extraction of environment variables, make the relevant functions into classmethods and add an expansion function which relies on the Variables instance. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Lift __extract_variables to be classmethodDaniel Silverstone2019-05-171-4/+4
| | | | | | | | | | | | | | Since __extract_variables only needs to be a class method, lift it to clarify what it depends on. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Remove self.__is_junctionDaniel Silverstone2019-05-171-13/+12
| | | | | | | | | | | | | | | | As a clarity-inducing change, remove the __is_junction property from elements since that is only used during initialisation and only by code which has access to the metaelement anyway. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * element.py: Plugin default initialisation as classmethodDaniel Silverstone2019-05-171-11/+11
|/ | | | | | | | Since plugins are initialised once per plugin kind+source combo, we can do that as a class method which renders the flow of information more transparently. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'raoul/440-fix' into 'master'Raoul Hidalgo Charman2019-05-172-5/+30
|\ | | | | | | | | Fix init_workspace for multiple sources See merge request BuildStream/buildstream!1345
| * element.py: fix init_workspace for multiple sourcesraoul/440-fixRaoul Hidalgo Charman2019-05-172-5/+30
|/ | | | | Noticed a mistake that got into !1214, so only the last source gets initialised in a workspace. Added a test for this.
* Merge branch 'shared/split_out_update_state_2' into 'master'bst-marge-bot2019-05-161-111/+171
|\ | | | | | | | | Split Element._update_state into cache key and artifact state methods See merge request BuildStream/buildstream!1342
| * Split Element._update_state into cache key and artifact state methodsshared/split_out_update_state_2Jonathan Maw2019-05-161-111/+171
|/ | | | | i.e. there are now __update_cache_keys(), __update_artifact_state() __reset_cache_data() and __update_cache_key_non_strict() methods.
* Merge branch 'raoul/974-Artifact-Rework' into 'master'bst-marge-bot2019-05-1524-701/+916
|\ | | | | | | | | | | | | Artifact as a Proto: rework Closes #974 See merge request BuildStream/buildstream!1292
| * element.py: Tweak pull_buildtrees logic in _pull_pending()raoul/974-Artifact-ReworkTom Pollard2019-05-151-1/+2
| | | | | | | | | | | | | | If an element is cached without a buildtree it not longer entails that it may be possible to pull a buildtree for it, if it wasn't cached with one at creation. Query _buildtree_exists() to reduce possible redudant pull attempts.
| * _cas/cascache.py: Remove contains_subdir_artifact()Tom Pollard2019-05-152-39/+0
| | | | | | | | This method is no longer used and as such can be removed.
| * Remove unused progress callbackRaoul Hidalgo Charman2019-05-154-19/+10
| |
| * element.py: Remove __get_cache_keys_for_commit()Tom Pollard2019-05-151-11/+0
| | | | | | | | This private method is no longer used and as such can be removed.
| * Remove excluded_subdir/subdir optionsRaoul Hidalgo Charman2019-05-153-56/+17
| | | | | | | | | | | | | | With artifact as a proto, it doesn't make sense to do it this way, bits of code can be removed. Part of #974
| * _artifact.py: Rework to use artifact protoRaoul Hidalgo Charman2019-05-1521-447/+723
| | | | | | | | | | | | | | | | This will replace the previous use of a directory structure. Quite a lot is changed here, predominantly _artifact and _artifactcache modules. Part of #974
| * element.py: Name normalisation & artifact path constructer helpersTom Pollard2019-05-1513-64/+79
| | | | | | | | | | | | By extracting the functionality from the Element() it allows the removal of code duplication for artifact assertion in ArtifactShare(), via a new get_artifact_name() method in Cli().
| * cascache: Make diff_trees publicRaoul Hidalgo Charman2019-05-151-54/+54
| | | | | | | | Part of #974
| * ArtifactServicer: Make GetArtifact update mtime blobsRaoul Hidalgo Charman2019-05-151-6/+17
| | | | | | | | Part of #974