summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* .gitlab-ci.yml: Use recent toxtristan/optional-coverageTristan Van Berkom2019-02-281-2/+2
| | | | | | | There appears to be multiple versions of tox in the WSL test environment, prepending /root/.local/bin to the PATH while running tests ensures that we are using a recent enough tox to execute the tests.
* .gitlab-ci.yml: Adding tox --version output to test diagnosticsTristan Van Berkom2019-02-281-0/+2
| | | | | This is generally useful to know, since the tox version is not enforced by BuildStream.
* tox.ini: Support running tests without coverage present at allTristan Van Berkom2019-02-281-7/+13
| | | | | | | | | | | This change simply makes it possible to run the tests without coverage and without any coverage related dependencies present in the tox venvs. To run tests without coverage, simply choose the python version and append "-nocover" to the env name, e.g.: tox -e py36-nocover
* setup.cfg, tox.ini: Moved coverage support to the tox.ini levelTristan Van Berkom2019-02-282-2/+2
| | | | | | | This disables any attempt to use coverage when running tests directly against installed dependencies with setup.py. This fixes #916
* tox.ini / requirements: Separated coverage requirementsTristan Van Berkom2019-02-285-14/+27
| | | | Since coverage is optional, lets make it really optional.
* Merge branch 'juerg/directory-import' into 'master'Jürg Billeter2019-02-272-134/+175
|\ | | | | | | | | Directory.import_files() improvements See merge request BuildStream/buildstream!1183
| * _filebaseddirectory.py: Support importing files from CASjuerg/directory-importJürg Billeter2019-02-271-12/+82
| |
| * _filebaseddirectory.py: Add _get_filetype() methodJürg Billeter2019-02-271-0/+16
| |
| * _casbaseddirectory.py: Replace file list with filter callbackJürg Billeter2019-02-271-106/+76
| |
| * _casbaseddirectory.py: Add result parameter to import methodsJürg Billeter2019-02-271-21/+19
| | | | | | | | This eliminates the need to combine subdir results.
| * _casbaseddirectory.py: Remove docstring for import_files()Jürg Billeter2019-02-271-14/+1
|/ | | | | The docstring in the superclass should be used as reference. This matches FileBasedDirectory.import_files().
* Merge branch 'aevri/macos' into 'master'bst-marge-bot2019-02-271-2/+2
|\ | | | | | | | | _platform/darwin: fix missing space, say MacOS See merge request BuildStream/buildstream!1191
| * _platform/darwin: fix missing space, say MacOSAngelos Evripiotis2019-02-271-2/+2
|/ | | | | Note that 'OSX' is now called 'MacOS', as of 2016: https://en.wikipedia.org/wiki/MacOS
* Merge branch 'danielsilverstone-ct/gc-play' into 'master'bst-marge-bot2019-02-273-0/+36
|\ | | | | | | | | _stream.py, _project.py: Manage GC during pipeline load See merge request BuildStream/buildstream!1164
| * _stream.py, _project.py: Manage GC during pipeline loadDaniel Silverstone2019-02-272-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | Since during pipeline load we are generating a lot of objects which need to live for the duration of the BuildStream instance there is benefit to disabling GC for the duration of the load since that drastically reduces wasted CPU cycles iterating data which will remain around always. In order to limit the increase in the peak memory consumption though, we do an explicit gc.collect() after loading the YAML in, since without that, we use 60% more memory at peak, and with it, only 20%. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.UK.
| * utils.py: Add a _with_gc_disabled() decoratorDaniel Silverstone2019-02-271-0/+31
|/ | | | | | | | | | This decorator can be used to wrapper any function to disable the GC for the duration of the function. At the end it will be re-enabled. This is not recursive, so only use this decorator sparingly and with care. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'juerg/import-filter' into 'master'Jürg Billeter2019-02-267-86/+144
|\ | | | | | | | | Replace file lists with filter callback for file import See merge request BuildStream/buildstream!1182
| * utils.py: Remove unused filelist parameter from _process_list()Jürg Billeter2019-02-261-8/+5
| |
| * utils.py: Remove unused presorted parameter from _process_list()Jürg Billeter2019-02-261-13/+3
| |
| * utils.py: Remove unused files parameter from copy_files and link_filesJürg Billeter2019-02-261-12/+6
| |
| * storage: Remove files parameter from Directory.import_filesJürg Billeter2019-02-263-18/+10
| |
| * compose.py: Replace file list with filter callbackJürg Billeter2019-02-261-1/+4
| |
| * local.py: Remove unnecessary files argument from copy_files() callJürg Billeter2019-02-261-1/+1
| |
| * element.py: Replace file lists with filter callbacks in stage_artifact()Jürg Billeter2019-02-261-8/+17
| |
| * element.py: Add __split_filter_func() methodJürg Billeter2019-02-261-11/+36
| |
| * element.py: Extract __split_filter() from __compute_splits()Jürg Billeter2019-02-261-21/+40
| |
| * storage: Add filter_callback parameter to Directory.import_filesJürg Billeter2019-02-263-0/+12
| |
| * utils.py: Add filter_callback parameter to copy_files and link_filesJürg Billeter2019-02-261-4/+21
|/
* Merge branch 'juerg/directory' into 'master'bst-marge-bot2019-02-266-180/+152
|\ | | | | | | | | Virtual directory improvements See merge request BuildStream/buildstream!1181
| * _casbaseddirectory.py: Use CASCache.checkout() in export_files()Jürg Billeter2019-02-261-35/+2
| | | | | | | | This eliminates code duplication.
| * cascache.py: Add can_link parameter to checkout()Jürg Billeter2019-02-261-4/+8
| | | | | | | | | | This allows CasBasedDirectory.export_files() to use CASCache.checkout(), eliminating code duplication.
| * cascache.py: Make _checkout() publicJürg Billeter2019-02-261-29/+37
| | | | | | | | | | This allows CasBasedDirectory.export_files() to use CASCache.checkout(), eliminating code duplication.
| * _casbaseddirectory.py: Use lazy instantiation for subdirectoriesJürg Billeter2019-02-261-11/+20
| | | | | | | | | | This avoids creation of directory objects for the whole tree, if only a part of the tree is used.
| * _casbaseddirectory.py: Replace isinstance() with _FileType checksJürg Billeter2019-02-261-28/+25
| | | | | | | | | | Using isinstance() with concrete classes should typically be avoided, and it's also more expensive than necessary.
| * _casbaseddirectory.py: Add type field to IndexEntry classJürg Billeter2019-02-261-9/+12
| |
| * storage/directory.py: Add _FileType enumJürg Billeter2019-02-261-0/+25
| |
| * _casbaseddirectory.py: Use unordered dict for the indexJürg Billeter2019-02-261-3/+1
| | | | | | | | | | There is no reason for using an ordered dict for the index. This improves descend() performance by about 20%.
| * _casbaseddirectory.py: _add_directory(): Drop unused code pathJürg Billeter2019-02-261-10/+3
| | | | | | | | | | The code path in _add_directory() where the entry already exists was unused. Drop it and add assertion.
| * _casbaseddirectory.py: Populate files_written in CAS-to-CAS importJürg Billeter2019-02-261-0/+1
| |
| * _filebaseddirectory.py: Drop indexJürg Billeter2019-02-261-45/+16
| | | | | | | | | | It's used very sparingly. The overhead of populating the index is likely higher than the benefit of using it.
| * storage: Rename update_utimes parameter to update_mtimeJürg Billeter2019-02-264-10/+10
| | | | | | | | utime is short for 'update time' and only the mtime is significant.
| * tests/internals/storage_vdir_import.py: Use public API in testJürg Billeter2019-02-261-5/+1
|/
* Merge branch 'juerg/unlink' into 'master'Jürg Billeter2019-02-261-9/+13
|\ | | | | | | | | utils.py: safe_link(): Unlink only if target already exists See merge request BuildStream/buildstream!1185
| * utils.py: safe_link(): Unlink only if target already existsjuerg/unlinkJürg Billeter2019-02-261-9/+13
|/ | | | | | | Unlike shutil.copyfile(), os.link() will not modify the target file if it exists already. This improves staging performance by about 10% in a simple test.
* Merge branch 'chandan/junction-dependency-format' into 'master'bst-marge-bot2019-02-257-40/+233
|\ | | | | | | | | | | | | loader: Allow dependencies to use ":" to refer to junctioned elements Closes #809 See merge request BuildStream/buildstream!998
| * Document how to refer to cross-junction dependencies inlinechandan/junction-dependency-formatChandan Singh2019-02-262-0/+37
| | | | | | | | | | | | In the previous commit, we added support to express cross-junction dependencies inline as simple strings. Document it along with the version in which the feature was added.
| * NEWS: Cross-junction dependencies can now be specified inlineChandan Singh2019-02-261-0/+3
| |
| * loader: Allow dependencies to use ":" to refer to junctioned elementsChandan Singh2019-02-264-40/+193
|/ | | | | | | | | | | | | | | | | | | | | | | | | | This will allow cross-junction dependencies to be listed as strings on a single line. As part of this, some logic around initializing `Dependency()` objects have been moved out of `_extract_depends_from_node()` method into the constructor of `Dependency()` class, to keep all related code in one place. * _loader/types.py: While initializing `Dependency` objects, attempt to split filenames, only if no `junction` was specified explicitly. If a `junction` was specified, then filenames with `:` in their names will result in an error. * _loader/loadelement.py: Refactor logic to initialize `Dependency()` objects to move it to the `Dependency()` constructor. * tests/frontend/buildcheckout.py: Add tests to ensure the above. * _versions.py: Bump BST_FORMAT_VERSION. Fixes #809.
* Merge branch 'bschubert/allow-no-cov' into 'master'bst-marge-bot2019-02-251-2/+2
|\ | | | | | | | | Make code coverage reporting optional See merge request BuildStream/buildstream!1179
| * Make code coverage reporting optionalbschubert/allow-no-covBenjamin Schubert2019-02-251-2/+2
|/ | | | | | Previously, without a coverage file, the tests would fail if combined with "--no-cov". This makes the coverage reporting optional and allows the usage of "--no-cov"