summaryrefslogtreecommitdiff
path: root/without-test-modules
Commit message (Collapse)AuthorAgeFilesLines
* Remove non-existing files from without-test-modulesTiago Gomes2015-12-151-6/+0
| | | | Change-Id: I0399dbfe3f9f71ca9895fef435bb604edb224928
* Remove tests for branch-and-merge pluginTiago Gomes2015-11-241-1/+0
| | | | Change-Id: Ie66b543eb67282c08a2651062727a4583057ee2d
* Add support for Baserock definitions version 7Sam Thursfield2015-10-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | This adds a new 'Defaults' class to represent definitions defaults The Python 'jsonschema' module is used to validate the contents of the Defaults file. This module is already included in Baserock 'build' and 'devel' reference systems by way of the 'openstack-common' stratum. This commit embeds a copy of the JSON-Schema schema for the DEFAULTS file. I think the canonical location of this schema should be in the reference definitions.git, for now. In future, the schemas should maybe have their own repos. Either way, Morph should embed a copy for the time being so that we are sure the schema matches how Morph expects to parse the file. Morph's automated tests are all updated to use definitions version 7. I removed most of the tests for built-in build systems, because the built-ins themselves are no longer part of Morph. Only the mechanism for defining them needs to be tested now. Change-Id: I65f8f1c967683ef605852bfae5c68518e53f9981
* Remove support for Baserock definitions format versions 3, 4 and 5Sam Thursfield2015-09-181-1/+1
| | | | Change-Id: Iad95af65bd5c528d2e72f5b2ffa80a01152f50ff
* Add DefinitionsRepo classSam Thursfield2015-06-161-0/+1
| | | | | | | | | | | | | | The intention is for this class to take over the from the Workspace and SystemBranch classes. It allows Morph to load and parse definitions from a Git repo, without requiring the user to run `morph checkout` or `morph branch`: it can operate from any normal Git repository. The class behaves differently when the Git repository is inside a Morph system-branch checkout made with `morph branch` or `morph checkout`, to avoid changing things under the feet of people who are used to those commands. Change-Id: I52a898efb9f6fb7f7e94c65b9ed38516bd51f49d
* morph: Add morph diff subcommandRichard Maw2015-05-111-0/+1
| | | | Change-Id: If74c97ccd81aa4d92ef247d2be59282f9552d4a1
* morphlib: Add utility module for parsing argv into lists of systemsRichard Maw2015-05-111-0/+1
| | | | | | | | | | | | | | The `morph anchor`, `morph build-morphology` and a potential `morph diff` command would all benefit from having a unified way to parse the argv for the systems it must operate on, especially in the case of the potential `morph diff`, which needs to be able to handle being given two sets of systems. `morph anchor` may make use of it now by passing the list of systems to the Source resolver, but `morph build-morphology` would have to iterate over the systems and graph each independently. Change-Id: I91ab4764ffca3aa16f144f89f68f37cc21b9f643
* Add `morph anchor` commandRichard Maw2015-05-011-0/+1
| | | | Change-Id: If9d92d7c75b9c4276b69c482c076c6fc1d4ccbbf
* morphlib: Add command to get build-log for a given chunk and system fileLauren Perry2015-04-301-0/+1
| | | | Change-Id: I09e9b17ef2e0fb94dbf5a96dca91062d64433add
* Add a CSV manifest generator.Adam Coldrick2015-04-301-0/+1
| | | | | | | | | | | | | | | | This adds a `morph generate-manifest-csv` command which creates a CSV manifest for a system containing license information, upstream URL and a guess at the version of each component in a system. The existing `morph generate-manifest` is also renamed to `morph generate-manifest-genivi` to improve the distinction between the two commands which generate a manifest. The basis of this commit was done by Michael Drake <michael.drake@codethink.co.uk>. Change-Id: I712de11eaa6e1564c5bae5f5f396e5e77484972f
* Add a `morph get-repo` command to clone repositories from the cacheAdam Coldrick2015-04-271-0/+1
| | | | Change-Id: I0c175649bd98067ea69d471ebe6880aac5a25598
* distbuild: Move SubprocessEventSource into its own moduleSam Thursfield2015-04-211-0/+1
| | | | | | | | | Previously it was only available in the distbuild-helper program. Moving it to its own module means we can test it and reuse it. This commit also adds a docstring to the class. Change-Id: Iaf7854048cf0ff463a87894f1f500cdcb6a34d8b
* Add `morph certify` plugin to check for build reproducibility.Michael Drake2015-04-011-0/+1
| | | | | | | | | | | Currently does three checks: 1. Checks that all chunks in given system(s) have sha1 refs. 2. Checks that all sha1 refs are anchored. Unanchored refs can be removed on `git gc`. 3. Checks that all chunk repos are on the trove-host. Change-Id: Iaf105b1614a45616684e68a08f28b8529d4321fa
* Add plugin to get details of given chunk name.Michael Drake2015-03-301-0/+1
| | | | | | | | | | | | | | | | | | | | | Usefully, this provides the resolved repo URL, which may be aliased in the actual defintions. This makes it easier to use a workflow that avoids `morph edit`. Resolved repo URLs can be git cloned. It also provides other info about the chunk. In the case of multiple strata containing the same chunk name, the tool lists them all, saying which stratum they come from. Example output: # morph get-chunk-details libpng In stratum graphics-common: Chunk: libpng Repo: git://git.baserock.org/delta/libpng Ref: 259fb7761d747655c607efcec7a12ff1f3c24561 Change-Id: I6b540e38d9521f3f473b51c9031e508ddcb458ee
* Add tests for sourceresolverAdam Coldrick2015-02-181-4/+0
| | | | | | | | | This only adds tests for the bits which were moved from morphologyfactory into sourceresolver, namely detection of build systems and the '_get_morphology()' function. These are just the morphologyfactory tests reworked slightly to work properly with the modified API.
* Move create_source_pool code into new 'sourceresolver' moduleSam Thursfield2014-11-101-0/+4
| | | | | | | | This code is an essential part of 'morph build'. It's quite complex and really shouldn't be mixed in with the base Application class. Given a dedicated class we can store some state in the object and avoid functions with seven parameters, too.
* Fix and integrate distbuild unit testsRichard Maw2014-10-011-0/+19
|
* Remove update-gits commandRichard Maw2014-07-221-1/+0
| | | | | | | | | This hasn't been used in a long time to my knowledge, its API completely misses the point, and its implementation relies on old APIs that need to change. Until we discover we need it, and work out what it should look like, I think the best thing to do is get rid of it.
* Exorcise some old and unused commandsRichard Maw2014-07-071-1/+0
| | | | | | | | | | | | | `morph merge` only worked for a small subset of cases, and has been left to bit-rot, since we don't use it. `morph tag` is just a `git tag` when we have petrified definitions repository. We don't use it, nor do we need it, so it can go away rather than take up valuable development time fixing it when requirements change. `old-foo` have all been superceded by newer versions and are no-longer used.
* Rewrite copy_artifacts plugin as list_artifactsSam Thursfield2014-06-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | The copy-artifacts and list-artifacts commands are mainly useful for making releases. As part of the release process we copy artifacts for the entire build graph of the release to the artifact cache on trove.baserock.org, to provide Baserock users with ready-built artifacts. This part of the release process is now automated, and the automation require the list-artifacts command to function as a 'plumbing' command. The copy-artifacts command is no longer required. It can be replaced with: morph list-artifacts --quiet REPO REF MORPH | rsync --files-from=- $TARGET The previous version of this plugin looked in the system artifact's metadata for the list of artifacts. This is flawed as the final system does not necessarily contain every build dependency. The new version of the plugin calculates the build graph from source, using the same process as the 'buildcommand' module. It also required looking in Morph's artifact cache for the system artifact file to analyse.
* Add utilities for listing and finding extensions.Mark Doffman2014-03-311-0/+1
| | | | | | | | | Add a module to morphlib that can list all write and configuration extensions either in morph itself or the morphology repository. The module also contains methods to find an extension filename from the name and type.
* Merge branch 'baserock/richardipsum/merge_distbuild'Richard Ipsum2014-03-271-0/+1
|\ | | | | | | | | | | | | | | | | | | Conflicts: morphlib/plugins/deploy_plugin.py without-test-modules Reviewed by: Richard Maw Lars Wirzenius
| * Don't test distbuild pluginRichard Ipsum2014-03-211-0/+1
| |
* | Implement morph push and morph pulladamcoldrick/ingest-binaries-v5-rebaseAdam Coldrick2014-03-201-0/+2
|/ | | | | | | | | | | Add a plugin to implement both `morph push` and `morph pull`. These commands are wrappers around the corresponding git commands push and pull, which also implement the functionality of pushing and pulling large files provided by git-fat. For example, running `morph pull` will pull any commits from the remote branch not on your local branch, and then pull any large files from the separate git-fat/rsync store on the Trove.
* Revert "Add utilities for listing and finding extensions."Mark Doffman2014-02-211-1/+0
| | | | This reverts commit ab0a83a09a93ca33aa402d9c4d3b916a48a1a882.
* Add utilities for listing and finding extensions.Mark Doffman2014-02-211-0/+1
|
* Add split rules to sourcesRichard Maw2014-01-161-0/+1
| | | | | | | | | This introduces a new artifactsplitrule module, which tries to provide a nice abstraction over matching a sequence of things to a bunch of outputs, to be used by both chunks splitting, for separating files out into chunk artifacts, the stratum splitting, where chunks are aggregated into stratum artifacts, and systems selecting the right strata to go into the artifact.
* morphlib: Add BuildBranch abstractionRichard Maw2013-11-291-0/+2
| | | | | | | This is an abstraction on top of SystemBranchDirectories, providing the ability to add uncommitted changes to the temporary build branch, push temporary build branches and retrieve the correct repository URI and ref to build the system.
* Merge branch 'baserock/richardmaw/S8563/bootstrap-rootfs-protection'Richard Maw2013-08-231-1/+0
|\ | | | | | | Reviewed-by: Lars Wirzenius
| * fsutils: add invert_paths functionRichard Maw2013-08-151-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will list all the paths generated by the walker generator function that aren't in the specified set. It removes directories from those returned by the walker, since with os.walk(topdown=True) this culls the search space. In the set of provided paths and the set of returned paths, if a directory is given, then its contents are virtually part of the set. This oddly specific behaviour is because invert_paths is to be used with linux-user-chroot to mark subtrees as read-only, when it only has a set of paths it wants to keep writable. It takes a walker, rather than being given a path and using os.walk, so that it is a pure function, so is easier to unit test.
* | Add "morph print-architecture" subcommandLars Wirzenius2013-08-151-0/+1
|/
* Re-implement "morph init" using the new Workspace classLars Wirzenius2013-07-311-0/+1
| | | | | | | Move "morph init" into a plugin, and remove it from the previously existing plugin. This keeps all the old, tricky code in the old plugin, and moves new, clean code into a new plugin. Eventually the old plugin can be removed, since it'll be empty.
* Add morph cross-bootstrapSam Thursfield2013-07-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cross-bootstrap is a way to build baserock on an architecture that does not currently have Baserock. It can be used by `morph cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact that can be used as a root filesystem with a basic build environment with a script named `native-bootstrap` which will build and install every chunk in the system. If done with a devel system, this will give you a suitable environment for building a proper Baserock system. This does not currently provide a kernel for the target architecture. Apart from adding the cross-bootstrap plugin, it also makes the following changes: * Moves the lit of valid_archs into morphlib (instead of locally-scoped in MorphologyFactory) * BuildCommand takes an extra argument, build_env * split BuildCommand's get_artifact_object into create_source_pool and resolve_artifacts (plus changes things that use get_artifact_object to use the new way) * setup_mounts finds out whether to do so by whether build_mode is 'staging', instead of by whether the setting 'staging-chroot' is true. * Makes ChunkBuilder's get_sources use the morphlib.builder2.extract_sources() method, and moved set_mtime_recursively into morphlib.builder2, since it's not currently used anywhere else. * moved ChunkBuilder's get_commands into the Morphology class (plus changes to anything that used get_commands)
* Remove "morph make-patch" since it doesn't workLars Wirzenius2013-07-051-1/+0
| | | | | | In addition, when we start using tbdiff for upgrades, all of this will need to be re-designed and re-implemented anyway. The current plugin is dead code. Note that the test suite had already been disabled.
* plugins: add morph gc subcommandRichard Maw2013-06-071-0/+1
| | | | | | | | | | This removes staging areas and extracted chunks from --tempdir. Then asks the local artifact cache what artifacts it has and how old they are, removing all sources older than --cachedir-artifact-delete-older-than, and may delete other sources that are younger than --cachedir-artifact-keep-younger-than if it still needs to make space.
* Add the trovectl plugin to the morph plugins directory.Tiago Gomes2013-05-231-0/+1
|
* Acknowledge the lack of test coverageJonathan Maw2013-03-141-0/+1
|
* Add image inspection pluginJannis Pohlmann2013-02-211-0/+1
| | | | | | | | | | | | This adds a `run-in-artifact` command which allows another command to be run in a system. There is also a `content-manifest` command which gives a manifest of the artifacts, which commits they were built from, and if possible, a version. This adds a morphlib.bins.call_in_artifact_directory() method to run a command inside an artifact and to generate a manifest.
* Add ExtractedTarball class and method to extract/mount an artifactJannis Pohlmann2013-02-211-0/+1
| | | | | | | | | | | | | ExtractedTarball is more or less the equivalent to MountableImage for artifacts that are not mountable images. So in order to inspect root file system tarballs, ExtractedTarball can be used, for disk images, MountableImage can be used. The morphlib.bins.call_in_artifact_directory() method combines these two classes and provides a way to extract/mount an artifact and call a callback with the temporary directory / mount point as its first argument. Using this, a plugin that runs a command relative to an artifact's root directory can be written easily.
* Move MountableImage class into morphlibJannis Pohlmann2013-02-211-0/+1
| | | | | | | This class would be pretty useful in other cases where a system image needs to be mounted and inspected. Also updates the Trebuchet plugin to use this class.
* Add morphlib module for common write extension codeLars Wirzenius2013-02-061-0/+1
|
* Add deployment pluginLars Wirzenius2013-02-061-1/+2
| | | | | | | | | | | This adds a new optional field to system morphologies: "configuration-extensions". The deployment plugin relies heavily on code from the branch and merge plugin. This needs to be eventually fixed by refactoring the codebase so that the shared code is in morphlib and not in plugins. However, doing that is beyond the scope of adding a deployment plugin.
* Make morph get its version from git.Richard Maw2013-01-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When morph is built, it writes various version information from git into morphlib. When morphlib is loaded it attempts to read this version information. if it cannot be found then it checks whether morphlib is being run from inside a git checkout, if it is then it reads the information that way. If it isn't in a git checkout then it raises an exception as builds made in such a fashion are not reproducible. The git version information retained is: 1. The output of git describe This is a relatively human-friendly way of knowing a version and gives a reasonably short output string. This will end with `-unreproducible` if there were uncommitted changes. 2. The commit sha1, so the exact part of Morph's history can be found 3. The tree sha1, so if the branch has been rebased rather than merged such that the commit is lost, you may still be able to find it, though it requires a git-wizard to check it out 4. The branch of morph, so that it's easier to see if the Further possible changes to increase reproducibility include: 1. Not allowing `python setup.py build` if there are uncommitted changes 2. Failing to run with uncommitted changes (recommended against since it will just annoy developers who are making changes to morph, and make them commit just to shut it up, then destroy the history later) Requiring an extra flag to build in this case may work better. 3. Reading the uncommitted changes into a tree object and including that would allow it to be recovered if the tree was later committed. 4. Checking whether the commit has been pushed upstream as well. Too annoying to work.
* Add plugin to without-test-modules listLars Wirzenius2012-11-071-0/+1
|
* Initial non-syslinux disk image kindDaniel Silverstone2012-08-141-0/+1
| | | | | | | This adds a disk system image kind which does everything the syslinux-disk one does, but without syslinux. It deliberately carries stubbed bootloader operations so that we can later make syslinux-disk inherit from this one and override it.
* Move cmd_build to a separate pluginRichard Maw2012-08-011-0/+1
| | | | | It's such a small amount of code, it's possibly not worth it, but now all commands are in plugins.
* Move BuildCommand from app into its own moduleRichard Maw2012-08-011-0/+1
| | | | | | | | morphlib.app should be for application bringup and providing command line options used by the library code. Any external plugins that refer to morphlib.app.BuildCommand will need fixing.
* Move init and minedir to branch_and_merge pluginRichard Maw2012-08-011-0/+1
| | | | | deduce_mine_directory is duplicated in the plugin and morphlib.app until all the commands are moved into the plugin.
* Move command make-patch to Trebuchet pluginRichard Maw2012-08-011-0/+1
| | | | | | | | The plugin is called trebuchet, so that if any other functionality is required from morph it has a place to go. make-patch generates a trebuchet patch, so it should go in the trebuchet plugin.
* morph: move update-gits to pluginRichard Maw2012-08-011-0/+1
| | | | This also publicizes cache_repo_and_submodules and traverse_morphs
View Lorry Controller Status