| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
Change-Id: If74c97ccd81aa4d92ef247d2be59282f9552d4a1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `morph anchor`, `morph build-morphology` and a potential `morph
diff` command would all benefit from having a unified way to parse the
argv for the systems it must operate on, especially in the case of the
potential `morph diff`, which needs to be able to handle being given two
sets of systems.
`morph anchor` may make use of it now by passing the list of systems to
the Source resolver, but `morph build-morphology` would have to iterate
over the systems and graph each independently.
Change-Id: I91ab4764ffca3aa16f144f89f68f37cc21b9f643
|
|
|
|
| |
Change-Id: If9d92d7c75b9c4276b69c482c076c6fc1d4ccbbf
|
|
|
|
| |
Change-Id: I09e9b17ef2e0fb94dbf5a96dca91062d64433add
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a `morph generate-manifest-csv` command which creates
a CSV manifest for a system containing license information,
upstream URL and a guess at the version of each component in a
system.
The existing `morph generate-manifest` is also renamed to
`morph generate-manifest-genivi` to improve the distinction
between the two commands which generate a manifest.
The basis of this commit was done by
Michael Drake <michael.drake@codethink.co.uk>.
Change-Id: I712de11eaa6e1564c5bae5f5f396e5e77484972f
|
|
|
|
| |
Change-Id: I0c175649bd98067ea69d471ebe6880aac5a25598
|
|
|
|
|
|
|
|
|
| |
Previously it was only available in the distbuild-helper program. Moving
it to its own module means we can test it and reuse it.
This commit also adds a docstring to the class.
Change-Id: Iaf7854048cf0ff463a87894f1f500cdcb6a34d8b
|
|
|
|
|
|
|
|
|
|
|
| |
Currently does three checks:
1. Checks that all chunks in given system(s) have sha1 refs.
2. Checks that all sha1 refs are anchored.
Unanchored refs can be removed on `git gc`.
3. Checks that all chunk repos are on the trove-host.
Change-Id: Iaf105b1614a45616684e68a08f28b8529d4321fa
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Usefully, this provides the resolved repo URL, which may be
aliased in the actual defintions. This makes it easier to
use a workflow that avoids `morph edit`. Resolved repo URLs
can be git cloned.
It also provides other info about the chunk. In the case
of multiple strata containing the same chunk name, the tool
lists them all, saying which stratum they come from.
Example output:
# morph get-chunk-details libpng
In stratum graphics-common:
Chunk: libpng
Repo: git://git.baserock.org/delta/libpng
Ref: 259fb7761d747655c607efcec7a12ff1f3c24561
Change-Id: I6b540e38d9521f3f473b51c9031e508ddcb458ee
|
|
|
|
|
|
|
|
|
| |
This only adds tests for the bits which were moved from
morphologyfactory into sourceresolver, namely detection
of build systems and the '_get_morphology()' function.
These are just the morphologyfactory tests reworked
slightly to work properly with the modified API.
|
|
|
|
|
|
|
|
| |
This code is an essential part of 'morph build'. It's quite complex and
really shouldn't be mixed in with the base Application class.
Given a dedicated class we can store some state in the object and avoid
functions with seven parameters, too.
|
| |
|
|
|
|
|
|
|
|
|
| |
This hasn't been used in a long time to my knowledge, its API completely
misses the point, and its implementation relies on old APIs that need to
change.
Until we discover we need it, and work out what it should look like, I
think the best thing to do is get rid of it.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`morph merge` only worked for a small subset of cases, and has been left
to bit-rot, since we don't use it.
`morph tag` is just a `git tag` when we have petrified definitions
repository. We don't use it, nor do we need it, so it can go away rather
than take up valuable development time fixing it when requirements
change.
`old-foo` have all been superceded by newer versions and are no-longer
used.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The copy-artifacts and list-artifacts commands are mainly useful for
making releases. As part of the release process we copy artifacts for
the entire build graph of the release to the artifact cache on
trove.baserock.org, to provide Baserock users with ready-built
artifacts.
This part of the release process is now automated, and the automation
require the list-artifacts command to function as a 'plumbing' command.
The copy-artifacts command is no longer required. It can be replaced
with:
morph list-artifacts --quiet REPO REF MORPH | rsync --files-from=- $TARGET
The previous version of this plugin looked in the system artifact's
metadata for the list of artifacts. This is flawed as the final system
does not necessarily contain every build dependency. The new version of
the plugin calculates the build graph from source, using the same
process as the 'buildcommand' module. It also required looking in
Morph's artifact cache for the system artifact file to analyse.
|
|
|
|
|
|
|
|
|
| |
Add a module to morphlib that can list all write
and configuration extensions either in morph itself
or the morphology repository.
The module also contains methods to find an extension
filename from the name and type.
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
morphlib/plugins/deploy_plugin.py
without-test-modules
Reviewed by:
Richard Maw
Lars Wirzenius
|
| | |
|
|/
|
|
|
|
|
|
|
|
|
| |
Add a plugin to implement both `morph push` and `morph pull`.
These commands are wrappers around the corresponding git commands
push and pull, which also implement the functionality of pushing
and pulling large files provided by git-fat.
For example, running `morph pull` will pull any commits from the
remote branch not on your local branch, and then pull any large
files from the separate git-fat/rsync store on the Trove.
|
|
|
|
| |
This reverts commit ab0a83a09a93ca33aa402d9c4d3b916a48a1a882.
|
| |
|
|
|
|
|
|
|
|
|
| |
This introduces a new artifactsplitrule module, which tries to provide a
nice abstraction over matching a sequence of things to a bunch of
outputs, to be used by both chunks splitting, for separating files out
into chunk artifacts, the stratum splitting, where chunks are
aggregated into stratum artifacts, and systems selecting the right
strata to go into the artifact.
|
|
|
|
|
|
|
| |
This is an abstraction on top of SystemBranchDirectories, providing the
ability to add uncommitted changes to the temporary build branch, push
temporary build branches and retrieve the correct repository URI and ref
to build the system.
|
|\
| |
| |
| | |
Reviewed-by: Lars Wirzenius
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This will list all the paths generated by the walker generator function
that aren't in the specified set.
It removes directories from those returned by the walker, since with
os.walk(topdown=True) this culls the search space.
In the set of provided paths and the set of returned paths, if a directory
is given, then its contents are virtually part of the set.
This oddly specific behaviour is because invert_paths is to be used
with linux-user-chroot to mark subtrees as read-only, when it only
has a set of paths it wants to keep writable.
It takes a walker, rather than being given a path and using os.walk, so
that it is a pure function, so is easier to unit test.
|
|/ |
|
|
|
|
|
|
|
| |
Move "morph init" into a plugin, and remove it from the previously
existing plugin. This keeps all the old, tricky code in the old
plugin, and moves new, clean code into a new plugin. Eventually the
old plugin can be removed, since it'll be empty.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cross-bootstrap is a way to build baserock on an architecture that
does not currently have Baserock. It can be used by `morph
cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact
that can be used as a root filesystem with a basic build environment
with a script named `native-bootstrap` which will build and install
every chunk in the system.
If done with a devel system, this will give you a suitable environment
for building a proper Baserock system.
This does not currently provide a kernel for the target architecture.
Apart from adding the cross-bootstrap plugin, it also makes the
following changes:
* Moves the lit of valid_archs into morphlib (instead of locally-scoped
in MorphologyFactory)
* BuildCommand takes an extra argument, build_env
* split BuildCommand's get_artifact_object into create_source_pool and
resolve_artifacts (plus changes things that use get_artifact_object to
use the new way)
* setup_mounts finds out whether to do so by whether build_mode is
'staging', instead of by whether the setting 'staging-chroot' is true.
* Makes ChunkBuilder's get_sources use the
morphlib.builder2.extract_sources() method, and moved
set_mtime_recursively into morphlib.builder2, since it's not currently
used anywhere else.
* moved ChunkBuilder's get_commands into the Morphology class (plus
changes to anything that used get_commands)
|
|
|
|
|
|
| |
In addition, when we start using tbdiff for upgrades, all of this will
need to be re-designed and re-implemented anyway. The current plugin
is dead code. Note that the test suite had already been disabled.
|
|
|
|
|
|
|
|
|
|
| |
This removes staging areas and extracted chunks from --tempdir.
Then asks the local artifact cache what artifacts it
has and how old they are, removing all sources older than
--cachedir-artifact-delete-older-than, and may delete other sources that
are younger than --cachedir-artifact-keep-younger-than if it still needs
to make space.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a `run-in-artifact` command which allows another command
to be run in a system.
There is also a `content-manifest` command which gives a
manifest of the artifacts, which commits they were built from,
and if possible, a version.
This adds a morphlib.bins.call_in_artifact_directory() method
to run a command inside an artifact and to generate a manifest.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ExtractedTarball is more or less the equivalent to MountableImage for
artifacts that are not mountable images. So in order to inspect root
file system tarballs, ExtractedTarball can be used, for disk images,
MountableImage can be used.
The morphlib.bins.call_in_artifact_directory() method combines these
two classes and provides a way to extract/mount an artifact and call
a callback with the temporary directory / mount point as its first
argument. Using this, a plugin that runs a command relative to an
artifact's root directory can be written easily.
|
|
|
|
|
|
|
| |
This class would be pretty useful in other cases where a system image
needs to be mounted and inspected.
Also updates the Trebuchet plugin to use this class.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a new optional field to system morphologies:
"configuration-extensions".
The deployment plugin relies heavily on code from the branch and
merge plugin. This needs to be eventually fixed by refactoring
the codebase so that the shared code is in morphlib and not in
plugins. However, doing that is beyond the scope of adding a
deployment plugin.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When morph is built, it writes various version information from git
into morphlib.
When morphlib is loaded it attempts to read this version information.
if it cannot be found then it checks whether morphlib is being run from
inside a git checkout, if it is then it reads the information that way.
If it isn't in a git checkout then it raises an exception as builds made
in such a fashion are not reproducible.
The git version information retained is:
1. The output of git describe
This is a relatively human-friendly way of knowing a version and
gives a reasonably short output string.
This will end with `-unreproducible` if there were uncommitted changes.
2. The commit sha1, so the exact part of Morph's history can be found
3. The tree sha1, so if the branch has been rebased rather than
merged such that the commit is lost, you may still be able to find
it, though it requires a git-wizard to check it out
4. The branch of morph, so that it's easier to see if the
Further possible changes to increase reproducibility include:
1. Not allowing `python setup.py build` if there are uncommitted changes
2. Failing to run with uncommitted changes (recommended against since it
will just annoy developers who are making changes to morph, and make
them commit just to shut it up, then destroy the history later)
Requiring an extra flag to build in this case may work better.
3. Reading the uncommitted changes into a tree object and including
that would allow it to be recovered if the tree was later committed.
4. Checking whether the commit has been pushed upstream as well.
Too annoying to work.
|
| |
|
|
|
|
|
|
|
| |
This adds a disk system image kind which does everything the syslinux-disk
one does, but without syslinux. It deliberately carries stubbed bootloader
operations so that we can later make syslinux-disk inherit from this one
and override it.
|
|
|
|
|
| |
It's such a small amount of code, it's possibly not worth it,
but now all commands are in plugins.
|
|
|
|
|
|
|
|
| |
morphlib.app should be for application bringup and providing
command line options used by the library code.
Any external plugins that refer to morphlib.app.BuildCommand
will need fixing.
|
|
|
|
|
| |
deduce_mine_directory is duplicated in the plugin and morphlib.app
until all the commands are moved into the plugin.
|
|
|
|
|
|
|
|
| |
The plugin is called trebuchet, so that if any other functionality
is required from morph it has a place to go.
make-patch generates a trebuchet patch, so it should go in the
trebuchet plugin.
|
|
|
|
| |
This also publicizes cache_repo_and_submodules and traverse_morphs
|
| |
|
|
|
|
|
| |
Also, rename the syslinux-disk builder plugin file to make it clear
it's a system builder plugin.
|
| |
|
|
|
|
| |
Mea culpa. I managed to make changes without running ./check properly.
|
|
|
|
|
|
|
| |
The artifact cache doesn't need to know the whole of an artifact object
to be able to retrieve something from the cache, just the basename.
This can be generated from the contents, or just saved itself.
|