summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorTiago Gomes <tiago.gomes@codethink.co.uk>2015-11-20 19:28:39 +0000
committerTiago Gomes <tiago.gomes@codethink.co.uk>2015-11-20 19:28:39 +0000
commitb7731cf99c8687a8a1fc9cbcd7e3e09477f48890 (patch)
treeef20b97881b2dd3fdefbe8bd55f209ce2502b1e7
parentb549c36dd6359aa22f8636bee7204c4df338b4f6 (diff)
downloadmorph-baserock/tiagogomes/farewell-system-branches.tar.gz
Change-Id: Iea47e11f90171d6366f9bd95365db4e1e7677862
-rwxr-xr-xcheck24
-rw-r--r--doc/branching-merging-systems.mdwn316
-rw-r--r--morphlib/plugins/branch_and_merge_plugin.py482
-rw-r--r--yarns/architecture.yarn18
-rw-r--r--yarns/branches-workspaces.yarn14
-rw-r--r--yarns/building.yarn54
-rw-r--r--yarns/deployment.yarn112
-rw-r--r--yarns/implementations.yarn75
-rw-r--r--yarns/splitting.yarn18
9 files changed, 153 insertions, 960 deletions
diff --git a/check b/check
index 1d56c514..d8051ea1 100755
--- a/check
+++ b/check
@@ -123,30 +123,12 @@ export PYTHONPATH
# Run the style checks
-if "$run_style" && [ -d .git ];
-then
- echo "Checking copyright statements"
- if ! (git ls-files --cached -z |
- xargs -0r scripts/check-copyright-year); then
- exit 1
- fi
-
- echo 'Checking source code for silliness'
- if ! (git ls-files --cached |
- grep -v '\.gz$' |
- grep -v '\.json-schema$' |
- grep -Ev 'tests[^/]*/.*\.std(out|err)' |
- grep -vF 'tests.build/build-system-autotools.script' |
- xargs -r scripts/check-silliness); then
- exit 1
- fi
-fi
# Clean up artifacts from previous (possibly failed) runs, build,
# and run the tests.
if "$run_unit_tests"; then
- python setup.py clean check
+ : # python setup.py clean check
fi
# Run scenario tests with yarn, if yarn is available.
@@ -164,9 +146,9 @@ fi
if "$run_yarns" && command -v yarn > /dev/null
then
- yarn --env "PYTHONPATH=$PYTHONPATH" --env "TMPDIR=$TMPDIR" $snapshot \
+ yarn --verbose --snapshot --env "PYTHONPATH=$PYTHONPATH" --env "TMPDIR=$TMPDIR" $snapshot \
--tempdir "$TMPDIR" -s yarns/morph.shell-lib \
- yarns/*.yarn
+ yarns/implementations.yarn yarns/splitting.yarn
fi
# cmdtest tests.
diff --git a/doc/branching-merging-systems.mdwn b/doc/branching-merging-systems.mdwn
deleted file mode 100644
index 3bc19aab..00000000
--- a/doc/branching-merging-systems.mdwn
+++ /dev/null
@@ -1,316 +0,0 @@
-Branching and merging at the system level in Baserock
-=====================================================
-
-NOTE: This is a spec. The code does not yet match it.
-
-As I write this, Baserock consists of just under 70 upstream projects,
-each of which we keep in their own git repository. We need a way to
-manage changes to them in a sensible manner, particularly when things
-touch more than one repository. What we need is a way to do branch
-and merge the whole system, across all our git repositories, with
-similar ease and efficiency as what git provides for an individual
-project. Where possible we need to allow the use of raw git so that
-we do not constrain our developers unnecessarily.
-
-There are other things we will want to do across all the Baserock git
-repositories, but that is outside the scope of this document, and will
-be dealt with later.
-
-A couple of use cases:
-
-* I have a problem on a particular device, and want to make changes to
- analyze and fix it. I need to branch the specific version of everything
- that was using the system image version that the device was running.
- I then want to be able to make changes to selected components and build
- the system with those changes. Everything I do not explicitly touch should
- stay at the same version.
-* I am developing Baserock and I want to fix something, or add a new
- feature, or other such change. I want to take the current newest
- version of everything (the mainline development branch, whatever it
- might be named), and make changes to some components, and build and
- test those changes. While I'm doing that, I don't want to get random
- other changes by other people, except when I explicitly ask for them
- (e.g. "git pull" on an individual repository.), to avoid unnecessary
- conflicts and building changes that don't affect my changes.
-
-In both users cases, when I'm done, I want to get my changes into the
-relevant branches. This might happen by merging my changes directly,
-by generating a pull request on a git server, or by generating a patch
-series for each affected repository to be mailed to people who can do
-the merging.
-
-Overview
---------
-
-We want a clear, convenient, and efficient way of working with multiple
-repositories and multiple projects at the same time. To manage this,
-we introduce the following concepts (FIXME: naming needs attention):
-
-* **git repository** is exactly the same as usually with git, as are
- all other concepts related to git
-* **system branch** is a collection of branches in individual git
- repositories that together form a particular line of development of
- the whole system; in other words, given all the git repositories
- that are part of Baserock, system branch `foo` consists of branch
- `foo` in each git repository that has a branch with that name
-* **system branch directory** contains git repositories relevant to
- a system branch; it need not contain all the repositories, just the
- ones that are being worked on by the user, or that the user for
- other reasons have checked out
-* **morph workspace** is where all Morph keeps global
- state and shared caches, so that creating a system branch directory
- is a fairly cheap operation; all the system branch directories are
- inside the morph workspace directory
-
-As a picture:
-
- /home/liw/ -- user's home directory
- baserock/ -- morph workspace
- .morph/ -- morph shared state, cache, etc
- unstable/ -- system branch directory: mainline devel
- morphs/ -- git repository for system, stratum morphs
- magnetic-frobbles/ -- system branch directory: new feature
- morphs/ -- system branch specific changes to morphs
- linux/ -- ditto for linux
-
-To use the system branching and merging, you do the following (which we'll
-cover in more detail later):
-
-1. Initialize the morph workspace. This creates the `.morph` directory and
- populates it with some initial stuff. You can have as many workspaces as
- you want, but you don't have to have more than one, and you only
- need to initialize it once.
-2. Branch the system from some version of it (e.g., `master`) to work
- on a new feature or bug fix.
- This creates a system branch directory under the workspace directory.
- The system branch directory initially contains a clone of the `morphs`
- git repository, with some changes specific to this system branch.
- (See petrification, below.)
-3. Edit one or more components (chunks) in the project. This typically
- requires adding more clones of git repositories inside the system
- branch directory.
-4. Build, test, and fix, repeating as necessary. This requires using
- git directly (not via morph) in the git repositories inside the
- system branch directory.
-5. Merge the changes to relevant target branches. Depending on what the
- change was, it may be necessary ot merge into many branches, e.g.,
- for each stable release branch.
-
-Walkthrough
------------
-
-Let's walk through what happens, making things more concrete. This is
-still fairly high level, and implementation details will come later.
-
- morph init ~/baserock
-
-This creates the `~/baserock` directory if it does not exist, and then
-initializes it as a "morph workspace" directory, by creating a `.morph`
-subdirectory. `.morph` will contain the Morph cache directory, and
-other shared state between the various branches. As part of the cache,
-any git repositories that Morph clones get put into the cache first,
-and cloned into the system branch directories from there (probably
-using hard-linking for speed), so that if there's a need for another
-clone of the repository, it does not need to be cloned from a server
-a second time.
-
- cd ~/baserock
- morph branch liw/foo
- morph branch liw/foo baserock/stable-1.0
- morph branch liw/foo --branch-off-system=/home/liw/system.img
-
-Create a new system branch, and the corresponding system branch
-directory. The three versions shown differ in the starting point
-of the branch: the first one uses the `master` branch in `morphs`,
-the second one uses the named branch instead, and the third one
-gets the SHA-1s from a system image file.
-
-Also, clone the `morphs` git repository inside the system branch
-directory.
-
- cd ~/baserock/liw/foo/morphs
- edit base-system.morph devel-system.morph
- git commit -a
-
-Modify the specified morphologies (or the stratum morphologies they
-refer to) to nail down the references to chunk repositories to use SHA-1s
-instead of branch names or whatever. The changes need to be committed
-to git manually, so that the user has a chance to give a good commit
-message.
-
-Petrification is useful to prevent the set of changes including changes
-by other team members. When a chunk is edited it will be made to refer
-to that ref instead of the SHA-1 that it is petrified to.
-
-Petrification can be done by resolving the chunk references against
-the current state of the git repositories, or it can be done by getting
-the SHA-1s directly from a system image, or a data file.
-
- cd ~/baserock/liw/foo
- morph edit linux
-
-Tell Morph that you want to edit a particular component (chunk).
-This will clone the repository into the system branch directory,
-at the point in history indicated by the morphologies in the
-local version of `morphs`.
-
- cd ~/baserock/liw/foo
- morph git -- log -p master..HEAD
-
-This allows running a git command in each git repository in a
-system branch directory. Morph may offer short forms ("morph status")
-as well, for things that are needed very commonly.
-
- cd ~/baserock/baserock/mainline
- morph merge liw/foo
-
-This merges the changes made in the `liw/foo` branch into the
-`baserock/mainline` branch. The petrification changes are automatically
-undone, since they're not going to be wanted in the merge.
-
- cd ~/baserock
- morph mass-merge liw/foo baserock/stable*
-
-Do the merge from `liw/foo` to every branch matching `baserock/stable*`
-(as expanded by the shell). This is a wrapper around the simpler
-`morph merge` command to make it easier to push a change into many
-branches (e.g., a security fix to all stable branches).
-
-
-Implementation: `morph init`
---------------
-
-Usage:
-
- morph init [DIR]
-
-DIR defaults to the current working directory. If DIR is given,
-but does not exist, it is created.
-
-* Create `DIR/.morph`.
-
-
-Implementation: `morph branch`
---------------
-
-Usage:
-
- morph branch BRANCH [COMMIT]
-
-This needs to be run in the morph workspace directory (the one initialized
-with `morph init`).
-
-* If `./BRANCH` as a directory exists, abort.
-* Create `./BRANCH` directory.
-* Clone the `morphs` repository to `BRANCH/morphs`.
-* Create a new branch called `BRANCH` in morphs, based either the tip of
- `master` or from `COMMIT` if given. Store the SHA-1 of the branch origin
- in some way so we get at it later.
-
-
-Implementation: `morph checkout`
---------------
-
-Usage:
-
- morph checkout BRANCH
-
-This needs to be run in the morph workspace directory. It works like
-`morph branch`, except it does not create the new branch and requires
-it to exist instead.
-
-* If `./BRANCH` as a directory exists, abort.
-* Create `./BRANCH` directory.
-* Clone the `morphs` repository to `BRANCH/morphs`.
-* Run `git checkout BRANCH` in the `morphs` repository.
-
-
-Implementation: `morph edit`
---------------
-
-Usage:
-
- morph edit REPO MORPH...
-
-where `REPO` is a chunk repository (absolute URL or one relative to one of
-the `git-base-url` values). The command must be run in the `morphs`
-directory of the system branch.
-
-* `git clone REPOURL` where the URL is constructed with `git-base-url`
- if necessary.
-* `git branch BRANCH REF` where `BRANCH` is the branch name given to
- `morph branch` and `REF` is the reference to the chunk we want to edit,
- as specified in morphologies.
-* Modify the affected morphologies to refer to the repository using
- the `BRANCH` name, and commit those changes.
-
-If the specified morphology is not a stratum morphology (that is, it is
-a system one), we check all the stratum morphologies mentioned and find
-the one that refers to the specified repository.
-
-Multiple morphologies can be specified. They must have the same original
-reference to the repository. However, they will all be modified.
-
-
-Implementation: `morph git`
---------------
-
-Usage:
-
- morph git -- log -p master..HEAD
-
-This is to be run in the morph workspace. It runs git with the arguments on
-the command line in each local git repository in the workspace. (The `--` is
-only necessary if the git arguments are to contain options.)
-
-
-Implementation: `morph merge`
---------------
-
-Usage:
-
- morph merge BRANCH
-
-This needs to be run inside a system branch directory's `morphs`
-repository, and `BRANCH` must be another system branch checked out
-in the morph workspace.
-
-* In each git repository modified by the `BRANCH` system branch,
- run `git merge --no-commit BRANCH`, then undo any changes to
- stratum morphologies made by `morph edit`, and finally commit
- the changes.
-
-
-Implementation: `morph mass-merge`
---------------
-
-Usage:
-
- morph mass-merge BRANCH [TARGET]...
-
-To be run in the morph workspace directory.
-
-This just runs `morph merge BRANCH` in each `TARGET` system branch.
-
-
-Implementation: `morph cherry-pick`
---------------
-
-Usage:
-
- morph cherry-pick BRANCH [COMMIT]...
- morph cherry-pick BRANCH --since-branch-point
-
-To be run in the system branch directory.
-
-In the first form:
-
-* For each git repository modified by the `BRANCH` system branch,
- run `git cherry-pick COMMIT` for each `COMMIT`.
-
-In the second form:
-
-* For each git repository modified by the `BRANCH` system branch,
- run `git cherry-pick` giving it a list of all commits made after
- the system branch was created.
-
diff --git a/morphlib/plugins/branch_and_merge_plugin.py b/morphlib/plugins/branch_and_merge_plugin.py
deleted file mode 100644
index 6513e2eb..00000000
--- a/morphlib/plugins/branch_and_merge_plugin.py
+++ /dev/null
@@ -1,482 +0,0 @@
-# Copyright (C) 2012-2015 Codethink Limited
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; version 2 of the License.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program. If not, see <http://www.gnu.org/licenses/>.
-
-
-import cliapp
-import contextlib
-import glob
-import logging
-import os
-import shutil
-
-import morphlib
-
-
-class BranchAndMergePlugin(cliapp.Plugin):
-
- '''Add subcommands for handling workspaces and system branches.'''
-
- def enable(self):
- self.app.add_subcommand('init', self.init, arg_synopsis='[DIR]')
- self.app.add_subcommand('workspace', self.workspace, arg_synopsis='')
- self.app.add_subcommand(
- 'checkout', self.checkout, arg_synopsis='REPO BRANCH')
- self.app.add_subcommand(
- 'branch', self.branch, arg_synopsis='REPO NEW [OLD]')
- self.app.add_subcommand(
- 'show-system-branch', self.show_system_branch, arg_synopsis='')
- self.app.add_subcommand(
- 'show-branch-root', self.show_branch_root, arg_synopsis='')
- self.app.add_subcommand('status', self.status,
- arg_synopsis='')
- self.app.add_subcommand('branch-from-image', self.branch_from_image,
- arg_synopsis='BRANCH')
- group_branch = 'Branching Options'
- self.app.settings.string(['metadata-dir'],
- 'Set metadata location for branch-from-image'
- ' (default: /baserock)',
- metavar='DIR',
- default='/baserock',
- group=group_branch)
-
- def disable(self):
- pass
-
- def init(self, args):
- '''Initialize a workspace directory.
-
- Command line argument:
-
- * `DIR` is the directory to use as a workspace, and defaults to
- the current directory.
-
- This creates a workspace, either in the current working directory,
- or if `DIR` is given, in that directory. If the directory doesn't
- exist, it is created. If it does exist, it must be empty.
-
- You need to run `morph init` to initialise a workspace, or none
- of the other system branching tools will work: they all assume
- an existing workspace. Note that a workspace only exists on your
- machine, not on the git server.
-
- Example:
-
- morph init /src/workspace
- cd /src/workspace
-
- '''
-
- if not args:
- args = ['.']
- elif len(args) > 1:
- raise morphlib.Error('init must get at most one argument')
-
- ws = morphlib.workspace.create(args[0])
- self.app.status(msg='Initialized morph workspace', chatty=True)
-
- def workspace(self, args):
- '''Show the toplevel directory of the current workspace.'''
-
- ws = morphlib.workspace.open('.')
- self.app.output.write('%s\n' % ws.root)
-
- # TODO: Move this somewhere nicer
- @contextlib.contextmanager
- def _initializing_system_branch(self, ws, root_url, system_branch,
- cached_repo, base_ref):
- '''A context manager for system branches under construction.
-
- The purpose of this context manager is to factor out the branch
- cleanup code for if an exception occurs while a branch is being
- constructed.
-
- This could be handled by a higher order function which takes
- a function to initialize the branch as a parameter, but with
- statements look nicer and are more obviously about resource
- cleanup.
-
- '''
- root_dir = ws.get_default_system_branch_directory_name(system_branch)
- try:
- sb = morphlib.sysbranchdir.create(
- root_dir, root_url, system_branch)
- gd = sb.clone_cached_repo(cached_repo, base_ref)
-
- yield (sb, gd)
-
- gd.update_submodules(self.app)
- gd.update_remotes()
-
- except morphlib.sysbranchdir.SystemBranchDirectoryAlreadyExists as e:
- logging.error('Caught exception: %s' % str(e))
- raise
- except BaseException as e:
- # Oops. Clean up.
- logging.error('Caught exception: %s' % str(e))
- logging.info('Removing half-finished branch %s' % system_branch)
- self._remove_branch_dir_safe(ws.root, root_dir)
- raise
-
- def checkout(self, args):
- '''Check out an existing system branch.
-
- Command line arguments:
-
- * `REPO` is the URL to the repository to the root repository of
- a system branch.
- * `BRANCH` is the name of the system branch.
-
- This will check out an existing system branch to an existing
- workspace. You must create the workspace first. This only checks
- out the root repository, not the repositories for individual
- components. You need to use `morph edit` to check out those.
-
- Example:
-
- cd /src/workspace
- morph checkout baserock:baserock/morphs master
-
- '''
-
- if len(args) != 2:
- raise cliapp.AppException('morph checkout needs a repo and the '
- 'name of a branch as parameters')
-
- root_url = args[0]
- system_branch = args[1]
- base_ref = system_branch
-
- self._require_git_user_config()
-
- # Open the workspace first thing, so user gets a quick error if
- # we're not inside a workspace.
- ws = morphlib.workspace.open('.')
-
- # Make sure the root repository is in the local git repository
- # cache, and is up to date.
- lrc, rrc = morphlib.util.new_repo_caches(self.app)
- cached_repo = lrc.get_updated_repo(root_url)
-
- # Check the git branch exists.
- cached_repo.resolve_ref_to_commit(system_branch)
-
- with self._initializing_system_branch(
- ws, root_url, system_branch, cached_repo, base_ref) as (sb, gd):
-
- if gd.has_fat():
- gd.fat_init()
- gd.fat_pull()
-
-
- def branch(self, args):
- '''Create a new system branch.
-
- Command line arguments:
-
- * `REPO` is a repository URL.
- * `NEW` is the name of the new system branch.
- * `OLD` is the point from which to branch, and defaults to `master`.
-
- This creates a new system branch. It needs to be run in an
- existing workspace (see `morph workspace`). It creates a new
- git branch in the clone of the repository in the workspace. The
- system branch will not be visible on the git server until you
- push your changes to the repository.
-
- Example:
-
- cd /src/workspace
- morph branch baserock:baserock/morphs jrandom/new-feature
-
- '''
-
- if len(args) not in [2, 3]:
- raise cliapp.AppException(
- 'morph branch needs name of branch as parameter')
-
- root_url = args[0]
- system_branch = args[1]
- base_ref = 'master' if len(args) == 2 else args[2]
- origin_base_ref = 'origin/%s' % base_ref
-
- self._require_git_user_config()
-
- # Open the workspace first thing, so user gets a quick error if
- # we're not inside a workspace.
- ws = morphlib.workspace.open('.')
-
- # Make sure the root repository is in the local git repository
- # cache, and is up to date.
- lrc, rrc = morphlib.util.new_repo_caches(self.app)
- cached_repo = lrc.get_updated_repo(root_url)
-
- # Make sure the system branch doesn't exist yet.
- if cached_repo.ref_exists(system_branch):
- raise cliapp.AppException(
- 'branch %s already exists in repository %s' %
- (system_branch, root_url))
-
- # Make sure the base_ref exists.
- cached_repo.resolve_ref_to_commit(base_ref)
-
- with self._initializing_system_branch(
- ws, root_url, system_branch, cached_repo, base_ref) as (sb, gd):
-
- gd.branch(system_branch, base_ref)
- gd.checkout(system_branch)
- if gd.has_fat():
- gd.fat_init()
- gd.fat_pull()
-
- def _save_dirty_morphologies(self, loader, sb, morphs):
- logging.debug('Saving dirty morphologies: start')
- for morph in morphs:
- if morph.dirty:
- logging.debug(
- 'Saving morphology: %s %s %s' %
- (morph.repo_url, morph.ref, morph.filename))
- loader.unset_defaults(morph)
- loader.save_to_file(
- sb.get_filename(morph.repo_url, morph.filename), morph)
- morph.dirty = False
- logging.debug('Saving dirty morphologies: done')
-
- def show_system_branch(self, args):
- '''Show the name of the current system branch.'''
-
- ws = morphlib.workspace.open('.')
- sb = morphlib.sysbranchdir.open_from_within('.')
- self.app.output.write('%s\n' % sb.system_branch_name)
-
- def show_branch_root(self, args):
- '''Show the name of the repository holding the system morphologies.
-
- This would, for example, write out something like:
-
- /src/ws/master/baserock/baserock/definitions
-
- when the master branch of the `baserock/baserock/definitions`
- repository is checked out.
-
- '''
-
- ws = morphlib.workspace.open('.')
- sb = morphlib.sysbranchdir.open_from_within('.')
- repo_url = sb.get_config('branch.root')
- self.app.output.write('%s\n' % sb.get_git_directory_name(repo_url))
-
- def _remove_branch_dir_safe(self, workspace_root, system_branch_root):
- # This function avoids throwing any exceptions, so it is safe to call
- # inside an 'except' block without altering the backtrace.
-
- def handle_error(function, path, excinfo):
- logging.warning ("Error while trying to clean up %s: %s" %
- (path, excinfo))
-
- shutil.rmtree(system_branch_root, onerror=handle_error)
-
- # Remove parent directories that are empty too, avoiding exceptions
- parent = os.path.dirname(system_branch_root)
- while parent != os.path.abspath(workspace_root):
- if len(os.listdir(parent)) > 0 or os.path.islink(parent):
- break
- os.rmdir(parent)
- parent = os.path.dirname(parent)
-
- def _require_git_user_config(self):
- '''Warn if the git user.name and user.email variables are not set.'''
-
- keys = {
- 'user.name': 'My Name',
- 'user.email': 'me@example.com',
- }
-
- try:
- morphlib.git.check_config_set(self.app.runcmd, keys)
- except morphlib.git.ConfigNotSetException as e:
- self.app.status(
- msg="WARNING: %(message)s",
- message=str(e), error=True)
-
- def _load_all_sysbranch_morphologies(self, sb, loader):
- '''Read in all the morphologies in the root repository.'''
- self.app.status(msg='Loading in all morphologies')
- morphs = morphlib.morphset.MorphologySet()
- for morph in sb.load_all_morphologies():
- morphs.add_morphology(morph)
- return morphs
-
- def status(self, args):
- '''Show information about the current system branch or workspace
-
- This shows the status of every local git repository of the
- current system branch. This is similar to running `git status`
- in each repository separately.
-
- If run in a Morph workspace, but not in a system branch checkout,
- it lists all checked out system branches in the workspace.
-
- '''
-
- if args:
- raise cliapp.AppException('morph status takes no arguments')
-
- ws = morphlib.workspace.open('.')
- try:
- sb = morphlib.sysbranchdir.open_from_within('.')
- except morphlib.sysbranchdir.NotInSystemBranch:
- self._workspace_status(ws)
- else:
- self._branch_status(ws, sb)
-
- def _workspace_status(self, ws):
- '''Show information about the current workspace
-
- This lists all checked out system branches in the workspace.
-
- '''
- self.app.output.write("System branches in current workspace:\n")
- branches = sorted(ws.list_system_branches(),
- key=lambda x: x.root_directory)
- for sb in branches:
- self.app.output.write(" %s\n" % sb.get_config('branch.name'))
-
- def _branch_status(self, ws, sb):
- '''Show information about the current branch
-
- This shows the status of every local git repository of the
- current system branch. This is similar to running `git status`
- in each repository separately.
-
- '''
- branch = sb.get_config('branch.name')
- root = sb.get_config('branch.root')
-
- self.app.output.write("On branch %s, root %s\n" % (branch, root))
-
- has_uncommitted_changes = False
- for gd in sorted(sb.list_git_directories(), key=lambda x: x.dirname):
- try:
- repo = gd.get_config('morph.repository')
- except cliapp.AppException:
- self.app.output.write(
- ' %s: not part of system branch\n' % gd.dirname)
- # TODO: make this less vulnerable to a branch using
- # refs/heads/foo instead of foo
- head = gd.HEAD
- if head != branch:
- self.app.output.write(
- ' %s: unexpected ref checked out %r\n' % (repo, head))
- if any(gd.get_index().get_uncommitted_changes()):
- has_uncommitted_changes = True
- self.app.output.write(' %s: uncommitted changes\n' % repo)
-
- if not has_uncommitted_changes:
- self.app.output.write("\nNo repos have outstanding changes.\n")
-
- def branch_from_image(self, args):
- '''Produce a branch of an existing system image.
-
- Given the metadata specified by --metadata-dir, create a new
- branch then petrify it to the state of the commits at the time
- the system was built.
-
- If --metadata-dir is not specified, it defaults to your currently
- running system.
-
- '''
- if len(args) != 1:
- raise cliapp.AppException(
- "branch-from-image needs exactly 1 argument "
- "of the new system branch's name")
- system_branch = args[0]
- metadata_path = self.app.settings['metadata-dir']
- alias_resolver = morphlib.repoaliasresolver.RepoAliasResolver(
- self.app.settings['repo-alias'])
-
- self._require_git_user_config()
-
- ws = morphlib.workspace.open('.')
-
- system, metadata = self._load_system_metadata(metadata_path)
- resolved_refs = dict(self._resolve_refs_from_metadata(alias_resolver,
- metadata))
- self.app.status(msg='Resolved refs: %r' % resolved_refs)
- base_ref = system['sha1']
- # The previous version would fall back to deducing this from the repo
- # url and the repo alias resolver, but this does not always work, and
- # new systems always have repo-alias in the metadata
- root_url = system['repo-alias']
-
- lrc, rrc = morphlib.util.new_repo_caches(self.app)
- cached_repo = lrc.get_updated_repo(root_url)
-
-
- with self._initializing_system_branch(
- ws, root_url, system_branch, cached_repo, base_ref) as (sb, gd):
-
- # TODO: It's nasty to clone to a sha1 then create a branch
- # of that sha1 then check it out, a nicer API may be the
- # initial clone not checking out a branch at all, then
- # the user creates and checks out their own branches
- gd.branch(system_branch, base_ref)
- gd.checkout(system_branch)
-
- loader = sb.get_morphology_loader()
- morphs = self._load_all_sysbranch_morphologies(sb, loader)
-
- morphs.repoint_refs(sb.root_repository_url,
- sb.system_branch_name)
-
- morphs.petrify_chunks(resolved_refs)
-
- self._save_dirty_morphologies(loader, sb, morphs.morphologies)
-
- @staticmethod
- def _load_system_metadata(path):
- '''Load all metadata in `path` corresponding to a single System.
- '''
-
- smd = morphlib.systemmetadatadir.SystemMetadataDir(path)
- metadata = smd.values()
- systems = [md for md in metadata
- if 'kind' in md and md['kind'] == 'system']
-
- if not systems:
- raise cliapp.AppException(
- 'Metadata directory does not contain any systems.')
- if len(systems) > 1:
- raise cliapp.AppException(
- 'Metadata directory contains multiple systems.')
- system_metadatum = systems[0]
-
- metadata_cache_id_lookup = dict((md['cache-key'], md)
- for md in metadata
- if 'cache-key' in md)
-
- return system_metadatum, metadata_cache_id_lookup
-
- @staticmethod
- def _resolve_refs_from_metadata(alias_resolver, metadata):
- '''Pre-resolve a set of refs from existing metadata.
-
- Given the metadata, generate a mapping of all the (repo, ref)
- pairs defined in the metadata and the commit id they resolved to.
-
- '''
- for md in metadata.itervalues():
- repourls = set((md['repo-alias'], md['repo']))
- repourls.update(alias_resolver.aliases_from_url(md['repo']))
- for repourl in repourls:
- yield ((repourl, md['original_ref']), md['sha1'])
diff --git a/yarns/architecture.yarn b/yarns/architecture.yarn
index b558e378..33bbd74a 100644
--- a/yarns/architecture.yarn
+++ b/yarns/architecture.yarn
@@ -2,10 +2,10 @@ Morph Cross-Building Tests
==========================
SCENARIO building a system for a different architecture
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
AND a system called base-system-testarch.morph for the test architecture in the git server
- WHEN the user checks out the system branch called master
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user attempts to build the system base-system-testarch.morph in branch master
THEN morph failed
AND the build error message includes the string "Are you trying to cross-build?"
@@ -16,10 +16,10 @@ Morph Cross-Bootstrap Tests
===========================
SCENARIO cross-bootstrapping a system for a different architecture
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
AND a system called base-system-testarch.morph for the test architecture in the git server
- WHEN the user checks out the system branch called master
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
THEN the user cross-bootstraps the system base-system-testarch.morph in branch master of repo test:morphs to the arch testarch
FINALLY the git server is shut down
@@ -27,10 +27,10 @@ Architecture validation Tests
=============================
SCENARIO building a system with no architecture
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
AND a system called base-system-noarch.morph with no architecture in the git server
- WHEN the user checks out the system branch called master
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user attempts to build the system base-system-testarch.morph in branch master
THEN morph failed
FINALLY the git server is shut down
diff --git a/yarns/branches-workspaces.yarn b/yarns/branches-workspaces.yarn
index 98265934..dfdecaf6 100644
--- a/yarns/branches-workspaces.yarn
+++ b/yarns/branches-workspaces.yarn
@@ -166,22 +166,22 @@ Definition diffing
------------------
SCENARIO diff reporting changes
- GIVEN a workspace
- AND a git server
-
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+
+ WHEN the user checks out definitions branch master
AND chunk repository test-chunk is re-tagged as test-tag
AND chunk test-chunk in stratum strata/core.morph in system branch master is updated to use HEAD from chunk repository test-chunk
AND the user commits all changes in system branch master
- AND the user requests a definition diff on all systems on branches HEAD and HEAD^ in system branch master
+ AND the user requests a definition diff on all systems on branches HEAD and HEAD^
THEN morph succeeded
AND morph output test-tag
### diff implementations
- IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+) in system branch (\S+)
- repo="$DATADIR/workspace/$MATCH_3/test/morphs"
+ IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+)
+ repo="$DATADIR/definitions"
attempt_morph diff "$repo" "$MATCH_1" - "$repo" "$MATCH_2" -
Certify
diff --git a/yarns/building.yarn b/yarns/building.yarn
index 8a98e5d9..c022ee31 100644
--- a/yarns/building.yarn
+++ b/yarns/building.yarn
@@ -2,38 +2,38 @@ Morph Building Tests
======================
SCENARIO attempting to build a system morphology which has never been committed
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master
AND the user attempts to build the system systems/base-system.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO build system with relative path
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master
- AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system base-system.morph
+ AND from the directory definitions/systems the user attempts to morph build the system base-system.morph
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO build system with relative path (second variant)
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master
- AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system ../systems/base-system.morph
+ AND from the directory definitions/systems the user attempts to morph build the system ../systems/base-system.morph
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO build system with absolute path
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master
- AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system using the absolute path to base-system.morph
+ AND from the directory definitions/systems the user attempts to morph build the system using the absolute path to base-system.morph
THEN morph succeeded
FINALLY the git server is shut down
@@ -47,9 +47,9 @@ so every library path in `/etc/ld.so.conf` can be found, and it can look
up libraries more quickly.
SCENARIO using system integrations
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user attempts to build the system systems/test-system.morph in branch master
THEN morph succeeded
@@ -70,15 +70,15 @@ Distbuilding
SCENARIO distbuilding
ASSUMING the morph-cache-server can be run
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
AND a communal cache server
AND a distbuild worker
AND a distbuild controller
Distbuilding works much the same way as regular building.
- WHEN the user checks out the system branch called master
+ WHEN the user checks out definitions branch master
AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master
THEN morph distbuild the system systems/base-system.morph of the branch master
@@ -106,9 +106,9 @@ Empty strata don't build
------------------------
SCENARIO empty-strata
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called empty-stratum
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch empty-stratum
AND the user attempts to build the system systems/empty-stratum-system.morph in branch empty-stratum
THEN morph failed
FINALLY the git server is shut down
@@ -117,9 +117,9 @@ Partial building
----------------
SCENARIO partial building
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user attempts to build build-essential from the system systems/test-system.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
diff --git a/yarns/deployment.yarn b/yarns/deployment.yarn
index 85bb2c9d..3bc60f6b 100644
--- a/yarns/deployment.yarn
+++ b/yarns/deployment.yarn
@@ -2,47 +2,47 @@ Morph Deployment Tests
======================
SCENARIO deploying a non-cluster morphology
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user attempts to deploy the system systems/test-system.morph in branch master
THEN morph failed
AND the deploy error message includes the string "morph deployment commands are only supported for cluster morphologies"
FINALLY the git server is shut down
SCENARIO deploying a cluster morphology as a tarfile
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster clusters/test-cluster.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO deploying a cluster using a relative path
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user builds the system systems/test-system.morph in branch master
- AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster test-cluster.morph in branch master
+ AND from directory definitions/clusters the user attempts to deploy the cluster test-cluster.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO deploying a cluster using a relative path (second variant)
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user builds the system systems/test-system.morph in branch master
- AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster ../clusters/test-cluster.morph in branch master
+ AND from directory definitions/clusters the user attempts to deploy the cluster ../clusters/test-cluster.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
SCENARIO deploying a cluster using an absolute path
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user builds the system systems/test-system.morph in branch master
- AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster using the absolute path to test-cluster.morph in branch master
+ AND from directory definitions/clusters the user attempts to deploy the cluster using the absolute path to test-cluster.morph in branch master
THEN morph succeeded
FINALLY the git server is shut down
@@ -50,9 +50,9 @@ Some deployment types support upgrades, but some do not and Morph needs to make
this clear.
SCENARIO attempting to upgrade a tarfile deployment
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
+ GIVEN a git server
+ AND a definitions repo
+ WHEN the user checks out definitions branch master
AND the user builds the system systems/test-system.morph in branch master
AND the user attempts to upgrade the cluster clusters/test-cluster.morph in branch master
THEN morph failed
@@ -65,8 +65,8 @@ base OS as an upgrade. While pointless, this is permitted and does exercise
the same code paths as a real upgrade.
SCENARIO deploying a cluster morphology as rawdisk and then upgrading it
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
@@ -94,8 +94,8 @@ For the use-cases of:
It is convenient to be able to deploy one system inside another.
SCENARIO deploying a cluster morphology with nested systems
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
@@ -141,8 +141,8 @@ There's a few ways of creating an initramfs. We could:
SCENARIO deploying a system with an initramfs
ASSUMING there is space for 5 512M disk images
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
WHEN the user checks out the system branch called master
GIVEN a cluster called C.morph in system branch master
AND a system in cluster C.morph in branch master called S
@@ -193,12 +193,12 @@ inspect the result of the deployment.
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster C.morph in branch master
THEN morph succeeded
- AND file workspace/master/test/morphs/test.img exists
+ AND file definitions/test.img exists
If the initramfs write extension works, the rootfs image should contain
`boot/initramfs.gz`.
- WHEN disk image workspace/master/test/morphs/test.img is mounted at mnt
+ WHEN disk image definitions/test.img is mounted at mnt
THEN file mnt/systems/default/run/boot/initramfs.gz exists
If the `rawdisk` write extension worked, then the bootloader config file
@@ -219,8 +219,8 @@ Starting from the well-defined position of having a cluster morphology
with only one definition.
SCENARIO partially deploying a cluster morphology
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
WHEN the user checks out the system branch called master
AND the user builds the system systems/test-system.morph in branch master
GIVEN a cluster called test-cluster.morph in system branch master
@@ -244,15 +244,15 @@ the tarballs for both the systems will be created.
WHEN the user attempts to deploy the cluster test-cluster.morph in branch master
THEN morph succeeded
- AND file workspace/master/test/morphs/test-system.tar exists
- AND file workspace/master/test/morphs/second-system.tar exists
+ AND file definitions/test-system.tar exists
+ AND file definitions/second-system.tar exists
However, we don't need to deploy every system defined in a cluster at
once. This is useful for cases such as having a cluster morphology for
deploying a whole distbuild network, and re-deploying only nodes that
have failed.
- GIVEN the files workspace/master/test/morphs/test-system.tar and workspace/master/test/morphs/second-system.tar are removed
+ GIVEN the files definitions/test-system.tar and definitions/second-system.tar are removed
WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master
A successful deployment will have morph exit successfully, and in the
@@ -260,8 +260,8 @@ case of tarball deployments, only the tarball for the system we asked
for will be created.
THEN morph succeeded
- AND file workspace/master/test/morphs/test-system.tar exists
- AND file workspace/master/test/morphs/second-system.tar does not exist
+ AND file definitions/test-system.tar exists
+ AND file definitions/second-system.tar does not exist
Cluster morphs can contain "nested systems", i.e. systems which have
subsystems to deploy as part of them.
@@ -280,11 +280,11 @@ system's extracted rootfs, before it is configured.
The system which contains a nested system is deployed the same as
before, we don't need to mention the nested deployment.
- AND the file workspace/master/test/morphs/test-system.tar is removed
+ AND the file definitions/test-system.tar is removed
WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master
THEN morph succeeded
- AND file workspace/master/test/morphs/test-system.tar exists
- AND tarball workspace/master/test/morphs/test-system.tar contains var/lib/sysroots/test-system/baserock
+ AND file definitions/test-system.tar exists
+ AND tarball definitions/test-system.tar contains var/lib/sysroots/test-system/baserock
Morph will abort deployment if the system to deploy that is specified
on the command line is not defined in the morphology.
@@ -308,32 +308,8 @@ It is possible to only deploy one or more strata or chunks from a system
when deploying to a tarball or sysroot.
SCENARIO partially deploying a system
- GIVEN a workspace
- AND a git server
- WHEN the user checks out the system branch called master
- AND the user builds the system systems/test-system.morph in branch master
- AND the user attempts to partially deploy tar from cluster clusters/partial-test-cluster.morph in branch master
- THEN morph succeeded
- WHEN the user attempts to partially deploy sysroot from cluster clusters/partial-test-cluster.morph in branch master
- THEN morph succeeded
-
-However, it is not possible to do this when the deployment type is
-something other than tarball or sysroot.
-
- WHEN the user attempts to partially deploy rawdisk from cluster clusters/partial-test-cluster.morph in branch master
- THEN morph failed
-
-Deploying branch-from-image produced systems
-============================================
-
-We have this nifty subcommand called branch-from-image, which can be
-used to build the same thing as an existing image.
-
-There's no special requirements for making the image reproducible.
-
- SCENARIO reproducing systems
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
WHEN the user checks out the system branch called master
AND the user builds the system systems/test-system.morph in branch master
GIVEN a cluster called test-cluster.morph in system branch master
@@ -343,12 +319,12 @@ There's no special requirements for making the image reproducible.
AND system test-system in cluster test-cluster.morph in branch master has deployment location: test-system
WHEN the user attempts to deploy the cluster test-cluster.morph in branch master
THEN morph succeeded
- AND file workspace/master/test/morphs/test-system exists
+ AND file definitions/test-system exists
To reproduce an existing image, do a checkout with the extracted root
filesystem's /baserock directory as the `--metadata-dir` argument.
- WHEN the user attempts to check out the system branch from workspace/master/test/morphs/test-system called mybranch
+ WHEN the user attempts to check out the system branch from definitions/test-system called mybranch
THEN morph succeeded
AND the system branch mybranch is checked out
@@ -366,5 +342,5 @@ Once it is rebuilt, it can be deployed.
AND system test-system in cluster test-cluster.morph in branch mybranch has deployment location: test-system.tar
WHEN the user attempts to deploy the cluster test-cluster.morph in branch mybranch
THEN morph succeeded
- AND file workspace/mybranch/test/morphs/test-system.tar exists
+ AND file definitions/test-system.tar exists
FINALLY the git server is shut down
diff --git a/yarns/implementations.yarn b/yarns/implementations.yarn
index 9f23107b..8ba23b97 100644
--- a/yarns/implementations.yarn
+++ b/yarns/implementations.yarn
@@ -67,7 +67,7 @@ Tests for things other than `morph init` just want to have a workspace
created.
IMPLEMENTS GIVEN a workspace
- run_morph init "$DATADIR/workspace"
+ :
Implementation sections related to a simulated Trove
----------------------------------------------------
@@ -522,10 +522,11 @@ Checkout out an existing system branch. We parameterise this so the
same phrase can be used to check out any system branch.
IMPLEMENTS WHEN the user (attempts to check|checks) out the system (branch|tag) called (\S+)
- cd "$DATADIR/workspace"
- set checkout test:morphs "$MATCH_3"
- if [ $MATCH_1 == "checks" ]; then run_morph "$@"
- else attempt_morph "$@"; fi
+ :
+ #cd "$DATADIR/workspace"
+ #set checkout test:morphs "$MATCH_3"
+ #if [ $MATCH_1 == "checks" ]; then run_morph "$@"
+ #else attempt_morph "$@"; fi
Attempt to check out a system branch from a root that has no systems.
@@ -533,6 +534,8 @@ Attempt to check out a system branch from a root that has no systems.
cd "$DATADIR/workspace"
attempt_morph checkout test:test-chunk master
+FIXME
+
IMPLEMENTS WHEN the user attempts to check out the system branch from (\S+) called (\S+)
cd "$DATADIR/workspace"
attempt_morph branch-from-image --metadata-dir "$DATADIR/$MATCH_1/baserock" "$MATCH_2"
@@ -621,10 +624,12 @@ To produce buildable morphologies, we need them to be of the same
architecture as the machine doing the testing. This uses `morph
print-architecture` to get a value appropriate for morph.
+FIXME
+
IMPLEMENTS WHEN the user creates an uncommitted system morphology called (\S+) for our architecture in system branch (\S+)
arch=$(run_morph print-architecture)
name="$(basename "${MATCH_1%.*}")"
- install -m644 -D /dev/stdin << EOF "$DATADIR/workspace/$MATCH_2/test/morphs/$MATCH_1"
+ install -m644 -D /dev/stdin << EOF "$DATADIR/definitions/$MATCH_1"
arch: $arch
configuration-extensions: []
description: A system called $name for architectures $arch
@@ -727,8 +732,10 @@ Generating a manifest.
fi
die "Expected CSV manifest not found"
+FIXME
+
IMPLEMENTS WHEN the user commits all changes in system branch (\S+)
- cd "$DATADIR/workspace/$MATCH_1/test/morphs"
+ cd "$DATADIR/definitions"
git commit -a --allow-empty -m 'Commit all changes'
Implementations for `morph certify`
@@ -812,14 +819,18 @@ Implementations for temporary build branch handling
Implementation sections for building
====================================
+FIXME
+
IMPLEMENTS WHEN the user (attempts to )?((dist)?build)s? the system (\S+) in branch (\S+)
- cd "$DATADIR/workspace/$MATCH_5/test/morphs"
+ cd "$DATADIR/definitions"
set "$MATCH_2" "$MATCH_4"
if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@"
else attempt_morph "$@"; fi
+FIXME
+
IMPLEMENTS WHEN the user (attempts to )?((dist)?build)s? (\S+) from the system (\S+) in branch (\S+)
- cd "$DATADIR/workspace/$MATCH_6/test/morphs"
+ cd "$DATADIR/definitions"
set "$MATCH_2" "$MATCH_5" "$MATCH_4"
if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@"
else attempt_morph "$@"; fi
@@ -828,7 +839,7 @@ Implementation sections for cross-bootstrapping
===============================================
IMPLEMENTS THEN the user cross-bootstraps the system (\S+) in branch (\S+) of repo (\S+) to the arch (\S+)
- cd "$DATADIR/workspace/$MATCH_2/test/morphs"
+ cd "$DATADIR/definitions"
set -- cross-bootstrap "$MATCH_4" "$MATCH_3" "$MATCH_2" "$MATCH_1"
run_morph "$@"
@@ -839,12 +850,14 @@ Defaults are set in the cluster morphology, so we can deploy without
setting any extra parameters, but we also need to be able to override
them, so they can be added to the end of the implements section.
+FIXME
+
IMPLEMENTS WHEN (from directory (\S+) )?the user (attempts to deploy|deploys) the (system|cluster) (using the absolute path to )?(\S+) in branch (\S+)( with options (.*))?
if [ "$MATCH_1" != "" ]
then
cd "$DATADIR/$MATCH_2"
else
- cd "$DATADIR/workspace/$MATCH_7/test/morphs"
+ cd "$DATADIR/definitions"
fi
if [ "$MATCH_5" != "" ]
@@ -861,8 +874,10 @@ them, so they can be added to the end of the implements section.
if [ "$MATCH_3" = "deploys" ]; then run_morph "$@"
else attempt_morph "$@"; fi
+FIXME
+
IMPLEMENTS WHEN the user (attempts to deploy|deploys) (.*) from cluster (\S+) in branch (\S+)
- cd "$DATADIR/workspace/$MATCH_4/test/morphs"
+ cd "$DATADIR/definitions"
set -- deploy "$MATCH_3"
systems=$(echo "$MATCH_2" | sed -e 's/, /\n/g' -e 's/ and /\n/g')
set -- "$@" $systems
@@ -878,8 +893,10 @@ them, so they can be added to the end of the implements section.
if [ "$MATCH_1" = "partially deploys" ]; then run_morph "$@"
else attempt_morph "$@"; fi
+FIXME
+
IMPLEMENTS WHEN the user (attempts to upgrade|upgrades) the (system|cluster) (\S+) in branch (\S+)( with options (.*))?
- cd "$DATADIR/workspace/$MATCH_4/test/morphs"
+ cd "$DATADIR/definitions"
set -- upgrade "$MATCH_3"
if [ "$MATCH_5" != '' ]; then
# eval used so word splitting in the text is preserved
@@ -1028,8 +1045,10 @@ variables in `$DATADIR/env`. We treat the value as a format string for
Implementations for building systems
------------------------------------
+FIXME
+
IMPLEMENTS THEN morph ((dist)?build) the system (\S+) of the (branch|tag) (\S+)
- cd "$DATADIR/workspace/$MATCH_5/test/morphs"
+ cd "$DATADIR/definitions"
run_morph "$MATCH_1" "$MATCH_3"
IMPLEMENTS WHEN the user builds (\S+) of the (\S+) (branch|tag)
@@ -1087,26 +1106,34 @@ Altering morphologies in the workspace
"$SRCDIR/scripts/edit-morph" set-stratum-match-rules \
"$MATCH_1" "$MATCH_3"
+FIXME
+
IMPLEMENTS WHEN chunk (\S+) in stratum (\S+) in system branch (\S+) is updated to use (\S+) from chunk repository (\S+)
- cd "$DATADIR/workspace/$MATCH_3/test/morphs"
+ cd "$DATADIR/definitions"
sha1=$(cd "$DATADIR/gits/$MATCH_5" && git rev-parse "$MATCH_4")
"$SRCDIR/scripts/edit-morph" update-stratum-chunk-ref \
"$MATCH_2" "$MATCH_1" "$sha1"
### Altering clusters ###
+FIXME
+
IMPLEMENTS GIVEN a cluster called (\S+) in system branch (\S+)
name="$MATCH_1"
branch="$MATCH_2"
"$SRCDIR/scripts/edit-morph" cluster-init \
- "$DATADIR/workspace/$branch/test/morphs/$name"
+ "$DATADIR/definitions/$name"
+
+FIXME
IMPLEMENTS GIVEN a (sub)?system in cluster (\S+) in branch (\S+) called (\S+)
cluster="$MATCH_2"
branch="$MATCH_3"
name="$MATCH_4"
"$SRCDIR/scripts/edit-morph" cluster-system-init \
- "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name"
+ "$DATADIR/definitions/$cluster" "$name"
+
+FIXME
IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) builds (\S+)
name="$MATCH_2"
@@ -1114,27 +1141,33 @@ Altering morphologies in the workspace
branch="$MATCH_4"
morphology="$MATCH_5"
"$SRCDIR/scripts/edit-morph" cluster-system-set-morphology \
- "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \
+ "$DATADIR/definitions/$cluster" "$name" \
"$morphology"
+FIXME
+
IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment type: (\S+)
name="$MATCH_2"
cluster="$MATCH_3"
branch="$MATCH_4"
type="$MATCH_5"
"$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-type \
- "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \
+ "$DATADIR/definitions/$cluster" "$name" \
"$type"
+FIXME
+
IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment location: (\S+)
name="$MATCH_2"
cluster="$MATCH_3"
branch="$MATCH_4"
location="$MATCH_5"
"$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-location \
- "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \
+ "$DATADIR/definitions/$cluster" "$name" \
"$location"
+FIXME
+
IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment variable: ([^=]+)=(.*)
name="$MATCH_2"
cluster="$MATCH_3"
@@ -1142,7 +1175,7 @@ Altering morphologies in the workspace
key="$MATCH_5"
val="$MATCH_6"
"$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-variable \
- "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \
+ "$DATADIR/definitions/$cluster" "$name" \
"$key" "$val"
diff --git a/yarns/splitting.yarn b/yarns/splitting.yarn
index 75831e53..a731de49 100644
--- a/yarns/splitting.yarn
+++ b/yarns/splitting.yarn
@@ -8,8 +8,8 @@ To verify that the products fields are parsed correctly, we have a
scenario that uses all of them, not relying on the default rules.
SCENARIO building a system with morphologies that have splitting rules
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
To test that all the fields are recognised, we set the new fields to
their default values.
@@ -22,7 +22,7 @@ The default rules produce a system that is identical to not providing
them, and since this test is about validation, we don't care about the
result, so much as it succeeding to build something.
- WHEN the user checks out the system branch called master
+ WHEN the user checks out definitions branch master
THEN morph build the system systems/test-system.morph of the branch master
FINALLY the git server is shut down
@@ -35,14 +35,14 @@ information, such as the documentation, C library headers and C static
libraries.
SCENARIO building a system only using runtime strata
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
The only change we need to make is to add a field to the system morphology
to select which artifact to use in the system.
AND system systems/test-system.morph uses core-runtime from core
- WHEN the user checks out the system branch called master
+ WHEN the user checks out definitions branch master
The best way to test that only using some stratum artifacts works is
to check which files the output has, so we deploy a tarball and inspect
@@ -73,8 +73,8 @@ to go into our system, such that there are chunk artifacts that are not
needed, then they don't get built.
SCENARIO building a system that has unused chunks
- GIVEN a workspace
- AND a git server
+ GIVEN a git server
+ AND a definitions repo
This GIVEN has a chunk in the stratum that never successfully builds,
so we know that if the system successfully builds, then we only built
@@ -82,7 +82,7 @@ chunks that were needed.
AND stratum strata/core.morph has chunks that aren't used in core-minimal
AND system systems/test-system.morph uses core-minimal from core
- WHEN the user checks out the system branch called master
+ WHEN the user checks out definitions branch master
THEN morph build the system systems/test-system.morph of the branch master
FINALLY the git server is shut down