diff options
Diffstat (limited to 'assets/js/search-data.json')
| -rw-r--r-- | assets/js/search-data.json | 793 |
1 files changed, 793 insertions, 0 deletions
diff --git a/assets/js/search-data.json b/assets/js/search-data.json new file mode 100644 index 00000000..1d8ca663 --- /dev/null +++ b/assets/js/search-data.json @@ -0,0 +1,793 @@ +{"0": { + "doc": "Contributing", + "title": "Contributing", + "content": ". | Submitting patches | Commit message style | Running the test suite | Coding style | Contributing Tutorial | Release process | . ", + "url": "/ostree/CONTRIBUTING/", + + "relUrl": "/CONTRIBUTING/" + },"1": { + "doc": "Contributing", + "title": "Submitting patches", + "content": "A majority of current maintainers prefer the GitHub pull request model, and this motivated moving the primary git repository to https://github.com/ostreedev/ostree. However, we do not use the “Merge pull request” button, because we do not like merge commits for one-patch pull requests, among other reasons. See this issue for more information. Instead, we use an instance of Homu, currently known as cgwalters-bot. As a review proceeds, the preferred method is to push fixup! commits. Any commits committed with the --fixup option will have have the word fixup! in its commit title. This is to indicate that this particular commit will be squashed with the commit that was specified in this command, git commit --fixup <commit ref or hash>. Homu knows how to use --autosquash when performing the final merge. See the Git documentation for more information. Alternative methods if you don’t like GitHub (also fully supported): . | Send mail to ostree-list@gnome.org, with the patch attached | Attach them to https://bugzilla.gnome.org/ | . It is likely however once a patch is ready to apply a maintainer will push it to a GitHub PR, and merge via Homu. ", + "url": "/ostree/CONTRIBUTING/#submitting-patches", + + "relUrl": "/CONTRIBUTING/#submitting-patches" + },"2": { + "doc": "Contributing", + "title": "Commit message style", + "content": "Please look at git log and match the commit log style, which is very similar to the Linux kernel. You may use Signed-off-by, but we’re not requiring it. General Commit Message Guidelines: . | Title . | Specify the context or category of the changes e.g. lib for library changes, docs for document changes, bin/<command-name> for command changes, etc. | Begin the title with the first letter of the first word capitalized. | Aim for less than 50 characters, otherwise 72 characters max. | Do not end the title with a period. | Use an imperative tone. | . | Body . | Separate the body with a blank line after the title. | Begin a paragraph with the first letter of the first word capitalized. | Each paragraph should be formatted within 72 characters. | Content should be about what was changed and why this change was made. | If your commit fixes an issue, the commit message should end with Closes: #<number>. | . | . Commit Message example: . <context>: Less than 50 characters for subject title A paragraph of the body should be within 72 characters. This paragraph is also less than 72 characters. For more information see How to Write a Git Commit Message . Editing a Committed Message: . To edit the message from the most recent commit run git commit --amend. To change older commits on the branch use git rebase -i. For a successful rebase have the branch track upstream main. Once the changes have been made and saved, run git push --force origin <branch-name>. ", + "url": "/ostree/CONTRIBUTING/#commit-message-style", + + "relUrl": "/CONTRIBUTING/#commit-message-style" + },"3": { + "doc": "Contributing", + "title": "Running the test suite", + "content": "OSTree uses both make check and supports the Installed Tests model as well (if --enable-installed-tests is provided). ", + "url": "/ostree/CONTRIBUTING/#running-the-test-suite", + + "relUrl": "/CONTRIBUTING/#running-the-test-suite" + },"4": { + "doc": "Contributing", + "title": "Coding style", + "content": "Indentation is GNU. Files should start with the appropriate mode lines. Use GCC __attribute__((cleanup)) wherever possible. If interacting with a third party library, try defining local cleanup macros. Use GError and GCancellable where appropriate. Prefer returning gboolean to signal success/failure, and have output values as parameters. Prefer linear control flow inside functions (aside from standard loops). In other words, avoid “early exits” or use of goto besides goto out;. This is an example of an “early exit”: . static gboolean myfunc (...) { gboolean ret = FALSE; /* some code */ /* some more code */ if (condition) return FALSE; /* some more code */ ret = TRUE; out: return ret; } . If you must shortcut, use: . if (condition) { ret = TRUE; goto out; } . A consequence of this restriction is that you are encouraged to avoid deep nesting of loops or conditionals. Create internal static helper functions, particularly inside loops. For example, rather than: . while (condition) { /* some code */ if (condition) { for (i = 0; i < somevalue; i++) { if (condition) { /* deeply nested code */ } /* more nested code */ } } } . Instead do this: . static gboolean helperfunc (..., GError **error) { if (condition) { /* deeply nested code */ } /* more nested code */ return ret; } while (condition) { /* some code */ if (!condition) continue; for (i = 0; i < somevalue; i++) { if (!helperfunc (..., i, error)) goto out; } } . ", + "url": "/ostree/CONTRIBUTING/#coding-style", + + "relUrl": "/CONTRIBUTING/#coding-style" + },"5": { + "doc": "Contributing", + "title": "Contributing Tutorial", + "content": "For a detailed walk-through on building, modifying, and testing, see this tutorial on how to start contributing to OSTree. ", + "url": "/ostree/CONTRIBUTING/#contributing-tutorial", + + "relUrl": "/CONTRIBUTING/#contributing-tutorial" + },"6": { + "doc": "Contributing", + "title": "Release process", + "content": "Releases can be performed by creating a new release ticket and following the steps in the checklist there. ", + "url": "/ostree/CONTRIBUTING/#release-process", + + "relUrl": "/CONTRIBUTING/#release-process" + },"7": { + "doc": "Historical OSTree README", + "title": "OSTree", + "content": " ", + "url": "/ostree/README-historical/#ostree", + + "relUrl": "/README-historical/#ostree" + },"8": { + "doc": "Historical OSTree README", + "title": "Problem statement", + "content": "Hacking on the core operating system is painful - this includes most of GNOME from upower and NetworkManager up to gnome-shell. I want a system that matches these requirements: . | Does not disturb your existing OS | Is not terribly slow to use | Shares your $HOME - you have your data | Allows easy rollback | Ideally allows access to existing apps | . ", + "url": "/ostree/README-historical/#problem-statement", + + "relUrl": "/README-historical/#problem-statement" + },"9": { + "doc": "Historical OSTree README", + "title": "Comparison with existing tools", + "content": ". | Virtualization . Fails on points 2) and 3). Actually qemu-kvm can be pretty fast, but in a lot of cases there is no substitute for actually booting on bare metal; GNOME 3 really needs some hardware GPU acceleration. | Rebuilding distribution packages . Fails on points 1) and 4). Is also just very annoying: dpkg/rpm both want tarballs, which you don’t have since you’re working from git. The suggested “mock/pbuilder” type chroot builds are slow. And even with non-chroot builds there is lots of pointless build wrapping going on. Both dpkg and rpm also are poor at helping you revert back to the original system. All of this can be scripted to be less bad of course - and I have worked on such scripts. But fundamentally you’re still fighting the system, and if you’re hacking on a lowlevel library like say glib, you can easily get yourself to the point where you need a recovery CD - at that point your edit/compile/debug cycle is just too long. | “sudo make install” . Now your system is in an undefined state. You can use e.g. rpm -qV to try to find out what you overwrote, but neither dpkg nor rpm will help clean up any files left over that aren’t shipped by the old package. This is most realistic option for people hacking on system components currently, but ostree will be better. | LXC / containers . Fails on 3), and 4) is questionable. Also shares the annoying part of rebuilding distribution packages. LXC is focused on running multiple server systems at the same time, which isn’t what we want (at least, not right now), and honestly even trying to support that for a graphical desktop would be a lot of tricky work, for example getting two GDM instances not to fight over VT allocations. But some bits of the technology may make sense to use. | jhbuild + distribution packages . The state of the art in GNOME - but can only build non-root things - this means you can’t build NetworkManager, and thus are permanently stuck on whatever the distro provides. | . ", + "url": "/ostree/README-historical/#comparison-with-existing-tools", + + "relUrl": "/README-historical/#comparison-with-existing-tools" + },"10": { + "doc": "Historical OSTree README", + "title": "Who is ostree for?", + "content": "First - operating system developers and testers. I specifically keep a few people in mind - Dan Williams and Eric Anholt, as well as myself obviously. For Eric Anholt, a key use case for him is being able to try out the latest gnome-shell, and combine it with his work on Mesa, and see how it works/performs - while retaining the ability to roll back if one or both breaks. The rollback concept is absolutely key for shipping anything to enthusiasts or knowledable testers. With a system like this, a tester can easily perform a local rollback - something just not well supported by dpkg/rpm. (What about Conary? See below.) . Also, distributing operating system trees (instead of packages) gives us a sane place to perform automated QA before we ship it to testers. We should never be wasting these people’s time. Even better, this system would allow testers to bisect across operating system builds, and do so very efficiently. ", + "url": "/ostree/README-historical/#who-is-ostree-for", + + "relUrl": "/README-historical/#who-is-ostree-for" + },"11": { + "doc": "Historical OSTree README", + "title": "The core idea - chroots", + "content": "chroots are the original lightweight “virtualization”. Let’s use them. So basically, you install a mainstream distribution (say Debian). It has a root filesystem like this: . /usr /etc /home ... Now, what we can do is have a system that installs chroots as a subdirectory of the root, like: . /ostree/gnomeos-3.0-opt-393a4555/{usr,etc,sbin,...} /ostree/gnomeos-3.2-opt-7e9788a2/{usr,etc,sbin,...} . These live in the same root filesystem as your regular distribution (Note though, the root partition should be reasonably sized, or hopefully you’ve used just one big partition). You should be able to boot into one of these roots. Since ostree lives inside a distro created partition, a tricky part here is that we need to know how to interact with the installed distribution’s grub. This is an annoying but tractable problem. First, we install a kernel+initramfs alongside the distribution’s. Then, we have a “trampoline” ostree-init binary which is statically linked, and boot the kernel with init=/ostree/ostree-init. This then takes care of chrooting and running the init binary. An important note here is that we bind mount the real /home. This means you have your data. This also implies we share uid/gid, so /etc/passwd will have to be in sync. Probably what we’ll do is have a script to pull the data from the “host” OS. I’ve decided for now to move /var into /ostree to avoid sharing it with the “host” distribution, because in practice we’re likely to hit incompatibilities. Do note however /etc lives inside the OSTree; it’s presently versioned and readonly like everything else. On a pure OSTree system, the filesystem layout will look like this: . |-- boot |-- home |-- ostree |-- var |-- current -> gnomeos-3.2-opt-7e9788a2 |-- gnomeos-3.0-opt-393a4555 | |-- etc | |-- lib | |-- mnt | |-- proc | |-- run | |-- sbin | |-- srv | |-- sys | `-- usr | `-- gnomeos-3.2-opt-7e9788a2 |-- etc |-- lib |-- mnt |-- proc |-- run |-- sbin |-- srv |-- sys | `-- usr |-- root . ", + "url": "/ostree/README-historical/#the-core-idea---chroots", + + "relUrl": "/README-historical/#the-core-idea---chroots" + },"12": { + "doc": "Historical OSTree README", + "title": "Making this efficient", + "content": "One of the first things you’ll probably ask is “but won’t that use a lot of disk space”? Indeed, it will, if you just unpack a set of RPMs or .debs into each root. Besides chroots, there’s another old Unix idea we can take advantage of - hard links. These allow sharing the underlying data of a file, with the tradeoff that changing any one file will change all names that point to it. This mutability means that we have to either: . | Make sure everything touching the operating system breaks hard links This is probably tractable over a long period of time, but if anything has a bug, then it corrupts the file effectively. | Make the core OS read-only, with a well-defined mechanism for mutating under the control of ostree. | . I chose 2. ", + "url": "/ostree/README-historical/#making-this-efficient", + + "relUrl": "/README-historical/#making-this-efficient" + },"13": { + "doc": "Historical OSTree README", + "title": "A userspace content-addressed versioning filesystem", + "content": "At its very core, that’s what ostree is. Just like git. If you understand git, you know it’s not like other revision control systems. git is effectively a specialized, userspace filesystem, and that is a very powerful idea. At the core of git is the idea of “content-addressed objects”. For background on this, see http://book.git-scm.com/7_how_git_stores_objects.html . Why not just use git? Basically because git is designed mainly for source trees - it goes to effort to be sure it’s compressing text for example, under the assumption that you have a lot of text. Its handling of binaries is very generic and unoptimized. In contrast, ostree is explicitly designed for binaries, and in particular one type of binary - ELF executables (or it will be once we start using bsdiff). Another big difference versus git is that ostree uses hard links between “checkouts” and the repository. This means each checkout uses almost no additional space, and is extremely fast to check out. We can do this because again each checkout is designed to be read-only. So we mentioned above there are: . /ostree/gnomeos-3.2-opt-7e9788a2 /ostree/gnomeos-3.2-opt-393a4555 . There is also a “repository” that looks like this: . /ostree/repo/objects/17/a95e8ca0ba655b09cb68d7288342588e867ee0.file /ostree/repo/objects/17/68625e7ff5a8db77904c77489dc6f07d4afdba.meta /ostree/repo/objects/17/cc01589dd8540d85c0f93f52b708500dbaa5a9.file /ostree/repo/objects/30 /ostree/repo/objects/30/6359b3ca7684358a3988afd005013f13c0c533.meta /ostree/repo/objects/30/8f3c03010cedd930b1db756ce659c064f0cd7f.meta /ostree/repo/objects/30/8cf0fd8e63dfff6a5f00ba5a48f3b92fb52de7.file /ostree/repo/objects/30/6cad7f027d69a46bb376044434bbf28d63e88d.file . Each object is either metadata (like a commit or tree), or a hard link to a regular file. Note that also unlike git, the checksum here includes metadata such as uid, gid, permissions, and extended attributes. (It does not include file access times, since those shouldn’t matter for the OS) . This is another important component to allowing the hardlinks. We wouldn’t want say all empty files to be shared necessarily. (Though maybe this is wrong, and since the OS is readonly, we can make all files owned by root without loss of generality). However this tradeoff means that we probably need a second index by content, so we don’t have to redownload the entire OS if permissions change =) . ", + "url": "/ostree/README-historical/#a-userspace-content-addressed-versioning-filesystem", + + "relUrl": "/README-historical/#a-userspace-content-addressed-versioning-filesystem" + },"14": { + "doc": "Historical OSTree README", + "title": "Atomic upgrades, rollback", + "content": "OSTree is designed to atomically swap operating systems - such that during an upgrade and reboot process, you either get the full new system, or the old one. There is no “Please don’t turn off your computer”. We do this by simply using a symbolic link like: . /ostree/current -> /ostree/gnomeos-3.4-opt-e3b0c4429 . Where gnomeos-e3b0c4429 has the full regular filesystem tree with usr/ etc/ directories as above. To upgrade or rollback (there is no difference internally), we simply check out a new tree into gnomeos-b90ae4763 for example, then swap the “current” symbolic link, then remove the old tree. But does this mean you have to reboot for OS upgrades? Very likely, yes - and this is no different from RPM/deb or whatever. They just typically lie to you about it =) . A typical model with RPM/deb is to unpack the new files, then use some IPC mechanism (SIGHUP, a control binary like /usr/sbin/apachectl) to signal the running process to reload. There are multiple problems with this - one is that in the new state, daemon A may depend on the updated configuration in daemon B. This may not be particularly common in default configurations, but it’s highly likely that that some deployments will have e.g. apache talking to a local MySQL instance. So you really want to do is only apply the updated configuration when all the files are in place; not after each RPM or .deb is installed. What’s even harder is the massive set of race conditions that are possible while RPM/deb are in the process of upgrading. Cron jobs are very likely to hit this. If we want the ability to apply updates to a live system, we could first pause execution of non-upgrade userspace tasks. This could be done via SIGSTOP for example. Then, we can swap around the filesystem tree, and then finally attempt to apply updates via SIGHUP, and if possible, restart processes. ", + "url": "/ostree/README-historical/#atomic-upgrades-rollback", + + "relUrl": "/README-historical/#atomic-upgrades-rollback" + },"15": { + "doc": "Historical OSTree README", + "title": "Configuration Management", + "content": "By now if you’ve thought about this problem domain before, you’re wondering about configuration management. In other words, if the OS is read only, how do I edit /etc/sudoers? . Well, have you ever been a system administrator on a zypper/yum system, done an RPM update, which then drops .rpmnew files in your /etc/ that you have to go and hunt for with “find” or something, and said to yourself, “Wow, this system is awesome!!!” ? Right, that’s what I thought. Configuration (and systems) management is a tricky problem, and I certainly don’t have a magic bullet. However, one large conceptual improvement I think is defaulting to “rebase” versus “merge”. This means that we won’t permit direct modification of /etc - instead, you HAVE to write a script which accomplishes your goals. To generate a tree, we check out a new copy, then run your script on top. If the script fails, we can roll back the update, or drop to a shell if interactive. However, we also need to consider cases where the administrator modifies state indirectly by a program. Think “adduser” for example. Possible approaches: . | Patch all of these programs to know how to write to the writable location, instead of the R/O bind mount overlay. | Move the data to /var | . ", + "url": "/ostree/README-historical/#configuration-management", + + "relUrl": "/README-historical/#configuration-management" + },"16": { + "doc": "Historical OSTree README", + "title": "What about “packages”?", + "content": "There are several complex and separate issues hiding in this seemingly simple question. I think OSTree always makes sense to use as a core operating system builder and updater. By “core” here I mean the parts that aren’t removable. Debian has Essential: yes, any other distribution has this too implicitly in the set of dependencies for their updater tool. Now, let me just say I will absolutely support using something like apt/yum/zypper (and consequently deb/rpm) on top of OSTree. This isn’t trivial, but there aren’t any conceptual issues. Concretely for example, RPM or .deb might make sense as a delivery vehicle for third party OS extensions. A canoncial example is the NVidia graphics driver. If one is using OSTree to build an operating system, then there has to be some API for applications. And that demands its own targeted solution - something like an evolved glick (zeroinstall is also similar). Current package systems are totally broken for application deployment though; for example, they will remove files away from under running applications on update. And we clearly need the ability to install and upgrade applications without rebooting the OS. ", + "url": "/ostree/README-historical/#what-about-packages", + + "relUrl": "/README-historical/#what-about-packages" + },"17": { + "doc": "Historical OSTree README", + "title": "Details of RPM installation", + "content": "We should be able to install LSB rpms. This implies providing “rpm”. The tricky part here is since the OS itself is not assembled via RPMs, we need to fake up a database of “provides” as if we were. Even harder would be maintaining binary compatibilty with any arbitrary %post scripts that may be run. ", + "url": "/ostree/README-historical/#details-of-rpm-installation", + + "relUrl": "/README-historical/#details-of-rpm-installation" + },"18": { + "doc": "Historical OSTree README", + "title": "What about BTRFS? Doesn’t it solve everything?", + "content": "In short, BTRFS is not a magic bullet, but yes - it helps significantly. The obvious thing to do is layer BTRFS under dpkg/rpm, and have a separate subvolume for /home so rollbacks don’t lose your data. See e.g. http://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs . As a general rule an issue with the BTRFS is that it can’t roll back just changes to things installed by RPM (i.e. what’s in rpm -qal). For example, it’s possible to e.g. run yum update, then edit something in /etc, reboot and notice things are broken, then roll back and have silently lost your changes to /etc. Another example is adding a new binary in /usr/local. You could say, “OK, we’ll use subvolumes for those!”. But then what about /var (and your VM images that live in /var/lib/libvirt ?) . Finally, probably the biggest disadvantage of the rpm/dpkg + BTRFS approach is it doesn’t solve the race conditions that happen when unpacking packages into the live system. This problem is really important to me. Note though ostree can definitely take advantage of BTRFS features! In particular, we could use “reflink” http://lwn.net/Articles/331808/ instead of hard links, and avoid having the object store corrupted if somehow the files are modified directly. ", + "url": "/ostree/README-historical/#what-about-btrfs--doesnt-it-solve-everything", + + "relUrl": "/README-historical/#what-about-btrfs--doesnt-it-solve-everything" + },"19": { + "doc": "Historical OSTree README", + "title": "Other systems", + "content": "I’ve spent a long time thinking about this problem, and here are some of the other possible solutions out there I looked at, and why I didn’t use them: . | Git: http://git-scm.com/ . Really awesome, and the core inspiration here. But like I mentioned above, not at all designed for binaries - we can make different tradeoffs. | bup: https://github.com/apenwarr/bup . bup is cool. But it shares the negative tradeoffs with git, though it does add positives of its own. It also inspired me. | git-annex: http://git-annex.branchable.com/git-annex/ . Looks interesting; I think this will share the same negative tradeoffs with git as far as using it for an OS goes. | schroot: http://www.debian-administration.org/articles/566 . Like LXC/containers, but just using a chroot. | NixOS: http://nixos.org/ . The NixOS people have a lot of really good ideas, and they’ve definitely thought about the problem space. However, their approach of checksumming all inputs to a package is pretty wacky. I don’t see the point, and moreover it uses gobs of disk space. | Conary: http://wiki.rpath.com/wiki/Conary:Updates_and_Rollbacks . If rpm/dpkg are like CVS, Conary is closer to Subversion. It’s not bad, but ostree is better than it for the exact same reasons git is better than Subversion. | BTRFS: http://en.wikipedia.org/wiki/Btrfs . See above. | Solaris IPS: http://hub.opensolaris.org/bin/view/Project+pkg/ . Rollback is ZFS level, so I think this shares the same tradeoffs as BTRFS+RPM/deb. They probably have some vertical integration though which definitely helps. Obviously we can’t use ZFS. | Jhbuild: https://live.gnome.org/Jhbuild . What we’ve been using in GNOME, and has the essential property of allowing you to “fall back” to a stable system. But ostree will blow it out of the water. | . ", + "url": "/ostree/README-historical/#other-systems", + + "relUrl": "/README-historical/#other-systems" + },"20": { + "doc": "Historical OSTree README", + "title": "Development", + "content": ". | OSTree wiki page: https://live.gnome.org/OSTree . | ostbuild wiki page: https://live.gnome.org/OSTree/Ostbuild . | Git repository: http://git.gnome.org/browse/ostree/ . | Deploying OSTree in the Gnome servers: https://bugzilla.gnome.org/show_bug.cgi?id=669772 . | . ", + "url": "/ostree/README-historical/#development", + + "relUrl": "/README-historical/#development" + },"21": { + "doc": "Historical OSTree README", + "title": "Historical OSTree README", + "content": "This file is outdated, but some of the text here is still useful for historical context. I’m preserving it (explicitly still in the tree) for posterity. ", + "url": "/ostree/README-historical/", + + "relUrl": "/README-historical/" + },"22": { + "doc": "Adapting existing mainstream distributions", + "title": "Adapting existing mainstream distributions", + "content": ". | System layout | Booting and initramfs technology | /usr/lib/passwd | Adapting existing package managers . | Licensing for this document: | . | . ", + "url": "/ostree/adapting-existing/", + + "relUrl": "/adapting-existing/" + },"23": { + "doc": "Adapting existing mainstream distributions", + "title": "System layout", + "content": "First, OSTree encourages systems to implement UsrMove This is simply to avoid the need for more bind mounts. By default OSTree’s dracut hook creates a read-only bind mount over /usr; you can of course generate individual bind-mounts for /bin, all the /lib variants, etc. So it is not intended to be a hard requirement. Remember, because by default the system is booted into a chroot equivalent, there has to be some way to refer to the actual physical root filesystem. Therefore, your operating system tree should contain an empty /sysroot directory; at boot time, OSTree will make this a bind mount to the physical / root directory. There is precedent for this name in the initramfs context. You should furthermore make a toplevel symbolic link /ostree which points to /sysroot/ostree, so that the OSTree tool at runtime can consistently find the system data regardless of whether it’s operating on a physical root or inside a deployment. Because OSTree only preserves /var across upgrades (each deployment’s chroot directory will be garbage collected eventually), you will need to choose how to handle other toplevel writable directories specified by the Filesystem Hierarchy Standard. Your operating system may of course choose not to support some of these such as /usr/local, but following is the recommended set: . | /home → /var/home | /opt → /var/opt | /srv → /var/srv | /root → /var/roothome | /usr/local → /var/usrlocal | /mnt → /var/mnt | /tmp → /sysroot/tmp | . Furthermore, since /var is empty by default, your operating system will need to dynamically create the targets of these at boot. A good way to do this is using systemd-tmpfiles, if your OS uses systemd. For example: . d /var/log/journal 0755 root root - L /var/home - - - - ../sysroot/home d /var/opt 0755 root root - d /var/srv 0755 root root - d /var/roothome 0700 root root - d /var/usrlocal 0755 root root - d /var/usrlocal/bin 0755 root root - d /var/usrlocal/etc 0755 root root - d /var/usrlocal/games 0755 root root - d /var/usrlocal/include 0755 root root - d /var/usrlocal/lib 0755 root root - d /var/usrlocal/man 0755 root root - d /var/usrlocal/sbin 0755 root root - d /var/usrlocal/share 0755 root root - d /var/usrlocal/src 0755 root root - d /var/mnt 0755 root root - d /run/media 0755 root root - . Particularly note here the double indirection of /home. By default, each deployment will share the global toplevel /home directory on the physical root filesystem. It is then up to higher levels of management tools to keep /etc/passwd or equivalent synchronized between operating systems. Each deployment can easily be reconfigured to have its own home directory set simply by making /var/home a real directory. ", + "url": "/ostree/adapting-existing/#system-layout", + + "relUrl": "/adapting-existing/#system-layout" + },"24": { + "doc": "Adapting existing mainstream distributions", + "title": "Booting and initramfs technology", + "content": "OSTree comes with optional dracut+systemd integration code which follows this logic: . | Parse the ostree= kernel command line argument in the initramfs | Set up a read-only bind mount on /usr | Bind mount the deployment’s /sysroot to the physical / | Use mount(MS_MOVE) to make the deployment root appear to be the root filesystem | . After these steps, systemd switches root. If you are not using dracut or systemd, using OSTree should still be possible, but you will have to write the integration code. See the existing sources in src/switchroot as a reference. Patches to support other initramfs technologies and init systems, if sufficiently clean, will likely be accepted upstream. A further specific note regarding sysvinit: OSTree used to support recording device files such as the /dev/initctl FIFO, but no longer does. It’s recommended to just patch your initramfs to create this at boot. ", + "url": "/ostree/adapting-existing/#booting-and-initramfs-technology", + + "relUrl": "/adapting-existing/#booting-and-initramfs-technology" + },"25": { + "doc": "Adapting existing mainstream distributions", + "title": "/usr/lib/passwd", + "content": "Unlike traditional package systems, OSTree trees contain numeric uid and gids. Furthermore, it does not have a %post type mechanism where useradd could be invoked. In order to ship an OS that contains both system users and users dynamically created on client machines, you will need to choose a solution for /etc/passwd. The core problem is that if you add a user to the system for a daemon, the OSTree upgrade process for /etc will simply notice that because /etc/passwd differs from the previous default, it will keep the modified config file, and your new OS user will not be visible. The solution chosen for the Gnome Continuous operating system is to create /usr/lib/passwd, and to include a NSS module nss-altfiles which instructs glibc to read from it. Then, the build system places all system users there, freeing up /etc/passwd to be purely a database of local users. See also a more recent effort from Systemd Stateless . ", + "url": "/ostree/adapting-existing/#usrlibpasswd", + + "relUrl": "/adapting-existing/#usrlibpasswd" + },"26": { + "doc": "Adapting existing mainstream distributions", + "title": "Adapting existing package managers", + "content": "The largest endeavor is likely to be redesigning your distribution’s package manager to be on top of OSTree, particularly if you want to keep compatibility with the “old way” of installing into the physical /. This section will use examples from both dpkg and rpm as the author has familiarity with both; but the abstract concepts should apply to most traditional package managers. There are many levels of possible integration; initially, we will describe the most naive implementation which is the simplest but also the least efficient. We will assume here that the admin is booted into an OSTree-enabled system, and wants to add a set of packages. Many package managers store their state in /var; but since in the OSTree model that directory is shared between independent versions, the package database must first be found in the per-deployment /usr directory. It becomes read-only; remember, all upgrades involve constructing a new filesystem tree, so your package manager will also need to create a copy of its database. Most likely, if you want to continue supporting non-OSTree deployments, simply have your package manager fall back to the legacy /var location if the one in /usr is not found. To install a set of new packages (without removing any existing ones), enumerate the set of packages in the currently booted deployment, and perform dependency resolution to compute the complete set of new packages. Download and unpack these new packages to a temporary directory. Now, because we are merely installing new packages and not removing anything, we can make the major optimization of reusing our existing filesystem tree, and merely layering the composed filesystem tree of these new packages on top. A command like this: . ostree commit -b osname/releasename/description --tree=ref=$osname/$releasename/$description --tree=dir=/var/tmp/newpackages.13A8D0/ . will create a new commit in the $osname/$releasename/$description branch. The OSTree SHA256 checksum of all the files in /var/tmp/newpackages.13A8D0/ will be computed, but we will not re-checksum the present existing tree. In this layering model, earlier directories will take precedence, but files in later layers will silently override earlier layers. Then to actually deploy this tree for the next boot: ostree admin deploy $osname/$releasename/$description . This is essentially what rpm-ostree does to support its package layering model. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/adapting-existing/#adapting-existing-package-managers", + + "relUrl": "/adapting-existing/#adapting-existing-package-managers" + },"27": { + "doc": "Atomic Upgrades", + "title": "Atomic Upgrades", + "content": ". | You can turn off the power anytime you want… | Simple upgrades via HTTP | Upgrades via external tools (e.g. package managers) | Assembling a new deployment directory | Atomically swapping boot configuration | The bootversion | The /ostree/boot directory . | Licensing for this document: | . | . ", + "url": "/ostree/atomic-upgrades/", + + "relUrl": "/atomic-upgrades/" + },"28": { + "doc": "Atomic Upgrades", + "title": "You can turn off the power anytime you want…", + "content": "OSTree is designed to implement fully atomic and safe upgrades; more generally, atomic transitions between lists of bootable deployments. If the system crashes or you pull the power, you will have either the old system, or the new one. ", + "url": "/ostree/atomic-upgrades/#you-can-turn-off-the-power-anytime-you-want", + + "relUrl": "/atomic-upgrades/#you-can-turn-off-the-power-anytime-you-want" + },"29": { + "doc": "Atomic Upgrades", + "title": "Simple upgrades via HTTP", + "content": "First, the most basic model OSTree supports is one where it replicates pre-generated filesystem trees from a server over HTTP, tracking exactly one ref, which is stored in the .origin file for the deployment. The command ostree admin upgrade implements this. To begin a simple upgrade, OSTree fetches the contents of the ref from the remote server. Suppose we’re tracking a ref named exampleos/buildmain/x86_64-runtime. OSTree fetches the URL http://example.com/repo/refs/heads/exampleos/buildmain/x86_64-runtime, which contains a SHA256 checksum. This determines the tree to deploy, and /etc will be merged from currently booted tree. If we do not have this commit, then we perform a pull process. At present (without static deltas), this involves quite simply just fetching each individual object that we do not have, asynchronously. Put in other words, we only download changed files (zlib-compressed). Each object has its checksum validated and is stored in /ostree/repo/objects/. Once the pull is complete, we have downloaded all the objects that we need to perform a deployment. ", + "url": "/ostree/atomic-upgrades/#simple-upgrades-via-http", + + "relUrl": "/atomic-upgrades/#simple-upgrades-via-http" + },"30": { + "doc": "Atomic Upgrades", + "title": "Upgrades via external tools (e.g. package managers)", + "content": "As mentioned in the introduction, OSTree is also designed to allow a model where filesystem trees are computed on the client. It is completely agnostic as to how those trees are generated; they could be computed with traditional packages, packages with post-deployment scripts on top, or built by developers directly from revision control locally, etc. At a practical level, most package managers today (dpkg and rpm) operate “live” on the currently booted filesystem. The way they could work with OSTree is to, instead, take the list of installed packages in the currently booted tree, and compute a new filesystem from that. A later chapter describes in more details how this could work: Adapting Existing Systems. For the purposes of this section, let’s assume that we have a newly generated filesystem tree stored in the repo (which shares storage with the existing booted tree). We can then move on to checking it back out of the repo into a deployment. ", + "url": "/ostree/atomic-upgrades/#upgrades-via-external-tools-eg-package-managers", + + "relUrl": "/atomic-upgrades/#upgrades-via-external-tools-eg-package-managers" + },"31": { + "doc": "Atomic Upgrades", + "title": "Assembling a new deployment directory", + "content": "Given a commit to deploy, OSTree first allocates a directory for it. This is of the form /boot/loader/entries/ostree-$stateroot-$checksum.$serial.conf. The $serial is normally 0, but if a given commit is deployed more than once, it will be incremented. This is supported because the previous deployment may have configuration in /etc that we do not want to use or overwrite. Now that we have a deployment directory, a 3-way merge is performed between the (by default) currently booted deployment’s /etc, its default configuration, and the new deployment (based on its /usr/etc). How it works is: . | Files in the currently booted deployment’s /etc which were modified from the default /usr/etc (of the same deployment) are retained. | Files in the currently booted deployment’s /etc which were not modified from the default /usr/etc (of the same deployment) are upgraded to the new defaults from the new deployment’s /usr/etc. | . Roughly, this means that as soon as you modify or add a file in /etc, this file will be propagated forever as is (though there is a corner-case, where if your modification eventually exactly matches a future default file, then the file will go back to following future default updates from that point on). You can use ostree admin config-diff to see the differences between your booted deployment’s /etc and the OSTree defaults. A command like diff {/usr,}/etc will additional print line-level differences. ", + "url": "/ostree/atomic-upgrades/#assembling-a-new-deployment-directory", + + "relUrl": "/atomic-upgrades/#assembling-a-new-deployment-directory" + },"32": { + "doc": "Atomic Upgrades", + "title": "Atomically swapping boot configuration", + "content": "At this point, a new deployment directory has been created as a hardlink farm; the running system is untouched, and the bootloader configuration is untouched. We want to add this deployment to the “deployment list”. To support a more general case, OSTree supports atomic transitioning between arbitrary sets of deployments, with the restriction that the currently booted deployment must always be in the new set. In the normal case, we have exactly one deployment, which is the booted one, and we want to add the new deployment to the list. A more complex command might allow creating 100 deployments as part of one atomic transaction, so that one can set up an automated system to bisect across them. ", + "url": "/ostree/atomic-upgrades/#atomically-swapping-boot-configuration", + + "relUrl": "/atomic-upgrades/#atomically-swapping-boot-configuration" + },"33": { + "doc": "Atomic Upgrades", + "title": "The bootversion", + "content": "OSTree allows swapping between boot configurations by implementing the “swapped directory pattern” in /boot. This means it is a symbolic link to one of two directories /ostree/boot.[0|1]. To swap the contents atomically, if the current version is 0, we create /ostree/boot.1, populate it with the new contents, then atomically swap the symbolic link. Finally, the old contents can be garbage collected at any point. ", + "url": "/ostree/atomic-upgrades/#the-bootversion", + + "relUrl": "/atomic-upgrades/#the-bootversion" + },"34": { + "doc": "Atomic Upgrades", + "title": "The /ostree/boot directory", + "content": "However, we want to optimize for the case where the set of kernel/initramfs/devicetree sets is the same between both the old and new deployment lists. This happens when doing an upgrade that does not include the kernel; think of a simple translation update. OSTree optimizes for this case because on some systems /boot may be on a separate medium such as flash storage not optimized for significant amounts of write traffic. Related to this, modern OSTree has support for having /boot be a read-only mount by default - it will automatically remount read-write just for the portion of time necessary to update the bootloader configuration. To implement this, OSTree also maintains the directory /ostree/boot.$bootversion, which is a set of symbolic links to the deployment directories. The $bootversion here must match the version of /boot. However, in order to allow atomic transitions of this directory, this is also a swapped directory, so just like /boot, it has a version of 0 or 1 appended. Each bootloader entry has a special ostree= argument which refers to one of these symbolic links. This is parsed at runtime in the initramfs. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/atomic-upgrades/#the-ostreeboot-directory", + + "relUrl": "/atomic-upgrades/#the-ostreeboot-directory" + },"35": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Writing a buildsystem and managing repositories", + "content": ". | Build vs buy | Initializing | Writing your own OSTree buildsystem | Constructing trees from unions | Migrating content between repositories | More sophisticated repository management . | Licensing for this document: | . | . OSTree is not a package system. It does not directly support building source code. Rather, it is a tool for transporting and managing content, along with package-system independent aspects like bootloader management for updates. We’ll assume here that we’re planning to generate commits on a build server, then have client systems replicate it. Doing client-side assembly is also possible of course, but this discussion will focus primarily on server-side concerns. ", + "url": "/ostree/buildsystem-and-repos/", + + "relUrl": "/buildsystem-and-repos/" + },"36": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Build vs buy", + "content": "Therefore, you need to either pick an existing tool for writing content into an OSTree repository, or write your own. An example tool is rpm-ostree - it takes as input RPMs, and commits them (currently oriented for server-side, but aiming to do client-side too). ", + "url": "/ostree/buildsystem-and-repos/#build-vs-buy", + + "relUrl": "/buildsystem-and-repos/#build-vs-buy" + },"37": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Initializing", + "content": "For this initial discussion, we’re assuming you have a single archive repository: . mkdir repo ostree --repo=repo init --mode=archive . You can export this via a static webserver, and configure clients to pull from it. ", + "url": "/ostree/buildsystem-and-repos/#initializing", + + "relUrl": "/buildsystem-and-repos/#initializing" + },"38": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Writing your own OSTree buildsystem", + "content": "There exist many, many systems that basically follow this pattern: . $pkg --installroot=/path/to/tmpdir install foo bar baz $imagesystem commit --root=/path/to/tmpdir . For various values of $pkg such as yum, apt-get, etc., and values of $imagesystem could be simple tarballs, Amazon Machine Images, ISOs, etc. Now obviously in this document, we’re going to talk about the situation where $imagesystem is OSTree. The general idea with OSTree is that wherever you might store a series of tarballs for applications or OS images, OSTree is likely going to be better. For example, it supports GPG signatures, binary deltas, writing bootloader configuration, etc. OSTree does not include a package/component build system simply because there already exist plenty of good ones - rather, it is intended to provide an infrastructure layer. The above mentioned rpm-ostree compose tree chooses RPM as the value of $pkg - so binaries are built as RPMs, then committed as a whole into an OSTree commit. But let’s discuss building our own. If you’re just experimenting, it’s quite easy to start with the command line. We’ll assume for this purpose that you have a build process that outputs a directory tree - we’ll call this tool $pkginstallroot (which could be yum --installroot or debootstrap, etc.). Your initial prototype is going to look like: . $pkginstallroot /path/to/tmpdir ostree --repo=repo commit -s 'build' -b exampleos/x86_64/standard --tree=dir=/path/to/tmpdir . Alternatively, if your build system can generate a tarball, you can commit that tarball into OSTree. For example, OpenEmbedded can output a tarball, and one can commit it via: . ostree commit -s 'build' -b exampleos/x86_64/standard --tree=tar=myos.tar . ", + "url": "/ostree/buildsystem-and-repos/#writing-your-own-ostree-buildsystem", + + "relUrl": "/buildsystem-and-repos/#writing-your-own-ostree-buildsystem" + },"39": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Constructing trees from unions", + "content": "The above is a very simplistic model, and you will quickly notice that it’s slow. This is because OSTree has to re-checksum and recompress the content each time it’s committed. (Most of the CPU time is spent in compression which gets thrown away if the content turns out to be already stored). A more advanced approach is to store components in OSTree itself, then union them, and recommit them. At this point, we recommend taking a look at the OSTree API, and choose a programming language supported by GObject Introspection to write your buildsystem scripts. Python may be a good choice, or you could choose custom C code, etc. For the purposes of this tutorial we will use shell script, but it’s strongly recommended to choose a real programming language for your build system. Let’s say that your build system produces separate artifacts (whether those are RPMs, zip files, or whatever). These artifacts should be the result of make install DESTDIR= or similar. Basically equivalent to RPMs/debs. Further, in order to make things fast, we will need a separate bare-user repository in order to perform checkouts quickly via hardlinks. We’ll then export content into the archive repository for use by client systems. mkdir build-repo ostree --repo=build-repo init --mode=bare-user . You can begin committing those as individual branches: . ostree --repo=build-repo commit -b exampleos/x86_64/bash --tree=tar=bash-4.2-bin.tar.gz ostree --repo=build-repo commit -b exampleos/x86_64/systemd --tree=tar=systemd-224-bin.tar.gz . Set things up so that whenever a package changes, you redo the commit with the new package version - conceptually, the branch tracks the individual package versions over time, and defaults to “latest”. This isn’t required - one could also include the version in the branch name, and have metadata outside to determine “latest” (or the desired version). Now, to construct our final tree: . rm -rf exampleos-build for package in bash systemd; do ostree --repo=build-repo checkout -U --union exampleos/x86_64/${package} exampleos-build done # Set up a \"rofiles-fuse\" mount point; this ensures that any processes # we run for post-processing of the tree don't corrupt the hardlinks. mkdir -p mnt rofiles-fuse exampleos-build mnt # Now run global \"triggers\", generate cache files: ldconfig -r mnt (Insert other programs here) fusermount -u mnt ostree --repo=build-repo commit -b exampleos/x86_64/standard --link-checkout-speedup exampleos-build . There are a number of interesting things going on here. The major architectural change is that we’re using --link-checkout-speedup. This is a way to tell OSTree that our checkout is made via hardlinks, and to scan the repository in order to build up a reverse (device, inode) -> checksum mapping. In order for this mapping to be accurate, we needed the rofiles-fuse to ensure that any changed files had new inodes (and hence a new checksum). ", + "url": "/ostree/buildsystem-and-repos/#constructing-trees-from-unions", + + "relUrl": "/buildsystem-and-repos/#constructing-trees-from-unions" + },"40": { + "doc": "Writing a buildsystem and managing repositories", + "title": "Migrating content between repositories", + "content": "Now that we have content in our build-repo repository (in bare-user mode), we need to move the exampleos/x86_64/standard branch content into the repository just named repo (in archive mode) for export, which will involve zlib compression of new objects. We likely want to generate static deltas after that as well. Let’s copy the content: . ostree --repo=repo pull-local build-repo exampleos/x86_64/standard . Clients can now incrementally download new objects - however, this would also be a good time to generate a delta from the previous commit. ostree --repo=repo static-delta generate exampleos/x86_64/standard . ", + "url": "/ostree/buildsystem-and-repos/#migrating-content-between-repositories", + + "relUrl": "/buildsystem-and-repos/#migrating-content-between-repositories" + },"41": { + "doc": "Writing a buildsystem and managing repositories", + "title": "More sophisticated repository management", + "content": "Next, see Repository Management for the next steps in managing content in OSTree repositories. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/buildsystem-and-repos/#more-sophisticated-repository-management", + + "relUrl": "/buildsystem-and-repos/#more-sophisticated-repository-management" + },"42": { + "doc": "OSTree Contributing Tutorial", + "title": "OSTree Contributing Tutorial", + "content": "The following guide is about OSTree forking, building, adding a command, testing the command, and submitting the change. | Getting Started | Building OSTree . | Install Build Dependencies | OSTree Build Commands . | Notes | Tip | . | . | Testing a Build . | Testing in a Container | Testing in a Virtual Machine | . | Tutorial: Adding a basic builtin command to ostree . | Modifying OSTree | Adding a new API function to libostree | OSTree Tests | Submitting a Patch | Returning Workflow | . | . ", + "url": "/ostree/contributing-tutorial/", + + "relUrl": "/contributing-tutorial/" + },"43": { + "doc": "OSTree Contributing Tutorial", + "title": "Getting Started", + "content": "Fork https://github.com/ostreedev/ostree, then run the following commands. $ git clone https://github.com/<username>/ostree && cd ostree $ git remote add upstream https://github.com/ostreedev/ostree $ git checkout main $ git fetch upstream && git branch --set-upstream-to=upstream/main main . Make a branch from main for your patch. $ git checkout -b <name-of-branch> $ git branch --set-upstream-to=upstream/main <name-of-branch> . ", + "url": "/ostree/contributing-tutorial/#getting-started", + + "relUrl": "/contributing-tutorial/#getting-started" + },"44": { + "doc": "OSTree Contributing Tutorial", + "title": "Building OSTree", + "content": "Install Build Dependencies . Execute one of the following group commands as superuser depending on your machine’s package manager. For Fedora: . $ dnf install @buildsys-build dnf-plugins-core && \\ dnf builddep ostree . For CentOS: . $ yum install yum-utils dnf-plugins-core && \\ yum-builddep ostree . For Debian based distros: . $ apt-get update && \\ apt-get install build-essential && \\ apt-get build-dep ostree . build.sh will have a list of packages needed to build ostree. OSTree Build Commands . These are the basic commands to build OSTree. Depending on the OS that OSTree will be built for, the flags or options for ./autogen.sh and ./configure will vary. See ostree-build.sh in this tutorial below for specific commands to building OSTree for Fedora 28 and Fedora 28 Atomic Host. # optional: autogen.sh will run this if necessary git submodule update --init env NOCONFIGURE=1 ./autogen.sh # run ./configure if makefile does not exist ./configure make make install DESTDIR=/path/to/install/binary . Notes . Running git submodule update --init is optional since autogen.sh will check to see if one of the submodule files for example from libglnx/ or from bsdiff/ exists. Additionally, autogen.sh will check to see if the environment variable NOCONFIGURE is set. To run ./configure manually, run autogen in a modified environment as such, env NOCONFIGURE=1 ./autogen.sh. Otherwise, leave NOCONFIGURE empty and autogen.sh will run ./configure as part of the autogen.sh command when it executes. For more information on --prefix see Variables for Installation Directories. make install will generate files for /bin and /lib. If DESTDIR is unspecified then OSTree will be installed in the default directory i.e. /usr/local/bin and its static libraries in /usr/local/lib. Note that the /usr/local portion of the path can be changed using the --prefix option for ./configure. See this GNU guide on DESTDIR Staged Installs for more information. Tip . Make allows parallel execution of recipes. Use make -j<N> to speed up the build. <N> is typically $((2 * $(nproc))) for optimal performance, where nproc is the number of processing units (CPU cores) available. See page 106 of the GNU Make Manual for more information about the --jobs or -j option. ", + "url": "/ostree/contributing-tutorial/#building-ostree", + + "relUrl": "/contributing-tutorial/#building-ostree" + },"45": { + "doc": "OSTree Contributing Tutorial", + "title": "Testing a Build", + "content": "It is best practice to build software (definitely including ostree) in a container or virtual machine first. Testing in a Container . There are a variety of container engines available and many distributions have pre-packaged versions of e.g. Podman and Docker. If you choose to use Docker upstream, you may want to follow this post-installation guide for Docker. This will allow you to run Docker as a non-root user on a Linux based host machine. You will need to have pushed a remote git branch $REMOTE_BRANCH (see ostree-git.sh below) in order to pull your changes into a container. The example below uses Docker to manage containers. Save the contents of this Dockerfile somewhere on your machine: . # this pulls the fedora 28 image FROM registry.fedoraproject.org/fedora:28 # install ostree dependencies RUN dnf update -y && \\ dnf -y install @buildsys-build dnf-plugins-core && \\ dnf -y builddep ostree && \\ dnf clean all # clone ostree and update main branch COPY ostree-git.sh / RUN ../ostree-git.sh # builds ostree + any additional commands COPY ostree-build.sh / # entry into the container will start at this directory WORKDIR /ostree # run the following as `/bin/sh -c` # or enter the container to execute ./ostree-build.sh RUN ../ostree-build.sh . Save the following bash scripts in the same directory as the Dockerfile. Then change the mode bit of these files so that they are executable, by running chmod +x ostree-git.sh ostree-build.sh . #!/bin/bash # ostree-git.sh # Clone ostree and update main branch set -euo pipefail # Set $USERNAME to your GitHub username here. USERNAME=\"\" # clone your fork of the OSTree repo, this will be in the \"/\" directory git clone https://github.com/$USERNAME/ostree.git cd ostree # Add upstream as remote and update main branch git checkout main git remote add upstream https://github.com/ostreedev/ostree.git git pull --rebase upstream main . #!/bin/bash # ostree-build.sh # Build and test OSTree set -euo pipefail # $REMOTE_BRANCH is the name of the remote branch in your # repository that contains changes (e.g. my-patch). REMOTE_BRANCH=\"\" # fetch updates from origin # origin url should be your forked ostree repository git fetch origin # go to branch with changes # if this branch already exists then checkout that branch exit_code=\"$(git checkout --track origin/$REMOTE_BRANCH; echo $?)\" if [[ \"$exit_code\" == 1 ]] then echo \"This branch:\" $REMOTE_BRANCH \"is not a remote branch.\" exit fi # make sure branch with changes is up-to-date git pull origin $REMOTE_BRANCH # build OSTree commands for Fedora 28 and Fedora 28 Atomic Host ./autogen.sh --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc ./configure --prefix=/usr make -j$((2 * $(nproc))) make install # any additional commands go here . Build the container . Run docker build in the same directory of the Dockerfile like so: . $ docker build -t ostree-fedora-test . When this build is done, the -t option tags the image as ostree-fedora-test. Note: Do not forget the dot . at the end of the above command which specifies the location of the Dockerfile. You will see ostree-fedora-test listed when running docker images: . REPOSITORY TAG IMAGE ID CREATED SIZE ostree-fedora-test latest 817c04cc3656 1 day ago 978MB . Entering the Container . To start the ostree-fedora-test container, run: . $ docker run -it --rm --entrypoint /bin/sh --name ostree-testing ostree-fedora-test . Note: . --rm option tells Docker to automatically clean up the container and remove the file system when the container exits. Otherwise remove it manually by running docker rm <container name>. The state of the container will not be saved when the shell prompt exits. Best practice is modify the Dockerfile to modify the image. Testing in a Container Workflow . | Edit the changes to OSTree on your local machine. | git add to stage the changed files, git commit and then git push origin <local-branch>:<remote-branch>. | Testing on a new container vs. Testing on an existing container: . If the ostree-testing container was newly built right after your changes have been committed, then the container’s build of ostree should contain your changes. Else: Within the ostree-testing container, run ../ostree-build.sh in the ostree directory. This will pull in changes from your branch and create a new ostree build. | make install will install OSTree in the default location i.e. /usr/..in a Fedora 28 container. | Test ostree. | . Testing in a Virtual Machine . To create a Fedora 28 Atomic Host Vagrant VM, run the following commands: . $ mkdir atomic && cd atomic $ vagrant init fedora/28-atomic-host && vagrant up . An option is to use rsync to transfer ostree files to a Vagrant VM. To find the IP address of a Vagrant VM, run vagrant ssh-config in the same directory as the Vagrantfile. Steps to rsync files to test an ostree build: . | Copy the contents of your public ssh key on your host machine e.g. id_rsa.pub to /home/vagrant/.ssh/authorized_keys on the VM. | Run sudo su, followed by ssh localhost then press Ctrl+c to exit from the decision prompt. This will create the .ssh directory with the right permissions. | Using Vagrant as the user, run sudo cp ~/.ssh/authorized_keys /root/.ssh/. So that user root has the the same login credentials. | To override the Read-only file system warning, run sudo ostree admin unlock. | <ostree-install-dir> will serve as the local install location for ostree and the path to this directory should be absolute when specified in DESTDIR. | Set rsync to sync changes in /etc and /usr from <ostree-install-dir>/ on the host to the VM: . $ rsync -av <ostree-install-dir>/etc/ root@<ip-address>:/etc $ rsync -av <ostree-install-dir>/usr/ root@<ip-address>:/usr . Using option -n will execute the commands as a trial, which is helpful to list the files that will be synced. | Run the commands in step 6 each time a new ostree build is executed to update the change. Running ls -lt in the directory where the changed file is expected, is a simple way to check when a particular file was last modified. Proceed to the test changes ostree with the most recent changes. | . ", + "url": "/ostree/contributing-tutorial/#testing-a-build", + + "relUrl": "/contributing-tutorial/#testing-a-build" + },"46": { + "doc": "OSTree Contributing Tutorial", + "title": "Tutorial: Adding a basic builtin command to ostree", + "content": "Modifying OSTree . This will add a command which prints Hello OSTree! when ostree hello-ostree is entered. | Create a file in src/ostree named ot-builtin-hello-ostree.c. Code that lives in here belongs to OSTree, and uses functionality from libostree. | Add the following to ot-builtin-hello-ostree.c: . #include \"config.h\" #include \"ot-main.h\" #include \"ot-builtins.h\" #include \"ostree.h\" #include \"otutil.h\" // Structure for options such as ostree hello-ostree --option. static GOptionEntry options[] = { { NULL }, }; gboolean ostree_builtin_hello_ostree (int argc, char **argv, OstreeCommandInvocation *invocation, GCancellable *cancellable, GError **error) { g_autoptr(OstreeRepo) repo = NULL; // Creates new command context, ready to be parsed. // Input to g_option_context_new shows when running ostree <command> --help g_autoptr(GOptionContext) context = g_option_context_new (\"\"); // Parses the command context according to the ostree CLI. if (!ostree_option_context_parse (context, options, &argc, &argv, invocation, &repo, cancellable, error)) return FALSE; g_print(\"Hello OSTree!\\n\"); return TRUE; } . This defines the functionality for hello-ostree. Now we have to make sure the CLI can refer to the execution function, and that autotools knows to build this file. Note: libostree codebase supports C99 features. | Add the following in src/ostree/main.c: . { \"hello-ostree\", // The name of the command OSTREE_BUILTIN_FLAG_NO_REPO, // Flag not to require the `--repo` argument, see \"ot-main.h\" ostree_builtin_hello_ostree, // Execution function for the command \"Print hello message\" }, // Short description to appear when `ostree hello-ostree --help` is entered . | Add a macro for the function declaration of ostree_builtin_hello_ostree, in ot-builtins.h: . BUILTINPROTO(hello_ostree); . This makes the function definition visible to the CLI. | Configure automake to include ot-builtin-hello-ostree.c in the build, by adding an entry in Makefile-ostree.am under ostree_SOURCES: . src/ostree/ot-builtin-hello-ostree.c \\ . | Rebuild ostree: . $ make && make install DESTDIR=/path/to/install/the/content . | Execute the new ostree binary, from where you installed it: . $ ostree hello-ostree Hello OSTree! . | . Adding a new API function to libostree . This will add a new API function ostree_kernel_args_foo() in src/libostree/ostree-kernel-args.c. | Add the following to src/libostree/ostree-kernel-args.h: _OSTREE_PUBLIC gboolean ostree_kernel_args_foo (const char *arg, GCancellable *cancellable, GError **error); . | Add the following to ostree-kernel-args.c: /** * ostree_kernel_args_foo: * @arg: Description of the arg * * Description of the function * * Since: $NEWVERSION //The new libostree version, for example 2022.5 **/ gboolean ostree_kernel_args_foo (const char *arg, GCancellable *cancellable, GError **error) { //Add code here } . | Add the following to src/libostree/libostree-devel.sym: LIBOSTREE_$NEWVERSION { // The new libostree version global: ostree_kernel_args_foo; // Function name } LIBOSTREE_$LASTSTABLE; // The last stable libostree version . | Add the following to Makefile-libostree.am: if BUILDOPT_IS_DEVEL_BUILD symbol_files += $(top_srcdir)/src/libostree/libostree-devel.sym endif . | Add function name ostree_kernel_args_foo to apidoc/ostree-sections.txt under <FILE>ostree-kernel-args</FILE>. | Call function ostree_kernel_args_foo() in your code. | . OSTree Tests . Tests for OSTree are done by shell scripting, by running OSTree commands and examining output. These steps will go through adding a test for hello-ostree. | Create a file in tests called test-hello-ostree.sh. | Add the following to test-hello-ostree.sh: . set -euo pipefail # Ensure the test will not silently fail . $(dirname $0)/libtest.sh # Make libtest.sh functions available echo \"1..1\" # Declare which test is being run out of how many pushd ${test_tmpdir} ${CMD_PREFIX} ostree hello-ostree > hello-output.txt assert_file_has_content hello-output.txt \"Hello OSTree!\" popd echo \"ok hello ostree\" # Indicate test success . Many tests require a fake repository setting up (as most OSTree commands require --repo to be specified). See test-pull-depth.sh for an example of this setup. | Configure automake to include test-hello-ostree.sh in the build, by adding an entry in Makefile-tests.am under _installed_or_uninstalled_test_scripts: . tests/test-hello-ostree.sh \\ . | Make sure test-hello-ostree.sh has executable permissions! . $ chmod +x tests/test-hello-ostree.sh . | Run the test: . $ make check TESTS=\"tests/test-hello-ostree.sh\" . Multiple tests can be specified: make check TESTS=\"test1 test2 ...\". To run all tests, use make check. Hopefully, the test passes! The following will be printed: . PASS: tests/test-hello-ostree.sh 1 hello ostree ============================================================================ Testsuite summary for libostree 2018.8 ============================================================================ # TOTAL: 1 # PASS: 1 # SKIP: 0 # XFAIL: 0 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ . | . Submitting a Patch . After you have committed your changes and tested, you are ready to submit your patch! . You should make sure your commits are placed on top of the latest changes from upstream/main: . $ git pull --rebase upstream main . To submit your patch, open a pull request from your forked repository. Most often, you’ll be merging into ostree:main from <username>:<branch name>. If some of your changes are complete and you would like feedback, you may also open a pull request that has WIP (Work In Progress) in the title. Before a pull request is considered merge ready, your commit messages should fall within the specified guideline. See Commit message style. See CONTRIBUTING.md for information on squashing commits, and alternative options to submit patches. Returning Workflow . When returning to work on a patch, it is recommended to update your fork with the latest changes in the upstream main branch. If creating a new branch: . $ git checkout main $ git pull upstream main $ git checkout -b <name-of-patch> . If continuing on a branch already created: . $ git checkout <name-of-patch> $ git pull --rebase upstream main . ", + "url": "/ostree/contributing-tutorial/#tutorial-adding-a-basic-builtin-command-to-ostree", + + "relUrl": "/contributing-tutorial/#tutorial-adding-a-basic-builtin-command-to-ostree" + },"47": { + "doc": "Deployments", + "title": "Deployments", + "content": ". | Overview . | “stateroot” (AKA “osname”): Group of deployments that share /var | Contents of a deployment | Staged deployments | The system /boot . | Licensing for this document: | . | . | . ", + "url": "/ostree/deployment/", + + "relUrl": "/deployment/" + },"48": { + "doc": "Deployments", + "title": "Overview", + "content": "Built on top of the OSTree versioning filesystem core is a layer that knows how to deploy, parallel install, and manage Unix-like operating systems (accessible via ostree admin). The core content of these operating systems are treated as read-only, but they transparently share storage. A deployment is physically located at a path of the form /ostree/deploy/$stateroot/deploy/$checksum. OSTree is designed to boot directly into exactly one deployment at a time; each deployment is intended to be a target for chroot() or equivalent. “stateroot” (AKA “osname”): Group of deployments that share /var . Each deployment is grouped in exactly one “stateroot” (also known as an “osname”); the former term is preferred. From above, you can see that a stateroot is physically represented in the /ostree/deploy/$stateroot directory. For example, OSTree can allow parallel installing Debian in /ostree/deploy/debian and Red Hat Enterprise Linux in /ostree/deploy/rhel (subject to operating system support, present released versions of these operating systems may not support this). Each stateroot has exactly one copy of the traditional Unix /var, stored physically in /ostree/deploy/$stateroot/var. OSTree provides support tools for systemd to create a Linux bind mount that ensures the booted deployment sees the shared copy of /var. OSTree does not touch the contents of /var. Operating system components such as daemon services are required to create any directories they require there at runtime (e.g. /var/cache/$daemonname), and to manage upgrading data formats inside those directories. Contents of a deployment . A deployment begins with a specific commit (represented as a SHA256 hash) in the OSTree repository in /ostree/repo. This commit refers to a filesystem tree that represents the underlying basis of a deployment. For short, we will call this the “tree”, to distinguish it from the concept of a deployment. First, the tree must include a kernel (and optionally an initramfs). The current standard locations for these are /usr/lib/modules/$kver/vmlinuz and /usr/lib/modules/$kver/initramfs.img. The “boot checksum” will be computed automatically. This follows the current Fedora kernel layout, and is the current recommended path. However, older versions of libostree don’t support this; you may need to also put kernels in the previous (legacy) paths, which are vmlinuz(-.*)?-$checksum in either /boot or /usr/lib/ostree-boot. The checksum should be a SHA256 hash of the kernel contents; it must be pre-computed before storing the kernel in the repository. Optionally, the directory can also contain an initramfs, stored as initramfs(-.*)?-$checksum and/or a device tree, stored as devicetree(-.*)?-$checksum. If an initramfs or devicetree exist, the checksum must include all of the kernel, initramfs and devicetree contents. OSTree will use this to determine which kernels are shared. The rationale for this is to avoid computing checksums on the client by default. The deployment should not have a traditional UNIX /etc; instead, it should include /usr/etc. This is the “default configuration”. When OSTree creates a deployment, it performs a 3-way merge using the old default configuration, the active system’s /etc, and the new default configuration. In the final filesystem tree for a deployment then, /etc is a regular writable directory. Besides the exceptions of /var and /etc then, the rest of the contents of the tree are checked out as hard links into the repository. It’s strongly recommended that operating systems ship all of their content in /usr, but this is not a hard requirement. Finally, a deployment may have a .origin file, stored next to its directory. This file tells ostree admin upgrade how to upgrade it. At the moment, OSTree only supports upgrading a single refspec. However, in the future OSTree may support a syntax for composing layers of trees, for example. Staged deployments . As mentioned above, when OSTree creates a new deployment, a 3-way merge is done to update its /etc. Depending on the nature of the system, this can cause an issue: if a user or program modifies the booted /etc after the pending deployment is created but before rebooting, those modifications will be lost. OSTree does not do a second /etc merge on reboot. To counter this, OSTree supports staged deployments. In this flow, deployments are created using e.g. ostree admin upgrade --stage on the CLI. The new deployment is still created when the command is invoked, but the 3-way /etc merge is delayed until the system is rebooted or shut down. Additionally, updating the bootloader is also delayed. This is done by the ostree-finalize-staged.service systemd unit. The main disadvantage of this approach is that rebooting can take longer and the failure mode can be confusing (the machine will reboot into the same deployment). In systems where the workload is well-understood and not subject to the /etc issue above, it may be better to not stage deployments. The system /boot . While OSTree parallel installs deployments cleanly inside the /ostree directory, ultimately it has to control the system’s /boot directory. The way this works is via the Boot Loader Specification, which is a standard for bootloader-independent drop-in configuration files. When a tree is deployed, it will have a configuration file generated of the form /boot/loader/entries/ostree-$stateroot-$checksum.$serial.conf. This configuration file will include a special ostree= kernel argument that allows the initramfs to find (and chroot() into) the specified deployment. At present, not all bootloaders implement the BootLoaderSpec, so OSTree contains code for some of these to regenerate native config files (such as /boot/syslinux/syslinux.conf) based on the entries. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/deployment/#overview", + + "relUrl": "/deployment/#overview" + },"49": { + "doc": "OSTree data formats", + "title": "OSTree data formats", + "content": ". | On the topic of “smart servers” | The archive format | archive efficiency | Aside: bare formats | Static deltas | Static delta repository layout | Static delta internal structure . | The delta superblock | . | A delta part | Fallback objects . | Licensing for this document: | . | . ", + "url": "/ostree/formats/", + + "relUrl": "/formats/" + },"50": { + "doc": "OSTree data formats", + "title": "On the topic of “smart servers”", + "content": "One really crucial difference between OSTree and git is that git has a “smart server”. Even when fetching over https://, it isn’t just a static webserver, but one that e.g. dynamically computes and compresses pack files for each client. In contrast, the author of OSTree feels that for operating system updates, many deployments will want to use simple static webservers, the same target most package systems were designed to use. The primary advantages are security and compute efficiency. Services like Amazon S3 and CDNs are a canonical target, as well as a stock static nginx server. ", + "url": "/ostree/formats/#on-the-topic-of-smart-servers", + + "relUrl": "/formats/#on-the-topic-of-smart-servers" + },"51": { + "doc": "OSTree data formats", + "title": "The archive format", + "content": "In the repo section, the concept of objects was introduced, where file/content objects are checksummed and managed individually. (Unlike a package system, which operates on compressed aggregates). The archive format simply gzip-compresses each content object. Metadata objects are stored uncompressed. This means that it’s easy to serve via static HTTP. Note: the repo config file still uses the historical term archive-z2 as mode. But this essentially indicates the modern archive format. When you commit new content, you will see new .filez files appearing in objects/. ", + "url": "/ostree/formats/#the-archive-format", + + "relUrl": "/formats/#the-archive-format" + },"52": { + "doc": "OSTree data formats", + "title": "archive efficiency", + "content": "The advantages of archive: . | It’s easy to understand and implement | Can be served directly over plain HTTP by a static webserver | Clients can download/unpack updates incrementally | Space efficient on the server | . The biggest disadvantage of this format is that for a client to perform an update, one HTTP request per changed file is required. In some scenarios, this actually isn’t bad at all, particularly with techniques to reduce HTTP overhead, such as HTTP/2. In order to make this format work well, you should design your content such that large data that changes infrequently (e.g. graphic images) are stored separately from small frequently changing data (application code). Other disadvantages of archive: . | It’s quite bad when clients are performing an initial pull (without HTTP/2), | One doesn’t know the total size (compressed or uncompressed) of content before downloading everything | . ", + "url": "/ostree/formats/#archive-efficiency", + + "relUrl": "/formats/#archive-efficiency" + },"53": { + "doc": "OSTree data formats", + "title": "Aside: bare formats", + "content": "The most common operation is to pull from a remote archive repository into a local one. This latter is not compressed on disk. In other words, pulling to a local repository is similar to unpacking (but not installing) the content of an RPM/deb package. The bare repository format is the simplest one. In this mode regular files are directly stored to disk, and all metadata (e.g. uid/gid and xattrs) is reflected to the filesystem. It allows further direct access to content and metadata, but it may require elevated privileges when writing objects to the repository. The bare-user format is a bit special in that the uid/gid and xattrs from the content are ignored. This is primarily useful if you want to have the same OSTree-managed content that can be run on a host system or an unprivileged container. Similarly, the bare-split-xattrs format is a special mode where xattrs are stored as separate repository objects, and not directly reflected to the filesystem. This is primarily useful when transporting xattrs through lossy environments (e.g. tar streams and containerized environments). It also allows carrying security-sensitive xattrs (e.g. SELinux labels) out-of-band without involving OS filesystem logic. ", + "url": "/ostree/formats/#aside-bare-formats", + + "relUrl": "/formats/#aside-bare-formats" + },"54": { + "doc": "OSTree data formats", + "title": "Static deltas", + "content": "OSTree itself was originally focused on a continuous delivery model, where client systems are expected to update regularly. However, many OS vendors would like to supply content that’s updated e.g. once a month or less often. For this model, we can do a lot better to support batched updates than a basic archive repo. However, we still want to preserve the model of “static webserver only”. Given this, OSTree has gained the concept of a “static delta”. These deltas are targeted to be a delta between two specific commit objects, including “bsdiff” and “rsync-style” deltas within a content object. Static deltas also support from NULL, where the client can more efficiently download a commit object from scratch - this is mostly useful when using OSTree for containers, rather than OS images. For OS images, one tends to download an installer ISO or qcow2 image which is a single file that contains the tree data already. Effectively, we’re spending server-side storage (and one-time compute cost), and gaining efficiency in client network bandwidth. ", + "url": "/ostree/formats/#static-deltas", + + "relUrl": "/formats/#static-deltas" + },"55": { + "doc": "OSTree data formats", + "title": "Static delta repository layout", + "content": "Since static deltas may not exist, the client first needs to attempt to locate one. Suppose a client wants to retrieve commit ${new} while currently running ${current}. In order to save space, these two commits are “modified base64” - the / character is replaced with _. Like the commit objects, a “prefix directory” is used to make management easier for filesystem tools. A delta is named $(mbase64 $from)-$(mbase64 $to), for example GpTyZaVut2jXFPWnO4LJiKEdRTvOw_mFUCtIKW1NIX0-L8f+VVDkEBKNc1Ncd+mDUrSVR4EyybQGCkuKtkDnTwk, which in SHA256 format is 1a94f265a56eb768d714f5a73b82c988a11d453bcec3f985502b48296d4d217d-2fc7fe5550e410128d73535c77e98352b495478132c9b4060a4b8ab640e74f09. Finally, the actual content can be found in deltas/$fromprefix/$fromsuffix-$to. ", + "url": "/ostree/formats/#static-delta-repository-layout", + + "relUrl": "/formats/#static-delta-repository-layout" + },"56": { + "doc": "OSTree data formats", + "title": "Static delta internal structure", + "content": "A delta is itself a directory. Inside, there is a file called superblock which contains metadata. The rest of the files will be integers bearing packs of content. The file format of static deltas should be currently considered an OSTree implementation detail. Obviously, nothing stops one from writing code which is compatible with OSTree today. However, we would like the flexibility to expand and change things, and having multiple codebases makes that more problematic. Please contact the authors with any requests. That said, one critical thing to understand about the design is that delta payloads are a bit more like “restricted programs” than they are raw data. There’s a “compilation” phase which generates output that the client executes. This “updates as code” model allows for multiple content generation strategies. The design of this was inspired by that of Chromium: ChromiumOS Autoupdate. The delta superblock . The superblock contains: . | arbitrary metadata | delta generation timestamp | the new commit object | An array of recursive deltas to apply | An array of per-part metadata, including total object sizes (compressed and uncompressed), | An array of fallback objects | . Let’s define a delta part, then return to discuss details: . ", + "url": "/ostree/formats/#static-delta-internal-structure", + + "relUrl": "/formats/#static-delta-internal-structure" + },"57": { + "doc": "OSTree data formats", + "title": "A delta part", + "content": "A delta part is a combination of a raw blob of data, plus a very restricted bytecode that operates on it. Say for example two files happen to share a common section. It’s possible for the delta compilation to include that section once in the delta data blob, then generate instructions to write out that blob twice when generating both objects. Realistically though, it’s very common for most of a delta to just be “stream of new objects” - if one considers it, it doesn’t make sense to have too much duplication inside operating system content at this level. So then, what’s more interesting is that OSTree static deltas support a per-file delta algorithm called bsdiff that most notably works well on executable code. The current delta compiler scans for files with matching basenames in each commit that have a similar size, and attempts a bsdiff between them. (It would make sense later to have a build system provide a hint for this - for example, files within a same package). A generated bsdiff is included in the payload blob, and applying it is an instruction. ", + "url": "/ostree/formats/#a-delta-part", + + "relUrl": "/formats/#a-delta-part" + },"58": { + "doc": "OSTree data formats", + "title": "Fallback objects", + "content": "It’s possible for there to be large-ish files which might be resistant to bsdiff. A good example is that it’s common for operating systems to use an “initramfs”, which is itself a compressed filesystem. This “internal compression” defeats bsdiff analysis. For these types of objects, the delta superblock contains an array of “fallback objects”. These objects aren’t included in the delta parts - the client simply fetches them from the underlying .filez object. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/formats/#fallback-objects", + + "relUrl": "/formats/#fallback-objects" + },"59": { + "doc": "Using Linux IMA with OSTree", + "title": "Using Linux IMA with OSTree", + "content": ". | Linux IMA | IMA signatures and OSTree checksum | Signing . | Generating a key | Signing a commit | Applying a policy | Linux EVM | . | Further references | . ", + "url": "/ostree/ima/", + + "relUrl": "/ima/" + },"60": { + "doc": "Using Linux IMA with OSTree", + "title": "Linux IMA", + "content": "The Linux Integrity Measurement Architecture provides a mechanism to cryptographically sign the digest of a regular file, and policies can be applied to e.g. require that code executed by the root user have a valid signed digest. The alignment between Linux IMA and ostree is quite strong. OSTree provides a content-addressable object store, where files are intended to be immutable. This is implemented with a basic read-only bind mount. While IMA does not actually prevent mutating files, any changed (or unsigned) files would (depending on policy) not be readable or executable. ", + "url": "/ostree/ima/#linux-ima", + + "relUrl": "/ima/#linux-ima" + },"61": { + "doc": "Using Linux IMA with OSTree", + "title": "IMA signatures and OSTree checksum", + "content": "Mechanically, IMA signatures appear as a security.ima extended attribute on the file. This is a signed digest of just the file content (and not any metadata) . OSTree’s checksums in contrast include not just the file content, but also metadata such as uid, gid and mode and extended attributes; . Together, this means that adding an IMA signature to a file in the OSTree model appears as a new object (with a new digest). A nice property is that this enables the transactional addition (or removal) of IMA signatures. However, adding IMA signatures to files that were previously unsigned also today duplicates disk space. ", + "url": "/ostree/ima/#ima-signatures-and-ostree-checksum", + + "relUrl": "/ima/#ima-signatures-and-ostree-checksum" + },"62": { + "doc": "Using Linux IMA with OSTree", + "title": "Signing", + "content": "To apply IMA signatures to an OSTree commit, there is an ima-sign command implemented currently in the ostree-rs-ext project. Generating a key . There is documentation for this in man evmctl and the upstream IMA page; we will not replicate it here. Signing a commit . ima-sign requires 4 things: . | An OSTree repository (could be any mode; archive or e.g. bare-user) | A ref or commit digest (e.g. exampleos/x86_64/stable) | A digest algorithm (usually sha256, but you may use e.g. sha512 as well) | An RSA private key | . You can then add IMA signatures to all regular files in the commit: . $ ostree-ext-cli ima-sign --repo=repo exampleos/x86_64/stable sha256 /path/to/key.pem . Many different choices are possible for the signing model. For example, your build system could store individual components/packages in their own ostree refs, and sign them at build time. This would avoid re-signing all binaries when creating production builds. Although note you still likely want to sign generated artifacts from unioning individual components, such as a dpkg/rpm database or equivalent and cache files such as the ldconfig and GTK+ icon caches, etc. Applying a policy . Signing a commit by itself will have little to no effect. You will also need to include in your builds an IMA policy. Linux EVM . The EVM subsystem builds on IMA, and adds another signature which covers most file data, such as the uid, gid and mode and selected security-relevant extended attributes. This is quite close to the ostree native checksum - the ordering of the fields is different so the checksums are physically different, but logically they are very close. However, the focus of the EVM design seems to mostly be on machine-specific signatures with keys stored in a TPM. Note that doing this on a per-machine basis would add a new security.evm extended attribute, and crucially that changes the ostree digest - so from ostree’s perspective, these objects will appear corrupt. In the future, ostree may learn to ignore the presence of security.evm extended attributes. There is also some support for “portable” EVM signatures - by default, EVM signatures also include the inode number and generation which are inherently machine-specific. A future ostree enhancement may instead also focus on supporting signing commits with these “portable” EVM signatures in addition to IMA. ", + "url": "/ostree/ima/#signing", + + "relUrl": "/ima/#signing" + },"63": { + "doc": "Using Linux IMA with OSTree", + "title": "Further references", + "content": ". | https://sourceforge.net/p/linux-ima/wiki/Home/ | https://en.opensuse.org/SDB:Ima_evm | https://wiki.gentoo.org/wiki/Integrity_Measurement_Architecture | https://fedoraproject.org/wiki/Changes/Signed_RPM_Contents | https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/enhancing-security-with-the-kernel-integrity-subsystem_managing-monitoring-and-updating-the-kernel | . ", + "url": "/ostree/ima/#further-references", + + "relUrl": "/ima/#further-references" + },"64": { + "doc": "libostree", + "title": "libostree", + "content": ". | Operating systems and distributions using OSTree | Distribution build tools | Projects linking to libostree | Language bindings | Building | API Reference | Manual Pages | Contributing | Licensing | . This project is now known as “libostree”, though it is still appropriate to use the previous name: “OSTree” (or “ostree”). The focus is on projects which use libostree’s shared library, rather than users directly invoking the command line tools (except for build systems). However, in most of the rest of the documentation, we will use the term “OSTree”, since it’s slightly shorter, and changing all documentation at once is impractical. We expect to transition to the new name over time. As implied above, libostree is both a shared library and suite of command line tools that combines a “git-like” model for committing and downloading bootable filesystem trees, along with a layer for deploying them and managing the bootloader configuration. The core OSTree model is like git in that it checksums individual files and has a content-addressed-object store. It’s unlike git in that it “checks out” the files via hardlinks, and they thus need to be immutable to prevent corruption. Therefore, another way to think of OSTree is that it’s just a more polished version of Linux VServer hardlinks. Features: . | Transactional upgrades and rollback for the system | Replicating content incrementally over HTTP via GPG signatures and “pinned TLS” support | Support for parallel installing more than just 2 bootable roots | Binary history on the server side (and client) | Introspectable shared library API for build and deployment systems | Flexible support for multiple branches and repositories, supporting projects like flatpak which use libostree for applications, rather than hosts. | . ", + "url": "/ostree/", + + "relUrl": "/" + },"65": { + "doc": "libostree", + "title": "Operating systems and distributions using OSTree", + "content": "Endless OS uses libostree for their host system as well as flatpak. See their eos-updater and deb-ostree-builder projects. Fedora derivatives use rpm-ostree (noted below); there are 4 variants using OSTree: . | Fedora CoreOS | Fedora Silverblue | Fedora Kinoite | Fedora IoT | . Red Hat Enterprise Linux CoreOS is a derivative of Fedora CoreOS, used in OpenShift 4. The machine-config-operator manages upgrades. RHEL CoreOS is also the successor to RHEL Atomic Host, which uses rpm-ostree as well. GNOME Continuous is where OSTree was born - as a high performance continuous delivery/testing system for GNOME. Liri OS has the option to install their distribution using ostree. TorizonCore uses libostree and Aktualizr as the base for OTA updates from compatible platforms, including Torizon OTA. ", + "url": "/ostree/#operating-systems-and-distributions-using-ostree", + + "relUrl": "/#operating-systems-and-distributions-using-ostree" + },"66": { + "doc": "libostree", + "title": "Distribution build tools", + "content": "meta-updater is a layer available for OpenEmbedded systems. QtOTA is Qt’s over-the-air update framework which uses libostree. The BuildStream build and integration tool supports importing and exporting from libostree repos. Fedora coreos-assembler is the build tool used to generate Fedora CoreOS derivatives. ", + "url": "/ostree/#distribution-build-tools", + + "relUrl": "/#distribution-build-tools" + },"67": { + "doc": "libostree", + "title": "Projects linking to libostree", + "content": "rpm-ostree is used by the Fedora-derived operating systems listed above. It is a full hybrid image/package system. By default it uses libostree to atomically replicate a base OS (all dependency resolution is done on the server), but it supports “package layering”, where additional RPMs can be layered on top of the base. This brings a “best of both worlds”” model for image and package systems. eos-updater is a daemon that implements updates on EndlessOS. flatpak uses libostree for desktop application containers. Unlike most of the other systems here, flatpak does not use the “libostree host system” aspects (e.g. bootloader management), just the “git-like hardlink dedup”. For example, flatpak supports a per-user OSTree repository. ", + "url": "/ostree/#projects-linking-to-libostree", + + "relUrl": "/#projects-linking-to-libostree" + },"68": { + "doc": "libostree", + "title": "Language bindings", + "content": "libostree is accessible via GObject Introspection; any language which has implemented the GI binding model should work. For example, Both pygobject and gjs are known to work and further are actually used in libostree’s test suite today. Some bindings take the approach of using GI as a lower level and write higher level manual bindings on top; this is more common for statically compiled languages. Here’s a list of such bindings: . | ostree-go | ostree-rs | . ", + "url": "/ostree/#language-bindings", + + "relUrl": "/#language-bindings" + },"69": { + "doc": "libostree", + "title": "Building", + "content": "Releases are available as GPG signed git tags, and most recent versions support extended validation using git-evtag. However, in order to build from a git clone, you must update the submodules. If you’re packaging OSTree and want a tarball, I recommend using a “recursive git archive” script. There are several available online; this code in OSTree is an example. Once you have a git clone or recursive archive, building is the same as almost every autotools project: . git submodule update --init env NOCONFIGURE=1 ./autogen.sh ./configure --prefix=... make make install DESTDIR=/path/to/dest . ", + "url": "/ostree/#building", + + "relUrl": "/#building" + },"70": { + "doc": "libostree", + "title": "API Reference", + "content": "The libostree API documentation is available in Reference. ", + "url": "/ostree/#api-reference", + + "relUrl": "/#api-reference" + },"71": { + "doc": "libostree", + "title": "Manual Pages", + "content": "The ostree manual pages are available in Manual. ", + "url": "/ostree/#manual-pages", + + "relUrl": "/#manual-pages" + },"72": { + "doc": "libostree", + "title": "Contributing", + "content": "See Contributing. ", + "url": "/ostree/#contributing", + + "relUrl": "/#contributing" + },"73": { + "doc": "libostree", + "title": "Licensing", + "content": "The licensing for the code of libostree can be canonically found in the individual files; and the overall status in the COPYING file in the source. Currently, that’s LGPLv2+. This also covers the man pages and API docs. The license for the manual documentation in the doc/ directory is: SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) This is intended to allow use by Wikipedia and other projects. In general, files should have a SPDX-License-Identifier and that is canonical. ", + "url": "/ostree/#licensing", + + "relUrl": "/#licensing" + },"74": { + "doc": "OSTree Overview", + "title": "OSTree Overview", + "content": ". | Introduction | Hello World example | Comparison with “package managers” | Comparison with block/image replication | Atomic transitions between parallel-installable read-only filesystem trees . | Licensing for this document: | . | . ", + "url": "/ostree/introduction/", + + "relUrl": "/introduction/" + },"75": { + "doc": "OSTree Overview", + "title": "Introduction", + "content": "OSTree is an upgrade system for Linux-based operating systems that performs atomic upgrades of complete filesystem trees. It is not a package system; rather, it is intended to complement them. A primary model is composing packages on a server, and then replicating them to clients. The underlying architecture might be summarized as “git for operating system binaries”. It operates in userspace, and will work on top of any Linux filesystem. At its core is a git-like content-addressed object store with branches (or “refs”) to track meaningful filesystem trees within the store. Similarly, one can check out or commit to these branches. Layered on top of that is bootloader configuration, management of /etc, and other functions to perform an upgrade beyond just replicating files. You can use OSTree standalone in the pure replication model, but another approach is to add a package manager on top, thus creating a hybrid tree/package system. ", + "url": "/ostree/introduction/#introduction", + + "relUrl": "/introduction/#introduction" + },"76": { + "doc": "OSTree Overview", + "title": "Hello World example", + "content": "OSTree is mostly used as a library, but a quick tour of using its CLI tools can give a general idea of how it works at its most basic level. You can create a new OSTree repository using init: . $ ostree --repo=repo init . This will create a new repo directory containing your repository. Feel free to inspect it. Now, let’s prepare some data to add to the repo: . $ mkdir tree $ echo \"Hello world!\" > tree/hello.txt . We can now import our tree/ directory using the commit command: . $ ostree --repo=repo commit --branch=foo tree/ . This will create a new branch foo pointing to the full tree imported from tree/. In fact, we could now delete tree/ if we wanted to. To check that we indeed now have a foo branch, you can use the refs command: . $ ostree --repo=repo refs foo . We can also inspect the filesystem tree using the ls and cat commands: . $ ostree --repo=repo ls foo d00775 1000 1000 0 / -00664 1000 1000 13 /hello.txt $ ostree --repo=repo cat foo /hello.txt Hello world! . And finally, we can check out our tree from the repository: . $ ostree --repo=repo checkout foo tree-checkout/ $ cat tree-checkout/hello.txt Hello world! . ", + "url": "/ostree/introduction/#hello-world-example", + + "relUrl": "/introduction/#hello-world-example" + },"77": { + "doc": "OSTree Overview", + "title": "Comparison with “package managers”", + "content": "Because OSTree is designed for deploying core operating systems, a comparison with traditional “package managers” such as dpkg and rpm is illustrative. Packages are traditionally composed of partial filesystem trees with metadata and scripts attached, and these are dynamically assembled on the client machine, after a process of dependency resolution. In contrast, OSTree only supports recording and deploying complete (bootable) filesystem trees. It has no built-in knowledge of how a given filesystem tree was generated or the origin of individual files, or dependencies, descriptions of individual components. Put another way, OSTree only handles delivery and deployment; you will likely still want to include inside each tree metadata about the individual components that went into the tree. For example, a system administrator may want to know what version of OpenSSL was included in your tree, so you should support the equivalent of rpm -q or dpkg -L. The OSTree core emphasizes replicating read-only OS trees via HTTP, and where the OS includes (if desired) an entirely separate mechanism to install applications, stored in /var if they’re system global, or /home for per-user application installation. An example application mechanism is http://docker.io/ . However, it is entirely possible to use OSTree underneath a package system, where the contents of /usr are computed on the client. For example, when installing a package, rather than changing the currently running filesystem, the package manager could assemble a new filesystem tree that layers the new packages on top of a base tree, record it in the local OSTree repository, and then set it up for the next boot. To support this model, OSTree provides an (introspectable) C shared library. ", + "url": "/ostree/introduction/#comparison-with-package-managers", + + "relUrl": "/introduction/#comparison-with-package-managers" + },"78": { + "doc": "OSTree Overview", + "title": "Comparison with block/image replication", + "content": "OSTree shares some similarity with “dumb” replication and stateless deployments, such as the model common in “cloud” deployments where nodes are booted from an (effectively) readonly disk, and user data is kept on a different volumes. The advantage of “dumb” replication, shared by both OSTree and the cloud model, is that it’s reliable and predictable. But unlike many default image-based deployments, OSTree supports exactly two persistent writable directories that are preserved across upgrades: /etc and /var. Because OSTree operates at the Unix filesystem layer, it works on top of any filesystem or block storage layout; it’s possible to replicate a given filesystem tree from an OSTree repository into plain ext4, BTRFS, XFS, or in general any Unix-compatible filesystem that supports hard links. Note: OSTree will transparently take advantage of some BTRFS features if deployed on it. OSTree is orthogonal to virtualization mechanisms like AMIs and qcow2 images, though it’s most useful though if you plan to update stateful VMs in-place, rather than generating new images. In practice, users of “bare metal” configurations will find the OSTree model most useful. ", + "url": "/ostree/introduction/#comparison-with-blockimage-replication", + + "relUrl": "/introduction/#comparison-with-blockimage-replication" + },"79": { + "doc": "OSTree Overview", + "title": "Atomic transitions between parallel-installable read-only filesystem trees", + "content": "Another deeply fundamental difference between both package managers and image-based replication is that OSTree is designed to parallel-install multiple versions of multiple independent operating systems. OSTree relies on a new toplevel ostree directory; it can in fact parallel install inside an existing OS or distribution occupying the physical / root. On each client machine, there is an OSTree repository stored in /ostree/repo, and a set of “deployments” stored in /ostree/deploy/$STATEROOT/$CHECKSUM. Each deployment is primarily composed of a set of hardlinks into the repository. This means each version is deduplicated; an upgrade process only costs disk space proportional to the new files, plus some constant overhead. The model OSTree emphasizes is that the OS read-only content is kept in the classic Unix /usr; it comes with code to create a Linux read-only bind mount to prevent inadvertent corruption. There is exactly one /var writable directory shared between each deployment for a given OS. The OSTree core code does not touch content in this directory; it is up to the code in each operating system for how to manage and upgrade state. Finally, each deployment has its own writable copy of the configuration store /etc. On upgrade, OSTree will perform a basic 3-way diff, and apply any local changes to the new copy, while leaving the old untouched. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/introduction/#atomic-transitions-between-parallel-installable-read-only-filesystem-trees", + + "relUrl": "/introduction/#atomic-transitions-between-parallel-installable-read-only-filesystem-trees" + },"80": { + "doc": "Related Projects", + "title": "Related Projects", + "content": ". | Combining dpkg/rpm + (BTRFS/LVM) | ChromiumOS updater | Ubuntu Image Based Updates | Clear Linux Software update | casync | Mender.io | OLPC update | NixOS / Nix | Solaris IPS | Google servers (custom rsync-like approach, live updates) | Conary | bmap | Git | Conda | rpm-ostree | GNOME Continuous | Docker | Docker-related: Balena | Torizon Platform . | TorizonCore | TorizonCore Builder | Torizon OTA . | Licensing for this document: | . | . | . OSTree is in many ways very evolutionary. It builds on concepts and ideas introduced from many different projects such as Systemd Stateless, Systemd Bootloader Spec, Chromium Autoupdate, the much older Fedora/Red Hat Stateless Project, Linux VServer and many more. As mentioned elsewhere, OSTree is strongly influenced by package manager designs as well. This page is not intended to be an exhaustive list of such projects, but we will try to keep it up to date, and relatively agnostic. Broadly speaking, projects in this area fall into two camps; either a tool to snapshot systems on the client side (dpkg/rpm + BTRFS/LVM), or a tool to compose on a server and replicate (ChromiumOS, Clear Linux). OSTree is flexible enough to do both. Note that this section of the documentation is almost entirely focused on the “ostree for host” model; the flatpak project uses libostree to store application data, distinct from the host system management model. ", + "url": "/ostree/related-projects/", + + "relUrl": "/related-projects/" + },"81": { + "doc": "Related Projects", + "title": "Combining dpkg/rpm + (BTRFS/LVM)", + "content": "In this approach, one uses a block/filesystem snapshot tool underneath the system package manager. The oVirt Node imgbased tool is an example of this approach, as are a few others below. Regarding BTRFS in particular - the OSTree author believes that Linux storage is a wide world, and while BTRFS is quite good, it is not everywhere now, nor will it be in the near future. There are other recently developed filesystems like f2fs, and Red Hat Enterprise Linux still defaults to XFS. Using a snapshot tool underneath a package manager does help significantly. In the rest of this text, we will use “BTRFS” as a mostly generic tool for filesystem snapshots. The obvious thing to do is layer BTRFS under dpkg/rpm, and have a separate subvolume for /home so rollbacks don’t lose your data. See e.g. Fedora BTRFS Rollback Feature. More generally, if you want to use BTRFS to roll back changes made by dpkg/rpm, you have to carefully set up the partition layout so that the files laid out by dpkg/rpm are installed in a subvolume to snapshot. This problem in many ways is addressed by the changes OSTree forces, such as putting all local state in /var (e.g. /usr/local -> /var/usrlocal). Then one can BTRFS snapshot /usr. This gets pretty far, except handling /etc is messy. This is something OSTree does well. In general, if one really tries to flesh out the BTRFS approach, a nontrivial middle layer of code between dpkg/rpm and BTRFS (or deep awareness of BTRFS in dpkg/rpm itself) will be required. A good example of this is the snapper.io project. The OSTree author believes that having total freedom at the block storage layer is better for general purpose operating systems. For example, the ability to choose dm-crypt per deployment is quite useful; not every site wants to pay the performance penalty. One can choose LVM or not, etc. Where applicable, OSTree does take advantage of copy-on-write/reflink features offered by the kernel for /etc. It uses the now generic ioctl(FICLONE) and copy_file_range(). Another major distinction between the default OSTree usage and package managers is whether updates are “online” or “offline” by default. The default OSTree design writes updates into a new root, leaving the running system unchanged. This means preparing updates is completely non-disruptive and safe - if the system runs out of disk space in the middle, it’s easy to recover. However, there is work in the rpm-ostree project to support online updates as well. OSTree supports using “bare-user” repositories, which do not require root to use. Using a filesystem-level layer without root is more difficult and would likely require a setuid helper or privileged service. Finally, see the next portion around ChromiumOS for why a hybrid but integrated package/image system improves on this. ", + "url": "/ostree/related-projects/#combining-dpkgrpm--btrfslvm", + + "relUrl": "/related-projects/#combining-dpkgrpm--btrfslvm" + },"82": { + "doc": "Related Projects", + "title": "ChromiumOS updater", + "content": "Many people who look at OSTree are most interested in using it as an updater for embedded or fixed-purpose systems, similar to use cases from the ChromiumOS updater. The ChromiumOS approach uses two partitions that are swapped via the bootloader. It has a very network-efficient update protocol, using a custom binary delta scheme between filesystem snapshots. This model even allows for switching filesystem types in an update. A major downside of this approach is that the OS size is doubled on disk always. In contrast, OSTree uses plain Unix hardlinks, which means it essentially only requires disk space proportional to the changed files, plus some small fixed overhead. This means with OSTree, one can easily have more than two trees (deployments). Another example is that the system OSTree repository could also be used for application containers. Finally, the author of OSTree believes that what one really wants for many cases is image replication with the ability to layer on some additional components (e.g. packages) - a hybrid model. This is what rpm-ostree is aiming to support. ", + "url": "/ostree/related-projects/#chromiumos-updater", + + "relUrl": "/related-projects/#chromiumos-updater" + },"83": { + "doc": "Related Projects", + "title": "Ubuntu Image Based Updates", + "content": "See https://wiki.ubuntu.com/ImageBasedUpgrades. Very architecturally similar to ChromeOS, although more interesting is discussion for supporting package installation on top, similar to rpm-ostree package layering. ", + "url": "/ostree/related-projects/#ubuntu-image-based-updates", + + "relUrl": "/related-projects/#ubuntu-image-based-updates" + },"84": { + "doc": "Related Projects", + "title": "Clear Linux Software update", + "content": "The Clear Linux Software update system is not very well documented. This mailing list post has some reverse-engineered design documentation. Like OSTree static deltas, it also uses bsdiff for network efficiency. More information will be filled in here over time. The OSTree author believes that at the moment, the “CL updater” is not truly atomic in the sense that because it applies updates live, there is a window where the OS root may be inconsistent. ", + "url": "/ostree/related-projects/#clear-linux-software-update", + + "relUrl": "/related-projects/#clear-linux-software-update" + },"85": { + "doc": "Related Projects", + "title": "casync", + "content": "The systemd casync project is relatively new. Currently, it is more of a storage library, and doesn’t support higher level logic for things like GPG signatures, versioning information, etc. This is mostly the OstreeRepo layer. Moving up to the OstreeSysroot level - things like managing the bootloader configuration, and most importantly implementing correct merging for /etc are missing. casync also is unaware of SELinux. OSTree is really today a shared library, and has been for quite some time. This has made it easy to build higher level projects such as rpm-ostree which has quite a bit more, such as a DBus API and other projects consume that, such as Cockpit. A major issue with casync today is that it doesn’t support garbage collection on the server side. OSTree’s GC works symmetrically on the server and client side. Broadly speaking, casync is a twist on the dual partition approach, and shares the general purpose disadvantages of those. ", + "url": "/ostree/related-projects/#casync", + + "relUrl": "/related-projects/#casync" + },"86": { + "doc": "Related Projects", + "title": "Mender.io", + "content": "Mender.io is another implementation of the dual partition approach. ", + "url": "/ostree/related-projects/#menderio", + + "relUrl": "/related-projects/#menderio" + },"87": { + "doc": "Related Projects", + "title": "OLPC update", + "content": "OSTree is basically a generalization of olpc-update, except using plain HTTP instead of rsync. OSTree has the notion of separate trees that one can track independently or parallel install, while still sharing storage via the hardlinked repository, whereas olpc-update uses version numbers for a single OS. OSTree has built-in plain old HTTP replication which can be served from a static webserver, whereas olpc-update uses rsync (more server load, but more efficient on the network side). The OSTree solution to improving network bandwidth consumption is via static deltas. See this comment for a comparison. ", + "url": "/ostree/related-projects/#olpc-update", + + "relUrl": "/related-projects/#olpc-update" + },"88": { + "doc": "Related Projects", + "title": "NixOS / Nix", + "content": "See NixOS. It was a very influential project for OSTree. NixOS and OSTree both support the idea of independent “roots” that are bootable. In NixOS, files in a package are accessed by a path depending on the checksums of package inputs (build dependencies) - see Nix store. However, OSTree uses a commit/deploy model - it isn’t tied to any particular directory layout, and you can put whatever data you want inside an OSTree, for example the standard FHS layout. A both positive and negative of the Nix model is that a change in the build dependencies (e.g. being built with a newer gcc), requires a cascading rebuild of everything. It’s good because it makes it easy to do massive system-wide changes such as gcc upgrades, and allows installing multiple versions of packages at once. However, a security update to e.g. glibc forces a rebuild of everything from scratch, and so Nix is not practical at scale. OSTree supports using a build system that just rebuilds individual components (packages) as they change, without forcing a rebuild of their dependencies. Nix automatically detects runtime package dependencies by scanning content for hashes. OSTree only supports only system-level images, and doesn’t do dependency management. Nix can store arbitrary files, using nix-store --add, but, more commonly, paths are added as the result of running a derivation file generated using the Nix language. OSTree is build-system agnostic; filesystem trees are committed using a simple C API, and this is the only way to commit files. OSTree automatically shares the storage of identical data using hard links into a content-addressed store. Nix can deduplicate using hard links as well, using the auto-optimise-store option, but this is not on by default, and Nix does not guarantee that all of its files are in the content-addressed store. OSTree provides a git-like command line interface for browsing the content-addressed store, while Nix does not have this functionality. Nix used to use the immutable bit to prevent modifications to /nix/store, but now it uses a read-only bind mount. The bind mount can be privately remounted, allowing per-process privileged write access. OSTree uses the immutable bit on the root of the deployment, and mounts /usr as read-only. NixOS supports switching OS images on-the-fly, by maintaining both booted-system and current-system roots. It is not clear how well this approach works. OSTree currently requries a reboot to switch images. Finally, NixOS supports installing user-specific packages from trusted repositories without requiring root, using a trusted daemon. Flatpak, based on OSTree, similarly has a policykit-based system helper that allows you to authenticate via polkit to install into the system repository. ", + "url": "/ostree/related-projects/#nixos--nix", + + "relUrl": "/related-projects/#nixos--nix" + },"89": { + "doc": "Related Projects", + "title": "Solaris IPS", + "content": "See Solaris IPS. Broadly, this is a similar design as to a combination of BTRFS+RPM/deb. There is a bootloader management system which combines with the snapshots. It’s relatively well thought through - however, it is a client-side system assembly. If one wants to image servers and replicate reliably, that’d be a different system. ", + "url": "/ostree/related-projects/#solaris-ips", + + "relUrl": "/related-projects/#solaris-ips" + },"90": { + "doc": "Related Projects", + "title": "Google servers (custom rsync-like approach, live updates)", + "content": "This paper talks about how Google was (at least at one point) managing updates for the host systems for some servers: Live Upgrading Thousands of Servers from an Ancient Red Hat Distribution to 10 Year Newer Debian Based One (USENIX LISA 2013) . ", + "url": "/ostree/related-projects/#google-servers-custom-rsync-like-approach-live-updates", + + "relUrl": "/related-projects/#google-servers-custom-rsync-like-approach-live-updates" + },"91": { + "doc": "Related Projects", + "title": "Conary", + "content": "See Conary Updates and Rollbacks. If rpm/dpkg are like CVS, Conary is closer to Subversion. It’s not bad, but e.g. its rollback model is rather ad-hoc and not atomic. It also is a fully client side system and doesn’t have an image-like replication with deltas. ", + "url": "/ostree/related-projects/#conary", + + "relUrl": "/related-projects/#conary" + },"92": { + "doc": "Related Projects", + "title": "bmap", + "content": "See bmap. A tool for optimized copying of disk images. Intended for offline use, so not directly comparable. ", + "url": "/ostree/related-projects/#bmap", + + "relUrl": "/related-projects/#bmap" + },"93": { + "doc": "Related Projects", + "title": "Git", + "content": "Although OSTree has been called “Git for Binaries”, and the two share the idea of a hashed content store, the implementation details are quite different. OSTree supports extended attributes and uses SHA256 instead of Git’s SHA1. It “checks out” files via hardlinks, rather than copying, and thus requires the checkout to be immutable. At the moment, OSTree commits may have at most one parent, as opposed to Git which allows an arbitrary number. Git uses a smart-delta protocol for updates, while OSTree uses 1 HTTP request per changed file, or can generate static deltas. ", + "url": "/ostree/related-projects/#git", + + "relUrl": "/related-projects/#git" + },"94": { + "doc": "Related Projects", + "title": "Conda", + "content": "Conda is an “OS-agnostic, system-level binary package manager and ecosystem”; although most well-known for its accompanying Python distribution anaconda, its scope has been expanding quickly. The package format is very similar to well-known ones such as RPM. However, unlike typical RPMs, the packages are built to be relocatable. Also, the package manager runs natively on Windows. Conda’s main advantage is its ability to install collections of packages into “environments” by unpacking them all to the same directory. Conda reduces duplication across environments using hardlinks, similar to OSTree’s sharing between deployments (although Conda uses package / file path instead of file hash). Overall, it is quite similar to rpm-ostree in functionality and scope. ", + "url": "/ostree/related-projects/#conda", + + "relUrl": "/related-projects/#conda" + },"95": { + "doc": "Related Projects", + "title": "rpm-ostree", + "content": "This builds on top of ostree to support building RPMs into OSTree images, and even composing RPMs on-the-fly using an overlay filesystem. It is being developed by Fedora, Red Hat, and CentOS as part of Project Atomic. ", + "url": "/ostree/related-projects/#rpm-ostree", + + "relUrl": "/related-projects/#rpm-ostree" + },"96": { + "doc": "Related Projects", + "title": "GNOME Continuous", + "content": "This is a service that incrementally rebuilds and tests GNOME on every commit. The need to make and distribute snapshots for this system was the original inspiration for ostree. ", + "url": "/ostree/related-projects/#gnome-continuous", + + "relUrl": "/related-projects/#gnome-continuous" + },"97": { + "doc": "Related Projects", + "title": "Docker", + "content": "It makes sense to compare OSTree and Docker as far as wire formats go. OSTree is not itself a container tool, but can be used as a transport/storage format for container tools. Docker has (at the time of this writing) two format versions (v1 and v2). v1 is deprecated, so we’ll look at format version 2. A Docker image is a series of layers, and a layer is essentially JSON metadata plus a tarball. The tarballs capture changes between layers, including handling deleting files in higher layers. Because the payload format is just tar, Docker hence captures (numeric) uid/gid and xattrs. This “layering” model is an interesting and powerful part of Docker, allowing different images to reference a shared base. OSTree doesn’t implement this natively, but it’s not difficult to implement in higher level tools. For example in flatpak, there’s a concept of a SDK and runtime, and it would make a lot of sense for the SDK to depend on the runtime, to avoid clients downloading data twice (even if it’s deduplicated on disk). That gets to an advantage of OSTree over Docker; OSTree checksums individual files (not tarballs), and uses this for deduplication. Docker (natively) only shares storage via layering. The biggest feature OSTree has over Docker though is support for (static) deltas, and even without pre-configured static deltas, the archive format has “natural” deltas. Particularly for a “base operating system”, one really wants on-wire deltas. It’d likely be possible to extend Docker with this concept. A core challenge both share is around metadata (particularly signing) and search/discovery (the ostree summary file doesn’t scale very well). One major issue Docker has is that it checksums compressed data, and furthermore the tar format is flexible, with multiple ways to represent data, making it hard to impossible to reassemble and verify from on-disk state. The tarsum effort was intended to address this, but it was not adopted in the end for v2. ", + "url": "/ostree/related-projects/#docker", + + "relUrl": "/related-projects/#docker" + },"98": { + "doc": "Related Projects", + "title": "Docker-related: Balena", + "content": "The Balena project forks Docker and aims to even use Docker/OCI format for the root filesystem, and adds wire deltas using librsync. See also discussion on libostree-list. ", + "url": "/ostree/related-projects/#docker-related-balena", + + "relUrl": "/related-projects/#docker-related-balena" + },"99": { + "doc": "Related Projects", + "title": "Torizon Platform", + "content": "Torizon is an open-source software platform that simplifies the development and maintenance of embedded Linux software. It is designed to be used out-of-the-box on devices requiring high reliability, allowing you to focus on your application and not on building and maintaining the operating system. TorizonCore . The platform OS - TorizonCore - is a minimal OS with a Docker runtime and libostree + Aktualizr. The main goal of this system is to allow application developers to use containers, while the maintainers of TorizonCore focus on the base system updates. TorizonCore Builder . Since the TorizonCore OS is meant as a binary distribution, OS customization is made easier with TorizonCore Builder, as the tool abstracts the handling of OSTree concepts from the final users. Torizon OTA . Torizon OTA is a hosted OTA update system that provides OS updates to TorizonCore using OSTree and Aktualizr. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/related-projects/#torizon-platform", + + "relUrl": "/related-projects/#torizon-platform" + },"100": { + "doc": "Anatomy of an OSTree repository", + "title": "Anatomy of an OSTree repository", + "content": ". | Core object types and data model . | Commit objects | Dirtree objects | Dirmeta objects | Content objects | Xattrs objects | . | Repository types and locations . | Refs | The summary file . | Licensing for this document: | . | . | . ", + "url": "/ostree/repo/", + + "relUrl": "/repo/" + },"101": { + "doc": "Anatomy of an OSTree repository", + "title": "Core object types and data model", + "content": "OSTree is deeply inspired by git; the core layer is a userspace content-addressed versioning filesystem. It is worth taking some time to familiarize yourself with Git Internals, as this section will assume some knowledge of how git works. Its object types are similar to git; it has commit objects and content objects. Git has “tree” objects, whereas OSTree splits them into “dirtree” and “dirmeta” objects. But unlike git, OSTree’s checksums are SHA256. And most crucially, its content objects include uid, gid, and extended attributes (but still no timestamps). Commit objects . A commit object contains metadata such as a timestamp, a log message, and most importantly, a reference to a dirtree/dirmeta pair of checksums which describe the root directory of the filesystem. Also like git, each commit in OSTree can have a parent. It is designed to store a history of your binary builds, just like git stores a history of source control. However, OSTree also makes it easy to delete data, under the assumption that you can regenerate it from source code. Dirtree objects . A dirtree contains a sorted array of (filename, checksum) pairs for content objects, and a second sorted array of (filename, dirtree checksum, dirmeta checksum), which are subdirectories. This type of object is stored as files ending with .dirtree in the objects directory. Dirmeta objects . In git, tree objects contain the metadata such as permissions for their children. But OSTree splits this into a separate object to avoid duplicating extended attribute listings. These type of objects are stored as files ending with .dirmeta in the objects directory. Content objects . Unlike the first three object types which are metadata, designed to be mmap()ed, the content object has a separate internal header and payload sections. The header contains uid, gid, mode, and symbolic link target (for symlinks), as well as extended attributes. After the header, for regular files, the content follows. These parts together form the SHA256 hash for content objects. The content type objects in this format exist only in archive OSTree repositories. Today the content part is gzip’ed and the objects are stored as files ending with .filez in the objects directory. Because the SHA256 hash is formed over the uncompressed content, these files do not match the hash they are named as. The OSTree data format intentionally does not contain timestamps. The reasoning is that data files may be downloaded at different times, and by different build systems, and so will have different timestamps but identical physical content. These files may be large, so most users would like them to be shared, both in the repository and between the repository and deployments. This could cause problems with programs that check if files are out-of-date by comparing timestamps. For Git, the logical choice is to not mess with timestamps, because unnecessary rebuilding is better than a broken tree. However, OSTree has to hardlink files to check them out, and commits are assumed to be internally consistent with no build steps needed. For this reason, OSTree acts as though all timestamps are set to time_t 0, so that comparisons will be considered up-to-date. Note that for a few releases, OSTree used 1 to fix warnings such as GNU Tar emitting “implausibly old time stamp” with 0; however, until we have a mechanism to transition cleanly to 1, for compatibilty OSTree is reverted to use zero again. Xattrs objects . In some repository modes (e.g. bare-split-xattrs), xattrs are stored on the side of the content objects they refer to. This is done via two dedicated object types, file-xattrs and file-xattrs-link. file-xattrs store xattrs data, encoded as GVariant. Each object is keyed by the checksum of the xattrs content, allowing for multiple references. file-xattrs-link are hardlinks which are associated to file objects. Each object is keyed by the same checksum of the corresponding file object. The target of the hardlink is an existing file-xattrs object. In case of reaching the limit of too many links, this object could be a plain file too. ", + "url": "/ostree/repo/#core-object-types-and-data-model", + + "relUrl": "/repo/#core-object-types-and-data-model" + },"102": { + "doc": "Anatomy of an OSTree repository", + "title": "Repository types and locations", + "content": "Also unlike git, an OSTree repository can be in one of five separate modes: bare, bare-split-xattrs, bare-user, bare-user-only, and archive. A bare repository is one where content files are just stored as regular files; it’s designed to be the source of a “hardlink farm”, where each operating system checkout is merely links into it. If you want to store files owned by e.g. root in this mode, you must run OSTree as root. The bare-split-xattrs mode is similar to the above one, but it does store xattrs as separate objects. This is meant to avoid conflicts with kernel-enforced constraints (e.g. on SELinux labels) and with other softwares that may perform ephemeral changes to xattrs (e.g. container runtimes). The bare-user mode is a later addition that is like bare in that files are unpacked, but it can (and should generally) be created as non-root. In this mode, extended metadata such as owner uid, gid, and extended attributes are stored in extended attributes under the name user.ostreemeta but not actually applied. The bare-user mode is useful for build systems that run as non-root but want to generate root-owned content, as well as non-root container systems. The bare-user-only mode is a variant to the bare-user mode. Unlike bare-user, neither ownership nor extended attributes are stored. These repos are meant to to be checked out in user mode (with the -U flag), where this information is not applied anyway. Hence this mode may lose metadata. The main advantage of bare-user-only is that repos can be stored on filesystems which do not support extended attributes, such as tmpfs. In contrast, the archive mode is designed for serving via plain HTTP. Like tar files, it can be read/written by non-root users. On an OSTree-deployed system, the “system repository” is /ostree/repo. It can be read by any uid, but only written by root. The ostree command will by default operate on the system repository; you may provide the --repo argument to override this, or set the $OSTREE_REPO environment variable. ", + "url": "/ostree/repo/#repository-types-and-locations", + + "relUrl": "/repo/#repository-types-and-locations" + },"103": { + "doc": "Anatomy of an OSTree repository", + "title": "Refs", + "content": "Like git, OSTree uses the terminology “references” (abbreviated “refs”) which are text files that name (refer to) particular commits. See the Git Documentation for information on how git uses them. Unlike git though, it doesn’t usually make sense to have a “main” branch. There is a convention for references in OSTree that looks like this: exampleos/buildmain/x86_64-runtime and exampleos/buildmain/x86_64-devel-debug. These two refs point to two different generated filesystem trees. In this example, the “runtime” tree contains just enough to run a basic system, and “devel-debug” contains all of the developer tools and debuginfo. The ostree supports a simple syntax using the caret ^ to refer to the parent of a given commit. For example, exampleos/buildmain/x86_64-runtime^ refers to the previous build, and exampleos/buildmain/x86_64-runtime^^ refers to the one before that. ", + "url": "/ostree/repo/#refs", + + "relUrl": "/repo/#refs" + },"104": { + "doc": "Anatomy of an OSTree repository", + "title": "The summary file", + "content": "A later addition to OSTree is the concept of a “summary” file, created via the ostree summary -u command. This was introduced for a few reasons. A primary use case is to be compatible with Metalink, which requires a single file with a known checksum as a target. The summary file primarily contains two mappings: . | A mapping of the refs and their checksums, equivalent to fetching the ref file individually | A list of all static deltas, along with their metadata checksums | . This currently means that it grows linearly with both items. On the other hand, using the summary file, a client can enumerate branches. Further, fetching the summary file over e.g. pinned TLS creates a strong end-to-end verification of the commit or static delta. The summary file can also be GPG signed (detached). This is currently the only way to provide GPG signatures (transitively) on deltas. If a repository administrator creates a summary file, they must thereafter run ostree summary -u to update it whenever a ref is updated or a static delta is generated. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/repo/#the-summary-file", + + "relUrl": "/repo/#the-summary-file" + },"105": { + "doc": "Managing content in OSTree repositories", + "title": "Managing content in OSTree repositories", + "content": ". | Mirroring repositories | Separate development vs release repositories | Promoting content along OSTree branches - “buildmain”, “smoketested” | Promoting content between OSTree repositories | Derived data - static deltas and the summary file | Pruning our build and dev repositories | Generating “scratch” deltas for efficient initial downloads . | Licensing for this document: | . | . Once you have a build system going, if you actually want client systems to retrieve the content, you will quickly feel a need for “repository management”. The command line tool ostree does cover some core functionality, but doesn’t include very high level workflows. One reason is that how content is delivered and managed has concerns very specific to the organization. For example, some operating system content vendors may want integration with a specific errata notification system when generating commits. In this section, we will describe some high level ideas and methods for managing content in OSTree repositories, mostly independent of any particular model or tool. That said, there is an associated upstream project ostree-releng-scripts which has some scripts that are intended to implement portions of this document. Another example of software which can assist in managing OSTree repositories today is the Pulp Project, which has a Pulp OSTree plugin. ", + "url": "/ostree/repository-management/", + + "relUrl": "/repository-management/" + },"106": { + "doc": "Managing content in OSTree repositories", + "title": "Mirroring repositories", + "content": "It’s very common to want to perform a full or partial mirror, in particular across organizational boundaries (e.g. an upstream OS provider, and a user that wants offline and faster access to the content). OSTree supports both full and partial mirroring of the base archive content, although not yet of static deltas. To create a mirror, first create an archive repository (you don’t need to run this as root), then add the upstream as a remote, then use pull --mirror. ostree --repo=repo init --mode=archive ostree --repo=repo remote add exampleos https://exampleos.com/ostree/repo ostree --repo=repo pull --mirror exampleos:exampleos/x86_64/standard . You can use the --depth=-1 option to retrieve all history, or a positive integer like 3 to retrieve just the last 3 commits. See also the rsync-repos script in ostree-releng-scripts. ", + "url": "/ostree/repository-management/#mirroring-repositories", + + "relUrl": "/repository-management/#mirroring-repositories" + },"107": { + "doc": "Managing content in OSTree repositories", + "title": "Separate development vs release repositories", + "content": "By default, OSTree accumulates server side history. This is actually optional in that your build system can (using the API) write a commit with no parent. But first, we’ll investigate the ramifications of server side history. Many content vendors will want to separate their internal development with what is made public to the world. Therefore, you will want (at least) two OSTree repositories, we’ll call them “dev” and “prod”. To phrase this another way, let’s say you have a continuous delivery system which is building from git and committing into your “dev” OSTree repository. This might happen tens to hundreds of times per day. That’s a substantial amount of history over time, and it’s unlikely most of your content consumers (i.e. not developers/testers) will be interested in all of it. The original vision of OSTree was to fulfill this “dev” role, and in particular the “archive” format was designed for it. Then, what you’ll want to do is promote content from “dev” to “prod”. We’ll discuss this later, but first, let’s talk about promotion inside our “dev” repository. ", + "url": "/ostree/repository-management/#separate-development-vs-release-repositories", + + "relUrl": "/repository-management/#separate-development-vs-release-repositories" + },"108": { + "doc": "Managing content in OSTree repositories", + "title": "Promoting content along OSTree branches - “buildmain”, “smoketested”", + "content": "Besides multiple repositories, OSTree also supports multiple branches inside one repository, equivalent to git’s branches. We saw in an earlier section an example branch name like exampleos/x86_64/standard. Choosing the branch name for your “prod” repository is absolutely critical as client systems will reference it. It becomes an important part of your face to the world, in the same way the “main” branch in a git repository is. But with your “dev” repository internally, it can be very useful to use OSTree’s branching concepts to represent different stages in a software delivery pipeline. Deriving from exampleos/x86_64/standard, let’s say our “dev” repository contains exampleos/x86_64/buildmain/standard. We choose the term “buildmain” to represent something that came straight from git main. It may not be tested very much. Our next step should be to hook up a testing system (Jenkins, Buildbot, etc.) to this. When a build (commit) passes some tests, we want to “promote” that commit. Let’s create a new branch called smoketested to say that some basic sanity checks pass on the complete system. This might be where human testers get involved, for example. This is a basic way to “promote” the buildmain commit that passed testing: . ostree commit -b exampleos/x86_64/smoketested/standard -s 'Passed tests' --tree=ref=aec070645fe53... Here we’re generating a new commit object (perhaps include in the commit log links to build logs, etc.), but we’re reusing the content from the buildmain commit aec070645fe53 that passed the smoketests. For a more sophisticated implementation of this model, see the do-release-tags script, which includes support for things like propagating version numbers across commit promotion. We can easily generalize this model to have an arbitrary number of stages like exampleos/x86_64/stage-1-pass/standard, exampleos/x86_64/stage-2-pass/standard, etc. depending on business requirements and logic. In this suggested model, the “stages” are increasingly expensive. The logic is that we don’t want to spend substantial time on e.g. network performance tests if something basic like a systemd unit file fails on bootup. ", + "url": "/ostree/repository-management/#promoting-content-along-ostree-branches---buildmain-smoketested", + + "relUrl": "/repository-management/#promoting-content-along-ostree-branches---buildmain-smoketested" + },"109": { + "doc": "Managing content in OSTree repositories", + "title": "Promoting content between OSTree repositories", + "content": "Now, we have our internal continuous delivery stream flowing, it’s being tested and works. We want to periodically take the latest commit on exampleos/x86_64/stage-3-pass/standard and expose it in our “prod” repository as exampleos/x86_64/standard, with a much smaller history. We’ll have other business requirements such as writing release notes (and potentially putting them in the OSTree commit message), etc. In Build Systems we saw how the pull-local command can be used to migrate content from the “build” repository (in bare-user mode) into an archive repository for serving to client systems. Following this section, we now have three repositories, let’s call them repo-build, repo-dev, and repo-prod. We’ve been pulling content from repo-build into repo-dev (which involves gzip compression among other things since it is a format change). When using pull-local to migrate content between two archive repositories, the binary content is taken unmodified. Let’s go ahead and generate a new commit in our prod repository: . checksum=$(ostree --repo=repo-dev rev-parse exampleos/x86_64/stage-3-pass/standard`) ostree --repo=repo-prod pull-local repo-dev ${checksum} ostree --repo=repo-prod commit -b exampleos/x86_64/standard \\ -s 'Release 1.2.3' --add-metadata-string=version=1.2.3 \\ --tree=ref=${checksum} . There are a few things going on here. First, we found the latest commit checksum for the “stage-3 dev”, and told pull-local to copy it, without using the branch name. We do this because we don’t want to expose the exampleos/x86_64/stage-3-pass/standard branch name in our “prod” repository. Next, we generate a new commit in prod that’s referencing the exact binary content in dev. If the “dev” and “prod” repositories are on the same Unix filesystem, (like git) OSTree will make use of hard links to avoid copying any content at all - making the process very fast. Another interesting thing to notice here is that we’re adding an version metadata string to the commit. This is an optional piece of metadata, but we are encouraging its use in the OSTree ecosystem of tools. Commands like ostree admin status show it by default. ", + "url": "/ostree/repository-management/#promoting-content-between-ostree-repositories", + + "relUrl": "/repository-management/#promoting-content-between-ostree-repositories" + },"110": { + "doc": "Managing content in OSTree repositories", + "title": "Derived data - static deltas and the summary file", + "content": "As discussed in Formats, the archive repository we use for “prod” requires one HTTP fetch per client request by default. If we’re only performing a release e.g. once a week, it’s appropriate to use “static deltas” to speed up client updates. So once we’ve used the above command to pull content from repo-dev into repo-prod, let’s generate a delta against the previous commit: . ostree --repo=repo-prod static-delta generate exampleos/x86_64/standard . We may also want to support client systems upgrading from two commits previous. ostree --repo=repo-prod static-delta generate --from=exampleos/x86_64/standard^^ --to=exampleos/x86_64/standard . Generating a full permutation of deltas across all prior versions can get expensive, and there is some support in the OSTree core for static deltas which “recurse” to a parent. This can help create a model where clients download a chain of deltas. Support for this is not fully implemented yet however. Regardless of whether or not you choose to generate static deltas, you should update the summary file: . ostree --repo=repo-prod summary -u . (Remember, the summary command cannot be run concurrently, so this should be triggered serially by other jobs). There is some more information on the design of the summary file in Repo. ", + "url": "/ostree/repository-management/#derived-data---static-deltas-and-the-summary-file", + + "relUrl": "/repository-management/#derived-data---static-deltas-and-the-summary-file" + },"111": { + "doc": "Managing content in OSTree repositories", + "title": "Pruning our build and dev repositories", + "content": "First, the OSTree author believes you should not use OSTree as a “primary content store”. The binaries in an OSTree repository should be derived from a git repository. Your build system should record proper metadata such as the configuration options used to generate the build, and you should be able to rebuild it if necessary. Art assets should be stored in a system that’s designed for that (e.g. Git LFS). Another way to say this is that five years down the line, we are unlikely to care about retaining the exact binaries from an OS build on Wednesday afternoon three years ago. We want to save space and prune our “dev” repository. ostree --repo=repo-dev prune --refs-only --keep-younger-than=\"6 months ago\" . That will truncate the history older than 6 months. Deleted commits will have “tombstone markers” added so that you know they were explicitly deleted, but all content in them (that is not referenced by a still retained commit) will be garbage collected. ", + "url": "/ostree/repository-management/#pruning-our-build-and-dev-repositories", + + "relUrl": "/repository-management/#pruning-our-build-and-dev-repositories" + },"112": { + "doc": "Managing content in OSTree repositories", + "title": "Generating “scratch” deltas for efficient initial downloads", + "content": "In general, the happy path for OSTree downloads is via static deltas. If you are in a situation where you want to download an OSTree commit from an uninitialized repo (or one with unrelated history), you can generate “scratch” (aka --empty deltas) which bundle all objects for that commit. The tradeoff here is increasing server disk space in return for many fewer client HTTP requests. For example: . $ ostree --repo=/path/to/repo static-delta generate --empty --to=exampleos/x86_64/testing-devel $ ostree --repo=/path/to/repo summary -u . After that, clients fetching that commit will prefer fetching the “scratch” delta if they don’t have the original ref. Licensing for this document: . SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later) . ", + "url": "/ostree/repository-management/#generating-scratch-deltas-for-efficient-initial-downloads", + + "relUrl": "/repository-management/#generating-scratch-deltas-for-efficient-initial-downloads" + } +} |