summaryrefslogtreecommitdiff
path: root/src/libostree/ostree-repo.h
Commit message (Collapse)AuthorAgeFilesLines
* Update FSF license notices to use URL instead of addressJoseph Marrero2021-12-071-3/+1
|
* Add an API to verify a commit signature explicitlyColin Walters2021-08-301-0/+23
| | | | | | | | | | | | | We have a bunch of APIs to do GPG verification of a commit, but that doesn't generalize to signapi. Further, they require the caller to check the signature status explicitly which seems like a trap. This much higher level API works with both GPG and signapi. The intention is to use this in things that are doing "external pulls" like the ostree-ext tar import support. There we will get the commitmeta from the tarball and we want to verify it at the same time we import the commit.
* lib/commit: automatically skip xattrs in bare-user-only modeLuca BRUNO2021-08-241-1/+1
|
* Merge pull request #2412 from lucab/ups/lib-commit-canonicalizeColin Walters2021-08-201-1/+5
|\ | | | | lib/commit: autofix permissions for bare-user-only
| * lib/commit: autofix permissions for bare-user-onlyLuca BRUNO2021-08-201-1/+5
| | | | | | | | | | This tweaks commit logic to detect bare-user-only repositories and canonicalize permissions automatically.
* | fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys()Dan Nicholson2021-07-151-3/+3
| |
* | lib/repo: Include WKD update URLs in GPG key listingDan Nicholson2021-07-151-0/+2
| | | | | | | | | | If the key UID contains a valid email address, include the GPG WKD update URLs in GVariant returned by ostree_repo_remote_get_gpg_keys().
* | lib/repo: Add ostree_repo_remote_get_gpg_keys()Dan Nicholson2021-07-151-0/+40
|/ | | | | | | | | | | | | | | | This function enumerates the trusted GPG keys for a remote and returns an array of `GVariant`s describing them. This is useful to see which keys are collected by ostree for a particular remote. The same information can be gathered with `gpg`. However, since ostree allows multiple keyring locations, that's only really useful if you have knowledge of how ostree collects GPG keyrings. The format of the variants is documented in `OSTREE_GPG_KEY_GVARIANT_FORMAT`. This format is primarily a copy of selected fields within `gpgme_key_t` and its subtypes. The fields are placed within vardicts rather than using a more efficient tuple of concrete types. This will allow flexibility if more components of `gpgme_key_t` are desired in the future.
* repo: Require lock type in ostree_repo_lock_popDan Nicholson2021-06-051-4/+5
| | | | | | | | This simplifies the lock state management considerably since the previously pushed type doesn't need to be tracked. Instead, 2 counters are kept to track how many times each lock type has been pushed. When the number of exclusive locks drops to 0, the lock transitions back to shared.
* repo: Make locking APIs publicColin Walters2021-06-051-0/+50
| | | | | | | | Doing anything even somewhat sophisticated requires this; turns out our own `ostree prune` CLI wants this, e.g. https://github.com/ostreedev/ostree/issues/2337 Closes: https://github.com/ostreedev/ostree/issues/2286
* repo: Add ostree_repo_write_regfileColin Walters2021-04-091-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | This API is push rather than pull, which makes it much more suitable to use cases like parsing a tar file from external code. Now, we have a large mess in this area internally because the original file writing code was pull based, but static deltas hit the same problem of wanting a push API, so I added this special `OstreeRepoBareContent` just for writing regular files from a push API. Eventually...I'd like to deprecate the pull based API, and rework things so that for regular files the push API is the default, and then `write_content_object()` would be split up into archive/bare cases. In this world the `ostree_repo_write_content()` API would then need to hackily bridge pull to push and it'd be less efficient. Anyways for now due to this bifurcation, this API only works on non-archive repositories, but that's fine for now because that's what I want for the `ostree-ext-container` bits.
* repo: Add ostree_repo_write_symlinkColin Walters2021-04-081-0/+10
| | | | | | | Continuation of the addition of `ostree_repo_write_regfile_inline()`. This will be helpful for ostree-rs-ext and importing from tar, it's quite inefficient and awkward for small files to end up creating a whole `GInputStream` and `GFileInfo` and etc. for small files.
* repo: Add ostree_repo_write_regfile_inlineColin Walters2021-04-081-0/+12
| | | | | | | | | | | | | | | When working on ostree-ext and importing from tar, it's quite inefficient and awkward for small files to end up creating a whole `GInputStream` and `GFileInfo` and etc. for small files. Plus the gtk-rs binding API to map from `impl Read` to Gio https://docs.rs/gio/0.9.1/gio/struct.ReadInputStream.html requires that the input stream is `Send` but the Rust `tar` API isn't. This is only 1/3 of the problem; we also need similar APIs to directly create a symlink, and to stream large objects via a push-based API.
* Add ostree_repo_gpg_sign_data()Alexander Larsson2020-10-231-0/+10
| | | | | This is similar to ostree_sign_data() but for the old gpg code. Flatpak will need this to reproduce a signed summary.
* deltas: Make ostree_repo_static_delta_reindex() publicAlexander Larsson2020-10-231-0/+17
| | | | | | It is useful to be able to trigger this without having to regenerate the summary. For example, if you are not using summaries, or ar generating the summaries yourself.
* deltas: Add ostree_repo_list_static_delta_indexes() functionAlexander Larsson2020-10-231-0/+6
| | | | This lists all the available delta indexes.
* lib/deltas: Check signed delta in execute_offlineFrédéric Danis2020-09-141-0/+8
| | | | | | | | | | | | | | | | Add a new function `ostree_repo_static_delta_execute_offline_with_signature` which takes a signature engine to verify the delta before applying it. The `ostree_repo_static_delta_execute_offline` is just a wrapper to this new function, passing a NULL signature engine. When this function is called without signature engine, but with a sign delta, it will only fails if `sign-verify-deltas` is set to true in repo core options. This commits move signature existence check and delta signature verification to share common parts between existing APIs and the new function. Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
* lib/deltas: Add signature check API for static-delta superblockFrédéric Danis2020-09-141-0/+8
| | | | | | | This retrieves the signatures and pass the static delta block as an array of bytes to ostree_sign_data_verify(). Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
* lib: Coerce flags enums to GIR bitfieldsDan Nicholson2020-04-301-3/+3
| | | | | | | | | | | | | | | | | | The GI scanner decides if an `enum` is really a `bitfield` if it finds any values that have left shifts. With an `enumeration`, the introspecting language may error or convert to a different type if the user tries to combine values. Change all Flags `enum`s to use left-shifted values so that they're represented as `bitfield`s in the GIR. The primary bug here is that you can't combine `REFS_ONLY` and `NO_PRUNE` when calling `OSTree.Repo.prune()` from an introspected language. This is an IABI break since the typelib will change from `enumeration` to `bitfield`. `OstreeRepoImportFlags` is internal but the change is included here to prepare for a subsequent name that would require bit shifting to operate correctly as a flag.
* lib: Squash two gtk-doc warningsColin Walters2020-04-041-1/+1
| | | | Just noticed these while doing a build.
* repo/commit: Add support for --selinux-policy-from-baseColin Walters2020-03-241-0/+7
| | | | | | | | | | | | | | | The [dev-overlay](https://github.com/coreos/coreos-assembler/blob/332c6ab3b91778d904224c3c960d9cc4739d60bd/src/cmd-dev-overlay) script shipped in coreos-assembler mostly exists to deal with the nontrivial logic around SELinux policy. Let's make the use case of "commit some binaries overlaying a base tree, using the base's selinux policy" just require a magical `--selinux-policy-from-base` argument to `ostree commit`. A new C API was added to implement this in the case of `--tree=ref`; when the base directory is already checked out, we can just reuse the existing logic that `--selinux-policy` was using. Requires: https://github.com/ostreedev/ostree/pull/2039
* fsck: Fix version in docs, tweak error textColin Walters2019-09-091-1/+1
| | | | | | | I think this error message is clearer. Closes: #1918 Approved by: jlebon
* fsck: Implement a partial commit reason bitmaskJason Wessel2019-09-091-17/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After the corruption has been fixed with "ostree fsck -a --delete", a second run of the "ostree fsck" command will print X partial commits not verified and exit with a zero. The zero exit code makes it hard to detect if a repair operation needs to be run. When ever fsck creates a partial commit it should add a reason for the partial commit to the state file found in state/<hash>.commitpartial. This will allow a future execution of the fsck to still return an error indicating that the repository is still in the damaged state, awaiting repair. Additional reason codes could be added in the future for why a partial commit exists. Text from: https://github.com/ostreedev/ostree/pull/1880 ==== cgwalters commented: To restate, the core issue is that it's valid to have partial commits for reasons other than fsck pruned bad objects, and libostree doesn't have a way to distinguish. Another option perhaps is to write e.g. fsck-partial into the statefile state/<hash>.commitpartial which would mean "partial, and expected to exist but was pruned by fsck" and fsck would continue to error out until the commit was re-pulled. Right now the partial stamp file is empty, so it'd be fully compatible to write a rationale into it. ==== Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Closes: #1910 Approved by: cgwalters
* gpg: conditionally build GPG-related code for sign/verificationDenis Pynkin2019-08-011-34/+34
| | | | | | | | | | | | Do not build the code related to GPG sign and verification if GPGME support is disabled. Public functions return error 'G_IO_ERROR_NOT_SUPPORTED' in case if gpg-related check is rquested. Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com> Closes: #1889 Approved by: cgwalters
* lib/commit: Added new ostree_repo_write_archive_to_mtree_from_fd()Tristan Van Berkom2019-05-281-0/+10
| | | | | | | | | | | | Similar to ostree_repo_write_archive_to_mtree(), but takes a file descriptor to read the archive from instead of mandating a file path. Usefull for importing archives into an OSTree repo over a socket or from standard input in command line tools. Closes: #1862 Approved by: jlebon
* lib/repo-refs: Allow resolving local collection-refsMatthew Leeds2019-04-151-0/+2
| | | | | | | | | | | | | Currently for a "normal" refspec you can choose to use ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if you only want to look at local refs (in refs/heads/) not remote ones. This commit provides the analogous functionality for ostree_repo_resolve_collection_ref() by adding a flag OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This will be used by Flatpak. Closes: #1825 Approved by: jlebon
* lib/repo-refs: Add a flag to exclude listing from refs/mirrorsMatthew Leeds2019-04-151-0/+2
| | | | | | | | | | | | | | | | | | | | | | | Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for ostree_repo_list_collection_refs() means that refs in refs/remotes/ should be excluded but refs in refs/mirrors/ should still be checked, in addition to refs/heads/ which is always checked. However in some situations you want to exclude both remote and mirrored refs and only check local "owned" ones. So this commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which lets you exclude refs/mirrors/ from the listing. This way we can avoid breaking API but still allow the listing of local collection-refs. The impetus for this change is that I'm changing Flatpak to make more use of refs/mirrors, and we need a way to specify that a collection-ref is local when using ostree_repo_resolve_collection_ref() in, for example, the implementation of the repo command. The subsequent commit will make the changes needed there. Closes: #1825 Approved by: jlebon
* Add sysroot.bootloader repo config keyRobert Fairley2019-03-011-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The sysroot.bootloader key configures the bootloader that OSTree uses when deploying a sysroot. Having this key allows specifying behavior not to use the default bootloader backend code, which is preferable when creating a first deployment from the sysroot (#1774). As of now, the key can take the values "auto" or "none". If the key is not given, the value defaults to "auto". "auto" causes _ostree_sysroot_query_bootloader() to be used when writing a new deployment, which is the original behavior that dynamically detects which bootloader to use. "none" avoids querying the bootloader dynamically. The BLS config fragments are still written to sysroot/boot/loader/entries for use by higher-level software. More values can be supported in future to specify a single bootloader, different behavior for the bootloader code, or a list of bootloaders to try. Resolves: #1774 Closes: #1814 Approved by: jlebon
* lib/repo: Fix Since: annotation for REMOTE_CHANGE_REPLACEMatthew Leeds2019-02-081-1/+1
| | | | | Closes: #1813 Approved by: jlebon
* repo: Add OstreeRepoRemoteChange replace operationDan Nicholson2019-02-081-1/+3
| | | | | | | | | | | Add the OSTREE_REPO_REMOTE_CHANGE_REPLACE operation to the OstreeRepoRemoteChange enum. This operation will add a remote or replace an existing one. It respects the location of the remote configuration file when replacing and the remotes config dir settings when adding a new remote. Closes: #1166 Approved by: cgwalters
* lib/commit: Add devino_cache_hits to txn statsJonathan Lebon2018-11-051-1/+2
| | | | | | | | | I found this useful while hacking on rpm-ostree but I think it might be useful enough to upstream. This stat is really helpful for validating that a pipeline is hitting the devino cache sweet spot. Closes: #1772 Approved by: cgwalters
* Rename core.repo-finders to core.default-repo-findersMatthew Leeds2018-10-231-1/+1
| | | | | | | | | | | This renames a config key to make its semantics more obvious. Despite what the commit message says, it only applies when a set of repo finders is not specified (either on the command line or in a library API call). This also renames the corresponding ostree_repo_get function. We can do this since it hasn't been released yet. Closes: #1763 Approved by: pwithnall
* Allow disabling pulling from LAN/USB/InternetMatthew Leeds2018-10-211-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently libostree essentially has two modes when it's pulling refs: the "legacy" code paths pull only from the Internet, and the code paths that are aware of collection IDs try to pull from the Internet, the local network, and mounted filesystems (such as USB drives). The problem is that while we eventually want to migrate everyone to using collection IDs, we don't want to force checking LAN and USB sources if the user just wants to pull from the Internet, since the LAN/USB code paths can have privacy[1], security[2], and performance[3] implications. So this commit implements a new repo config option called "repo-finders" which can be configured to, for example, "config;lan;mount;" to check all three sources or "config;mount;" to disable searching the LAN. The set of values mirror those used for the --finders option of the find-remotes command. This configuration affects pulls in three places: 1. the ostree_repo_find_remotes_async() API, regardless of whether or not the user of the API provided a list of OstreeRepoFinders 2. the ostree_repo_finder_resolve_async() / ostree_repo_finder_resolve_all_async() API 3. the find-remotes command This feature is especially important right now since we soon want to have Flathub publish a metadata key which will have Flatpak clients update the remote config to add a collection ID.[4] This effectively fixes https://github.com/flatpak/flatpak/issues/1863 but I'll patch Flatpak too, so it doesn't pass finders to libostree only to then have them be removed. [1] https://github.com/flatpak/flatpak/issues/1863#issuecomment-404128824 [2] https://github.com/ostreedev/ostree/issues/1527 [3] Based on how long the "ostree find-remotes" command takes to complete, having the LAN finder enabled slows down that step of the pull process by about 40%. See also https://github.com/flatpak/flatpak/issues/1862 [4] https://github.com/flathub/flathub/issues/676 Closes: #1758 Approved by: cgwalters
* repo: Add a checkout option to not hardlink zero-sized filesColin Walters2018-10-111-1/+2
| | | | | | | | | | | | | | | | | | | | | In rpm-ostree we've hit a few cases where hardlinking zero-sized files causes us problems. The most prominent is lock files in `/usr/etc`, such as `/usr/etc/selinux/semanage.LOCK`. If there are two zero-sized lock files to grab, but they're hardlinked, then locking will fail. Another case here is if one is using ostree inside a container and don't have access to FUSE (i.e. `rofiles-fuse`), then the ostree hardlinking can cause files that aren't ordinarily hardlinked to become so, and mutation of one mutates all. An example where this is concerning is Python `__init__.py` files. Now, these lock files should clearly not be in the tree to begin with, but - we're not gaining a huge amount by hardlinking these files either, so let's add an option to disable it. Closes: #1752 Approved by: jlebon
* lib/repo: Separate min-free-space-* calculation from transaction codepathUmang Jain2018-09-211-1/+3
| | | | | | | | | | | | | | Earlier, the actual reserved space (in blocks) were calculated inside the transaction codepath ostree_repo_prepare_transaction(). However, while reworking on ostree_repo_get_min_free_space_bytes() API, it was realized that this calculation can be done independently from the transaction's codepaths, hence enabling the usage for ostree_repo_get_min_free_space_bytes() API irrespective of whether there is an ongoing transaction or not. https://github.com/ostreedev/ostree/issues/1720 Closes: #1722 Approved by: pwithnall
* lib/repo: Define a metadata key, ostree.deploy-collection-idMatthew Leeds2018-09-211-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit defines a metadata key that tells clients to update their remote config to add a collection ID. This functionality is currently implemented in Flatpak for the key "xa.collection-id", but there are two good reasons for moving the key to OSTree: 1) Servers such as Flathub shouldn't set xa.collection-id in their metadata now or in the medium term future, because many users are still using old versions of Flatpak and OSTree[1] which would hit various bugs[2][3][4] on the P2P code paths that are enabled by collection IDs. Defining a new key means that only clients running recent (as-yet-unreleased) versions of Flatpak and OSTree will pay attention to it and deploy the collection ID, leaving the users on old versions unaffected. 2) OSTree is as "invested" in collection IDs as Flatpak, so there's no reason the key should be defined in Flatpak rather than here. According to Philip Withnall, the reason the key was put in Flatpak originally was that at the time there was uncertainty about tying OSTree to collection IDs. [1] https://ahayzen.com/direct/flathub.html#downloadsbyflatpakstacked [2] https://github.com/ostreedev/ostree/commit/e4e6d85ea [3] https://github.com/flatpak/flatpak/commit/5813639f [4] https://github.com/flatpak/flatpak/commit/5b21a5b7 Closes: #1726 Approved by: pwithnall
* lib/repo: Add an API to get min-free-space-* reserved bytesUmang Jain2018-09-041-0/+2
| | | | | | | https://phabricator.endlessm.com/T23694 Closes: #1715 Approved by: cgwalters
* Make P2P API public (no longer experimental)Matthew Leeds2018-06-041-27/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the API that allows P2P operations (e.g. pulling an ostree ref from a LAN or USB source) is hidden behind the configure flag --enable-experimental-api. This commit makes the API public and makes that flag essentially a no-op (leaving it in place in case we want to use it again in the future). The P2P API has been tested over the last several months and proven to work. This means that since we're no longer using the "experimental" feature flag, P2P builds of Flatpak will fail when using versions of OSTree from this commit onwards, until Flatpak is patched in the near future. If you want to build Flatpak < 0.11.8 with P2P enabled and link against OSTree 2018.6, you'll have to patch Flatpak. However, since Flatpak won't yet have a hard dependency on OSTree 2018.6, it needs a new way to determine if the P2P API in OSTree is available, so this commit adds a "p2p" feature flag. This way the feature set is more semantically correct than if we had continued to use the "experimental" feature flag. In addition to making the P2P API public, this commit makes the P2P unit tests run by default, removes the f27-experimental CI instance that's no longer needed, changes a few man pages to reflect the changes, and updates the bash completion script to accept the new commands and options. Closes: #1596 Approved by: cgwalters
* lib: Add a public helper method for pruning to find all ref'd commitsColin Walters2018-05-241-0/+8
| | | | | | | | | Prep for reworking how we do sysroot cleanup. We're going to start doing more lowlevel pruning work there, and I wanted to avoid duplicating the ref enumeration. Closes: #1566 Approved by: jlebon
* lib/repo: Enable locking by default, but drop external APIColin Walters2018-04-301-42/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code has been sitting around for a while but since I disabled it by default, I doubt anyone is really using it or relying on it. This patch and turns on locking by default, and also drops the API which was only public in the experimental API builds. Conceptually these are two distinct things, and we may actually want to split up the patches. I don't think this will break anyone, but it's hard to say for sure. It's also going to be hard to find out until we actually release I suspect... But anyone who is broken should be able to add `locking=false` into their repo config. On the flip side Endless has been shipping with this enabled and it is reported to help. The reason to drop the APIs: I'm a bit concerned about the interactions over time between libostree's use of the API and any apps that start using it. For example, if an app specifies a SHARED lock in their code, then later internally we decide to temporarily grab an `EXCLUSIVE`, but the app had a second thread/process that was `EXCLUSIVE` already, and that process was waiting on the first bit of code, then we could deadlock. I can't think of a real world situation where this would happen yet though. We are likely to in the future have say `fsck` take an external lock, `checkout` grab a shared one, etc. Closes: #1555 Approved by: jlebon
* lib/repo: Add ostree_repo_traverse_commit_union_with_parentsAlexander Larsson2018-04-141-0/+14
| | | | | | | | | | This is a version of ostree_repo_traverse_commit_union that also remembers where the objects came from, by recording the parent relationships in a hashtable. This can be used to later find which commits each object was from, which we want to use in fsck. Closes: #1533 Approved by: cgwalters
* lib/checkout: add filter API to skip over filesJonathan Lebon2018-02-061-1/+31
| | | | | | | | | | | | | | | | This is analogous to the filtering support for the commit API: we allow library users to skip over checking out specific files. This is useful in some tricky situations where we *know* that the files to be checked out will conflict with existing files in subtle ways. One such example is in rpm-ostree support for multilib. There, we want to allow checking out a package onto an existing tree, but skipping over files that are not coloured to our preferred value (e.g. not overwriting an i686 version of `ldconfig` if we already have the `x86_64` version). See https://github.com/projectatomic/rpm-ostree/pull/1227 for details. Closes: #1441 Approved by: cgwalters
* Add SPDX-License-Identifier to source filesMarcus Folkesson2018-01-301-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | SPDX License List is a list of (common) open source licenses that can be referred to by a “short identifier”. It has several advantages compared to the common "license header texts" usually found in source files. Some of the advantages: * It is precise; there is no ambiguity due to variations in license header text * It is language neutral * It is easy to machine process * It is concise * It is simple and can be used without much cost in interpreted environments like java Script, etc. * An SPDX license identifier is immutable. * It provides simple guidance for developers who want to make sure the license for their code is respected See http://spdx.org for further reading. Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com> Closes: #1439 Approved by: cgwalters
* lib/repo: Add an API to mark a commit as partialColin Walters2017-12-141-0/+6
| | | | | | | | | | | For the [rpm-ostree jigdo ♲📦](https://github.com/projectatomic/rpm-ostree/issues/1081) work. We're basically doing "pull" via a non-libostree mechanism, and this should be fully supported. As I mentioned earlier we should try to have `ostree-repo-pull.c` only use public APIs; this gets us closer to that. Closes: #1376 Approved by: jlebon
* Add public API for fsck, use it before loading metadataColin Walters2017-12-121-1/+8
| | | | | | | | | | | | | | | | | | | | | | A while ago I did `truncate -s 0 /path/to/repo/00/123.commit`, and expected a checksum error, but I actually got a validation error due to us loading the commit into a variant and trying to parse out the parent checksum, etc. I first started by changing the `load_and_fsck_one_object()` function to checksum before loading, but the problem is that we do a traverse of all objects first. Fixing this is going to require an `OSTREE_REPO_COMMIT_TRAVER_FLAG_FSCK` or something. In the meantime at least though, let's add a public API to fsck a single object which *does* checksum cleanly before parsing the object, and change the `fsck` command to use it. We then change the fsck binary to do this while iterating over the refs and finding the commit object. This way we'll at least get a checksum first for commit objects, even if not dirtree/dirmeta. Closes: #1364 Approved by: jlebon
* lib/repo: Add locking auto cleanup handlerDan Nicholson2017-12-051-0/+18
| | | | | | | | | | | | | | | | | Define an auto cleanup handler for use with repo locking. This is based on the existing auto transaction cleanup. A wrapper for ostree_repo_lock_push() is added with it. The intended usage is like so: g_autoptr(OstreeRepoAutoLock) lock = NULL; lock = ostree_repo_auto_lock_push (repo, lock_type, cancellable, error); if (!lock) return FALSE; The functions and type are marked to be skipped by introspection since I can't see them being usable from bindings. Closes: #1343 Approved by: cgwalters
* lib/repo: Add repo locking mechanismDan Nicholson2017-12-051-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently ostree has no method of guarding against concurrent pruning. When there are multiple repo writers, it's possible to have a pull or commit race against a prune and end up with missing objects. This adds a file based repo locking mechanism. The intention is to take a shared lock when writing objects and an exclusive lock when deleting them. In order to make use of the locking throughout the library in a fine grained fashion, the lock acts recursively with a stack of lock states. If the lock becomes exclusive, it will stay in that state until the stack is unwound past the initial exclusive push. The file locking is similar to GLnxLockFile in that it uses open file descriptor locks but falls back to flock when needed. The lock also attempts to be thread safe by storing the lock state in thread local storage with GPrivate. This means that each thread will have an independent lock for each repository it opens. There are some drawbacks to that, but it seemed impossible to manage the lock state coherently in the face of multithreaded access. The API is a push/pop interface in accordance with the recursive nature of the locking. The push interface uses an enum that's translated to LOCK_SH or LOCK_EX as needed. Both interfaces use an internal timeout field to decide whether to manage the lock in a blocking or non-blocking fashion. The intention is to allow ostree applications as well as administrators to control this timeout. For now, the default is a 30 second timeout. Note that the timeout is handled synchronously in thread since the lock is maintained in thread local storage. I.e., the thread that acquires the lock needs to be the same thread that runs the operation. There may be a way to offer an asynchronous version, but it's not clear exactly how that would work since it would likely involve a separate thread that invokes a callback when the locking operation completes. https://bugzilla.gnome.org/show_bug.cgi?id=759442 Closes: #1343 Approved by: cgwalters
* lib/repo: Add a DEVINO_CANONICAL commit modifier flagColin Walters2017-12-041-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | I was seeing the `Writing OSTree commit...` phase of rpm-ostree being very slow lately. This turns out to be more fallout from https://github.com/ostreedev/ostree/pull/1170 AKA commit: 8fe4536 Loading the xattrs is slow on my system (F27AW, XFS+LVM, NVMe). I haven't fully traced through why, but AIUI at least on XFS the xattrs are often stored outside of the inode so it's a little bit like doing an `open()+read()`. Plus there's the LSM overhead, etc. The thing is that for rpm-ostree's package layering use case, we basically always want to treat the on-disk state as canonical. (There's a subtle case here if one does overrides for something that contains policy but we'll fix that). Anyways, so we're in a state now where we do the slow but correct thing by default, which seems sane. But let's allow the app to opt-in to telling us "really trust devino". The difference between a `stat()` + hash table lookup versus the full xattr load on my test case of `rpm-ostree install ./tree-1.7.0-10.fc27.x86_64.rpm` is absolutely dramatic; consistently on the order of 10s without this support, and <1s with (800ms). Closes: #1357 Approved by: jlebon
* lib/repo: Add gtk-doc comment to OstreeRepoCommitStatePhilip Withnall2017-11-081-0/+12
| | | | | | | Signed-off-by: Philip Withnall <withnall@endlessm.com> Closes: #1335 Approved by: cgwalters
* lib/repo: Add OSTREE_REPO_COMMIT_STATE_NORMAL to represent most commitsPhilip Withnall2017-11-081-0/+1
| | | | | | | | | | This allows more explicit handling of commit state in code using libostree, rather than hard-coding a commit state of 0 for ‘normal’. Signed-off-by: Philip Withnall <withnall@endlessm.com> Closes: #1335 Approved by: cgwalters