summaryrefslogtreecommitdiff
path: root/src/libostree/ostree-repo.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #2496 from lucab/ups/repo-assertsColin Walters2021-12-071-4/+9
|\ | | | | lib/repo: fix problematic invariant checks
| * lib/repo: do no return a NULL on failureLuca BRUNO2021-12-071-2/+4
| | | | | | | | | | | | This turns an existing check into an assert. The previously returned NULL may result in confusing callers, as none of them is checking for that.
| * lib/repo: do no return an arbitrary mode on failureLuca BRUNO2021-12-071-1/+2
| | | | | | | | | | This turns the existing check into an assert. Otherwise, the previous code may return an arbitrary repo mode (bare) on failure.
| * lib/repo: assert that writable state and error agreeLuca BRUNO2021-12-071-1/+3
| | | | | | | | | | This adds an assertion to check that writable stable and error are in sync. The subsequent logic uses them interchangeably.
* | Merge pull request #2494 from jmarrero/fsf-addressColin Walters2021-12-071-3/+1
|\ \ | |/ |/| Update FSF license notices to use URL instead of address
| * Update FSF license notices to use URL instead of addressJoseph Marrero2021-12-071-3/+1
| |
* | repo: Change locking for summary regeneration to be sharedColin Walters2021-12-031-8/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is trying to address: https://pagure.io/fedora-iot/issue/48 Basically we changed rpm-ostree to start doing a shared lock during commit by default, but this broke because pungi is starting a process doing a commit for each architecture, and then trying to regenerate the summary after each one. This patch is deleting a big comment with a rationale for why summary regeneration should be exclusive. Point by point: > This makes sure the commits and deltas don't get > deleted while generating the summary. But prune operations require an exclusive lock, which means that data still can't be deleted when the summary grabs a shared lock. > It also means we can be sure refs > won't be created/updated/deleted during the operation, without having to > add exclusive locks to those operations which would prevent concurrent > commits from working. First: The status quo *has* prevented concurrent commits from working! There is no real locking solution to this problem. What we really need to do here is regenerate the summary after each commit *or* when the caller decides to do it and e.g. include deltas at the same time. It's OK if multiple threads race to regenerate the summary; last-one-wins behavior here is totally fine.
* repo/private: move OstreeRepoAutoTransaction to a boxed typeLuca BRUNO2021-10-111-14/+43
| | | | | | | This defines `OstreeRepoAutoTransaction` as a boxed type, in order to support auto-generating bindings for it. That first requires adding internal reference-counting to it, to allow freely copying/freeing references to a single transaction guard.
* repo/private: allow committing/aborting through a transaction guardLuca BRUNO2021-10-051-0/+112
| | | | | | | | | This enhances the auto-transaction logic, augmenting the scope of a transaction guard. It allows committing or aborting a transaction through its guard. It also supports tracking the completion status of a transaction guard, avoiding double commits/aborts, while retaining the auto-cleanup logic.
* Add support for "custom remotes"Colin Walters2021-09-081-13/+17
| | | | | | | | | | This will be helpful for the "ostree native container" work in https://github.com/ostreedev/ostree-rs-ext/ Basically in order to reuse GPG/signapi verification, we need to support adding a remote, even though it can't be used via `ostree pull`. (At least, not until we merge ostree-rs-ext into ostree, but even then I think the principle stands)
* fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys()Dan Nicholson2021-07-151-1/+1
|
* fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys()Dan Nicholson2021-07-151-5/+5
|
* lib/repo: Include WKD update URLs in GPG key listingDan Nicholson2021-07-151-0/+14
| | | | | If the key UID contains a valid email address, include the GPG WKD update URLs in GVariant returned by ostree_repo_remote_get_gpg_keys().
* lib/repo: Add ostree_repo_remote_get_gpg_keys()Dan Nicholson2021-07-151-0/+124
| | | | | | | | | | | | | | | | This function enumerates the trusted GPG keys for a remote and returns an array of `GVariant`s describing them. This is useful to see which keys are collected by ostree for a particular remote. The same information can be gathered with `gpg`. However, since ostree allows multiple keyring locations, that's only really useful if you have knowledge of how ostree collects GPG keyrings. The format of the variants is documented in `OSTREE_GPG_KEY_GVARIANT_FORMAT`. This format is primarily a copy of selected fields within `gpgme_key_t` and its subtypes. The fields are placed within vardicts rather than using a more efficient tuple of concrete types. This will allow flexibility if more components of `gpgme_key_t` are desired in the future.
* lib/repo: Allow preparing GPG verifier without global keyringsDan Nicholson2021-07-151-3/+4
| | | | | | | | Currently the verifier decides whether to include the global keyrings based on whether the specified remote has its own keyring or not. Allow callers to exclude the global keyrings even when that's not the case. This will be used in a subsequent commit in order to get the GPG keys only associated with a remote.
* lib/repo: Factor out GPG verifier preparationDan Nicholson2021-07-151-20/+43
| | | | | | | In order to use the GPG verifier, it needs to be seeded with GPG keys after instantation. Currently this is only used for verifying data, but it will also be used for getting a list of trusted GPG keys in a subsequent commit.
* repo: Use g_new for OstreeRepoAutoLockDan Nicholson2021-06-071-2/+2
| | | | | GSlice is effectively deprecated and has little to no advantage over using the system allocator on Linux.
* repo: Make locking precondition failures fatalDan Nicholson2021-06-051-5/+11
| | | | | | | | | | | Use `g_error` and `g_assert*` rather than `g_return*` when checking the locking preconditions so that failures result in the program terminating. Since this code is protecting filesystem data, we'd rather crash than delete or corrupt data unexpectedly. `g_error` is used when the error is due to the caller requesting an invalid transition like attempting to pop a lock type that hasn't been taken. It also provides a semi-useful message about what happened.
* repo: Make locking per-OstreeRepoDan Nicholson2021-06-051-106/+59
| | | | | | | | | | | | | | | | Previously each thread maintained its own lock file descriptor regardless of whether the thread was using the same `OstreeRepo` as another thread. This was very safe but it made certain multithreaded procedures difficult. For example, if a main thread took an exclusive lock and then spawned worker threads, it would deadlock if one of the worker threads tried to acquire the lock. This moves the file descriptor from thread local storage to the `OstreeRepo` structure so that threads using the same `OstreeRepo` can share the lock. A mutex guards against threads altering the lock state concurrently. Fixes: #2344
* repo: Require lock type in ostree_repo_lock_popDan Nicholson2021-06-051-61/+121
| | | | | | | | This simplifies the lock state management considerably since the previously pushed type doesn't need to be tracked. Instead, 2 counters are kept to track how many times each lock type has been pushed. When the number of exclusive locks drops to 0, the lock transitions back to shared.
* repo: Make locking APIs publicColin Walters2021-06-051-23/+28
| | | | | | | | Doing anything even somewhat sophisticated requires this; turns out our own `ostree prune` CLI wants this, e.g. https://github.com/ostreedev/ostree/issues/2337 Closes: https://github.com/ostreedev/ostree/issues/2286
* tests: Test without a cache directory by defaultDan Nicholson2021-04-191-1/+1
| | | | | | | | | | | | | | | | | | | Several tests generate summaries and then expect to use the generated summary immediately. However, this can cause intermittent test failures when they inadvertantly get a cached summary file. This typically happens when the test is run on a filesystem that doesn't support user extended attributes. In that case, the caching code can only use the last modified time, which only has 1 second granularity. If tests don't carefully manage the summary modification times or the repo cache then they are likely subject to races in some test environments. This introduces an environment variable `OSTREE_SKIP_CACHE` that prevents the repo from using a cache directory. This is enabled by default in tests and disabled for tests that are a explicitly trying to test the caching behavior. Fixes: #2313 Fixes: #2351
* repo: Ensure load_variant_if_exists sets NULL valueColin Walters2021-04-031-0/+4
| | | | | Spotted by @jlebon - we usually expect callers to zero, but let's be explicit.
* repo: Fix load_variant_if_exists to return a nullable valueColin Walters2021-04-031-2/+3
| | | | Another introspection fix I hit when trying to use this in Rust.
* Merge pull request #2263 from cgwalters/createat-nullableOpenShift Merge Robot2021-01-121-1/+1
|\ | | | | repo: Make ostree_repo_create_at take nullable options
| * repo: Make ostree_repo_create_at take nullable optionsColin Walters2021-01-121-1/+1
| | | | | | | | Hit this when trying to use the Rust bindings.
* | repo: Move fsverity bits to ostree-repo-verity.cColin Walters2021-01-111-26/+1
|/ | | | This file will get larger when we start doing more with fsverity.
* Merge pull request #2226 from alexlarsson/fix-gir-argOpenShift Merge Robot2020-10-281-1/+1
|\ | | | | ostree_repo_gpg_sign_data: Fix API doc argument name
| * ostree_repo_gpg_sign_data: Fix API doc argument nameAlexander Larsson2020-10-281-1/+1
| | | | | | | | | | I got: src/libostree/ostree-repo.c:5232: Warning: OSTree: ostree_repo_gpg_sign_data: unknown parameter 'out_signature' in documentation comment, should be 'out_signatures'
* | Refactor: sysroot.bootloader: Store enum value rather than stringWilliam Manley2020-10-261-21/+20
| | | | | | | | | | | | It's easier to extend and it centralises the config parsing. In other places we will no longer need to use `g_str_equal` to match these values, a `switch` statement will be sufficient.
* | ostree_repo_get_bootloader: Document transfer noneWilliam Manley2020-10-261-1/+1
|/ | | | I think this may affect bindings too.
* Add ostree_repo_gpg_sign_data()Alexander Larsson2020-10-231-0/+61
| | | | | This is similar to ostree_sign_data() but for the old gpg code. Flatpak will need this to reproduce a signed summary.
* deltas: Set `indexed-deltas` key in the config and summaryAlexander Larsson2020-10-231-0/+3
| | | | | | Clients can use these during pull and avoid downloading the summary if needed, or use the indexed-deltas instead of relying on the ones in the summary which may be left out.
* deltas: Make ostree_repo_static_delta_reindex() publicAlexander Larsson2020-10-231-1/+1
| | | | | | It is useful to be able to trigger this without having to regenerate the summary. For example, if you are not using summaries, or ar generating the summaries yourself.
* deltas: Add and document no-deltas-in-summary config optionAlexander Larsson2020-10-231-23/+31
| | | | By default this is FALSE to keep existing clients working.
* deltas: Update delta indexes when updating summaryAlexander Larsson2020-10-231-0/+3
| | | | | | | | | | | | When we update the summary file (and its list of deltas) we also update all delta indexes. The index format is a single `a{sv}` variant identical to the metadata-part of the summary with (currently) only the `ostree.static-deltas` key. Since we expect most delta indexes to change rarely, we avoid unnecessary writes when reindexing. New indexes are compared to existing ones and only the changed ones are written to disk. This avoids unnecessary write load and mtime changes on the repo server.
* lib/repo: Add mode and tombstone config options to the summary filePhilip Withnall2020-10-011-0/+18
| | | | | | | | | | | Currently, they are set in the `config` file and cause that to be downloaded on every pull. Given that the client is already pulling the `summary` file, it makes sense to avoid an additional network round trip and cache those options in the `summary` file. Signed-off-by: Philip Withnall <withnall@endlessm.com> Helps: #2165
* Avoid shadowing local variablesavoid-shadowsPhaedrus Leeds2020-09-131-9/+9
| | | | | | This should help with code readability. Fixes https://github.com/ostreedev/ostree/issues/2194
* deltas: Break out _ostree_repo_static_delta_superblock_digest() helperAlexander Larsson2020-09-111-14/+7
| | | | | | | This loads and makes a digest for a delta superblock. The previous code was used when generating the deltas section in the summary file. This changes nothing, but is in preparation for using similar formats in a separate delta index file.
* pull: Add --per-object-fsyncColin Walters2020-07-181-0/+4
| | | | | | | | | | | | This is the opposite of https://github.com/ostreedev/ostree/issues/1184 Motivated by OpenShift seeing etcd performance issues during OS updates: https://github.com/openshift/machine-config-operator/issues/1897 Basically, if we switch to invoking `fsync()` as we go, it makes ostree performance worse (in my tests, 31s to write 2G versus 7s if we delay sync) but it avoids *huge* outliers in `fsync()` time for etcd.
* lib/repo: Handle EACCES for POSIX lockingJonathan Lebon2020-06-301-1/+3
| | | | | | | | | | If `glnx_make_lock_file` falls back to `flock`, on NFS this uses POSIX locks (`F_SETLK`). As such, we need to be able to handle `EACCES` as well as `EAGAIN` (see `fnctl(2)`). I think this is what coreos-ostree-importer has been hitting, which runs on RHEL7 in the Fedora infra and does locking over an NFS share where multiple apps could concurrently pull things into the repo.
* lib: Rename function for staging dir checkJonathan Lebon2020-04-171-2/+2
| | | | | Rename the function to more accurately reflect what it does, which is to check whether the filename has the `staging-` prefix.
* lib: Move internal binding verification API to repo.cColin Walters2020-04-171-0/+104
| | | | | | | | `ostree-repo-pull.c` is rather monstrous; I plan to split it up a bit. There's actually already a `pull-private.h` but that's just for the binding verification API. I think that one isn't really pull specific. Let's move it into the "catchall" `repo.c`.
* Merge pull request #1878 from d4s/wip/d4s/no_gpgOpenShift Merge Robot2020-04-041-47/+9
|\ | | | | Alternative signing system
| * ostree-repo: improve error handlingDenis Pynkin2020-03-251-29/+9
| | | | | | | | | | | | | | | | | | | | Correctly return "error" from `ostree_repo_sign_commit()` in case if GPG is not enabled. Use glnx_* functions in signature related pull code for clear error handling if GPG isn't enabled. Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
| * gpg: do not fail GPG-related configuration get for remoteDenis Pynkin2020-03-251-18/+0
| | | | | | | | | | | | | | We don't need anymore stubs for verification options for remotes in case if ostree built without GPG support. Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
* | lib: Add error prefixing for sysroot load and repo openColin Walters2020-03-281-4/+3
|/ | | | | | | | Noticed this while writing tests for a core `ostree_sysroot_load()` entrypoint. And decided to do the same for `ostree_repo_open()`, and while there also noted we had a duplicate error prefixing for the open (more recently `glnx_opendirat()` automatically prefixes with the path).
* Merge pull request #1984 from jlebon/pr/supp-groupsOpenShift Merge Robot2020-02-201-6/+6
|\ | | | | lib/repo: Create repo directories as 0775
| * lib/repo: Create repo directories as 0775Jonathan Lebon2020-02-181-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | For repo structure directories like `objects`, `refs`, etc... we should be more permissive and let the system's `umask` narrow down the permission bits as wanted. This came up in a context where we want to be able to have read/write access on an OSTree repo on NFS from two separate OpenShift apps by using supplemental groups[1] so we don't require SCCs for running as the same UID (supplemental groups are part of the default restricted SCC). [1] https://docs.openshift.com/container-platform/3.11/install_config/persistent_storage/persistent_storage_nfs.html#nfs-supplemental-groups
* | Initial fs-verity supportColin Walters2020-01-271-0/+30
|/ | | | | | | | | | | | | | | | | | Using fs-verity is natural for OSTree because it's file-based, as opposed to block based (like dm-verity). This only covers files - not symlinks or directories. And we clearly need to have integrity for the deployment directories at least. Also, what we likely need is an API that supports signing files as they're committed. So making this truly secure would need a lot more work. Nevertheless, I think it's time to start experimenting with it. Among other things, it does *finally* add an API that makes files immutable, which will help against some accidental damage. This is basic enablement work that is being driven by Fedora CoreOS; see also https://github.com/coreos/coreos-assembler/pull/876
View Lorry Controller Status