| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
| |
Found by ASAN.
|
|
|
|
|
| |
Without this you can't create a scratch delta from GI. While here,
switch the deprecated allow-none annotations to nullable.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While the commits contained in the single static-delta file are signed so
we can check them and operate on trusted data, the superblock isn't signed
in any way, so it end up operating on untrusted data to:
1. actually find where the trusted data is, and
2. check whether the update is fit for the current device by looking at
the collection id stored in the metadata
This commit generates signatures of all static data, and concatenate them
to the existing static delta format, i.e. as a GVariant layout `a{sv}ay`
where
- a{sv}: signatures
- ay: existing delta variant
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
|
|
|
|
|
|
|
| |
The "new style" code generally avoids `goto err` because it conflicts
with `__attribute__((cleanup))`. This fixes a compiler warning.
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For repo structure directories like `objects`, `refs`, etc... we should
be more permissive and let the system's `umask` narrow down the
permission bits as wanted.
This came up in a context where we want to be able to have read/write
access on an OSTree repo on NFS from two separate OpenShift apps by
using supplemental groups[1] so we don't require SCCs for running as the
same UID (supplemental groups are part of the default restricted SCC).
[1] https://docs.openshift.com/container-platform/3.11/install_config/persistent_storage/persistent_storage_nfs.html#nfs-supplemental-groups
|
|
|
|
|
|
|
| |
There is no need to ref the argument of g_variant_builder_add_value
Closes: #1701
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
| |
I initially was going to add a `G_DEFINE_AUTOPTR_CLEANUP_FUNC` for
`FetchStaticDeltaData`, but it honestly didn't seem worth mucking around
ownership everywhere and potentially getting it wrong.
Discovered by Coverity.
Closes: #1692
Approved by: cgwalters
|
|
|
|
|
|
|
|
| |
These show up in the RPM build, I didn't yet try to figure out why we're not
reproducing them outside of that.
Closes: #1510
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.
Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
license for their code is respected
See http://spdx.org for further reading.
Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Closes: #1439
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently we were parsing `opt_filename` twice...I dug through
the history a bit and it looks like it may have been an accident
from refactoring.
What we're fixing here concretely is that using relative subdirectories
like `--filename somesubdir/foo` broke because we were incorrectly
passing the `somesubdir/` again.
Closes: #1423
Closes: #1427
Approved by: jlebon
|
|
|
|
|
|
|
| |
I used this to find the bsdiff'd objects.
Closes: #1314
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
| |
It's the slowest part, let's show admins something. This "update every 10%" code
was copied from the fsck command; obviously a better approach would be "progress
every N seconds" but doing that somewhat accurately requires making things
async; not worth it here yet.
Closes: #1314
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
First, the manual crosscheck script bitrotted; it got caught up
in the "use libtest repo creation wrapper" bit, and also it
seems like at some point `pull --require-static-deltas` changed
meaning when dealing with `file:///` repos. I have more work to
unwind that.
Next, I'm seeing a delta failure which looks like a static delta
miscompilation with rollsums; change the compiler to print out
the source object too, which helped me debug this.
And finally in the processing code, fix incorrect error prefixing, which was
misleading.
Closes: #1311
Approved by: ashcrow
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Directly when we allocate a new part we finish the old one,
writing the compressed data to a temporary file and generating
the delta header for it.
When all these are done we loop over them and collect the headers,
sizes and either copy the tempfile data into the inlined superblock
or link the tempfiles to disk with the proper names.
Closes: #1309
Approved by: cgwalters
|
|
|
|
|
|
|
| |
We will do some changes later that need these earliers, so move them up.
Closes: #1309
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This allows us to create the final delta desciptor directly on disk
rather than having it all in memory. This is nice because it can
become quite large if inlined parts are used.
Note however, that we currently generate all the delta parts in
memory before adding them to the delta, so we still keep all individual
parts in memory. Fixing that is the next step.
Closes: #1309
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This ends up a lot better IMO. This commit is *mostly* just
`s/glnx_close_fd/glnx_autofd`, but there's also a number of hunks like:
```
- if (self->sysroot_fd != -1)
- {
- (void) close (self->sysroot_fd);
- self->sysroot_fd = -1;
- }
+ glnx_close_fd (&self->sysroot_fd);
```
Update submodule: libglnx
Closes: #1259
Approved by: jlebon
|
|
|
|
|
|
|
|
|
| |
What the deltas code is doing is weird/unfortunate. The name
`ot_variant_read()` conflicts too much with `ot_variant_read_fd()`.
Since nothing else uses it, move it into the deltas code.
Closes: #1254
Approved by: jlebon
|
|
|
|
|
|
|
|
|
| |
We added a `.dir-locals.el` in commit: 9a77017d87b74c5e2895cdd64ad098018929403f
There's no need to have it per-file, with that people might think
to add other editors, which is the wrong direction.
Closes: #1206
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I added `glnx_open_anonymous_tmpfile()`, but then later noticed
that the usage of this was really to be combined with `mmap()`,
and we had two versions of that in the delta code. Add a helper.
(Bigger picture...how is this different from glibc's "mmap() of /dev/zero"
approach for large chunks? One advantage is the storage can be "swapped" to
`/var/tmp`, but still deleted automatically, rather than requiring swap space)
Closes: #973
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Use the new tmpfile bits
- `glnx_try_fallocate`
- `glnx_renameat()`
Depends: https://github.com/GNOME/libglnx/pull/57
Update submodule: libglnx
Closes: #970
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There's lots of mechanically replacing `OtTmpFile` with `GLnxTmpfile`;
the biggest changes are in the commit path. Symlink commits are now
very clearly separated from regular files. Symlinks are `OtCleanupUnlinkat`,
and regular files are `GLnxTmpfile`.
The commit codepath separates those as `_ostree_repo_commit_path_final()` and
`_ostree_repo_commit_tmpf_final()`. A nice aspect of all of this is that they
both *consume* the temporary on success. This avoids an extra spurious
`unlink()` call.
One of the biggest bits of code motion is in `commit_loose_regfile_object()`,
which no longer needs to care about symlinks. For the most parth though it's
just removing conditionals.
Update submodule: libglnx
Closes: #958
Approved by: jlebon
|
|
|
|
|
|
|
| |
Looking at the uses of `ostree_repo_load_file()` here.
Closes: #953
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In https://github.com/ostreedev/ostree/pull/634 we introduced
a subtle regression - the unreadable object was added to the *new*
reachable objects, when it shouldn't have been. Because it
was a *from* object, clients already had it.
This became more obvious now that I'm working on fixing delta
progress - I noticed my deltas were always starting out with 40MB
fetched, which turned out to be a non-world-readable initramfs object.
This code should simply *skip* the unreadable object, and the delta processing
below properly iterates over "new objects", so we'll pick it up from there.
Closes: #678
Approved by: giuseppe
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will prevent including in the delta the bits to update files that
are not world readable, so that we don't run into a permissions problem
when applying the deltas from a bare-user repository that has a bare
repository set as its parent.
This is the case for Endless when updating flatpak runtimes, as the
temporary directory created in ~/.local/share/flatpak/system-cache will
be of type bare-user with its parent set to /var/lib/flatpak which is a
bare repository in EOS, as it's shared with the one at /ostree/repo.
https://phabricator.endlessm.com/T14159
Closes: #634
Approved by: cgwalters
|
|
|
|
|
|
|
|
| |
Now that I remembered to do `env G_SLICE=always-malloc`, lots more
leaks become apparent. Nothing major.
Closes: #598
Approved by: jlebon
|
|
|
|
|
|
|
| |
We need to ref-sik the new varian for g_autoptr to work
Closes: #597
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is actually fine in practice, but it triggers this
`-fsanitize=undefined` warning I saw in the test suite log:
```
src/libostree/ostree-repo-static-delta-compilation.c:160:10: runtime error: null pointer passed as argument 1, which is declared to never be null
```
Closes: #584
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Otherwise it's possible for us to exhaust available file descriptors
or (on 32 bit) run up against mmap limits.
In the rollsum case, we didn't need to hold open the "from" object
at all. And in the bsdiff case, we weren't even looking at either of
the files until we started processing.
Also, while we have the patient open, switch to using O_TMPFILE
if available.
Closes: #567
Approved by: giuseppe
|
|
|
|
|
|
|
|
| |
So we build warning-free on GLib (< 2.50, >= 2.50). This
is a band aid until we hard-require >= 2.50.
Closes: #547
Approved by: jlebon
|
|
|
|
|
|
|
|
|
|
| |
CLang finds these, whereas GCC treats having
`__attribute__((cleanup))` as a use.
This obsoletes https://github.com/ostreedev/ostree/pull/411
Closes: #548
Approved by: jlebon
|
|
|
|
|
|
|
|
| |
This should help avoid address space exhaustion on 32 bit systems,
and in general is obviously going to improve efficiency.
Closes: #498
Approved by: alexlarsson
|
|
|
|
|
|
|
|
| |
This was hit in practice when generating a delta for a flatpak app on ARM
it looks like.
Closes: #497
Approved by: alexlarsson
|
|
|
|
|
|
|
|
|
|
| |
You need to supply an argument to F_DUPFD_CLOEXEC or fcntl will return
EINVAL. Use 3 as the minimum fd number as is standard.
Closes: #454
Closes: #448
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
| |
If there's an early error, part_temp_fds will be NULL and dereferencing
the len member will segfault.
Closes: #454
Closes: #448
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
| |
I think we'd end up closing stdin...yuck. Need to investigate
having this be fatal, but CentOS 7 `gcc-4.8.5` doesn't understand
`-Werror=int-conversion`.
Closes: #430
Approved by: giuseppe
|
|
|
|
|
|
|
|
|
|
| |
This was the last use of libgsystem tmpfile APIs. The change here is
a bit uglier than it needs to be because we support creating the delta
in place as well as in the repo, so we needed to abstract over
locations.
Closes: #424
Approved by: giuseppe
|
|
|
|
|
|
|
|
| |
The init-fs code in particular gets (IMO) a lot cleaner being fd
relative.
Closes: #312
Approved by: giuseppe
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Apparently I got the bracketing wrong in
862e6ecdcc58f025696b1394adfc0fcf7322df23:
src/libostree/ostree-repo.c: In function 'ostree_repo_delete_object':
src/libostree/ostree-repo.c:3538:11: warning: missing braces around
initializer [-Wmissing-braces]
g_auto(GVariantBuilder) builder = {0,};
Closes: #298
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This tries to avoid leaking GVariantBuilders and GVariants in some
situations. The leaks were usually happening when some error occurred
or because of unclear variant ownership situation.
The former is mostly about making sure that g_variant_builder_clear is
called on builders that didn't finish their variant building process.
The latter is surely more work - sometimes the result of
g_variant_builder_end() should not be passed directly to a function,
but rather stored in a g_autoptr(GVariant), sunk and then passed to a
function. IMO, with an advent of g_autoptr, GVariants should be always
sunk instead of relying on some receiver function sinking it. This
would make an easy-to-follow policy of always sinking your
variants. Functions could then assume that the passed variant is
already sunk. These leaks are still happenning in commands, but they
are less harmful, since that code will not be used by some daemon as a
library routine.
Closes: #291
Approved by: cgwalters
|
|
|
|
|
| |
Closes: #267
Approved by: cgwalters
|
|
|
|
|
|
|
| |
We may not have write permissions in the current directory.
Closes: #259
Approved by: cgwalters
|
|
|
|
|
|
|
|
|
| |
:warning: There is a notable spiked pit trap here around
`posix_fallocate()` and `errno`. This has bit other projects,
see e.g.
https://github.com/systemd/systemd/commit/7bb87460e691d30c1a7fd23a1a8240776957e05f
Otherwise the port was straightforward.
|
|
|
|
| |
It's a bit more verbose but...eh.
|
|
|
|
| |
I'd like to eventually kill libgsystem.
|
|
|
|
|
| |
Extend the `static-delta show` and `pull` commands to use the
endianness information (if available).
|
|
|
|
|
|
|
| |
We screwed up and had delta integers use host endianness. Start
digging out by at least annotating the endianness.
https://bugzilla.gnome.org/show_bug.cgi?id=762515
|
|
|
|
| |
And use it internally. This way it's a bit less magical.
|
|
|
|
|
|
|
|
|
|
| |
This is very useful for the inline-parts case, as you can then include
detached signatures in a single file representing the commit.
It is not as important for the generic pull case, as the detached
metadata is only a single small file. Additionally the detached
metadata is not content referenced and may change after the static
delta file was created, so we need to pull the latest version anyway.
|