summaryrefslogtreecommitdiff
path: root/src/import/pull-raw.c
Commit message (Collapse)AuthorAgeFilesLines
* import: Only keep RO copy if ETag header is setKevin Kuehler2020-03-231-7/+10
| | | | | | | | | | | | | | | | | | | | We fix the case when the webserver servers container images without setting the ETag header in the response. When an image is downloaded to image root, a read only copy is stored alongside it. The filename has the following form: .raw-<encoded-url-of-image>.\x22<ETAG-header>\22.raw. This is so, if the same resource is fetched multiple times, importd can avoid extra downloads by creating the new image using the local read-only copy. The current code assumes the ETag header is set because, if the server does not set the ETag header, the file is stored without the ETag value in the filename. When importd fetches a duplicate image, it will run rename_noreplace and fail: Failed to rename raw file to /var/lib/machines/.raw-http:\x2f\x2flocalhost:8000\x2fwalkthroughd.raw: File exists This patch makes importd only store a read-only image if the webserver has set the ETag header.
* import: don't complain if FS_NOCOW_FL is not availableLennart Poettering2020-01-201-15/+5
| | | | | Let's downgrade the log message to LOG_DEBUG if triggered by an fs that doesn't support the flag.
* util-lib: when copying files make sure to apply some chattrs early, some lateLennart Poettering2019-03-281-1/+1
| | | | | | | Some chattrs only work sensible if you set them right after opening a file for create (think: FS_NOCOW_FL). Others only work when they are applied when the file is fully written (think: FS_IMMUTABLE_FL). Let's take that into account when copying files and applying a chattr to them.
* tree-wide: (void)ify a few unlink() and rmdir()Lennart Poettering2019-03-271-3/+3
| | | | | | Let's be helpful to static analyzers which care about whether we knowingly ignore return values. We do in these cases, since they are usually part of error paths.
* copy: don't synthesize a 'user.crtime_usec' xattr on copy unless explicitly ↵Lennart Poettering2019-03-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | requested Previously, when we'd copy an individual file we'd synthesize a user.crtime_usec xattr with the source's creation time if we can determine it. As the creation/birth time was until recently not queriable form userspace this effectively just propagated the same xattr on the source to the same xattr on the destination. However, current kernels now allow to query the birthtime using statx() and we do make use of that now. Which means that suddenly we started synthesizing these xattrs much more regularly. Doing this actually does make sense, but only in very few cases: not for the typical regular files we copy, but certainly when dealing with disk images. Hence, let's keep this kind of propagation, but let's make it a flag and default to off. Then turn it on whenever we deal with disk images, and leave it off otherwise. This is particularly relevant as overlayfs combining a real fs, and a tmpfs on top will result in EOPNOTSUPP when it is attempted to open a file with xattrs for writing, as tmpfs does not support xattrs, and hence the copy-up cannot work. Hence, let's avoid synthesizing this needlessly, to increase compat with overlayfs.
* util-lib: split out all temporary file related calls into tmpfiles-util.cLennart Poettering2018-12-021-1/+1
| | | | | | | | This splits out a bunch of functions from fileio.c that have to do with temporary files. Simply to make the header files a bit shorter, and to group things more nicely. No code changes, just some rearranging of source files.
* import: drop logic of setting up /var/lib/machines as btrfs loopback mountLennart Poettering2018-11-261-6/+0
| | | | | | | | | | | | | | | | | | | Let's simplify things and drop the logic that /var/lib/machines is setup as auto-growing btrfs loopback file /var/lib/machines.raw. THis was done in order to make quota available for machine management, but quite frankly never really worked properly, as we couldn't grow the file system in sync with its use properly. Moreover philosophically it's problematic overriding the admin's choice of file system like this. Let's hence drop this, and simplify things. Deleting code is a good feeling. Now that regular file systems provide project quota we could probably add per-machine quota support based on that, hence the btrfs quota argument is not that interesting anymore (though btrfs quota is a bit more powerful as it allows recursive quota, i.e. that the machine pool gets an overall quota in addition to per-machine quota).
* import: use structured initializersYu Watanabe2018-11-241-13/+23
|
* chattr: optionally, return the old flags when updating themLennart Poettering2018-10-081-3/+3
|
* tree-wide: remove Lennart's copyright linesLennart Poettering2018-06-141-3/+0
| | | | | | | | | | | These lines are generally out-of-date, incomplete and unnecessary. With SPDX and git repository much more accurate and fine grained information about licensing and authorship is available, hence let's drop the per-file copyright notice. Of course, removing copyright lines of others is problematic, hence this commit only removes my own lines and leaves all others untouched. It might be nicer if sooner or later those could go away too, making git the only and accurate source of authorship information.
* tree-wide: drop 'This file is part of systemd' blurbLennart Poettering2018-06-141-2/+0
| | | | | | | | | | | | | | | | This part of the copyright blurb stems from the GPL use recommendations: https://www.gnu.org/licenses/gpl-howto.en.html The concept appears to originate in times where version control was per file, instead of per tree, and was a way to glue the files together. Ultimately, we nowadays don't live in that world anymore, and this information is entirely useless anyway, as people are very welcome to copy these files into any projects they like, and they shouldn't have to change bits that are part of our copyright header for that. hence, let's just get rid of this old cruft, and shorten our codebase a bit.
* tree-wide: drop license boilerplateZbigniew Jędrzejewski-Szmek2018-04-061-13/+0
| | | | | | | | | | Files which are installed as-is (any .service and other unit files, .conf files, .policy files, etc), are left as is. My assumption is that SPDX identifiers are not yet that well known, so it's better to retain the extended header to avoid any doubt. I also kept any copyright lines. We can probably remove them, but it'd nice to obtain explicit acks from all involved authors before doing that.
* tree-wide: use TAKE_PTR() and TAKE_FD() macrosYu Watanabe2018-04-051-2/+1
|
* macro: introduce new TAKE_FD() macroLennart Poettering2018-03-221-2/+1
| | | | | | | This is similar to TAKE_PTR() but operates on file descriptors, and thus assigns -1 to the fd parameter after returning it. Removes 60 lines from our codebase. Pretty good too I think.
* Replace free and reassignment with free_and_replaceDaniel Lockyer2017-11-241-3/+1
|
* Add SPDX license identifiers to source files under the LGPLZbigniew Jędrzejewski-Szmek2017-11-191-0/+1
| | | | | This follows what the kernel is doing, c.f. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5fd54ace4721fc5ce2bb5aef6318fcf17f421460.
* importd: log more information when renaming filesLennart Poettering2017-06-261-1/+1
|
* import: remove misplaced assert()Lennart Poettering2017-06-261-1/+1
| | | | | | | | We shouldn't assume the final path for the settings file is already known, it unlikely is unless we already downloaded the image once. Also add some commenting to explain the code surrounding the assert a bit. Fixes: #6188
* only check signature job error if signature job exists (#6118)tblume2017-06-211-1/+1
| | | otherwise it will segfault when accessing signature jobs error status
* tree-wide: remove unused variablesThomas Hindoe Paaboel Andersen2017-04-281-1/+0
|
* importd: support SUSE style checksums (#5206)tblume2017-04-241-3/+9
| | | | | | | | | | | In order to verify a pulled container or disk image, importd only supports SHA256SUMS files with the detached signature in SHA256SUMS.gpg. SUSE is using an inline signed file with the name of the image itself and the suffix .sha256 instead. This commit adds support for this type of signature files. It is first attempted to pull the .sha256 file. If this fails with error 404, the SHA256SUMS and SHA256SUMS.gpg files are pulled and used for verification.
* copy: change the various copy_xyz() calls to take a unified flags parameterLennart Poettering2017-02-171-2/+2
| | | | | | | | This adds a unified "copy_flags" parameter to all copy_xyz() function calls, replacing the various boolean flags so far used. This should make many invocations more readable as it is clear what behaviour is precisely requested. This also prepares ground for adding support for more modes later on.
* importd: automatically download .roothash in addition to .nspawn for raw ↵Lennart Poettering2016-12-211-79/+192
| | | | | | | downloads Now that nspawn looks for these files, and mkosi generates them, we should also make sure importd downloads them if it can.
* tree-wide: use mfree moreZbigniew Jędrzejewski-Szmek2016-10-161-3/+1
|
* import: don't claim we had copied a settings file if we didn'tLennart Poettering2016-02-211-2/+4
|
* tree-wide: remove Emacs lines from all filesDaniel Mack2016-02-101-2/+0
| | | | | This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.
* import: report error before losing errnoMichal Schmidt2015-11-051-1/+2
| | | | | unlink() may change errno, so report the error from rename() first, then unlink.
* treewide: use the negative error codes returned by our functionsMichal Schmidt2015-11-051-3/+3
| | | | | Our functions return negative error codes. Do not rely on errno being set after calling our own functions.
* util-lib: split out allocation calls into alloc-util.[ch]Lennart Poettering2015-10-271-0/+1
|
* util-lib: move web-related calls into web-util.[ch]Lennart Poettering2015-10-271-0/+1
|
* util-lib: move a number of fs operations into fs-util.[ch]Lennart Poettering2015-10-271-0/+1
|
* util-lib: split out file attribute calls to chattr-util.[ch]Lennart Poettering2015-10-271-0/+1
|
* util-lib: move more file I/O related calls into fileio.[ch]Lennart Poettering2015-10-271-0/+1
|
* util-lib: split out fd-related operations into fd-util.[ch]Lennart Poettering2015-10-251-1/+2
| | | | | There are more than enough to deserve their own .c file, hence move them over.
* util-lib: split our string related calls from util.[ch] into its own file ↵Lennart Poettering2015-10-241-13/+15
| | | | | | | | | | | | | | string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.
* import: don't claim we moved .nspawn file into place when in fact we did notLennart Poettering2015-10-221-3/+3
|
* tree-wide: never use the off_t unless glibc makes us use itLennart Poettering2015-09-101-1/+1
| | | | | | | | | | | off_t is a really weird type as it is usually 64bit these days (at least in sane programs), but could theoretically be 32bit. We don't support off_t as 32bit builds though, but still constantly deal with safely converting from off_t to other types and back for no point. Hence, never use the type anymore. Always use uint64_t instead. This has various benefits, including that we can expose these values directly as D-Bus properties, and also that the values parse the same in all cases.
* importd: for .raw and .tar images, try to download .nspawn settings file tooLennart Poettering2015-09-081-17/+141
|
* util: make machine_name_is_valid() a macro and move it to hostname-util.hLennart Poettering2015-08-241-0/+1
| | | | | | | | | | | | | As it turns out machine_name_is_valid() does the exact same thing as hostname_is_valid() these days, as it just invoked that and checked the name length was < 64. However, hostname_is_valid() checks the length against HOST_NAME_MAX anyway (which is 64 on Linux), hence any additional check is redundant. We hence replace machine_name_is_valid() by a macro that simply maps it to hostname_is_valid() but sets the allow_trailing_dot parameter to false. We also move this this call to hostname-util.h, to the same place as the hostname_is_valid() declaration.
* util: when creating temporary file names, allow including extra id string in itLennart Poettering2015-06-151-3/+3
| | | | | | | | | | | This adds a "char *extra" parameter to tempfn_xxxxxx(), tempfn_random(), tempfn_ranomd_child(). If non-NULL this string is included in the middle of the newly created file name. This is useful for being able to distuingish the kind of temporary file when we see one. This also adds tests for the three call. For now, we don't make use of this at all, but port all users over.
* util: merge change_attr_fd() and chattr_fd()Lennart Poettering2015-04-081-3/+3
|
* btrfs: support recursively removing btrfs snapshotsLennart Poettering2015-04-061-4/+2
|
* util: rework rm_rf() logicLennart Poettering2015-04-061-1/+2
| | | | | | | | - Move to its own file rm-rf.c - Change parameters into a single flags parameter - Remove "honour sticky" logic, it's unused these days
* util: add rename_noreplaceAlban Crequy2015-03-101-2/+2
| | | | | | | | | renameat2() exists since Linux 3.15 but btrfs support for the flag RENAME_NOREPLACE was added later. This patch implements a fallback when renameat2() returns EINVAL. EINVAL is the error returned when the filesystem does not support one of the flags.
* importd: add new bus calls for importing local tar and raw imagesLennart Poettering2015-03-051-8/+8
| | | | | | | | This also adds "machinectl import-raw" and "machinectl import-tar" to wrap these new bus calls. THe commands basically do for local files that "machinectl pull-raw" and friends do for remote files.
* import: rename download code from "import" to "pull"Lennart Poettering2015-03-051-0/+518
That way we can call the code for local container/VM imports "import" without confusion.