summaryrefslogtreecommitdiff
path: root/src/import
Commit message (Collapse)AuthorAgeFilesLines
* time-util: assume CLOCK_BOOTTIME always existsLennart Poettering2022-03-281-1/+1
| | | | | | | | Let's raise our supported baseline a bit: CLOCK_BOOTTIME started to work with timerfd in kernel 3.15 (i.e. back in 2014), let's require support for it now. This will raise our baseline only modestly from 3.13 → 3.15.
* import: improve error messageLennart Poettering2022-03-151-1/+1
| | | | As suggested: https://github.com/systemd/systemd/pull/20156#discussion_r810941489
* Merge pull request #22092 from keszybz/docs-linksZbigniew Jędrzejewski-Szmek2022-01-121-1/+1
|\ | | | | Add more doc pages, adjust links, add explanatory headers to examples and relax license to CC-0
| * policy files: adjust landing page linkZbigniew Jędrzejewski-Szmek2022-01-121-1/+1
| |
* | meson: Use files() for testsJan Janssen2022-01-111-3/+3
|/ | | | | | Not having to provide the full path in the source tree is much nicer and the produced lists can also be used anywhere in the source tree.
* meson: stop building out convenience libraries by defaultZbigniew Jędrzejewski-Szmek2021-12-161-1/+2
| | | | | | | | | | | | | | | | | The meson default for static_library() are: build_by_default=true, install=false. We never interact with the static libraries, and we only care about them as a stepping-stone towards the installable executables or libraries. Thus let's only build them if they are a dependency of something else we are building. While at it, let's drop install:false, since this appears to be the default. This change would have fixed the issue with lib_import_common failing to build too: we wouldn't attempt to build it. In practice this changes very little, because we generally only declare static libraries where there's something in the default target that will make use of them. But it seems to be a better pattern to set build_by_default to false.
* meson: don't compile import sources four timesZbigniew Jędrzejewski-Szmek2021-12-161-51/+41
| | | | | | | | | | Use a 'convenience library' to do the compilation once and then link the objects into all the files that need it. Those files are small, so this probably doesn't matter too much for speed, but has the advantage that we don't get the same error four times if something goes wrong. The library is conditionalized in the same way importd itself, because we cannot build it without the deps.
* tree-wide: make FOREACH_DIRENT_ALL define the iterator variableZbigniew Jędrzejewski-Szmek2021-12-152-7/+6
| | | | | | | | | The variable is not useful outside of the loop (it'll always be null after the loop is finished), so we can declare it inline in the loop. This saves one variable declaration and reduces the chances that somebody tries to use the variable outside of the loop. For consistency, 'de' is used everywhere for the var name.
* Merge pull request #21170 from keszybz/delibgcryptifyZbigniew Jędrzejewski-Szmek2021-12-032-14/+55
|\ | | | | Allow systemd-resolved and systemd-importd to use libgcrypt or libopenssl
| * import: port importd from libgcrypt to openssl^gcryptZbigniew Jędrzejewski-Szmek2021-11-302-14/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is heavily based on Kevin Kuehler's work, but the logic is also significantly changed: instead of a straighforward port to openssl, both versions of the code are kept, and at compile time we pick one or the other. The code is purposefully kept "dumb" — the idea is that the libgcrypt codepaths are only temporary and will be removed after everybody upgrades to openssl 3. Thus, a separate abstraction layer is not introduced. Instead, very simple ifdefs are used to select one or the other. If we added an abstraction layer, we'd have to remove it again afterwards, and it don't think it makes sense to do that for a temporary solution. Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> # Conflicts: # meson.build
* | tree-wide: use ERRNO_IS_TRANSIENT()Yu Watanabe2021-11-301-3/+4
|/
* shared: clean up mkdir.h/label.h situationLennart Poettering2021-11-166-6/+6
| | | | | | | | | | Previously the mkdir_label() family of calls was implemented in src/shared/mkdir-label.c but its functions partly declared ins src/shared/label.h and partly in src/basic/mkdir.h (!!). That's weird (and wrong). Let's clean this up, and add a proper mkdir-label.h matching the .c file.
* tree-wide: use new RET_NERRNO() helper at various placesLennart Poettering2021-11-162-2/+2
|
* tree-wide: always use TAKE_FD() when calling rearrange_stdio()Lennart Poettering2021-11-033-5/+6
| | | | | | | | | | | | rearrange_stdio() invalidates specified fds even on failure, which means we should always invalidate the fds we pass in no matter what. Let's make this explicit by using TAKE_FD() for that everywhere. Note that in many places we such invalidation doesnt get us much behaviour-wise, since we don't use the variables anymore later. But TAKE_FD() in a way is also documentation, it encodes explicitly that the fds are invalidated here, so I think it's a good thing to always make this explicit here.
* tree-wide: port more code to sigkill_wait()Lennart Poettering2021-11-034-16/+8
|
* tree-wide: port various places to use TAKE_PID()Lennart Poettering2021-11-034-8/+4
|
* basic: split out sync() family of calls from fs-util.[ch] into new c/h fileLennart Poettering2021-10-051-1/+1
| | | | No changes in code, just some splitting out.
* import-fs: create paren dirs in --direct mode tooLennart Poettering2021-08-201-1/+2
| | | | There's no reason not to create these in advance if they are missing.
* import: don't attempt full-file clones if we only are supposed to write a ↵Lennart Poettering2021-08-201-0/+3
| | | | | | part of the file Otherwise we'll copy more than we were told to.
* import: enable sparse file writing logic only for files we createLennart Poettering2021-08-202-2/+2
| | | | | | | | | | | Only if we create a file we know for sure that it is empty and hence our sparse file logic of skipping over NUL bytes can can work. If we hwoever are called to write data to some existing file/block device, we must do regular writes to override everything that might be in place before. Hence, conditionalize sparse file writing on the write offset not being configured (which is how we internally distinguish write to existing file and write to new file)
* import: when completed, say how many bytes we wrote/acquiredLennart Poettering2021-08-202-0/+5
| | | | This is just too useful.
* import: allow file:// in addition to HTTP(S)Lennart Poettering2021-08-205-37/+49
| | | | | | | | | Previously we only allows http/https urls, let's open this up a bit. Why? Because it makes testing *so* *much* *easier* as we don't need to run a HTTP server all the time. CURL mostly abstracts the differences of http/https away from us, hence we can get away with very little extra work.
* import: turn off weird protocols in curlLennart Poettering2021-08-201-0/+3
| | | | | | | | | | | | | | Let's lock things down a bit and now allow curl's weirder protocols to be used with our use. i.e. stick to http:// + https:// + file:// and turn everything else off. (Gopher!) This is cde that interfaces with the network after all, and we better shouldn't support protocols needlessly that are much less tested. (Given that HTTP redirects (and other redirects) exist, this should give us a security benefit, since we will then be sure that noone can forward us to a weird protocol, which we never tested, and other people test neither)
* tree-wide: fix typoYu Watanabe2021-08-181-1/+1
|
* import: drop some now unused functions from import-common.cLennart Poettering2021-08-172-54/+0
|
* import-fs: make various options controllable via cmdline/env varLennart Poettering2021-08-171-97/+165
| | | | | | | | | This basically does what the previous two commits did for systemd-import + systemd-pull but for systemd-import-fs. This commit is a bit simpler though, as a --direct mode doesn't change that much. It's mostly about not searching for existing, conflicting images and not much else.
* pull: add --direct mode + make various eatures optional + explicit checksum ↵Lennart Poettering2021-08-179-516/+1111
| | | | | | | | | | | | | | verification This does what the previous commit did for systemd-import the same way for systemd-pull. It also adds one more thing: the checksum validation is extended, in addition of doing SHA256SUMS/gpg verification it is now possible to immediately specify a hash value on the command line that the download needs to match. This is particularly useful in --direct mode as we can download/decompress/unpack arbitrary files and check the hash of the downloaded file on-the-fly.
* import: add new "--direct" mode + add controls for turning certain features ↵Lennart Poettering2021-08-176-200/+543
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | on/off This reworks/modernizes the tar/raw import logic and adds the following new features: - Adds the ability to control btrfs subvol and quota behaviour which was previously always on via an env var and cmdline arg - Adds control whether to sync() stuff after writing it, similar via env var + cmdline arg - Similar, the QCOW2 unpacking logic that was previously the implied default may now be controlled via env var + cmdline arg. - adds a "direct" mode. In this mode, the systemd-import tool can be used as a simple tool for decompressing/unpacking/installing arbitrary files, without all the additional meta data and auxiliary resources, i.e. outside of the immediate disk image context. Via the new --offset= and --size-max= switches the downloaded data can be written to specific locations of a file (which is particularly useful to use the tool to download fs images and write them to a partition location before actually creating the partition). We'll later use the latter feature for "sysupdate" concept, where images can be directly be written to partitions. That way the systemd-import binary will be used as backend for both "systemd-importd" and "systemd-sysupdate" and share most of the same code.
* shared/copy: add a new flag COPY_ALL_XATTRSAndrej Lajovic2021-08-113-3/+3
| | | | | | | | When the flag COPY_ALL_XATTRS is set, it causes the complete set of xattrs to be copied. If the flag is unset, only xattrs from the "user" namespace are copied. Fixes #17178.
* tree-wide: use memmem_safe()Lennart Poettering2021-08-101-8/+9
| | | | | | | | Let's be paranoid and do something useful if we operate with empty haystack/needle. This doesn't actually fix anything, as the places as far as I can see check for non-emptyness already beforehand, but I will sleep safer at night, if we don't even allow the trap to be fallen in, ever, even if the code is changed sooner or later.
* Drop the text argument from assert_not_reached()Zbigniew Jędrzejewski-Szmek2021-08-039-9/+9
| | | | | | | | | | | | | | | | | In general we almost never hit those asserts in production code, so users see them very rarely, if ever. But either way, we just need something that users can pass to the developers. We have quite a few of those asserts, and some have fairly nice messages, but many are like "WTF?" or "???" or "unexpected something". The error that is printed includes the file location, and function name. In almost all functions there's at most one assert, so the function name alone is enough to identify the failure for a developer. So we don't get much extra from the message, and we might just as well drop them. Dropping them makes our code a tiny bit smaller, and most importantly, improves development experience by making it easy to insert such an assert in the code without thinking how to phrase the argument.
* Merge pull request #20109 from keszybz/timestamp-macrosYu Watanabe2021-07-142-14/+6
|\ | | | | Add macros that define scratch buffer internally for timestamp/timespan formatting
| * import: use SYNTHETIC_ERRNO in one more placeZbigniew Jędrzejewski-Szmek2021-07-091-2/+1
| |
| * tree-wide: add FORMAT_BYTES()Zbigniew Jędrzejewski-Szmek2021-07-092-10/+4
| |
| * tree-wide: add FORMAT_TIMESPAN()Zbigniew Jędrzejewski-Szmek2021-07-091-2/+1
| |
* | tree-wide: make cunescape*() functions return ssize_tZbigniew Jędrzejewski-Szmek2021-07-091-11/+12
|/ | | | | Strictly speaking, we are returning the size of a memory chunk of arbitrary size, so ssize_t is more appropriate than int.
* alloc-util: simplify GREEDY_REALLOC() logic by relying on malloc_usable_size()Lennart Poettering2021-05-192-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | We recently started making more use of malloc_usable_size() and rely on it (see the string_erase() story). Given that we don't really support sytems where malloc_usable_size() cannot be trusted beyond statistics anyway, let's go fully in and rework GREEDY_REALLOC() on top of it: instead of passing around and maintaining the currenly allocated size everywhere, let's just derive it automatically from malloc_usable_size(). I am mostly after this for the simplicity this brings. It also brings minor efficiency improvements I guess, but things become so much nicer to look at if we can avoid these allocation size variables everywhere. Note that the malloc_usable_size() man page says relying on it wasn't "good programming practice", but I think it does this for reasons that don't apply here: the greedy realloc logic specifically doesn't rely on the returned extra size, beyond the fact that it is equal or larger than what was requested. (This commit was supposed to be a quick patch btw, but apparently we use the greedy realloc stuff quite a bit across the codebase, so this ends up touching *a*lot* of code.)
* meson: simplify the BUILD_MODE conditionalZbigniew Jędrzejewski-Szmek2021-04-141-1/+1
| | | | | | | Using a enum is all nice and generic, but at this point it seems unlikely that we'll add further build modes. But having an enum means that we need to include the header file with the enumeration whenerever the conditional is used. I want to use the conditional in log.h, which makes it hard to avoid circular imports.
* tree-wide: sd_bus_error_setf → set_bus_error_setZbigniew Jędrzejewski-Szmek2021-04-071-1/+1
| | | | strdup() is more efficient than asprintf().
* tree-wide: use UINT64_MAX or friendsYu Watanabe2021-03-057-14/+14
|
* btrfs-util: add helper that abstracts "might be btrfs subvol?" checkLennart Poettering2021-03-021-1/+1
| | | | | Let#s not hardcode inode nr 256 everywhere, but abstract this check slightly.
* signal-util: make -1 termination of ignore_signals() argument list unnecessaryLennart Poettering2021-02-253-3/+3
| | | | | | | | | | Clean up ignore_signals() + default_signals() + sigaction_many() a bit: make it unnecessary to explicitly terminate the signal list with -1. Merge all three calls into a single function that is just called with slightly different parameters. And eliminate an unnecessary extra iteration in its inner for() loop. No change in behaviour.
* util: move percent/permille/permyriad parser into percent-util.[ch]Lennart Poettering2021-02-181-0/+1
| | | | | | | A good chunk of parse-util.[ch] has been about parsing parts per hundred/thousand/ten-thousand. Let's split that out into its own file. No code changes, just some shuffling around.
* tree-wide: reset the cleaned-up variable in cleanup functionsZbigniew Jędrzejewski-Szmek2021-02-161-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the cleanup function returns the appropriate type, use that to reset the variable. For other functions (usually the foreign ones which return void), add an explicit value to reset to. This causes a bit of code churn, but I think it might be worth it. In a following patch static destructors will be called from a fuzzer, and this change allows them to be called multiple times. But I think such a change might help with detecting unitialized code reuse too. We hit various bugs like this, and things are more obvious when a pointer has been set to NULL. I was worried whether this change increases text size, but it doesn't seem to: -Dbuildtype=debug: before "tree-wide: return NULL from freeing functions": -rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 4494520 Feb 16 15:06 build/systemd* after "tree-wide: return NULL from freeing functions": -rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 4494576 Feb 16 15:10 build/systemd* now: -rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 4494640 Feb 16 15:15 build/systemd* -Dbuildtype=release: before "tree-wide: return NULL from freeing functions": -rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:09 build-rawhide/systemd* after "tree-wide: return NULL from freeing functions": -rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:10 build-rawhide/systemd* now: -rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0* -rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:16 build-rawhide/systemd* I would expect that the compiler would be able to elide the setting of a variable if the variable is never used again. And this seems to be the case: in optimized builds there is no change in size whatsoever. And the change in size in unoptimized build is negligible. Something strange is happening with size of libsystemd: it's bigger in optimized builds. Something to figure out, but unrelated to this patch.
* tree-wide: use -EINVAL for enum invalid valuesZbigniew Jędrzejewski-Szmek2021-02-104-4/+4
| | | | | | | | | As suggested in https://github.com/systemd/systemd/pull/11484#issuecomment-775288617. This does not touch anything exposed in src/systemd. Changing the defines there would be a compatibility break. Note that tests are broken after this commit. They will be fixed in the next one.
* Merge pull request #18375 from yuwata/cli-tools-also-read-kernel-command-lineZbigniew Jędrzejewski-Szmek2021-02-081-1/+6
|\ | | | | tree-wide: make CLI tools also read kernel command line when run as service
| * import,home: update $SYSTEMD_EXEC_PID= if it is setYu Watanabe2021-02-011-0/+5
| | | | | | | | | | The subsequent execv() or execl() always calls tools provided by systemd. So, it is safe to update the variable.
| * tree-wide: enable colorized logging for daemons when run in consoleYu Watanabe2021-02-011-1/+1
| | | | | | | | It may be useful when debugging daemons.
* | import: use +i (immutable) chattr flag for marking OS images read-onlyLennart Poettering2021-02-041-14/+25
| | | | | | | | | | This is what nspawn and machine-image.[ch] actually look for, hence us it here too.
* | import: fix etag detection supportLennart Poettering2021-02-041-16/+29
| | | | | | | | | | | | | | | | Let's make sure we still look at the etags reported by http 304 (i.e. the cache management code). Otherwise we won't properly realize we already downloaded this before. This fixes a bug introduced in 6792cbbcf84b730f465decbeaf247c6b1ccf1c18
View Lorry Controller Status