summaryrefslogtreecommitdiff
path: root/src/journal/journal-internal.h
Commit message (Collapse)AuthorAgeFilesLines
* sd-journal: move source files for sd-journal to src/libsystemd/sd-journalYu Watanabe2021-01-191-138/+0
|
* license: LGPL-2.1+ -> LGPL-2.1-or-laterYu Watanabe2020-11-091-1/+1
|
* sd-journal: when enumerating, continue even after an inaccessible fieldZbigniew Jędrzejewski-Szmek2020-07-211-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | SD_JOURNAL_FOREACH_DATA() and SD_JOURNAL_FOREACH_UNIQUE() would immediately terminate when a field couldn't be accessed. This can happen for example when a field is compressed with an unavailable compression format. But it's likely that this is the wrong thing to do: the caller for example might want to iterate over the fields but isn't interested in all of them. coredumpctl is like this: it uses SD_JOURNAL_FOREACH_DATA() but only uses a subset of the fields. Add two new functions sd_journal_enumerate_good_data() and sd_journal_enumerate_good_unique() that retry sd_journal_enumerate_data() and sd_journal_enumerate_unique() if the return value is something that applies to a single field: ENOBUS, E2BIG, EOPNOTSUPP. Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1856037. An alternative would be to make the macros themselves smarter instead of adding new symbols, and do the looping internally in the macro. I don't like that approach for two reasons. First, it would embed the logic in the macro, so recompilation would be required if we decide to update the logic. With the current version of the patch, recompilation is required to use the new symbols, but after that, library upgrades are enough. So the current approach is safer in case further updates are needed. Second, our headers use primitive C, and it is hard to do the macros without using newer features.
* journal: use a different hash function for each journal fileLennart Poettering2020-06-251-1/+1
| | | | | | | | | | | | | | This adds a new (incompatible) feature to journal files: if enabled the hash function used for the hash tables is no longer jenkins hash with a zero key, but siphash keyed by the file uuid that is included in the file header anyway. This should make our hash tables more robust against collision attacks, as long as the attacker has no read access to the journal files. We switch from jenkins to siphash simply because it's more well-known and we standardize for the rest of our codebase onto it. This is hardening in order to make collision attacks harder for clients that can forge log messages but have no read access to the logs. It has no effect on clients that have read access.
* journal: store NE hash instead of LE hash in Match objectLennart Poettering2020-06-251-1/+1
| | | | | We keep converting forth and back though we never need it in LE. Let's stop doing those conversions hence.
* journal: use a bitfield where appropriateLennart Poettering2020-04-231-4/+4
|
* journal: allow opening journal files specific to some namespaceLennart Poettering2020-01-311-0/+1
|
* tree-wide: remove Lennart's copyright linesLennart Poettering2018-06-141-4/+0
| | | | | | | | | | | These lines are generally out-of-date, incomplete and unnecessary. With SPDX and git repository much more accurate and fine grained information about licensing and authorship is available, hence let's drop the per-file copyright notice. Of course, removing copyright lines of others is problematic, hence this commit only removes my own lines and leaves all others untouched. It might be nicer if sooner or later those could go away too, making git the only and accurate source of authorship information.
* tree-wide: drop 'This file is part of systemd' blurbLennart Poettering2018-06-141-2/+0
| | | | | | | | | | | | | | | | This part of the copyright blurb stems from the GPL use recommendations: https://www.gnu.org/licenses/gpl-howto.en.html The concept appears to originate in times where version control was per file, instead of per tree, and was a way to glue the files together. Ultimately, we nowadays don't live in that world anymore, and this information is entirely useless anyway, as people are very welcome to copy these files into any projects they like, and they shouldn't have to change bits that are part of our copyright header for that. hence, let's just get rid of this old cruft, and shorten our codebase a bit.
* tree-wide: drop license boilerplateZbigniew Jędrzejewski-Szmek2018-04-061-13/+0
| | | | | | | | | | Files which are installed as-is (any .service and other unit files, .conf files, .policy files, etc), are left as is. My assumption is that SPDX identifiers are not yet that well known, so it's better to retain the extended header to avoid any doubt. I also kept any copyright lines. We can probably remove them, but it'd nice to obtain explicit acks from all involved authors before doing that.
* sd-journal: properly handle inotify queue overflowLennart Poettering2018-02-121-0/+2
| | | | | | | | | | | | | | | | | This adds proper handling of IN_Q_OVERFLOW: when the inotify queue runs over we'll reiterate all directories we are looking at. At the same time we'll mark all files and directories we encounter that way with a generation counter we first increased. All files and directories not marked like this are then unloaded. With this logic we do the best when the inotify queue overflows: we synchronize our in-memory state again with what's on disk. This contains some refactoring of the directory logic, to share more code between uuid directories and "root" directories and generally make things a bit more readable by splitting things up into smaller bits. See: #7998 #8032
* journal: use IteratedCache in sd-journalVito Caputo2018-01-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes real_journal_next() to leverage the IteratedCache for accelerating iteration across the open journal files. journalctl timing comparisons with 100 journal files of 8MiB size party to this boot: Pre (~v235): # time ./journalctl -b --no-pager > /dev/null real 0m9.613s user 0m9.560s sys 0m0.053s # time ./journalctl -b --no-pager > /dev/null real 0m9.548s user 0m9.525s sys 0m0.023s # time ./journalctl -b --no-pager > /dev/null real 0m9.612s user 0m9.582s sys 0m0.030s Post-IteratedCache: # time ./journalctl -b --no-pager > /dev/null real 0m8.449s user 0m8.425s sys 0m0.024s # time ./journalctl -b --no-pager > /dev/null real 0m8.409s user 0m8.382s sys 0m0.027s # time ./journalctl -b --no-pager > /dev/null real 0m8.410s user 0m8.350s sys 0m0.061s ~12.5% improvement, the benefit increases the more log files there are.
* Add SPDX license identifiers to source files under the LGPLZbigniew Jędrzejewski-Szmek2017-11-191-0/+1
| | | | | This follows what the kernel is doing, c.f. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5fd54ace4721fc5ce2bb5aef6318fcf17f421460.
* sd-journal: add API for opening journal files or directories by fdLennart Poettering2016-04-251-0/+3
| | | | | | | Also, expose this via the "journalctl --file=-" syntax for STDIN. This feature remains undocumented though, as it is probably not too useful in real-life as this still requires fds that support mmaping and seeking, i.e. does not work for pipes, for which reading from STDIN is most commonly used.
* tree-wide: remove Emacs lines from all filesDaniel Mack2016-02-101-2/+0
| | | | | This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.
* sd-journal: add an API to enumerate known field names of the journalLennart Poettering2016-02-011-6/+15
| | | | | | This adds two new calls to get the list of all journal fields names currently in use. This is the low-level support to implement the feature requested in #2176 in a more optimized way.
* sd-journal: introduce has_runtime_files and has_persistent_filesJan Synacek2016-02-011-0/+2
| | | | | | | Also introduce sd_journal_has_runtime_files() and sd_journal_has_persistent_files() to the public API. These functions can be used to easily find out if the open journal files are runtime and/or persistent.
* tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easyLennart Poettering2015-11-271-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.
* tree-wide: sort includes in *.hThomas Hindoe Paaboel Andersen2015-11-181-1/+1
| | | | | This is a continuation of the previous include sort patch, which only sorted for .c files.
* journalctl: when we fail to open a journal file, print whyLennart Poettering2015-11-031-1/+1
| | | | | | | | | | | | | When we enumerate journal files and encounter an invalid one, remember which this, and show it to the user. Note the possibly slightly surprising logic here: we store only one path per error code. This means we show all error kinds but not every actual error we encounter. This has the benefit of not requiring us to keep a potentially unbounded list of errors with their sources around, but can still provide a pretty complete overview on the errors we encountered. Fixes #1669.
* util-lib: split our string related calls from util.[ch] into its own file ↵Lennart Poettering2015-10-241-4/+4
| | | | | | | | | | | | | | string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.
* Add missing includes in header filesThomas Hindoe Paaboel Andersen2015-02-121-0/+1
| | | | | This fixes various issues found by globally reordering the include sections of all .c files.
* journal: move definition of LocationType to journal-file.hMichal Schmidt2014-12-181-14/+0
| | | | | In preparation for individual JournalFiles maintaining a location of their own.
* journal: make sd_journal::files a OrderedHashmapMichal Schmidt2014-10-231-1/+1
| | | | | Anything that uses hashmap_next() almost certainly cares about the order and needs to be an OrderedHashmap.
* sd-journal: do not reset sd_j_enumerate_unique position on errorZbigniew Jędrzejewski-Szmek2014-10-091-0/+4
| | | | | | | | | | | | systemctl would call sd_j_enumerate_unique() interleaved with sd_j_next(). But the latter can remove a file if it detects an error in it. In those circumstances sd_j_enumerate_unique would restart with the first file in hashmap. With many corrupted files sd_j_enumerate_unique might iterate over the list multiple times. Avoid this by jumping to the next file in unique list if possible, or setting a flag that tells sd_j_enumerate_unique that it is done otherwise.
* Always prefer our headers to system headersZbigniew Jędrzejewski-Szmek2014-07-311-1/+1
| | | | | | In practice this shouldn't make much difference, but sometimes our headers might be newer, and we want to test them.
* journal: when listing logs of a container make sure we don't accidentally ↵Lennart Poettering2013-12-111-0/+1
| | | | show messages from host too
* util: allow trailing semicolons on define_trivial_cleanup_func linesLennart Poettering2013-10-141-1/+1
| | | | | | | | | | | | | | | | Emacs C indenting really gets confused by these lines if they carry no trailing semicolon, hence let's make this nicer for good old emacs. The other macros which define functions already do this too, so let's copy the scheme here. Also, let's use an uppercase name for the macro. So far our rough rule was that macros that are totally not function-like (like this ones, which define a function) are uppercase. (Well, admittedly it is a rough rule only, for example function and variable decorators are all lower-case SINCE THE CONSTANT YELLING IN THE SOURCES WOULD SUCK, and also they at least got underscore prefixes.) Also, the macros that define functions that we already have are all uppercase, so let's do the same here...
* Introduce udev object cleanup functionsZbigniew Jędrzejewski-Szmek2013-10-131-5/+2
|
* journal: return -ECHILD after a forkZbigniew Jędrzejewski-Szmek2013-07-161-9/+9
| | | | | | | | A few asserts are replaced with 'return -EINVAL'. I think that assert should not be used to check argument in public functions. Fields in struct sd_journal are rearranged to make it less swiss-cheesy.
* journal: add sd_journal_open_filesZbigniew Jędrzejewski-Szmek2013-06-101-0/+1
| | | | | | | | | | | | This allows the caller to explicitly specify which journal files should be opened. The same functionality could be achieved before by creating a directory and playing around with symlinks. It is useful to debug stuff and explore the journal, and has been requested before. Waiting is supported, the journal will notice modifications on the files supplied when opening the journal, but will not add any new files.
* journalctl: fix verbose output when no logs are foundZbigniew Jędrzejewski-Szmek2013-06-091-0/+3
| | | | | | | | | | | | | | | | | $ journalctl -o verbose _EXE=/quiet/binary -f -- Logs begin at Sun 2013-03-17 17:28:22 EDT. -- Failed to get realtime timestamp: Cannot assign requested address JOURNAL_FOREACH_DATA_RETVAL is added, which allows the caller to get the return value from sd_journal_enumerate_data. I think we might want to expose this macro like SD_JOURNAL_FOREACH_DATA, but for now it is in journal-internal.h. There's a change in behaviour for output_*, not only in output_verbose, that errors in sd_j_enumerate_data are not silently ignored anymore. https://bugs.freedesktop.org/show_bug.cgi?id=56459
* Rearrange a few fields to reduce holesZbigniew Jędrzejewski-Szmek2013-05-081-4/+5
|
* journal: add one more level on top with ANDHarald Hoyer2013-04-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using "-p" and "-b" in combination with "-u", the output is not what you would expect. The reason is the sd_journal_add_disjunction() call in add_matches_for_unit() and add_matches_for_user_unit(), which adds two ORs without taking the other conditions to every OR. Adding another level on top with AND and sd_journal_add_conjunction() solves the problem. Output before: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Reboot -- [ 3.216305] lenovo systemd[1]: Starting OpenSSH server daemon... -- Reboot -- [ 3.168666] lenovo systemd[1]: Starting OpenSSH server daemon... [ 3.169639] lenovo systemd[1]: Started OpenSSH server daemon. [36285.635389] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 10.838657] lenovo systemd[1]: Starting OpenSSH server daemon... [ 10.913698] lenovo systemd[1]: Started OpenSSH server daemon. [ 6881.035183] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22. As we see, the output is from _every_ boot and priority 0 is not taken into account. Output after patch: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:58:21 CET. -- Increasing the priority: $ journalctl -o short-monotonic -ab -p 6 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:59:12 CET. -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22.
* macro: rework how we define cleanup macrosLennart Poettering2013-04-161-0/+6
| | | | | | | | There's now a generic _cleanup_ macro with an argument. The macros for specific types are now defined using this macro, and in the header files where they belong. All cleanup handlers are now inline functions.
* journal: add sd_journal_get_timeout() call to public APILennart Poettering2013-04-041-0/+2
| | | | | Let's do the wake-up logic on NFS internally, making things simpler for users.
* journalctl: be smarter about journal error checksZbigniew Jędrzejewski-Szmek2013-03-221-0/+3
| | | | | | | | | | There are many ways in which we can get those checks wrong, so it is better to warn and then error out on a real access failure. The error messages are wrapped to <80 lines, because their primary use is to be displayed in the terminal, and it is easier to read them this way. Reading them in the journal can be a bit trickier, but this is a bug in logs-show.c.
* journal: by default do not decompress dat objects larger than 64KLennart Poettering2012-11-211-0/+2
| | | | | | | | | | This introduces a new data threshold setting for sd_journal objects which controls the maximum size of objects to decompress. This is relieves the library from having to decompress full data objects even if a client program is only interested in the initial part of them. This speeds up "systemd-coredumpctl" drastically when invoked without parameters.
* journal: provide an API that allows client to figure out whether they need ↵Lennart Poettering2012-10-261-0/+2
| | | | | | | | | | | | to recheck the journal manually for changes in regular intervals Network file systems generally do not offer inotify() that would work across the network. We hence cannot rely on inotify() exclusiely in those case. Provide an API to determine these cases, and suggest doing manual regular rechecks. Note that this is not complete yet, as we need to rescan journal dirs on network file systems explicitly to find new/removed files
* journal: add ability to list values a specified field can take in all ↵Lennart Poettering2012-10-181-0/+4
| | | | | | | | | | | | | | | | | entries of the journal The new 'unique' API allows listing all unique field values that a field specified by a field name can take in all entries of the journal. This allows answering queries such as "What units logged to the journal?", "What hosts have logged into the journal?", "Which boot IDs have logged into the journal?". Ultimately this allows implementation of tools similar to lastlog based on journal data. Note that listing these field values will not work for journal files created with older journald, as the field values are not indexed in older files.
* journal: update comments a bitLennart Poettering2012-10-101-0/+7
|
* journal: make sure sd_journal_seek_cursor() seeks to the specified entry if ↵Lennart Poettering2012-10-101-1/+2
| | | | it exists, not one after it
* journal: implement generic sharable mmap caching logicLennart Poettering2012-08-161-1/+1
| | | | | | instead of having one simple per-file cache implement an more comprehensive one that works for multiple files and can actually maintain multiple maps per file and per object type.
* use #pragma once instead of foo*foo #define guardsShawn Landden2012-07-191-3/+1
| | | | | | | | | | | | | | | | | #pragma once has been "un-deprecated" in gcc since 3.3, and is widely supported in other compilers. I've been using and maintaining (rebasing) this patch for a while now, as it annoyed me to see #ifndef fooblahfoo, etc all over the place, almost arrogant about the annoyance of having to define all these names to perform a commen but neccicary functionality, when a completely superior alternative exists. I havn't sent it till now, cause its kindof a style change, and it is bad voodoo to mess with style that has been established by more established editors. So feel free to lambast me as a crazy bafoon. v2 - preserve externally used headers
* journal: when watching directories actually watch the directories asked forLennart Poettering2012-07-191-0/+2
|
* journal: automatically rotate journal files if the data hash table is full > 75%Lennart Poettering2012-07-171-0/+1
| | | | | | | Previously, when the main data hash table grows too full the performance simply started to decrease drastically. Instead, now simply rotate to a new journal file as the hash table gets to full, so that we can start with a new fresh empty hash table.
* journal: avoid re-definition of enums for older gcc versionsKay Sievers2012-07-151-3/+0
| | | | | | | | | On Sun, Jul 15, 2012 at 2:00 PM, Koen Kooi <koen@dominion.thruhere.net> wrote: > | In file included from src/journal/sd-journal.c:37:0: > | src/journal/journal-internal.h:47:3: error: redefinition of typedef 'MatchType' > | src/journal/journal-internal.h:36:24: note: previous declaration of 'MatchType' was here > | src/journal/journal-internal.h:67:3: error: redefinition of typedef 'LocationType' > | src/journal/journal-internal.h:37:27: note: previous declaration of 'LocationType' was here
* journal: beef up journal matches considerablyLennart Poettering2012-07-131-9/+29
| | | | | | we now can take multiple matches, and they will apply as AND if they apply to different fields and OR if they apply to the same fields. Also, terms of this kind can be combined with an overreaching OR.
* journal: rework directory enumeration/watch logicLennart Poettering2012-07-111-10/+21
| | | | | | | | | | There's now sd_journal_new_directory() for watching specific journal directories. This is exposed in journalctl -D. sd_journal_wait() and sd_journal_process() now return whether changes in the journal are invalidating or just appending. We now create inotify kernel watches only when we actually need them
* relicense to LGPLv2.1 (with exceptions)Lennart Poettering2012-04-121-4/+4
| | | | | | | | | | | | | | We finally got the OK from all contributors with non-trivial commits to relicense systemd from GPL2+ to LGPL2.1+. Some udev bits continue to be GPL2+ for now, but we are looking into relicensing them too, to allow free copy/paste of all code within systemd. The bits that used to be MIT continue to be MIT. The big benefit of the relicensing is that closed source code may now link against libsystemd-login.so and friends.
View Lorry Controller Status