journald: make maximum size of stream log lines configurable and bump it to 48K (#6838)

This adds a new setting LineMax= to journald.conf, and sets it by
default to 48K. When we convert stream-based stdout/stderr logging into
record-based log entries, read up to the specified amount of bytes
before forcing a line-break.

This also makes three related changes:

- When a NUL byte is read we'll not recognize this as alternative line
  break, instead of silently dropping everything after it. (see #4863)

- The reason for a line-break is now encoded in the log record, if it
  wasn't a plain newline. Specifically, we distuingish "nul",
  "line-max" and "eof", for line breaks due to NUL byte, due to the
  maximum line length as configured with LineMax= or due to end of
  stream. This data is stored in the new implicit _LINE_BREAK= field.
  It's not synthesized for plain \n line breaks.

- A randomized 128bit ID is assigned to each log stream.

With these three changes in place it's (mostly) possible to reconstruct
the original byte streams from log data, as (most) of the context of
the conversion from the byte stream to log records is saved now. (So,
the only bits we still drop are empty lines. Which might be something to
look into in a future change, and which is outside of the scope of this
work)

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465
See: #4863
Replaces: #4875

1 files changed, 22 insertions, 0 deletions

diff --git a/man/systemd.journal-fields.xml b/man/systemd.journal-fields.xml
index b82c1300ca..e488affe3e 100644
--- a/man/systemd.journal-fields.xml
+++ b/man/systemd.journal-fields.xml
@@ -333,6 +333,28 @@
           </variablelist>
         </listitem>
       </varlistentry>
+      <varlistentry>
+        <term><varname>_STREAM_ID=</varname></term>
+        <listitem>
+          <para>Only applies to <literal>_TRANSPORT=stream</literal> records: specifies a randomized 128bit ID assigned
+          to the stream connection when it was first created. This ID is useful to reconstruct individual log streams
+          from the log records: all log records carrying the same stream ID originate from the same stream.</para>
+        </listitem>
+      </varlistentry>
+      <varlistentry>
+        <term><varname>_LINE_BREAK=</varname></term>
+        <listitem>
+          <para>Only applies to <literal>_TRANSPORT=stream</literal> records: indicates that the log message in the
+          standard output/error stream was not terminated with a normal newline character (<literal>\n</literal>,
+          i.e. ASCII 10). Specifically, when set this field is one of <option>nul</option> (in case the line was
+          terminated by a NUL byte), <option>line-max</option> (in case the maximum log line length was reached, as
+          configured with <varname>LineMax=</varname> in
+          <citerefentry><refentrytitle>journald.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry>) or
+          <option>eof</option> (if this was the last log record of a stream and the stream ended without a final
+          newline character). Note that this record is not generated when a normal newline character was used for
+          marking the log line end.</para>
+        </listitem>
+      </varlistentry>
     </variablelist>
   </refsect1>