diff options
| author | Sebastian Andrzej Siewior <bigeasy@linutronix.de> | 2022-02-10 21:55:22 +0100 |
|---|---|---|
| committer | Sebastian Andrzej Siewior <bigeasy@linutronix.de> | 2022-02-10 21:55:22 +0100 |
| commit | b8150f7cb443de70d32f91a7a585fc2dd72eb892 (patch) | |
| tree | 123d1daa8b11615b040b2e4e413dfba394b27fc8 | |
| parent | 46792e5f5fed1417678610601daefa3666fe5f24 (diff) | |
| download | linux-rt-b8150f7cb443de70d32f91a7a585fc2dd72eb892.tar.gz | |
[ANNOUNCE] v5.17-rc3-rt6v5.17-rc3-rt6-patches
Dear RT folks!
I'm pleased to announce the v5.17-rc3-rt6 patch set.
Changes since v5.17-rc3-rt5:
- Update John's printk series. It supports now direct printing from
irqwork.
- Correct atomic access to a variable in prink. Patch by John Ogness.
- Add a warning if there is a ksoftirqd wakeup from idle.
- Jason A. Donenfeld patches against the random subsystem were updated
to v4. There is an additional RT related change on top.
- The known issue
netconsole triggers WARN.
has been removed from the list since it also triggers with
CONFIG_PREEMPT and v5.17-rc3 (without the PREEMPT_RT patch).
Known issues
- Valentin Schneider reported a few splats on ARM64, see
https://lkml.kernel.org/r/20210810134127.1394269-1-valentin.schneider@arm.com
The delta patch against v5.17-rc3-rt5 is appended below and can be found here:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.17/incr/patch-5.17-rc3-rt5-rt6.patch.xz
You can get this release via the git tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git v5.17-rc3-rt6
The RT patch against v5.17-rc3 can be found here:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.17/older/patch-5.17-rc3-rt6.patch.xz
The split quilt queue is available at:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.17/older/patches-5.17-rc3-rt6.tar.xz
Sebastian
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
31 files changed, 527 insertions, 191 deletions
diff --git a/patches/0001-printk-rename-cpulock-functions.patch b/patches/0001-printk-rename-cpulock-functions.patch index 3b537e3c3c10..b65342bd5fe3 100644 --- a/patches/0001-printk-rename-cpulock-functions.patch +++ b/patches/0001-printk-rename-cpulock-functions.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:15 +0106 +Date: Mon, 7 Feb 2022 15:53:38 +0106 Subject: [PATCH 01/16] printk: rename cpulock functions Since the printk cpulock is CPU-reentrant and since it is used diff --git a/patches/0001-random-use-computational-hash-for-entropy-extraction.patch b/patches/0001-random-use-computational-hash-for-entropy-extraction.patch index 067494403142..80774c1b72e8 100644 --- a/patches/0001-random-use-computational-hash-for-entropy-extraction.patch +++ b/patches/0001-random-use-computational-hash-for-entropy-extraction.patch @@ -1,6 +1,6 @@ From: "Jason A. Donenfeld" <Jason@zx2c4.com> Date: Sun, 16 Jan 2022 14:23:10 +0100 -Subject: [PATCH 1/2] random: use computational hash for entropy extraction +Subject: [PATCH 1/3] random: use computational hash for entropy extraction The current 4096-bit LFSR used for entropy collection had a few desirable attributes for the context in which it was created. For @@ -344,7 +344,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } static void __mix_pool_bytes(const void *in, int nbytes) -@@ -954,15 +782,14 @@ static int crng_slow_load(const u8 *cp, +@@ -953,15 +781,14 @@ static int crng_slow_load(const u8 *cp, static void crng_reseed(struct crng_state *crng, bool use_input_pool) { unsigned long flags; diff --git a/patches/0002-printk-cpu-sync-always-disable-interrupts.patch b/patches/0002-printk-cpu-sync-always-disable-interrupts.patch index 87a9766a0e5f..5a43c7a7ff8c 100644 --- a/patches/0002-printk-cpu-sync-always-disable-interrupts.patch +++ b/patches/0002-printk-cpu-sync-always-disable-interrupts.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:15 +0106 +Date: Mon, 7 Feb 2022 15:53:41 +0106 Subject: [PATCH 02/16] printk: cpu sync always disable interrupts The CPU sync functions are a NOP for !CONFIG_SMP. But for diff --git a/patches/0002-random-remove-batched-entropy-locking.patch b/patches/0002-random-remove-batched-entropy-locking.patch new file mode 100644 index 000000000000..a8a7f3bec631 --- /dev/null +++ b/patches/0002-random-remove-batched-entropy-locking.patch @@ -0,0 +1,150 @@ +From: "Jason A. Donenfeld" <Jason@zx2c4.com> +Date: Wed, 9 Feb 2022 13:56:43 +0100 +Subject: [PATCH 2/3] random: remove batched entropy locking +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Rather than use spinlocks to protect batched entropy, we can instead +disable interrupts locally, since we're dealing with per-cpu data, and +manage resets with a basic generation counter. At the same time, we +can't quite do this on PREEMPT_RT, where we still want spinlocks-as- +mutexes semantics. So we use a local_lock_t, which provides the right +behavior for each. Because this is a per-cpu lock, that generation +counter is still doing the necessary CPU-to-CPU communication. + +This should improve performance a bit. It will also fix the linked splat +that Jonathan received with a PROVE_RAW_LOCK_NESTING=y. + +Suggested-by: Andy Lutomirski <luto@kernel.org> +Reported-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net> +Tested-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net> +Link: https://lore.kernel.org/lkml/YfMa0QgsjCVdRAvJ@latitude/ +Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +Cc: Sultan Alsawaf <sultan@kerneltoast.com> +Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> +Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +--- + drivers/char/random.c | 55 +++++++++++++++++++++++++------------------------- + 1 file changed, 28 insertions(+), 27 deletions(-) + +--- a/drivers/char/random.c ++++ b/drivers/char/random.c +@@ -1876,13 +1876,16 @@ static int __init random_sysctls_init(vo + device_initcall(random_sysctls_init); + #endif /* CONFIG_SYSCTL */ + ++static atomic_t batch_generation = ATOMIC_INIT(0); ++ + struct batched_entropy { + union { + u64 entropy_u64[CHACHA_BLOCK_SIZE / sizeof(u64)]; + u32 entropy_u32[CHACHA_BLOCK_SIZE / sizeof(u32)]; + }; ++ local_lock_t lock; + unsigned int position; +- spinlock_t batch_lock; ++ int generation; + }; + + /* +@@ -1894,7 +1897,7 @@ struct batched_entropy { + * point prior. + */ + static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = { +- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock), ++ .lock = INIT_LOCAL_LOCK(batched_entropy_u64.lock) + }; + + u64 get_random_u64(void) +@@ -1903,67 +1906,65 @@ u64 get_random_u64(void) + unsigned long flags; + struct batched_entropy *batch; + static void *previous; ++ int next_gen; + + warn_unseeded_randomness(&previous); + ++ local_lock_irqsave(&batched_entropy_u64.lock, flags); + batch = raw_cpu_ptr(&batched_entropy_u64); +- spin_lock_irqsave(&batch->batch_lock, flags); +- if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) { ++ ++ next_gen = atomic_read(&batch_generation); ++ if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0 || ++ next_gen != batch->generation) { + extract_crng((u8 *)batch->entropy_u64); + batch->position = 0; ++ batch->generation = next_gen; + } ++ + ret = batch->entropy_u64[batch->position++]; +- spin_unlock_irqrestore(&batch->batch_lock, flags); ++ local_unlock_irqrestore(&batched_entropy_u64.lock, flags); + return ret; + } + EXPORT_SYMBOL(get_random_u64); + + static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = { +- .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock), ++ .lock = INIT_LOCAL_LOCK(batched_entropy_u32.lock) + }; ++ + u32 get_random_u32(void) + { + u32 ret; + unsigned long flags; + struct batched_entropy *batch; + static void *previous; ++ int next_gen; + + warn_unseeded_randomness(&previous); + ++ local_lock_irqsave(&batched_entropy_u32.lock, flags); + batch = raw_cpu_ptr(&batched_entropy_u32); +- spin_lock_irqsave(&batch->batch_lock, flags); +- if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) { ++ ++ next_gen = atomic_read(&batch_generation); ++ if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0 || ++ next_gen != batch->generation) { + extract_crng((u8 *)batch->entropy_u32); + batch->position = 0; ++ batch->generation = next_gen; + } ++ + ret = batch->entropy_u32[batch->position++]; +- spin_unlock_irqrestore(&batch->batch_lock, flags); ++ local_unlock_irqrestore(&batched_entropy_u32.lock, flags); + return ret; + } + EXPORT_SYMBOL(get_random_u32); + + /* It's important to invalidate all potential batched entropy that might + * be stored before the crng is initialized, which we can do lazily by +- * simply resetting the counter to zero so that it's re-extracted on the +- * next usage. */ ++ * bumping the generation counter. ++ */ + static void invalidate_batched_entropy(void) + { +- int cpu; +- unsigned long flags; +- +- for_each_possible_cpu(cpu) { +- struct batched_entropy *batched_entropy; +- +- batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu); +- spin_lock_irqsave(&batched_entropy->batch_lock, flags); +- batched_entropy->position = 0; +- spin_unlock(&batched_entropy->batch_lock); +- +- batched_entropy = per_cpu_ptr(&batched_entropy_u64, cpu); +- spin_lock(&batched_entropy->batch_lock); +- batched_entropy->position = 0; +- spin_unlock_irqrestore(&batched_entropy->batch_lock, flags); +- } ++ atomic_inc(&batch_generation); + } + + /** diff --git a/patches/0003-printk-use-percpu-flag-instead-of-cpu_online.patch b/patches/0003-printk-use-percpu-flag-instead-of-cpu_online.patch index 5d63c9856ab8..b0aa5173df4d 100644 --- a/patches/0003-printk-use-percpu-flag-instead-of-cpu_online.patch +++ b/patches/0003-printk-use-percpu-flag-instead-of-cpu_online.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:15 +0106 +Date: Mon, 7 Feb 2022 15:53:45 +0106 Subject: [PATCH 03/16] printk: use percpu flag instead of cpu_online() The CON_ANYTIME console flag is used to label consoles that will diff --git a/patches/0002-random-do-not-take-spinlocks-in-irq-handler.patch b/patches/0003-random-defer-fast-pool-mixing-to-worker.patch index a1f1550ed6dc..b8197e898491 100644 --- a/patches/0002-random-do-not-take-spinlocks-in-irq-handler.patch +++ b/patches/0003-random-defer-fast-pool-mixing-to-worker.patch @@ -1,23 +1,21 @@ From: "Jason A. Donenfeld" <Jason@zx2c4.com> -Date: Fri, 4 Feb 2022 16:31:49 +0100 -Subject: [PATCH 2/2] random: do not take spinlocks in irq handler +Date: Wed, 9 Feb 2022 13:56:44 +0100 +Subject: [PATCH 3/3] random: defer fast pool mixing to worker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit -On PREEMPT_RT, it's problematic to take spinlocks from hard IRQ +On PREEMPT_RT, it's problematic to take spinlocks from hard irq handlers. We can fix this by deferring to a work queue the dumping of the fast pool into the input pool. -We accomplish this by making `u8 count` an `atomic_t count`, with the -following rules: +We accomplish this with some careful rules on fast_pool->count: - When it's incremented to >= 64, we schedule the work. - If the top bit is set, we never schedule the work, even if >= 64. - The worker is responsible for setting it back to 0 when it's done. - - If we need to retry the worker later, we clear the top bit. -In the worst case, an IRQ handler is mixing a new IRQ into the pool at +In the worst case, an irq handler is mixing a new irq into the pool at the same time as the worker is dumping it into the input pool. In this case, we only ever set the count back to 0 _after_ we're done, so that subsequent cycles will require a full 64 to dump it in again. In other @@ -33,11 +31,10 @@ Cc: Sultan Alsawaf <sultan@kerneltoast.com> Cc: Jonathan Neuschäfer <j.neuschaefer@gmx.net> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> -Link: https://lkml.kernel.org/r/20220204153149.51428-1-Jason@zx2c4.com --- - drivers/char/random.c | 67 ++++++++++++++++++++++-------------------- - include/trace/events/random.h | 6 --- - 2 files changed, 36 insertions(+), 37 deletions(-) + drivers/char/random.c | 54 ++++++++++++++++++++++++++---------------- + include/trace/events/random.h | 6 ---- + 2 files changed, 34 insertions(+), 26 deletions(-) --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -62,7 +59,7 @@ Link: https://lkml.kernel.org/r/20220204153149.51428-1-Jason@zx2c4.com + struct work_struct mix; unsigned long last; + u32 pool[4]; -+ atomic_t count; ++ unsigned int count; u16 reg_idx; - u8 count; }; @@ -78,32 +75,29 @@ Link: https://lkml.kernel.org/r/20220204153149.51428-1-Jason@zx2c4.com } static void process_random_ready_list(void) -@@ -1047,12 +1042,37 @@ static u32 get_reg(struct fast_pool *f, +@@ -1047,12 +1042,34 @@ static u32 get_reg(struct fast_pool *f, return *ptr; } +static void mix_interrupt_randomness(struct work_struct *work) +{ + struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix); ++ u8 pool[sizeof(fast_pool->pool)]; + -+ fast_pool->last = jiffies; -+ -+ /* Since this is the result of a trip through the scheduler, xor in ++ /* ++ * Since this is the result of a trip through the scheduler, xor in + * a cycle counter. It can't hurt, and might help. + */ + fast_pool->pool[3] ^= random_get_entropy(); ++ /* Copy the pool to the stack so that the mixer always has a consistent view. */ ++ memcpy(pool, fast_pool->pool, sizeof(pool)); ++ /* We take care to zero out the count only after we're done reading the pool. */ ++ WRITE_ONCE(fast_pool->count, 0); ++ fast_pool->last = jiffies; + -+ if (unlikely(crng_init == 0)) { -+ if (crng_fast_load((u8 *)&fast_pool->pool, sizeof(fast_pool->pool)) > 0) -+ atomic_set(&fast_pool->count, 0); -+ else -+ atomic_and(~FAST_POOL_MIX_INFLIGHT, &fast_pool->count); -+ return; -+ } -+ -+ mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool)); -+ atomic_set(&fast_pool->count, 0); ++ mix_pool_bytes(pool, sizeof(pool)); + credit_entropy_bits(1); ++ memzero_explicit(pool, sizeof(pool)); +} + void add_interrupt_randomness(int irq) @@ -116,32 +110,30 @@ Link: https://lkml.kernel.org/r/20220204153149.51428-1-Jason@zx2c4.com u32 c_high, j_high; u64 ip; -@@ -1070,29 +1090,14 @@ void add_interrupt_randomness(int irq) +@@ -1069,9 +1086,10 @@ void add_interrupt_randomness(int irq) + fast_mix(fast_pool); add_interrupt_bench(cycles); ++ new_count = ++fast_pool->count; -- if (unlikely(crng_init == 0)) { + if (unlikely(crng_init == 0)) { - if ((fast_pool->count >= 64) && -- crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) { -- fast_pool->count = 0; -- fast_pool->last = now; -- } -- return; -+ new_count = (unsigned int)atomic_inc_return(&fast_pool->count); -+ if (new_count >= 64 && new_count < FAST_POOL_MIX_INFLIGHT && -+ (time_after(now, fast_pool->last + HZ) || unlikely(crng_init == 0))) { -+ if (unlikely(!fast_pool->mix.func)) -+ INIT_WORK(&fast_pool->mix, mix_interrupt_randomness); -+ atomic_or(FAST_POOL_MIX_INFLIGHT, &fast_pool->count); -+ schedule_work(&fast_pool->mix); ++ if (new_count >= 64 && + crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) { + fast_pool->count = 0; + fast_pool->last = now; +@@ -1079,20 +1097,16 @@ void add_interrupt_randomness(int irq) + return; } -- + - if ((fast_pool->count < 64) && !time_after(now, fast_pool->last + HZ)) -- return; -- ++ if (new_count & FAST_POOL_MIX_INFLIGHT) + return; + - if (!spin_trylock(&input_pool.lock)) -- return; -- ++ if (new_count < 64 && !time_after(now, fast_pool->last + HZ)) + return; + - fast_pool->last = now; - __mix_pool_bytes(&fast_pool->pool, sizeof(fast_pool->pool)); - spin_unlock(&input_pool.lock); @@ -150,6 +142,10 @@ Link: https://lkml.kernel.org/r/20220204153149.51428-1-Jason@zx2c4.com - - /* award one bit for the contents of the fast pool */ - credit_entropy_bits(1); ++ if (unlikely(!fast_pool->mix.func)) ++ INIT_WORK(&fast_pool->mix, mix_interrupt_randomness); ++ fast_pool->count |= FAST_POOL_MIX_INFLIGHT; ++ queue_work_on(raw_smp_processor_id(), system_highpri_wq, &fast_pool->mix); } EXPORT_SYMBOL_GPL(add_interrupt_randomness); diff --git a/patches/0004-printk-get-caller_id-timestamp-after-migration-disab.patch b/patches/0004-printk-get-caller_id-timestamp-after-migration-disab.patch index cf30e431c911..2f24c819cc68 100644 --- a/patches/0004-printk-get-caller_id-timestamp-after-migration-disab.patch +++ b/patches/0004-printk-get-caller_id-timestamp-after-migration-disab.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:15 +0106 +Date: Mon, 7 Feb 2022 15:53:48 +0106 Subject: [PATCH 04/16] printk: get caller_id/timestamp after migration disable Currently the local CPU timestamp and caller_id for the record are diff --git a/patches/0005-printk-call-boot_delay_msec-in-printk_delay.patch b/patches/0005-printk-call-boot_delay_msec-in-printk_delay.patch index a7c1e803a729..0e58681b35ff 100644 --- a/patches/0005-printk-call-boot_delay_msec-in-printk_delay.patch +++ b/patches/0005-printk-call-boot_delay_msec-in-printk_delay.patch @@ -1,10 +1,11 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:16 +0106 +Date: Mon, 7 Feb 2022 15:53:51 +0106 Subject: [PATCH 05/16] printk: call boot_delay_msec() in printk_delay() boot_delay_msec() is always called immediately before printk_delay() so just call it from within printk_delay(). +Signed-off-by: John Ogness <john.ogness@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- kernel/printk/printk.c | 7 ++++--- diff --git a/patches/0006-printk-refactor-and-rework-printing-logic.patch b/patches/0006-printk-refactor-and-rework-printing-logic.patch index 6a5411038a91..2204201ea6b2 100644 --- a/patches/0006-printk-refactor-and-rework-printing-logic.patch +++ b/patches/0006-printk-refactor-and-rework-printing-logic.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:16 +0106 +Date: Mon, 7 Feb 2022 15:54:14 +0106 Subject: [PATCH 06/16] printk: refactor and rework printing logic Refactor/rework printing logic in order to prepare for moving to threaded diff --git a/patches/0007-printk-move-buffer-definitions-into-console_emit_nex.patch b/patches/0007-printk-move-buffer-definitions-into-console_emit_nex.patch index 9851f4c70872..d083e0c61dd7 100644 --- a/patches/0007-printk-move-buffer-definitions-into-console_emit_nex.patch +++ b/patches/0007-printk-move-buffer-definitions-into-console_emit_nex.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:16 +0106 +Date: Mon, 7 Feb 2022 15:54:22 +0106 Subject: [PATCH 07/16] printk: move buffer definitions into console_emit_next_record() caller @@ -26,8 +26,8 @@ from the code that does the various types of string printing. Signed-off-by: John Ogness <john.ogness@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- - kernel/printk/printk.c | 58 ++++++++++++++++++++++++++++++++++--------------- - 1 file changed, 41 insertions(+), 17 deletions(-) + kernel/printk/printk.c | 60 +++++++++++++++++++++++++++++++++++-------------- + 1 file changed, 43 insertions(+), 17 deletions(-) --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -73,17 +73,19 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> #define printk_time false #define prb_read_valid(rb, seq, r) false -@@ -2265,7 +2269,8 @@ static ssize_t msg_print_ext_body(char * +@@ -2265,7 +2269,10 @@ static ssize_t msg_print_ext_body(char * struct dev_printk_info *dev_info) { return 0; } static void console_lock_spinning_enable(void) { } static int console_lock_spinning_disable_and_check(void) { return 0; } -static void call_console_driver(struct console *con, const char *text, size_t len) {} +static void call_console_driver(struct console *con, const char *text, size_t len, -+ char *dropped_text) {} ++ char *dropped_text) ++{ ++} static bool suppress_message_printing(int level) { return false; } static void printk_delay(int level) {} -@@ -2561,6 +2566,14 @@ static void __console_unlock(void) +@@ -2561,6 +2568,14 @@ static void __console_unlock(void) * Print one record for the given console. The record printed is whatever * record is the next available record for the given console. * @@ -93,12 +95,12 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + * CONSOLE_EXT_LOG_MAX. Otherwise @ext_text must be NULL. + * + * If dropped messages should be printed, @dropped_text is a buffer of size -+ * DROPPED_TEXT_MAX. Otherise @dropped_text must be NULL. ++ * DROPPED_TEXT_MAX. Otherwise @dropped_text must be NULL. + * * Requires the console_lock. * * Returns false if the given console has no next record to print, otherwise -@@ -2570,17 +2583,16 @@ static void __console_unlock(void) +@@ -2570,17 +2585,16 @@ static void __console_unlock(void) * console_lock, in which case the caller is no longer holding the * console_lock. Otherwise it is set to false. */ @@ -119,7 +121,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> *handover = false; -@@ -2598,13 +2610,13 @@ static bool console_emit_next_record(str +@@ -2598,13 +2612,13 @@ static bool console_emit_next_record(str goto skip; } @@ -138,7 +140,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> len = record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, printk_time); } -@@ -2622,7 +2634,7 @@ static bool console_emit_next_record(str +@@ -2622,7 +2636,7 @@ static bool console_emit_next_record(str console_lock_spinning_enable(); stop_critical_timings(); /* don't trace print latency */ @@ -147,7 +149,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> start_critical_timings(); con->seq++; -@@ -2651,6 +2663,9 @@ static bool console_emit_next_record(str +@@ -2651,6 +2665,9 @@ static bool console_emit_next_record(str */ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handover) { @@ -157,7 +159,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> bool any_usable = false; struct console *con; bool any_progress; -@@ -2668,7 +2683,16 @@ static bool console_flush_all(bool do_co +@@ -2668,7 +2685,16 @@ static bool console_flush_all(bool do_co continue; any_usable = true; diff --git a/patches/0008-printk-add-pr_flush.patch b/patches/0008-printk-add-pr_flush.patch index 28c790211bc3..ffdc30eada19 100644 --- a/patches/0008-printk-add-pr_flush.patch +++ b/patches/0008-printk-add-pr_flush.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:16 +0106 +Date: Mon, 7 Feb 2022 15:54:26 +0106 Subject: [PATCH 08/16] printk: add pr_flush() Provide a might-sleep function to allow waiting for threaded console @@ -41,7 +41,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> return 0; --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c -@@ -2448,6 +2448,7 @@ void suspend_console(void) +@@ -2450,6 +2450,7 @@ void suspend_console(void) if (!console_suspend_enabled) return; pr_info("Suspending console(s) (use no_console_suspend to debug)\n"); @@ -49,7 +49,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_lock(); console_suspended = 1; up_console_sem(); -@@ -2460,6 +2461,7 @@ void resume_console(void) +@@ -2462,6 +2463,7 @@ void resume_console(void) down_console_sem(); console_suspended = 0; console_unlock(); @@ -57,7 +57,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } /** -@@ -2801,8 +2803,10 @@ void console_unblank(void) +@@ -2803,8 +2805,10 @@ void console_unblank(void) if (oops_in_progress) { if (down_trylock_console_sem() != 0) return; @@ -69,7 +69,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_locked = 1; console_may_schedule = 0; -@@ -2868,6 +2872,7 @@ struct tty_driver *console_device(int *i +@@ -2870,6 +2874,7 @@ struct tty_driver *console_device(int *i */ void console_stop(struct console *console) { @@ -77,7 +77,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_lock(); console->flags &= ~CON_ENABLED; console_unlock(); -@@ -2879,6 +2884,7 @@ void console_start(struct console *conso +@@ -2881,6 +2886,7 @@ void console_start(struct console *conso console_lock(); console->flags |= CON_ENABLED; console_unlock(); @@ -85,7 +85,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL(console_start); -@@ -3249,6 +3255,71 @@ static int __init printk_late_init(void) +@@ -3251,6 +3257,71 @@ static int __init printk_late_init(void) late_initcall(printk_late_init); #if defined CONFIG_PRINTK diff --git a/patches/0009-printk-add-functions-to-allow-direct-printing.patch b/patches/0009-printk-add-functions-to-allow-direct-printing.patch index 972888c7915d..dddee01b1439 100644 --- a/patches/0009-printk-add-functions-to-allow-direct-printing.patch +++ b/patches/0009-printk-add-functions-to-allow-direct-printing.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:16 +0106 +Date: Mon, 7 Feb 2022 15:54:30 +0106 Subject: [PATCH 09/16] printk: add functions to allow direct printing Once kthread printing is introduced, console printing will no longer @@ -26,14 +26,14 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> drivers/tty/sysrq.c | 2 ++ include/linux/printk.h | 11 +++++++++++ kernel/hung_task.c | 11 ++++++++++- - kernel/printk/printk.c | 25 +++++++++++++++++++++++++ + kernel/printk/printk.c | 39 ++++++++++++++++++++++++++++++++++++++- kernel/rcu/tree_stall.h | 2 ++ kernel/reboot.c | 14 +++++++++++++- kernel/watchdog.c | 4 ++++ kernel/watchdog_hld.c | 4 ++++ lib/dump_stack.c | 2 ++ lib/nmi_backtrace.c | 2 ++ - 10 files changed, 75 insertions(+), 2 deletions(-) + 10 files changed, 88 insertions(+), 3 deletions(-) --- a/drivers/tty/sysrq.c +++ b/drivers/tty/sysrq.c @@ -125,7 +125,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> + +/** + * printk_direct_enter - cause console printing to occur in the context of -+ * printk() callers ++ * printk() callers + * + * This globally effects all printk() callers. + * @@ -149,6 +149,48 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> DECLARE_WAIT_QUEUE_HEAD(log_wait); /* All 3 protected by @syslog_lock. */ /* the next printk record to read by syslog(READ) or /proc/kmsg */ +@@ -3327,6 +3352,7 @@ EXPORT_SYMBOL(pr_flush); + */ + #define PRINTK_PENDING_WAKEUP 0x01 + #define PRINTK_PENDING_OUTPUT 0x02 ++#define PRINTK_DIRECT_OUTPUT 0x04 + + static DEFINE_PER_CPU(int, printk_pending); + +@@ -3335,9 +3361,15 @@ static void wake_up_klogd_work_func(stru + int pending = __this_cpu_xchg(printk_pending, 0); + + if (pending & PRINTK_PENDING_OUTPUT) { ++ if (pending & PRINTK_DIRECT_OUTPUT) ++ printk_direct_enter(); ++ + /* If trylock fails, someone else is doing the printing */ + if (console_trylock()) + console_unlock(); ++ ++ if (pending & PRINTK_DIRECT_OUTPUT) ++ printk_direct_exit(); + } + + if (pending & PRINTK_PENDING_WAKEUP) +@@ -3362,11 +3394,16 @@ void wake_up_klogd(void) + + void defer_console_output(void) + { ++ int val = PRINTK_PENDING_OUTPUT; ++ + if (!printk_percpu_data_ready()) + return; + ++ if (atomic_read(&printk_direct)) ++ val |= PRINTK_DIRECT_OUTPUT; ++ + preempt_disable(); +- this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT); ++ this_cpu_or(printk_pending, val); + irq_work_queue(this_cpu_ptr(&wake_up_klogd_work)); + preempt_enable(); + } --- a/kernel/rcu/tree_stall.h +++ b/kernel/rcu/tree_stall.h @@ -587,6 +587,7 @@ static void print_cpu_stall(unsigned lon diff --git a/patches/0010-printk-add-kthread-console-printers.patch b/patches/0010-printk-add-kthread-console-printers.patch index 4b429e670f8d..ad8b57cad2df 100644 --- a/patches/0010-printk-add-kthread-console-printers.patch +++ b/patches/0010-printk-add-kthread-console-printers.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:17 +0106 +Date: Mon, 7 Feb 2022 15:54:42 +0106 Subject: [PATCH 10/16] printk: add kthread console printers Create a kthread for each console to perform console printing. During @@ -80,8 +80,8 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> #else /* CONFIG_PRINTK */ #define CONSOLE_LOG_MAX 0 -@@ -2298,6 +2315,8 @@ static void call_console_driver(struct c - char *dropped_text) {} +@@ -2300,6 +2317,8 @@ static void call_console_driver(struct c + } static bool suppress_message_printing(int level) { return false; } static void printk_delay(int level) {} +static void start_printk_kthread(struct console *con) {} @@ -89,7 +89,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> #endif /* CONFIG_PRINTK */ -@@ -2486,6 +2505,10 @@ void resume_console(void) +@@ -2488,6 +2507,10 @@ void resume_console(void) down_console_sem(); console_suspended = 0; console_unlock(); @@ -100,7 +100,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> pr_flush(1000, true); } -@@ -2701,6 +2724,10 @@ static bool console_flush_all(bool do_co +@@ -2703,6 +2726,10 @@ static bool console_flush_all(bool do_co *handover = false; do { @@ -111,7 +111,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> any_progress = false; for_each_console(con) { -@@ -2909,6 +2936,10 @@ void console_start(struct console *conso +@@ -2911,6 +2938,10 @@ void console_start(struct console *conso console_lock(); console->flags |= CON_ENABLED; console_unlock(); @@ -122,7 +122,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> pr_flush(1000, true); } EXPORT_SYMBOL(console_start); -@@ -3113,6 +3144,8 @@ void register_console(struct console *ne +@@ -3115,6 +3146,8 @@ void register_console(struct console *ne /* Begin with next message. */ newcon->seq = prb_next_seq(prb); } @@ -131,7 +131,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> console_unlock(); console_sysfs_notify(); -@@ -3169,6 +3202,11 @@ int unregister_console(struct console *c +@@ -3171,6 +3204,11 @@ int unregister_console(struct console *c } } @@ -143,7 +143,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> if (res) goto out_disable_unlock; -@@ -3275,6 +3313,13 @@ static int __init printk_late_init(void) +@@ -3277,6 +3315,13 @@ static int __init printk_late_init(void) console_cpu_notify, NULL); WARN_ON(ret < 0); printk_sysctl_init(); @@ -157,7 +157,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> return 0; } late_initcall(printk_late_init); -@@ -3345,6 +3390,116 @@ bool pr_flush(int timeout_ms, bool reset +@@ -3347,6 +3392,116 @@ bool pr_flush(int timeout_ms, bool reset } EXPORT_SYMBOL(pr_flush); @@ -274,7 +274,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> /* * Delayed printk version, for scheduler-internal messages: */ -@@ -3364,7 +3519,7 @@ static void wake_up_klogd_work_func(stru +@@ -3373,7 +3528,7 @@ static void wake_up_klogd_work_func(stru } if (pending & PRINTK_PENDING_WAKEUP) diff --git a/patches/0011-printk-reimplement-console_lock-for-proper-kthread-s.patch b/patches/0011-printk-reimplement-console_lock-for-proper-kthread-s.patch index 350dcad42742..4b57e7153c5d 100644 --- a/patches/0011-printk-reimplement-console_lock-for-proper-kthread-s.patch +++ b/patches/0011-printk-reimplement-console_lock-for-proper-kthread-s.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:17 +0106 +Date: Mon, 7 Feb 2022 15:54:47 +0106 Subject: [PATCH 11/16] printk: reimplement console_lock for proper kthread support @@ -19,7 +19,7 @@ situation is needed. The kthread printers do not need to synchronize against each other, but they do need to synchronize against console_lock() callers. To -provide this synchonization, introduce a per-console mutex. The +provide this synchronization, introduce a per-console mutex. The mutex is taken by the kthread printer during printing and is also taken by console_lock() callers. Since mutexes have owners, when calling console_lock(), the scheduler is able to schedule any @@ -43,28 +43,36 @@ kthread printers is as follows (pseudo code): console_lock() { - down(&console_sem); - for_each_console(con) { - mutex_lock(&con->lock); - con->flags |= CON_PAUSED; - mutex_unlock(&con->lock); - } + down(&console_sem); + for_each_console(con) { + mutex_lock(&con->lock); + con->flags |= CON_PAUSED; + mutex_unlock(&con->lock); + } + /* console lock acquired */ } console_trylock() { - assert(down_trylock(&console_sem)); - assert(atomic_cmpxchg(&console_lock_count, 0, -1)); + if (down_trylock(&console_sem) == 0) { + if (atomic_cmpxchg(&console_lock_count, 0, -1) == 0) { + /* console lock acquired */ + } + } } -kthread_printer() +threaded_printer() { - mutex_lock(&con->lock); - assert(con->flags & CON_PAUSED); - assert(atomic_inc_unless_negative(&console_lock_count)); - con->write(); - atomic_dec(&console_lock_count); - mutex_unlock(&con->lock); + mutex_lock(&con->lock); + if (!(con->flags & CON_PAUSED)) { + if (atomic_inc_unless_negative(&console_lock_count)) { + /* console locking now blocked */ + + con->write(); + atomic_dec(&console_lock_count); + } + } + mutex_unlock(&con->lock); } Also note that the console owner and waiter logic now only applies @@ -196,7 +204,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> * This is used for debugging the mess that is the VT code by * keeping track if we have the console semaphore held. It's * definitely not the perfect debug tool (we don't know if _WE_ -@@ -2505,10 +2556,6 @@ void resume_console(void) +@@ -2507,10 +2558,6 @@ void resume_console(void) down_console_sem(); console_suspended = 0; console_unlock(); @@ -207,7 +215,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> pr_flush(1000, true); } -@@ -2546,6 +2593,7 @@ void console_lock(void) +@@ -2548,6 +2595,7 @@ void console_lock(void) down_console_sem(); if (console_suspended) return; @@ -215,7 +223,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_locked = 1; console_may_schedule = 1; } -@@ -2567,15 +2615,45 @@ int console_trylock(void) +@@ -2569,15 +2617,45 @@ int console_trylock(void) up_console_sem(); return 0; } @@ -262,7 +270,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL(is_console_locked); -@@ -2609,6 +2687,19 @@ static inline bool console_is_usable(str +@@ -2611,6 +2689,19 @@ static inline bool console_is_usable(str static void __console_unlock(void) { console_locked = 0; @@ -282,7 +290,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> up_console_sem(); } -@@ -2631,7 +2722,8 @@ static void __console_unlock(void) +@@ -2633,7 +2724,8 @@ static void __console_unlock(void) * * @handover will be set to true if a printk waiter has taken over the * console_lock, in which case the caller is no longer holding the @@ -292,7 +300,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> */ static bool console_emit_next_record(struct console *con, char *text, char *ext_text, char *dropped_text, bool *handover) -@@ -2639,12 +2731,14 @@ static bool console_emit_next_record(str +@@ -2641,12 +2733,14 @@ static bool console_emit_next_record(str struct printk_info info; struct printk_record r; unsigned long flags; @@ -308,7 +316,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (!prb_read_valid(prb, con->seq, &r)) return false; -@@ -2670,18 +2764,23 @@ static bool console_emit_next_record(str +@@ -2672,18 +2766,23 @@ static bool console_emit_next_record(str len = record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, printk_time); } @@ -344,7 +352,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> stop_critical_timings(); /* don't trace print latency */ call_console_driver(con, write_text, len, dropped_text); -@@ -2689,8 +2788,10 @@ static bool console_emit_next_record(str +@@ -2691,8 +2790,10 @@ static bool console_emit_next_record(str con->seq++; @@ -357,7 +365,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> printk_delay(r.info->level); skip: -@@ -2824,7 +2925,7 @@ void console_unlock(void) +@@ -2826,7 +2927,7 @@ void console_unlock(void) * Re-check if there is a new record to flush. If the trylock * fails, another context is already handling the printing. */ @@ -366,7 +374,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL(console_unlock); -@@ -2855,6 +2956,10 @@ void console_unblank(void) +@@ -2857,6 +2958,10 @@ void console_unblank(void) if (oops_in_progress) { if (down_trylock_console_sem() != 0) return; @@ -377,7 +385,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } else { pr_flush(1000, true); console_lock(); -@@ -2936,10 +3041,6 @@ void console_start(struct console *conso +@@ -2938,10 +3043,6 @@ void console_start(struct console *conso console_lock(); console->flags |= CON_ENABLED; console_unlock(); @@ -388,7 +396,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> pr_flush(1000, true); } EXPORT_SYMBOL(console_start); -@@ -3134,7 +3235,11 @@ void register_console(struct console *ne +@@ -3136,7 +3237,11 @@ void register_console(struct console *ne if (newcon->flags & CON_EXTENDED) nr_ext_console_drivers++; @@ -400,7 +408,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (newcon->flags & CON_PRINTBUFFER) { /* Get a consistent copy of @syslog_seq. */ mutex_lock(&syslog_lock); -@@ -3397,16 +3502,17 @@ static bool printer_should_wake(struct c +@@ -3399,16 +3504,17 @@ static bool printer_should_wake(struct c if (kthread_should_stop()) return true; @@ -422,7 +430,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> return prb_read_valid(prb, seq, NULL); } -@@ -3417,7 +3523,6 @@ static int printk_kthread_func(void *dat +@@ -3419,7 +3525,6 @@ static int printk_kthread_func(void *dat char *dropped_text = NULL; char *ext_text = NULL; bool progress; @@ -430,7 +438,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> u64 seq = 0; char *text; int error; -@@ -3450,9 +3555,17 @@ static int printk_kthread_func(void *dat +@@ -3452,9 +3557,17 @@ static int printk_kthread_func(void *dat continue; do { @@ -451,7 +459,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> break; } -@@ -3466,14 +3579,13 @@ static int printk_kthread_func(void *dat +@@ -3468,14 +3581,13 @@ static int printk_kthread_func(void *dat */ console_may_schedule = 0; progress = console_emit_next_record(con, text, ext_text, diff --git a/patches/0012-printk-remove-console_locked.patch b/patches/0012-printk-remove-console_locked.patch index ad98f119ae36..af56470ce706 100644 --- a/patches/0012-printk-remove-console_locked.patch +++ b/patches/0012-printk-remove-console_locked.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:17 +0106 +Date: Mon, 7 Feb 2022 15:54:54 +0106 Subject: [PATCH 12/16] printk: remove @console_locked The static global variable @console_locked is used to help debug VT @@ -37,7 +37,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * Array of consoles built from command line options (console=) -@@ -2594,7 +2586,6 @@ void console_lock(void) +@@ -2596,7 +2588,6 @@ void console_lock(void) if (console_suspended) return; pause_all_consoles(); @@ -45,7 +45,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_may_schedule = 1; } EXPORT_SYMBOL(console_lock); -@@ -2619,7 +2610,6 @@ int console_trylock(void) +@@ -2621,7 +2612,6 @@ int console_trylock(void) up_console_sem(); return 0; } @@ -53,7 +53,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> console_may_schedule = 0; return 1; } -@@ -2646,14 +2636,25 @@ static int console_trylock_sched(bool ma +@@ -2648,14 +2638,25 @@ static int console_trylock_sched(bool ma return 0; } pause_all_consoles(); @@ -81,7 +81,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL(is_console_locked); -@@ -2686,8 +2687,6 @@ static inline bool console_is_usable(str +@@ -2688,8 +2689,6 @@ static inline bool console_is_usable(str static void __console_unlock(void) { @@ -90,7 +90,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * Depending on whether console_lock() or console_trylock() was used, * appropriately allow the kthread printers to continue. -@@ -2965,7 +2964,6 @@ void console_unblank(void) +@@ -2967,7 +2966,6 @@ void console_unblank(void) console_lock(); } diff --git a/patches/0013-console-introduce-CON_MIGHT_SLEEP-for-vt.patch b/patches/0013-console-introduce-CON_MIGHT_SLEEP-for-vt.patch index d6746a857504..4d14b008d00c 100644 --- a/patches/0013-console-introduce-CON_MIGHT_SLEEP-for-vt.patch +++ b/patches/0013-console-introduce-CON_MIGHT_SLEEP-for-vt.patch @@ -1,5 +1,5 @@ From: John Ogness <john.ogness@linutronix.de> -Date: Fri, 4 Feb 2022 16:01:17 +0106 +Date: Mon, 7 Feb 2022 15:54:58 +0106 Subject: [PATCH 13/16] console: introduce CON_MIGHT_SLEEP for vt Deadlocks and the framebuffer console have been a recurring issue @@ -43,7 +43,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> char name[16]; --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c -@@ -2835,6 +2835,8 @@ static bool console_flush_all(bool do_co +@@ -2837,6 +2837,8 @@ static bool console_flush_all(bool do_co if (!console_is_usable(con)) continue; diff --git a/patches/0014-printk-add-infrastucture-for-atomic-consoles.patch b/patches/0014-printk-add-infrastucture-for-atomic-consoles.patch index d8df9dd2c209..155fe3b30261 100644 --- a/patches/0014-printk-add-infrastucture-for-atomic-consoles.patch +++ b/patches/0014-printk-add-infrastucture-for-atomic-consoles.patch @@ -245,12 +245,12 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> static void console_lock_spinning_enable(void) { } static int console_lock_spinning_disable_and_check(void) { return 0; } static void call_console_driver(struct console *con, const char *text, size_t len, -- char *dropped_text) {} -+ char *dropped_text, bool atomic_printing) {} +- char *dropped_text) ++ char *dropped_text, bool atomic_printing) + { + } static bool suppress_message_printing(int level) { return false; } - static void printk_delay(int level) {} - static void start_printk_kthread(struct console *con) {} -@@ -2664,13 +2746,23 @@ EXPORT_SYMBOL(is_console_locked); +@@ -2666,13 +2748,23 @@ EXPORT_SYMBOL(is_console_locked); * * Requires the console_lock. */ @@ -276,7 +276,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * Console drivers may assume that per-cpu resources have been -@@ -2702,6 +2794,66 @@ static void __console_unlock(void) +@@ -2704,6 +2796,66 @@ static void __console_unlock(void) up_console_sem(); } @@ -343,16 +343,16 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * Print one record for the given console. The record printed is whatever * record is the next available record for the given console. -@@ -2714,6 +2866,8 @@ static void __console_unlock(void) +@@ -2716,6 +2868,8 @@ static void __console_unlock(void) * If dropped messages should be printed, @dropped_text is a buffer of size - * DROPPED_TEXT_MAX. Otherise @dropped_text must be NULL. + * DROPPED_TEXT_MAX. Otherwise @dropped_text must be NULL. * + * @atomic_printing specifies if atomic printing should be used. + * * Requires the console_lock. * * Returns false if the given console has no next record to print, otherwise -@@ -2725,7 +2879,8 @@ static void __console_unlock(void) +@@ -2727,7 +2881,8 @@ static void __console_unlock(void) * to disable allowing the console_lock to be taken over by a printk waiter. */ static bool console_emit_next_record(struct console *con, char *text, char *ext_text, @@ -362,7 +362,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> { struct printk_info info; struct printk_record r; -@@ -2733,23 +2888,27 @@ static bool console_emit_next_record(str +@@ -2735,23 +2890,27 @@ static bool console_emit_next_record(str bool allow_handover; char *write_text; size_t len; @@ -395,7 +395,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> goto skip; } -@@ -2782,10 +2941,10 @@ static bool console_emit_next_record(str +@@ -2784,10 +2943,10 @@ static bool console_emit_next_record(str } stop_critical_timings(); /* don't trace print latency */ @@ -408,7 +408,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (allow_handover) { *handover = console_lock_spinning_disable_and_check(); -@@ -2833,7 +2992,7 @@ static bool console_flush_all(bool do_co +@@ -2835,7 +2994,7 @@ static bool console_flush_all(bool do_co for_each_console(con) { bool progress; @@ -417,7 +417,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> continue; if ((con->flags & CON_MIGHT_SLEEP) && !do_cond_resched) continue; -@@ -2843,11 +3002,11 @@ static bool console_flush_all(bool do_co +@@ -2845,11 +3004,11 @@ static bool console_flush_all(bool do_co /* Extended consoles do not print "dropped messages". */ progress = console_emit_next_record(con, &text[0], &ext_text[0], NULL, @@ -431,7 +431,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } if (*handover) return true; -@@ -2868,6 +3027,67 @@ static bool console_flush_all(bool do_co +@@ -2870,6 +3029,67 @@ static bool console_flush_all(bool do_co return any_usable; } @@ -499,7 +499,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /** * console_unlock - unlock the console system * -@@ -2981,6 +3201,11 @@ void console_unblank(void) +@@ -2983,6 +3203,11 @@ void console_unblank(void) */ void console_flush_on_panic(enum con_flush_mode mode) { @@ -511,7 +511,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * If someone else is holding the console lock, trylock will fail * and may_schedule may be set. Ignore and proceed to unlock so -@@ -2997,7 +3222,7 @@ void console_flush_on_panic(enum con_flu +@@ -2999,7 +3224,7 @@ void console_flush_on_panic(enum con_flu seq = prb_first_valid_seq(prb); for_each_console(c) @@ -520,7 +520,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } console_unlock(); } -@@ -3238,16 +3463,19 @@ void register_console(struct console *ne +@@ -3240,16 +3465,19 @@ void register_console(struct console *ne if (consoles_paused) newcon->flags |= CON_PAUSED; @@ -543,7 +543,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } if (kthreads_started) start_printk_kthread(newcon); -@@ -3329,6 +3557,10 @@ int unregister_console(struct console *c +@@ -3331,6 +3559,10 @@ int unregister_console(struct console *c console_unlock(); console_sysfs_notify(); @@ -554,7 +554,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (console->exit) res = console->exit(console); -@@ -3463,7 +3695,7 @@ bool pr_flush(int timeout_ms, bool reset +@@ -3465,7 +3697,7 @@ bool pr_flush(int timeout_ms, bool reset console_lock(); for_each_console(con) { @@ -563,7 +563,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> continue; printk_seq = con->seq; if (printk_seq < seq) -@@ -3531,6 +3763,11 @@ static int printk_kthread_func(void *dat +@@ -3533,6 +3765,11 @@ static int printk_kthread_func(void *dat (con->flags & CON_BOOT) ? "boot" : "", con->name, con->index); @@ -575,7 +575,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> text = kmalloc(CONSOLE_LOG_MAX, GFP_KERNEL); if (!text) goto out; -@@ -3559,7 +3796,7 @@ static int printk_kthread_func(void *dat +@@ -3561,7 +3798,7 @@ static int printk_kthread_func(void *dat if (error) break; @@ -584,7 +584,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> mutex_unlock(&con->lock); break; } -@@ -3579,7 +3816,7 @@ static int printk_kthread_func(void *dat +@@ -3581,7 +3818,7 @@ static int printk_kthread_func(void *dat */ console_may_schedule = 0; progress = console_emit_next_record(con, text, ext_text, diff --git a/patches/0016-printk-avoid-preempt_disable-for-PREEMPT_RT.patch b/patches/0016-printk-avoid-preempt_disable-for-PREEMPT_RT.patch index d0b421420b18..7fcd95e13dee 100644 --- a/patches/0016-printk-avoid-preempt_disable-for-PREEMPT_RT.patch +++ b/patches/0016-printk-avoid-preempt_disable-for-PREEMPT_RT.patch @@ -82,7 +82,7 @@ Signed-off-by: John Ogness <john.ogness@linutronix.de> } wake_up_klogd(); -@@ -2922,8 +2936,13 @@ static bool console_emit_next_record(str +@@ -2924,8 +2938,13 @@ static bool console_emit_next_record(str len = record_print_text(&r, console_msg_format & MSG_FORMAT_SYSLOG, printk_time); } diff --git a/patches/Add_localversion_for_-RT_release.patch b/patches/Add_localversion_for_-RT_release.patch index f2d35e0c0528..7b3d2414e699 100644 --- a/patches/Add_localversion_for_-RT_release.patch +++ b/patches/Add_localversion_for_-RT_release.patch @@ -15,4 +15,4 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- /dev/null +++ b/localversion-rt @@ -0,0 +1 @@ -+-rt5 ++-rt6 diff --git a/patches/KVM__arm_arm64__downgrade_preempt_disabled_region_to_migrate_disable.patch b/patches/KVM__arm_arm64__downgrade_preempt_disabled_region_to_migrate_disable.patch index bf841eb60e43..bb18775b7325 100644 --- a/patches/KVM__arm_arm64__downgrade_preempt_disabled_region_to_migrate_disable.patch +++ b/patches/KVM__arm_arm64__downgrade_preempt_disabled_region_to_migrate_disable.patch @@ -27,7 +27,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c -@@ -846,7 +846,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v +@@ -864,7 +864,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v * involves poking the GIC, which must be done in a * non-preemptible context. */ @@ -36,7 +36,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> kvm_pmu_flush_hwstate(vcpu); -@@ -870,7 +870,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v +@@ -888,7 +888,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v kvm_timer_sync_user(vcpu); kvm_vgic_sync_hwstate(vcpu); local_irq_enable(); @@ -45,7 +45,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> continue; } -@@ -943,7 +943,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v +@@ -958,7 +958,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_v /* Exit types that need handling before we can be preempted */ handle_exit_early(vcpu, ret); diff --git a/patches/irq_poll-Use-raise_softirq_irqoff-in-cpu_dead-notifi.patch b/patches/irq_poll-Use-raise_softirq_irqoff-in-cpu_dead-notifi.patch index b18cc8753b8d..4d94fe3915e6 100644 --- a/patches/irq_poll-Use-raise_softirq_irqoff-in-cpu_dead-notifi.patch +++ b/patches/irq_poll-Use-raise_softirq_irqoff-in-cpu_dead-notifi.patch @@ -1,10 +1,10 @@ From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Date: Thu, 2 Apr 2020 21:16:30 +0200 -Subject: [PATCH] irq_poll: Use raise_softirq_irqoff() in cpu_dead notifier +Subject: [PATCH] irq_poll: Add local_bh_disable() in cpu_dead notifier __raise_softirq_irqoff() adds a bit to the pending sofirq mask and this is it. The softirq won't be handled in a deterministic way but randomly -when an interrupt fires and handles softirq in its irq_exit() routine or +when an interrupt fires and handles the softirq in its irq_exit() routine or if something randomly checks and handles pending softirqs in the call chain before the CPU goes idle. @@ -12,7 +12,7 @@ Add a local_bh_disable/enable() around the IRQ-off section which will handle pending softirqs. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> -Link: https://lkml.kernel.org/r/20210930103754.2128949-1-bigeasy@linutronix.de +Link: https://lore.kernel.org/all/YgJ%2FXWVxxWDVBBVA@linutronix.de --- lib/irq_poll.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/patches/locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch b/patches/locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch index 6db72b56436f..d3b374583bc6 100644 --- a/patches/locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch +++ b/patches/locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch @@ -10,6 +10,7 @@ because RT_MUTEX based locking is always used. Enable CONFIG_RT_MUTEXES by default on PREEMPT_RT builds. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +Link: https://lkml.kernel.org/r/YgKmhjkcuqWXdUjQ@linutronix.de --- init/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/patches/locking-local_lock-Make-the-empty-local_lock_-functi.patch b/patches/locking-local_lock-Make-the-empty-local_lock_-functi.patch index 52501259d245..e13835bb9ea5 100644 --- a/patches/locking-local_lock-Make-the-empty-local_lock_-functi.patch +++ b/patches/locking-local_lock-Make-the-empty-local_lock_-functi.patch @@ -4,7 +4,7 @@ Subject: [PATCH] locking/local_lock: Make the empty local_lock_*() function a macro. It has been said that local_lock() does not add any overhead compared to -preempt_disable() in a !LOCKDEP configuration. A microbenchmark showed +preempt_disable() in a !LOCKDEP configuration. A micro benchmark showed an unexpected result which can be reduced to the fact that local_lock() was not entirely optimized away. In the !LOCKDEP configuration local_lock_acquire() is an empty static @@ -12,13 +12,14 @@ inline function. On x86 the this_cpu_ptr() argument of that function is fully evaluated leading to an additional mov+add instructions which are not needed and not used. -Replace the static inline function whith a macro. The typecheck() macro +Replace the static inline function with a macro. The typecheck() macro ensures that the argument is of proper type while the resulting -dissasembly shows no traces of this_cpu_ptr(). +disassembly shows no traces of this_cpu_ptr(). -Link: https://lkml.kernel.org/r/20220105202623.1118172-1-bigeasy@linutronix.de Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Reviewed-by: Waiman Long <longman@redhat.com> +Reviewed-by: Davidlohr Bueso <dbueso@suse.de> +Link: https://lkml.kernel.org/r/YgKjciR60fZft2l4@linutronix.de --- include/linux/local_lock_internal.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/patches/printk-defer_console_output-use-atomic-update.patch b/patches/printk-defer_console_output-use-atomic-update.patch new file mode 100644 index 000000000000..af1f8e1250d2 --- /dev/null +++ b/patches/printk-defer_console_output-use-atomic-update.patch @@ -0,0 +1,50 @@ +From: John Ogness <john.ogness@linutronix.de> +Date: Mon, 7 Feb 2022 10:52:20 +0106 +Subject: [PATCH] printk: defer_console_output: use atomic update + +The per-cpu @printk_pending variable can be updated from +sleepable contexts, such as: + + get_random_bytes() + warn_unseeded_randomness() + printk_deferred() + defer_console_output() + +and can be updated from interrupt contexts, such as: + + handle_irq_event_percpu() + __irq_wake_thread() + wake_up_process() + try_to_wake_up() + select_task_rq() + select_fallback_rq() + printk_deferred() + defer_console_output() + +and can be updated from NMI contexts, such as: + + vprintk() + if (in_nmi()) defer_console_output() + +Therefore this_cpu_or(), the atomic variant of __this_cpu_or(), +should be used to update the variable. + +Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +Reviewed-by: Sergey Senozhatsky <senozhatsky@chromium.org> +Signed-off-by: John Ogness <john.ogness@linutronix.de> +Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +--- + kernel/printk/printk.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +--- a/kernel/printk/printk.c ++++ b/kernel/printk/printk.c +@@ -3262,7 +3262,7 @@ void defer_console_output(void) + return; + + preempt_disable(); +- __this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT); ++ this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT); + irq_work_queue(this_cpu_ptr(&wake_up_klogd_work)); + preempt_enable(); + } diff --git a/patches/ptrace__fix_ptrace_vs_tasklist_lock_race.patch b/patches/ptrace__fix_ptrace_vs_tasklist_lock_race.patch index 54dec73215bb..4fd942cdb07a 100644 --- a/patches/ptrace__fix_ptrace_vs_tasklist_lock_race.patch +++ b/patches/ptrace__fix_ptrace_vs_tasklist_lock_race.patch @@ -49,7 +49,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> /* * Special states are those that do not use the normal wait-loop pattern. See * the comment with set_special_state(). -@@ -2007,6 +2003,81 @@ static inline int test_tsk_need_resched( +@@ -2006,6 +2002,81 @@ static inline int test_tsk_need_resched( return unlikely(test_tsk_thread_flag(tsk,TIF_NEED_RESCHED)); } diff --git a/patches/random-Move-crng_fast_load-to-the-worker.patch b/patches/random-Move-crng_fast_load-to-the-worker.patch new file mode 100644 index 000000000000..32ad4a6d5da5 --- /dev/null +++ b/patches/random-Move-crng_fast_load-to-the-worker.patch @@ -0,0 +1,59 @@ +From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +Date: Thu, 10 Feb 2022 18:22:05 +0100 +Subject: [PATCH] random: Move crng_fast_load() to the worker. + +crng_fast_load() is invoked from hard IRQ context and acquires a +spinlock_t via a trylock. If the lock is locked in hard IRQ context then +the following locking attempt (on another CPU) will PI-boost the wrong +task. + +Move the crng_fast_load() invocation into the worker. + +Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +--- + drivers/char/random.c | 27 ++++++++++++++++++++++----- + 1 file changed, 22 insertions(+), 5 deletions(-) + +--- a/drivers/char/random.c ++++ b/drivers/char/random.c +@@ -1047,6 +1047,17 @@ static void mix_interrupt_randomness(str + struct fast_pool *fast_pool = container_of(work, struct fast_pool, mix); + u8 pool[sizeof(fast_pool->pool)]; + ++ if (unlikely(crng_init == 0)) { ++ size_t ret; ++ ++ ret = crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)); ++ if (ret) { ++ WRITE_ONCE(fast_pool->count, 0); ++ fast_pool->last = jiffies; ++ return; ++ } ++ } ++ + /* + * Since this is the result of a trip through the scheduler, xor in + * a cycle counter. It can't hurt, and might help. +@@ -1089,11 +1100,17 @@ void add_interrupt_randomness(int irq) + new_count = ++fast_pool->count; + + if (unlikely(crng_init == 0)) { +- if (new_count >= 64 && +- crng_fast_load((u8 *)fast_pool->pool, sizeof(fast_pool->pool)) > 0) { +- fast_pool->count = 0; +- fast_pool->last = now; +- } ++ if (new_count & FAST_POOL_MIX_INFLIGHT) ++ return; ++ ++ if (new_count < 64) ++ return; ++ ++ fast_pool->count |= FAST_POOL_MIX_INFLIGHT; ++ if (unlikely(!fast_pool->mix.func)) ++ INIT_WORK(&fast_pool->mix, mix_interrupt_randomness); ++ queue_work_on(raw_smp_processor_id(), system_highpri_wq, ++ &fast_pool->mix); + return; + } + diff --git a/patches/sched__Add_support_for_lazy_preemption.patch b/patches/sched__Add_support_for_lazy_preemption.patch index b2a60232bcfd..b20d79b1664f 100644 --- a/patches/sched__Add_support_for_lazy_preemption.patch +++ b/patches/sched__Add_support_for_lazy_preemption.patch @@ -177,7 +177,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- a/include/linux/sched.h +++ b/include/linux/sched.h -@@ -2007,6 +2007,43 @@ static inline int test_tsk_need_resched( +@@ -2006,6 +2006,43 @@ static inline int test_tsk_need_resched( return unlikely(test_tsk_thread_flag(tsk,TIF_NEED_RESCHED)); } diff --git a/patches/series b/patches/series index 000fa5282f02..13f33300fc6f 100644 --- a/patches/series +++ b/patches/series @@ -3,6 +3,7 @@ ########################################################################### # John's printk queue ########################################################################### +printk-defer_console_output-use-atomic-update.patch 0001-printk-rename-cpulock-functions.patch 0002-printk-cpu-sync-always-disable-interrupts.patch 0003-printk-use-percpu-flag-instead-of-cpu_online.patch @@ -32,6 +33,7 @@ irq_poll-Use-raise_softirq_irqoff-in-cpu_dead-notifi.patch smp_wake_ksoftirqd_on_preempt_rt_instead_do_softirq.patch tcp-Don-t-acquire-inet_listen_hashbucket-lock-with-d.patch locking-local_lock-Make-the-empty-local_lock_-functi.patch +locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch # sched 0001_kernel_fork_redo_ifdefs_around_task_s_handling.patch @@ -49,21 +51,27 @@ locking-local_lock-Make-the-empty-local_lock_-functi.patch 0003-mm-memcg-Add-a-local_lock_t-for-IRQ-and-TASK-object.patch 0004-mm-memcg-Allow-the-task_obj-optimization-only-on-non.patch +# net +0001-net-dev-Remove-preempt_disable-and-get_cpu-in-netif_.patch +0002-net-dev-Make-rps_lock-disable-interrupts.patch +0003-net-dev-Makes-sure-netif_rx-can-be-invoked-in-any-co.patch + ########################################################################### # Post ########################################################################### cgroup__use_irqsave_in_cgroup_rstat_flush_locked.patch mm__workingset__replace_IRQ-off_check_with_a_lockdep_assert..patch softirq-Use-a-dedicated-thread-for-timer-wakeups.patch -locking-Enable-RT_MUTEXES-by-default-on-PREEMPT_RT.patch # These two need some feedback. genirq-Provide-generic_handle_irq_safe.patch Use-generic_handle_irq_safe-where-it-makes-sense.patch -# Random, WIP +# Random, WIP, v4 0001-random-use-computational-hash-for-entropy-extraction.patch -0002-random-do-not-take-spinlocks-in-irq-handler.patch +0002-random-remove-batched-entropy-locking.patch +0003-random-defer-fast-pool-mixing-to-worker.patch +random-Move-crng_fast_load-to-the-worker.patch ########################################################################### # Kconfig bits: @@ -75,11 +83,6 @@ jump-label__disable_if_stop_machine_is_used.patch ########################################################################### sched-Make-preempt_enable_no_resched-behave-like-pre.patch -# net -0001-net-dev-Remove-preempt_disable-and-get_cpu-in-netif_.patch -0002-net-dev-Make-rps_lock-disable-interrupts.patch -0003-net-dev-Makes-sure-netif_rx-can-be-invoked-in-any-co.patch - ########################################################################### # sched: ########################################################################### diff --git a/patches/smp_wake_ksoftirqd_on_preempt_rt_instead_do_softirq.patch b/patches/smp_wake_ksoftirqd_on_preempt_rt_instead_do_softirq.patch index b3c5c704c708..5d420eafab2d 100644 --- a/patches/smp_wake_ksoftirqd_on_preempt_rt_instead_do_softirq.patch +++ b/patches/smp_wake_ksoftirqd_on_preempt_rt_instead_do_softirq.patch @@ -2,44 +2,66 @@ From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Subject: smp: Wake ksoftirqd on PREEMPT_RT instead do_softirq(). Date: Mon, 27 Sep 2021 09:38:14 +0200 -The softirq implementation on PREEMPT_RT does not provide do_softirq(). -The other user of do_softirq() is replaced with a local_bh_disable() -+ enable() around the possible raise-softirq invocation. This can not be -done here because migration_cpu_stop() is invoked with disabled -preemption. +The softirq implementation on PREEMPT_RT does not provide do_softirq(). The +softirq can not be handled directly here because migration_cpu_stop() is +invoked with disabled preemption/ interrupts. -Wake the softirq thread on PREEMPT_RT if there are any pending softirqs. +A known user of scheduling softirqs from a remote function call is the block +layer. It won't happen on PREEMPT_RT because it doesn't make sense for +latency/ performance reasons and is disabled. Nevertheless this should +be handled in case of a new user pops up rather than simply ignoring it. + +Waking ksoftirqd unconditionally can be problematic if softirqs were already +pending but not yet handled. This can happen since the migration thread +is running at a high priority and able to preempt a threaded-interrupt. +The woken-up ksoftirqd would catch-up all pending (and later raised) +softirqs which is not desired on PREEMPT_RT since it is no longer +handled where it has been originally raised. This in turn delays the +actual processing until a SCHED_OTHER task can run. + +Wake the softirq thread on PREEMPT_RT if a remote function call raised +softirqs. Add warning in this case since this condition is not desired. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> -Link: https://lore.kernel.org/r/20210927073814.x5h6osr4dgiu44sc@linutronix.de +Link: https://lkml.kernel.org/r/YgKgL6aPj8aBES6G@linutronix.de --- +v2…v3: + - Only wake ksoftirqd if the softirqs were raised wthin + flush_smp_call_function_queue(). + - Add a warning in the wake case. v1…v2: Drop an empty line. - kernel/smp.c | 14 ++++++++++++-- - 1 file changed, 12 insertions(+), 2 deletions(-) + kernel/smp.c | 21 ++++++++++++++++++--- + 1 file changed, 18 insertions(+), 3 deletions(-) --- --- a/kernel/smp.c +++ b/kernel/smp.c -@@ -690,10 +690,20 @@ void flush_smp_call_function_from_idle(v - +@@ -691,10 +691,25 @@ void flush_smp_call_function_from_idle(v cfd_seq_store(this_cpu_ptr(&cfd_seq_local)->idle, CFD_SEQ_NOCPU, smp_processor_id(), CFD_SEQ_IDLE); -+ local_irq_save(flags); - flush_smp_call_function_queue(true); +- flush_smp_call_function_queue(true); - if (local_softirq_pending()) - do_softirq(); -+ -+ if (local_softirq_pending()) { -+ if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { ++ if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { ++ flush_smp_call_function_queue(true); ++ if (local_softirq_pending()) + do_softirq(); -+ } else { ++ } else { ++ unsigned int pending_prev; ++ unsigned int pending_post; + ++ pending_prev = local_softirq_pending(); ++ flush_smp_call_function_queue(true); ++ pending_post = local_softirq_pending(); ++ ++ if (WARN_ON_ONCE(!pending_prev && pending_post)) { + struct task_struct *ksoftirqd = this_cpu_ksoftirqd(); + + if (ksoftirqd && !task_is_running(ksoftirqd)) + wake_up_process(ksoftirqd); + } + } - local_irq_restore(flags); } + diff --git a/patches/tcp-Don-t-acquire-inet_listen_hashbucket-lock-with-d.patch b/patches/tcp-Don-t-acquire-inet_listen_hashbucket-lock-with-d.patch index a0065abf3e7d..d11df4740e5a 100644 --- a/patches/tcp-Don-t-acquire-inet_listen_hashbucket-lock-with-d.patch +++ b/patches/tcp-Don-t-acquire-inet_listen_hashbucket-lock-with-d.patch @@ -49,6 +49,7 @@ Reported-by: Mike Galbraith <efault@gmx.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Link: https://lkml.kernel.org/r/12d6f9879a97cd56c09fb53dee343cbb14f7f1f7.camel@gmx.de Link: https://lkml.kernel.org/r/X9CheYjuXWc75Spa@hirez.programming.kicks-ass.net +Link: https://lkml.kernel.org/r/YgKh9fbQ2dcBu3e1@linutronix.de --- net/ipv4/inet_hashtables.c | 53 ++++++++++++++++++++++++++------------------ net/ipv6/inet6_hashtables.c | 5 ---- diff --git a/patches/x86__kvm_Require_const_tsc_for_RT.patch b/patches/x86__kvm_Require_const_tsc_for_RT.patch index ad1ccab7b8fa..d42e28b4402b 100644 --- a/patches/x86__kvm_Require_const_tsc_for_RT.patch +++ b/patches/x86__kvm_Require_const_tsc_for_RT.patch @@ -18,7 +18,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c -@@ -8811,6 +8811,12 @@ int kvm_arch_init(void *opaque) +@@ -8813,6 +8813,12 @@ int kvm_arch_init(void *opaque) goto out; } |