diff options
author | Sebastian Andrzej Siewior <bigeasy@linutronix.de> | 2020-10-24 00:33:30 +0200 |
---|---|---|
committer | Sebastian Andrzej Siewior <bigeasy@linutronix.de> | 2020-10-24 00:33:30 +0200 |
commit | e8077fadb604e13404de3e31ed2d674b8fca3e5c (patch) | |
tree | 0f357d6f21661a6e638a6012e70cd2c77523c135 | |
parent | a17bee0e4461c412981e4bcae695c2a51b6b231d (diff) | |
download | linux-rt-e8077fadb604e13404de3e31ed2d674b8fca3e5c.tar.gz |
[ANNOUNCE] v5.9.1-rt19v5.9.1-rt19-patches
Dear RT folks!
I'm pleased to announce the v5.9.1-rt19 patch set.
Changes since v5.9.1-rt18:
- Mike Galbraith reported a possible circular locking dependency with
a seqcount. Backported a patch from upstream solving the issue.
- David Runge reported a crash in the block layer.
- The migrate-disable series by Peter Zijlstra has been update to v4.
- Mike Galbraith a possible possible hang due to the new printk code.
Dropped a patch from the printk code which was causing the problem
as suggested by John Ogness.
- The rtmutex clean up in v5.9-rc8-rt12 restructured the code path and
removed the blk_schedule_flush_plug() invocation from the locking
path. It turns out that it is still required and has been added
back.
- A small rtmutex related clean up:
- Remove rt_mutex_lock_killable(), it has no users.
- Use _mutex_lock_io_nested() for _mutex_lock_io() to avoid
duplicated code.
Known issues
- It has been pointed out that due to changes to the printk code the
internal buffer representation changed. This is only an issue if tools
like `crash' are used to extract the printk buffer from a kernel memory
image.
The delta patch against v5.9.1-rt18 is appended below and can be found here:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/incr/patch-5.9.1-rt18-rt19.patch.xz
You can get this release via the git tree at:
git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git v5.9.1-rt19
The RT patch against v5.9.1 can be found here:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/older/patch-5.9.1-rt19.patch.xz
The split quilt queue is available at:
https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/older/patches-5.9.1-rt19.tar.xz
Sebastian
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
39 files changed, 379 insertions, 314 deletions
diff --git a/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch b/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch index a63db738db6e..8c337cede0f9 100644 --- a/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch +++ b/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Mon, 21 Sep 2020 12:58:17 +0200 +Date: Fri, 23 Oct 2020 12:11:59 +0200 Subject: [PATCH 01/19] stop_machine: Add function and caller debug info Crashes in stop-machine are hard to connect to the calling code, add a diff --git a/patches/0002-sched-Fix-balance_callback.patch b/patches/0002-sched-Fix-balance_callback.patch index 6254ec3ce16f..61e82a06f432 100644 --- a/patches/0002-sched-Fix-balance_callback.patch +++ b/patches/0002-sched-Fix-balance_callback.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Mon, 11 May 2020 14:13:00 +0200 +Date: Fri, 23 Oct 2020 12:12:00 +0200 Subject: [PATCH 02/19] sched: Fix balance_callback() The intent of balance_callback() has always been to delay executing diff --git a/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch b/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch index 3efaa2838aa4..6e3ba750fb66 100644 --- a/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch +++ b/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Fri, 11 Sep 2020 09:54:27 +0200 +Date: Fri, 23 Oct 2020 12:12:01 +0200 Subject: [PATCH 03/19] sched/hotplug: Ensure only per-cpu kthreads run during hotplug diff --git a/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch b/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch index 3bb9eee23e6d..25f1d4c5c561 100644 --- a/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch +++ b/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch @@ -1,5 +1,5 @@ From: Thomas Gleixner <tglx@linutronix.de> -Date: Mon, 14 Sep 2020 14:47:28 +0200 +Date: Fri, 23 Oct 2020 12:12:02 +0200 Subject: [PATCH 04/19] sched/core: Wait for tasks being pushed away on hotplug RT kernels need to ensure that all tasks which are not per CPU kthreads diff --git a/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch b/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch index e1b826d0b49b..60c3800a80de 100644 --- a/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch +++ b/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Fri, 25 Sep 2020 15:45:11 +0200 +Date: Fri, 23 Oct 2020 12:12:03 +0200 Subject: [PATCH 05/19] workqueue: Manually break affinity on hotplug Don't rely on the scheduler to force break affinity for us -- it will diff --git a/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch b/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch index 4988b287dd9f..c5d20e3a4ac7 100644 --- a/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch +++ b/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch @@ -1,5 +1,5 @@ From: Thomas Gleixner <tglx@linutronix.de> -Date: Wed, 16 Sep 2020 09:27:18 +0200 +Date: Fri, 23 Oct 2020 12:12:04 +0200 Subject: [PATCH 06/19] sched/hotplug: Consolidate task migration on CPU unplug With the new mechanism which kicks tasks off the outgoing CPU at the end of diff --git a/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch b/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch deleted file mode 100644 index dd8bc592bdc4..000000000000 --- a/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch +++ /dev/null @@ -1,49 +0,0 @@ -From: Thomas Gleixner <tglx@linutronix.de> -Date: Thu, 9 Jun 2011 11:43:52 +0200 -Subject: [PATCH 07/23] locking/rtmutex: Add rtmutex_lock_killable() - -Add "killable" type to rtmutex. We need this since rtmutex are used as -"normal" mutexes which do use this type. - -Signed-off-by: Thomas Gleixner <tglx@linutronix.de> ---- - include/linux/rtmutex.h | 1 + - kernel/locking/rtmutex.c | 17 +++++++++++++++++ - 2 files changed, 18 insertions(+) - ---- a/include/linux/rtmutex.h -+++ b/include/linux/rtmutex.h -@@ -99,6 +99,7 @@ extern void rt_mutex_lock(struct rt_mute - #endif - - extern int rt_mutex_lock_interruptible(struct rt_mutex *lock); -+extern int rt_mutex_lock_killable(struct rt_mutex *lock); - extern int rt_mutex_trylock(struct rt_mutex *lock); - - extern void rt_mutex_unlock(struct rt_mutex *lock); ---- a/kernel/locking/rtmutex.c -+++ b/kernel/locking/rtmutex.c -@@ -1538,6 +1538,23 @@ int __sched __rt_mutex_futex_trylock(str - } - - /** -+ * rt_mutex_lock_killable - lock a rt_mutex killable -+ * -+ * @lock: the rt_mutex to be locked -+ * -+ * Returns: -+ * 0 on success -+ * -EINTR when interrupted by a signal -+ */ -+int __sched rt_mutex_lock_killable(struct rt_mutex *lock) -+{ -+ might_sleep(); -+ -+ return rt_mutex_fastlock(lock, TASK_KILLABLE, rt_mutex_slowlock); -+} -+EXPORT_SYMBOL_GPL(rt_mutex_lock_killable); -+ -+/** - * rt_mutex_trylock - try to lock a rt_mutex - * - * @lock: the rt_mutex to be locked diff --git a/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch b/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch index 40a091e57c7a..e26dfb31fbc0 100644 --- a/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch +++ b/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Fri, 25 Sep 2020 16:42:31 +0200 +Date: Fri, 23 Oct 2020 12:12:05 +0200 Subject: [PATCH 07/19] sched: Fix hotplug vs CPU bandwidth control Since we now migrate tasks away before DYING, we should also move diff --git a/patches/0008-sched-Massage-set_cpus_allowed.patch b/patches/0008-sched-Massage-set_cpus_allowed.patch index f9dd57152f4d..3c79a4c11ade 100644 --- a/patches/0008-sched-Massage-set_cpus_allowed.patch +++ b/patches/0008-sched-Massage-set_cpus_allowed.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Wed, 16 Sep 2020 14:59:08 +0200 +Date: Fri, 23 Oct 2020 12:12:06 +0200 Subject: [PATCH 08/19] sched: Massage set_cpus_allowed() Thread a u32 flags word through the *set_cpus_allowed*() callchain. diff --git a/patches/0009-sched-Add-migrate_disable.patch b/patches/0009-sched-Add-migrate_disable.patch index d8d641c2a456..3f736a398e02 100644 --- a/patches/0009-sched-Add-migrate_disable.patch +++ b/patches/0009-sched-Add-migrate_disable.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Thu, 17 Sep 2020 10:38:30 +0200 +Date: Fri, 23 Oct 2020 12:12:07 +0200 Subject: [PATCH 09/19] sched: Add migrate_disable() Add the base migrate_disable() support (under protest). diff --git a/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch b/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch index 8a84be8f2b31..7511b4b4fc7d 100644 --- a/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch +++ b/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Fri, 18 Sep 2020 17:24:31 +0200 +Date: Fri, 23 Oct 2020 12:12:08 +0200 Subject: [PATCH 10/19] sched: Fix migrate_disable() vs set_cpus_allowed_ptr() Concurrent migrate_disable() and set_cpus_allowed_ptr() has @@ -30,12 +30,15 @@ that is exactly the case where there is a pending set_cpus_allowed_ptr(), so let that provide storage for the async stop machine. +Much thanks to Valentin who used TLA+ most effective and found lots of +'interesting' cases. + Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- include/linux/sched.h | 1 - kernel/sched/core.c | 162 ++++++++++++++++++++++++++++++++++++++++++-------- - 2 files changed, 140 insertions(+), 23 deletions(-) + kernel/sched/core.c | 234 +++++++++++++++++++++++++++++++++++++++++++------- + 2 files changed, 205 insertions(+), 30 deletions(-) --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -82,67 +85,143 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL_GPL(migrate_enable); -@@ -1811,6 +1822,7 @@ static struct rq *move_queued_task(struc +@@ -1809,8 +1820,16 @@ static struct rq *move_queued_task(struc + } + struct migration_arg { - struct task_struct *task; - int dest_cpu; -+ struct completion *done; +- struct task_struct *task; +- int dest_cpu; ++ struct task_struct *task; ++ int dest_cpu; ++ struct set_affinity_pending *pending; ++}; ++ ++struct set_affinity_pending { ++ refcount_t refs; ++ struct completion done; ++ struct cpu_stop_work stop_work; ++ struct migration_arg arg; }; /* -@@ -1845,6 +1857,7 @@ static int migration_cpu_stop(void *data +@@ -1842,16 +1861,19 @@ static struct rq *__migrate_task(struct + */ + static int migration_cpu_stop(void *data) + { ++ struct set_affinity_pending *pending; struct migration_arg *arg = data; struct task_struct *p = arg->task; ++ int dest_cpu = arg->dest_cpu; struct rq *rq = this_rq(); + bool complete = false; struct rq_flags rf; /* -@@ -1867,15 +1880,27 @@ static int migration_cpu_stop(void *data + * The original target CPU might have gone down and we might + * be on another CPU but it doesn't matter. + */ +- local_irq_disable(); ++ local_irq_save(rf.flags); + /* + * We need to explicitly wake pending tasks before running + * __migrate_task() such that we will not miss enforcing cpus_ptr +@@ -1861,21 +1883,83 @@ static int migration_cpu_stop(void *data + + raw_spin_lock(&p->pi_lock); + rq_lock(rq, &rf); ++ ++ pending = p->migration_pending; + /* + * If task_rq(p) != rq, it cannot be migrated here, because we're + * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because * we're holding p->pi_lock. */ if (task_rq(p) == rq) { + if (is_migration_disabled(p)) + goto out; + ++ if (pending) { ++ p->migration_pending = NULL; ++ complete = true; ++ } ++ ++ /* migrate_enable() -- we must not race against SCA */ ++ if (dest_cpu < 0) { ++ /* ++ * When this was migrate_enable() but we no longer ++ * have a @pending, a concurrent SCA 'fixed' things ++ * and we should be valid again. Nothing to do. ++ */ ++ if (!pending) { ++ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq))); ++ goto out; ++ } ++ ++ dest_cpu = cpumask_any_distribute(&p->cpus_mask); ++ } ++ if (task_on_rq_queued(p)) - rq = __migrate_task(rq, &rf, p, arg->dest_cpu); +- rq = __migrate_task(rq, &rf, p, arg->dest_cpu); ++ rq = __migrate_task(rq, &rf, p, dest_cpu); else - p->wake_cpu = arg->dest_cpu; +- p->wake_cpu = arg->dest_cpu; ++ p->wake_cpu = dest_cpu; + -+ if (arg->done) { -+ p->migration_pending = NULL; -+ complete = true; ++ } else if (dest_cpu < 0) { ++ /* ++ * This happens when we get migrated between migrate_enable()'s ++ * preempt_enable() and scheduling the stopper task. At that ++ * point we're a regular task again and not current anymore. ++ * ++ * A !PREEMPT kernel has a giant hole here, which makes it far ++ * more likely. ++ */ ++ ++ /* ++ * When this was migrate_enable() but we no longer have an ++ * @pending, a concurrent SCA 'fixed' things and we should be ++ * valid again. Nothing to do. ++ */ ++ if (!pending) { ++ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq))); ++ goto out; + } ++ ++ /* ++ * When migrate_enable() hits a rq mis-match we can't reliably ++ * determine is_migration_disabled() and so have to chase after ++ * it. ++ */ ++ task_rq_unlock(rq, p, &rf); ++ stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop, ++ &pending->arg, &pending->stop_work); ++ return 0; } +- rq_unlock(rq, &rf); +- raw_spin_unlock(&p->pi_lock); +out: - rq_unlock(rq, &rf); - raw_spin_unlock(&p->pi_lock); -- - local_irq_enable(); ++ task_rq_unlock(rq, p, &rf); + + if (complete) -+ complete_all(arg->done); ++ complete_all(&pending->done); + ++ /* For pending->{arg,stop_work} */ ++ pending = arg->pending; ++ if (pending && refcount_dec_and_test(&pending->refs)) ++ wake_up_var(&pending->refs); + +- local_irq_enable(); return 0; } -@@ -1944,6 +1969,112 @@ void do_set_cpus_allowed(struct task_str - __do_set_cpus_allowed(p, new_mask, 0); +@@ -1945,6 +2029,110 @@ void do_set_cpus_allowed(struct task_str } -+struct set_affinity_pending { -+ refcount_t refs; -+ struct completion done; -+ struct cpu_stop_work stop_work; -+ struct migration_arg arg; -+}; -+ -+/* + /* + * This function is wildly self concurrent, consider at least 3 times. + */ -+static int affine_move_task(struct rq *rq, struct rq_flags *rf, -+ struct task_struct *p, int dest_cpu, unsigned int flags) ++static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf, ++ int dest_cpu, unsigned int flags) +{ + struct set_affinity_pending my_pending = { }, *pending = NULL; + struct migration_arg arg = { @@ -194,12 +273,17 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + if (WARN_ON_ONCE(!pending)) + return -EINVAL; + -+ arg.done = &pending->done; -+ + if (flags & SCA_MIGRATE_ENABLE) { + ++ refcount_inc(&pending->refs); /* pending->{arg,stop_work} */ + task_rq_unlock(rq, p, rf); -+ pending->arg = arg; ++ ++ pending->arg = (struct migration_arg) { ++ .task = p, ++ .dest_cpu = -1, ++ .pending = pending, ++ }; ++ + stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, + &pending->arg, &pending->stop_work); + @@ -237,17 +321,18 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + return 0; +} + - /* ++/* * Change a given task's CPU affinity. Migrate the thread to a * proper CPU and schedule it away if the CPU it's executing on -@@ -2013,23 +2144,8 @@ static int __set_cpus_allowed_ptr(struct + * is removed from the allowed bitmask. +@@ -2013,23 +2201,8 @@ static int __set_cpus_allowed_ptr(struct p->nr_cpus_allowed != 1); } - /* Can the task run on the task's current CPU? If so, we're done */ - if (cpumask_test_cpu(task_cpu(p), new_mask)) - goto out; -+ return affine_move_task(rq, &rf, p, dest_cpu, flags); ++ return affine_move_task(rq, p, &rf, dest_cpu, flags); - if (task_running(rq, p) || p->state == TASK_WAKING) { - struct migration_arg arg = { p, dest_cpu }; @@ -265,3 +350,11 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> out: task_rq_unlock(rq, p, &rf); +@@ -3209,6 +3382,7 @@ static void __sched_fork(unsigned long c + init_numa_balancing(clone_flags, p); + #ifdef CONFIG_SMP + p->wake_entry.u_flags = CSD_TYPE_TTWU; ++ p->migration_pending = NULL; + #endif + } + diff --git a/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch b/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch index 30854aa465b9..6f4bcbc0f964 100644 --- a/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch +++ b/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch @@ -1,5 +1,5 @@ From: Thomas Gleixner <tglx@linutronix.de> -Date: Wed, 26 Aug 2020 14:08:10 +0200 +Date: Fri, 23 Oct 2020 12:12:09 +0200 Subject: [PATCH 11/19] sched/core: Make migrate disable and CPU hotplug cooperative @@ -59,7 +59,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> #endif /* -@@ -2638,6 +2651,11 @@ static inline bool is_migration_disabled +@@ -2695,6 +2708,11 @@ static inline bool is_migration_disabled return false; } @@ -71,7 +71,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> #endif static void -@@ -7006,15 +7024,20 @@ static void balance_push(struct rq *rq) +@@ -7064,15 +7082,20 @@ static void balance_push(struct rq *rq) * Both the cpu-hotplug and stop task are in this case and are * required to complete the hotplug process. */ @@ -94,7 +94,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> raw_spin_unlock(&rq->lock); rcuwait_wake_up(&rq->hotplug_wait); raw_spin_lock(&rq->lock); -@@ -7061,7 +7084,8 @@ static void balance_hotplug_wait(void) +@@ -7119,7 +7142,8 @@ static void balance_hotplug_wait(void) { struct rq *rq = this_rq(); @@ -104,7 +104,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> TASK_UNINTERRUPTIBLE); } -@@ -7306,7 +7330,7 @@ int sched_cpu_dying(unsigned int cpu) +@@ -7364,7 +7388,7 @@ int sched_cpu_dying(unsigned int cpu) sched_tick_stop(cpu); rq_lock_irqsave(rq, &rf); diff --git a/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch b/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch index da82575dbd0e..e9a57312e8cb 100644 --- a/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch +++ b/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Thu, 1 Oct 2020 15:54:14 +0200 +Date: Fri, 23 Oct 2020 12:12:10 +0200 Subject: [PATCH 12/19] sched,rt: Use cpumask_any*_distribute() Replace a bunch of cpumask_any*() instances with diff --git a/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch b/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch index 128aa4e97b69..c1f377bad590 100644 --- a/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch +++ b/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Thu, 1 Oct 2020 16:05:39 +0200 +Date: Fri, 23 Oct 2020 12:12:11 +0200 Subject: [PATCH 13/19] sched,rt: Use the full cpumask for balancing We want migrate_disable() tasks to get PULLs in order for them to PUSH diff --git a/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch b/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch index 2b2e6dac8e99..da06ec23a1bd 100644 --- a/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch +++ b/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch @@ -8,9 +8,9 @@ Required for lock implementation ontop of rtmutex. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- - kernel/locking/rtmutex.c | 58 ++++++++++++++++++++++++++-------------- + kernel/locking/rtmutex.c | 54 ++++++++++++++++++++++++++++------------ kernel/locking/rtmutex_common.h | 3 ++ - 2 files changed, 42 insertions(+), 19 deletions(-) + 2 files changed, 41 insertions(+), 16 deletions(-) --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -68,16 +68,9 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } EXPORT_SYMBOL_GPL(rt_mutex_lock_interruptible); -@@ -1552,12 +1564,18 @@ int __sched __rt_mutex_futex_trylock(str - */ - int __sched rt_mutex_lock_killable(struct rt_mutex *lock) - { -- might_sleep(); -- -- return rt_mutex_fastlock(lock, TASK_KILLABLE, rt_mutex_slowlock); -+ return rt_mutex_lock_state(lock, 0, TASK_KILLABLE); +@@ -1541,6 +1553,14 @@ int __sched __rt_mutex_futex_trylock(str + return __rt_mutex_slowtrylock(lock); } - EXPORT_SYMBOL_GPL(rt_mutex_lock_killable); +int __sched __rt_mutex_trylock(struct rt_mutex *lock) +{ @@ -90,7 +83,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /** * rt_mutex_trylock - try to lock a rt_mutex * -@@ -1573,10 +1591,7 @@ int __sched rt_mutex_trylock(struct rt_m +@@ -1556,10 +1576,7 @@ int __sched rt_mutex_trylock(struct rt_m { int ret; @@ -102,7 +95,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (ret) mutex_acquire(&lock->dep_map, 0, 1, _RET_IP_); -@@ -1584,6 +1599,11 @@ int __sched rt_mutex_trylock(struct rt_m +@@ -1567,6 +1584,11 @@ int __sched rt_mutex_trylock(struct rt_m } EXPORT_SYMBOL_GPL(rt_mutex_trylock); diff --git a/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch b/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch index 763ba836d58c..5401e0670dd4 100644 --- a/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch +++ b/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Thu, 1 Oct 2020 16:13:01 +0200 +Date: Fri, 23 Oct 2020 12:12:12 +0200 Subject: [PATCH 14/19] sched, lockdep: Annotate ->pi_lock recursion There's a valid ->pi_lock recursion issue where the actual PI code @@ -14,7 +14,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -2603,6 +2603,7 @@ int select_task_rq(struct task_struct *p +@@ -2660,6 +2660,7 @@ int select_task_rq(struct task_struct *p void sched_set_stop_task(int cpu, struct task_struct *stop) { @@ -22,7 +22,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> struct sched_param param = { .sched_priority = MAX_RT_PRIO - 1 }; struct task_struct *old_stop = cpu_rq(cpu)->stop; -@@ -2618,6 +2619,20 @@ void sched_set_stop_task(int cpu, struct +@@ -2675,6 +2676,20 @@ void sched_set_stop_task(int cpu, struct sched_setscheduler_nocheck(stop, SCHED_FIFO, ¶m); stop->sched_class = &stop_sched_class; diff --git a/patches/0015-printk-add-pr_flush.patch b/patches/0015-printk-add-pr_flush.patch deleted file mode 100644 index 00aaed1b31ff..000000000000 --- a/patches/0015-printk-add-pr_flush.patch +++ /dev/null @@ -1,96 +0,0 @@ -From: John Ogness <john.ogness@linutronix.de> -Date: Mon, 19 Oct 2020 23:14:40 +0206 -Subject: [PATCH 15/15] printk: add pr_flush() - -Add a function to allow a caller to wait until all printing threads -have caught up. A timeout can be set to limit the waiting. The -timeout is reset if forward progress is detected. - -Signed-off-by: John Ogness <john.ogness@linutronix.de> -Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> ---- - kernel/printk/printk.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++ - 1 file changed, 66 insertions(+) - ---- a/kernel/printk/printk.c -+++ b/kernel/printk/printk.c -@@ -3165,6 +3165,66 @@ const char *kmsg_dump_reason_str(enum km - EXPORT_SYMBOL_GPL(kmsg_dump_reason_str); - - /** -+ * pr_flush() - Wait for printing threads to catch up. -+ * -+ * @timeout_ms: The maximum time (in ms) to wait. -+ * @reset_on_progress: Reset the timeout if forward progress is seen. -+ * -+ * A value of 0 for @timeout_ms means no waiting will occur. A value of -1 -+ * represents infinite waiting. -+ * -+ * If @reset_on_progress is true, the timeout will be reset whenever any -+ * printer has been seen to make some forward progress. -+ * -+ * Context: Any context if @timeout_ms is 0. Otherwise process context and -+ * may sleep if a printer is not caught up. -+ * Return: true if all enabled printers are caught up. -+ */ -+static bool pr_flush(int timeout_ms, bool reset_on_progress) -+{ -+ int remaining = timeout_ms; -+ struct console *con; -+ u64 last_diff = 0; -+ u64 printk_seq; -+ u64 diff; -+ u64 seq; -+ -+ seq = prb_next_seq(prb); -+ -+ for (;;) { -+ diff = 0; -+ -+ for_each_console(con) { -+ if (!(con->flags & CON_ENABLED)) -+ continue; -+ printk_seq = atomic64_read(&con->printk_seq); -+ if (printk_seq < seq) -+ diff += seq - printk_seq; -+ } -+ -+ if (diff != last_diff && reset_on_progress) -+ remaining = timeout_ms; -+ -+ if (!diff || remaining == 0) -+ break; -+ -+ if (remaining < 0) { -+ msleep(100); -+ } else if (remaining < 100) { -+ msleep(remaining); -+ remaining = 0; -+ } else { -+ msleep(100); -+ remaining -= 100; -+ } -+ -+ last_diff = diff; -+ } -+ -+ return (diff == 0); -+} -+ -+/** - * kmsg_dump - dump kernel log to kernel message dumpers. - * @reason: the reason (oops, panic etc) for dumping - * -@@ -3186,6 +3246,12 @@ void kmsg_dump(enum kmsg_dump_reason rea - sync_mode = true; - pr_info("enabled sync mode\n"); - } -+ -+ /* -+ * Give the printing threads time to flush, allowing up to 1 -+ * second of no printing forward progress before giving up. -+ */ -+ pr_flush(1000, true); - } - - rcu_read_lock(); diff --git a/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch b/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch index 5b3a2108f981..20af7d2fe22b 100644 --- a/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch +++ b/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch @@ -37,7 +37,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> #ifdef CONFIG_SMP --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -3221,7 +3221,7 @@ try_to_wake_up(struct task_struct *p, un +@@ -3278,7 +3278,7 @@ try_to_wake_up(struct task_struct *p, un int cpu, success = 0; preempt_disable(); @@ -46,7 +46,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> /* * We're waking current, this means 'p->on_rq' and 'task_cpu(p) * == smp_processor_id()'. Together this means we can special -@@ -3251,8 +3251,26 @@ try_to_wake_up(struct task_struct *p, un +@@ -3308,8 +3308,26 @@ try_to_wake_up(struct task_struct *p, un */ raw_spin_lock_irqsave(&p->pi_lock, flags); smp_mb__after_spinlock(); @@ -74,7 +74,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> trace_sched_waking(p); -@@ -3442,6 +3460,18 @@ int wake_up_process(struct task_struct * +@@ -3499,6 +3517,18 @@ int wake_up_process(struct task_struct * } EXPORT_SYMBOL(wake_up_process); diff --git a/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch b/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch index 80feca872798..7d44d63cb595 100644 --- a/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch +++ b/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Mon, 28 Sep 2020 17:06:07 +0200 +Date: Fri, 23 Oct 2020 12:12:13 +0200 Subject: [PATCH 15/19] sched: Fix migrate_disable() vs rt/dl balancing In order to minimize the interference of migrate_disable() on lower @@ -107,7 +107,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> static inline bool rq_has_pinned_tasks(struct rq *rq) { return rq->nr_pinned; -@@ -1917,6 +1912,49 @@ static int migration_cpu_stop(void *data +@@ -1976,6 +1971,49 @@ static int migration_cpu_stop(void *data return 0; } @@ -157,7 +157,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * sched_class::set_cpus_allowed must do the below, but is not required to * actually call this function. -@@ -2004,6 +2042,14 @@ static int affine_move_task(struct rq *r +@@ -2056,6 +2094,14 @@ static int affine_move_task(struct rq *r /* Can the task run on the task's current CPU? If so, we're done */ if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) { @@ -172,7 +172,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> pending = p->migration_pending; if (pending) { refcount_inc(&pending->refs); -@@ -2012,6 +2058,11 @@ static int affine_move_task(struct rq *r +@@ -2064,6 +2110,11 @@ static int affine_move_task(struct rq *r } task_rq_unlock(rq, p, rf); @@ -184,15 +184,15 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (complete) goto do_complete; -@@ -2049,6 +2100,7 @@ static int affine_move_task(struct rq *r - +@@ -2100,6 +2151,7 @@ static int affine_move_task(struct rq *r if (flags & SCA_MIGRATE_ENABLE) { + refcount_inc(&pending->refs); /* pending->{arg,stop_work} */ + p->migration_flags &= ~MDF_PUSH; task_rq_unlock(rq, p, rf); - pending->arg = arg; - stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, -@@ -2661,11 +2713,6 @@ static inline int __set_cpus_allowed_ptr + + pending->arg = (struct migration_arg) { +@@ -2718,11 +2770,6 @@ static inline int __set_cpus_allowed_ptr static inline void migrate_disable_switch(struct rq *rq, struct task_struct *p) { } diff --git a/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch b/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch index 5147454efed8..3830e36e4126 100644 --- a/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch +++ b/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch @@ -936,7 +936,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } int __sched __rt_mutex_lock_state(struct rt_mutex *lock, int state) -@@ -1612,16 +1941,13 @@ void __sched __rt_mutex_unlock(struct rt +@@ -1597,16 +1926,13 @@ void __sched __rt_mutex_unlock(struct rt void __sched rt_mutex_unlock(struct rt_mutex *lock) { mutex_release(&lock->dep_map, _RET_IP_); @@ -957,7 +957,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> { lockdep_assert_held(&lock->wait_lock); -@@ -1638,23 +1964,35 @@ bool __sched __rt_mutex_futex_unlock(str +@@ -1623,23 +1949,35 @@ bool __sched __rt_mutex_futex_unlock(str * avoid inversion prior to the wakeup. preempt_disable() * therein pairs with rt_mutex_postunlock(). */ @@ -996,7 +996,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } /** -@@ -1690,7 +2028,7 @@ void __rt_mutex_init(struct rt_mutex *lo +@@ -1675,7 +2013,7 @@ void __rt_mutex_init(struct rt_mutex *lo if (name && key) debug_rt_mutex_init(lock, name, key); } @@ -1005,7 +1005,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /** * rt_mutex_init_proxy_locked - initialize and lock a rt_mutex on behalf of a -@@ -1710,6 +2048,14 @@ void rt_mutex_init_proxy_locked(struct r +@@ -1695,6 +2033,14 @@ void rt_mutex_init_proxy_locked(struct r struct task_struct *proxy_owner) { __rt_mutex_init(lock, NULL, NULL); @@ -1020,7 +1020,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> debug_rt_mutex_proxy_lock(lock, proxy_owner); rt_mutex_set_owner(lock, proxy_owner); } -@@ -1733,6 +2079,26 @@ void rt_mutex_proxy_unlock(struct rt_mut +@@ -1718,6 +2064,26 @@ void rt_mutex_proxy_unlock(struct rt_mut rt_mutex_set_owner(lock, NULL); } @@ -1047,7 +1047,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /** * __rt_mutex_start_proxy_lock() - Start lock acquisition for another task * @lock: the rt_mutex to take -@@ -1805,6 +2171,9 @@ int __rt_mutex_start_proxy_lock(struct r +@@ -1790,6 +2156,9 @@ int __rt_mutex_start_proxy_lock(struct r ret = 0; } @@ -1057,7 +1057,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> return ret; } -@@ -1894,6 +2263,9 @@ int rt_mutex_wait_proxy_lock(struct rt_m +@@ -1879,6 +2248,9 @@ int rt_mutex_wait_proxy_lock(struct rt_m * have to fix that up. */ fixup_rt_mutex_waiters(lock); diff --git a/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch b/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch index 9a02d96c393c..94883b8bbfa1 100644 --- a/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch +++ b/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Mon, 5 Oct 2020 12:49:16 +0200 +Date: Fri, 23 Oct 2020 12:12:14 +0200 Subject: [PATCH 16/19] sched/proc: Print accurate cpumask vs migrate_disable() Ensure /proc/*/status doesn't print 'random' cpumasks due to diff --git a/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch b/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch index 420f7499bb16..cc03f0bb1b3d 100644 --- a/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch +++ b/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch @@ -15,7 +15,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c -@@ -1899,7 +1899,11 @@ EXPORT_SYMBOL_GPL(rt_mutex_lock_killable +@@ -1884,7 +1884,11 @@ int __sched __rt_mutex_futex_trylock(str int __sched __rt_mutex_trylock(struct rt_mutex *lock) { diff --git a/patches/0017-sched-Add-migrate_disable-tracepoints.patch b/patches/0017-sched-Add-migrate_disable-tracepoints.patch index 76f0fa2f699b..016f62d84838 100644 --- a/patches/0017-sched-Add-migrate_disable-tracepoints.patch +++ b/patches/0017-sched-Add-migrate_disable-tracepoints.patch @@ -1,5 +1,5 @@ From: Peter Zijlstra <peterz@infradead.org> -Date: Fri, 2 Oct 2020 20:29:46 +0200 +Date: Fri, 23 Oct 2020 12:12:15 +0200 Subject: [PATCH 17/19] sched: Add migrate_disable() tracepoints XXX write a tracer: diff --git a/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch b/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch index e2887c3a6d98..21037b2580d8 100644 --- a/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch +++ b/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch @@ -6,15 +6,15 @@ Subject: [PATCH 18/23] locking/rtmutex: add mutex implementation based on Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- - include/linux/mutex_rt.h | 131 +++++++++++++++++++++++++++ - kernel/locking/mutex-rt.c | 222 ++++++++++++++++++++++++++++++++++++++++++++++ - 2 files changed, 353 insertions(+) + include/linux/mutex_rt.h | 130 ++++++++++++++++++++++++++ + kernel/locking/mutex-rt.c | 224 ++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 354 insertions(+) create mode 100644 include/linux/mutex_rt.h create mode 100644 kernel/locking/mutex-rt.c --- /dev/null +++ b/include/linux/mutex_rt.h -@@ -0,0 +1,131 @@ +@@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0-only +#ifndef __LINUX_MUTEX_RT_H +#define __LINUX_MUTEX_RT_H @@ -46,7 +46,6 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + +extern void __mutex_do_init(struct mutex *lock, const char *name, struct lock_class_key *key); +extern void __lockfunc _mutex_lock(struct mutex *lock); -+extern void __lockfunc _mutex_lock_io(struct mutex *lock); +extern void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass); +extern int __lockfunc _mutex_lock_interruptible(struct mutex *lock); +extern int __lockfunc _mutex_lock_killable(struct mutex *lock); @@ -63,7 +62,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +#define mutex_lock_killable(l) _mutex_lock_killable(l) +#define mutex_trylock(l) _mutex_trylock(l) +#define mutex_unlock(l) _mutex_unlock(l) -+#define mutex_lock_io(l) _mutex_lock_io(l); ++#define mutex_lock_io(l) _mutex_lock_io_nested(l, 0); + +#define __mutex_owner(l) ((l)->lock.owner) + @@ -94,7 +93,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +# define mutex_lock_killable_nested(l, s) \ + _mutex_lock_killable(l) +# define mutex_lock_nest_lock(lock, nest_lock) mutex_lock(lock) -+# define mutex_lock_io_nested(l, s) _mutex_lock_io(l) ++# define mutex_lock_io_nested(l, s) _mutex_lock_io_nested(l, s) +#endif + +# define mutex_init(mutex) \ @@ -148,7 +147,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +#endif --- /dev/null +++ b/kernel/locking/mutex-rt.c -@@ -0,0 +1,222 @@ +@@ -0,0 +1,224 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Real-Time Preemption Support @@ -216,6 +215,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +#include <linux/fs.h> +#include <linux/futex.h> +#include <linux/hrtimer.h> ++#include <linux/blkdev.h> + +#include "rtmutex_common.h" + @@ -236,29 +236,43 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +} +EXPORT_SYMBOL(__mutex_do_init); + ++static int _mutex_lock_blk_flush(struct mutex *lock, int state) ++{ ++ /* ++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too ++ * late if one of the callbacks needs to acquire a sleeping lock. ++ */ ++ if (blk_needs_flush_plug(current)) ++ blk_schedule_flush_plug(current); ++ return __rt_mutex_lock_state(&lock->lock, state); ++} ++ +void __lockfunc _mutex_lock(struct mutex *lock) +{ + mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_); -+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE); ++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE); +} +EXPORT_SYMBOL(_mutex_lock); + -+void __lockfunc _mutex_lock_io(struct mutex *lock) ++void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass) +{ + int token; + + token = io_schedule_prepare(); -+ _mutex_lock(lock); ++ ++ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_); ++ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE); ++ + io_schedule_finish(token); +} -+EXPORT_SYMBOL_GPL(_mutex_lock_io); ++EXPORT_SYMBOL_GPL(_mutex_lock_io_nested); + +int __lockfunc _mutex_lock_interruptible(struct mutex *lock) +{ + int ret; + + mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_); -+ ret = __rt_mutex_lock_state(&lock->lock, TASK_INTERRUPTIBLE); ++ ret = _mutex_lock_blk_flush(lock, TASK_INTERRUPTIBLE); + if (ret) + mutex_release(&lock->dep_map, _RET_IP_); + return ret; @@ -270,7 +284,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + int ret; + + mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_); -+ ret = __rt_mutex_lock_state(&lock->lock, TASK_KILLABLE); ++ ret = _mutex_lock_blk_flush(lock, TASK_KILLABLE); + if (ret) + mutex_release(&lock->dep_map, _RET_IP_); + return ret; @@ -281,27 +295,14 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +void __lockfunc _mutex_lock_nested(struct mutex *lock, int subclass) +{ + mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_); -+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE); ++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE); +} +EXPORT_SYMBOL(_mutex_lock_nested); + -+void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass) -+{ -+ int token; -+ -+ token = io_schedule_prepare(); -+ -+ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_); -+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE); -+ -+ io_schedule_finish(token); -+} -+EXPORT_SYMBOL_GPL(_mutex_lock_io_nested); -+ +void __lockfunc _mutex_lock_nest_lock(struct mutex *lock, struct lockdep_map *nest) +{ + mutex_acquire_nest(&lock->dep_map, 0, 0, nest, _RET_IP_); -+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE); ++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE); +} +EXPORT_SYMBOL(_mutex_lock_nest_lock); + @@ -310,7 +311,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + int ret; + + mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_); -+ ret = __rt_mutex_lock_state(&lock->lock, TASK_INTERRUPTIBLE); ++ ret = _mutex_lock_blk_flush(lock, TASK_INTERRUPTIBLE); + if (ret) + mutex_release(&lock->dep_map, _RET_IP_); + return ret; @@ -322,7 +323,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + int ret; + + mutex_acquire(&lock->dep_map, subclass, 0, _RET_IP_); -+ ret = __rt_mutex_lock_state(&lock->lock, TASK_KILLABLE); ++ ret = _mutex_lock_blk_flush(lock, TASK_KILLABLE); + if (ret) + mutex_release(&lock->dep_map, _RET_IP_); + return ret; diff --git a/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch b/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch index 75b9d9525f00..12e20edee164 100644 --- a/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch +++ b/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch @@ -1,5 +1,5 @@ From: Valentin Schneider <valentin.schneider@arm.com> -Date: Tue, 13 Oct 2020 15:01:15 +0100 +Date: Fri, 23 Oct 2020 12:12:16 +0200 Subject: [PATCH 18/19] sched: Deny self-issued __set_cpus_allowed_ptr() when migrate_disable() @@ -17,7 +17,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -2187,8 +2187,17 @@ static int __set_cpus_allowed_ptr(struct +@@ -2244,8 +2244,17 @@ static int __set_cpus_allowed_ptr(struct goto out; } diff --git a/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch b/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch index 9e9b59eccfde..c3d3173593f4 100644 --- a/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch +++ b/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch @@ -43,8 +43,8 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- include/linux/rwsem-rt.h | 69 ++++++++++ - kernel/locking/rwsem-rt.c | 292 ++++++++++++++++++++++++++++++++++++++++++++++ - 2 files changed, 361 insertions(+) + kernel/locking/rwsem-rt.c | 307 ++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 376 insertions(+) create mode 100644 include/linux/rwsem-rt.h create mode 100644 kernel/locking/rwsem-rt.c @@ -122,12 +122,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +#endif --- /dev/null +++ b/kernel/locking/rwsem-rt.c -@@ -0,0 +1,292 @@ +@@ -0,0 +1,307 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/rwsem.h> +#include <linux/sched/debug.h> +#include <linux/sched/signal.h> +#include <linux/export.h> ++#include <linux/blkdev.h> + +#include "rtmutex_common.h" + @@ -212,6 +213,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + if (__down_read_trylock(sem)) + return 0; + ++ /* ++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too ++ * late if one of the callbacks needs to acquire a sleeping lock. ++ */ ++ if (blk_needs_flush_plug(current)) ++ blk_schedule_flush_plug(current); ++ + might_sleep(); + raw_spin_lock_irq(&m->wait_lock); + /* @@ -334,6 +342,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + struct rt_mutex *m = &sem->rtmutex; + unsigned long flags; + ++ /* ++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too ++ * late if one of the callbacks needs to acquire a sleeping lock. ++ */ ++ if (blk_needs_flush_plug(current)) ++ blk_schedule_flush_plug(current); ++ + /* Take the rtmutex as a first step */ + if (__rt_mutex_lock_state(m, state)) + return -EINTR; diff --git a/patches/0019-sched-Comment-affine_move_task.patch b/patches/0019-sched-Comment-affine_move_task.patch index 203f4dc2f648..272305c2edf6 100644 --- a/patches/0019-sched-Comment-affine_move_task.patch +++ b/patches/0019-sched-Comment-affine_move_task.patch @@ -1,5 +1,5 @@ From: Valentin Schneider <valentin.schneider@arm.com> -Date: Tue, 13 Oct 2020 15:01:16 +0100 +Date: Fri, 23 Oct 2020 12:12:17 +0200 Subject: [PATCH 19/19] sched: Comment affine_move_task() Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> @@ -12,8 +12,8 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -2032,7 +2032,75 @@ struct set_affinity_pending { - }; +@@ -2084,7 +2084,75 @@ void do_set_cpus_allowed(struct task_str + } /* - * This function is wildly self concurrent, consider at least 3 times. @@ -87,9 +87,9 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> + * pending affinity completion is preceded an uninstallion of + * p->migration_pending done with p->pi_lock held. */ - static int affine_move_task(struct rq *rq, struct rq_flags *rf, - struct task_struct *p, int dest_cpu, unsigned int flags) -@@ -2076,6 +2144,7 @@ static int affine_move_task(struct rq *r + static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf, + int dest_cpu, unsigned int flags) +@@ -2128,6 +2196,7 @@ static int affine_move_task(struct rq *r if (!(flags & SCA_MIGRATE_ENABLE)) { /* serialized by p->pi_lock */ if (!p->migration_pending) { @@ -97,7 +97,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> refcount_set(&my_pending.refs, 1); init_completion(&my_pending.done); p->migration_pending = &my_pending; -@@ -2114,7 +2183,11 @@ static int affine_move_task(struct rq *r +@@ -2171,7 +2240,11 @@ static int affine_move_task(struct rq *r } if (task_running(rq, p) || p->state == TASK_WAKING) { @@ -110,7 +110,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> task_rq_unlock(rq, p, rf); stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg); -@@ -2139,6 +2212,10 @@ static int affine_move_task(struct rq *r +@@ -2196,6 +2269,10 @@ static int affine_move_task(struct rq *r if (refcount_dec_and_test(&pending->refs)) wake_up_var(&pending->refs); diff --git a/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch b/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch index 7ed201e99523..e59b8c3c3c11 100644 --- a/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch +++ b/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch @@ -303,7 +303,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } /** -@@ -2261,7 +2407,7 @@ int rt_mutex_wait_proxy_lock(struct rt_m +@@ -2246,7 +2392,7 @@ int rt_mutex_wait_proxy_lock(struct rt_m raw_spin_lock_irq(&lock->wait_lock); /* sleep on the mutex */ set_current_state(TASK_INTERRUPTIBLE); @@ -312,7 +312,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * try_to_take_rt_mutex() sets the waiter bit unconditionally. We might * have to fix that up. -@@ -2331,3 +2477,97 @@ bool rt_mutex_cleanup_proxy_lock(struct +@@ -2316,3 +2462,97 @@ bool rt_mutex_cleanup_proxy_lock(struct return cleanup; } @@ -430,7 +430,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> struct rt_mutex_waiter *waiter, --- a/kernel/locking/rwsem-rt.c +++ b/kernel/locking/rwsem-rt.c -@@ -130,7 +130,7 @@ static int __sched __down_read_common(st +@@ -138,7 +138,7 @@ static int __sched __down_read_common(st */ rt_mutex_init_waiter(&waiter, false); ret = rt_mutex_slowlock_locked(m, state, NULL, RT_MUTEX_MIN_CHAINWALK, diff --git a/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch b/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch index 91368081d64e..0837da1c0e57 100644 --- a/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch +++ b/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch @@ -120,7 +120,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -4899,7 +4899,7 @@ pick_next_task(struct rq *rq, struct tas +@@ -4957,7 +4957,7 @@ pick_next_task(struct rq *rq, struct tas * * WARNING: must be called with preemption disabled! */ @@ -129,7 +129,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> { struct task_struct *prev, *next; unsigned long *switch_count; -@@ -4952,7 +4952,7 @@ static void __sched notrace __schedule(b +@@ -5010,7 +5010,7 @@ static void __sched notrace __schedule(b * - ptrace_{,un}freeze_traced() can change ->state underneath us. */ prev_state = prev->state; @@ -138,7 +138,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> if (signal_pending_state(prev_state, prev)) { prev->state = TASK_RUNNING; } else { -@@ -5036,7 +5036,7 @@ void __noreturn do_task_dead(void) +@@ -5094,7 +5094,7 @@ void __noreturn do_task_dead(void) /* Tell freezer to ignore us: */ current->flags |= PF_NOFREEZE; @@ -147,7 +147,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> BUG(); /* Avoid "noreturn function does return" - but don't continue if BUG() is a NOP: */ -@@ -5066,9 +5066,6 @@ static inline void sched_submit_work(str +@@ -5124,9 +5124,6 @@ static inline void sched_submit_work(str preempt_enable_no_resched(); } @@ -157,7 +157,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * If we are going to sleep and we have plugged IO queued, * make sure to submit it to avoid deadlocks. -@@ -5094,7 +5091,7 @@ asmlinkage __visible void __sched schedu +@@ -5152,7 +5149,7 @@ asmlinkage __visible void __sched schedu sched_submit_work(tsk); do { preempt_disable(); @@ -166,7 +166,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> sched_preempt_enable_no_resched(); } while (need_resched()); sched_update_worker(tsk); -@@ -5122,7 +5119,7 @@ void __sched schedule_idle(void) +@@ -5180,7 +5177,7 @@ void __sched schedule_idle(void) */ WARN_ON_ONCE(current->state); do { @@ -175,7 +175,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> } while (need_resched()); } -@@ -5175,7 +5172,7 @@ static void __sched notrace preempt_sche +@@ -5233,7 +5230,7 @@ static void __sched notrace preempt_sche */ preempt_disable_notrace(); preempt_latency_start(1); @@ -184,7 +184,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> preempt_latency_stop(1); preempt_enable_no_resched_notrace(); -@@ -5205,6 +5202,19 @@ asmlinkage __visible void __sched notrac +@@ -5263,6 +5260,19 @@ asmlinkage __visible void __sched notrac NOKPROBE_SYMBOL(preempt_schedule); EXPORT_SYMBOL(preempt_schedule); @@ -204,7 +204,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /** * preempt_schedule_notrace - preempt_schedule called by tracing * -@@ -5248,7 +5258,7 @@ asmlinkage __visible void __sched notrac +@@ -5306,7 +5316,7 @@ asmlinkage __visible void __sched notrac * an infinite recursion. */ prev_ctx = exception_enter(); @@ -213,7 +213,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> exception_exit(prev_ctx); preempt_latency_stop(1); -@@ -5277,7 +5287,7 @@ asmlinkage __visible void __sched preemp +@@ -5335,7 +5345,7 @@ asmlinkage __visible void __sched preemp do { preempt_disable(); local_irq_enable(); diff --git a/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch b/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch new file mode 100644 index 000000000000..4754e90fd194 --- /dev/null +++ b/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch @@ -0,0 +1,37 @@ +From: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +Date: Fri, 23 Oct 2020 12:21:51 +0200 +Subject: [PATCH] blk-mq: Don't IPI requests on PREEMPT_RT + +blk_mq_complete_request_remote() will dispatch request completion to +another CPU via IPI if the CPU belongs to a different cache domain. + +This breaks on PREEMPT_RT because the IPI function will complete the +request in IRQ context which includes acquiring spinlock_t typed locks. +Completing the IPI request in softirq on the remote CPU is probably less +efficient because it would require to wake ksoftirqd for this task +(which runs at SCHED_OTHER). + +Ignoring the IPI request and completing the request locally is probably +the best option. It be completed either in the IRQ-thread or at the end +of the routine in softirq context. + +Let blk_mq_complete_need_ipi() return that there is no need for IPI on +PREEMPT_RT. + +Reported-by: David Runge <dave@sleepmap.de> +Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +--- + block/blk-mq.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +--- a/block/blk-mq.c ++++ b/block/blk-mq.c +@@ -645,7 +645,7 @@ static inline bool blk_mq_complete_need_ + { + int cpu = raw_smp_processor_id(); + +- if (!IS_ENABLED(CONFIG_SMP) || ++ if (!IS_ENABLED(CONFIG_SMP) || IS_ENABLED(CONFIG_PREEMPT_RT) || + !test_bit(QUEUE_FLAG_SAME_COMP, &rq->q->queue_flags)) + return false; + diff --git a/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch b/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch index b128b5d1f642..66713a1c3cc3 100644 --- a/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch +++ b/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch @@ -54,7 +54,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> security_task_free(tsk); --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -4178,15 +4178,6 @@ static struct rq *finish_task_switch(str +@@ -4236,15 +4236,6 @@ static struct rq *finish_task_switch(str if (prev->sched_class->task_dead) prev->sched_class->task_dead(prev); diff --git a/patches/localversion.patch b/patches/localversion.patch index 48a458c6f3b7..19d7ea05016c 100644 --- a/patches/localversion.patch +++ b/patches/localversion.patch @@ -10,4 +10,4 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> --- /dev/null +++ b/localversion-rt @@ -0,0 +1 @@ -+-rt18 ++-rt19 diff --git a/patches/preempt-lazy-support.patch b/patches/preempt-lazy-support.patch index 4a3ca64d24fc..c3281447651c 100644 --- a/patches/preempt-lazy-support.patch +++ b/patches/preempt-lazy-support.patch @@ -321,7 +321,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> preempt_enable(); trace_sched_migrate_enable_tp(p); -@@ -3758,6 +3802,9 @@ int sched_fork(unsigned long clone_flags +@@ -3816,6 +3860,9 @@ int sched_fork(unsigned long clone_flags p->on_cpu = 0; #endif init_task_preempt_count(p); @@ -331,7 +331,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> #ifdef CONFIG_SMP plist_node_init(&p->pushable_tasks, MAX_PRIO); RB_CLEAR_NODE(&p->pushable_dl_tasks); -@@ -4999,6 +5046,7 @@ static void __sched notrace __schedule(b +@@ -5057,6 +5104,7 @@ static void __sched notrace __schedule(b next = pick_next_task(rq, prev, &rf); clear_tsk_need_resched(prev); @@ -339,7 +339,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> clear_preempt_need_resched(); if (likely(prev != next)) { -@@ -5195,6 +5243,30 @@ static void __sched notrace preempt_sche +@@ -5253,6 +5301,30 @@ static void __sched notrace preempt_sche } while (need_resched()); } @@ -370,7 +370,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> #ifdef CONFIG_PREEMPTION /* * This is the entry point to schedule() from in-kernel preemption -@@ -5208,7 +5280,8 @@ asmlinkage __visible void __sched notrac +@@ -5266,7 +5338,8 @@ asmlinkage __visible void __sched notrac */ if (likely(!preemptible())) return; @@ -380,7 +380,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> preempt_schedule_common(); } NOKPROBE_SYMBOL(preempt_schedule); -@@ -5248,6 +5321,9 @@ asmlinkage __visible void __sched notrac +@@ -5306,6 +5379,9 @@ asmlinkage __visible void __sched notrac if (likely(!preemptible())) return; @@ -390,7 +390,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> do { /* * Because the function tracer can trace preempt_count_sub() -@@ -7086,7 +7162,9 @@ void init_idle(struct task_struct *idle, +@@ -7144,7 +7220,9 @@ void init_idle(struct task_struct *idle, /* Set the preempt count _outside_ the spinlocks! */ init_idle_preempt_count(idle, cpu); diff --git a/patches/printk-Tiny-cleanup.patch b/patches/printk-Tiny-cleanup.patch index 54a8fec446c3..5fed4d33a19f 100644 --- a/patches/printk-Tiny-cleanup.patch +++ b/patches/printk-Tiny-cleanup.patch @@ -125,7 +125,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> { return vprintk_emit(0, LOGLEVEL_DEFAULT, NULL, fmt, args); } -@@ -3369,7 +3345,7 @@ EXPORT_SYMBOL_GPL(kmsg_dump_get_line); +@@ -3303,7 +3279,7 @@ EXPORT_SYMBOL_GPL(kmsg_dump_get_line); * @syslog: include the "<4>" prefixes * @buf: buffer to copy the line to * @size: maximum size of the buffer @@ -134,7 +134,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> * * Start at the end of the kmsg buffer and fill the provided buffer * with as many of the the *youngest* kmsg records that fit into it. -@@ -3536,7 +3512,7 @@ static bool __prb_trylock(struct prb_cpu +@@ -3470,7 +3446,7 @@ static bool __prb_trylock(struct prb_cpu * * It is safe to call this function from any context and state. */ @@ -143,7 +143,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> { for (;;) { if (__prb_trylock(cpu_lock, cpu_store)) -@@ -3554,7 +3530,7 @@ void prb_lock(struct prb_cpulock *cpu_lo +@@ -3488,7 +3464,7 @@ void prb_lock(struct prb_cpulock *cpu_lo * * It is safe to call this function from any context and state. */ diff --git a/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch b/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch index 6b940147385f..7fd06b76cf3a 100644 --- a/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch +++ b/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch @@ -115,7 +115,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> spin_unlock_irq(&task->sighand->siglock); --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -2511,6 +2511,18 @@ int migrate_swap(struct task_struct *cur +@@ -2568,6 +2568,18 @@ int migrate_swap(struct task_struct *cur } #endif /* CONFIG_NUMA_BALANCING */ @@ -134,7 +134,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> /* * wait_task_inactive - wait for a thread to unschedule. * -@@ -2555,7 +2567,7 @@ unsigned long wait_task_inactive(struct +@@ -2612,7 +2624,7 @@ unsigned long wait_task_inactive(struct * is actually now running somewhere else! */ while (task_running(rq, p)) { @@ -143,7 +143,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> return 0; cpu_relax(); } -@@ -2570,7 +2582,8 @@ unsigned long wait_task_inactive(struct +@@ -2627,7 +2639,8 @@ unsigned long wait_task_inactive(struct running = task_running(rq, p); queued = task_on_rq_queued(p); ncsw = 0; diff --git a/patches/sched-might-sleep-do-not-account-rcu-depth.patch b/patches/sched-might-sleep-do-not-account-rcu-depth.patch index 305bcf173a15..bebb0df42aec 100644 --- a/patches/sched-might-sleep-do-not-account-rcu-depth.patch +++ b/patches/sched-might-sleep-do-not-account-rcu-depth.patch @@ -36,7 +36,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> /* Internal to kernel */ --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -7795,7 +7795,7 @@ void __init sched_init(void) +@@ -7853,7 +7853,7 @@ void __init sched_init(void) #ifdef CONFIG_DEBUG_ATOMIC_SLEEP static inline int preempt_count_equals(int preempt_offset) { diff --git a/patches/sched-mmdrop-delayed.patch b/patches/sched-mmdrop-delayed.patch index 4d4ea6476ff1..1177da334795 100644 --- a/patches/sched-mmdrop-delayed.patch +++ b/patches/sched-mmdrop-delayed.patch @@ -77,7 +77,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> struct mm_struct *mm; --- a/kernel/sched/core.c +++ b/kernel/sched/core.c -@@ -4166,9 +4166,13 @@ static struct rq *finish_task_switch(str +@@ -4224,9 +4224,13 @@ static struct rq *finish_task_switch(str * provided by mmdrop(), * - a sync_core for SYNC_CORE. */ @@ -92,7 +92,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de> } if (unlikely(prev_state == TASK_DEAD)) { if (prev->sched_class->task_dead) -@@ -7183,6 +7187,7 @@ void sched_setnuma(struct task_struct *p +@@ -7241,6 +7245,7 @@ void sched_setnuma(struct task_struct *p #endif /* CONFIG_NUMA_BALANCING */ #ifdef CONFIG_HOTPLUG_CPU diff --git a/patches/seqlock-Unbreak-lockdep.patch b/patches/seqlock-Unbreak-lockdep.patch new file mode 100644 index 000000000000..9a5ec6509764 --- /dev/null +++ b/patches/seqlock-Unbreak-lockdep.patch @@ -0,0 +1,68 @@ +From: "peterz@infradead.org" <peterz@infradead.org> +Date: Tue, 15 Sep 2020 16:30:28 +0200 +Subject: [PATCH] seqlock: Unbreak lockdep + +Upstream commit 267580db047ef428a70bef8287ca62c5a450c139 + +seqcount_LOCKNAME_init() needs to be a macro due to the lockdep +annotation in seqcount_init(). Since a macro cannot define another +macro, we need to effectively revert commit: e4e9ab3f9f91 ("seqlock: +Fold seqcount_LOCKNAME_init() definition"). + +Fixes: e4e9ab3f9f91 ("seqlock: Fold seqcount_LOCKNAME_init() definition") +Reported-by: Qian Cai <cai@redhat.com> +Debugged-by: Boqun Feng <boqun.feng@gmail.com> +Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> +Tested-by: Qian Cai <cai@redhat.com> +Link: https://lkml.kernel.org/r/20200915143028.GB2674@hirez.programming.kicks-ass.net +Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> +--- + include/linux/seqlock.h | 22 ++++++++++++++-------- + 1 file changed, 14 insertions(+), 8 deletions(-) + +--- a/include/linux/seqlock.h ++++ b/include/linux/seqlock.h +@@ -173,6 +173,19 @@ static inline void seqcount_lockdep_read + * @lock: Pointer to the associated lock + */ + ++#define seqcount_LOCKNAME_init(s, _lock, lockname) \ ++ do { \ ++ seqcount_##lockname##_t *____s = (s); \ ++ seqcount_init(&____s->seqcount); \ ++ __SEQ_LOCK(____s->lock = (_lock)); \ ++ } while (0) ++ ++#define seqcount_raw_spinlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, raw_spinlock) ++#define seqcount_spinlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, spinlock) ++#define seqcount_rwlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, rwlock); ++#define seqcount_mutex_init(s, lock) seqcount_LOCKNAME_init(s, lock, mutex); ++#define seqcount_ww_mutex_init(s, lock) seqcount_LOCKNAME_init(s, lock, ww_mutex); ++ + /* + * SEQCOUNT_LOCKNAME() - Instantiate seqcount_LOCKNAME_t and helpers + * seqprop_LOCKNAME_*() - Property accessors for seqcount_LOCKNAME_t +@@ -190,13 +203,6 @@ typedef struct seqcount_##lockname { + __SEQ_LOCK(locktype *lock); \ + } seqcount_##lockname##_t; \ + \ +-static __always_inline void \ +-seqcount_##lockname##_init(seqcount_##lockname##_t *s, locktype *lock) \ +-{ \ +- seqcount_init(&s->seqcount); \ +- __SEQ_LOCK(s->lock = lock); \ +-} \ +- \ + static __always_inline seqcount_t * \ + __seqprop_##lockname##_ptr(seqcount_##lockname##_t *s) \ + { \ +@@ -284,8 +290,8 @@ SEQCOUNT_LOCKNAME(ww_mutex, struct w + __SEQ_LOCK(.lock = (assoc_lock)) \ + } + +-#define SEQCNT_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) + #define SEQCNT_RAW_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) ++#define SEQCNT_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) + #define SEQCNT_RWLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) + #define SEQCNT_MUTEX_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) + #define SEQCNT_WW_MUTEX_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock) diff --git a/patches/series b/patches/series index aa5087908836..4a85452aa9e4 100644 --- a/patches/series +++ b/patches/series @@ -42,8 +42,8 @@ # https://lkml.kernel.org/r/20200828100022.1099682-2-npiggin@gmail.com mm-fix-exec-activate_mm-vs-TLB-shootdown-and-lazy-tl.patch -# 2020-10-15 13:05 Peter Zijlstra [PATCH v3 00/19] sched: Migrate disable support -# 20201015110532.738127234@infradead.org +# 2020-10-23 12:11 Peter Zijlstra [PATCH v4 00/19] sched: Migrate disable support +# 20201023101158.088940906@infradead.org 0001-stop_machine-Add-function-and-caller-debug-info.patch 0002-sched-Fix-balance_callback.patch 0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch @@ -79,6 +79,9 @@ io_wq-Make-io_wqe-lock-a-raw_spinlock_t.patch # 20200915074816.52zphpywj4zidspk@linutronix.de bus-mhi-Remove-include-of-rwlock_types.h.patch +# 20201023110400.bx3uzsb7xy5jtsea@linutronix.de +blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch + ############################################################ # Ready for posting ############################################################ @@ -98,7 +101,6 @@ bus-mhi-Remove-include-of-rwlock_types.h.patch 0012-printk-move-console-printing-to-kthreads.patch 0013-printk-remove-deferred-printing.patch 0014-printk-add-console-handover.patch -0015-printk-add-pr_flush.patch printk-Tiny-cleanup.patch ############################################################ @@ -144,7 +146,6 @@ tasklets-Use-static-line-for-functions.patch 0004-locking-rtmutex-Remove-rt_mutex_timed_lock.patch 0005-locking-rtmutex-Handle-the-various-new-futex-race-co.patch 0006-futex-Fix-bug-on-when-a-requeued-RT-task-times-out.patch -0007-locking-rtmutex-Add-rtmutex_lock_killable.patch 0008-locking-rtmutex-Make-lock_killable-work.patch 0009-locking-spinlock-Split-the-lock-types-header.patch 0010-locking-rtmutex-Avoid-include-hell.patch @@ -199,6 +200,8 @@ seqlock-Fix-multiple-kernel-doc-warnings.patch 0011-seqlock-seqcount_t-Implement-all-read-APIs-as-statem.patch 0012-seqlock-seqcount_LOCKNAME_t-Introduce-PREEMPT_RT-sup.patch 0013-seqlock-PREEMPT_RT-Do-not-starve-seqlock_t-writers.patch +# 267580db047ef428a70bef8287ca62c5a450c139 +seqlock-Unbreak-lockdep.patch ## 0024-xfrm-Use-sequence-counter-with-associated-spinlock.patch u64_stats-Disable-preemption-on-32bit-UP-SMP-with-RT.patch |