summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSebastian Andrzej Siewior <bigeasy@linutronix.de>2020-10-24 00:33:30 +0200
committerSebastian Andrzej Siewior <bigeasy@linutronix.de>2020-10-24 00:33:30 +0200
commite8077fadb604e13404de3e31ed2d674b8fca3e5c (patch)
tree0f357d6f21661a6e638a6012e70cd2c77523c135
parenta17bee0e4461c412981e4bcae695c2a51b6b231d (diff)
downloadlinux-rt-e8077fadb604e13404de3e31ed2d674b8fca3e5c.tar.gz
[ANNOUNCE] v5.9.1-rt19v5.9.1-rt19-patches
Dear RT folks! I'm pleased to announce the v5.9.1-rt19 patch set. Changes since v5.9.1-rt18: - Mike Galbraith reported a possible circular locking dependency with a seqcount. Backported a patch from upstream solving the issue. - David Runge reported a crash in the block layer. - The migrate-disable series by Peter Zijlstra has been update to v4. - Mike Galbraith a possible possible hang due to the new printk code. Dropped a patch from the printk code which was causing the problem as suggested by John Ogness. - The rtmutex clean up in v5.9-rc8-rt12 restructured the code path and removed the blk_schedule_flush_plug() invocation from the locking path. It turns out that it is still required and has been added back. - A small rtmutex related clean up: - Remove rt_mutex_lock_killable(), it has no users. - Use _mutex_lock_io_nested() for _mutex_lock_io() to avoid duplicated code. Known issues - It has been pointed out that due to changes to the printk code the internal buffer representation changed. This is only an issue if tools like `crash' are used to extract the printk buffer from a kernel memory image. The delta patch against v5.9.1-rt18 is appended below and can be found here: https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/incr/patch-5.9.1-rt18-rt19.patch.xz You can get this release via the git tree at: git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git v5.9.1-rt19 The RT patch against v5.9.1 can be found here: https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/older/patch-5.9.1-rt19.patch.xz The split quilt queue is available at: https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.9/older/patches-5.9.1-rt19.tar.xz Sebastian Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
-rw-r--r--patches/0001-stop_machine-Add-function-and-caller-debug-info.patch2
-rw-r--r--patches/0002-sched-Fix-balance_callback.patch2
-rw-r--r--patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch2
-rw-r--r--patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch2
-rw-r--r--patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch2
-rw-r--r--patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch2
-rw-r--r--patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch49
-rw-r--r--patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch2
-rw-r--r--patches/0008-sched-Massage-set_cpus_allowed.patch2
-rw-r--r--patches/0009-sched-Add-migrate_disable.patch2
-rw-r--r--patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch167
-rw-r--r--patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch10
-rw-r--r--patches/0012-sched-rt-Use-cpumask_any-_distribute.patch2
-rw-r--r--patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch2
-rw-r--r--patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch19
-rw-r--r--patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch6
-rw-r--r--patches/0015-printk-add-pr_flush.patch96
-rw-r--r--patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch6
-rw-r--r--patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch18
-rw-r--r--patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch14
-rw-r--r--patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch2
-rw-r--r--patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch2
-rw-r--r--patches/0017-sched-Add-migrate_disable-tracepoints.patch2
-rw-r--r--patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch63
-rw-r--r--patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch4
-rw-r--r--patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch21
-rw-r--r--patches/0019-sched-Comment-affine_move_task.patch16
-rw-r--r--patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch6
-rw-r--r--patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch20
-rw-r--r--patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch37
-rw-r--r--patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch2
-rw-r--r--patches/localversion.patch2
-rw-r--r--patches/preempt-lazy-support.patch12
-rw-r--r--patches/printk-Tiny-cleanup.patch6
-rw-r--r--patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch6
-rw-r--r--patches/sched-might-sleep-do-not-account-rcu-depth.patch2
-rw-r--r--patches/sched-mmdrop-delayed.patch4
-rw-r--r--patches/seqlock-Unbreak-lockdep.patch68
-rw-r--r--patches/series11
39 files changed, 379 insertions, 314 deletions
diff --git a/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch b/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch
index a63db738db6e..8c337cede0f9 100644
--- a/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch
+++ b/patches/0001-stop_machine-Add-function-and-caller-debug-info.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Mon, 21 Sep 2020 12:58:17 +0200
+Date: Fri, 23 Oct 2020 12:11:59 +0200
Subject: [PATCH 01/19] stop_machine: Add function and caller debug info
Crashes in stop-machine are hard to connect to the calling code, add a
diff --git a/patches/0002-sched-Fix-balance_callback.patch b/patches/0002-sched-Fix-balance_callback.patch
index 6254ec3ce16f..61e82a06f432 100644
--- a/patches/0002-sched-Fix-balance_callback.patch
+++ b/patches/0002-sched-Fix-balance_callback.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Mon, 11 May 2020 14:13:00 +0200
+Date: Fri, 23 Oct 2020 12:12:00 +0200
Subject: [PATCH 02/19] sched: Fix balance_callback()
The intent of balance_callback() has always been to delay executing
diff --git a/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch b/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch
index 3efaa2838aa4..6e3ba750fb66 100644
--- a/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch
+++ b/patches/0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Fri, 11 Sep 2020 09:54:27 +0200
+Date: Fri, 23 Oct 2020 12:12:01 +0200
Subject: [PATCH 03/19] sched/hotplug: Ensure only per-cpu kthreads run during
hotplug
diff --git a/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch b/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch
index 3bb9eee23e6d..25f1d4c5c561 100644
--- a/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch
+++ b/patches/0004-sched-core-Wait-for-tasks-being-pushed-away-on-hotpl.patch
@@ -1,5 +1,5 @@
From: Thomas Gleixner <tglx@linutronix.de>
-Date: Mon, 14 Sep 2020 14:47:28 +0200
+Date: Fri, 23 Oct 2020 12:12:02 +0200
Subject: [PATCH 04/19] sched/core: Wait for tasks being pushed away on hotplug
RT kernels need to ensure that all tasks which are not per CPU kthreads
diff --git a/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch b/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch
index e1b826d0b49b..60c3800a80de 100644
--- a/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch
+++ b/patches/0005-workqueue-Manually-break-affinity-on-hotplug.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Fri, 25 Sep 2020 15:45:11 +0200
+Date: Fri, 23 Oct 2020 12:12:03 +0200
Subject: [PATCH 05/19] workqueue: Manually break affinity on hotplug
Don't rely on the scheduler to force break affinity for us -- it will
diff --git a/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch b/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch
index 4988b287dd9f..c5d20e3a4ac7 100644
--- a/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch
+++ b/patches/0006-sched-hotplug-Consolidate-task-migration-on-CPU-unpl.patch
@@ -1,5 +1,5 @@
From: Thomas Gleixner <tglx@linutronix.de>
-Date: Wed, 16 Sep 2020 09:27:18 +0200
+Date: Fri, 23 Oct 2020 12:12:04 +0200
Subject: [PATCH 06/19] sched/hotplug: Consolidate task migration on CPU unplug
With the new mechanism which kicks tasks off the outgoing CPU at the end of
diff --git a/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch b/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch
deleted file mode 100644
index dd8bc592bdc4..000000000000
--- a/patches/0007-locking-rtmutex-Add-rtmutex_lock_killable.patch
+++ /dev/null
@@ -1,49 +0,0 @@
-From: Thomas Gleixner <tglx@linutronix.de>
-Date: Thu, 9 Jun 2011 11:43:52 +0200
-Subject: [PATCH 07/23] locking/rtmutex: Add rtmutex_lock_killable()
-
-Add "killable" type to rtmutex. We need this since rtmutex are used as
-"normal" mutexes which do use this type.
-
-Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
----
- include/linux/rtmutex.h | 1 +
- kernel/locking/rtmutex.c | 17 +++++++++++++++++
- 2 files changed, 18 insertions(+)
-
---- a/include/linux/rtmutex.h
-+++ b/include/linux/rtmutex.h
-@@ -99,6 +99,7 @@ extern void rt_mutex_lock(struct rt_mute
- #endif
-
- extern int rt_mutex_lock_interruptible(struct rt_mutex *lock);
-+extern int rt_mutex_lock_killable(struct rt_mutex *lock);
- extern int rt_mutex_trylock(struct rt_mutex *lock);
-
- extern void rt_mutex_unlock(struct rt_mutex *lock);
---- a/kernel/locking/rtmutex.c
-+++ b/kernel/locking/rtmutex.c
-@@ -1538,6 +1538,23 @@ int __sched __rt_mutex_futex_trylock(str
- }
-
- /**
-+ * rt_mutex_lock_killable - lock a rt_mutex killable
-+ *
-+ * @lock: the rt_mutex to be locked
-+ *
-+ * Returns:
-+ * 0 on success
-+ * -EINTR when interrupted by a signal
-+ */
-+int __sched rt_mutex_lock_killable(struct rt_mutex *lock)
-+{
-+ might_sleep();
-+
-+ return rt_mutex_fastlock(lock, TASK_KILLABLE, rt_mutex_slowlock);
-+}
-+EXPORT_SYMBOL_GPL(rt_mutex_lock_killable);
-+
-+/**
- * rt_mutex_trylock - try to lock a rt_mutex
- *
- * @lock: the rt_mutex to be locked
diff --git a/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch b/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch
index 40a091e57c7a..e26dfb31fbc0 100644
--- a/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch
+++ b/patches/0007-sched-Fix-hotplug-vs-CPU-bandwidth-control.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Fri, 25 Sep 2020 16:42:31 +0200
+Date: Fri, 23 Oct 2020 12:12:05 +0200
Subject: [PATCH 07/19] sched: Fix hotplug vs CPU bandwidth control
Since we now migrate tasks away before DYING, we should also move
diff --git a/patches/0008-sched-Massage-set_cpus_allowed.patch b/patches/0008-sched-Massage-set_cpus_allowed.patch
index f9dd57152f4d..3c79a4c11ade 100644
--- a/patches/0008-sched-Massage-set_cpus_allowed.patch
+++ b/patches/0008-sched-Massage-set_cpus_allowed.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Wed, 16 Sep 2020 14:59:08 +0200
+Date: Fri, 23 Oct 2020 12:12:06 +0200
Subject: [PATCH 08/19] sched: Massage set_cpus_allowed()
Thread a u32 flags word through the *set_cpus_allowed*() callchain.
diff --git a/patches/0009-sched-Add-migrate_disable.patch b/patches/0009-sched-Add-migrate_disable.patch
index d8d641c2a456..3f736a398e02 100644
--- a/patches/0009-sched-Add-migrate_disable.patch
+++ b/patches/0009-sched-Add-migrate_disable.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Thu, 17 Sep 2020 10:38:30 +0200
+Date: Fri, 23 Oct 2020 12:12:07 +0200
Subject: [PATCH 09/19] sched: Add migrate_disable()
Add the base migrate_disable() support (under protest).
diff --git a/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch b/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch
index 8a84be8f2b31..7511b4b4fc7d 100644
--- a/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch
+++ b/patches/0010-sched-Fix-migrate_disable-vs-set_cpus_allowed_ptr.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Fri, 18 Sep 2020 17:24:31 +0200
+Date: Fri, 23 Oct 2020 12:12:08 +0200
Subject: [PATCH 10/19] sched: Fix migrate_disable() vs set_cpus_allowed_ptr()
Concurrent migrate_disable() and set_cpus_allowed_ptr() has
@@ -30,12 +30,15 @@ that is exactly the case where there is a pending
set_cpus_allowed_ptr(), so let that provide storage for the async stop
machine.
+Much thanks to Valentin who used TLA+ most effective and found lots of
+'interesting' cases.
+
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/sched.h | 1
- kernel/sched/core.c | 162 ++++++++++++++++++++++++++++++++++++++++++--------
- 2 files changed, 140 insertions(+), 23 deletions(-)
+ kernel/sched/core.c | 234 +++++++++++++++++++++++++++++++++++++++++++-------
+ 2 files changed, 205 insertions(+), 30 deletions(-)
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -82,67 +85,143 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
}
EXPORT_SYMBOL_GPL(migrate_enable);
-@@ -1811,6 +1822,7 @@ static struct rq *move_queued_task(struc
+@@ -1809,8 +1820,16 @@ static struct rq *move_queued_task(struc
+ }
+
struct migration_arg {
- struct task_struct *task;
- int dest_cpu;
-+ struct completion *done;
+- struct task_struct *task;
+- int dest_cpu;
++ struct task_struct *task;
++ int dest_cpu;
++ struct set_affinity_pending *pending;
++};
++
++struct set_affinity_pending {
++ refcount_t refs;
++ struct completion done;
++ struct cpu_stop_work stop_work;
++ struct migration_arg arg;
};
/*
-@@ -1845,6 +1857,7 @@ static int migration_cpu_stop(void *data
+@@ -1842,16 +1861,19 @@ static struct rq *__migrate_task(struct
+ */
+ static int migration_cpu_stop(void *data)
+ {
++ struct set_affinity_pending *pending;
struct migration_arg *arg = data;
struct task_struct *p = arg->task;
++ int dest_cpu = arg->dest_cpu;
struct rq *rq = this_rq();
+ bool complete = false;
struct rq_flags rf;
/*
-@@ -1867,15 +1880,27 @@ static int migration_cpu_stop(void *data
+ * The original target CPU might have gone down and we might
+ * be on another CPU but it doesn't matter.
+ */
+- local_irq_disable();
++ local_irq_save(rf.flags);
+ /*
+ * We need to explicitly wake pending tasks before running
+ * __migrate_task() such that we will not miss enforcing cpus_ptr
+@@ -1861,21 +1883,83 @@ static int migration_cpu_stop(void *data
+
+ raw_spin_lock(&p->pi_lock);
+ rq_lock(rq, &rf);
++
++ pending = p->migration_pending;
+ /*
+ * If task_rq(p) != rq, it cannot be migrated here, because we're
+ * holding rq->lock, if p->on_rq == 0 it cannot get enqueued because
* we're holding p->pi_lock.
*/
if (task_rq(p) == rq) {
+ if (is_migration_disabled(p))
+ goto out;
+
++ if (pending) {
++ p->migration_pending = NULL;
++ complete = true;
++ }
++
++ /* migrate_enable() -- we must not race against SCA */
++ if (dest_cpu < 0) {
++ /*
++ * When this was migrate_enable() but we no longer
++ * have a @pending, a concurrent SCA 'fixed' things
++ * and we should be valid again. Nothing to do.
++ */
++ if (!pending) {
++ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq)));
++ goto out;
++ }
++
++ dest_cpu = cpumask_any_distribute(&p->cpus_mask);
++ }
++
if (task_on_rq_queued(p))
- rq = __migrate_task(rq, &rf, p, arg->dest_cpu);
+- rq = __migrate_task(rq, &rf, p, arg->dest_cpu);
++ rq = __migrate_task(rq, &rf, p, dest_cpu);
else
- p->wake_cpu = arg->dest_cpu;
+- p->wake_cpu = arg->dest_cpu;
++ p->wake_cpu = dest_cpu;
+
-+ if (arg->done) {
-+ p->migration_pending = NULL;
-+ complete = true;
++ } else if (dest_cpu < 0) {
++ /*
++ * This happens when we get migrated between migrate_enable()'s
++ * preempt_enable() and scheduling the stopper task. At that
++ * point we're a regular task again and not current anymore.
++ *
++ * A !PREEMPT kernel has a giant hole here, which makes it far
++ * more likely.
++ */
++
++ /*
++ * When this was migrate_enable() but we no longer have an
++ * @pending, a concurrent SCA 'fixed' things and we should be
++ * valid again. Nothing to do.
++ */
++ if (!pending) {
++ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq)));
++ goto out;
+ }
++
++ /*
++ * When migrate_enable() hits a rq mis-match we can't reliably
++ * determine is_migration_disabled() and so have to chase after
++ * it.
++ */
++ task_rq_unlock(rq, p, &rf);
++ stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop,
++ &pending->arg, &pending->stop_work);
++ return 0;
}
+- rq_unlock(rq, &rf);
+- raw_spin_unlock(&p->pi_lock);
+out:
- rq_unlock(rq, &rf);
- raw_spin_unlock(&p->pi_lock);
--
- local_irq_enable();
++ task_rq_unlock(rq, p, &rf);
+
+ if (complete)
-+ complete_all(arg->done);
++ complete_all(&pending->done);
+
++ /* For pending->{arg,stop_work} */
++ pending = arg->pending;
++ if (pending && refcount_dec_and_test(&pending->refs))
++ wake_up_var(&pending->refs);
+
+- local_irq_enable();
return 0;
}
-@@ -1944,6 +1969,112 @@ void do_set_cpus_allowed(struct task_str
- __do_set_cpus_allowed(p, new_mask, 0);
+@@ -1945,6 +2029,110 @@ void do_set_cpus_allowed(struct task_str
}
-+struct set_affinity_pending {
-+ refcount_t refs;
-+ struct completion done;
-+ struct cpu_stop_work stop_work;
-+ struct migration_arg arg;
-+};
-+
-+/*
+ /*
+ * This function is wildly self concurrent, consider at least 3 times.
+ */
-+static int affine_move_task(struct rq *rq, struct rq_flags *rf,
-+ struct task_struct *p, int dest_cpu, unsigned int flags)
++static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
++ int dest_cpu, unsigned int flags)
+{
+ struct set_affinity_pending my_pending = { }, *pending = NULL;
+ struct migration_arg arg = {
@@ -194,12 +273,17 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ if (WARN_ON_ONCE(!pending))
+ return -EINVAL;
+
-+ arg.done = &pending->done;
-+
+ if (flags & SCA_MIGRATE_ENABLE) {
+
++ refcount_inc(&pending->refs); /* pending->{arg,stop_work} */
+ task_rq_unlock(rq, p, rf);
-+ pending->arg = arg;
++
++ pending->arg = (struct migration_arg) {
++ .task = p,
++ .dest_cpu = -1,
++ .pending = pending,
++ };
++
+ stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop,
+ &pending->arg, &pending->stop_work);
+
@@ -237,17 +321,18 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ return 0;
+}
+
- /*
++/*
* Change a given task's CPU affinity. Migrate the thread to a
* proper CPU and schedule it away if the CPU it's executing on
-@@ -2013,23 +2144,8 @@ static int __set_cpus_allowed_ptr(struct
+ * is removed from the allowed bitmask.
+@@ -2013,23 +2201,8 @@ static int __set_cpus_allowed_ptr(struct
p->nr_cpus_allowed != 1);
}
- /* Can the task run on the task's current CPU? If so, we're done */
- if (cpumask_test_cpu(task_cpu(p), new_mask))
- goto out;
-+ return affine_move_task(rq, &rf, p, dest_cpu, flags);
++ return affine_move_task(rq, p, &rf, dest_cpu, flags);
- if (task_running(rq, p) || p->state == TASK_WAKING) {
- struct migration_arg arg = { p, dest_cpu };
@@ -265,3 +350,11 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
out:
task_rq_unlock(rq, p, &rf);
+@@ -3209,6 +3382,7 @@ static void __sched_fork(unsigned long c
+ init_numa_balancing(clone_flags, p);
+ #ifdef CONFIG_SMP
+ p->wake_entry.u_flags = CSD_TYPE_TTWU;
++ p->migration_pending = NULL;
+ #endif
+ }
+
diff --git a/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch b/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch
index 30854aa465b9..6f4bcbc0f964 100644
--- a/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch
+++ b/patches/0011-sched-core-Make-migrate-disable-and-CPU-hotplug-coop.patch
@@ -1,5 +1,5 @@
From: Thomas Gleixner <tglx@linutronix.de>
-Date: Wed, 26 Aug 2020 14:08:10 +0200
+Date: Fri, 23 Oct 2020 12:12:09 +0200
Subject: [PATCH 11/19] sched/core: Make migrate disable and CPU hotplug
cooperative
@@ -59,7 +59,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
#endif
/*
-@@ -2638,6 +2651,11 @@ static inline bool is_migration_disabled
+@@ -2695,6 +2708,11 @@ static inline bool is_migration_disabled
return false;
}
@@ -71,7 +71,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
#endif
static void
-@@ -7006,15 +7024,20 @@ static void balance_push(struct rq *rq)
+@@ -7064,15 +7082,20 @@ static void balance_push(struct rq *rq)
* Both the cpu-hotplug and stop task are in this case and are
* required to complete the hotplug process.
*/
@@ -94,7 +94,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
raw_spin_unlock(&rq->lock);
rcuwait_wake_up(&rq->hotplug_wait);
raw_spin_lock(&rq->lock);
-@@ -7061,7 +7084,8 @@ static void balance_hotplug_wait(void)
+@@ -7119,7 +7142,8 @@ static void balance_hotplug_wait(void)
{
struct rq *rq = this_rq();
@@ -104,7 +104,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
TASK_UNINTERRUPTIBLE);
}
-@@ -7306,7 +7330,7 @@ int sched_cpu_dying(unsigned int cpu)
+@@ -7364,7 +7388,7 @@ int sched_cpu_dying(unsigned int cpu)
sched_tick_stop(cpu);
rq_lock_irqsave(rq, &rf);
diff --git a/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch b/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch
index da82575dbd0e..e9a57312e8cb 100644
--- a/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch
+++ b/patches/0012-sched-rt-Use-cpumask_any-_distribute.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Thu, 1 Oct 2020 15:54:14 +0200
+Date: Fri, 23 Oct 2020 12:12:10 +0200
Subject: [PATCH 12/19] sched,rt: Use cpumask_any*_distribute()
Replace a bunch of cpumask_any*() instances with
diff --git a/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch b/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch
index 128aa4e97b69..c1f377bad590 100644
--- a/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch
+++ b/patches/0013-sched-rt-Use-the-full-cpumask-for-balancing.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Thu, 1 Oct 2020 16:05:39 +0200
+Date: Fri, 23 Oct 2020 12:12:11 +0200
Subject: [PATCH 13/19] sched,rt: Use the full cpumask for balancing
We want migrate_disable() tasks to get PULLs in order for them to PUSH
diff --git a/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch b/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch
index 2b2e6dac8e99..da06ec23a1bd 100644
--- a/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch
+++ b/patches/0014-locking-rtmutex-export-lockdep-less-version-of-rt_mu.patch
@@ -8,9 +8,9 @@ Required for lock implementation ontop of rtmutex.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
- kernel/locking/rtmutex.c | 58 ++++++++++++++++++++++++++--------------
+ kernel/locking/rtmutex.c | 54 ++++++++++++++++++++++++++++------------
kernel/locking/rtmutex_common.h | 3 ++
- 2 files changed, 42 insertions(+), 19 deletions(-)
+ 2 files changed, 41 insertions(+), 16 deletions(-)
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -68,16 +68,9 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
}
EXPORT_SYMBOL_GPL(rt_mutex_lock_interruptible);
-@@ -1552,12 +1564,18 @@ int __sched __rt_mutex_futex_trylock(str
- */
- int __sched rt_mutex_lock_killable(struct rt_mutex *lock)
- {
-- might_sleep();
--
-- return rt_mutex_fastlock(lock, TASK_KILLABLE, rt_mutex_slowlock);
-+ return rt_mutex_lock_state(lock, 0, TASK_KILLABLE);
+@@ -1541,6 +1553,14 @@ int __sched __rt_mutex_futex_trylock(str
+ return __rt_mutex_slowtrylock(lock);
}
- EXPORT_SYMBOL_GPL(rt_mutex_lock_killable);
+int __sched __rt_mutex_trylock(struct rt_mutex *lock)
+{
@@ -90,7 +83,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/**
* rt_mutex_trylock - try to lock a rt_mutex
*
-@@ -1573,10 +1591,7 @@ int __sched rt_mutex_trylock(struct rt_m
+@@ -1556,10 +1576,7 @@ int __sched rt_mutex_trylock(struct rt_m
{
int ret;
@@ -102,7 +95,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
if (ret)
mutex_acquire(&lock->dep_map, 0, 1, _RET_IP_);
-@@ -1584,6 +1599,11 @@ int __sched rt_mutex_trylock(struct rt_m
+@@ -1567,6 +1584,11 @@ int __sched rt_mutex_trylock(struct rt_m
}
EXPORT_SYMBOL_GPL(rt_mutex_trylock);
diff --git a/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch b/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch
index 763ba836d58c..5401e0670dd4 100644
--- a/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch
+++ b/patches/0014-sched-lockdep-Annotate-pi_lock-recursion.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Thu, 1 Oct 2020 16:13:01 +0200
+Date: Fri, 23 Oct 2020 12:12:12 +0200
Subject: [PATCH 14/19] sched, lockdep: Annotate ->pi_lock recursion
There's a valid ->pi_lock recursion issue where the actual PI code
@@ -14,7 +14,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -2603,6 +2603,7 @@ int select_task_rq(struct task_struct *p
+@@ -2660,6 +2660,7 @@ int select_task_rq(struct task_struct *p
void sched_set_stop_task(int cpu, struct task_struct *stop)
{
@@ -22,7 +22,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
struct sched_param param = { .sched_priority = MAX_RT_PRIO - 1 };
struct task_struct *old_stop = cpu_rq(cpu)->stop;
-@@ -2618,6 +2619,20 @@ void sched_set_stop_task(int cpu, struct
+@@ -2675,6 +2676,20 @@ void sched_set_stop_task(int cpu, struct
sched_setscheduler_nocheck(stop, SCHED_FIFO, &param);
stop->sched_class = &stop_sched_class;
diff --git a/patches/0015-printk-add-pr_flush.patch b/patches/0015-printk-add-pr_flush.patch
deleted file mode 100644
index 00aaed1b31ff..000000000000
--- a/patches/0015-printk-add-pr_flush.patch
+++ /dev/null
@@ -1,96 +0,0 @@
-From: John Ogness <john.ogness@linutronix.de>
-Date: Mon, 19 Oct 2020 23:14:40 +0206
-Subject: [PATCH 15/15] printk: add pr_flush()
-
-Add a function to allow a caller to wait until all printing threads
-have caught up. A timeout can be set to limit the waiting. The
-timeout is reset if forward progress is detected.
-
-Signed-off-by: John Ogness <john.ogness@linutronix.de>
-Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
----
- kernel/printk/printk.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++
- 1 file changed, 66 insertions(+)
-
---- a/kernel/printk/printk.c
-+++ b/kernel/printk/printk.c
-@@ -3165,6 +3165,66 @@ const char *kmsg_dump_reason_str(enum km
- EXPORT_SYMBOL_GPL(kmsg_dump_reason_str);
-
- /**
-+ * pr_flush() - Wait for printing threads to catch up.
-+ *
-+ * @timeout_ms: The maximum time (in ms) to wait.
-+ * @reset_on_progress: Reset the timeout if forward progress is seen.
-+ *
-+ * A value of 0 for @timeout_ms means no waiting will occur. A value of -1
-+ * represents infinite waiting.
-+ *
-+ * If @reset_on_progress is true, the timeout will be reset whenever any
-+ * printer has been seen to make some forward progress.
-+ *
-+ * Context: Any context if @timeout_ms is 0. Otherwise process context and
-+ * may sleep if a printer is not caught up.
-+ * Return: true if all enabled printers are caught up.
-+ */
-+static bool pr_flush(int timeout_ms, bool reset_on_progress)
-+{
-+ int remaining = timeout_ms;
-+ struct console *con;
-+ u64 last_diff = 0;
-+ u64 printk_seq;
-+ u64 diff;
-+ u64 seq;
-+
-+ seq = prb_next_seq(prb);
-+
-+ for (;;) {
-+ diff = 0;
-+
-+ for_each_console(con) {
-+ if (!(con->flags & CON_ENABLED))
-+ continue;
-+ printk_seq = atomic64_read(&con->printk_seq);
-+ if (printk_seq < seq)
-+ diff += seq - printk_seq;
-+ }
-+
-+ if (diff != last_diff && reset_on_progress)
-+ remaining = timeout_ms;
-+
-+ if (!diff || remaining == 0)
-+ break;
-+
-+ if (remaining < 0) {
-+ msleep(100);
-+ } else if (remaining < 100) {
-+ msleep(remaining);
-+ remaining = 0;
-+ } else {
-+ msleep(100);
-+ remaining -= 100;
-+ }
-+
-+ last_diff = diff;
-+ }
-+
-+ return (diff == 0);
-+}
-+
-+/**
- * kmsg_dump - dump kernel log to kernel message dumpers.
- * @reason: the reason (oops, panic etc) for dumping
- *
-@@ -3186,6 +3246,12 @@ void kmsg_dump(enum kmsg_dump_reason rea
- sync_mode = true;
- pr_info("enabled sync mode\n");
- }
-+
-+ /*
-+ * Give the printing threads time to flush, allowing up to 1
-+ * second of no printing forward progress before giving up.
-+ */
-+ pr_flush(1000, true);
- }
-
- rcu_read_lock();
diff --git a/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch b/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch
index 5b3a2108f981..20af7d2fe22b 100644
--- a/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch
+++ b/patches/0015-sched-Add-saved_state-for-tasks-blocked-on-sleeping-.patch
@@ -37,7 +37,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
#ifdef CONFIG_SMP
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -3221,7 +3221,7 @@ try_to_wake_up(struct task_struct *p, un
+@@ -3278,7 +3278,7 @@ try_to_wake_up(struct task_struct *p, un
int cpu, success = 0;
preempt_disable();
@@ -46,7 +46,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
/*
* We're waking current, this means 'p->on_rq' and 'task_cpu(p)
* == smp_processor_id()'. Together this means we can special
-@@ -3251,8 +3251,26 @@ try_to_wake_up(struct task_struct *p, un
+@@ -3308,8 +3308,26 @@ try_to_wake_up(struct task_struct *p, un
*/
raw_spin_lock_irqsave(&p->pi_lock, flags);
smp_mb__after_spinlock();
@@ -74,7 +74,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
trace_sched_waking(p);
-@@ -3442,6 +3460,18 @@ int wake_up_process(struct task_struct *
+@@ -3499,6 +3517,18 @@ int wake_up_process(struct task_struct *
}
EXPORT_SYMBOL(wake_up_process);
diff --git a/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch b/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch
index 80feca872798..7d44d63cb595 100644
--- a/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch
+++ b/patches/0015-sched-Fix-migrate_disable-vs-rt-dl-balancing.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Mon, 28 Sep 2020 17:06:07 +0200
+Date: Fri, 23 Oct 2020 12:12:13 +0200
Subject: [PATCH 15/19] sched: Fix migrate_disable() vs rt/dl balancing
In order to minimize the interference of migrate_disable() on lower
@@ -107,7 +107,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
static inline bool rq_has_pinned_tasks(struct rq *rq)
{
return rq->nr_pinned;
-@@ -1917,6 +1912,49 @@ static int migration_cpu_stop(void *data
+@@ -1976,6 +1971,49 @@ static int migration_cpu_stop(void *data
return 0;
}
@@ -157,7 +157,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/*
* sched_class::set_cpus_allowed must do the below, but is not required to
* actually call this function.
-@@ -2004,6 +2042,14 @@ static int affine_move_task(struct rq *r
+@@ -2056,6 +2094,14 @@ static int affine_move_task(struct rq *r
/* Can the task run on the task's current CPU? If so, we're done */
if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) {
@@ -172,7 +172,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
pending = p->migration_pending;
if (pending) {
refcount_inc(&pending->refs);
-@@ -2012,6 +2058,11 @@ static int affine_move_task(struct rq *r
+@@ -2064,6 +2110,11 @@ static int affine_move_task(struct rq *r
}
task_rq_unlock(rq, p, rf);
@@ -184,15 +184,15 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
if (complete)
goto do_complete;
-@@ -2049,6 +2100,7 @@ static int affine_move_task(struct rq *r
-
+@@ -2100,6 +2151,7 @@ static int affine_move_task(struct rq *r
if (flags & SCA_MIGRATE_ENABLE) {
+ refcount_inc(&pending->refs); /* pending->{arg,stop_work} */
+ p->migration_flags &= ~MDF_PUSH;
task_rq_unlock(rq, p, rf);
- pending->arg = arg;
- stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop,
-@@ -2661,11 +2713,6 @@ static inline int __set_cpus_allowed_ptr
+
+ pending->arg = (struct migration_arg) {
+@@ -2718,11 +2770,6 @@ static inline int __set_cpus_allowed_ptr
static inline void migrate_disable_switch(struct rq *rq, struct task_struct *p) { }
diff --git a/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch b/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch
index 5147454efed8..3830e36e4126 100644
--- a/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch
+++ b/patches/0016-locking-rtmutex-add-sleeping-lock-implementation.patch
@@ -936,7 +936,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
}
int __sched __rt_mutex_lock_state(struct rt_mutex *lock, int state)
-@@ -1612,16 +1941,13 @@ void __sched __rt_mutex_unlock(struct rt
+@@ -1597,16 +1926,13 @@ void __sched __rt_mutex_unlock(struct rt
void __sched rt_mutex_unlock(struct rt_mutex *lock)
{
mutex_release(&lock->dep_map, _RET_IP_);
@@ -957,7 +957,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
{
lockdep_assert_held(&lock->wait_lock);
-@@ -1638,23 +1964,35 @@ bool __sched __rt_mutex_futex_unlock(str
+@@ -1623,23 +1949,35 @@ bool __sched __rt_mutex_futex_unlock(str
* avoid inversion prior to the wakeup. preempt_disable()
* therein pairs with rt_mutex_postunlock().
*/
@@ -996,7 +996,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
}
/**
-@@ -1690,7 +2028,7 @@ void __rt_mutex_init(struct rt_mutex *lo
+@@ -1675,7 +2013,7 @@ void __rt_mutex_init(struct rt_mutex *lo
if (name && key)
debug_rt_mutex_init(lock, name, key);
}
@@ -1005,7 +1005,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/**
* rt_mutex_init_proxy_locked - initialize and lock a rt_mutex on behalf of a
-@@ -1710,6 +2048,14 @@ void rt_mutex_init_proxy_locked(struct r
+@@ -1695,6 +2033,14 @@ void rt_mutex_init_proxy_locked(struct r
struct task_struct *proxy_owner)
{
__rt_mutex_init(lock, NULL, NULL);
@@ -1020,7 +1020,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
debug_rt_mutex_proxy_lock(lock, proxy_owner);
rt_mutex_set_owner(lock, proxy_owner);
}
-@@ -1733,6 +2079,26 @@ void rt_mutex_proxy_unlock(struct rt_mut
+@@ -1718,6 +2064,26 @@ void rt_mutex_proxy_unlock(struct rt_mut
rt_mutex_set_owner(lock, NULL);
}
@@ -1047,7 +1047,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/**
* __rt_mutex_start_proxy_lock() - Start lock acquisition for another task
* @lock: the rt_mutex to take
-@@ -1805,6 +2171,9 @@ int __rt_mutex_start_proxy_lock(struct r
+@@ -1790,6 +2156,9 @@ int __rt_mutex_start_proxy_lock(struct r
ret = 0;
}
@@ -1057,7 +1057,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
return ret;
}
-@@ -1894,6 +2263,9 @@ int rt_mutex_wait_proxy_lock(struct rt_m
+@@ -1879,6 +2248,9 @@ int rt_mutex_wait_proxy_lock(struct rt_m
* have to fix that up.
*/
fixup_rt_mutex_waiters(lock);
diff --git a/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch b/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch
index 9a02d96c393c..94883b8bbfa1 100644
--- a/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch
+++ b/patches/0016-sched-proc-Print-accurate-cpumask-vs-migrate_disable.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Mon, 5 Oct 2020 12:49:16 +0200
+Date: Fri, 23 Oct 2020 12:12:14 +0200
Subject: [PATCH 16/19] sched/proc: Print accurate cpumask vs migrate_disable()
Ensure /proc/*/status doesn't print 'random' cpumasks due to
diff --git a/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch b/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch
index 420f7499bb16..cc03f0bb1b3d 100644
--- a/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch
+++ b/patches/0017-locking-rtmutex-Allow-rt_mutex_trylock-on-PREEMPT_RT.patch
@@ -15,7 +15,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
-@@ -1899,7 +1899,11 @@ EXPORT_SYMBOL_GPL(rt_mutex_lock_killable
+@@ -1884,7 +1884,11 @@ int __sched __rt_mutex_futex_trylock(str
int __sched __rt_mutex_trylock(struct rt_mutex *lock)
{
diff --git a/patches/0017-sched-Add-migrate_disable-tracepoints.patch b/patches/0017-sched-Add-migrate_disable-tracepoints.patch
index 76f0fa2f699b..016f62d84838 100644
--- a/patches/0017-sched-Add-migrate_disable-tracepoints.patch
+++ b/patches/0017-sched-Add-migrate_disable-tracepoints.patch
@@ -1,5 +1,5 @@
From: Peter Zijlstra <peterz@infradead.org>
-Date: Fri, 2 Oct 2020 20:29:46 +0200
+Date: Fri, 23 Oct 2020 12:12:15 +0200
Subject: [PATCH 17/19] sched: Add migrate_disable() tracepoints
XXX write a tracer:
diff --git a/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch b/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch
index e2887c3a6d98..21037b2580d8 100644
--- a/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch
+++ b/patches/0018-locking-rtmutex-add-mutex-implementation-based-on-rt.patch
@@ -6,15 +6,15 @@ Subject: [PATCH 18/23] locking/rtmutex: add mutex implementation based on
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
- include/linux/mutex_rt.h | 131 +++++++++++++++++++++++++++
- kernel/locking/mutex-rt.c | 222 ++++++++++++++++++++++++++++++++++++++++++++++
- 2 files changed, 353 insertions(+)
+ include/linux/mutex_rt.h | 130 ++++++++++++++++++++++++++
+ kernel/locking/mutex-rt.c | 224 ++++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 354 insertions(+)
create mode 100644 include/linux/mutex_rt.h
create mode 100644 kernel/locking/mutex-rt.c
--- /dev/null
+++ b/include/linux/mutex_rt.h
-@@ -0,0 +1,131 @@
+@@ -0,0 +1,130 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#ifndef __LINUX_MUTEX_RT_H
+#define __LINUX_MUTEX_RT_H
@@ -46,7 +46,6 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+
+extern void __mutex_do_init(struct mutex *lock, const char *name, struct lock_class_key *key);
+extern void __lockfunc _mutex_lock(struct mutex *lock);
-+extern void __lockfunc _mutex_lock_io(struct mutex *lock);
+extern void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass);
+extern int __lockfunc _mutex_lock_interruptible(struct mutex *lock);
+extern int __lockfunc _mutex_lock_killable(struct mutex *lock);
@@ -63,7 +62,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+#define mutex_lock_killable(l) _mutex_lock_killable(l)
+#define mutex_trylock(l) _mutex_trylock(l)
+#define mutex_unlock(l) _mutex_unlock(l)
-+#define mutex_lock_io(l) _mutex_lock_io(l);
++#define mutex_lock_io(l) _mutex_lock_io_nested(l, 0);
+
+#define __mutex_owner(l) ((l)->lock.owner)
+
@@ -94,7 +93,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+# define mutex_lock_killable_nested(l, s) \
+ _mutex_lock_killable(l)
+# define mutex_lock_nest_lock(lock, nest_lock) mutex_lock(lock)
-+# define mutex_lock_io_nested(l, s) _mutex_lock_io(l)
++# define mutex_lock_io_nested(l, s) _mutex_lock_io_nested(l, s)
+#endif
+
+# define mutex_init(mutex) \
@@ -148,7 +147,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+#endif
--- /dev/null
+++ b/kernel/locking/mutex-rt.c
-@@ -0,0 +1,222 @@
+@@ -0,0 +1,224 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Real-Time Preemption Support
@@ -216,6 +215,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+#include <linux/fs.h>
+#include <linux/futex.h>
+#include <linux/hrtimer.h>
++#include <linux/blkdev.h>
+
+#include "rtmutex_common.h"
+
@@ -236,29 +236,43 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+}
+EXPORT_SYMBOL(__mutex_do_init);
+
++static int _mutex_lock_blk_flush(struct mutex *lock, int state)
++{
++ /*
++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too
++ * late if one of the callbacks needs to acquire a sleeping lock.
++ */
++ if (blk_needs_flush_plug(current))
++ blk_schedule_flush_plug(current);
++ return __rt_mutex_lock_state(&lock->lock, state);
++}
++
+void __lockfunc _mutex_lock(struct mutex *lock)
+{
+ mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_);
-+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE);
++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE);
+}
+EXPORT_SYMBOL(_mutex_lock);
+
-+void __lockfunc _mutex_lock_io(struct mutex *lock)
++void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass)
+{
+ int token;
+
+ token = io_schedule_prepare();
-+ _mutex_lock(lock);
++
++ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_);
++ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE);
++
+ io_schedule_finish(token);
+}
-+EXPORT_SYMBOL_GPL(_mutex_lock_io);
++EXPORT_SYMBOL_GPL(_mutex_lock_io_nested);
+
+int __lockfunc _mutex_lock_interruptible(struct mutex *lock)
+{
+ int ret;
+
+ mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_);
-+ ret = __rt_mutex_lock_state(&lock->lock, TASK_INTERRUPTIBLE);
++ ret = _mutex_lock_blk_flush(lock, TASK_INTERRUPTIBLE);
+ if (ret)
+ mutex_release(&lock->dep_map, _RET_IP_);
+ return ret;
@@ -270,7 +284,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ int ret;
+
+ mutex_acquire(&lock->dep_map, 0, 0, _RET_IP_);
-+ ret = __rt_mutex_lock_state(&lock->lock, TASK_KILLABLE);
++ ret = _mutex_lock_blk_flush(lock, TASK_KILLABLE);
+ if (ret)
+ mutex_release(&lock->dep_map, _RET_IP_);
+ return ret;
@@ -281,27 +295,14 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+void __lockfunc _mutex_lock_nested(struct mutex *lock, int subclass)
+{
+ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_);
-+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE);
++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE);
+}
+EXPORT_SYMBOL(_mutex_lock_nested);
+
-+void __lockfunc _mutex_lock_io_nested(struct mutex *lock, int subclass)
-+{
-+ int token;
-+
-+ token = io_schedule_prepare();
-+
-+ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_);
-+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE);
-+
-+ io_schedule_finish(token);
-+}
-+EXPORT_SYMBOL_GPL(_mutex_lock_io_nested);
-+
+void __lockfunc _mutex_lock_nest_lock(struct mutex *lock, struct lockdep_map *nest)
+{
+ mutex_acquire_nest(&lock->dep_map, 0, 0, nest, _RET_IP_);
-+ __rt_mutex_lock_state(&lock->lock, TASK_UNINTERRUPTIBLE);
++ _mutex_lock_blk_flush(lock, TASK_UNINTERRUPTIBLE);
+}
+EXPORT_SYMBOL(_mutex_lock_nest_lock);
+
@@ -310,7 +311,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ int ret;
+
+ mutex_acquire_nest(&lock->dep_map, subclass, 0, NULL, _RET_IP_);
-+ ret = __rt_mutex_lock_state(&lock->lock, TASK_INTERRUPTIBLE);
++ ret = _mutex_lock_blk_flush(lock, TASK_INTERRUPTIBLE);
+ if (ret)
+ mutex_release(&lock->dep_map, _RET_IP_);
+ return ret;
@@ -322,7 +323,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ int ret;
+
+ mutex_acquire(&lock->dep_map, subclass, 0, _RET_IP_);
-+ ret = __rt_mutex_lock_state(&lock->lock, TASK_KILLABLE);
++ ret = _mutex_lock_blk_flush(lock, TASK_KILLABLE);
+ if (ret)
+ mutex_release(&lock->dep_map, _RET_IP_);
+ return ret;
diff --git a/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch b/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch
index 75b9d9525f00..12e20edee164 100644
--- a/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch
+++ b/patches/0018-sched-Deny-self-issued-__set_cpus_allowed_ptr-when-m.patch
@@ -1,5 +1,5 @@
From: Valentin Schneider <valentin.schneider@arm.com>
-Date: Tue, 13 Oct 2020 15:01:15 +0100
+Date: Fri, 23 Oct 2020 12:12:16 +0200
Subject: [PATCH 18/19] sched: Deny self-issued __set_cpus_allowed_ptr() when
migrate_disable()
@@ -17,7 +17,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -2187,8 +2187,17 @@ static int __set_cpus_allowed_ptr(struct
+@@ -2244,8 +2244,17 @@ static int __set_cpus_allowed_ptr(struct
goto out;
}
diff --git a/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch b/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch
index 9e9b59eccfde..c3d3173593f4 100644
--- a/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch
+++ b/patches/0019-locking-rtmutex-add-rwsem-implementation-based-on-rt.patch
@@ -43,8 +43,8 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/rwsem-rt.h | 69 ++++++++++
- kernel/locking/rwsem-rt.c | 292 ++++++++++++++++++++++++++++++++++++++++++++++
- 2 files changed, 361 insertions(+)
+ kernel/locking/rwsem-rt.c | 307 ++++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 376 insertions(+)
create mode 100644 include/linux/rwsem-rt.h
create mode 100644 kernel/locking/rwsem-rt.c
@@ -122,12 +122,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+#endif
--- /dev/null
+++ b/kernel/locking/rwsem-rt.c
-@@ -0,0 +1,292 @@
+@@ -0,0 +1,307 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/rwsem.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/signal.h>
+#include <linux/export.h>
++#include <linux/blkdev.h>
+
+#include "rtmutex_common.h"
+
@@ -212,6 +213,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ if (__down_read_trylock(sem))
+ return 0;
+
++ /*
++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too
++ * late if one of the callbacks needs to acquire a sleeping lock.
++ */
++ if (blk_needs_flush_plug(current))
++ blk_schedule_flush_plug(current);
++
+ might_sleep();
+ raw_spin_lock_irq(&m->wait_lock);
+ /*
@@ -334,6 +342,13 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ struct rt_mutex *m = &sem->rtmutex;
+ unsigned long flags;
+
++ /*
++ * Flush blk before ->pi_blocked_on is set. At schedule() time it is too
++ * late if one of the callbacks needs to acquire a sleeping lock.
++ */
++ if (blk_needs_flush_plug(current))
++ blk_schedule_flush_plug(current);
++
+ /* Take the rtmutex as a first step */
+ if (__rt_mutex_lock_state(m, state))
+ return -EINTR;
diff --git a/patches/0019-sched-Comment-affine_move_task.patch b/patches/0019-sched-Comment-affine_move_task.patch
index 203f4dc2f648..272305c2edf6 100644
--- a/patches/0019-sched-Comment-affine_move_task.patch
+++ b/patches/0019-sched-Comment-affine_move_task.patch
@@ -1,5 +1,5 @@
From: Valentin Schneider <valentin.schneider@arm.com>
-Date: Tue, 13 Oct 2020 15:01:16 +0100
+Date: Fri, 23 Oct 2020 12:12:17 +0200
Subject: [PATCH 19/19] sched: Comment affine_move_task()
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
@@ -12,8 +12,8 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -2032,7 +2032,75 @@ struct set_affinity_pending {
- };
+@@ -2084,7 +2084,75 @@ void do_set_cpus_allowed(struct task_str
+ }
/*
- * This function is wildly self concurrent, consider at least 3 times.
@@ -87,9 +87,9 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+ * pending affinity completion is preceded an uninstallion of
+ * p->migration_pending done with p->pi_lock held.
*/
- static int affine_move_task(struct rq *rq, struct rq_flags *rf,
- struct task_struct *p, int dest_cpu, unsigned int flags)
-@@ -2076,6 +2144,7 @@ static int affine_move_task(struct rq *r
+ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
+ int dest_cpu, unsigned int flags)
+@@ -2128,6 +2196,7 @@ static int affine_move_task(struct rq *r
if (!(flags & SCA_MIGRATE_ENABLE)) {
/* serialized by p->pi_lock */
if (!p->migration_pending) {
@@ -97,7 +97,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
refcount_set(&my_pending.refs, 1);
init_completion(&my_pending.done);
p->migration_pending = &my_pending;
-@@ -2114,7 +2183,11 @@ static int affine_move_task(struct rq *r
+@@ -2171,7 +2240,11 @@ static int affine_move_task(struct rq *r
}
if (task_running(rq, p) || p->state == TASK_WAKING) {
@@ -110,7 +110,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
task_rq_unlock(rq, p, rf);
stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg);
-@@ -2139,6 +2212,10 @@ static int affine_move_task(struct rq *r
+@@ -2196,6 +2269,10 @@ static int affine_move_task(struct rq *r
if (refcount_dec_and_test(&pending->refs))
wake_up_var(&pending->refs);
diff --git a/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch b/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch
index 7ed201e99523..e59b8c3c3c11 100644
--- a/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch
+++ b/patches/0022-locking-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch
@@ -303,7 +303,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
}
/**
-@@ -2261,7 +2407,7 @@ int rt_mutex_wait_proxy_lock(struct rt_m
+@@ -2246,7 +2392,7 @@ int rt_mutex_wait_proxy_lock(struct rt_m
raw_spin_lock_irq(&lock->wait_lock);
/* sleep on the mutex */
set_current_state(TASK_INTERRUPTIBLE);
@@ -312,7 +312,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/*
* try_to_take_rt_mutex() sets the waiter bit unconditionally. We might
* have to fix that up.
-@@ -2331,3 +2477,97 @@ bool rt_mutex_cleanup_proxy_lock(struct
+@@ -2316,3 +2462,97 @@ bool rt_mutex_cleanup_proxy_lock(struct
return cleanup;
}
@@ -430,7 +430,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
struct rt_mutex_waiter *waiter,
--- a/kernel/locking/rwsem-rt.c
+++ b/kernel/locking/rwsem-rt.c
-@@ -130,7 +130,7 @@ static int __sched __down_read_common(st
+@@ -138,7 +138,7 @@ static int __sched __down_read_common(st
*/
rt_mutex_init_waiter(&waiter, false);
ret = rt_mutex_slowlock_locked(m, state, NULL, RT_MUTEX_MIN_CHAINWALK,
diff --git a/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch b/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch
index 91368081d64e..0837da1c0e57 100644
--- a/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch
+++ b/patches/0023-locking-rtmutex-Use-custom-scheduling-function-for-s.patch
@@ -120,7 +120,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -4899,7 +4899,7 @@ pick_next_task(struct rq *rq, struct tas
+@@ -4957,7 +4957,7 @@ pick_next_task(struct rq *rq, struct tas
*
* WARNING: must be called with preemption disabled!
*/
@@ -129,7 +129,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
{
struct task_struct *prev, *next;
unsigned long *switch_count;
-@@ -4952,7 +4952,7 @@ static void __sched notrace __schedule(b
+@@ -5010,7 +5010,7 @@ static void __sched notrace __schedule(b
* - ptrace_{,un}freeze_traced() can change ->state underneath us.
*/
prev_state = prev->state;
@@ -138,7 +138,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
if (signal_pending_state(prev_state, prev)) {
prev->state = TASK_RUNNING;
} else {
-@@ -5036,7 +5036,7 @@ void __noreturn do_task_dead(void)
+@@ -5094,7 +5094,7 @@ void __noreturn do_task_dead(void)
/* Tell freezer to ignore us: */
current->flags |= PF_NOFREEZE;
@@ -147,7 +147,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
BUG();
/* Avoid "noreturn function does return" - but don't continue if BUG() is a NOP: */
-@@ -5066,9 +5066,6 @@ static inline void sched_submit_work(str
+@@ -5124,9 +5124,6 @@ static inline void sched_submit_work(str
preempt_enable_no_resched();
}
@@ -157,7 +157,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/*
* If we are going to sleep and we have plugged IO queued,
* make sure to submit it to avoid deadlocks.
-@@ -5094,7 +5091,7 @@ asmlinkage __visible void __sched schedu
+@@ -5152,7 +5149,7 @@ asmlinkage __visible void __sched schedu
sched_submit_work(tsk);
do {
preempt_disable();
@@ -166,7 +166,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
sched_preempt_enable_no_resched();
} while (need_resched());
sched_update_worker(tsk);
-@@ -5122,7 +5119,7 @@ void __sched schedule_idle(void)
+@@ -5180,7 +5177,7 @@ void __sched schedule_idle(void)
*/
WARN_ON_ONCE(current->state);
do {
@@ -175,7 +175,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
} while (need_resched());
}
-@@ -5175,7 +5172,7 @@ static void __sched notrace preempt_sche
+@@ -5233,7 +5230,7 @@ static void __sched notrace preempt_sche
*/
preempt_disable_notrace();
preempt_latency_start(1);
@@ -184,7 +184,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
preempt_latency_stop(1);
preempt_enable_no_resched_notrace();
-@@ -5205,6 +5202,19 @@ asmlinkage __visible void __sched notrac
+@@ -5263,6 +5260,19 @@ asmlinkage __visible void __sched notrac
NOKPROBE_SYMBOL(preempt_schedule);
EXPORT_SYMBOL(preempt_schedule);
@@ -204,7 +204,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/**
* preempt_schedule_notrace - preempt_schedule called by tracing
*
-@@ -5248,7 +5258,7 @@ asmlinkage __visible void __sched notrac
+@@ -5306,7 +5316,7 @@ asmlinkage __visible void __sched notrac
* an infinite recursion.
*/
prev_ctx = exception_enter();
@@ -213,7 +213,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
exception_exit(prev_ctx);
preempt_latency_stop(1);
-@@ -5277,7 +5287,7 @@ asmlinkage __visible void __sched preemp
+@@ -5335,7 +5345,7 @@ asmlinkage __visible void __sched preemp
do {
preempt_disable();
local_irq_enable();
diff --git a/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch b/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch
new file mode 100644
index 000000000000..4754e90fd194
--- /dev/null
+++ b/patches/blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch
@@ -0,0 +1,37 @@
+From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+Date: Fri, 23 Oct 2020 12:21:51 +0200
+Subject: [PATCH] blk-mq: Don't IPI requests on PREEMPT_RT
+
+blk_mq_complete_request_remote() will dispatch request completion to
+another CPU via IPI if the CPU belongs to a different cache domain.
+
+This breaks on PREEMPT_RT because the IPI function will complete the
+request in IRQ context which includes acquiring spinlock_t typed locks.
+Completing the IPI request in softirq on the remote CPU is probably less
+efficient because it would require to wake ksoftirqd for this task
+(which runs at SCHED_OTHER).
+
+Ignoring the IPI request and completing the request locally is probably
+the best option. It be completed either in the IRQ-thread or at the end
+of the routine in softirq context.
+
+Let blk_mq_complete_need_ipi() return that there is no need for IPI on
+PREEMPT_RT.
+
+Reported-by: David Runge <dave@sleepmap.de>
+Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+---
+ block/blk-mq.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+--- a/block/blk-mq.c
++++ b/block/blk-mq.c
+@@ -645,7 +645,7 @@ static inline bool blk_mq_complete_need_
+ {
+ int cpu = raw_smp_processor_id();
+
+- if (!IS_ENABLED(CONFIG_SMP) ||
++ if (!IS_ENABLED(CONFIG_SMP) || IS_ENABLED(CONFIG_PREEMPT_RT) ||
+ !test_bit(QUEUE_FLAG_SAME_COMP, &rq->q->queue_flags))
+ return false;
+
diff --git a/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch b/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch
index b128b5d1f642..66713a1c3cc3 100644
--- a/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch
+++ b/patches/kernel-sched-move-stack-kprobe-clean-up-to-__put_tas.patch
@@ -54,7 +54,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
security_task_free(tsk);
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -4178,15 +4178,6 @@ static struct rq *finish_task_switch(str
+@@ -4236,15 +4236,6 @@ static struct rq *finish_task_switch(str
if (prev->sched_class->task_dead)
prev->sched_class->task_dead(prev);
diff --git a/patches/localversion.patch b/patches/localversion.patch
index 48a458c6f3b7..19d7ea05016c 100644
--- a/patches/localversion.patch
+++ b/patches/localversion.patch
@@ -10,4 +10,4 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
--- /dev/null
+++ b/localversion-rt
@@ -0,0 +1 @@
-+-rt18
++-rt19
diff --git a/patches/preempt-lazy-support.patch b/patches/preempt-lazy-support.patch
index 4a3ca64d24fc..c3281447651c 100644
--- a/patches/preempt-lazy-support.patch
+++ b/patches/preempt-lazy-support.patch
@@ -321,7 +321,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
preempt_enable();
trace_sched_migrate_enable_tp(p);
-@@ -3758,6 +3802,9 @@ int sched_fork(unsigned long clone_flags
+@@ -3816,6 +3860,9 @@ int sched_fork(unsigned long clone_flags
p->on_cpu = 0;
#endif
init_task_preempt_count(p);
@@ -331,7 +331,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
#ifdef CONFIG_SMP
plist_node_init(&p->pushable_tasks, MAX_PRIO);
RB_CLEAR_NODE(&p->pushable_dl_tasks);
-@@ -4999,6 +5046,7 @@ static void __sched notrace __schedule(b
+@@ -5057,6 +5104,7 @@ static void __sched notrace __schedule(b
next = pick_next_task(rq, prev, &rf);
clear_tsk_need_resched(prev);
@@ -339,7 +339,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
clear_preempt_need_resched();
if (likely(prev != next)) {
-@@ -5195,6 +5243,30 @@ static void __sched notrace preempt_sche
+@@ -5253,6 +5301,30 @@ static void __sched notrace preempt_sche
} while (need_resched());
}
@@ -370,7 +370,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
#ifdef CONFIG_PREEMPTION
/*
* This is the entry point to schedule() from in-kernel preemption
-@@ -5208,7 +5280,8 @@ asmlinkage __visible void __sched notrac
+@@ -5266,7 +5338,8 @@ asmlinkage __visible void __sched notrac
*/
if (likely(!preemptible()))
return;
@@ -380,7 +380,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
preempt_schedule_common();
}
NOKPROBE_SYMBOL(preempt_schedule);
-@@ -5248,6 +5321,9 @@ asmlinkage __visible void __sched notrac
+@@ -5306,6 +5379,9 @@ asmlinkage __visible void __sched notrac
if (likely(!preemptible()))
return;
@@ -390,7 +390,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
do {
/*
* Because the function tracer can trace preempt_count_sub()
-@@ -7086,7 +7162,9 @@ void init_idle(struct task_struct *idle,
+@@ -7144,7 +7220,9 @@ void init_idle(struct task_struct *idle,
/* Set the preempt count _outside_ the spinlocks! */
init_idle_preempt_count(idle, cpu);
diff --git a/patches/printk-Tiny-cleanup.patch b/patches/printk-Tiny-cleanup.patch
index 54a8fec446c3..5fed4d33a19f 100644
--- a/patches/printk-Tiny-cleanup.patch
+++ b/patches/printk-Tiny-cleanup.patch
@@ -125,7 +125,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
{
return vprintk_emit(0, LOGLEVEL_DEFAULT, NULL, fmt, args);
}
-@@ -3369,7 +3345,7 @@ EXPORT_SYMBOL_GPL(kmsg_dump_get_line);
+@@ -3303,7 +3279,7 @@ EXPORT_SYMBOL_GPL(kmsg_dump_get_line);
* @syslog: include the "<4>" prefixes
* @buf: buffer to copy the line to
* @size: maximum size of the buffer
@@ -134,7 +134,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
*
* Start at the end of the kmsg buffer and fill the provided buffer
* with as many of the the *youngest* kmsg records that fit into it.
-@@ -3536,7 +3512,7 @@ static bool __prb_trylock(struct prb_cpu
+@@ -3470,7 +3446,7 @@ static bool __prb_trylock(struct prb_cpu
*
* It is safe to call this function from any context and state.
*/
@@ -143,7 +143,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
{
for (;;) {
if (__prb_trylock(cpu_lock, cpu_store))
-@@ -3554,7 +3530,7 @@ void prb_lock(struct prb_cpulock *cpu_lo
+@@ -3488,7 +3464,7 @@ void prb_lock(struct prb_cpulock *cpu_lo
*
* It is safe to call this function from any context and state.
*/
diff --git a/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch b/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch
index 6b940147385f..7fd06b76cf3a 100644
--- a/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch
+++ b/patches/ptrace-fix-ptrace-vs-tasklist_lock-race.patch
@@ -115,7 +115,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
spin_unlock_irq(&task->sighand->siglock);
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -2511,6 +2511,18 @@ int migrate_swap(struct task_struct *cur
+@@ -2568,6 +2568,18 @@ int migrate_swap(struct task_struct *cur
}
#endif /* CONFIG_NUMA_BALANCING */
@@ -134,7 +134,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
/*
* wait_task_inactive - wait for a thread to unschedule.
*
-@@ -2555,7 +2567,7 @@ unsigned long wait_task_inactive(struct
+@@ -2612,7 +2624,7 @@ unsigned long wait_task_inactive(struct
* is actually now running somewhere else!
*/
while (task_running(rq, p)) {
@@ -143,7 +143,7 @@ Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
return 0;
cpu_relax();
}
-@@ -2570,7 +2582,8 @@ unsigned long wait_task_inactive(struct
+@@ -2627,7 +2639,8 @@ unsigned long wait_task_inactive(struct
running = task_running(rq, p);
queued = task_on_rq_queued(p);
ncsw = 0;
diff --git a/patches/sched-might-sleep-do-not-account-rcu-depth.patch b/patches/sched-might-sleep-do-not-account-rcu-depth.patch
index 305bcf173a15..bebb0df42aec 100644
--- a/patches/sched-might-sleep-do-not-account-rcu-depth.patch
+++ b/patches/sched-might-sleep-do-not-account-rcu-depth.patch
@@ -36,7 +36,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
/* Internal to kernel */
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -7795,7 +7795,7 @@ void __init sched_init(void)
+@@ -7853,7 +7853,7 @@ void __init sched_init(void)
#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
static inline int preempt_count_equals(int preempt_offset)
{
diff --git a/patches/sched-mmdrop-delayed.patch b/patches/sched-mmdrop-delayed.patch
index 4d4ea6476ff1..1177da334795 100644
--- a/patches/sched-mmdrop-delayed.patch
+++ b/patches/sched-mmdrop-delayed.patch
@@ -77,7 +77,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
struct mm_struct *mm;
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
-@@ -4166,9 +4166,13 @@ static struct rq *finish_task_switch(str
+@@ -4224,9 +4224,13 @@ static struct rq *finish_task_switch(str
* provided by mmdrop(),
* - a sync_core for SYNC_CORE.
*/
@@ -92,7 +92,7 @@ Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
}
if (unlikely(prev_state == TASK_DEAD)) {
if (prev->sched_class->task_dead)
-@@ -7183,6 +7187,7 @@ void sched_setnuma(struct task_struct *p
+@@ -7241,6 +7245,7 @@ void sched_setnuma(struct task_struct *p
#endif /* CONFIG_NUMA_BALANCING */
#ifdef CONFIG_HOTPLUG_CPU
diff --git a/patches/seqlock-Unbreak-lockdep.patch b/patches/seqlock-Unbreak-lockdep.patch
new file mode 100644
index 000000000000..9a5ec6509764
--- /dev/null
+++ b/patches/seqlock-Unbreak-lockdep.patch
@@ -0,0 +1,68 @@
+From: "peterz@infradead.org" <peterz@infradead.org>
+Date: Tue, 15 Sep 2020 16:30:28 +0200
+Subject: [PATCH] seqlock: Unbreak lockdep
+
+Upstream commit 267580db047ef428a70bef8287ca62c5a450c139
+
+seqcount_LOCKNAME_init() needs to be a macro due to the lockdep
+annotation in seqcount_init(). Since a macro cannot define another
+macro, we need to effectively revert commit: e4e9ab3f9f91 ("seqlock:
+Fold seqcount_LOCKNAME_init() definition").
+
+Fixes: e4e9ab3f9f91 ("seqlock: Fold seqcount_LOCKNAME_init() definition")
+Reported-by: Qian Cai <cai@redhat.com>
+Debugged-by: Boqun Feng <boqun.feng@gmail.com>
+Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
+Tested-by: Qian Cai <cai@redhat.com>
+Link: https://lkml.kernel.org/r/20200915143028.GB2674@hirez.programming.kicks-ass.net
+Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
+---
+ include/linux/seqlock.h | 22 ++++++++++++++--------
+ 1 file changed, 14 insertions(+), 8 deletions(-)
+
+--- a/include/linux/seqlock.h
++++ b/include/linux/seqlock.h
+@@ -173,6 +173,19 @@ static inline void seqcount_lockdep_read
+ * @lock: Pointer to the associated lock
+ */
+
++#define seqcount_LOCKNAME_init(s, _lock, lockname) \
++ do { \
++ seqcount_##lockname##_t *____s = (s); \
++ seqcount_init(&____s->seqcount); \
++ __SEQ_LOCK(____s->lock = (_lock)); \
++ } while (0)
++
++#define seqcount_raw_spinlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, raw_spinlock)
++#define seqcount_spinlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, spinlock)
++#define seqcount_rwlock_init(s, lock) seqcount_LOCKNAME_init(s, lock, rwlock);
++#define seqcount_mutex_init(s, lock) seqcount_LOCKNAME_init(s, lock, mutex);
++#define seqcount_ww_mutex_init(s, lock) seqcount_LOCKNAME_init(s, lock, ww_mutex);
++
+ /*
+ * SEQCOUNT_LOCKNAME() - Instantiate seqcount_LOCKNAME_t and helpers
+ * seqprop_LOCKNAME_*() - Property accessors for seqcount_LOCKNAME_t
+@@ -190,13 +203,6 @@ typedef struct seqcount_##lockname {
+ __SEQ_LOCK(locktype *lock); \
+ } seqcount_##lockname##_t; \
+ \
+-static __always_inline void \
+-seqcount_##lockname##_init(seqcount_##lockname##_t *s, locktype *lock) \
+-{ \
+- seqcount_init(&s->seqcount); \
+- __SEQ_LOCK(s->lock = lock); \
+-} \
+- \
+ static __always_inline seqcount_t * \
+ __seqprop_##lockname##_ptr(seqcount_##lockname##_t *s) \
+ { \
+@@ -284,8 +290,8 @@ SEQCOUNT_LOCKNAME(ww_mutex, struct w
+ __SEQ_LOCK(.lock = (assoc_lock)) \
+ }
+
+-#define SEQCNT_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
+ #define SEQCNT_RAW_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
++#define SEQCNT_SPINLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
+ #define SEQCNT_RWLOCK_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
+ #define SEQCNT_MUTEX_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
+ #define SEQCNT_WW_MUTEX_ZERO(name, lock) SEQCOUNT_LOCKNAME_ZERO(name, lock)
diff --git a/patches/series b/patches/series
index aa5087908836..4a85452aa9e4 100644
--- a/patches/series
+++ b/patches/series
@@ -42,8 +42,8 @@
# https://lkml.kernel.org/r/20200828100022.1099682-2-npiggin@gmail.com
mm-fix-exec-activate_mm-vs-TLB-shootdown-and-lazy-tl.patch
-# 2020-10-15 13:05 Peter Zijlstra [PATCH v3 00/19] sched: Migrate disable support
-# 20201015110532.738127234@infradead.org
+# 2020-10-23 12:11 Peter Zijlstra [PATCH v4 00/19] sched: Migrate disable support
+# 20201023101158.088940906@infradead.org
0001-stop_machine-Add-function-and-caller-debug-info.patch
0002-sched-Fix-balance_callback.patch
0003-sched-hotplug-Ensure-only-per-cpu-kthreads-run-durin.patch
@@ -79,6 +79,9 @@ io_wq-Make-io_wqe-lock-a-raw_spinlock_t.patch
# 20200915074816.52zphpywj4zidspk@linutronix.de
bus-mhi-Remove-include-of-rwlock_types.h.patch
+# 20201023110400.bx3uzsb7xy5jtsea@linutronix.de
+blk-mq-Don-t-IPI-requests-on-PREEMPT_RT.patch
+
############################################################
# Ready for posting
############################################################
@@ -98,7 +101,6 @@ bus-mhi-Remove-include-of-rwlock_types.h.patch
0012-printk-move-console-printing-to-kthreads.patch
0013-printk-remove-deferred-printing.patch
0014-printk-add-console-handover.patch
-0015-printk-add-pr_flush.patch
printk-Tiny-cleanup.patch
############################################################
@@ -144,7 +146,6 @@ tasklets-Use-static-line-for-functions.patch
0004-locking-rtmutex-Remove-rt_mutex_timed_lock.patch
0005-locking-rtmutex-Handle-the-various-new-futex-race-co.patch
0006-futex-Fix-bug-on-when-a-requeued-RT-task-times-out.patch
-0007-locking-rtmutex-Add-rtmutex_lock_killable.patch
0008-locking-rtmutex-Make-lock_killable-work.patch
0009-locking-spinlock-Split-the-lock-types-header.patch
0010-locking-rtmutex-Avoid-include-hell.patch
@@ -199,6 +200,8 @@ seqlock-Fix-multiple-kernel-doc-warnings.patch
0011-seqlock-seqcount_t-Implement-all-read-APIs-as-statem.patch
0012-seqlock-seqcount_LOCKNAME_t-Introduce-PREEMPT_RT-sup.patch
0013-seqlock-PREEMPT_RT-Do-not-starve-seqlock_t-writers.patch
+# 267580db047ef428a70bef8287ca62c5a450c139
+seqlock-Unbreak-lockdep.patch
##
0024-xfrm-Use-sequence-counter-with-associated-spinlock.patch
u64_stats-Disable-preemption-on-32bit-UP-SMP-with-RT.patch