summaryrefslogtreecommitdiff
path: root/drivers/irqchip/irq-gic-v3-its.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote-tracking branch 'kvm-arm/next'Stephen Rothwell2020-06-241-0/+8
|\
| * KVM: arm64: vgic-v4: Plug race between non-residency and v4.1 doorbellMarc Zyngier2020-06-231-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When making a vPE non-resident because it has hit a blocking WFI, the doorbell can fire at any time after the write to the RD. Crucially, it can fire right between the write to GICR_VPENDBASER and the write to the pending_last field in the its_vpe structure. This means that we would overwrite pending_last with stale data, and potentially not wakeup until some unrelated event (such as a timer interrupt) puts the vPE back on the CPU. GICv4 isn't affected by this as we actively mask the doorbell on entering the guest, while GICv4.1 automatically manages doorbell delivery without any hypervisor-driven masking. Use the vpe_lock to synchronize such update, which solves the problem altogether. Fixes: ae699ad348cdc ("irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer") Reported-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
* | irqchip/gic-v4.1: Use readx_poll_timeout_atomic() to fix sleep in atomicZenghui Yu2020-06-211-4/+4
|/ | | | | | | | | | | | | | | readx_poll_timeout() can sleep if @sleep_us is specified by the caller, and is therefore unsafe to be used inside the atomic context, which is this case when we use it to poll the GICR_VPENDBASER.Dirty bit in irq_set_vcpu_affinity() callback. Let's convert to its atomic version instead which helps to get the v4.1 board back to life! Fixes: 96806229ca03 ("irqchip/gic-v4.1: Add support for VPENDBASER's Dirty+Valid signaling") Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200605052345.1494-1-yuzenghui@huawei.com
* irqchip/gic-v3-its: Balance initial LPI affinity across CPUsMarc Zyngier2020-05-201-27/+100
| | | | | | | | | | | | | | | | | | | | | When mapping a LPI, the ITS driver picks the first possible affinity, which is in most cases CPU0, assuming that if that's not suitable, someone will come and set the affinity to something more interesting. It apparently isn't the case, and people complain of poor performance when many interrupts are glued to the same CPU. So let's place the interrupts by finding the "least loaded" CPU (that is, the one that has the fewer LPIs mapped to it). So called 'managed' interrupts are an interesting case where the affinity is actually dictated by the kernel itself, and we should honor this. Reported-by: John Garry <john.garry@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Tested-by: John Garry <john.garry@huawei.com> Link: https://lore.kernel.org/r/1575642904-58295-1-git-send-email-john.garry@huawei.com Link: https://lore.kernel.org/r/20200515165752.121296-3-maz@kernel.org
* irqchip/gic-v3-its: Track LPI distribution on a per CPU basisMarc Zyngier2020-05-181-3/+46
| | | | | | | | | | In order to improve the distribution of LPIs among CPUs, let start by tracking the number of LPIs assigned to CPUs, both for managed and non-managed interrupts (as separate counters). Signed-off-by: Marc Zyngier <maz@kernel.org> Tested-by: John Garry <john.garry@huawei.com> Link: https://lore.kernel.org/r/20200515165752.121296-2-maz@kernel.org
* irqchip/gic-v4.1: Update effective affinity of virtual SGIsMarc Zyngier2020-04-161-0/+1
| | | | | | | | | | | | | Although the vSGIs are not directly visible to the host, they still get moved around by the CPU hotplug, for example. This results in the kernel moaning on the console, such as: genirq: irq_chip GICv4.1-sgi did not update eff. affinity mask of irq 38 Updating the effective affinity on set_affinity() fixes it. Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
* irqchip/gic-v4.1: Add support for VPENDBASER's Dirty+Valid signalingMarc Zyngier2020-04-161-0/+19
| | | | | | | | | | | | | | | | | | | | | When a vPE is made resident, the GIC starts parsing the virtual pending table to deliver pending interrupts. This takes place asynchronously, and can at times take a long while. Long enough that the vcpu enters the guest and hits WFI before any interrupt has been signaled yet. The vcpu then exits, blocks, and now gets a doorbell. Rince, repeat. In order to avoid the above, a (optional on GICv4, mandatory on v4.1) feature allows the GIC to feedback to the hypervisor whether it is done parsing the VPT by clearing the GICR_VPENDBASER.Dirty bit. The hypervisor can then wait until the GIC is ready before actually running the vPE. Plug the detection code as well as polling on vPE schedule. While at it, tidy-up the kernel message that displays the GICv4 optional features. Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org>
* Merge branch 'irq/gic-v4.1' into irq/irqchip-nextMarc Zyngier2020-03-241-22/+400
|\ | | | | | | Signed-off-by: Marc Zyngier <maz@kernel.org>
| * irqchip/gic-v4.1: Eagerly vmap vPEsMarc Zyngier2020-03-241-10/+42
| | | | | | | | | | | | | | | | | | | | | | Now that we have HW-accelerated SGIs being delivered to VPEs, it becomes required to map the VPEs on all ITSs instead of relying on the lazy approach that we would use when using the ITS-list mechanism. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20200304203330.4967-17-maz@kernel.org
| * irqchip/gic-v4.1: Plumb set_vcpu_affinity SGI callbacksMarc Zyngier2020-03-241-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Just like for vLPIs, there is some configuration information that cannot be directly communicated through the normal irqchip API, and we have to use our good old friend set_vcpu_affinity as a side-band communication mechanism. This is used to configure group and priority for a given vSGI. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-13-maz@kernel.org
| * irqchip/gic-v4.1: Plumb get/set_irqchip_state SGI callbacksMarc Zyngier2020-03-241-0/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To implement the get/set_irqchip_state callbacks (limited to the PENDING state), we have to use a particular set of hacks: - Reading the pending state is done by using a pair of new redistributor registers (GICR_VSGIR, GICR_VSGIPENDR), which allow the 16 interrupts state to be retrieved. - Setting the pending state is done by generating it as we'd otherwise do for a guest (writing to GITS_SGIR). - Clearing the pending state is done by emitting a VSGI command with the "clear" bit set. This requires some interesting locking though: - When talking to the redistributor, we must make sure that the VPE affinity doesn't change, hence taking the VPE lock. - At the same time, we must ensure that nobody accesses the same redistributor's GICR_VSGIR registers for a different VPE, which would corrupt the reading of the pending bits. We thus take the per-RD spinlock. Much fun. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20200304203330.4967-12-maz@kernel.org
| * irqchip/gic-v4.1: Plumb mask/unmask SGI callbacksMarc Zyngier2020-03-241-0/+18
| | | | | | | | | | | | | | | | | | | | Implement mask/unmask for virtual SGIs by calling into the configuration helper. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-11-maz@kernel.org
| * irqchip/gic-v4.1: Add initial SGI configurationMarc Zyngier2020-03-241-1/+91
| | | | | | | | | | | | | | | | | | | | | | The GICv4.1 ITS has yet another new command (VSGI) which allows a VPE-targeted SGI to be configured (or have its pending state cleared). Add support for this command and plumb it into the activate irqdomain callback so that it is ready to be used. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20200304203330.4967-10-maz@kernel.org
| * irqchip/gic-v4.1: Plumb skeletal VSGI irqchipMarc Zyngier2020-03-241-1/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since GICv4.1 has the capability to inject 16 SGIs into each VPE, and that I'm keen not to invent too many specific interfaces to manipulate these interrupts, let's pretend that each of these SGIs is an actual Linux interrupt. For that matter, let's introduce a minimal irqchip and irqdomain setup that will get fleshed up in the following patches. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-9-maz@kernel.org
| * irqchip/gic-v4.1: Map the ITS SGIR register pageMarc Zyngier2020-03-201-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One of the new features of GICv4.1 is to allow virtual SGIs to be directly signaled to a VPE. For that, the ITS has grown a new 64kB page containing only a single register that is used to signal a SGI to a given VPE. Add a second mapping covering this new 64kB range, and take this opportunity to limit the original mapping to 64kB, which is enough to cover the span of the ITS registers. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-8-maz@kernel.org
| * irqchip/gic-v4.1: Advertise support v4.1 to KVMMarc Zyngier2020-03-201-1/+8
| | | | | | | | | | | | | | | | | | Tell KVM that we support v4.1. Nothing uses this information so far. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-7-maz@kernel.org
| * irqchip/gic-v4.1: Ensure mutual exclusion betwen invalidations on the same RDMarc Zyngier2020-03-201-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The GICv4.1 spec says that it is CONTRAINED UNPREDICTABLE to write to any of the GICR_INV{LPI,ALL}R registers if GICR_SYNCR.Busy == 1. To deal with it, we must ensure that only a single invalidation can happen at a time for a given redistributor. Add a per-RD lock to that effect and take it around the invalidation/syncr-read to deal with this. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-6-maz@kernel.org
| * irqchip/gic-v4.1: Wait for completion of redistributor's INVALL operationZenghui Yu2020-03-201-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In GICv4.1, we emulate a guest-issued INVALL command by a direct write to GICR_INVALLR. Before we finish the emulation and go back to guest, let's make sure the physical invalidate operation is actually completed and no stale data will be left in redistributor. Per the specification, this can be achieved by polling the GICR_SYNCR.Busy bit (to zero). Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200302092145.899-1-yuzenghui@huawei.com Link: https://lore.kernel.org/r/20200304203330.4967-5-maz@kernel.org
| * irqchip/gic-v4.1: Ensure mutual exclusion between vPE affinity change and RD ↵Marc Zyngier2020-03-191-8/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | access Before GICv4.1, all operations would be serialized with the affinity changes by virtue of using the same ITS command queue. With v4.1, things change, as invalidations (and a number of other operations) are issued using the redistributor MMIO frame. We must thus make sure that these redistributor accesses cannot race against aginst the affinity change, or we may end-up talking to the wrong redistributor. To ensure this, we expand the irq_to_cpuid() helper to take a spinlock when the LPI is mapped to a vLPI (a new per-VPE lock) on each operation that requires mutual exclusion. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20200304203330.4967-4-maz@kernel.org
| * irqchip/gic-v4.1: Skip absent CPUs while iterating over redistributorsMarc Zyngier2020-03-191-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | In a system that is only sparsly populated with CPUs, we can end-up with redistributors structures that are not initialized. Let's make sure we don't try and access those when iterating over them (in this case when checking we have a L2 VPE table). Fixes: 4e6437f12d6e ("irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD level") Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Link: https://lore.kernel.org/r/20200304203330.4967-3-maz@kernel.org
* | irqchip/gic-v4: Use Inner-Shareable attributes for virtual pending tablesHeyi Guo2020-03-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is no special reason to set virtual LPI pending table as non-shareable. If we choose to hard code the shareability without probing, Inner-Shareable is likely to be a better choice, as the VPEs can move around and benefit from having the redistributors snooping each other's cache, if that's something they can do. Furthermore, Hisilicon hip08 ends up with unspecified errors when mixing shareability attributes. So let's move to IS attributes for the VPT. This has also been tested on D05 and didn't show any regression. Signed-off-by: Heyi Guo <guoheyi@huawei.com> [maz: rewrote commit message] Signed-off-by: Marc Zyngier <maz@kernel.org> Tested-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191130073849.38378-1-guoheyi@huawei.com
* | irqchip/gic-v4: Provide irq_retrigger to avoid circular locking dependencyMarc Zyngier2020-03-161-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On a very heavily loaded D05 with GICv4, I managed to trigger the following lockdep splat: [ 6022.598864] ====================================================== [ 6022.605031] WARNING: possible circular locking dependency detected [ 6022.611200] 5.6.0-rc4-00026-geee7c7b0f498 #680 Tainted: G E [ 6022.618061] ------------------------------------------------------ [ 6022.624227] qemu-system-aar/7569 is trying to acquire lock: [ 6022.629789] ffff042f97606808 (&p->pi_lock){-.-.}, at: try_to_wake_up+0x54/0x7a0 [ 6022.637102] [ 6022.637102] but task is already holding lock: [ 6022.642921] ffff002fae424cf0 (&irq_desc_lock_class){-.-.}, at: __irq_get_desc_lock+0x5c/0x98 [ 6022.651350] [ 6022.651350] which lock already depends on the new lock. [ 6022.651350] [ 6022.659512] [ 6022.659512] the existing dependency chain (in reverse order) is: [ 6022.666980] [ 6022.666980] -> #2 (&irq_desc_lock_class){-.-.}: [ 6022.672983] _raw_spin_lock_irqsave+0x50/0x78 [ 6022.677848] __irq_get_desc_lock+0x5c/0x98 [ 6022.682453] irq_set_vcpu_affinity+0x40/0xc0 [ 6022.687236] its_make_vpe_non_resident+0x6c/0xb8 [ 6022.692364] vgic_v4_put+0x54/0x70 [ 6022.696273] vgic_v3_put+0x20/0xd8 [ 6022.700183] kvm_vgic_put+0x30/0x48 [ 6022.704182] kvm_arch_vcpu_put+0x34/0x50 [ 6022.708614] kvm_sched_out+0x34/0x50 [ 6022.712700] __schedule+0x4bc/0x7f8 [ 6022.716697] schedule+0x50/0xd8 [ 6022.720347] kvm_arch_vcpu_ioctl_run+0x5f0/0x978 [ 6022.725473] kvm_vcpu_ioctl+0x3d4/0x8f8 [ 6022.729820] ksys_ioctl+0x90/0xd0 [ 6022.733642] __arm64_sys_ioctl+0x24/0x30 [ 6022.738074] el0_svc_common.constprop.3+0xa8/0x1e8 [ 6022.743373] do_el0_svc+0x28/0x88 [ 6022.747198] el0_svc+0x14/0x40 [ 6022.750761] el0_sync_handler+0x124/0x2b8 [ 6022.755278] el0_sync+0x140/0x180 [ 6022.759100] [ 6022.759100] -> #1 (&rq->lock){-.-.}: [ 6022.764143] _raw_spin_lock+0x38/0x50 [ 6022.768314] task_fork_fair+0x40/0x128 [ 6022.772572] sched_fork+0xe0/0x210 [ 6022.776484] copy_process+0x8c4/0x18d8 [ 6022.780742] _do_fork+0x88/0x6d8 [ 6022.784478] kernel_thread+0x64/0x88 [ 6022.788563] rest_init+0x30/0x270 [ 6022.792390] arch_call_rest_init+0x14/0x1c [ 6022.796995] start_kernel+0x498/0x4c4 [ 6022.801164] [ 6022.801164] -> #0 (&p->pi_lock){-.-.}: [ 6022.806382] __lock_acquire+0xdd8/0x15c8 [ 6022.810813] lock_acquire+0xd0/0x218 [ 6022.814896] _raw_spin_lock_irqsave+0x50/0x78 [ 6022.819761] try_to_wake_up+0x54/0x7a0 [ 6022.824018] wake_up_process+0x1c/0x28 [ 6022.828276] wakeup_softirqd+0x38/0x40 [ 6022.832533] __tasklet_schedule_common+0xc4/0xf0 [ 6022.837658] __tasklet_schedule+0x24/0x30 [ 6022.842176] check_irq_resend+0xc8/0x158 [ 6022.846609] irq_startup+0x74/0x128 [ 6022.850606] __enable_irq+0x6c/0x78 [ 6022.854602] enable_irq+0x54/0xa0 [ 6022.858431] its_make_vpe_non_resident+0xa4/0xb8 [ 6022.863557] vgic_v4_put+0x54/0x70 [ 6022.867469] kvm_arch_vcpu_blocking+0x28/0x38 [ 6022.872336] kvm_vcpu_block+0x48/0x490 [ 6022.876594] kvm_handle_wfx+0x18c/0x310 [ 6022.880938] handle_exit+0x138/0x198 [ 6022.885022] kvm_arch_vcpu_ioctl_run+0x4d4/0x978 [ 6022.890148] kvm_vcpu_ioctl+0x3d4/0x8f8 [ 6022.894494] ksys_ioctl+0x90/0xd0 [ 6022.898317] __arm64_sys_ioctl+0x24/0x30 [ 6022.902748] el0_svc_common.constprop.3+0xa8/0x1e8 [ 6022.908046] do_el0_svc+0x28/0x88 [ 6022.911871] el0_svc+0x14/0x40 [ 6022.915434] el0_sync_handler+0x124/0x2b8 [ 6022.919951] el0_sync+0x140/0x180 [ 6022.923773] [ 6022.923773] other info that might help us debug this: [ 6022.923773] [ 6022.931762] Chain exists of: [ 6022.931762] &p->pi_lock --> &rq->lock --> &irq_desc_lock_class [ 6022.931762] [ 6022.942101] Possible unsafe locking scenario: [ 6022.942101] [ 6022.948007] CPU0 CPU1 [ 6022.952523] ---- ---- [ 6022.957039] lock(&irq_desc_lock_class); [ 6022.961036] lock(&rq->lock); [ 6022.966595] lock(&irq_desc_lock_class); [ 6022.973109] lock(&p->pi_lock); [ 6022.976324] [ 6022.976324] *** DEADLOCK *** This is happening because we have a pending doorbell that requires retrigger. As SW retriggering is done in a tasklet, we trigger the circular dependency above. The easy cop-out is to provide a retrigger callback that doesn't require acquiring any extra lock. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200310184921.23552-5-maz@kernel.org
* | irqchip/gic-v3-its: Probe ITS page size for all GITS_BASERn registersMarc Zyngier2020-03-161-34/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The GICv3 ITS driver assumes that once it has latched on a page size for a given BASER register, it can use the same page size as the maximum page size for all subsequent BASER registers. Although it worked so far, nothing in the architecture guarantees this, and Nianyao Tang hit this problem on some undisclosed implementation. Let's bite the bullet and probe the the supported page size on all BASER registers before starting to populate the tables. This simplifies the setup a bit, at the expense of a few additional MMIO accesses. Signed-off-by: Marc Zyngier <maz@kernel.org> Reported-by: Nianyao Tang <tangnianyao@huawei.com> Tested-by: Nianyao Tang <tangnianyao@huawei.com> Link: https://lore.kernel.org/r/1584089195-63897-1-git-send-email-zhangshaokun@hisilicon.com
* | irqchip/gic-v3-its: Fix access width for gicr_syncrHeyi Guo2020-03-081-1/+1
|/ | | | | | | | | GICR_SYNCR is a 32bit register, so it is better to access it with 32bit access width, though we have not seen any real problem. Signed-off-by: Heyi Guo <guoheyi@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200225090023.28020-1-guoheyi@huawei.com
* irqchip/gic-v4.1: Avoid 64bit division for the sake of 32bit ARMMarc Zyngier2020-02-091-2/+2
| | | | | | | | | | | | In order to allow the GICv4 code to link properly on 32bit ARM, make sure we don't use 64bit divisions when it isn't strictly necessary. Fixes: 4e6437f12d6e ("irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD level") Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* irqchip/gic-v3-its: Rename VPENDBASER/VPROPBASER accessorsZenghui Yu2020-02-081-14/+14
| | | | | | | | | | | V{PEND,PROP}BASER registers are actually located in VLPI_base frame of the *redistributor*. Rename their accessors to reflect this fact. No functional changes. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-7-yuzenghui@huawei.com
* irqchip/gic-v3-its: Remove superfluous WARN_ONZenghui Yu2020-02-081-1/+0
| | | | | | | | | "ITS virtual pending table not cleaning" is already complained inside its_clear_vpend_valid(), there's no need to trigger a WARN_ON again. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-6-yuzenghui@huawei.com
* irqchip/gic-v4.1: Drop 'tmp' in inherit_vpe_l1_table_from_rd()Zenghui Yu2020-02-081-3/+1
| | | | | | | | | The variable 'tmp' in inherit_vpe_l1_table_from_rd() is actually not needed, drop it. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-5-yuzenghui@huawei.com
* irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD levelZenghui Yu2020-02-081-0/+80
| | | | | | | | | | | | | | | | | | | | | In GICv4, we will ensure that level2 vPE table memory is allocated for the specified vpe_id on all v4 ITS, in its_alloc_vpe_table(). This still works well for the typical GICv4.1 implementation, where the new vPE table is shared between the ITSs and the RDs. To make it explicit, let us introduce allocate_vpe_l2_table() to make sure that the L2 tables are allocated on all v4.1 RDs. We're likely not need to allocate memory in it because the vPE table is shared and (L2 table is) already allocated at ITS level, except for the case where the ITS doesn't share anything (say SVPET == 0, practically unlikely but architecturally allowed). The implementation of allocate_vpe_l2_table() is mostly copied from its_alloc_table_entry(). Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-4-yuzenghui@huawei.com
* irqchip/gic-v4.1: Set vpe_l1_base for all redistributorsZenghui Yu2020-02-081-1/+4
| | | | | | | | | | | | | | | Currently, we will not set vpe_l1_page for the current RD if we can inherit the vPE configuration table from another RD (or ITS), which results in an inconsistency between RDs within the same CommonLPIAff group. Let's rename it to vpe_l1_base to indicate the base address of the vPE configuration table of this RD, and set it properly for *all* v4.1 redistributors. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-3-yuzenghui@huawei.com
* irqchip/gic-v4.1: Fix programming of GICR_VPROPBASER_4_1_SIZEZenghui Yu2020-02-081-1/+1
| | | | | | | | | | The Size field of GICv4.1 VPROPBASER register indicates number of pages minus one and together Page_Size and Size control the vPEID width. Let's respect this requirement of the architecture. Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20200206075711.1275-2-yuzenghui@huawei.com
* irqchip/gic-v3-its: Reference to its_invall_cmd descriptor when building INVALLZenghui Yu2020-02-031-1/+1
| | | | | | | | | | | | It looks like an obvious mistake to use its_mapc_cmd descriptor when building the INVALL command block. It so far worked by luck because both its_mapc_cmd.col and its_invall_cmd.col sit at the same offset of the ITS command descriptor, but we should not rely on it. Fixes: cc2d3216f53c ("irqchip: GICv3: ITS command queue") Signed-off-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191202071021.1251-1-yuzenghui@huawei.com
* irqchip/gic-v4.1: Allow direct invalidation of VLPIsMarc Zyngier2020-01-221-19/+36
| | | | | | | | | Just like for INVALL, GICv4.1 has grown a VPE-aware INVLPI register. Let's plumb it in and make use of the DirectLPI code in that case. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-16-maz@kernel.org
* irqchip/gic-v4.1: Suppress per-VLPI doorbellMarc Zyngier2020-01-221-2/+9
| | | | | | | | | | Since GICv4.1 gives us a per-VPE doorbell, avoid programming anything else on VMOVI/VMAPI/VMAPTI and on any other action that would have otherwise resulted in a per-VLPI doorbell to be programmed. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-15-maz@kernel.org
* irqchip/gic-v4.1: Add VPE INVALL callbackMarc Zyngier2020-01-221-0/+14
| | | | | | | | | | GICv4.1 redistributors have a VPE-aware INVALL register. Progress! We can now emulate a guest-requested INVALL without emiting a VINVALL command. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-14-maz@kernel.org
* irqchip/gic-v4.1: Add VPE eviction callbackMarc Zyngier2020-01-221-11/+42
| | | | | | | | | | | When descheduling a VPE, special care must be taken to tell the GIC about whether we want to receive a doorbell or not. This is a major improvement on GICv4.0, where the doorbell had to be separately enabled/disabled. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-13-maz@kernel.org
* irqchip/gic-v4.1: Add VPE residency callbackMarc Zyngier2020-01-221-0/+17
| | | | | | | | | | Making a VPE resident on GICv4.1 is pretty simple, as it is just a single write to the local redistributor. We just need extra information about which groups to enable, which the KVM code will have to provide. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-12-maz@kernel.org
* irqchip/gic-v4.1: Add mask/unmask doorbell callbacksMarc Zyngier2020-01-221-0/+73
| | | | | | | | | | | | masking/unmasking doorbells on GICv4.1 relies on a new INVDB command, which broadcasts the invalidation to all RDs. Implement the new command as well as the masking callbacks, and plug the whole thing into the v4.1 VPE irqchip. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-11-maz@kernel.org
* irqchip/gic-v4.1: Plumb skeletal VPE irqchipMarc Zyngier2020-01-221-1/+31
| | | | | | | | | | | Just like for GICv4.0, each VPE has its own doorbell interrupt, and thus an irqchip that manages them. Since the doorbell management is quite different on GICv4.1, let's introduce an almost empty irqchip the will get populated over the next new patches. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-10-maz@kernel.org
* irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVPMarc Zyngier2020-01-221-7/+33
| | | | | | | | | | | | | | With GICv4.1, VMOVP is extended to allow a default doorbell to be specified, as well as a validity bit for this doorbell. As an added bonus, VMOVP isn't required anymore of moving a VPE between redistributors that share the same affinity. Let's add this support to the VMOVP builder, and make sure we don't issue the command if we don't really need to. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-9-maz@kernel.org
* irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is setMarc Zyngier2020-01-221-1/+24
| | | | | | | | | | | | | | The infamous VPE proxy device isn't used with GICv4.1 because: - we can invalidate any LPI from the DirectLPI MMIO interface - the ITS and redistributors understand the life cycle of the doorbell, so we don't need to enable/disable it all the time So let's escape early from the proxy related functions. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-8-maz@kernel.org
* irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPPMarc Zyngier2020-01-221-5/+55
| | | | | | | | | | | | | | | | | | The ITS VMAPP command gains some new fields with GICv4.1: - a default doorbell, which allows a single doorbell to be used for all the VLPIs routed to a given VPE - a pointer to the configuration table (instead of having it in a register that gets context switched) - a flag indicating whether this is the first map or the last unmap for this particular VPE - a flag indicating whether the pending table is known to be zeroed, or not Plumb in the new fields in the VMAPP builder, and add the map/unmap refcounting so that the ITS can do the right thing. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-7-maz@kernel.org
* irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocationMarc Zyngier2020-01-221-3/+307
| | | | | | | | | | | | | | | | | | GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules. To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different. The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-6-maz@kernel.org
* irqchip/gic-v3: Add GICv4.1 VPEID size discoveryMarc Zyngier2020-01-221-1/+10
| | | | | | | | | | While GICv4.0 mandates 16 bit worth of VPEIDs, GICv4.1 allows smaller implementations to be built. Add the required glue to dynamically compute the limit. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191224111055.11836-3-maz@kernel.org
* irqchip/gic-v3-its: Fix get_vlpi_map() breakage with doorbellsMarc Zyngier2020-01-221-5/+6
| | | | | | | | | | | | | | | | | When updating an LPI configuration, get_vlpi_map() may be passed a irq_data structure relative to an ITS domain (the normal case) or one that is relative to the core GICv3 domain in the case of a GICv4 doorbell. In the latter case, special care must be take not to dereference the irq_chip data as an its_dev structure, as that isn't what is stored there. Instead, check *first* whether the IRQ is forwarded to a vcpu, and only then try to obtain the vlpi mapping. Fixes: c1d4d5cd203c ("irqchip/gic-v3-its: Add its_vlpi_map helpers") Signed-off-by: Marc Zyngier <maz@kernel.org> Reported-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20200122085609.658-1-yuzenghui@huawei.com
* irqchip/gic-v3-its: Make vlpi_lock a spinlockMarc Zyngier2019-11-101-9/+9
| | | | | | | | | | | The VLPI map is currently a mutex, and that's a bad idea as this lock can be taken in non-preemptible contexts. Convert it to a raw spinlock, and turn the memory allocation of the VLPI map to be atomic. Reported-by: Heyi Guo <guoheyi@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191108165805.3071-12-maz@kernel.org
* irqchip/gic-v3-its: Lock VLPI map array before translating itMarc Zyngier2019-11-101-2/+4
| | | | | | | | | Obtaining the mapping ivformation for a VLPI should always be done with the vlpi_lock for this device held. Otherwise, we expose ourselves to races against a concurrent unmap. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191108165805.3071-11-maz@kernel.org
* irqchip/gic-v3-its: Synchronise INT/CLEAR commands targetting a VLPI using VSYNCMarc Zyngier2019-11-101-4/+75
| | | | | | | | | | | | | | | | | We have so far always injected/cleared VLPIs using either INT+SYNC or CLEAR+SYNC sequences, but that's pretty wrong for two reasons: - SYNC only synchronises physical LPIs - The collection ID that for the associated LPI doesn't match the redistributor the vPE is associated with Instead, send an {INT,CLEAR}+VSYNC for forwarded LPIs, ensuring that the ITS synchronises against the virtual pending table. Reported-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191108165805.3071-10-maz@kernel.org
* irqchip/gic-v3-its: Synchronise INV command targetting a VLPI using VSYNCMarc Zyngier2019-11-101-1/+35
| | | | | | | | | | | | | | | | | We have so far alwways invalidated VLPIs usinc an INV+SYNC sequence, but that's pretty wrong for two reasons: - SYNC only synchronises physical LPIs - The collection ID that for the associated LPI doesn't match the redistributor the vPE is associated with Instead, send an INV+VSYNC for forwarded LPIs, ensuring that the ITS can properly synchronise the invalidation of VLPIs. Fixes: 015ec0386ab6 ("irqchip/gic-v3-its: Add VLPI configuration handling") Reported-by: Zenghui Yu <yuzenghui@huawei.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20191108165805.3071-9-maz@kernel.org
* irqchip/gic-v3-its: Add its_vlpi_map helpersMarc Zyngier2019-11-101-15/+32
| | | | | | | | | | | Obtaining the mapping information for a VLPI is something quite common, and the GICv4.1 code is going to make even more use of it. Expose it as a separate set of helpers. Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Zenghui Yu <yuzenghui@huawei.com> Link: https://lore.kernel.org/r/20191027144234.8395-8-maz@kernel.org Link: https://lore.kernel.org/r/20191108165805.3071-8-maz@kernel.org
View Lorry Controller Status