summaryrefslogtreecommitdiff
path: root/nova/virt
Commit message (Collapse)AuthorAgeFilesLines
* Use force=True for os-brick disconnect during deletemelanie witt2023-05-1014-33/+40
| | | | | | | | | | | | | | | | | | | The 'force' parameter of os-brick's disconnect_volume() method allows callers to ignore flushing errors and ensure that devices are being removed from the host. We should use force=True when we are going to delete an instance to avoid leaving leftover devices connected to the compute host which could then potentially be reused to map to volumes to an instance that should not have access to those volumes. We can use force=True even when disconnecting a volume that will not be deleted on termination because os-brick will always attempt to flush and disconnect gracefully before forcefully removing devices. Closes-Bug: #2004555 Change-Id: I3629b84d3255a8fe9d8a7cea8c6131d7c40899e8
* Merge "Handle zero pinned CPU in a cell with mixed policy"Zuul2023-05-091-4/+6
|\
| * Handle zero pinned CPU in a cell with mixed policyBalazs Gibizer2022-12-131-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When cpu_policy is mixed the scheduler tries to find a valid CPU pinning for each instance NUMA cell. However if there is an instance NUMA cell that does not request any pinned CPUs then such logic will calculate empty pinning information for that cell. Then the scheduler logic wrongly assumes that an empty pinning result means there was no valid pinning. However there is difference between a None result when no valid pinning found, from an empty result [] which means there was nothing to pin. This patch makes sure that pinning == None is differentiated from pinning == []. Closes-Bug: #1994526 Change-Id: I5a35a45abfcfbbb858a94927853777f112e73e5b
* | Merge "Have host look for CPU controller of cgroupsv2 location."Zuul2023-05-041-1/+30
|\ \
| * | Have host look for CPU controller of cgroupsv2 location.Jorge San Emeterio2023-05-031-1/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make the host class look under '/sys/fs/cgroup/cgroup.controllers' for support of the cpu controller. The host will try searching through cgroupsv1 first, just like up until now, and in the case that fails, it will try cgroupsv2 then. The host will not support the feature if both checks fail. This new check needs to be mocked by all tests that focus on this piece of code, as it touches a system file that requires privileges. For such thing, the CGroupsFixture is defined to easily add suck mocking to all test cases that require so. I also removed old mocking at test_driver.py in favor of the fixture from above. Partial-Bug: #2008102 Change-Id: I99b57c27c8a4425389bec2b7f05af660bab85610
* | | Save cell socket correctly when updating host NUMA topologyArtom Lifshitz2023-04-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, in numa_usage_from_instance_numa(), any new NUMACell objects we created did not have the `socket` attribute. In some cases this was persisted all the way down to the database. Fix this by copying `socket` from the old_cell. Change-Id: I9ed3c31ccd3220b02d951fc6dbc5ea049a240a68 Closes-Bug: 1995153
* | | Merge "hyperv: Mark driver as experimental"Zuul2023-04-191-0/+8
|\ \ \
| * | | hyperv: Mark driver as experimentalStephen Finucane2022-11-071-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cloudbase have changed priorities and will no longer be testing the Hyper-V driver. We need to mark this as experimental and consider removing it in the future. Change-Id: I823fbf660948c062581d4e0aaaadc6a6983de2a3 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* | | | mypy: Fix implicit optional usageEric Harney2023-03-273-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current versions of mypy run with no-implicit-optional by default. This change gets Nova's mypy test environment to pass again. Change-Id: Ie50c8d364ad9c339355cc138b560ec4df14fe307
* | | | Merge "fup for power management series"Zuul2023-03-092-23/+1
|\ \ \ \
| * | | | fup for power management seriesSylvain Bauza2023-02-232-23/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Emptying the cpu init file and directly calling the submodule API. Relates to blueprint libvirt-cpu-state-mgmt Change-Id: I1299ca4b49743f58bec6f541785dd9fbee0ae9e2
* | | | | Merge "Revert "Add logging to find test cases leaking libvirt threads""27.0.0.0rc127.0.0Zuul2023-03-041-18/+0
|\ \ \ \ \
| * | | | | Revert "Add logging to find test cases leaking libvirt threads"Sylvain Bauza2023-02-141-18/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 1778a9c589cf24e17b44f556680b17af9577df11. Reason for revert: We said we wouldn't have it in RC1. Change-Id: Idf0c9a8adeac231f099b312fc24b9cf9726687e0
* | | | | | Merge "Fix logging in MemEncryption-related checks"Zuul2023-02-281-4/+14
|\ \ \ \ \ \ | |_|/ / / / |/| | | | |
| * | | | | Fix logging in MemEncryption-related checksAlexey Stupnikov2023-02-111-4/+14
| | |_|/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently Nova produces ambigous error when volume-backed instance is started using flavor with hw:mem_encryption extra_specs flag: ImageMeta doesn't contain name if it represents Cinder volume. This fix sligtly changes steps to get image_meta.name for some MemEncryption-related checks where it could make any difference. Closes-bug: #2006952 Change-Id: Ia69e7cb18cd862f01ecfdbdc358c87af1ab8fbf6
* | | | | Merge "Enable cpus when an instance is spawning"Zuul2023-02-183-0/+118
|\ \ \ \ \
| * | | | | Enable cpus when an instance is spawningSylvain Bauza2023-02-103-0/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By this patch, we now automatically power down or up cores when an instance is either stopped or started. Also, by default, we now powersave or offline dedicated cores when starting the compute service. Implements: blueprint libvirt-cpu-state-mgmt Change-Id: Id645fd1ba909683af903f3b8f11c7f06db3401cb
* | | | | | Merge "libvirt: let CPUs be power managed"Zuul2023-02-182-5/+16
|\ \ \ \ \ \ | |/ / / / /
| * | | | | libvirt: let CPUs be power managedSylvain Bauza2023-02-102-5/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before going further, we need to somehow return the list of CPUs even offline if they are power managed by Nova. Co-Authored-By: Sean Mooney <smooney@redhat.com> Partially-Implements: blueprint libvirt-cpu-state-mgmt Change-Id: I5dca10acde0eff554ed139587aefaf2f5fad2ca5
* | | | | | Merge "libvirt: Add configuration options to set SPICE compression settings"Zuul2023-02-172-0/+29
|\ \ \ \ \ \
| * | | | | | libvirt: Add configuration options to set SPICE compression settingsManuel Bentele2023-01-112-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the following SPICE-related options to the 'spice' configuration group of a Nova configuration: - image_compression - jpeg_compression - zlib_compression - playback_compression - streaming_mode These configuration options can be used to enable and set the SPICE compression settings for libvirt (QEMU/KVM) provisioned instances. Each configuration option is optional and can be set explictly to configure the associated SPICE compression setting for libvirt. If all configuration options are not set, then none of the SPICE compression settings will be configured for libvirt, which corresponds to the behavior before this change. In this case, the built-in defaults from the libvirt backend (e.g. QEMU) are used. Note that those options are only taken into account if SPICE support is enabled (and the VNC support is disabled). Implements: blueprint nova-support-spice-compression-algorithm Change-Id: Ia7efeb1b1a04504721e1a5bdd1b5fa7a87cdb810
* | | | | | | Merge "cpu: interfaces for managing state and governor"Zuul2023-02-153-0/+156
|\ \ \ \ \ \ \ | | |/ / / / / | |/| | | | |
| * | | | | | cpu: interfaces for managing state and governorSylvain Bauza2023-02-093-0/+156
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first stage of the power management series. In order to be able to switch the CPU state or change the governor, we need a framework to access sysfs. As some bits can be reused, let's create a nova.filesystem helper module that will define read-write mechanisms for accessing sysfs-specific commands. Partially-Implements: blueprint libvirt-cpu-state-mgmt Change-Id: Icb913ed9be8d508de35e755a9c650ba25e45aca2
* | | | | | | Merge "Add logging to find test cases leaking libvirt threads"Zuul2023-02-141-0/+18
|\ \ \ \ \ \ \ | |_|_|/ / / / |/| | | / / / | | |_|/ / / | |/| | | |
| * | | | | Add logging to find test cases leaking libvirt threadsSylvain Bauza2023-02-101-0/+18
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We see functional test failures due to leaked libvirt event handling thread weaking up after its original test finished and importing libvirt. If it happens when the libvirt package import is poisoned then the currently executing test will fail. This patch logs the name of the test case that leaked the libvirt event handling thread. We will revert his before RC1. Change-Id: I3146e9afb411056d004fc118ccfa31126a3c6b15 Related-Bug: #1946339
* | | | | Stable compute uuid functional testsDan Smith2023-02-011-0/+20
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a number of functional test cases for the stable-compute-uuid error cases. Specifically around checks and aborted startups to make sure we're catching what we expect, and failing in the appropriate ways. Related to blueprint stable-compute-uuid Change-Id: I8bcb93a6887ed06dbd4b7c28c93a20a3705a6077
* | | | Check our nodes for hypervisor_hostname changesDan Smith2023-02-014-9/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we are loading our ComputeNode objects by UUID according to what the virt driver reported, we can sanity check the DB records against the virt driver's hostname. This covers the case where our CONF.host has not changed but the hostname reported by the virt driver has, assuming we can find the ComputeNode object(s) that match our persistent node uuid. Related to blueprint stable-compute-uuid Change-Id: I41635210d7d6f46b437b06d2570a26a80ed8676a
* | | | Merge "Add further workaround features for qemu_monitor_announce_self"Zuul2023-01-311-10/+31
|\ \ \ \
| * | | | Add further workaround features for qemu_monitor_announce_selfas02023-01-301-10/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases on Arista VXLAN fabrics, VMs are inaccessible via network after live migration, despite garps being observed on the fabric itself. This patch builds on the feature ``[workarounds]/enable_qemu_monitor_announce_self`` feature as reported in `bug 1815989 <https://bugs.launchpad.net/nova/+bug/1815989>` This patch adds the ability to config the number of times the QEMU announce_self monitor command is called, and add a new configuration option to specify a delay between calling the announce_self command multiple times, as in some cases, multiple announce_self monitor commands are required for the fabric to honor the garp packets and the VM to become accessible via the network after live migration. Closes-Bug: #1996995 Change-Id: I2f5bf7c9de621bb1dc7fae5b3374629a4fcc1f46
* | | | | Fixup patch for stable-compute-uuid seriesDan Smith2023-01-301-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Minor cleanups from feedback on earlier patches. Related to blueprint stable-compute-uuid Change-Id: I00505f1df47b46ed36645c781354258e255f0dcc
* | | | | Make resource tracker use UUIDs instead of namesDan Smith2023-01-302-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This makes the resource tracker look up and create ComputeNode objects by uuid instead of nodename. For drivers like ironic that already provide 'uuid' in the resources dict, we can use that. For those that do not, we force the uuid to be the locally-persisted node uuid, and use that to find/create the ComputeNode object. A (happy) side-effect of this is that if we find a deleted compute node object that matches that of our hypervisor, we undelete it instead of re-creating one with a new uuid, which may clash with our old one. This means we remove some of the special-casing of ironic rebalance, although the tests for that still largely stay the same. Change-Id: I6a582a38c302fd1554a49abc38cfeda7c324d911
* | | | | Merge "Add get_available_node_uuids() to virt driver"Zuul2023-01-275-3/+25
|\ \ \ \ \
| * | | | | Add get_available_node_uuids() to virt driverDan Smith2023-01-205-3/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a get_available_node_uuids() method to the virt driver interface. This aims to eventually replace the nodename-based interface, but currently provides an implementation that will work for most drivers. Any driver that does not override this method will get the locally-persistent UUID from nova.virt.node. Ironic obviously needs to override this (which is easy), as well as the fake driver because it supports multiple nodes for testing. The libvirt driver overrides it only because we test multiple libvirt driver instances on a single host and we need each instantiation of it to "capture" the UUID we have mocked out at the time it is started. Change-Id: Ibe14d2b223c737d82c217a74bc94e41603271a9d
* | | | | | Merge "Add virt/node module for stable uuids"Zuul2023-01-261-0/+107
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Add virt/node module for stable uuidsDan Smith2023-01-201-0/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Related to blueprint stable-compute-uuid Change-Id: Ie8897a843fadf325c696b411923f075e237a7342
* | | | | | Merge "libvirt: Replace usage of compareCPU() with compareHypervisorCPU()"Zuul2023-01-262-1/+17
|\ \ \ \ \ \
| * | | | | | libvirt: Replace usage of compareCPU() with compareHypervisorCPU()Kashyap Chamarthy2023-01-242-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The older libvirt API compareCPU() does not take into account the capabilities of the "host hypervisor" (KVM, QEMU and the details libvirt knows about the host) when comparing CPUs. This is causing unwanted failures during CPU compatibility checks. To fix this and other related problems, libvirt has introduced (in v4.4.0) a newer API, compareHypervisorCPU(), which _does_ take into account the host hypervisor's capabilities before comparing CPUs. This will help fix a range of problems, such as[1][2]. So let's switch to the newer API, which is largely a drop-in replacement. In this patch: - Introduce a wrapper method, compare_hypervisor_cpu() for libvirt's compareHypervisorCPU() API. - Update the _compare_cpu() method to use the wrapper method, compare_hypervisor_cpu(). - Update the unit tests to use the newer API, compareHypervisorCPU(). [1] https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1978064 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2138381 Change-Id: Ib523753f52993cfe72e35e0309e429ca879c125c Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
* | | | | | | Merge "libvirt: At start-up rework compareCPU() usage with a workaround"Zuul2023-01-261-27/+20
|\ \ \ \ \ \ \ | |/ / / / / /
| * | | | | | libvirt: At start-up rework compareCPU() usage with a workaroundKashyap Chamarthy2023-01-241-27/+20
| | |_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In this patch: - Remove the first compareCPU() call (called via the wrapper _compare_cpu()) in _check_cpu_compatibility(), and let libvirt handle it. (QEMU >=2.9 and libvirt >= 4.4.0 are the mininum required versions, and upstream Nova satisfies them by a good margin.) - Validate the user-configured CPU models from _get_cpu_model_mapping(). And take into account all the CPU flags before calling _compare_cpu(). (Suggested-by: Sean Mooney -- thanks!) - Add a workaround to allow skipping the remaining compareCPU() call in _check_cpu_compatibility() as a potential future-proof (because we cannot test all possible CPU models and hardware). Unlike the removed first call, this call takes into account the extra CPU flags provided by the user into account when evaluating guest CPU model compatibility. As a follow up comes the patch[1] that replaces the older libvirt CPU API with the newer one. [1] https://review.opendev.org/c/openstack/nova/+/869950 -- libvirt: Replace usage of compareCPU() with compareHypervisorCPU() Change-Id: I8ef9db851b37c5249d2efbe09a15a1ddbae8205d Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
* | | | | | Check VMDK create-type against an allowed listDan Smith2023-01-241-0/+31
| |/ / / / |/| | | | | | | | | | | | | | | | | | | Related-Bug: #1996188 Change-Id: I5a399f1d3d702bfb76c067893e9c924904c8c360
* | | | | Merge "libvirt: Report ephemeral encryption traits based on imagebackend"Zuul2023-01-142-1/+7
|\ \ \ \ \
| * | | | | libvirt: Report ephemeral encryption traits based on imagebackendLee Yarwood2022-10-102-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds ephemeral encryption capabilities to the libvirt driver and defaults image backends to SUPPORTS_LUKS = False. Image backends that support ephemeral encryption will set this attribute to True in later patches. Co-Authored-By: melanie witt <melwittt@gmail.com> Related to blueprint ephemeral-storage-encryption Change-Id: Ieee5805e28118fe72c2b0df6f6ceda3962431d49
* | | | | | Merge "libvirt: Add encryption support to qemu-img create command"Zuul2023-01-141-9/+60
|\ \ \ \ \ \ | |/ / / / / | | / / / / | |/ / / / |/| | | |
| * | | | libvirt: Add encryption support to qemu-img create commandLee Yarwood2022-10-101-9/+60
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds handling of options needed for qemu-img create to generate encrypted images. Co-Authored-By: melanie witt <melwittt@gmail.com> Related to blueprint ephemeral-storage-encryption Change-Id: I5d6d2a7b03b5ace0826af80c4004de852579ff12
* | | | Merge "Filter PCI pools based on Placement allocation"Zuul2022-12-201-1/+8
|\ \ \ \
| * | | | Filter PCI pools based on Placement allocationBalazs Gibizer2022-10-171-1/+8
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The stats module is used to decide if the InstancePCIRequests of a boot request can fit to a given compute host and to decide which PCI device pool can fulfill the requests. It is used both during scheduling and also during the PCI claim code. PCI devices now modelled in placement and the allocation_candidate query now requests PCI resources therefore each allocation candidate returned from placement already restricts which PCI devices can be used during the PciPassthroughFilter, the NumaTopologyFilter, and PCI claim code paths. This patch adapts the stats module to consider the PCI allocation candidate or the already made placement PCI allocation when filtering the PCI device pools. blueprint: pci-device-tracking-in-placement Change-Id: If363981c4aeeb09a96ee94b140070d3d0f6af48f
* | | | Merge "Improving logging at '_allocate_mdevs'."Zuul2022-12-201-0/+13
|\ \ \ \
| * | | | Improving logging at '_allocate_mdevs'.Jorge San Emeterio2022-10-131-0/+13
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding both 'info' and 'debug' messages with the intention of telling which mdevs are available, which get allocated and whether new ones are created. Closes-Bug: #1992451 Change-Id: Ibd331df51fd4eaeed4831a98469f06a4ce0cd452
* | | | Ironic: retry when node not availableJohn Garbutt2022-12-151-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After a baremetal instance is deleted, and its allocation is removed in placement, the ironic node might start cleaning. Eventually nova will notice and update the inventory to be reserved. During this window, a new instance may have already picked this ironic node. When that race happens today the build fails with an error: "Failed to reserve node ..." This change tries to ensure the remaining alternative hosts are attempted before aborting the build. Clearly the race is still there, but this makes it less painful. Related-Bug: #1974070 Change-Id: Ie5cdc17219c86927ab3769605808cb9d9fa9fa4d
* | | | Merge "Ironic nodes with instance reserved in placement"Zuul2022-12-141-8/+18
|\ \ \ \ | |_|_|/ |/| | |