From 00299dcaad9a42c7f81be62a3966c59b299dff1d Mon Sep 17 00:00:00 2001 From: Sylvain Bauza Date: Mon, 20 Feb 2023 22:02:59 +0100 Subject: Add docs for cpu management Relates to blueprint libvirt-cpu-state-mgmt Change-Id: Id22018a880b39f694a509c04cc6497ef42343049 --- doc/source/admin/cpu-topologies.rst | 91 +++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) (limited to 'doc') diff --git a/doc/source/admin/cpu-topologies.rst b/doc/source/admin/cpu-topologies.rst index 9770639c3a..082c88f655 100644 --- a/doc/source/admin/cpu-topologies.rst +++ b/doc/source/admin/cpu-topologies.rst @@ -730,6 +730,97 @@ CPU policy, meanwhile, will consume ``VCPU`` inventory. .. _configure-hyperv-numa: +Configuring CPU power management for dedicated cores +---------------------------------------------------- + +.. versionchanged:: 27.0.0 + + This feature was only introduced by the 2023.1 Antelope release + +.. important:: + + The functionality described below is currently only supported by the + libvirt/KVM driver. + +For power saving reasons, operators can decide to turn down the power usage of +CPU cores whether they are in use or not. For obvious reasons, Nova only allows +to change the power consumption of a dedicated CPU core and not a shared one. +Accordingly, usage of this feature relies on the reading of +:oslo.config:option:`compute.cpu_dedicated_set` config option to know which CPU +cores to handle. +The main action to enable the power management of dedicated cores is to set +:oslo.config:option:`libvirt.cpu_power_management` config option to ``True``. + +By default, if this option is enabled, Nova will lookup the dedicated cores and +power them down at the compute service startup. Then, once an instance starts +by being attached to a dedicated core, this below core will be powered up right +before the libvirt guest starts. On the other way, once an instance is stopped, +migrated or deleted, then the corresponding dedicated core will be powered down. + +There are two distinct strategies for powering up or down : + +- the default is to offline the CPU core and online it when needed. +- an alternative strategy is to use two distinct CPU governors for the up state + and the down state. + +The strategy can be chosen using +:oslo.config:option:`libvirt.cpu_power_management_strategy` config option. +``cpu_state`` supports the first online/offline strategy, while ``governor`` +sets the alternative strategy. +We default to turning off the cores as it provides you the best power savings +while there could be other tools outside Nova to manage the governor, like +tuned. That being said, we also provide a way to automatically change the +governors on the fly, as explained below. + +If the strategy is set to ``governor``, a couple of config options are provided +to define which exact CPU govenor to use for each of the up and down states : + +- :oslo.config:option:`libvirt.cpu_power_governor_low` will define the governor + to use for the powerdown state (defaults to ``powersave``) +- :oslo.config:option:`libvirt.cpu_power_governor_high` will define the + governor to use for the powerup state (defaults to ``performance``) + +.. important:: + This is the responsibility of the operator to ensure that the govenors + defined by the configuration options are currently supported by the OS + underlying kernel that runs the compute service. + + As a side note, we recommend the ``schedutil`` governor as an alternative for + the high-power state (if the kernel supports it) as the CPU frequency is + dynamically set based on CPU task states. Other governors may be worth to + be tested, including ``conservative`` and ``ondemand`` which are quite a bit + more power consuming than ``schedutil`` but more efficient than + ``performance``. See `Linux kernel docs`_ for further explanations. + +.. _`Linux kernel docs`: https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt + +As an example, a ``nova.conf`` part of configuration would look like:: + + [compute] + cpu_dedicated_set=2-17 + + [libvirt] + cpu_power_management=True + cpu_power_management_strategy=cpu_state + +.. warning:: + + The CPU core #0 has a special meaning in most of the recent Linux kernels. + This is always highly discouraged to use it for CPU pinning but please + refrain to have it power managed or you could have surprises if Nova turns + it off ! + +One last important note : you can decide to change the CPU management strategy +during the compute lifecycle, or you can currently already manage the CPU +states. For ensuring that Nova can correctly manage the CPU performances, we +added a couple of checks at startup that refuse to start nova-compute service +if those arbitrary rules aren't enforced : + +- if the operator opts for ``cpu_state`` strategy, then all dedicated CPU + governors *MUST* be identical. +- if they decide using ``governor``, then all dedicated CPU cores *MUST* be + online. + Configuring Hyper-V compute nodes for instance NUMA policies ------------------------------------------------------------ -- cgit v1.2.1