diff options
| author | John Garbutt <john.garbutt@stackhpc.com> | 2022-11-16 17:12:40 +0000 |
|---|---|---|
| committer | John Garbutt <john.garbutt@stackhpc.com> | 2022-11-17 14:09:08 +0000 |
| commit | 3c022e968375c1b2eadf3c2dd7190b9434c6d4c1 (patch) | |
| tree | a03e26c5c3d048b4df5edde704c610c4e8a01377 /releasenotes | |
| parent | 2eb358cdcec36fcfe5388ce6982d2961ca949d0a (diff) | |
| download | nova-3c022e968375c1b2eadf3c2dd7190b9434c6d4c1.tar.gz | |
Ironic nodes with instance reserved in placement
Currently, when you delete an ironic instance, we trigger
and undeploy in ironic and we release our allocation in placement.
We do this well before the ironic node is actually available.
We have attempted to fix this my marking unavailable nodes
as reserved in placement. This works great until you try
and re-image lots of nodes.
It turns out, ironic nodes that are waiting for their automatic
clean to finish, are returned as a valid allocation candidates
for quite some time. Eventually we mark then as reserved.
This patch takes a strange approach, if we mark all nodes as
reserved as soon as the instance lands, we close the race.
That is, when the allocation is removed the node is still
unavailable until the next update of placement is done and
notices that the node has become available. That may or may
not have been after automatic cleaning. The trade off is
that when you don't have automatic cleaning, we wait a bit
longer to notice the node is available again.
Note, this is also useful when a broken Ironic node is
marked as in-maintainance while it is in-use by a nova
instance. In a similar way, we mark the Nova as reserved
immmeidately, rather than first waiting for the instance to be
deleted before reserving the resources in Placement.
Closes-Bug: #1974070
Change-Id: Iab92124b5776a799c7f90d07281d28fcf191c8fe
Diffstat (limited to 'releasenotes')
| -rw-r--r-- | releasenotes/notes/fix-ironic-scheduler-race-08cf8aba0365f512.yaml | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/releasenotes/notes/fix-ironic-scheduler-race-08cf8aba0365f512.yaml b/releasenotes/notes/fix-ironic-scheduler-race-08cf8aba0365f512.yaml new file mode 100644 index 0000000000..4fd2cc1ca9 --- /dev/null +++ b/releasenotes/notes/fix-ironic-scheduler-race-08cf8aba0365f512.yaml @@ -0,0 +1,11 @@ +--- +fixes: + - | + Fixed when placement returns ironic nodes that have just started automatic + cleaning as possible valid candidates. This is done by marking all ironic + nodes with an instance on them as reserved, such that nova only makes them + available once we have double checked Ironic reports the node as available. + If you don't have automatic cleaning on, this might mean it takes longer + than normal for Ironic nodes to become available for new instances. + If you want the old behaviour use the following workaround config: + `[workarounds]skip_reserve_in_use_ironic_nodes=true` |