Fix potential race condition on node power on and reboot

Currently there is the possibility that a configuration job
does not transition to the correct state and start execution
during a power on or reboot operation.  If the boot device
is being changed, the system might complete its POST before
the job is ready, leaving the job in the queue, and the
system will boot from the wrong device.

This fix suggests the following change for the power control:

  o During internal call to set boot device, poll the iDRAC job
    queue for a configurable amount of time to ensure the job
    is in the correct state
  o Throw an exception on timeout if the job does not transition
    to the correct state
  o Proceed with normal logic as soon as the job is in the
    correct state

This will ensure the iDRAC is in a state to execute the job
prior to the reboot starting, removing this race condition.

Story: #2004909
Task: #29259

Change-Id: I5cc71fb3c9a7e0166aab5bd458bbd257cefa8f5b

1 files changed, 8 insertions, 1 deletions

diff --git a/ironic/conf/drac.py b/ironic/conf/drac.py
index f132574be..7696bb03f 100644
--- a/ironic/conf/drac.py
+++ b/ironic/conf/drac.py
@@ -21,7 +21,14 @@ opts = [
                min=1,
                help=_('Interval (in seconds) between periodic RAID job status '
                       'checks to determine whether the asynchronous RAID '
-                      'configuration was successfully finished or not.'))
+                      'configuration was successfully finished or not.')),
+    cfg.IntOpt('boot_device_job_status_timeout',
+               default=30,
+               min=1,
+               help=_('Maximum amount of time (in seconds) to wait for '
+                      'the boot device configuration job to transition '
+                      'to the correct state to allow a reboot or power '
+                      'on to complete.'))
 ]