Train only - Fix py3 support for bootloader default config load

When the support to load the default bootloader configuration was
backported to stable train, I modified it to pass python2 unit tests
as well as ultimately attempt to be compatible.

It might have actually worked! What did not work for Python3 when the
UTF-16 codepoint delimiter file was encountered in a production case.
This passed CI originally just fine, however Ubuntu images do not have
the delimiter codepoint, where as RHEL images *do* have a UTF-16
delimiter at the start of the file.

On python3, with the python2 compatible code, the file was being opened
as ascii or UTF-8, which we know as the same rule applies for both. All
code points must be <0xFF. This resulted in the file open itself to fail,
as encoding support differs between Python2 and Python3.

What this patch does now is separately handle the case of if python2,
to use the new preview python3 io subsystem previewed in python2 to
open the file properly, and present the UTF-16 encoded results.

This issue was identified in RHOSP testing with python 3.6, with
partition images where the deployment would fail with reporting
that the ascii codec could not decode 0xFF which indicates that the
contents are being handled as something other than utf-16, and it
is hitting a constraint in string handling in Python where ascii and
utf-8 character code points are expected to always be <0xFF.

Story: 2009020
Task: 42742
rhbz#1977417

Change-Id: I08664c2b3900448633837e40098a0ff9f0bd65f7

3 files changed, 16 insertions, 2 deletions

diff --git a/ironic_python_agent/extensions/image.py b/ironic_python_agent/extensions/image.py
index e500fb29..f581756f 100644
--- a/ironic_python_agent/extensions/image.py
+++ b/ironic_python_agent/extensions/image.py
@@ -13,6 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import io
 import os
 import re
 import shlex
@@ -23,6 +24,7 @@ import tempfile
 from oslo_concurrency import processutils
 from oslo_config import cfg
 from oslo_log import log
+import six
 
 from ironic_python_agent import errors
 from ironic_python_agent.extensions import base
@@ -277,7 +279,12 @@ def _run_efibootmgr(valid_efi_bootloaders, device, efi_partition,
         if 'csv' in v_bl.lower():
             # These files are always UTF-16 encoded, sometimes have a header.
             # Positive bonus is python silently drops the FEFF header.
-            with open(mount_point + '/' + v_bl, 'rt') as csv:
+            if six.PY2:
+                open_call = io.open
+            else:
+                open_call = open
+            with open_call(mount_point + '/' + v_bl, 'r',
+                           encoding='utf-16') as csv:
                 contents = str(csv.read())
             csv_contents = contents.split(',')
             csv_filename = v_bl.split('/')[-1]
diff --git a/ironic_python_agent/tests/unit/extensions/test_image.py b/ironic_python_agent/tests/unit/extensions/test_image.py
index 0ab6c355..8807b889 100644
--- a/ironic_python_agent/tests/unit/extensions/test_image.py
+++ b/ironic_python_agent/tests/unit/extensions/test_image.py
@@ -13,6 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import io
 import os
 import shutil
 import sys
@@ -1707,7 +1708,7 @@ efibootmgr: ** Warning ** : Boot0005 has same label ironic1\n
                             mock.mock_open(read_data=csv_file_data)):
                 result = image._manage_uefi(self.fake_dev, self.fake_root_uuid)
         else:
-            with mock.patch.object(image, 'open',
+            with mock.patch.object(io, 'open',
                                    mock.mock_open(read_data=csv_file_data)):
                 result = image._manage_uefi(self.fake_dev, self.fake_root_uuid)
         self.assertTrue(result)
diff --git a/releasenotes/notes/train-bootloader-csv-file-load-fix-for-py3-e04001cecb8ca8e8.yaml b/releasenotes/notes/train-bootloader-csv-file-load-fix-for-py3-e04001cecb8ca8e8.yaml
new file mode 100644
index 00000000..f07b235b
--- /dev/null
+++ b/releasenotes/notes/train-bootloader-csv-file-load-fix-for-py3-e04001cecb8ca8e8.yaml
@@ -0,0 +1,6 @@
+---
+fixes:
+  - |
+    Fixes Python3 based support for loading files as file loading of bootloader
+    configuration files requires an explicit open operation with an unicode
+    indicator, which was inadvertently broken in backporting for Python2 support.