Support nodes setting 'auto' python-path

The nodepool "python-path" config variable makes it's way through from
the node arguments and ends up as the "ansible_python_interpreter"
variable for the inventory when running the job.

Notably, Python 3 only distributions require this to be set to
/usr/bin/python3 to avoid what can often be confusing red-herring
errors (e.g. things like dnf packages incorrectly appearing to be
missing on Fedora, for example [1]).

Upstream is aware of this often confusing behaviour and has made an
"ansible_python_interpreter" value of "auto" to, essentially, "do the
right thing" [2] and choose the right python for the target
environment.  This is available in Ansible >=2.8 and will become
default in 2.12.

This allows, and defaults to, an interpreter value of "auto" when
running with Ansible >=2.8.  On the supported prior Ansible releases,
"auto" will be translated into "/usr/bin/python2" to maintain
backwards compatability.  Of course a node explicity setting
"python-path" already will override this.

Nodepool is updated to set this by default with
I02a1a618c8806b150049e91b644ec3c0cb826ba4.

I think this is much more user friendly as it puts the work of
figuring out what platform has what interpreter into Ansible.  It
alleviates the need for admins to know anything at all about
"python-path" for node configurations unless they are actually doing
something out of the ordinary like using a virtualenv.  At the moment,
if you put a modern Python-3 only distro into nodepool, Zuul always
does the wrong thing by selecting /usr/bin/python2; you are left to
debug the failures and need to know to go and manually update the
python-path to Python 3.

Documentation is updated.  Detailed discussion is moved into the
executor section; the README is simplified a bit to avoid confusion.

A release note is added.

A test-case is added.  Note that it is also self-testing in that jobs
using Ansible 2.8 use the updated value
(c.f. I7cdcfc760975871f7fa9949da1015d7cec92ee67)

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1696404
[2] https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html

Change-Id: I2b3bc6d4f873b7d653cfaccd1598464583c561e7

1 files changed, 51 insertions, 0 deletions

diff --git a/tests/unit/test_inventory.py b/tests/unit/test_inventory.py
index 65944689d..ef10ee455 100644
--- a/tests/unit/test_inventory.py
+++ b/tests/unit/test_inventory.py
@@ -76,6 +76,57 @@ class TestInventoryPythonPath(TestInventoryBase):
         self.waitUntilSettled()
 
 
+class TestInventoryAutoPython(TestInventoryBase):
+
+    def test_auto_python_ansible28_inventory(self):
+        inventory = self._get_build_inventory('ansible-version28-inventory')
+
+        all_nodes = ('ubuntu-xenial',)
+        self.assertIn('all', inventory)
+        self.assertIn('hosts', inventory['all'])
+        self.assertIn('vars', inventory['all'])
+        for node_name in all_nodes:
+            self.assertIn(node_name, inventory['all']['hosts'])
+            node_vars = inventory['all']['hosts'][node_name]
+            self.assertEqual(
+                'auto', node_vars['ansible_python_interpreter'])
+
+        self.assertIn('zuul', inventory['all']['vars'])
+        z_vars = inventory['all']['vars']['zuul']
+        self.assertIn('executor', z_vars)
+        self.assertIn('src_root', z_vars['executor'])
+        self.assertIn('job', z_vars)
+        self.assertEqual(z_vars['job'], 'ansible-version28-inventory')
+        self.assertEqual(z_vars['message'], 'QQ==')
+
+        self.executor_server.release()
+        self.waitUntilSettled()
+
+    def test_auto_python_ansible27_inventory(self):
+        inventory = self._get_build_inventory('ansible-version27-inventory')
+
+        all_nodes = ('ubuntu-xenial',)
+        self.assertIn('all', inventory)
+        self.assertIn('hosts', inventory['all'])
+        self.assertIn('vars', inventory['all'])
+        for node_name in all_nodes:
+            self.assertIn(node_name, inventory['all']['hosts'])
+            node_vars = inventory['all']['hosts'][node_name]
+            self.assertEqual(
+                '/usr/bin/python2', node_vars['ansible_python_interpreter'])
+
+        self.assertIn('zuul', inventory['all']['vars'])
+        z_vars = inventory['all']['vars']['zuul']
+        self.assertIn('executor', z_vars)
+        self.assertIn('src_root', z_vars['executor'])
+        self.assertIn('job', z_vars)
+        self.assertEqual(z_vars['job'], 'ansible-version27-inventory')
+        self.assertEqual(z_vars['message'], 'QQ==')
+
+        self.executor_server.release()
+        self.waitUntilSettled()
+
+
 class TestInventory(TestInventoryBase):
 
     def test_single_inventory(self):