Revert the logic of exception_cb in read_url.

In commit e9e8616, there was an inversion of the logic of the
exception_cb return value meaning, breaking the (network) OpenStack
DataSource, which implemented exception_cb as should_retry_cb, returning
True when a retry should be done and False when the retry loop should
be broken and the exception reraised again immediately.

The OpenStack DS was the only user of this callback at the time and not
touched by the commit (nor did the commit message mention an intended
change), so this almost certainly happened by mistake.

These days, we have a second user of the callback in DataSourceScaleway.
It uses the new logic, so it needs change if we fix the meaning of the
return value.

This patch reverts the meaning of url_helper.read_url() execption_cb
to the old semantics. It updates the comment and adjusts the Scaleway
datasource.

The patch has been tested on Open Telekom Cloud (which uses the
OpenStack network Datasource) where previously a missing user_data
and network_data.json would be retried 6 times each despite them
not being present (they are optional!) and the server repsonding
with a correct 404. After the patch, boot times are 10s faster,
as we no longer pointlessly retry these files.

LP: #1702160
LP: #1298921

1 files changed, 4 insertions, 3 deletions

diff --git a/cloudinit/url_helper.py b/cloudinit/url_helper.py
index 4e814a5f..36289af5 100644
--- a/cloudinit/url_helper.py
+++ b/cloudinit/url_helper.py
@@ -258,9 +258,10 @@ def readurl(url, data=None, timeout=None, retries=0, sec_between=1,
                     # ssl exceptions are not going to get fixed by waiting a
                     # few seconds
                     break
-            if exception_cb and exception_cb(req_args.copy(), excps[-1]):
-                # if an exception callback was given it should return None
-                # a true-ish value means to break and re-raise the exception
+            if exception_cb and not exception_cb(req_args.copy(), excps[-1]):
+                # if an exception callback was given, it should return True
+                # to continue retrying and False to break and re-raise the
+                # exception
                 break
             if i + 1 < manual_tries and sec_between > 0:
                 LOG.debug("Please wait %s seconds while we wait to try again",