Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:

nova/tests/functional/regressions/test_bug_1781286.py
  We need to avoid using 'fixtures.MockPatch' since fixtures is using
  'mock' (the library) under the hood and a call to 'mock.patch.stop'
  found in that test will now "stop" mocks from the wrong library. We
  have discussed making this configurable but the option proposed isn't
  that pretty [1] so this is better.

The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:

  import glob

  for path in glob.glob('nova/tests/**/*.py', recursive=True):
      with open(path) as fh:
          lines = fh.readlines()
      if 'import mock\n' not in lines:
          continue
      import_group_found = False
      create_first_party_group = False
      for num, line in enumerate(lines):
          line = line.strip()
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              for lib in (
                  'ddt', 'six', 'webob', 'fixtures', 'testtools'
                  'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
              ):
                  if lib in tokens[1]:
                      create_first_party_group = True
                      break
              if create_first_party_group:
                  break
              import_group_found = True
          if not import_group_found:
              continue
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              if tokens[1] > 'unittest':
                  break
              elif tokens[1] == 'unittest' and (
                  len(tokens) == 2 or tokens[4] > 'mock'
              ):
                  break
          elif not line:
              break
      if create_first_party_group:
          lines.insert(num, 'from unittest import mock\n\n')
      else:
          lines.insert(num, 'from unittest import mock\n')
      del lines[lines.index('import mock\n')]
      with open(path, 'w+') as fh:
          fh.writelines(lines)

Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.

[1] https://github.com/testing-cabal/fixtures/pull/49

Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.

This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.

This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.

The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.

Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d

Merge these, removing an unnecessary layer of abstraction, and place
them in the new 'nova.db.main' directory. The resulting change is huge,
but it's mainly the result of 's/sqlalchemy import api/main import api/'
and 's/nova.db.api/nova.db.main.api/' with some necessary cleanup. We
also need to rework how we do the blocking of API calls since we no
longer have a 'DBAPI' object that we can monkey patch as we were doing
before. This is now done via a global variable that is set by the 'main'
function of 'nova.cmd.compute'.

The main impact of this change is that it's no longer possible to set
'[database] use_db_reconnect' and have all APIs automatically wrapped in
a DB retry. Seeing as this behavior is experimental, isn't applied to
any of the API DB methods (which don't use oslo.db's 'DBAPI' helper),
and is used explicitly in what would appear to be the critical cases
(via the explicit 'oslo_db.api.wrap_db_retry' decorator), this doesn't
seem like a huge loss.

Change-Id: Iad2e4da4546b80a016e477577d23accb2606a6e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

Replace the following items with Python 3 style code.

- six.moves.configparser
- six.moves.StringIO
- six.moves.cStringIO
- six.moves.urllib
- six.moves.builtins
- six.moves.range
- six.moves.xmlrpc_client
- six.moves.http_client
- six.moves.http_cookies
- six.moves.queue
- six.moves.zip
- six.moves.reload_module
- six.StringIO
- six.BytesIO

Subsequent patches will replace other six usages.

Change-Id: Ib2c406327fef2fb4868d8050fc476a7d17706e23
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>

It seems that with certain host configuration the ipv6 wsgi tests are
failing as the test only skipped if no ipv6 interface is available but
it does not check if the test will be able to bind to a port on that
interface. So this patch extends the skip criteria by checking if
binding is possible.

Change-Id: Iffea0586f02e8d4020cadc8c513bafe984e07a6e
Closes-Bug: #1883885

Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.

Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri

Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96

Convert the remaining few unit test that aren't specific to nova-network
to "use neutron". In most cases, this simply means dropping unnecessary
'use_neutron=True' flag overrides, though there are some additional
things that need to be done.

Change-Id: I3d30fc9f823b02a1651646a01ad83b5c3e781325
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

As stated in a TODO ItemMatcher returns True on [1, 1, 2] ?= [1, 2, 2].
This patch changes the internal implementation from set() to
collection.Counter to remove this false positive.

Change-Id: I1e8fccf3967dfcb464674d9e2d8f7d93eb015a83

When an argument somewhere in a mock call was a list, but the order of
that list was unpredictable (usually because it came from a set/dict), I
got sick of having to do:

  my_mock.assert_called_once_with(1, mock.ANY, 3)
  self.assertItemsEqual(expected_list, my_mock.call_args[0][1])

This commit adds a little helper so I can instead do:

  my_mock.assert_called_once_with(1, ItemsMatcher(expected_list), 3)

...and uses it in a few (by no means all) appropriate places.

Change-Id: I1d4d64650d1fa39d247815c7dbd8d50bf55b222d

This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.

'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.

Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>