This change updates the cpu and ram initial
allocation ratios to 4.0 and 1.0 to reflect
the typical values that are suitable for non web
hosting workloads.

Change-Id: I283eb270a4e47da15cf01d098b4f3952f7b5b570
implements: bp/nova-change-default-overcommit-values

In py310 unittest.mock does not allow to mock the same function twice as
the second mocking will fail to autospec the Mock object created by the
first mocking.

This patch manually fixes the double mocking.

Fixed cases:
1) one of the mock was totally unnecessary so it was removed
2) the second mock specialized the behavior of the first generic mock.
   In this case the second mock is replaced with the configuration of
   the first mock
3) a test case with two test steps mocked the same function for each
   step with overlapping mocks. Here the overlap was removed to have
   the two mock exists independently

The get_connection injection in the libvirt functional test needed a
further tweak (yeah I know it has many already) to act like a single
mock (basically case #2) instead of a temporary re-mocking. Still the
globalness of the get_connection mocking warrant the special set / reset
logic there.

Change-Id: I3998d0d49583806ac1c3ae64f1b1fe343cefd20d

Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:

nova/tests/functional/regressions/test_bug_1781286.py
  We need to avoid using 'fixtures.MockPatch' since fixtures is using
  'mock' (the library) under the hood and a call to 'mock.patch.stop'
  found in that test will now "stop" mocks from the wrong library. We
  have discussed making this configurable but the option proposed isn't
  that pretty [1] so this is better.

The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:

  import glob

  for path in glob.glob('nova/tests/**/*.py', recursive=True):
      with open(path) as fh:
          lines = fh.readlines()
      if 'import mock\n' not in lines:
          continue
      import_group_found = False
      create_first_party_group = False
      for num, line in enumerate(lines):
          line = line.strip()
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              for lib in (
                  'ddt', 'six', 'webob', 'fixtures', 'testtools'
                  'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
              ):
                  if lib in tokens[1]:
                      create_first_party_group = True
                      break
              if create_first_party_group:
                  break
              import_group_found = True
          if not import_group_found:
              continue
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              if tokens[1] > 'unittest':
                  break
              elif tokens[1] == 'unittest' and (
                  len(tokens) == 2 or tokens[4] > 'mock'
              ):
                  break
          elif not line:
              break
      if create_first_party_group:
          lines.insert(num, 'from unittest import mock\n\n')
      else:
          lines.insert(num, 'from unittest import mock\n')
      del lines[lines.index('import mock\n')]
      with open(path, 'w+') as fh:
          fh.writelines(lines)

Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.

[1] https://github.com/testing-cabal/fixtures/pull/49

Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This change fixes some of the typos in unit tests as well
as in nova code-base.

Change-Id: I209bbb270baf889fcb2b9a4d1ce0ab4a962d0d0e

The vmwareapi driver uses Managed-Object references throughout the code
with the assumption that they are stable. It is however a database id,
which may change during the runtime of the compute node. e.g. If an
instance is unregistered and re-registerd in the vcenter, the moref will
change. By wrapping a moref in a proxy object, with an additional method
to resolve the openstack object to a moref, we can hide those changes
from a caller.

MoRef implementation with closure - should ease the transition to stable
mo-refs One simply has to pass the search function as a closure to the
MoRef instance, and the very same method will be called when an
exception is raised for the stored reference.

Stable Volume refs - The connection_info['data'] contains the
managed-object reference (moref) as well as the uuid of the volume.
When the moref become invalid for some reason, we can recover it by
searching for the volume-uuid as the `config.instanceUuid` attribute
of the shadow-vm.

Stable VM Ref - By encapsulating all the parameters for searching for
the vm-ref again, we can move the retry logic to the session object,
where we can try to recover the vm-ref should it result in a
ManagedObjectNotFound exception.

Use refs as index for fakedb -  It was previously using the object-id
to lookup an object, meaning that you couldn't pass a newly created
Managed-object-reference like you could over the vmware-api. Now the
lookup happens over the ref-id string, and in turn some functions
were refactored to take that into account.

Partial-Bug: #1962771

Change-Id: I2a3ddf95b7fe07630855b06e732f8764efb13e91

The VMwareAPISession object is not only used by the driver, but in
practically all modules of vmwareapi. It reduces a bit the scope of
the driver module itself.

Partial-Bug: #1962771

Change-Id: I4094b6031872bd3b5c871b9a82c7e01280a3352d

If instance memory is not multiple of 4, creating instance on ESXi
fails with error "['GenericVmConfigFault'] VimFaultException: Memory
(RAM) size is invalid.". However this is after instance is built and
tried to launch on ESXi. Add check in prepare_for_spawn to trigger
failure early and avoid further steps i.e. build as well as launch.

Closes-Bug: #1966987
Change-Id: I7ed8ac986283cd455e54e3f18ab955f43b3248d0

vSphere 6.5 introduced APIs to manage virtual disks (volumes)
as first class objects. The new managed disk entity is called
VStorageObject aka First Class Disk (FCD). Adding support for
volumes backed by VStorageObject.

Change-Id: I4a5a9d3537dc175508f0a0fd82507c498737d1a5