Cells: Add the main code.

This introduces *EXPERIMENTAL* compute cells functionality as a way to
scale nova in a more distributed fashion without having to use complicated
technologies like DB and message queue clustering.

Cells are configured as a tree and the top level cell should contain
nova-api without any nova-computes while child cells contain everything
except nova-api.  One can think of a cell as a normal nova deployment in
that each cell has its own DB server and message queue broker.

The top level cell keeps a subset of data about ALL instances in all
cells in its DB.  Child cells send messages to the top level cell when
instances change state.  Data in 1 child cell is not shared with another
child cell.

A new service, nova-cells, is introduced that handles communication
between cells and picking of a cell for new instances.  This service is
required for every cell.  Communication between cells is pluggable with
the only option currently implemented being communnication via RPC.

Cells scheduling is separate from host scheduling.  nova-cells first picks
a cell (currently randomly -- future patches add filtering/weighing
functionality and decisions can be based on broadcasts of
capacity/capabilities).  Once a cell has been selected and the new build
request has reached its nova-cells service, it'll be sent over to the host
scheduler in that cell and the build proceeds as it does without cells.

New config options are introduced for enabling and configuring the cells
code.  Cells is disabled by default.  All of the config options below go
under a '[cells]' section in nova.conf.  These are the options that one
may want to tweak:

enable -- Turn on cells code (default is False)
name -- Name of the current cell.
capabilities -- List of arbitrary key=value pairs defining capabilities
                of the current cell.  These are sent to parent cells,
                but aren't used in scheduling until later filter/weight
                support is added.
call_timeout -- How long to wait for replies from a calls between cells

When using cells, the compute API class must be changed in the API cell,
so that requests can be proxied via nova-cells down to the correct cell
properly.  Thus, config requirements for API cell:

--
[DEFAULT]
compute_api_class=nova.compute.cells_api.ComputeCellsAPI.
[cells]
enable=True
name=api-cell
--

Config requirements for child cell:

--
[cells]
enable=True
name=child-cell1
--

Another requirement is populating the 'cells' DB table in each cell.
Each cell needs to know about its parent and children and how to
communicate with them (message broker location, credentials, etc).

Implements blueprint nova-compute-cells

DocImpact

Change-Id: I1b52788ea9d7753365d175abf39bdbc22ba822fe

1 files changed, 1 insertions, 0 deletions

diff --git a/setup.py b/setup.py
index f3da546187..e13ae4f64e 100644
--- a/setup.py
+++ b/setup.py
@@ -50,6 +50,7 @@ setuptools.setup(name='nova',
                'bin/nova-api-metadata',
                'bin/nova-api-os-compute',
                'bin/nova-rpc-zmq-receiver',
+               'bin/nova-cells',
                'bin/nova-cert',
                'bin/nova-clear-rabbit-queues',
                'bin/nova-compute',