To remove duplicated data this patch introduce common json fragments for
the instance related notifications and refactors the sample file of
instance.delete.start and instance.delete.end.

The common sample fragments define an active instance as that is the
most common state in the instance action notifications. This way
we can minimize the difference between the common sample and the
generated notification. Still there will be differences. For example
instance.delete.end notification will contain different instance status
and some other deletion related fields are also set. To make sure
the the sample matches with the generated notification we have to
override some of the fields coming from the common sample fragment.
Such overrides could be added in the test case directly to the
_verify_notification() call as it is done in case of dynamic values
like instance uuid. However the sample files not only used in the
functional tests but also during the generation of the notification
dev-ref. To have meaningful samples in the doc we have to do such
overrides during the reference resolving and not directly in the
test case.

Change-Id: I0669a075c94ba2a435116791e3391d8d745bc639

In I18e7483ec9a484a660e1d306fdc0986e1d5f952b BDM was added to the instance
notifications. In general to add BDM to the payload an exta DB query is
needed. However in some places the BDM is already separately before the
notify_about_instance_action is called to send the notification. In this cases
loading the BDM again is unnecessary as the already loaded BDM can be reused.

This patch makes sure that notify_about_instance_action is called with the
already loaded BDM. There will be subsequent patches to do the same with
other notify calls.

Change-Id: I391554d3904a5a60b921ef4714a1cfd0a64a25c2
Related-Bug: #1718226

Change-Id: Iddc5ec3b5053ddf5dce73c86edca26c4552d451b
Implements: bp versioned-notification-transformation-pike
Closes-Bug: #1684860

This patch adds bdm related data to InstancePayload. As loading BDM
can be expensive a new configuration flag 'bdms_in_notifications' is
introduced. If this is set to false (as by default) then the instance
notifications will contain 'null' value in block_devices field.
If this configuration flag is set to true then the block devices will
be loaded from the db and will be included in the notification payload.

Implements: bp additional-notification-fields-for-searchlight
Change-Id: I18e7483ec9a484a660e1d306fdc0986e1d5f952b

Add key_name field to InstancePayload which means every instance.<action>
notification now have key_name field based on the instance.key_name.

Change-Id: I39059c5077c790aa85dff6b4563f581c54536b86
Implements: bp additional-notification-fields-for-searchlight

Add auto_disk_config field to InstancePayload based on
instance.auto_disk_config boolean value. If true, set it to
AUTO else MANUAL.

Implements: bp additional-notification-fields-for-searchlight
Change-Id: I18af99479562e2fe5e74e6e1252b804b074fee58

This ensures that instance.availability_zone reflects what AZ the
instance is actually in. In the case where no AZ was requested at
boot, previously this would always be some default value, which
isn't as useful to the things that need to consider what AZ the
instance is actually in (without doing all the aggregate math to
determine it on the fly).

Related to blueprint cells-aware-api
Change-Id: I8d426f2635232ffc4b510548a905794ca88d7f99

Due to the not strict handling of uninitialized fields during
notification payload population it is possible that the emitted
notification missing some of the fields defined in the schema.

There are two problematic cases:
1) no load tirggered for lazy loaded fields. If the field was not
   loaded it is not added to the payload.
2) uninitialized, not lazy loadable fields are not added to the
   payload.

This patch makes sure that lazy load is triggered. If the field is
not lazy loadable and also not initialized then related payload
field will be set to None. If the payload field is not nullable
the code will fail to make sure that the inconsistency is detected.

The following changes cannot be split to different commits because
as soon as the generic schema population code is fixed every listed
notification starts behaving differently.

In some cases the availability_zone field of the Service object is left
unitialized by both the constructor and the obj_load_attr function this
caused that the availability_zone field of the service.update
notification was missing from the emitted notification payload.

The extra_specs field of the Flavor object is not lazy loadable and not
initialized in the Flavor destroy case. So the extra_specs field of the
FlavorPayload needed to be made nullable.

The projects field of the Flavor object is lazy loaded but when the
Flavor object is loaded as part of an Instance object Flavor is
orphaned and no lazy load is allowed on that Flavor object. In this
case the projects field of the FlavorPayload will be set to None to
signal that the information is not available. This also means that
the projects field of the FlavorPayload needed to be changed to
nullable.

The hosts and id fields of the Aggregate object is not initialized
during the create of the Aggregate before the aggregate.create.start
needs to be sent. Therefore these fields needs to be nullable.

Closes-Bug: #1653221
Change-Id: Ib122cd98ee0cc31938d5ff1d5c753053267a3bd4