Add support for the audit middleware

This adds support for the audit middleware to Ironic, allowing
the middleware to send two notifications per API request,
one for the request and another for the response.
This adds an option to enable or disable audit middleware.
Also to properly audit API requests passing conf options
via audit map file.

AuditMiddleware docs:
   http://docs.openstack.org/developer/keystonemiddleware/audit.html

Co-Authored-By: Chris Krelle <nobodycam@gmail.com>

Closes-Bug: #1540232
Change-Id: I6de4751aa6b25e8457cae3eeab95a15f417662c5

11 files changed, 291 insertions, 0 deletions

diff --git a/doc/source/deploy/api-audit-support.rst b/doc/source/deploy/api-audit-support.rst
new file mode 100644
index 000000000..a83204c91
--- /dev/null
+++ b/doc/source/deploy/api-audit-support.rst
@@ -0,0 +1,110 @@
+.. _api-audit-support:
+
+API Audit Logging
+=================
+
+Audit middleware supports delivery of CADF audit events via Oslo messaging
+notifier capability. Based on `notification_driver` configuration, audit events
+can be routed to messaging infrastructure (notification_driver = messagingv2)
+or can be routed to a log file (notification_driver = log).
+
+Audit middleware creates two events per REST API interaction. First event has
+information extracted from request data and the second one has request outcome
+(response).
+
+Enabling API Audit Logging
+==========================
+
+Audit middleware is available as part of `keystonemiddleware` (>= 1.6) library.
+For infomation regarding how audit middleware functions refer `here.
+<http://docs.openstack.org/developer/keystonemiddleware/audit.html>`_
+
+Auditing can be enabled for the Bare Metal service by making the following changes
+to ``/etc/ironic/ironic.conf``.
+
+#. To enable audit logging of API requests::
+
+    [audit]
+    ...
+    enabled=true
+
+#. To customize auditing API requests, the audit middleware requires the audit_map_file setting
+   to be defined. Update the value of configuration setting 'audit_map_file' to set its
+   location. Audit map file configuration options for the Bare Metal service are included
+   in the etc/ironic/ironic_api_audit_map.conf.sample file. To understand CADF format
+   specified in ironic_api_audit_map.conf file refer to `CADF Format.
+   <http://www.dmtf.org/sites/default/files/standards/documents/DSP2038_1.0.0.pdf>`_::
+
+    [audit]
+    ...
+    audit_map_file=/etc/ironic/ironic_api_audit_map.conf
+
+#. Comma separated list of Ironic REST API HTTP methods to be ignored during audit.
+   For example: GET,POST. It is used only when API audit is enabled.
+
+    [audit]
+    ...
+    ignore_req_list=GET,POST
+
+Sample Audit Event
+==================
+
+Following is the sample of audit event for ironic node list request.
+
+.. code-block:: json
+
+    {
+       "event_type":"audit.http.request",
+       "timestamp":"2016-06-15 06:04:30.904397",
+       "payload":{
+          "typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event",
+          "eventTime":"2016-06-15T06:04:30.903071+0000",
+          "target":{
+             "id":"ironic",
+             "typeURI":"unknown",
+             "addresses":[
+                {
+                   "url":"http://{ironic_admin_host}:6385",
+                   "name":"admin"
+                },
+               {
+                   "url":"http://{ironic_internal_host}:6385",
+                   "name":"private"
+               },
+               {
+                   "url":"http://{ironic_public_host}:6385",
+                   "name":"public"
+               }
+             ],
+             "name":"ironic"
+          },
+          "observer":{
+             "id":"target"
+          },
+          "tags":[
+             "correlation_id?value=685f1abb-620e-5d5d-b74a-b4135fb32373"
+          ],
+          "eventType":"activity",
+          "initiator":{
+             "typeURI":"service/security/account/user",
+             "name":"admin",
+             "credential":{
+                "token":"***",
+                "identity_status":"Confirmed"
+             },
+             "host":{
+                "agent":"python-ironicclient",
+                "address":"10.1.200.129"
+             },
+             "project_id":"d8f52dd7d9e1475dbbf3ba47a4a83313",
+             "id":"8c1a948bad3948929aa5d5b50627a174"
+          },
+          "action":"read",
+          "outcome":"pending",
+          "id":"061b7aa7-5879-5225-a331-c002cf23cb6c",
+          "requestPath":"/v1/nodes/?associated=True"
+       },
+       "priority":"INFO",
+       "publisher_id":"ironic-api",
+       "message_id":"2f61ebaa-2d3e-4023-afba-f9fca6f21fc2"
+    }
diff --git a/doc/source/index.rst b/doc/source/index.rst
index a970bbeea..12d2f54f9 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -42,6 +42,7 @@ Administrator's Guide
   deploy/inspection
   deploy/security
   deploy/adoption
+  deploy/api-audit-support
   deploy/troubleshooting
   Release Notes <http://docs.openstack.org/releasenotes/ironic/>
   Dashboard (horizon) plugin <http://docs.openstack.org/developer/ironic-ui/>
diff --git a/etc/ironic/ironic.conf.sample b/etc/ironic/ironic.conf.sample
index fda45054a..cec6de3fa 100644
--- a/etc/ironic/ironic.conf.sample
+++ b/etc/ironic/ironic.conf.sample
@@ -487,6 +487,27 @@
 #enable_ssl_api = false
 
 
+[audit]
+
+#
+# From ironic
+#
+
+# Enable auditing of API requests (for ironic-api service).
+# (boolean value)
+#enabled = false
+
+# Path to audit map file for ironic-api service. Used only
+# when API audit is enabled. (string value)
+#audit_map_file = /etc/ironic/ironic_api_audit_map.conf
+
+# Comma separated list of Ironic REST API HTTP methods to be
+# ignored during audit. For example: auditing will not be done
+# on any GET or POST requests if this is set to "GET,POST". It
+# is used only when API audit is enabled. (string value)
+#ignore_req_list = <None>
+
+
 [cimc]
 
 #
diff --git a/etc/ironic/ironic_api_audit_map.conf.sample b/etc/ironic/ironic_api_audit_map.conf.sample
new file mode 100644
index 000000000..a8076e2ab
--- /dev/null
+++ b/etc/ironic/ironic_api_audit_map.conf.sample
@@ -0,0 +1,29 @@
+[DEFAULT]
+# default target endpoint type
+# should match the endpoint type defined in service catalog
+target_endpoint_type = None
+
+# possible end path of API requests
+# path of api requests for CADF target typeURI
+# Just need to include top resource path to identify class
+# of resources. Ex: Log audit event for API requests
+# path containing "nodes" keyword and node uuid.
+[path_keywords]
+nodes = node
+drivers = driver
+chassis = chassis
+ports = port
+states = state
+power = None
+provision = None
+maintenance = None
+validate = None
+boot_device = None
+supported = None
+console = None
+vendor_passthrus = vendor_passthru
+
+
+# map endpoint type defined in service catalog to CADF typeURI
+[service_endpoints]
+baremetal = service/compute/baremetal
diff --git a/ironic/api/app.py b/ironic/api/app.py
index f81b3e6c6..5621d9759 100644
--- a/ironic/api/app.py
+++ b/ironic/api/app.py
@@ -15,6 +15,8 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import keystonemiddleware.audit as audit_middleware
+from keystonemiddleware.audit import PycadfAuditApiConfigError
 from oslo_config import cfg
 import oslo_middleware.cors as cors_middleware
 import pecan
@@ -24,6 +26,7 @@ from ironic.api import config
 from ironic.api.controllers.base import Version
 from ironic.api import hooks
 from ironic.api import middleware
+from ironic.common import exception
 from ironic.conf import CONF
 
 
@@ -60,6 +63,19 @@ def setup_app(pecan_config=None, extra_hooks=None):
         wrap_app=middleware.ParsableErrorMiddleware,
     )
 
+    if CONF.audit.enabled:
+        try:
+            app = audit_middleware.AuditMiddleware(
+                app,
+                audit_map_file=CONF.audit.audit_map_file,
+                ignore_req_list=CONF.audit.ignore_req_list
+            )
+        except (EnvironmentError, OSError, PycadfAuditApiConfigError) as e:
+            raise exception.InputFileError(
+                file_name=CONF.audit.audit_map_file,
+                reason=e
+            )
+
     if pecan_config.app.enable_acl:
         app = acl.install(app, cfg.CONF, pecan_config.app.acl_public_routes)
 
diff --git a/ironic/common/exception.py b/ironic/common/exception.py
index f93305d21..cb8e6c631 100644
--- a/ironic/common/exception.py
+++ b/ironic/common/exception.py
@@ -255,6 +255,10 @@ class InstanceNotFound(NotFound):
     _msg_fmt = _("Instance %(instance)s could not be found.")
 
 
+class InputFileError(IronicException):
+    _msg_fmt = _("Error with file %(file_name)s. Reason: %(reason)s")
+
+
 class NodeNotFound(NotFound):
     _msg_fmt = _("Node %(node)s could not be found.")
 
diff --git a/ironic/conf/__init__.py b/ironic/conf/__init__.py
index 4bcb97319..e80854a92 100644
--- a/ironic/conf/__init__.py
+++ b/ironic/conf/__init__.py
@@ -16,6 +16,7 @@
 from oslo_config import cfg
 
 from ironic.conf import api
+from ironic.conf import audit
 from ironic.conf import cimc
 from ironic.conf import cisco_ucs
 from ironic.conf import conductor
@@ -42,6 +43,7 @@ from ironic.conf import virtualbox
 CONF = cfg.CONF
 
 api.register_opts(CONF)
+audit.register_opts(CONF)
 cimc.register_opts(CONF)
 cisco_ucs.register_opts(CONF)
 conductor.register_opts(CONF)
diff --git a/ironic/conf/audit.py b/ironic/conf/audit.py
new file mode 100644
index 000000000..5e1d4b5a0
--- /dev/null
+++ b/ironic/conf/audit.py
@@ -0,0 +1,38 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from oslo_config import cfg
+
+from ironic.common.i18n import _
+
+opts = [
+    cfg.BoolOpt('enabled',
+                default=False,
+                help=_('Enable auditing of API requests'
+                       ' (for ironic-api service).')),
+
+    cfg.StrOpt('audit_map_file',
+               default='/etc/ironic/ironic_api_audit_map.conf',
+               help=_('Path to audit map file for ironic-api service. '
+                      'Used only when API audit is enabled.')),
+
+    cfg.StrOpt('ignore_req_list',
+               help=_('Comma separated list of Ironic REST API HTTP methods '
+                      'to be ignored during audit. For example: auditing '
+                      'will not be done on any GET or POST requests '
+                      'if this is set to "GET,POST". It is used '
+                      'only when API audit is enabled.')),
+]
+
+
+def register_opts(conf):
+    conf.register_opts(opts, group='audit')
diff --git a/ironic/conf/opts.py b/ironic/conf/opts.py
index b1c105f48..67a0bee3a 100644
--- a/ironic/conf/opts.py
+++ b/ironic/conf/opts.py
@@ -43,6 +43,7 @@ _opts = [
         ironic.drivers.modules.amt.common.opts,
         ironic.drivers.modules.amt.power.opts)),
     ('api', ironic.conf.api.opts),
+    ('audit', ironic.conf.audit.opts),
     ('cimc', ironic.conf.cimc.opts),
     ('cisco_ucs', ironic.conf.cisco_ucs.opts),
     ('conductor', ironic.conf.conductor.opts),
diff --git a/ironic/tests/unit/api/test_audit.py b/ironic/tests/unit/api/test_audit.py
new file mode 100644
index 000000000..6e53fbfb1
--- /dev/null
+++ b/ironic/tests/unit/api/test_audit.py
@@ -0,0 +1,59 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+"""
+Tests to assert that audit middleware works as expected.
+"""
+
+from keystonemiddleware import audit
+import mock
+from oslo_config import cfg
+
+from ironic.common import exception
+from ironic.tests.unit.api import base
+
+
+CONF = cfg.CONF
+
+
+class TestAuditMiddleware(base.BaseApiTest):
+    """Provide a basic smoke test to ensure audit middleware is active.
+
+    The tests below provide minimal confirmation that the audit middleware
+    is called, and may be configured. For comprehensive tests, please consult
+    the test suite in keystone audit_middleware.
+    """
+
+    def setUp(self):
+        super(TestAuditMiddleware, self).setUp()
+
+    @mock.patch.object(audit, 'AuditMiddleware')
+    def test_enable_audit_request(self, mock_audit):
+        CONF.audit.enabled = True
+        self._make_app(enable_acl=True)
+        mock_audit.assert_called_once_with(
+            mock.ANY,
+            audit_map_file=CONF.audit.audit_map_file,
+            ignore_req_list=CONF.audit.ignore_req_list)
+
+    @mock.patch.object(audit, 'AuditMiddleware')
+    def test_enable_audit_request_error(self, mock_audit):
+        CONF.audit.enabled = True
+        mock_audit.side_effect = IOError("file access error")
+
+        self.assertRaises(exception.InputFileError,
+                          self._make_app, enable_acl=True)
+
+    @mock.patch.object(audit, 'AuditMiddleware')
+    def test_disable_audit_request(self, mock_audit):
+        CONF.audit.enabled = False
+        self._make_app(enable_acl=True)
+        self.assertFalse(mock_audit.called)
diff --git a/releasenotes/notes/adding-audit-middleware-b95f2a00baed9750.yaml b/releasenotes/notes/adding-audit-middleware-b95f2a00baed9750.yaml
new file mode 100644
index 000000000..ef804c304
--- /dev/null
+++ b/releasenotes/notes/adding-audit-middleware-b95f2a00baed9750.yaml
@@ -0,0 +1,10 @@
+---
+features:
+  - |
+    The ironic-api service now supports logging audit messages of
+    api calls. The following configuration parameters have been added.
+    By default auditing of ironic-api service is turned off.
+
+    * [audit]/enabled
+    * [audit]/ignore_req_list
+    * [audit]/audit_map_file