Merge "[goal] Deprecate the JSON formatted policy file"

13 files changed, 68 insertions, 21 deletions

diff --git a/doc/source/admin/cli-manage-projects-users-and-roles.rst b/doc/source/admin/cli-manage-projects-users-and-roles.rst
index f27979d2c..8d2f83702 100644
--- a/doc/source/admin/cli-manage-projects-users-and-roles.rst
+++ b/doc/source/admin/cli-manage-projects-users-and-roles.rst
@@ -10,8 +10,8 @@ define which actions users can perform. You assign roles to
 user-project pairs.
 
 You can define actions for OpenStack service roles in the
-``/etc/PROJECT/policy.json`` files. For example, define actions for
-Compute service roles in the ``/etc/nova/policy.json`` file.
+``/etc/PROJECT/policy.yaml`` files. For example, define actions for
+Compute service roles in the ``/etc/nova/policy.yaml`` file.
 
 You can manage projects, users, and roles independently from each other.
 
diff --git a/doc/source/admin/identity-concepts.rst b/doc/source/admin/identity-concepts.rst
index 3d615c0da..0f8cfc51f 100644
--- a/doc/source/admin/identity-concepts.rst
+++ b/doc/source/admin/identity-concepts.rst
@@ -122,9 +122,9 @@ Identity user management examples:
      Individual services assign meaning to roles, typically through
      limiting or granting access to users with the role to the
      operations that the service supports. Role access is typically
-     configured in the service's ``policy.json`` file. For example,
+     configured in the service's ``policy.yaml`` file. For example,
      to limit Compute access to the ``compute-user`` role, edit the
-     Compute service's ``policy.json`` file to require this role for
+     Compute service's ``policy.yaml`` file to require this role for
      Compute operations.
 
 The Identity service assigns a project and a role to a user. You might
@@ -139,25 +139,25 @@ A user can have different roles in different projects. For example, Alice
 might also have the ``admin`` role in the ``Cyberdyne`` project. A user
 can also have multiple roles in the same project.
 
-The ``/etc/[SERVICE_CODENAME]/policy.json`` file controls the
+The ``/etc/[SERVICE_CODENAME]/policy.yaml`` file controls the
 tasks that users can perform for a given service. For example, the
-``/etc/nova/policy.json`` file specifies the access policy for the
-Compute service, the ``/etc/glance/policy.json`` file specifies
+``/etc/nova/policy.yaml`` file specifies the access policy for the
+Compute service, the ``/etc/glance/policy.yaml`` file specifies
 the access policy for the Image service, and the
-``/etc/keystone/policy.json`` file specifies the access policy for
+``/etc/keystone/policy.yaml`` file specifies the access policy for
 the Identity service.
 
-The default ``policy.json`` files in the Compute, Identity, and
+The default ``policy.yaml`` files in the Compute, Identity, and
 Image services recognize only the ``admin`` role. Any user with
 any role in a project can access all operations that do not require the
 ``admin`` role.
 
 To restrict users from performing operations in, for example, the
 Compute service, you must create a role in the Identity service and
-then modify the ``/etc/nova/policy.json`` file so that this role
+then modify the ``/etc/nova/policy.yaml`` file so that this role
 is required for Compute operations.
 
-For example, the following line in the ``/etc/cinder/policy.json``
+For example, the following line in the ``/etc/cinder/policy.yaml``
 file does not restrict which users can create volumes:
 
 .. code-block:: none
diff --git a/doc/source/admin/upgrading.rst b/doc/source/admin/upgrading.rst
index 687dba4e3..709d98dac 100644
--- a/doc/source/admin/upgrading.rst
+++ b/doc/source/admin/upgrading.rst
@@ -42,7 +42,7 @@ Plan your upgrade:
   to re-read the release notes for the previous release (or two!).
 
 * Prepare your new configuration files, including ``keystone.conf``,
-  ``logging.conf``, ``policy.json``, ``keystone-paste.ini``, and anything else
+  ``logging.conf``, ``policy.yaml``, ``keystone-paste.ini``, and anything else
   in ``/etc/keystone/``, by customizing the corresponding files from the next
   release.
 
diff --git a/doc/source/configuration/policy.rst b/doc/source/configuration/policy.rst
index daafdea51..3f80c5ee2 100644
--- a/doc/source/configuration/policy.rst
+++ b/doc/source/configuration/policy.rst
@@ -2,6 +2,15 @@
 Policy configuration
 ====================
 
+.. warning::
+
+   JSON formatted policy file is deprecated since Keystone 19.0.0 (Wallaby).
+   This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing
+   JSON-formatted policy file to YAML in a backward-compatible way.
+
+.. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html
+
+
 Configuration
 ~~~~~~~~~~~~~
 
diff --git a/doc/source/getting-started/policy_mapping.rst b/doc/source/getting-started/policy_mapping.rst
index 2975b45b3..a7cb27cfa 100644
--- a/doc/source/getting-started/policy_mapping.rst
+++ b/doc/source/getting-started/policy_mapping.rst
@@ -2,7 +2,7 @@
 Mapping of policy target to API
 ===============================
 
-The following table shows the target in the policy.json file for each API.
+The following table shows the target in the policy.yaml file for each API.
 
 =========================================================  ===
 Target                                                     API
diff --git a/keystone/cmd/status.py b/keystone/cmd/status.py
index 3585c2e2b..64b2e62f3 100644
--- a/keystone/cmd/status.py
+++ b/keystone/cmd/status.py
@@ -12,6 +12,7 @@
 
 from oslo_policy import _checks
 from oslo_policy import policy
+from oslo_upgradecheck import common_checks
 from oslo_upgradecheck import upgradecheck
 
 from keystone.common import driver_hints
@@ -86,6 +87,8 @@ class Checks(upgradecheck.UpgradeCommands):
          check_trust_policies_are_not_empty),
         ("Check default roles are immutable",
          check_default_roles_are_immutable),
+        ("Policy File JSON to YAML Migration",
+         (common_checks.check_policy_json, {'conf': CONF})),
     )
 
 
diff --git a/keystone/common/rbac_enforcer/enforcer.py b/keystone/common/rbac_enforcer/enforcer.py
index ca6a8e7b2..7add048ce 100644
--- a/keystone/common/rbac_enforcer/enforcer.py
+++ b/keystone/common/rbac_enforcer/enforcer.py
@@ -14,6 +14,7 @@ import functools
 
 import flask
 from oslo_log import log
+from oslo_policy import opts
 from oslo_policy import policy as common_policy
 from oslo_utils import strutils
 
@@ -39,6 +40,13 @@ _POSSIBLE_TARGET_ACTIONS = frozenset([
 _ENFORCEMENT_CHECK_ATTR = 'keystone:RBAC:enforcement_called'
 
 
+# TODO(gmann): Remove setting the default value of config policy_file
+# once oslo_policy change the default value to 'policy.yaml'.
+# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49
+DEFAULT_POLICY_FILE = 'policy.yaml'
+opts.set_defaults(CONF, DEFAULT_POLICY_FILE)
+
+
 class RBACEnforcer(object):
     """Enforce RBAC on API calls."""
 
diff --git a/keystone/conf/__init__.py b/keystone/conf/__init__.py
index 77c26a11d..5de0ec183 100644
--- a/keystone/conf/__init__.py
+++ b/keystone/conf/__init__.py
@@ -18,6 +18,7 @@ from oslo_log import log
 from oslo_log import versionutils
 import oslo_messaging
 from oslo_middleware import cors
+from oslo_policy import opts as policy_opts
 from osprofiler import opts as profiler
 
 from keystone.conf import application_credential
@@ -185,6 +186,12 @@ def set_external_opts_defaults():
     # configure OSprofiler options
     profiler.set_defaults(CONF, enabled=False, trace_sqlalchemy=False)
 
+    # TODO(gmann): Remove setting the default value of config policy_file
+    # once oslo_policy change the default value to 'policy.yaml'.
+    # https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49
+    DEFAULT_POLICY_FILE = 'policy.yaml'
+    policy_opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE)
+
     # Oslo.cache is always enabled by default for request-local caching
     # TODO(morganfainberg): Fix this to not use internal interface when
     # oslo.cache has proper interface to set defaults added. This is
diff --git a/keystone/tests/unit/test_policy.py b/keystone/tests/unit/test_policy.py
index 8346549ec..d58ea6399 100644
--- a/keystone/tests/unit/test_policy.py
+++ b/keystone/tests/unit/test_policy.py
@@ -197,7 +197,7 @@ class PolicyJsonTestCase(unit.TestCase):
     def test_all_targets_documented(self):
         policy_keys = self._get_default_policy_rules()
 
-        # These keys are in the policy.json but aren't targets.
+        # These keys are in the policy.yaml but aren't targets.
         policy_rule_keys = [
             'admin_or_owner', 'admin_or_token_subject', 'admin_required',
             'owner', 'service_admin_or_token_subject', 'service_or_admin',
diff --git a/keystone/tests/unit/test_v3.py b/keystone/tests/unit/test_v3.py
index 56ebf4a90..7d6c6b11f 100644
--- a/keystone/tests/unit/test_v3.py
+++ b/keystone/tests/unit/test_v3.py
@@ -247,7 +247,7 @@ class RestfulTestCase(unit.SQLDriverOverrides, rest.RestfulTestCase,
             domain_id=DEFAULT_DOMAIN_ID)
         self.default_domain_user_id = self.default_domain_user['id']
 
-        # create & grant policy.json's default role for admin_required
+        # create & grant policy.yaml's default role for admin_required
         self.role = unit.new_role_ref(name='admin')
         self.role_id = self.role['id']
         PROVIDERS.role_api.create_role(self.role_id, self.role)
diff --git a/lower-constraints.txt b/lower-constraints.txt
index 9e9220199..431589d02 100644
--- a/lower-constraints.txt
+++ b/lower-constraints.txt
@@ -25,16 +25,16 @@ oauthlib==0.6.2
 os-api-ref==1.4.0
 oslo.cache==1.26.0
 oslo.concurrency==3.26.0
-oslo.config==5.2.0
+oslo.config==6.8.0
 oslo.context==2.22.0
 oslo.db==6.0.0
 oslo.i18n==3.15.3
 oslo.log==3.44.0
 oslo.messaging==5.29.0
 oslo.middleware==3.31.0
-oslo.policy==3.0.2
+oslo.policy==3.6.0
 oslo.serialization==2.18.0
-oslo.upgradecheck==0.1.0
+oslo.upgradecheck==1.3.0
 oslo.utils==3.33.0
 oslotest==3.2.0
 osprofiler==1.4.0
diff --git a/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml
new file mode 100644
index 000000000..c9c530004
--- /dev/null
+++ b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml
@@ -0,0 +1,20 @@
+---
+upgrade:
+  - |
+    The default value of ``[oslo_policy] policy_file`` config option has
+    been changed from ``policy.json`` to ``policy.yaml``.
+    Operators who are utilizing customized or previously generated
+    static policy JSON files (which are not needed by default), should
+    generate new policy files or convert them in YAML format. Use the
+    `oslopolicy-convert-json-to-yaml
+    <https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html>`_
+    tool to convert a JSON to YAML formatted policy file in
+    backward compatible way.
+deprecations:
+  - |
+    Use of JSON policy files was deprecated by the ``oslo.policy`` library
+    during the Victoria development cycle. As a result, this deprecation is
+    being noted in the Wallaby cycle with an anticipated future removal of support
+    by ``oslo.policy``. As such operators will need to convert to YAML policy
+    files. Please see the upgrade notes for details on migration of any
+    custom policy files.
diff --git a/requirements.txt b/requirements.txt
index 33a2c423e..3464fb3fc 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -16,16 +16,16 @@ keystonemiddleware>=7.0.0 # Apache-2.0
 bcrypt>=3.1.3 # Apache-2.0
 scrypt>=0.8.0 # BSD
 oslo.cache>=1.26.0 # Apache-2.0
-oslo.config>=5.2.0 # Apache-2.0
+oslo.config>=6.8.0 # Apache-2.0
 oslo.context>=2.22.0 # Apache-2.0
 oslo.messaging>=5.29.0 # Apache-2.0
 oslo.db>=6.0.0 # Apache-2.0
 oslo.i18n>=3.15.3 # Apache-2.0
 oslo.log>=3.44.0 # Apache-2.0
 oslo.middleware>=3.31.0 # Apache-2.0
-oslo.policy>=3.0.2 # Apache-2.0
+oslo.policy>=3.6.0 # Apache-2.0
 oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0
-oslo.upgradecheck>=0.1.0 # Apache-2.0
+oslo.upgradecheck>=1.3.0 # Apache-2.0
 oslo.utils>=3.33.0 # Apache-2.0
 oauthlib>=0.6.2 # BSD
 pysaml2>=5.0.0