Merge "Update unified limit documentation"

3 files changed, 473 insertions, 37 deletions

diff --git a/doc/requirements.txt b/doc/requirements.txt
index 8aebc182b..0c5da21e9 100644
--- a/doc/requirements.txt
+++ b/doc/requirements.txt
@@ -7,6 +7,7 @@ sphinx!=1.6.6,!=1.6.7,>=1.6.2;python_version>='3.4'  # BSD
 sphinxcontrib-apidoc>=0.2.0  # BSD
 sphinxcontrib-seqdiag>=0.8.4  # BSD
 sphinx-feature-classification>=0.3.2  # Apache-2.0
+sphinxcontrib-blockdiag>=1.5.5  # BSD
 reno>=2.5.0 # Apache-2.0
 os-api-ref>=1.4.0 # Apache-2.0
 python-ldap>=3.0.0  # PSF
diff --git a/doc/source/admin/unified-limits.rst b/doc/source/admin/unified-limits.rst
index d590674a9..1a37a4c5a 100644
--- a/doc/source/admin/unified-limits.rst
+++ b/doc/source/admin/unified-limits.rst
@@ -24,6 +24,7 @@ to different users or projects as needs change. Some information needed to
 establish a limit may include:
 
 - project_id
+- domain_id
 - API service type (e.g. compute, network, object-storage)
 - a resource type (e.g. ram_mb, vcpus, security-groups)
 - a default limit
@@ -54,22 +55,37 @@ service. The second part sets a default of 20 `vcpus` per project. This
 provides all the information needed for basic quota enforcement for any
 resource provided by a service.
 
+Domain limits
+-------------
+
+A domain limit is a limit associated to a specific domain and it acts as an
+override for a registered limit. Similar to registered limits, domain limits
+require a resource type and a service. Additionally, a registered limit must
+exist before you can create a domain-specific override. For example, let's
+assume a registered limit exists for `vcpus` provided by the compute service.
+It wouldn't be possible to create a domain limit for `cores` on the compute
+service. Domain limits can only override limits that have already been
+registered. In a general sense, registered limits are likely established when a
+new service or cloud is deployed. Domain limits are used continuously to manage
+the flow of resource allocation.
+
+Domain limits may affect the limits of projects within the domain. This is
+particularly important to keep in mind when choosing an enforcement model,
+documented below.
+
 Project limits
 --------------
 
-A project limit is a limit associated to a specific project and it acts as an
-override for a registered limit. A project limit still requires a resource type
-and service, both of which must exist as a registered limit. For example, let's
-assume a registered limit exists for `vcpus` provided by the compute service.
-It wouldn't be possible for a project limit to be created for `cores` on the
-compute service for a specific project. Project limits can only override limits
-that have already been registered. In a general sense, registered limits are
-likely established when a new service or cloud is deployed. Project limits are
-used continuously to manage the flow of resource allocation.
+Project limits have the same properties as domain limits, but are specific to
+projects instead of domains. You must register a limit before creating a
+project-specific override. Just like with domain limits, the flow of resources
+between related projects may vary depending on the configured enforcement
+model. The support enforcement models below describe how limit validation and
+enforcement behave between related projects and domains.
 
-Together, registered limits and project limits give deployments the ability to
-restrict resources across the deployment by default, while being flexible
-enough to freely marshal resources across projects.
+Together, registered limits, domain limits, and project limits give deployments
+the ability to restrict resources across the deployment by default, while being
+flexible enough to freely marshal resources across projects.
 
 Limits and usage
 ================
@@ -80,13 +96,12 @@ system for enforcing that usage does not exceed limits. While they are coupled,
 they are distinct.
 
 Up to this point, we've established that keystone is the system for maintaining
-limit information. Keystone’s responsibility is to ensuring that any changes to
+limit information. Keystone’s responsibility is to ensure that any changes to
 limits are consistent with related limits currently stored in keystone.
 
-Individual services maintain and enforce usage. Services check for enforcement
-against the current limits at the time a resource allocation is requested by a
-particular user. A usage reflects the actual allocation of units of a
-particular resource to a consumer.
+Individual services maintain and enforce usage. Services check enforcement
+against the current limits at the time a user requests a resource. Usage
+reflects the actual resource allocation in units to a consumer.
 
 Given the above, the following is a possible and legal flow:
 
@@ -98,7 +113,7 @@ Given the above, the following is a possible and legal flow:
   she (or others in the project) have deleted at least 9 CPUs to get under the
   new limit
 
-The following would be another permuation:
+The following would be another permutation:
 
 - User Jane is in project Foo
 - Project Foo has a default CPU limit of 20
@@ -108,8 +123,8 @@ The following would be another permuation:
   limit of CPUs
 - User Jane requests more resources
 - Administrator Kelly adjust the project limit for Foo to be 30 CPUs
-- User Jane resends her request for an instance, which successed since the
-  useage for project Foo is under the project limit of 30 CPUs
+- User Jane resends her request for an instance, which succeeds since the
+  usage for project Foo is under the project limit of 30 CPUs
 
 This behavior lets administrators set the policy of what the future should be
 when convenient, and prevent those projects from creating any more resources
@@ -141,22 +156,12 @@ deployment.
 
 Keystone exposes a ``GET /limits-model`` endpoint that returns the enforcement
 model selected by the deployment. This allows limit information to be
-discoverable and perserves interoperability between OpenStack deployments with
+discoverable and preserves interoperability between OpenStack deployments with
 different enforcement models.
 
-.. NOTE:: The API to expose the limit model configuration in keystone will be
-   implemented when keystone supports more than one enforcement model. Until
-   then, keystone only supports the following enforcement model.
-
 Flat
 ----
 
-.. NOTE::
-
-    The flat enforcement model is currently the only supported enforcement
-    model. Future work will add more enforcement models which will be
-    configurable via configuration files and discoverable via the API.
-
 Flat enforcement ignores all aspects of a project hierarchy. Each project is
 considered a peer to all other projects. The limits associated to the parents,
 siblings, or children have no affect on a particular project. This model
@@ -165,12 +170,437 @@ between limits, regardless of the hierarchy. Validation of limits via the API
 will allow operations that might not be considered accepted in other models.
 
 For example, assume project `Charlie` is a child of project `Beta`, which is a
-child of project `Alpha`. A default is set on a limit, all projects get that
-effective default, which is 10. The following operations would be possible:
+child of project `Alpha`. All projects assume a default limit of 10 cores via a
+registered limit. The labels in the diagrams below use shorthand notation for
+`limit` and `usage` as `l` and `u`, respectively:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (u=0)"];
+      Beta [label=" Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+Each project may use up to 10 cores because of the registered limit and none of
+the projects have an override. Using flat enforcement, you're allowed to
+``UPDATE LIMIT on Alpha to 20``:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (l=20, u=0)", textcolor = "#00af00"];
+      Beta [label=" Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
 
-- ``UPDATE LIMIT on Alpha to 20``
-- ``UPDATE LIMIT on Charlie to 30``
+You're also allowed to ``UPDATE LIMIT on Charlie to 30``, even though `Charlie`
+is a sub-project of both `Beta` and `Alpha`.
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (l=20, u=0)"];
+      Beta [label=" Beta (u=0)"];
+      Charlie [label="Charlie (l=30, u=0)", textcolor = "#00af00"];
+   }
 
 This is allowed with flat enforcement because the hierarchy is not taken into
-consideration during limit validation, even though a child project has a higher
-limit than a parent project.
+consideration during limit validation. Child projects may have a higher limit
+than a parent project.
+
+Conversely, you can simulate hierarchical enforcement by adjusting limits
+through the project tree manually. For example, let's still assume 10 is the
+default limit imposed by an existing registered limit:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (u=0)"];
+      Beta [label=" Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+You may set a project-specific override to ``UPDATE LIMIT on Alpha to 30``:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (l=30, u=0)", textcolor = "#00af00"];
+      Beta [label=" Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+Next you can ``UPDATE LIMIT on Beta to 20``:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (l=30, u=0)"];
+      Beta [label=" Beta (l=20, u=0)", textcolor = "#00af00"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+Theoretically, the entire project tree consisting of `Alpha`, `Beta`, and
+`Charlie` is limited to 60 cores. If you'd like to ensure only 30 cores are
+used within the entire hierarchy, you can ``UPDATE LIMIT on Alpha to 0``:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha [label="Alpha (l=0, u=0)", textcolor = "#00af00"];
+      Beta [label=" Beta (l=20, u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+You should use this model if you:
+
+* Have project hierarchies greater than two levels
+* Want extremely strict control of project usage and don't want resource usage
+  to bleed across projects or domains
+
+Advantages
+~~~~~~~~~~
+
+* Allows you to model specific and strict limits
+* Works with any project hierarchy or depth
+* Usage is only calculated for the project in question
+
+Disadvantages
+~~~~~~~~~~~~~
+
+* Resources aren't allowed to flow gracefully between projects in a hierarchy
+* Requires intervention and verification to move resources across projects
+* Project limit validation isn't performed with respect to other projects or
+  domains
+
+Strict Two Level
+----------------
+
+The ``strict_two_level`` enforcement model assumes the project hierarchy does
+not exceed two levels. The top layer can consist of projects or domains. For
+example, project `Alpha` can have a sub-project called `Beta` within this
+model. Project `Beta` cannot have a sub-project. The hierarchy is restrained to
+two layers. `Alpha` can also be a domain that contains project `Beta`, but
+`Beta` cannot have a sub-project. Regardless of the top layer consisting of
+projects or domains, the hierarchical depth is limited to two layers.
+
+Resource utilization is allowed to flow between projects in the hierarchy,
+depending on the limits. This property allows for more flexibility than the
+``flat`` enforcement model. The model is strict in that operators can set
+limits on parent projects or domains and the limits of the children may never
+exceed the parent.
+
+For example, assume domain `Alpha` contains two projects, `Beta` and `Charlie`.
+Projects `Beta` and `Charlie` are siblings so the hierarchy maintains a depth
+of two. A system administrator sets the limit of a resource on `Alpha` to 20.
+Both projects `Beta` and `Charlie` can consume resources until the total usage
+of `Alpha`, `Beta`, and `Charlie` reach 20. At that point, no more resources
+should be allocated to the tree. System administrators can also reserve
+portions of domain `Alpha`'s resource in sub-projects directly. Using the
+previous example, project `Beta` could have a limit of 12 resources, implicitly
+leaving 8 resources for `Charlie` to consume.
+
+The following diagrams illustrate the behaviors described above, using projects
+named `Alpha`, `Beta`, `Charlie`, and `Delta`. Assume the resource in question
+is cores and the default registered limit for cores is 10. Also assume we have
+the following project hierarchy where `Alpha` has a limit of 20 cores and its
+usage is currently 4:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+Technically, both `Beta` and `Charlie` can use up to 8 cores each:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (u=8)", textcolor = "#00af00"];
+      Charlie [label="Charlie (u=8)", textcolor = "#00af00"];
+   }
+
+If `Alpha` attempts to claim two cores the usage check will fail
+because the service will fetch the hierarchy from keystone using ``oslo.limit``
+and check the usage of each project in the hierarchy to see that the total
+usage of `Alpha`, `Beta`, and `Charlie` is equal to the limit of the tree, set
+by `Alpha.limit`:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=6)", textcolor = "#FF0000"];
+      Beta [label="Beta (u=8)"];
+      Charlie [label="Charlie (u=8)"];
+   }
+
+Despite the usage of the tree being equal to the limit, we can still add
+children to the tree:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+      Alpha -> Delta;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (u=8)"];
+      Charlie [label="Charlie (u=8)"];
+      Delta [label="Delta (u=0)", textcolor = "#00af00"];
+   }
+
+Even though the project can be created, the current usage of cores across the
+tree prevents `Delta` from claiming any cores:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+      Alpha -> Delta;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (u=8)"];
+      Charlie [label="Charlie (u=8)"];
+      Delta [label="Delta (u=2)", textcolor = "#FF0000"];
+   }
+
+Creating a grandchild of project `Alpha` is forbidden because it violates the
+two-level hierarchical constraint:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+      Charlie -> Delta;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (u=8)"];
+      Charlie [label="Charlie (u=8)"];
+      Delta [label="Delta (u=0)", textcolor = "#FF0000"];
+   }
+
+This is a fundamental constraint of this design because it provides a very
+clear escalation path. When a request fails because the tree limit has been
+exceeded, a user has all the information they need to provide meaningful
+context in a support ticket (e.g., their project ID and the parent project ID).
+An administrator should be able to reshuffle usage accordingly. Providing this
+information in tree structures with more than a depth of two is much harder,
+but may be implemented with a separate model.
+
+Granting `Beta` the ability to claim more cores can be done by giving `Beta` a
+project-specific override for cores
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=4)"];
+      Beta [label="Beta (l=12, u=8)", textcolor = "#00af00"];
+      Charlie [label="Charlie (u=8)"];
+   }
+
+Note that regardless of this update, any subsequent requests to claim more
+cores in the tree will be rejected since the usage is equal to the limit of the
+`Alpha`. `Beta` can claim cores if they are released from `Alpha` or `Charlie`:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=2)", textcolor = "#00af00"];
+      Beta [label="Beta (l=12, u=8)"];
+      Charlie [label="Charlie (u=6)", textcolor = "#00af00"];
+   }
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=2)"];
+      Beta [label="Beta (l=12, u=12)", textcolor = "#00af00"];
+      Charlie [label="Charlie (u=6)"];
+   }
+
+While `Charlie` is still under its default allocation of 10 cores, it won't be
+able to claim any more cores because the total usage of the tree is equal to
+the limit of `Alpha`, thus preventing `Charlie` from reclaiming the cores it
+had:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=2)"];
+      Beta [label="Beta (l=12, u=12)"];
+      Charlie [label="Charlie (u=8)", textcolor = "#FF0000"];
+   }
+
+Creating or updating a project with a limit that exceeds the limit of `Alpha`
+is forbidden. Even though it is possible for the sum of all limits under
+`Alpha` to exceed the limit of `Alpha`, the total usage is capped at
+`Alpha.limit`. Allowing children to have explicit overrides greater than the
+limit of the parent would result in strange user experience and be misleading
+since the total usage of the tree would be capped at the limit of the parent:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+
+      Alpha [label="Alpha (l=20, u=0)"];
+      Beta [label="Beta (l=30, u=0)", textcolor = "#FF0000"];
+      Charlie [label="Charlie (u=0)"];
+   }
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+      Alpha -> Delta;
+
+      Alpha [label="Alpha (l=20, u=0)"];
+      Beta [label="Beta (u=0)"];
+      Charlie [label="Charlie (u=0)"];
+      Delta [label="Delta (l=30, u=0)", textcolor = "#FF0000"];
+   }
+
+Finally, let's still assume the default registered limit for cores is 10, but
+we're going to create project `Alpha` with a limit of 6 cores.
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha;
+
+      Alpha [label="Alpha (l=6, u=0)", textcolor = "#00af00"];
+   }
+
+When we create project `Beta`, which is a child of project `Alpha`, the limit
+API ensures that project `Beta` doesn't assume the default of 10, despite the
+registered limit of 10 cores. Instead, the child assumes the parent's limit
+since no single child limit should exceed the limit of the parent:
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+
+      Alpha [label="Alpha (l=6, u=0)"];
+      Beta [label="Beta (l=6, u=0)", textcolor = "#00af00"];
+   }
+
+This behavior is consistent regardless of the number of children added under
+project `Alpha`.
+
+.. blockdiag::
+
+   blockdiag {
+      orientation = portrait;
+
+      Alpha -> Beta;
+      Alpha -> Charlie;
+      Alpha -> Delta;
+
+      Alpha [label="Alpha (l=6, u=0)"];
+      Beta [label="Beta (l=6, u=0)"];
+      Charlie [label="Charlie (l=6, u=0)", textcolor = "#00af00"];
+      Delta [label="Delta (l=6, u=0)", textcolor = "#00af00"];
+   }
+
+Creating limit overrides while creating projects seems counter-productive given
+the whole purpose of a registered default, but it also seems unlikely to
+throttle a parent project by specifying it's default to be less than a
+registered default. This behavior maintains consistency with the requirement
+that the sum of all child limits may exceed the parent limit, but the limit of
+any one child may not.
+
+You should use this model if you:
+
+* Want resources to flow between projects and domains within a hierarchy
+* Don't have a project depth greater than two levels
+* Are not concerned about usage calculation performance or don't have project
+  trees that are wide
+
+Advantages
+~~~~~~~~~~
+
+* Allows resources to flow between projects and domains within a strict
+  two-level hierarchy
+* Limits are validated when they are created and updated
+
+Disadvantages
+~~~~~~~~~~~~~
+
+* Project depth cannot exceed two levels
+* Performance may suffer in wide and flat project hierarchies during usage
+  calculation
diff --git a/doc/source/conf.py b/doc/source/conf.py
index a88fba24c..90b62f8d1 100644
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -42,8 +42,13 @@ extensions = ['sphinx.ext.coverage',
               'sphinxcontrib.apidoc',
               'sphinxcontrib.seqdiag',
               'sphinx_feature_classification.support_matrix',
+              'sphinxcontrib.blockdiag'
               ]
 
+blockdiag_html_image_format = 'SVG'
+
+blockdiag_fontpath = 'DejaVuSans.ttf'
+
 # sphinxcontrib.apidoc options
 apidoc_module_dir = '../../keystone'
 apidoc_output_dir = 'api'