| Commit message (Collapse) | Author | Age | Files | Lines |
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].
I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html
Change-Id: Ifcaf6993517d02bf54cd144efd247832947a009f
(cherry picked from commit 7b649af0c1569ebb61ed4e34da4a476ccb1474b9)
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An earlier patch[1] added a mapping for context 'system_scope'
to 'system' when enforce was called with a RequestContext
object. However, enforce can also be called with a creds dictionary
that may contain the context 'system_scope' element. When this
occured, 'system_scope' was not mapped to 'system' and the enforce
would fail with an InvalidScope exception.
This patch moves the 'system_scope' mapping from only occuring
with RequestContext objects to also map it when a creds dictonary
is passed to enforce.
[1] https://review.opendev.org/c/openstack/oslo.policy/+/578995
Change-Id: I83a22c3f825bad0c88018118f8630a20a445965e
(cherry picked from commit 9774108cf91408e9cb825b317f48a3a3f856e161)
(cherry picked from commit ff2a39fc619baafa956bee74a2574f5438d40767)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.
Until the requirements repository has as stable/wallaby branch, tests will
continue to use the upper-constraints list on master.
Change-Id: I2b34b7d28513a9f47d6210fbca59d3a6ee68ffd9
|
|
|
|
| |
Change-Id: I8a32a5648df151f3608e9e1bc2dc4ece974e4aaf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The goal here is to avoid conflicts between flake8 and hacking version each
2 days.
Inspired from nova's approach[1].
The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.
[1] https://opendev.org/openstack/nova/src/branch/master/.pre-commit-config.yaml#L26-L35
[2] https://opendev.org/openstack/hacking/src/branch/master/requirements.txt#L1
Change-Id: Id33601ce5949950e4791052ea34a4bf4b0e02158
(cherry picked from commit b11d9e8ab64d1eddd88ce30a0d272007fb68b563)
|
|
|
|
|
|
|
|
|
|
| |
There's no functional changes here other than moving the initializer
documentation for the 'DeprecatedRule' from the '__init__' docstring to
the class' docstring. This allows us to remove '__init__' entirely since
it was just calling the superclass.
Change-Id: I7437cf00b2ba5c02926dc8c2435e237ef730f2b1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
We obviously can't do much about people accessing the private functions,
but this should avoid some obvious bugs. If people want to change the
rules, they should either state a different definition in a file or
change the definition in code.
Change-Id: Ibcbf5292977e5264bf7eedd225cbab83e683e394
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are two big issues with this. Firstly, as noted in bug #1914095,
we're not copying the provided 'Rule' object which means if we create an
enforcer and the rule is modified, then a second enforcer in the same
process will ultimately end up using the same modified 'Rule' object
will be used. Secondly, while the 'check' attribute is modified the
'check_str' attribute is not. This is immensely confusing for people
debugging.
Resolve both issues by not modifying this check at all and instead build
the combined check and store this.
Change-Id: Iff85a9134f4db7c0355da2858deb8a704d044634
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Currently, the way you replace a rule with another rule is by using the
'deprecated_rule' parameter of '(Documented)RuleDefault'. For example:
deprecated_rule = policy.DeprecatedRule(
name='foo:bar',
check_str='role:bazz'
)
policy.RuleDefault(
name='foo:create_bar',
check_str='role:bang',
description='Create a bar.',
deprecated_rule=deprecated_rule,
deprecated_reason='foo:bar has been replaced by foo:create_bar',
deprecated_since='N',
)
In this instance, we're stating that the 'foo:create_bar' policy
replaces the 'foo:bar' policy and we've used (and indeed have to use, to
avoid a 'ValueError') the 'deprecated_reason' and 'deprecated_since'
parameters on the **new** rule to illustrate why. This is confusing. The
new rule clearly isn't the one that's deprecated, so why are we stating
the 'deprecated_reason' and 'deprecated_since' there? We can clarify
this by instead specifying the reason and timeline on the deprecated
rule, like so:
deprecated_rule = policy.DeprecatedRule(
name='foo:bar',
check_str='role:bazz'
deprecated_reason='foo:bar has been replaced by foo:create_bar',
deprecated_since='N',
)
policy.RuleDefault(
name='foo:create_bar',
check_str='role:bang',
description='Create a bar.',
deprecated_rule=deprecated_rule,
)
Add support for this, with appropriate warnings to nudge people over to
the new, improved way of doing things eventually.
Change-Id: Ie4809c7749242bd092a2677b7545ef281735d984
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
|\ \
| |/ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Many of requests' APIs accept a 'verify' parameter which can be a
boolean value or a path to either a CA cert bundle or a directory of CA
certs. If 'verify=True' is set, requests will look for two environment
variables, 'REQUESTS_CA_BUNDLE' and 'CURL_CA_BUNDLE', that, if set,
should specify a path to CA certs. If either of these are found,
'requests' overrides the 'True' value with the value of the environment
variable [1]. From the docs [2]:
This list of trusted CAs can also be specified through the
REQUESTS_CA_BUNDLE environment variable. If REQUESTS_CA_BUNDLE is not
set, CURL_CA_BUNDLE will be used as fallback.
This can cause test failures on environments where either of these are
set. Ensure this doesn't happen by using the 'EnvironmentVariable'
fixture to unset these environment variables.
[1] https://github.com/psf/requests/blob/v2.25.0/requests/sessions.py#L717-L719
[2] https://requests.readthedocs.io/en/master/user/advanced/#ssl-cert-verification
Change-Id: I808c9102b214aa25144e88e7773a9890ab0a5bdc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
|\ \ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We do not test the olso policy master code changes with
services unit or functional tests. Tempest job initialize
policy once and run with default rules so it will not be able
to catch all the scenario what unit or functional job does.
They initialize or override policy in parallel and sometime
that help to find the issue like
- https://bugs.launchpad.net/oslo.policy/+bug/1914095
Also this will help us to avoid any new breaking release which
is detected at the time when requirement u-c are updated
Example: https://review.opendev.org/c/openstack/requirements/+/773779
Currently this commit adds only nova & neutron testing but we can add
more service testing later.
Change-Id: Ic54b229a4bf1325adac2cab747bcd19b9f8ecb01
|
|\ \ \
| |/ /
|/| | |
|
| | |
| | |
| | |
| | | |
Change-Id: I2837959e8b03f98e8d947787d5c81569fe69acf6
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
When service register their policy rule oslo policy does not
copy the rule and instead work on the original object.
- https://github.com/openstack/oslo.policy/blob/bd9d47aa36ad6f2f4746f09a267d7ce809a820f4/oslo_policy/policy.py#L1104
policy enforcer modify the default rules in
_handle_deprecated_rule().
- https://github.com/openstack/oslo.policy/blob/bd9d47aa36ad6f2f4746f09a267d7ce809a820f4/oslo_policy/policy.py#L767-L774
In any case, oslo policy should make copy of the registered
rules.
Another thing it fix is setting of flag
RuleDefault._deprecated_rule_handled.
Flag _deprecated_rule_handled is set to True when
_handle_deprecated_rule() is called irrespective of it
actually handle the deprecated rule and add it in OR checks.
We should set this flag when acutally deprecated rule is
handled so that if any condition change like config flag or
file rules we correctly handle deprecated rules.
Closes-Bug: #1914095
Closes-Bug: #1914592
Story: 2008556
Task: 41687
Change-Id: I154213dabd4d9eef760f0a4c9a852d504638ca8d
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The policy engine converts simple strings into instances of rule
objects based on a policy DSL. This engine iterates checks and reduces
them after each iteration if performing the conversion on list of check
strings.
When we deprecate policies we apply a logical OR to make upgrades easier
for operators. The logical OR, implemented with an OrCheck, only needs
to be done once per deprecated rule. Today, we're re-initializing an
OrCheck instance each time we load rules, which happens every time
oslo_policy.policy.Enforcer.enforce() is called.
For most OpenStack usage, this isn't noticiable, especially if you're
only using it to enforce access to a specific endpoint. However, this
can get expensive if you're using the enforcer to protect the API,
protect each resource in a response, and protect each attrbute of the
resource (e.g., Neutron makes extensive usage of this pattern to
implement RBAC for resources it's responsible for).
This commit updates the RuleDefault object to track state of handling
deprecated logic ORs so that we only cast the check strings to OrCheck
instances once per rule no matter how many times we call load_rules().
Closes-Bug: 1913718
Change-Id: I539672fc220b8d7e3c47ab3dfa6670b88e3f4093
|
|\ \ \ |
|
| | |/
| |/|
| | |
| | |
| | |
| | |
| | | |
collections.MutableMapping has been deprecated since Python 3.3 and
is removed in Python 3.10. The functionality should be identical.
Change-Id: Ic96309fef409ba01dd24a3a70ff132a9f5352f9c
|
|\ \ \
| |/ /
|/| | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
UPPER_CONSTRAINTS_FILE is old name and deprecated
-https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file
This allows to use lower-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.
[1] https://review.opendev.org/#/c/722814/
Change-Id: I5d023d0c1e74b01bb3743c54f4a0536085b98e93
|
| |/
|/|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
hacking 3.1.0 depended on 'flake8<3.8.0,>=3.6.0', while we were
specifying flake 3.8.3. This resulted in an error when using the
dependency resolver introduced in pip 20.3. Resolve this by bumping to
hacking 3.2.0.
We also remove bandit and pre-commit from test-requirements, since these
are linters which are not managed by upper-constraints and are not
necessary to run tests. oslo.context is also specified in both
requirements.txt and test-requirements.txt, so we remove it from the
latter.
Change-Id: I829870c327b73b583877b9b969ee38f0bcaa1495
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Description and operators are mandatory parameter for
DocumentedRuleDefault but few services still using the
RuleDefault for exmaple glance
- https://github.com/openstack/glance/blob/1344c45772c1a77107f621632642cf1488e431db/glance/policies/image.py#L16
To make work on oslopolicy-j2y-convertor tool to covert
the JSON file for such services we need to pass description
in preparing DocumentedRuleDefault for this tool to generate
the yaml file with rule description. For such cases, it add
rule name as description.
Also for 'operations', do not write it in generate file
for RuleDefault as that end up with blank space.
Change-Id: I910291a152402051b1eac96c3ec16c3f0bb8bbb7
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
As part of community goal[1], each services are changing the default
value of 'CONF.oslo_policy.policy_file' config option from 'policy.json'
to 'policy.yaml'. oslo policy select the default value from
CONF.oslo_policy.policy_file which will be policy.yaml as service will
start changing the default. To avoid breaking the existing deployment which
are relying on old default (policy.json) file, a new fallback logic
is implemented. If new default file 'policy.yaml' does not exist but old
default 'policy.json' exist then fallback to use old default file.
Each services are going to add upgrade checks and warnings for using JSON
formatted policy file so in future we cna remove this fallback logic.
This logic was done in nova in Victoria cycle when nova changed the
default value - https://review.opendev.org/#/c/748059/ . Moving this
to oslo policy side will avoid the duplication on services side.
Also it provides a flag to disable this fallback.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: If72b2fcc3cfd8116b575ed7b9e3870df634fd9af
|
|\ \ |
|
| | |
| | |
| | |
| | | |
Change-Id: I138e5cf6deb82ad29a2bf70392c1859354b84800
|
|\ \ \
| |_|/
|/| | |
|
| |/
| |
| |
| |
| |
| | |
Replace six with Python 3 style code.
Change-Id: I3d0c35e237484409d8410601ec482fac0dacf30d
|
|\ \ |
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.
Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.
To avoid similar gate break in future, we need to bump the hacking min
version.
- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html
Change-Id: I20e3db3b092462c85d784aad51d5e98f3499f3f1
|
|/
|
|
|
|
|
|
| |
As `--format` option was dropped in oslopolicy-policy-generator, remove
examples of the option in documentation.
Change-Id: I20f748dadecc15eaad6326a20a121cab4c1a995e
Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: Ia9f7040f9966f1492c590a005f55ef7f3b67f0c9
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.
See also the PTI in governance [1].
[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: I921aa1033f414ddcf70858eb584bf8515a842dfb
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add file to the reno documentation build to show release notes for
stable/victoria.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.
Change-Id: Ic9fe5dc1119eee908d0381b89f2c50e86ba4a535
Sem-Ver: feature
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).
Fixing:
- bug#1886298
Bump the lower constraints for required deps which added python3.8 support
in their later version.
Story: #2007865
Task: #40207
Closes-Bug: #1886298
[1] https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal.h>
Change-Id: I97072055f880915cef6c5c2f0210730e7bbe5119
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit makes the following minor improvements
in YAML output of oslopolicy-sample-generator.
* Add a blank line between policies.
Previously when a deprecated rule exists there was no blank line
between the deprecated rule and the next rule.
It was not easy to identify the beginning of the next rule.
* Drop unnecessary blank line comment.
If a policy is defined by RuleDefault instead of DocumentedRuleDefault
there is no description and unnecessary blank line comment was added
in an output YAML file.
* Honor newlines in deprecated_text.
Previously newlines in deprecated_text were dropped by
_format_help_text(). Main deprecation message and reason are
processed separately and newlines are not dropped now.
Change-Id: I75889a1b05344a47135419d0553525f54c1a51b8
|
|
|
|
|
|
|
|
|
|
|
|
| |
If any rules present in policy file is exactly same as
defaults then operators do not need to keep these
redundant rules in files. 'oslopolicy-list-redundant' tool
is to detects such rule but we can log warnings also for
such rule to communicate it to the deployer in strong way.
Partial implement blueprint policy-json-to-yaml
Change-Id: Ie31ea13e8ea62bc495ceb1c1694407539e2cab8d
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
JSON support for policy_file has been problematic
since projects started policy-in-code. For example,
generating a sample policy file in JSON results in
all the policy-in-code rules being overridden because
it is not possible to comment out the default rules in JSON.
Asd part of migration of JSON format to YAML, this commit
deprecates the:
1. Deprecate JSON support in oslo.policy.
2. Deprecate JSON output in policy CLI tools including '--format'
option.
Partial implement blueprint policy-json-to-yaml
Change-Id: I5432a8cf80903620f48936cbbfb92ea6b6ff30fa
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add ``oslopolicy-convert-json-to-yaml`` tool which can be
used to convert the json formatted policy file to yaml format.
It takes json formatted policy file as input and convert it to
a yaml formatted policy file similar to 'oslopolicy-sample-generator'
tool except keeping the overridden rule as uncommented.
This tool does the following:
* Comment out any rules that match the default from policy-in-code.
* Keep rules uncommented if rule is overridden.
* Does not auto add the deprecated rules in the file unless it not already
present in the file.
* Keep any extra rules or already exist deprecated rules uncommented
but at the end of the file with a warning text.
I did not add the new functionality in existing 'oslopolicy-policy-upgrade'
tool because the above listed features of new tool end up creating a
complete different code path instead of reusing it from existing tool so it
better to have separate tool which can be removed in future once all deployments
are migrated to YAML formatted file.
This commits add doc and reno also for this tool
Partial implement blueprint policy-json-to-yaml
Change-Id: Icc245951b2992cc09a891516ffd14f3d4c009920
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| | |
The definition of an "access" file is very vague, but oslopolicy-checker
expects it to be a token response body from keystone. If you don't pass
a token response explicitly, oslopolicy-checker will fail.
Change-Id: I5362fabb0344b67996367382dbc173eeaf39b06b
|
| |
| |
| |
| |
| |
| |
| |
| | |
This patch bumps bandit allowed version to >=1.6.0,<1.7.0 in order to
avoid the errors detailed here https://github.com/PyCQA/bandit/pull/393
Change-Id: I0570c916cffc08bcbaebb385a9cc4a4c7038b215
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
| |
stevedore has switched to importlib_metadata[1] and
this breaked unit test as the test relied on internal
implementation of it.
Instead we should switch to mock NamedExtensionManager
that's what called by oslo_policy.
[1] https://review.opendev.org/#/c/739306/
Closes-Bug: #1888208
Change-Id: I993d743c53fa3506ceda3d1f291c12f4635eb60a
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When doing a literal comparison in a rule, it is necessary to enclose
the literal in single quotes. This is not apparent from the existing
docs and is only mentioned in a private module[0] which does not
appear in the published docs. This change adds an example that covers
literal comparisons and briefly discusses how to determine what
fields are available for comparison. The latter should be expanded
upon at some point as it is important for anyone writing their own
policy rules.
Change-Id: I383f179ce274c1cf00f83d006a1dcddd40c52084
0: https://github.com/openstack/oslo.policy/blob/de857746867344c1a3f9f1dadf87b7ae046a1fc1/oslo_policy/_checks.py#L299
|
|/
|
|
|
|
|
|
|
|
|
|
| |
As far as I can tell, mask_dict_password does not modify the object
passed in to it[0]. As such, this deepcopy only adds an unnecessary
requirement on the policy objects that makes it possible for a call
to fail in a different way when debug logging is enabled. Since this
is pretty terrible, let's get rid of it.
Change-Id: I34eace9806e6ed7c9c6206a34f55debc0c20bac6
Closes-Bug: 1886984
0: https://github.com/openstack/oslo.utils/blob/4fe75b7e1bd3144282f107ce7cb61880257c7c1e/oslo_utils/strutils.py#L349
|
|\ |
|
| |
| |
| |
| |
| | |
Change-Id: Ifcfc88a67b038528f03756d550e1ddf8726cb37a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
|