summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Fix host:port handling" into stable/trainstable/trainZuul2022-11-031-2/+4
|\
| * Fix host:port handlingBence Romsics2022-09-091-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we check the EC2 signature without the port part of the host value received, we should properly split host:port. Keep in mind the splitting should work for values like [fc00::]:123 too. Change-Id: I1d90dfcea3568e2a9b22069daa428ea6a2a38bd6 Closes-Bug: #1988168 (cherry picked from commit 6c35b366e3c8c6d7f47471b93f5315582301c5ef) (cherry picked from commit d39790ac4e9dc25af09cdddc6217e36bacbc2bb1) (cherry picked from commit 0bb9cdee71805af1a7cb0a7db110b336eae5da1e) (cherry picked from commit aa50b963cce20a76db0c4834b3716d3658c784af) (cherry picked from commit fe837d87c949f6a2347cf79d81b66214f0a449b3) (cherry picked from commit 1ab860a08e527ca9e0c82a49fbf004d415fec991)
* | Merge "Stop adding entry in local_user while updating ephemerals" into ↵Zuul2022-10-284-2/+80
|\ \ | |/ |/| | | stable/train
| * Stop adding entry in local_user while updating ephemeralsPedro Martins2022-07-214-2/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem description =================== Today we have a consistency problem when updating federated users via OpenStack. When I update a ephemeral user via OpenStack, a registry in the local_user table is created, making this user having entries in user, local_user and federated_user tables in the database. Furthermore, if I try to do some operations using this user (that has entries in all three tables), I get a "More than one user exists with the name ..." error from the OpenStack Keystone API. It happens because the user has an entry in both local_user and federated_user tables. I fix the persistence in the local_user table for ephemeral users when doing updates. Proposal ======== I fix the problem with creating an entry in the local_user table while updating an ephemeral user Closes-Bug: #1848342 Change-Id: I2ac6e90f24b94dc5c0d9c0758f008a388597036c (cherry picked from commit 7597ecc1350eb6918c09585e4116911102acb54a)
* | Merge "Remove the note of training-labs" into stable/trainZuul2022-07-253-24/+0
|\ \ | |/ |/|
| * Remove the note of training-labsHan Guangyu2022-04-223-24/+0
| | | | | | | | | | | | | | | | | | | | | | | | Training-labs had been officially retired as no maintainer. The information of training-labs has been deleting in the openstack documentatioan. It is not appropriate to continue the presentation in note form here. [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025586.html [2] https://opendev.org/openstack/training-labs/commit/e78d74f10558ab3e6a9a6fd7d45e617c15e9c3d8 Change-Id: I0ac3d05389041ac58fe2347171541ffaaf151fdf
* | Fix issue with LDAP backend returning bytes instead of stringGrzegorz Grasza2022-02-071-2/+17
|/ | | | | | | | | | | | | | When connecting to some LDAP server software, the ldap client returns bytes instances instead of the expected strings. This can result in either being transparently converted to strings, when the data is inserted via sqlalchemy into the database, or could be used as input to other functions, and/or cached, which causes unexpected results. Closes-Bug: #1952458 Resolves: rhbz#1964872 Change-Id: I77148641715efe09e3adc2e9432e66e50fb444b4 (cherry picked from commit 1e0cd90191663c100c165d4c6a2b1ca796b5af25)
* Merge "Fix typos in application credential policies" into stable/trainZuul2021-10-121-2/+2
|\
| * Fix typos in application credential policiesLance Bragstad2021-10-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We updated these policies when we introduces system scope and default roles, but the policy names accidentally changed, which makes the policy files render with an alias because oslo.policy thinks the names are changing. Conflicts: keystone/common/policies/application_credential.py in later releases the deprecated parameters were moved from the DocumentedRuleDefault object to the DeprecatedRule object, which is a non-functional change. Change-Id: I1121f1abe769ee83ffc285103a95ee95540ce727 (cherry picked from commit 60e898c47038667e66a54e0a9a6cd7b91e115f55) (cherry picked from commit 7b28f1b3b47d0e4a22afe99c64d047016a772da5) (cherry picked from commit 14d2f5944ce9ef0cc881b91463df7cf3a1114d5b) (cherry picked from commit 4063ad98e7bebc4c56eb92be742cfda7254c27e7) (cherry picked from commit aa9459447f61fd51df20e1cbba8146033696bd01)
* | Merge "Fix typos in ec2 credential policies" into stable/trainZuul2021-10-111-6/+6
|\ \
| * | Fix typos in ec2 credential policiesLance Bragstad2021-10-071-6/+6
| |/ | | | | | | | | | | | | | | | | | | | | | | | | There was a trailing s in two of these policies and it caused the policy names to mismatch, which causes confusion with the rendered policy files and potentially causes uses with deprecation logic. Change-Id: I54021986d17c57d7733d53caa4032c2767eaf25e (cherry picked from commit 82da8824df0f56ef4e137805bf32d647cef1ea59) (cherry picked from commit 6dff22b5baa1a19842aca435782fe1f9789f72cc) (cherry picked from commit a57ae85c9699e7a560b7ffe9786ed0a6453c1e86) (cherry picked from commit 65c99d6efb6a83fcc16ddf957e59297763e85b6c) (cherry picked from commit b38928d0a26e8ab4c1496e2eaea7598cf72e5b37)
* | Fix typo in identity provider policiesLance Bragstad2021-10-071-4/+4
|/ | | | | | | | | | | | | | | | | This cause the sample generated policy file to alias the old name with the new policy name, which isn't needed since we're not renaming these policies at all and it was likely a typo. Conflicts: keystone/common/policies/identity_provider.py In later releases the deprecation parameters were moved up to the deprecated options and not in the DocumentedRule defaults. Change-Id: Idfd9adbbe800bbc21814d94002a2b61524cce28a (cherry picked from commit c10d5c88ef40e63d4dfefb792d6c3d68acd72dd9) (cherry picked from commit bdd8f82f60d2e46e5f6951c4407366b89591cde5) (cherry picked from commit f742fadef1718e070ff4e151d400a96dc6acf74e) (cherry picked from commit 3fd71f5a933052b0a3fd4dadb3a754ac544348f3)
* Remove group deletion for non-sql driver when removing domains.Sami MAKKI2021-07-152-12/+19
| | | | | | | | | As LDAP is now read-only, trying to remove it was throwing an error. We now only try to delete it when the driver is sql-based. Change-Id: I15b92b35b31d0e5d735a629e7c154ddd7bdda03d Closes-bug: #1848238 (cherry picked from commit d6977a0e9b3ed8ae80527d6f6ace67b687b46c60)
* Merge "Hide AccountLocked exception from end users" into stable/traintrain-em16.0.2Zuul2021-06-044-6/+16
|\
| * Hide AccountLocked exception from end usersGage Hugo2021-05-104-6/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change hides the AccountLocked exception from being returned to the end user to hide sensitive information that a potential malicious person could gain insight from. The notification handler catches the AccountLocked exception as before, but after sending the audit notification, it instead bubbles up Unauthorized rather than AccountLocked. Co-Authored-By: Samuel de Medeiros Queiroz <samueldmq@gmail.com> Change-Id: Id51241989b22c52810391f3e8e1cadbf8613d873 Related-Bug: #1688137 (cherry picked from commit ac2631ae33445877094cdae796fbcdce8833a626)
* | Merge "Use app cred user ID in policy enforcement" into stable/trainZuul2021-06-033-2/+118
|\ \ | |/ |/|
| * Use app cred user ID in policy enforcementLance Bragstad2021-01-223-2/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The application credential policies use the `rule:owner` policy to allow users to manage their own credentials. The policy engine pulled the user_id attribute from the request path instead of the actual application credential. This allowed for users to exploit the enforcement and view or delete application credentials they don't own. This commit attempts to resolve the issue by updating the flask parameters before they're translated to policy arguments and target data, prior to policy enforcement. This commit also deviates slightly from backports to stable/ussuri, stable/victoria, and master (wallaby). This is because newer branches use `http.client` to assert status codes and in stable/train we are still using `http_client`. This change is functionally the same. Change-Id: I903d20fa41270499ca1c39d296120dd97cef5405 Closes-Bug: 1901207 (cherry picked from commit 2d7bf10a5a145ed3b6e34c3cb95e05fb7e33e0d1) (cherry picked from commit 80832de6ced534bccbf7cfe24a6a01c8262c1779) (cherry picked from commit 661bf6a1dca502f37b5da9995c45ba60581e5e97)
* | Merge "Retry update_user when sqlalchemy raises StaleDataErrors" into ↵Zuul2021-04-283-0/+51
|\ \ | | | | | | | | | stable/train
| * | Retry update_user when sqlalchemy raises StaleDataErrorsLance Bragstad2021-03-313-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Keystone's update_user() method in the SQL driver processes a lot of information about how to update users. This includes evaluating password logic and authentication attempts for PSI-DSS. This logic is evaluated after keystone pulls the user record from SQL and before it exits the context manager, which performs the write. When multiple clients are all updating the same user reference, it's more likely they will see an HTTP 500 because of race conditions exiting the context manager. The HTTP 500 is due to stale data when updating password expiration for old passwords, which happens when setting a new password for a user. This commit attempts to handle that case more gracefully than throwing a 500 by detecting StaleDataErrors from sqlalchemy and retrying. The identity sql backend will retry the request for clients that have stale data change from underneath them. Conflicts: keystone/tests/unit/test_backend_sql.py due to import order differences between train and ussuri. Also adjust the expected log message since the method path is different compared to older releases, which have the driver name in them (e.g., Identity). Change-Id: I75590c20e90170ed862f46f0de7d61c7810b5c90 Closes-Bug: 1885753 (cherry picked from commit ceae3566e83b26fd6a1679154eae9b0cef29da64) (cherry picked from commit f47e635b8041542faa05e64606e66d2fbbc5f284) (cherry picked from commit 5b7d4c80d484262018f937083050844648f07a11) (cherry picked from commit 07d3a3d3ff534a5295842d4f236042b30536cd82) (cherry picked from commit d4f48fc4e53f71d653e133104854f064fbb1b25f)
* | | Support bytes type in generate_public_ID()Keigo Noha2021-03-183-2/+35
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | python-ldap3.0 or later running on python3 uses str or bytes data type according to what fields are returned. local_id may be a bytes data type. To handle it properly, mapping[key] needs to be examined for identifying its data type and what python version is used. Closes-Bug: #1901654 Change-Id: Iac097235fd31e166028c169d14ec0937c663c21c (cherry picked from commit f7df9fba828328d8b20e85d711c1d27c77089632) (cherry picked from commit 5b860e0b3b4e318b91325996156bae3f99abd6c7) (cherry picked from commit f4819fe36f087b2f652dc68fd33880860570f8d9)
* | Merge "Delete system role assignments from system_assignment table" into ↵Zuul2021-03-183-0/+40
|\ \ | |/ |/| | | stable/train
| * Delete system role assignments from system_assignment tableVishakha Agarwal2020-10-303-0/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch ensures to delete the system role assignments from all the assignment tables in keystone after deleting the role user has over the system. This also make sure of deleting stale role assignments before deleting role for the deployments that are already in this state. Closes-Bug: #1878938 Change-Id: I4df19c45c870ff3fb78578ca1fb7dd0d35da3c82 (cherry picked from commit c1dcbb05b4488f1fa3e7af4d9171d11702d94119) (cherry picked from commit b83170a386ba8da2195c7494d04d832ce9b6d7b0) (cherry picked from commit 6f93063ff95f3c65af106a09281427e411d01850) (cherry picked from commit 7ac0891375656d156f1a9601e3520052dddb39d0)
* | Drop lower-constraints jobKristi Nikolla2021-01-221-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lower-constraints is not a requirement of the OpenStack Python PTI [0] and there currently is a discussion on the mailing list [1] about dropping the test, with the oslo team already having done so [2]. The new dependency resolver in pip fails due to incompatible dependency versions in our lower-constraints file, meaning that we were never providing any real guarantees with it. To unblock the CI, I am disabling lower-constraints job for now, with the option to reenable it in case we fix the constraints, and based on the outcome of the mailing list discussions and consensus. [0]. https://governance.openstack.org/tc/reference/pti/python.html [1]. http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019672.html [2]. http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019659.html Change-Id: I47e747058891596e7717848a0b0bc10f0a235b2f (cherry picked from commit d6610594d1b766a8ee3ac65182b951f2a3d431f7) (cherry picked from commit 42faeb277b6eb4b4a772f516a6e2b547f2b2a2b4) (cherry picked from commit 62c3d8eeb63ddc0f6b7f1505ed9a8b393a80e548)
* | Merge "fix link in release note of bug/1794527" into stable/trainZuul2021-01-061-1/+1
|\ \
| * | fix link in release note of bug/1794527Maurice Escher2020-05-261-1/+1
| | | | | | | | | | | | | | | Change-Id: I120ef1c0c8259c85b6030f2db0a649c71b990879 closes-bug: 1877393
* | | Merge "Properly handle octet (byte) strings when converting LDAP responses" ↵Zuul2020-11-183-1/+30
|\ \ \ | | | | | | | | | | | | into stable/train
| * | | Properly handle octet (byte) strings when converting LDAP responsesLance Bragstad2020-11-113-1/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If LDAP returns a UUID as an octet string the LDAP driver will fail to convert it to something meaningful. The error usually looks something like: ID attribute objectGUID not found in LDAP object Microsoft AD's `objectGUID` parameter is stored and transmitted as an octet string [0]. If you attempt to use the `objectGUID` to generate user or group IDs, you'll get an HTTP 404 because keystone can't decode it properly. This is unfortunate because `objectGUID` are a fixed length, UUID format, and ideal for generating IDs in keystone. As opposed to using the object's CN, which is variable length, and can generate hashes that are larger than keystone's database table limit for user IDs. [0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid Conflicts: keystone/identity/backends/ldap/common.py Due to python3 string detection differences between ussuri and train. Change-Id: Id80b17bdff015e10340e636102576b7435bd564f Closes-Bug: 1889936 (cherry picked from commit 8bf222ac5d390e25d306d35f69bd958b18bee4d8) (cherry picked from commit d5870f69c12c034dd97b164345e85e6259ee7abe)
* | | | Implement more robust connection handling for asynchronous LDAP callsLance Bragstad2020-11-032-38/+75
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Keystone's paging implementation contains a memory leak. The issue is noticeable if you integrate keystone with an LDAP server that supports paging and set `keystone.conf [ldap] page_size` to a low integer (e.g., 5). Keystone's LDAP backend uses `python-ldap` to interact with LDAP servers. For paged requests, it uses `search_ext()`, which is an asynchronous API [0]. The server responds with a message ID, which the client uses to retrieve all data for the request. In keystone's case, the `search_ext()` method is invoked with a page control that tells the server to deliver responses in increments according to the page size configured with `keystone.conf [ldap] page_size`. So long as the client has the original connection used to fetch the message ID, it can request the rest of the information associated to the request. Keystone's paging implementation loops continuously for paged requests. It takes the message ID it gets from `search_ext()` and calls `result3()`, asking the server for the data associated with that specific message. Keystone continues to do this until the server sends an indicator that it has no more data relevant to the query (via a cookie). The `search_ext()` and `result3()` methods must use the same LDAP connection. Given the above information, keystone uses context managers to provide connections. This is relevant when deploying connection pools, where certain connections are re-used from a pool. Keystone relies on Python context managers to handle connections, which is pretty typical use-case for context managers. Connection managers allow us to do the following (assuming pseudocode): with self.get_connection as conn: response = conn.search_s() return format(response) The above snippet assumes the `get_connection` method provides a connection object and a callable that implements `search_s`. Upon exiting the `with` statement, the connection is disconnected, or put back into the pool, or whatever the implementation of the context manager decides to do. Most connections in the LDAP backend are handled in this fashion. Unfortunately, the LDAP driver is somewhat oblivious to paging, it's control implementation, or the fact that it uses an asynchronous API. Instead, the driver leaves it up to the handler objects it uses for connections to determine if the request should be controlled via paging. This is an anti-pattern since the backend establishes the connection for the request but doesn't ensure that connection is safely handled for asynchronous APIs. This forces the `search_ext()` and `result3()` implementations in the PooledLDAPHandler to know how to handle connections and context managers, since it needs to ensure the same connection is used for paged requests. The current code tried to clean up the context manager responsible for connections after the results are collected from the server using the message ID. I believe it does this because it needs to get a new connection for each message in the paged results, even though it already operates from within a connection established via a context manager and the PooledLDAPHandler almost always returns the same connection object from the pool. The code tries to use a weak reference to create a callback that tears down the context manager when nothing else references it. At a high-level, the idea is similar to the following pseudocode: with self.get_connection as conn: while True: ldap_data = [] context_manager = self.get_connection() connection = context_manager.__enter__() message_id = connection.search_ext() results = connection.result3(message_id) ldap_data.append(results) context_manager.__exit__() I wasn't able to see the callback get invoked or work as described in comments, resulting in memory bloat, especially with low page sizes which results in more requests. A weak reference invokes the callback when the weak reference is called, but there are no other references to the original object [1]. In our case, I don't think we invoke that path because we don't actually do anything with the weak reference. We assume it's going to run the callback when the object is garbage collected. This commit attempts to address this issue by using the concept of a finalizer [2], which was designed for similar cases. It also attempts to hide the cleanup implementation in the AsynchronousMessage object, so that callers don't have to worry about making sure they invoke the finalizer. An alternative approach would be to push more of the paging logic and implementation up into the LDAP driver. This would make it easier to put the entire asynchronous API flow for paging into a `with` statement and relying on the normal behavior of context managers to clean up accordingly. This approach would remove the manual cleanup invocation, regardless of using weak references or finalizer objects. However, this approach would likely require a non-trivial amount of design work to refactor the entire LDAP backend. The LDAP backend has other issues that would complicate the re-design process: - Handlers and connection are generalized to mean the same thing - Method names don't follow a convention - Domain-specific language from python-ldap bleeds into keystone's implementation (e.g., get_all, _ldap_get_all, add_member) at different points in the backend (e.g., UserApi (BaseLdap), GroupApi (BaseLdap), KeystoneLDAPHandler, PooledLDAPHandler, PythonLDAPHandler) - Backend contains dead code from when keystone supported writeable LDAP backends - Responsibility for connections and connection handling is spread across objects (BaseLdap, LDAPHandler) - Handlers will invoke methods differently based on configuration at runtime, which is a sign that the relationship between the driver, handlers, and connection objects isn't truely polymorphic While keeping the logic for properly handling context managers and connections in the Handlers might not be ideal, it is a relatively minimal fix in comparison to a re-design or backend refactor. These issues can be considered during a refactor of the LDAP backend if or when the community decides to re-design the LDAP backend. [0] https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap.html#ldap.LDAPObject.search_ext [1] https://docs.python.org/3/library/weakref.html#weakref.ref [2] https://docs.python.org/3/library/weakref.html#finalizer-objects Closes-Bug: 1896125 Change-Id: Ia45a45ff852d0d4e3a713dae07a46d4ff8d370f3
* | | Make opensuse jobs nonvotingColleen Murphy2020-10-262-3/+2
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The opensuse images tend to be less stable, and the opensuse nodeset was moved to openSUSE 15.2 which does not work properly with devstack for the train and lower branches. Ussuri works for the moment, and Victoria+ have been moved to Ubuntu, so this patch is targeted for stable/train and prior to disable voting for opensuse-based jobs. To fix the gate, patch also pulls in this backport: Update amqp lower constraint Newer versions of kombu rely on updated versions of amqp [0]. Keystone's lower-constraints job is failing consistently across multiple branches because it's referencing amqp 2.2.2 when we should be relying on a newer version (5.0.0+). You can verify this by using `pip3 check` with the lower-constraints tox environment. Without this patch, you'll likely see the test_notifications tests fail with something like: ModuleNotFoundError: No module named 'vine.five' Which spirals into subsequent failures mocking messaging drivers. [0] https://github.com/celery/celery/issues/3547 (cherry picked from commit c495defc2bd4f2ef5a38d2a9e4d5aac5653fb406) (cherry picked from commit 12891a039fb1dae7ba22884cf78ad7d1faa8569d) (cherry picked from commit d435a915160a9876b58f2a04696d9430790e5cdc) Change-Id: I2895300556b83e943ce5e5fd72e4b86317a306a9
* | Fix lower-constraint for PyMySQLVishakha Agarwal2020-08-111-0/+1
|/ | | | | | | | | | | | | | | | keystone does not have any lower constraint for PyMySQL so the latest version 0.10.0 is picked by the job which is failing [1] In OpenStack, PyMySQL upper constraint is .9.3 means that version is tested not 0.10.0 [2] let's add PyMySQL lower constraint also so that we test lower-constraint job with correct lower version. [1]https://zuul.opendev.org/t/openstack/build/3077d96f4fff4b7985cb763d0635d471/log/job-output.txt#621 [2]https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L384 Change-Id: I3834b3b34641c006c70614d5331d292c41f8a346 Closes-Bug: #1888886 (cherry picked from commit 3de085b1eb9750cb0d0d25cb468250cf34804eaf) (cherry picked from commit 300e79e93b9966e988ec92a74bf9b49b71d77d1b)
* Merge "Change time faking for totp test" into stable/trainZuul2020-05-201-3/+3
|\
| * Change time faking for totp testGage Hugo2020-05-141-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This change moves the time mocking from using freezegun to using oslo.utils TimeFixture for the unit test test_with_passcode_in_previous_windows_extended, which was occasionally failing with 401 errors due to the totp creation time not properly be faked with 4 extended windows. Closes-Bug: #1843464 Change-Id: I3aefd99907fbc2d03538c9814f7279b282715679 (cherry picked from commit 6525203c1af9ecdf7c23af3f25e45f0db9f6fed2)
* | Merge "Revert "Temporarily disable k2k tests on train and stein"" into ↵Zuul2020-05-191-1/+2
|\ \ | | | | | | | | | stable/train
| * | Revert "Temporarily disable k2k tests on train and stein"Colleen Murphy2020-05-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 5c34cb6c7875e0332ccbb23e68457399a13530fc. Depends-On: https://review.opendev.org/727000 Depends-On: https://review.opendev.org/726727 Depends-On: https://review.opendev.org/727372 Change-Id: I1cb9044c3b08bf4367bf006e1c7f943df505b86f
* | | Merge "Refactor some ldap code to implement TODOs" into stable/trainZuul2020-05-192-26/+21
|\ \ \
| * | | Refactor some ldap code to implement TODOsRadosław Piliszek2020-03-062-26/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This implements TODOs added in [1], as promised in [2]. The first TODO is realised only partially because most ldap code actually relies on having two connections obtained from the pool. This optimizes mixin code by removing extra ldap calls. There is no change in the observed behaviour of integration. This also removes some duplication and refactors names to avoid some confusion related to dn/object_id. Backport to: Train, Stein (with [1]&[3]), Rocky (with [1]&[3]), Queens (with [1]&[3]) [1] c7fae97d873f72068ca65538ec5b5919c0ac7d5a [2] https://review.opendev.org/683303 [3] 19d4831daa3991bed48fb364fa05927740c96445 Change-Id: I22f3bce647182996dfc06084ee6d4989449e3d2d (cherry picked from commit a6bb81146ff2126f055834459f428cf97080466f)
* | | | Stop explicitly requiring pycodestyleStephen Finucane2020-05-143-4/+5
| |_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | pip doesn't have a dependency resolver. Instead, it "simply uses the first specification it finds for a project." [1] In Train, keystone switched from hacking 0.12.x/0.13.x to hacking 1.1.x [2]. That change explicitly added a pycodestyle dependency for reasons that aren't entirely clear to me, but pip's broken dependency resolution leads to the below funkiness when trying to install the dependencies. ERROR: flake8 2.6.2 has requirement pycodestyle<2.1,>=2.0, but you'll have pycodestyle 2.5.0 which is incompatible. As seen below, this can be easily reproduced and seems to happen because pip doesn't go further than one level of dependencies, meaning it knows about the dependency on flake8<2.7.0,>=2.6.0 from hacking, but not the dependency on pycodestyle<2.1,>=2.0 that this in-turn introduces. $ virtualenv venv $ source venv/bin/activate $ (venv) cat requirements.txt hacking>=1.1.0,<1.2.0 # Apache-2.0 pycodestyle>=2.0.0 # MIT License $ pip install -r requirements-new.txt Collecting hacking<1.2.0,>=1.1.0 Using cached ... Collecting pycodestyle>=2.0.0 Using cached ... Collecting six>=1.10.0 Using cached ... Collecting flake8<2.7.0,>=2.6.0 Using cached ... Collecting pbr!=2.1.0,>=2.0.0 Using cached ... Collecting mccabe<0.6,>=0.2.1 Using cached ... Collecting pyflakes!=1.2.0,!=1.2.1,!=1.2.2,<1.3,>=0.8.1 Using cached ... ERROR: flake8 2.6.2 has requirement pycodestyle<2.1,>=2.0, but you'll have pycodestyle 2.5.0 which is incompatible. Installing collected packages: six, pycodestyle, mccabe, pyflakes, flake8, pbr, hacking Successfully installed flake8-2.6.2 hacking-1.1.0 mccabe-0.5.3 pbr-5.4.3 pycodestyle-2.5.0 pyflakes-1.2.3 six-1.12.0 The solution is simple: stop explicitly requiring this dependency and instead rely on flake8 bringing it in. [1] https://pip.pypa.io/en/stable/user_guide/#requirements-files [2] I3fc591e09c1e25a3bd2a3922880772ea9617f1e3 This cherry-pick includes an update to setup.cfg to align the python-memcached version with global requirements. Change-Id: Ic0991d3eeae018609be0ecbd43fa0b0b9f13d6ba Signed-off-by: Stephen Finucane <sfinucan@redhat.com> (cherry picked from commit 5c71ebd7a92d25df83e2e7cc5fad9990e9eebbf5)
* | | Merge "Fix security issues with EC2 credentials" into stable/train16.0.1Zuul2020-05-118-62/+606
|\ \ \
| * | | Fix security issues with EC2 credentialsColleen Murphy2020-05-098-62/+606
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change addresses several issues in the creation and use of EC2/S3 credentials with keystone tokens. 1. Disable altering credential owner attributes or metadata Without this patch, an authenticated user can create an EC2 credential for themself for a project they have a role on, then update the credential to target a user and project completely unrelated to them. In the worst case, this could be the admin user and a project the admin user has a role assignment on. A token granted for an altered credential like this would allow the user to masquerade as the victim user. This patch ensures that when updating a credential, the new form of the credential is one the acting user has access to: if the system admin user is changing the credential, the new user ID or project ID could be anything, but regular users may only change the credential to be one that they still own. Relatedly, when a user uses an application credential or a trust to create an EC2 credential, keystone automatically adds the trust ID or application credential ID as metadata in the EC2 access blob so that it knows how the token can be scoped when it is used. Without this patch, a user who has created a credential in this way can update the access blob to remove or alter this metadata and escalate their privileges to be fully authorized for the trustor's, application credential creator's, or OAuth1 access token authorizor's privileges on the project. This patch fixes the issue by simply disallowing updates to keystone-controlled metadata in the credential. 2. Respect token roles when creating EC2 credentials Without this patch, a trustee, an application credential user, or an OAuth1 access token holder could create an EC2 credential or an application credential using any roles the trustor, application credential creator, or access token authorizor had on the project, regardless of whether the creator had delegated only a limited subset of roles. This was because the trust_id attribute of the EC2 access blob was ignored, and no metadata for the application credential or access token was recorded either. This change ensures that the access delegation resource is recorded in the metadata of the EC2 credential when created and passed to the token provider when used for authentication so that the token provider can look up the correct roles for the request. Conflicts (six removal in e2d83ae9, pep8 fixes in e2d83ae9, test helper in 52da4d0e12): keystone/api/credentials.py keystone/tests/unit/test_v3_application_credential.py keystone/tests/unit/test_v3_credential.py Depends-on: https://review.opendev.org/726526 Change-Id: I39d0d705839fbe31ac518ac9a82959e108cb7c1d Closes-bug: #1872733 Closes-bug: #1872755 Closes-bug: #1872735 (cherry picked from commit 37e9907a176dad6843819b1bec4946c3aecc4548) (cherry picked from commit 2f2736ebb267c757ad77fcf25ee0aaeefab2a09d)
* | | | Merge "Ensure OAuth1 authorized roles are respected" into stable/trainZuul2020-05-113-0/+50
|\ \ \ \ | |/ / / | | | / | |_|/ |/| |
| * | Ensure OAuth1 authorized roles are respectedColleen Murphy2020-05-033-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this patch, when an OAuth1 request token is authorized with a limited set of roles, the roles for the access token are ignored when the user uses it to request a keystone token. This means that user of an access token can use it to escallate their role assignments beyond what was authorized by the creator. This patch fixes the issue by ensuring the token model accounts for an OAuth1-scoped token and correctly populating the roles for it. Change-Id: I02f9836fbd4d7e629653977fc341476cfd89859e Closes-bug: #1873290 (cherry picked from commit 6c73690f779a42a5c62914b6bc37f0ac2f41a3e3) (cherry picked from commit ba89d27793c2d3a26ad95642660fa9bd820ed3be)
* | | Temporarily disable k2k tests on train and steinColleen Murphy2020-05-091-2/+1
|/ / | | | | | | | | | | | | | | | | | | | | | | The opensuse nodepool node is going through experimental changes that are affecting devstack runs on stable branches[1]. This change temporarily disables them so that we can get critical security bugfixes through the gate. These tests are specific to keystone federation, not general keystone or devstack functionality. [1] https://zuul.opendev.org/t/openstack/build/61a6a3c542e54b3589b201c0d827f0fc Change-Id: If9430fb854c9d14147f852883492cf8704cfade7
* | Check timestamp of signed EC2 token requestColleen Murphy2020-05-014-3/+191
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | EC2 token requests contain a signature that signs the entire request, including the access timestamp. While the signature is checked, the timestamp is not, and so these signed requests remain valid indefinitely, leaving the token API vulnerable to replay attacks. This change introduces a configurable TTL for signed token requests and ensures that the timestamp is actually validated against it. The check will work for either an AWS Signature v1/v2 'Timestamp' parameter[1] or the AWS Signature v4 'X-Aws-Date' header or parameter[2]. Although this technically adds a new feature and the default value of the feature changes behavior, this change is required to protect credential holders and therefore must be backported to all supported branches. [1] https://docs.aws.amazon.com/general/latest/gr/signature-version-2.html [2] https://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html Conflicts due to six removal in e2d83ae9: keystone/api/_shared/EC2_S3_Resource.py keystone/tests/unit/test_contrib_ec2_core.py Change-Id: Idb10267338b4204b435df233c636046a1ce5711f Closes-bug: #1872737 (cherry picked from commit ab89ea749013e7f2c46260f68504f5687763e019) (cherry picked from commit 8d5becbe4b463f6a5a24a1929dd0f48dab6ae027)
* Tell reno to ignore the kilo branchKristi Nikolla2020-02-212-0/+6
| | | | | | | | | | | | | | | | | | | | This patch was adopted from Idd56c3f37d5786daae7181c36d38d267cbf1885c Original description: When reno 3.x runs under setuptools, it scans all of the branches it can find, including any that look like they're closed and have an -eol tag. The old kilo branch in this repository has a jumbled history that somehow makes it look like it should include tags that it doesn't. We know that there are no release notes in that branch, because reno wasn't adopted while it was open. The releasenotes/source/index.rst links to separate release notes in the wiki. This patch tells reno to ignore that branch so that it doesn't throw an exception when it gets confused about the old tag. Change-Id: I28fd0fd499c40e33ff164fd643dadd6ac9009b17 Co-Authored-By: Doug Hellmann <doug@doughellmann.com> Co-Authored-By: Sean McGinnis <sean.mcginnis@gmail.com>
* Constraint dependencies for docs buildKristi Nikolla2020-02-191-0/+1
| | | | | | | | | | | | | | | | Without this patch, new library releases (even if not specifically related to sphinx or docs) can break the docs build. For example, the latest Werkzeug release is breaking the docs for the stable/stein branch due to an import error that was fixed in later branches. We shouldn't be letting new library releases affect the docs environment for the same reason we want to control the other tox environments. This change adds -c to the docs tox environment so ensure that all libraries installed in this environment are known good versions. Manual port of 13410383cf55699f756c6ce01301d7e67c65b575 edited to reference train instead of master. Change-Id: I987918023525ff165ada691feda23ee86c7e36de
* Merge "Fix token auth error if federated_groups_id is empty list" into ↵Zuul2020-02-123-3/+28
|\ | | | | | | stable/train
| * Fix token auth error if federated_groups_id is empty listshenjiatong2020-01-023-3/+28
| | | | | | | | | | | | | | | | | | `federation_group_ids` could be zero length list, so deciding whether a token is federated by checking if it is none. Change-Id: I0f4b9e24d949aa4838ee721a165999b29c684d32 Closes-Bug: #1856962 (cherry picked from commit f0d964e66675037d62ad17847a966e71720dbd54)
* | Merge "Always have username in CADF initiator" into stable/trainZuul2020-02-113-1/+39
|\ \
| * | Always have username in CADF initiatorGage Hugo2020-02-013-1/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current initiator object for CADF notifications does not include the username of the user who initiated the action, which leads to issues when using an LDAP backend and not having a direct way to map a username to a user id. This change makes it so that the initiator object for CADF notifications always contains the username for a user as well as the user id. This follows along with the CADF standard for OpenStack[0]. [0] https://www.dmtf.org/sites/default/files/standards/documents/DSP2038_1.1.0.pdf#page=12 Closes-Bug: #1856904 Change-Id: I833e6e0d7792acf49f816050ad7a63e8ea4f702f (cherry picked from commit 95edaaab06c6da761411ef97bc2545d86d579215)
* | | Add voting k2k testsColleen Murphy2020-02-042-0/+20
|/ / | | | | | | | | | | | | | | | | | | | | With the addition of K2K-specific tests in the tempest plugin and a config toggle in the plugin to disable use of the external IdP, we can safely add a voting federation job. This also fixes the devstack plugin to install the xmlsec1 tool which is needed for K2K. Change-Id: I9dc634e073657ff337751ec67363a57bd10e20d4 Depends-on: https://review.opendev.org/689222 (cherry picked from commit fb0be8e59917b5b637926d9526a64676af07ebf0)
View Lorry Controller Status