summaryrefslogtreecommitdiff
path: root/oslo_db
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Add release note for base test class removal"13.0.0Zuul2023-04-172-5/+5
|\
| * Add release note for base test class removalStephen Finucane2023-04-172-5/+5
| | | | | | | | | | | | | | | | | | In change I1e71150ba6daeba464b6ed8d46163f1f34959db3 we removed the legacy base test classes, first deprecated in 2015. We forgot to include a release note, however. Address this now. Change-Id: I4d66f0308b89a187143ef6c8495383fe60043c14 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Remove logic for SQLAlchemy < 1.4"Zuul2023-04-173-46/+3
|\ \
| * | Remove logic for SQLAlchemy < 1.4Stephen Finucane2023-04-173-46/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Change I8629225eeb51d95264d8a3e4b719268bb1597f4f bumped the minimum version for SQLAlchemy to 1.4, meaning this logic is now dead. Remove it. Change-Id: I4d4a58e15e840ecfa63e15c709617a65642c8323 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | | Merge "Match exceptions with multiple lines"Zuul2023-04-171-44/+45
|\ \ \ | |/ /
| * | Match exceptions with multiple linesStephen Finucane2023-04-171-44/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We were creating regexes without the DOTALL flag, which meant '.' wasn't matching newlines. This meant exceptions that contained multiple lines would not be caught. For example, in my environment where Kerberos is used, I see the following otherwise harmless message: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: could not initiate GSSAPI security context: No credentials were supplied, or the credentials were unavailable or inaccessible: Configuration file does not specify default realm connection to server at "localhost" (::1), port 5432 failed: FATAL: database "non_existent_database" does not exist The presence of that newline causes our matchers to fail and the exception is not wrapped. Correct this. In the meanwhile, we reformat the function that does the wrapping to make it a little flatter. This was difficult to modify (for debugging purposes) due to the level of indentation. Change-Id: I5396a5a3272e6984954d819cfc71507283c775db Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | | Merge "Remove dead code"Zuul2023-04-172-81/+0
|\ \ \ | |/ /
| * | Remove dead codeStephen Finucane2023-04-172-81/+0
| | | | | | | | | | | | | | | | | | | | | | | | This should have been removed in change Ic3d6bd318038d723b0d50d39e45f8e26289e9a57 but was missed. Change-Id: I4e1faa2c617ac19e7c9766e99ee9012ad9298d31 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | | Merge "Don't sleep in tests"Zuul2023-04-171-2/+4
|\ \ \ | |/ / | | / | |/ |/|
| * Don't sleep in testsStephen Finucane2023-04-171-2/+4
| | | | | | | | | | | | | | Mock it out, reducing test run of ~3 and ~15 seconds to milliseconds. Change-Id: Ice3a0c0d0a5b8c2920c7f775ff8ce974b572c66e Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Remove legacy base test classes"Zuul2023-04-175-344/+1
|\ \ | |/ |/|
| * Remove legacy base test classesStephen Finucane2023-03-225-344/+1
| | | | | | | | | | | | | | | | The fixtures have been available for a very long time now. We can drop these. Change-Id: I1e71150ba6daeba464b6ed8d46163f1f34959db3 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Use SQLAlchemy native pre-pingStephen Finucane2023-04-064-111/+267
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This functionality has been available upstream since SQLALchemy 1.2 [1]. However, for oslo.db to use this feature while maintaining full behavior, we need at least SQLAlchemy 2.0.5 to provide complete event support. In particular, oslo.db adds several new "is disconnect" conditions including those specific to Galera. Behavior of the handle_error event is modified to expect the "pre-ping" calling form, which may not have an "engine" present (only a dialect), and additionally takes advantage of the new is_pre_ping attribute which informs on the correct way to affect the disconnection status within the ping handler. Change-Id: I50d862d3cbb126987a63209795352c6e801ed919
* | Get test suite to full pass with SQLAlchemy 2.0Stephen Finucane2023-04-0611-153/+240
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remaining issues encountered when running with SQLAlchemy 2.0 for real: * Never call str() on a URL and expect it to be meaningful anymore. The password is aggressively obfuscated now (users absolultely wouldn't let us leave it as is) * More utilities and fixtures that were calling begin() within a block that would have already begun * isnot is now called is_not; mocking "isnot" leads into too many weird compat layers * ORM InstrumentedAttribute and internals use __slots__ now, mock seems to not be able to patch methods. Ideally these tests would use a comparator subclass or something * Connection.connection.connection is now called driver_connection, SQLAlchemy keeps the old name available however oslo.db test suite does not appear to tolerate the deprecation warning emitted, so add a compat layer * mapper() is fully removed from 2.0, not sure if there is another not-yet-committed gerrit that removes mapper() [1] https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.pool_pre_ping [2] https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-2fe37eaf2295cebd3bb4ee8e5b8c575c [3] https://github.com/sqlalchemy/sqlalchemy/issues/5648 Change-Id: Ifaca67c07f008d8bc0febeecd3e200cc7ee7a4b0
* | Do not convert to string ``url.URL`` objectsRodolfo Alonso Hernandez2023-03-277-36/+39
|/ | | | | | | | | | | | | | | The SQLAlchemy method ``url.make_url`` accepts strings and ``url.URL`` object types. By default, oslo.db was converting any object to string before calling ``url.make_url``, that parses this string. Since SQLAlchemy 2.0, the ``url.URL.__str__`` method is removed and the ``url.URL.__repr__`` method returns a string with the password hidden. The new utility method checks what type of object is passed and only if the object is not a string nor a ``url.URL`` object, is converted to a string. Closes-Bug: #2012928 Change-Id: I84f13f378f83e2a55078370ae2b4787f00982c23
* Remove sqlalchemy-migrateMike Bayer2023-03-219-1426/+10
| | | | | | | | | | | | | | | sqlalchemy-migrate does not support SQLAlchemy 2.x and we're not going to invest the time in changing this. Remove integration of sqlalchemy-migrate in oslo.db, allowing us to support SQLAlchemy 2.x fully. Note that we do not remove the 'migration_cli' module entirely yet. While this is deprecated, it is possible to use this with alembic. New users shouldn't be switching to it, but any existing users can continue to use this module for some time yet. Change-Id: Ic3d6bd318038d723b0d50d39e45f8e26289e9a57 Sem-Ver: api-break
* Rollback the connection after server ping methodRodolfo Alonso Hernandez2023-03-022-0/+29
| | | | | | | | | In the method ``engines._connect_ping_listener``, the connection should be rolled back after the ping execution. The rollback will revert the transaction and delete it. Closes-Bug: #2008209 Change-Id: Iba29ded227634e02795052acfd89b572bf21f54c
* Fix default value for wsrep_sync_wait option12.3.1Ayumu Ueha2023-01-184-5/+21
| | | | | | | | | | The default value for wsrep_sync_wait option should be `None`. However, since 0 is set incorrectly, an unintended process is executed. This patch fixes default value for wsrep_sync_wait option to `None` instead of `0`. Change-Id: Ifb1dc7ddcb127a69ea01234922caa7ca5ab111ce
* Add option for wsrep_sync_waitMike Bayer2022-12-054-8/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using Galera, the wsrep_sync_wait option [1] can change the behavior of a variety of Galera DQL/DML statements such that a particular operation will pause until outstanding write-sets are fully persisted to the local node. The setting supersedes the previous boolean parameter wsrep_causal_reads which only affected SELECT, with an updated approach that allows for fine-grained control of so-called "causality checks" on individual statement types. The legacy-compatible setting of '1' indicates that READ/SELECT/BEGIN operations should proceed only after any pending writesets are fully available. The use case for this setting is for an application that is running operations on multiple Galera nodes simultaenously. An application that commits data on one node, and then immediately uses a different connection (on a potentially different node) to SELECT that data, may fail to see those changes if "causality checks" for SELECT are not enabled. While a COMMIT operation in Galera will block locally until all other nodes approve of the writeset, the operation does not block for the subsequent period of time when other nodes are actually persisting that writeset. Setting up "causal reads" in this case indicates that a SELECT operation will wait until any writesets in progress are available, thus maintaining serialization between the COMMIT and subsequent SELECT. As the name implies, wsrep_sync_wait adds...waiting! to the operation, and thus directly impacts performance by adding latency to SELECT operations or to the operations that have been selected for causality checks, to the degree that concurrent writesets are expected to be present. Since it's not expected that most if any Openstack applications actually need this setting in order to be effective with Galera multi-master operation, and as the setting is available within client session scope and also impacts performance, making it available on a per-application basis means that specific applications which may see issues under load can choose to enable this setting, much in the way any other "transaction isolation" settings might be made, without having to add a cluster-wide performance penalty by setting it at the Galera server level. [1] https://mariadb.com/docs/ent/ref/mdb/system-variables/wsrep_sync_wait/ Change-Id: Iee7afcac8ba952a2d67a9ad9dd0e4eae3f42518e
* tests: Define a primary keyStephen Finucane2022-09-091-1/+1
| | | | | | | | | | | Avoids the following warning: SAWarning: Could not assemble any primary keys for locally mapped table 'fake_table_inh' - no rows will be persisted in this Table. class FakeTableJoinedInh(FakeTable): Change-Id: Ia7795be05e6364d8b2d70be70f5154094a55161a Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* tests: Fix compatibility with PostgreSQL 14+Stephen Finucane2022-09-091-3/+13
| | | | | | | | | | | | | | | | | | | | | | It seems postgres-client has changed the format of error messages. Previously we saw messages like: fatal: database "non_existent_database" does not exist These are now prefixed. For example: connection to server at "localhost" (::1), port 5432 failed: fatal: database "non_existent_database" does not exist You can see this in the docs. Compare the "Client Connection Problems" section for Postgres 13 [1] to Postgres 14 [2]. [1] https://www.postgresql.org/docs/13/server-start.html [2] https://www.postgresql.org/docs/14/server-start.html Change-Id: Id2c8eec202d128d142b8a8a8f904fcc14b6f52d7 Signed-off-by: Stephen Finucane <stephenfin@redhat.com> Closes-bug: #1989208
* Fix misuse of assert_has_callsTakashi Natsume2022-09-191-2/+2
| | | | | | | | The 'assert_has_calls' is a method, not a variable. Change-Id: I6aea978b8fc9adf4a062c149c84d6562f5adec04 Closes-Bug: 1989280 Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Merge "Deprecate MySQL NDB Cluster Support"Zuul2022-09-098-29/+138
|\
| * Deprecate MySQL NDB Cluster SupportStephen Finucane2022-07-278-29/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Traditionally, the MySQL support in oslo.db has assumed use of the InnoDB storage engine. However, this isn't the only storage engine available and a few years ago an effort was made to add support for another storage engine, MySQL Cluster (NDB). The oslo.db aspects of this effort were tracked via bug 1564110 [1] and from reading this bug and looking at other patches related to this effort [2], it becomes obvious that this was never seen through to the completion and the OpenStack-wide effort never took off [3]. As a result, much of what is here is in-effect dead code now. Given no one is using this engine, there's no reason to keep it around. Deprecate it with an eye on removing it sooner rather than later. [1] https://bugs.launchpad.net/oslo.db/+bug/1564110 [2] https://review.opendev.org/q/owner:octave.orgeron%2540oracle.com [3] https://review.opendev.org/c/openstack/openstack-specs/+/429940 Change-Id: Id5ddf1d6f47b8a572001f58ad8b9b8a7dbe4e8ac Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "trivial: Formatting changes for oslo_db.options"Zuul2022-09-091-112/+182
|\ \ | |/
| * trivial: Formatting changes for oslo_db.optionsStephen Finucane2022-07-141-112/+182
| | | | | | | | | | Change-Id: Iad89a0564b7851d75703f769c6eb96fbfe9a1e97 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | types: Set 'cache_ok' (redux)Stephen Finucane2022-09-081-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | In change Idf6fd858fad9521c7c5ba82c31b6d3077756abd9, we defined the 'cache_ok' key on a number of custom types. However, we relied on this attribute being inheritable which is not the case due to how that attribute is checked for [1]. As a result, we must set this attribute on every one of our custom types. [1] https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_41/lib/sqlalchemy/sql/type_api.py#L984 Change-Id: I005af8c7afe3b5104068ca153aecbfe54e163ca3 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Replace abc.abstractproperty with property and abc.abstractmethod12.1.0ljhuang2022-08-031-4/+8
| | | | | | | | | | | | | | | | | | Replace abc.abstractproperty with property and abc.abstractmethod, as abc.abstractproperty has been deprecated since python3.3[1] [1]https://docs.python.org/3.8/whatsnew/3.3.html?highlight=deprecated#abc Change-Id: Id90fbd2c53fd49341043bde740500a857a4339d3
* | trivial: Don't emit warnings for our own deprecationsStephen Finucane2022-02-081-0/+7
| | | | | | | | | | | | | | | | We've deprecated a number of modules recently. We don't need to emit these warnings when running unit tests. Silence things. Change-Id: I7aed7789584bf0070f11c22b5eaa0e80c42dfc9c Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | tests: Enable SAWarning warningsStephen Finucane2022-02-082-5/+18
| | | | | | | | | | | | | | | | | | We shouldn't be raising warnings from SQLAlchemy. Where we are intentionally doing so, we should capture these warnings at the test level. This requires some minor fixes. Change-Id: I9d4512dc337153edc48a2cc3bf95ab2b31c39ccf Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Remove the 'Session.autocommit' parameterStephen Finucane2022-02-085-77/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The Session.autocommit parameter is deprecated and will be removed in SQLAlchemy version 2.0. The Session now features "autobegin" behavior such that the Session.begin() method may be called if a transaction has not yet been started yet. See the section session_explicit_begin for background. Change-Id: I7867cdcea115b13f2e45e0674bb9ef2ad138aae9 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Add missing 'connect' wrapperStephen Finucane2022-02-081-1/+3
|/ | | | | | | | This was missed in change Ib789cd4d11a3d5dd01fcdb99822025b11bbc234e ("Don't rely on implicit autocommit") Change-Id: I9ec27650ae5e36099a6d2b2d59bb66cd820e8ffc Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* Don't call 'begin()' on existing transactionStephen Finucane2022-02-082-7/+1
| | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: Calling .begin() when a transaction is already begun, creating a 'sub' transaction, is deprecated and will be removed in 2.0. See the documentation section 'Migrating from the nesting pattern' for background on how to migrate from this pattern. Change-Id: I59e45dfea8fcbf72fc6e34345e510554cbdd138e Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* utils: Remove troublesome utility methods11.2.0Stephen Finucane2022-02-083-219/+0
| | | | | | | | | | These are not compatible with SQLAlchemy 2.0 due to their reliance on nested transactions. We should deprecate them first but doing so would push the boat out further wrt how long we have to wait before achieving compatibility with this new version. Change-Id: If3db4e8c1b681c0c62d3f04a57f92802639b3b9b Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* Merge "Remove use of Session.begin.subtransactions flag"Zuul2022-02-022-17/+2
|\
| * Remove use of Session.begin.subtransactions flagStephen Finucane2021-08-102-17/+2
| | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The Session.begin.subtransactions flag is deprecated and will be removed in SQLAlchemy version 2.0. See the documentation at session_subtransactions for background on a compatible alternative pattern. Change-Id: Ib2537bae77861ee60d17a48a72c57b88e043553e Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Don't rely on implicit autocommit"Zuul2022-02-026-103/+148
|\ \ | |/
| * Don't rely on implicit autocommitStephen Finucane2021-08-106-103/+148
| | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The current statement is being autocommitted using implicit autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin() method of Engine or Connection in order to use an explicit transaction for DML and DDL statements. Change-Id: Ib789cd4d11a3d5dd01fcdb99822025b11bbc234e Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | tests: Restore - don't reset - warning filtersStephen Finucane2021-11-191-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are more various warning filters pre-configured in a typical Python environment, including a few from third-party libraries such as requests [1][2] and urllib3 [3] as well as stdlib [4]. Our fixture to configure warnings, 'WarningsFixture', called 'warnings.resetwarnings' which *reset* all the warning filters [5]. This is clearly not something we want to do, and resulted in tests puking warnings after the initial test run. Resolve this by backing up the existing warning filters before applying the filter, and then *restoring* this original list of warning filters after the test run. [1] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L127 [2] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L152 [3] https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/__init__.py#L68-L78 [4] https://docs.python.org/3.8/library/warnings.html#default-warning-filter [5] https://docs.python.org/3.8/library/warnings.html#warnings.resetwarnings Change-Id: Ie74dad3f20002dd26fa9760c9ba452c4a40186c5 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Replace use of 'Engine.execute()'"11.1.0Zuul2021-11-155-175/+217
|\ \ | |/
| * Replace use of 'Engine.execute()'Stephen Finucane2021-08-105-175/+217
| | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. Change-Id: I4c47a690a94abcb3b4b6fb087a1bf86c5350b523 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Don't call mapper() outside of declarative registry"Zuul2021-11-153-9/+9
|\ \ | |/
| * Don't call mapper() outside of declarative registryStephen Finucane2021-08-103-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning: Calling the mapper() function directly outside of a declarative registry is deprecated. Please use the sqlalchemy.orm.registry.map_imperatively() function for a classical mapping. Change-Id: I92a7ccdd48eedd4c788384033743daf50a9dc113 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Don't pass kwargs to connection.execute()"Zuul2021-11-124-11/+9
|\ \ | |/
| * Don't pass kwargs to connection.execute()Stephen Finucane2021-08-104-11/+9
| | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The connection.execute() method in SQLAlchemy 2.0 will accept parameters as a single dictionary or a single sequence of dictionaries only. Parameters passed as keyword arguments, tuples or positionally oriented dictionaries and/or tuples will no longer be accepted. Change-Id: I44675fce86337696b6494abc03e8058af32686c6 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Replace use of Executable.execute method"Zuul2021-11-122-80/+83
|\ \ | |/
| * Replace use of Executable.execute methodStephen Finucane2021-08-102-80/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | Resolve the following RemovedIn20Warning warning: The Executable.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. Change-Id: Ie0acba4a315c85ec7236e44a22449e0ad920ca9b Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Remove unnecessary warning filter"Zuul2021-11-121-5/+0
|\ \ | |/
| * Remove unnecessary warning filterStephen Finucane2021-08-101-5/+0
| | | | | | | | | | | | | | Looks like we solved this along the way. Change-Id: I50469e58fe3fae695316cb344fea44ec18fffe60 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* | Merge "Replace use of Engine.scalar()"Zuul2021-11-123-12/+12
|\ \ | |/
View Lorry Controller Status