| Commit message (Collapse) | Author | Age | Files | Lines |
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Change I8629225eeb51d95264d8a3e4b719268bb1597f4f bumped the minimum
version for SQLAlchemy to 1.4, meaning this logic is now dead. Remove
it.
Change-Id: I4d4a58e15e840ecfa63e15c709617a65642c8323
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|\ \
| |/ |
|
| |
| |
| |
| |
| |
| |
| |
| | |
This should have been removed in change
Ic3d6bd318038d723b0d50d39e45f8e26289e9a57 but was missed.
Change-Id: I4e1faa2c617ac19e7c9766e99ee9012ad9298d31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|\ \
| |/ |
|
| |
| |
| |
| |
| |
| |
| | |
Mock it out, reducing test run of ~3 and ~15 seconds to milliseconds.
Change-Id: Ice3a0c0d0a5b8c2920c7f775ff8ce974b572c66e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|\ \
| |/
|/| |
|
| |
| |
| |
| |
| |
| |
| |
| | |
The fixtures have been available for a very long time now. We can drop
these.
Change-Id: I1e71150ba6daeba464b6ed8d46163f1f34959db3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This functionality has been available upstream since SQLALchemy 1.2 [1].
However, for oslo.db to use this feature while maintaining full
behavior, we need at least SQLAlchemy 2.0.5 to provide complete event
support. In particular, oslo.db adds several new "is disconnect"
conditions including those specific to Galera.
Behavior of the handle_error event is modified to expect the "pre-ping"
calling form, which may not have an "engine" present (only a dialect),
and additionally takes advantage of the new is_pre_ping attribute which
informs on the correct way to affect the disconnection status within the
ping handler.
Change-Id: I50d862d3cbb126987a63209795352c6e801ed919
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Remaining issues encountered when running with SQLAlchemy 2.0 for real:
* Never call str() on a URL and expect it to be meaningful anymore.
The password is aggressively obfuscated now (users absolultely
wouldn't let us leave it as is)
* More utilities and fixtures that were calling begin() within a
block that would have already begun
* isnot is now called is_not; mocking "isnot" leads into too many
weird compat layers
* ORM InstrumentedAttribute and internals use __slots__ now, mock
seems to not be able to patch methods. Ideally these tests would use
a comparator subclass or something
* Connection.connection.connection is now called driver_connection,
SQLAlchemy keeps the old name available however oslo.db test suite
does not appear to tolerate the deprecation warning emitted,
so add a compat layer
* mapper() is fully removed from 2.0, not sure if there is another
not-yet-committed gerrit that removes mapper()
[1] https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine.params.pool_pre_ping
[2] https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-2fe37eaf2295cebd3bb4ee8e5b8c575c
[3] https://github.com/sqlalchemy/sqlalchemy/issues/5648
Change-Id: Ifaca67c07f008d8bc0febeecd3e200cc7ee7a4b0
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The SQLAlchemy method ``url.make_url`` accepts strings and ``url.URL``
object types. By default, oslo.db was converting any object to string
before calling ``url.make_url``, that parses this string. Since
SQLAlchemy 2.0, the ``url.URL.__str__`` method is removed and the
``url.URL.__repr__`` method returns a string with the password hidden.
The new utility method checks what type of object is passed and only
if the object is not a string nor a ``url.URL`` object, is converted
to a string.
Closes-Bug: #2012928
Change-Id: I84f13f378f83e2a55078370ae2b4787f00982c23
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sqlalchemy-migrate does not support SQLAlchemy 2.x and we're not going
to invest the time in changing this. Remove integration of
sqlalchemy-migrate in oslo.db, allowing us to support SQLAlchemy 2.x
fully.
Note that we do not remove the 'migration_cli' module entirely yet.
While this is deprecated, it is possible to use this with alembic.
New users shouldn't be switching to it, but any existing users can
continue to use this module for some time yet.
Change-Id: Ic3d6bd318038d723b0d50d39e45f8e26289e9a57
Sem-Ver: api-break
|
|
|
|
|
|
|
|
|
| |
In the method ``engines._connect_ping_listener``, the connection should
be rolled back after the ping execution. The rollback will revert the
transaction and delete it.
Closes-Bug: #2008209
Change-Id: Iba29ded227634e02795052acfd89b572bf21f54c
|
|
|
|
|
|
|
|
|
|
| |
The default value for wsrep_sync_wait option should be `None`.
However, since 0 is set incorrectly, an unintended process is executed.
This patch fixes default value for wsrep_sync_wait option to `None`
instead of `0`.
Change-Id: Ifb1dc7ddcb127a69ea01234922caa7ca5ab111ce
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using Galera, the wsrep_sync_wait option [1]
can change the behavior of a variety of Galera DQL/DML statements
such that a particular operation will pause until outstanding
write-sets are fully persisted to the local node. The setting
supersedes the previous boolean parameter wsrep_causal_reads
which only affected SELECT, with an updated approach that allows
for fine-grained control of so-called "causality checks"
on individual statement types. The legacy-compatible setting
of '1' indicates that READ/SELECT/BEGIN operations should
proceed only after any pending writesets are fully available.
The use case for this setting is for an application that
is running operations on multiple Galera nodes simultaenously.
An application that commits data on one node, and then immediately
uses a different connection (on a potentially different node)
to SELECT that data, may fail to see those changes if
"causality checks" for SELECT are not enabled. While
a COMMIT operation in Galera will block locally until all other
nodes approve of the writeset, the operation does not block
for the subsequent period of time when other nodes are actually
persisting that writeset. Setting up "causal reads"
in this case indicates that a SELECT operation will wait until
any writesets in progress are available, thus maintaining
serialization between the COMMIT and subsequent SELECT.
As the name implies, wsrep_sync_wait adds...waiting! to the
operation, and thus directly impacts performance by adding
latency to SELECT operations or to the operations that have
been selected for causality checks, to the degree that
concurrent writesets are expected to be present.
Since it's not expected that most if any Openstack applications
actually need this setting in order to be effective with
Galera multi-master operation, and as the setting is available
within client session scope and also impacts performance,
making it available on a per-application basis means that
specific applications which may see issues under load can
choose to enable this setting, much in the way any other
"transaction isolation" settings might be made, without having
to add a cluster-wide performance penalty by setting it at the
Galera server level.
[1] https://mariadb.com/docs/ent/ref/mdb/system-variables/wsrep_sync_wait/
Change-Id: Iee7afcac8ba952a2d67a9ad9dd0e4eae3f42518e
|
|
|
|
|
|
|
|
|
|
|
| |
Avoids the following warning:
SAWarning: Could not assemble any primary keys for locally mapped
table 'fake_table_inh' - no rows will be persisted in this Table.
class FakeTableJoinedInh(FakeTable):
Change-Id: Ia7795be05e6364d8b2d70be70f5154094a55161a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems postgres-client has changed the format of error messages.
Previously we saw messages like:
fatal: database "non_existent_database" does not exist
These are now prefixed. For example:
connection to server at "localhost" (::1), port 5432 failed: fatal:
database "non_existent_database" does not exist
You can see this in the docs. Compare the "Client Connection Problems"
section for Postgres 13 [1] to Postgres 14 [2].
[1] https://www.postgresql.org/docs/13/server-start.html
[2] https://www.postgresql.org/docs/14/server-start.html
Change-Id: Id2c8eec202d128d142b8a8a8f904fcc14b6f52d7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-bug: #1989208
|
|
|
|
|
|
|
|
| |
The 'assert_has_calls' is a method, not a variable.
Change-Id: I6aea978b8fc9adf4a062c149c84d6562f5adec04
Closes-Bug: 1989280
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Traditionally, the MySQL support in oslo.db has assumed use of the
InnoDB storage engine. However, this isn't the only storage engine
available and a few years ago an effort was made to add support for
another storage engine, MySQL Cluster (NDB). The oslo.db aspects of this
effort were tracked via bug 1564110 [1] and from reading this bug and
looking at other patches related to this effort [2], it becomes obvious
that this was never seen through to the completion and the
OpenStack-wide effort never took off [3]. As a result, much of what is
here is in-effect dead code now.
Given no one is using this engine, there's no reason to keep it around.
Deprecate it with an eye on removing it sooner rather than later.
[1] https://bugs.launchpad.net/oslo.db/+bug/1564110
[2] https://review.opendev.org/q/owner:octave.orgeron%2540oracle.com
[3] https://review.opendev.org/c/openstack/openstack-specs/+/429940
Change-Id: Id5ddf1d6f47b8a572001f58ad8b9b8a7dbe4e8ac
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |
| |
| |
| |
| |
| |
| |
| | |
We've deprecated a number of modules recently. We don't need to emit
these warnings when running unit tests. Silence things.
Change-Id: I7aed7789584bf0070f11c22b5eaa0e80c42dfc9c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We shouldn't be raising warnings from SQLAlchemy. Where we are
intentionally doing so, we should capture these warnings at the test
level. This requires some minor fixes.
Change-Id: I9d4512dc337153edc48a2cc3bf95ab2b31c39ccf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|/
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The Session.autocommit parameter is deprecated and will be removed in
SQLAlchemy version 2.0. The Session now features "autobegin" behavior
such that the Session.begin() method may be called if a transaction
has not yet been started yet. See the section session_explicit_begin
for background.
Change-Id: I7867cdcea115b13f2e45e0674bb9ef2ad138aae9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
Calling .begin() when a transaction is already begun, creating a 'sub'
transaction, is deprecated and will be removed in 2.0. See the
documentation section 'Migrating from the nesting pattern' for
background on how to migrate from this pattern.
Change-Id: I59e45dfea8fcbf72fc6e34345e510554cbdd138e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
These are not compatible with SQLAlchemy 2.0 due to their reliance on
nested transactions. We should deprecate them first but doing so would
push the boat out further wrt how long we have to wait before achieving
compatibility with this new version.
Change-Id: If3db4e8c1b681c0c62d3f04a57f92802639b3b9b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Resolve the following RemovedIn20Warning warning:
The Session.begin.subtransactions flag is deprecated and will be
removed in SQLAlchemy version 2.0. See the documentation at
session_subtransactions for background on a compatible alternative
pattern.
Change-Id: Ib2537bae77861ee60d17a48a72c57b88e043553e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|\ \
| |/ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Resolve the following RemovedIn20Warning warning:
The current statement is being autocommitted using implicit
autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin()
method of Engine or Connection in order to use an explicit transaction
for DML and DDL statements.
Change-Id: Ib789cd4d11a3d5dd01fcdb99822025b11bbc234e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There are more various warning filters pre-configured in a typical
Python environment, including a few from third-party libraries such as
requests [1][2] and urllib3 [3] as well as stdlib [4]. Our fixture to
configure warnings, 'WarningsFixture', called 'warnings.resetwarnings'
which *reset* all the warning filters [5]. This is clearly not something
we want to do, and resulted in tests puking warnings after the initial
test run.
Resolve this by backing up the existing warning filters before applying
the filter, and then *restoring* this original list of warning filters
after the test run.
[1] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L127
[2] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L152
[3] https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/__init__.py#L68-L78
[4] https://docs.python.org/3.8/library/warnings.html#default-warning-filter
[5] https://docs.python.org/3.8/library/warnings.html#warnings.resetwarnings
Change-Id: Ie74dad3f20002dd26fa9760c9ba452c4a40186c5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The Engine.execute() method is considered legacy as of the 1.x series
of SQLAlchemy and will be removed in 2.0. All statement execution in
SQLAlchemy 2.0 is performed by the Connection.execute() method of
Connection, or in the ORM by the Session.execute() method of Session.
Change-Id: I4c47a690a94abcb3b4b6fb087a1bf86c5350b523
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning:
Calling the mapper() function directly outside of a declarative
registry is deprecated. Please use the
sqlalchemy.orm.registry.map_imperatively() function for a classical
mapping.
Change-Id: I92a7ccdd48eedd4c788384033743daf50a9dc113
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The connection.execute() method in SQLAlchemy 2.0 will accept
parameters as a single dictionary or a single sequence of dictionaries
only. Parameters passed as keyword arguments, tuples or positionally
oriented dictionaries and/or tuples will no longer be accepted.
Change-Id: I44675fce86337696b6494abc03e8058af32686c6
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The Executable.execute() method is considered legacy as of the 1.x
series of SQLAlchemy and will be removed in 2.0. All statement
execution in SQLAlchemy 2.0 is performed by the Connection.execute()
method of Connection, or in the ORM by the Session.execute() method of
Session.
Change-Id: Ie0acba4a315c85ec7236e44a22449e0ad920ca9b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
| |
Looks like we solved this along the way.
Change-Id: I50469e58fe3fae695316cb344fea44ec18fffe60
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The Engine.scalar() method is considered legacy as of the 1.x series
of SQLAlchemy and will be removed in 2.0. All statement execution in
SQLAlchemy 2.0 is performed by the Connection.execute() method of
Connection, or in the ORM by the Session.execute() method of Session;
the Result.scalar() method can then be used to return a scalar result.
Change-Id: Ic3c9c5bb008b6299f2ed9a59bda0329fab9d554d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The Row.keys() method is considered legacy as of the 1.x series of
SQLAlchemy and will be removed in 2.0. Use the namedtuple standard
accessor Row._fields, or for full mapping behavior use
row._mapping.keys()
Change-Id: I647a57909df56fec7b570ae29efbc731126df14d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
Using non-integer/slice indices on Row is deprecated and will be
removed in version 2.0; please use row._mapping[<key>], or the
mappings() accessor on the Result object.
Change-Id: I3a4845216914635e5802a70c2b1be757d82b7a49
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
Using plain strings to indicate SQL statements without using the
text() construct is deprecated and will be removed in version 2.0.
Ensure plain SQL statements are passed using the text() construct.
Change-Id: I8acdb54d168afa1a7eac270ad6165faa287311ec
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The autoload parameter is deprecated and will be removed in version
2.0. Please use the autoload_with parameter, passing an engine or
connection.
Change-Id: Ie039851005c6038591e0e1f6426922ca5b0dcb9a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning:
The update.whereclause parameter will be removed in SQLAlchemy 2.0.
Please refer to the Update.where() method.
Change-Id: I50064d16de6e1951b2282fcf9548a0ff2cd0fe55
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following MovedIn20Warning:
The ``declarative_base()`` function is now available as
sqlalchemy.orm.declarative_base()
Change-Id: I5fd707c62d0bfad6e54150c4a3b87aadadbeedb3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The update.values parameter will be removed in SQLAlchemy 2.0. Please
refer to the Update.values() method.
Change-Id: I426605aa8f4332fd0b7d9f6fa0e97d7e02c7cbdc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The insert.values parameter will be removed in SQLAlchemy 2.0. Please
refer to the Insert.values() method.
Change-Id: I52ff0c44039d8d1ee09210d9a990d0e40bbfa411
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
The ``bind`` argument for schema methods that invoke SQL against an
engine or connection will be required in SQLAlchemy 2.0.
Change-Id: If9492a3842055dde3841eaeb72fb4c050563b6ac
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following RemovedIn20Warning warning:
Passing a string to Connection.execute() is deprecated and will be
removed in version 2.0. Use the text() construct, or the
Connection.exec_driver_sql() method to invoke a driver-level SQL
string.
Change-Id: I1faa8c957649a04aa080518651045b432c6bd372
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Resolves the following RemovedIn20Warning warning:
The MetaData.bind argument is deprecated and will be removed in
SQLAlchemy 2.0.
Change-Id: I468048c01455d9dfe12004736c181806dc218e37
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Resolve the following SADeprecationWarning warning:
The legacy calling style of select() is deprecated and will be removed
in SQLAlchemy 2.0. Please use the new calling style described at
select().
Change-Id: Ic5f7240e790425d2689c6870483748650a49bc3d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
| |
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.
Change-Id: I32cad5fd9ddbfd2ff6b62e5e2f87fad6ece7a4c1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Highlight use of deprecated SQLAlchemy APIs to ensure we keep on top of
things. This requires resolving the following SADeprecationWarning
warnings:
The from_engine() method on Inspector is deprecated and will be
removed in a future release. Please use the sqlalchemy.inspect()
function on an Engine or Connection in order to acquire an Inspector.
The Column.copy() method is deprecated and will be removed in a future
release.
The ColumnCollectionConstraint.copy() method is deprecated and will be
removed in a future release.
There are more warnings to be resolved related to SQLAlchemy 2.0, but
those require a special environment option (SQLALCHEMY_WARN_20) to
trigger and a lot of work to resolve. We'll address those in a series of
follow-ups.
Change-Id: I34b395e6d50f4e4151178c327d94308e6f5d5b6e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|