| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
The default value for wsrep_sync_wait option should be `None`.
However, since 0 is set incorrectly, an unintended process is executed.
This patch fixes default value for wsrep_sync_wait option to `None`
instead of `0`.
Change-Id: Ifb1dc7ddcb127a69ea01234922caa7ca5ab111ce
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using Galera, the wsrep_sync_wait option [1]
can change the behavior of a variety of Galera DQL/DML statements
such that a particular operation will pause until outstanding
write-sets are fully persisted to the local node. The setting
supersedes the previous boolean parameter wsrep_causal_reads
which only affected SELECT, with an updated approach that allows
for fine-grained control of so-called "causality checks"
on individual statement types. The legacy-compatible setting
of '1' indicates that READ/SELECT/BEGIN operations should
proceed only after any pending writesets are fully available.
The use case for this setting is for an application that
is running operations on multiple Galera nodes simultaenously.
An application that commits data on one node, and then immediately
uses a different connection (on a potentially different node)
to SELECT that data, may fail to see those changes if
"causality checks" for SELECT are not enabled. While
a COMMIT operation in Galera will block locally until all other
nodes approve of the writeset, the operation does not block
for the subsequent period of time when other nodes are actually
persisting that writeset. Setting up "causal reads"
in this case indicates that a SELECT operation will wait until
any writesets in progress are available, thus maintaining
serialization between the COMMIT and subsequent SELECT.
As the name implies, wsrep_sync_wait adds...waiting! to the
operation, and thus directly impacts performance by adding
latency to SELECT operations or to the operations that have
been selected for causality checks, to the degree that
concurrent writesets are expected to be present.
Since it's not expected that most if any Openstack applications
actually need this setting in order to be effective with
Galera multi-master operation, and as the setting is available
within client session scope and also impacts performance,
making it available on a per-application basis means that
specific applications which may see issues under load can
choose to enable this setting, much in the way any other
"transaction isolation" settings might be made, without having
to add a cluster-wide performance penalty by setting it at the
Galera server level.
[1] https://mariadb.com/docs/ent/ref/mdb/system-variables/wsrep_sync_wait/
Change-Id: Iee7afcac8ba952a2d67a9ad9dd0e4eae3f42518e
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Traditionally, the MySQL support in oslo.db has assumed use of the
InnoDB storage engine. However, this isn't the only storage engine
available and a few years ago an effort was made to add support for
another storage engine, MySQL Cluster (NDB). The oslo.db aspects of this
effort were tracked via bug 1564110 [1] and from reading this bug and
looking at other patches related to this effort [2], it becomes obvious
that this was never seen through to the completion and the
OpenStack-wide effort never took off [3]. As a result, much of what is
here is in-effect dead code now.
Given no one is using this engine, there's no reason to keep it around.
Deprecate it with an eye on removing it sooner rather than later.
[1] https://bugs.launchpad.net/oslo.db/+bug/1564110
[2] https://review.opendev.org/q/owner:octave.orgeron%2540oracle.com
[3] https://review.opendev.org/c/openstack/openstack-specs/+/429940
Change-Id: Id5ddf1d6f47b8a572001f58ad8b9b8a7dbe4e8ac
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |
|
|
|
| |
Change-Id: Iad89a0564b7851d75703f769c6eb96fbfe9a1e97
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |
|
|
|
|
|
|
|
|
| |
The option was replaced 4 years ago[1] by connection_recycle_time. The
option is not anymore present in sqlalchemy. It's the good time to
remove it.
[1] https://opendev.org/openstack/oslo.db/commit/6634218415906192fca891362b8fa0ac50b66284
Change-Id: I0c26c3e14ad9ad9a4f30af80757e369d6042441a
|
| |
|
|
|
|
|
|
| |
The option was replaced 6 years ago[1] by max_pool_size.
[1] https://opendev.org/openstack/oslo.db/commit/1b7c295d522e26aa8bf09242dcc1355c3fb1ab3e
Change-Id: Ice1a90edc32603c85449b80fe78fd437aa7e8081
|
| |
|
|
|
|
|
|
|
|
| |
The database option min_pool_size is not actually
used in oslo.db.
This is a folllow up to
Ib07b7b4200b5cf3a34b8eed441e6754da7d446cf
Change-Id: I89754e9e4b42c45259729b0b0bb0a4bf464a981c
|
| |
|
|
|
|
|
|
|
|
| |
This option has for a long time done nothing in oslo.db
and sqlalchemy does not support this convention. As such I
recommend that we deprecate and eventually remove the option.
Closes-bug: #1764786
Change-Id: Ib07b7b4200b5cf3a34b8eed441e6754da7d446cf
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Nova project has made the decision to store the entire
contents of the "sql_connection" field in the database, with
alterations to the host value, such that various "cells" database
URLs can be located using Nova's database itself as a central
registry of database URLs.
This architecture has produced several problems. The first
is that it is impossible to apply parameters to the URL that
are local to the client machine; the leading example of this
is the MySQL "bind_host" variable, which must match the hostname
of the connecting host. Because cells puts these URLs into the
database and shares them with all controllers, we have to use a workaround
with the read_default_file parameter to specify a controller-local
file of options; this is not a standard thing
for other database drivers, and these parameters only apply to
the MySQL driver and not the SQLAlchemy engine.
The next issue is that it is inconvenient to add parameters
to the URL at all, once Nova has already been running, as one
must manually use Nova's command line tools
to alter all the URLs that have already been copied into the database
and alter the query parameters with each of those individually,
and then restart *all* services who will all receive the parameter
(no way to add params to just one controller).
Nova's "store the URL in the database" feature only needs to be able
to locate the host / database name of the alternate database,
and not change the URL tuning parameters. This patch adds a new
oslo.db parameter connection_parameters which allows the params to
be applied separately from the sql_connection parameter, so that
Nova can continue persisting sql_connection but the parameters remain
local to the nova.conf file. A URL parameter that truly had to remain
persisted in Nova's database (there aren't any) could still be applied
at the sql_connection level.
This feature is essential not just so that we can again place
simple parameters into controller-local files like "bind_host", but
also to allow for configuration of SQLAlchemy features such as
plugins that do connection pool monitoring.
Change-Id: Id4de4b09ec4719cbf8b372629fcf58cf368a33d4
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The "idle_timeout" name and documentation are misleading to users.
Rename the option to connection_recycle_time to more closely
match that of the underlying SQLAlchemy argument name
"pool_recycle", which refers to a time period for which
a connection present in the pool for longer than that number
of seconds will be closed and replaced with a new one *upon checkout*,
to prevent usage of stale connections that may have been closed by
server-side reapers.
Change-Id: I5e39d5ac81eaa8d85b1dfb18862fe27695ab8b96
|
| |
|
|
|
|
|
|
| |
Enables new functionality for selecting InnoDB or MySQL Cluster
as a DB storage backend in OpenStack services.
Closes-Bug: 1564110
Change-Id: I9f1fd2a87fdf75332de2339d3ff4f08ce9220dcf
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
We deprecated config options sqlite_db in July, 2016 [1], and
it shows up in many projects' config files but these projects
don't use it either, so it's confusing to our users. It's time
to remove it, just use config option connection or slave_connection
to connect to the database.
[1] https://review.openstack.org/#/c/338047/
Closes-Bug: #1329086
Change-Id: Id269d921e40edf95eb977b011f1753f633b79d18
|
| |
|
|
|
|
|
|
| |
This is follow up of commit 0a1bae9859079fb21a03716be947c5f1da6db0a2.
Log warning when pass keyword argument sqlite_db.
Related-Bug: #1329086
Change-Id: Ibed55dc9f28b7813bd96fb5e6f5138de0667591f
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| | |
The max_pool_size defaults to 5 instead of None. Setting this
option to 0 effective disables an upper limit to the pool.
Change-Id: I9a6918e1a808e9e70105d5b21d901bbe3d298f34
|
| |/
|
|
|
|
|
| |
There is a set allowable minimum and maximum to the connection_debug
option of 0 and 100. The config option should enforce this.
Change-Id: Ica610ad8fc0875a8be1042f7daba5d0a0c1aad03
|
| |
|
|
|
|
|
|
|
|
| |
We should use config option connection or slave_connection to connect
to the database. We can remove it when all consuming projects don't use
it anymore.
Related-Bug: #1329086
Change-Id: I1960e47b27b5f98f19a07f9899079269bac025cc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It turned out that distribution of WSGI requests between forks is not
even when eventlet is used, which can cause pool timeout issues under
load, while there are idle workers with open DB connections, which
should have served these HTTP requests.
The default max_overflow value should be increased to allow DB oriented
services (mostly APIs) handle spikes in number of concurrent requests
trying to use a DB.
At the same time, the default number of greenlets is decreased:
I65b40b9906b75146a0085bbe168f1e6bcae82f21
which effectively causes a particular worker to stop accepting new
requests it has no resources (DB connections) to handle and allow
other forks accept()'ing on the same FD to proceed.
Testing was performed using this script:
https://gist.github.com/zzzeek/c69138fd0d0b3e553a1f
With 100 greenthreads in the pool, 50 DB connections allowed overflow
no pool timeout issues were seen with up to 500 concurrent requests
done by ab, while current default values (1000/10) could not handle
even 100 concurrent requests.
See this ML thread for details:
http://lists.openstack.org/pipermail/openstack-dev/2015-December/082717.html
DocImpact
This change potentially increases the number of connections open to the
RDBMS server. If you start seeing "too many connections" errors, please
check these settings and adjust them appropriately:
https://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_max_connections
http://www.postgresql.org/docs/current/static/runtime-config-connection.html#GUC-MAX-CONNECTIONS
Closes-Bug: #1535375
Change-Id: I2e9c2a71d8231e0dfbefc6293ad319e1e459beec
|
| |
|
|
|
|
|
|
|
|
| |
In order for the config generator's new hook for updating default values
to work, we have to return the same Opt objects from list_opts() that
our set_defaults() function is going to modify. There's no real need to
be making copies of the objects anyway, so just stop doing it.
Change-Id: I62c2f733f1e7943449aa744ed15dac4addb3e1f7
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
|
| |
|
|
|
|
|
|
|
| |
When generating the docs some warnings were generated.
Also, configure doc build so that if there are warnings, the
build will fail.
Change-Id: I7e10499294cf1f06f766cbff392e9de32ea48e5a
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Add a decorator retry_on_deadlock(). All db.api functions marked
with this decorator will be retried if a DBDeadlock exception is
received.
Change-Id: I3488bfec67ec1c563292f1b61a7a9697f118809c
Closes-bug: 1348588
|
| |/
|
|
|
| |
Change-Id: I4ec9b2a310471e4e07867073e9577731ac34027d
Blueprint: drop-namespace-packages
|
|
|
Move the public API out of oslo.db to oslo_db. Retain the ability to
import from the old namespace package for backwards compatibility for
this release cycle.
Blueprint: drop-namespace-packages
Change-Id: Ie96b482b9fbcb1d85203ad35bb65c1f43e912a44
|
