| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.
From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.
As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.
Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.
As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.
An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.
py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.
Includes the following improvements:
Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.
For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.
Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.
For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.
Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.
Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
in I4940d184a4dc790782fcddfb9873af3cca844398 we reworked how async
tests run but apparently the async tests in test/ext/asyncio
are reporting success without being run. This patch pushes
pytestplugin further so that it won't instrument any test
or function overall that declares itself async. This removes
the need for the __async_wrap__ flag and also allows us to
use a more strict "run_async_test" function that always
runs the asyncio event loop from the top.
Also start working asyncio into main testing suite.
Change-Id: If7144e951a9db67eb7ea73b377f81c4440d39819
|
| |
|
|
| |
Change-Id: I4940d184a4dc790782fcddfb9873af3cca844398
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reworked the proxy creation used by scoped_session() to be
based on fully copied code with augmented docstrings and
moved it into langhelpers. asyncio session, engine,
connection can now take
advantage of it so that all non-async methods are availble.
Overall implementation of most important accessors / methods
on AsyncConnection, etc. , including awaitable versions
of invalidate, execution_options, etc.
In order to support an event dispatcher on the async
classes while still allowing them to hold __slots__,
make some adjustments to the event system to allow
that to be present, at least rudimentally.
Fixes: #5628
Change-Id: I5eb6929fc1e4fdac99e4b767dcfd49672d56e2b2
|
| |
|
|
|
| |
Fixes: #5223
Change-Id: I0952e54ed9af2952ea340be1945311376ffc1ad2
|
| |
|
|
|
|
|
| |
pytest has removed support for pytest.Class().collect()
and we need to use from_parent.
Change-Id: Ia5fed9b22e76c99f71489283acee207f996f52a4
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).
Fixes: #5362
Fixes: #5324
Fixes: #5360
Change-Id: Iecea6f3ceb36821e8b96f0b61049b580507a1875
|
| |
|
|
|
|
|
|
|
| |
With version 2.0.0 they removed the old terminology, which
I had no idea had been changed in any case for several years.
pushing this out for all branches
Change-Id: I51e4f577aeb9ef8205348b50489549f77072a613
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Using the approach introduced at
https://gist.github.com/zzzeek/6287e28054d3baddc07fa21a7227904e
We can now create asyncio endpoints that are then handled
in "implicit IO" form within the majority of the Core internals.
Then coroutines are re-exposed at the point at which we call
into asyncpg methods.
Patch includes:
* asyncpg dialect
* asyncio package
* engine, result, ORM session classes
* new test fixtures, tests
* some work with pep-484 and a short plugin for the
pyannotate package, which seems to have so-so results
Change-Id: Idbcc0eff72c4cad572914acdd6f40ddb1aef1a7d
Fixes: #3414
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Note the PR has a few remaining doc linking issues
listed in the comment that must be addressed separately.
Signed-off-by: aplatkouski <5857672+aplatkouski@users.noreply.github.com>
Closes: #5371
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5371
Pull-request-sha: 7e7d233cf3a0c66980c27db0fcdb3c7d93bc2510
Change-Id: I9c36e8d8804483950db4b42c38ee456e384c59e3
|
| |
|
|
| |
Change-Id: I431e1ef41e26d490343204a75a5c097768749768
|
| |
|
|
|
|
|
|
|
| |
py.test 5.4.0 emits deprecation warnings for pytest.Class.
make sure we don't raise for these, and log the code that will
be used for 5.4.0 when we bump requirements.
Fixes: #5201
Change-Id: I83e0402c4a6b2365a63b58d052c6989df3a37328
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The technique arrived at for doing exclusions inside of combinations
relies upon comparing all the arguments in a particular combination
to some set of combinations that were gathered as having
"exclusions". This logic is actually broken for the
case where the @testing.combinations has an "id", but if we fix
that, we still have the issue of all the arguments being
compared, which is complicated and also doesn't work for the
case of a py2/py3 incompatibility like a timezone that has
fractional minutes or seconds in it. It's also not clear
if a @testing.combinations that uses lambdas will work either
(maybe it does though because lambdax == lambdax compares...).
anyway, this patch reworks it so that we hit this on the decorator
side instead, where we add our own decorator and go through
the extra effort to create a decorator that accepts an extra
argument of "exclusions" which we can then check in a way that
is local to the whole pytest @combinations thing in the first place.
The only difficulty is that pytest is very sneaky about looking
at the test function so we need to make sure __wrapped__ isn't
set when doing this.
Change-Id: Ic57aae15b378e0f4ed009e4e82ae7ba73fb6dfc5
|
| |
|
|
|
|
|
|
|
|
|
|
| |
As the ORM's combinatoric tests mostly use entities and
table metadata that's defined in fixtures, we can't use
@testing.combinations directly as it takes place at the
module level. Instead we use lambdas, but to reduce
verbosity we use a code replacement so that the namespace
of the lambda can be provided at runtime rather than
module import time.
Change-Id: Ia63a510f9c1d08b055eef62cf047f1f427f0450c
|
| |
|
|
|
|
|
|
|
|
| |
- ensure we escape out percent signs when a CompiledSQL or RegexSQL
has percent signs in the SQL or in the parameter repr
- to support combinations, print out complete test name in skip
messages, py.test environment gives us a way to do this
Change-Id: Ia9e62f7c1026c1465986144c5757e35fc164a2b8
|
| |
|
|
|
|
|
|
|
|
|
| |
Like py.test we need to be able to mark certain combination
elements with exclusion rules. Add additional logic
to pytestlplugin and exclusions so that the exclusion decorators
can be added to the combination tuples, where they will be applied
to the decorated function along with a qualifier that the test
arguments need to match what's given.
Change-Id: I15d2839954d77a252bab5aaf6e3fd9f388c99dd5
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dialect tests tend to have a lot of lists of types,
SQL constructs etc, convert as many of these to @combinations
as possible.
This is exposing that we don't have per-combination
exclusion rules set up which is making things a little bit
cumbersome.
Also set up a fixture that does metadata + DDL.
Change-Id: Ief820e48c9202982b0b1e181b87862490cd7b0c3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add factilities to implement pytest.mark.parametrize and
pytest.fixtures patterns, which largely resemble things we are
already doing.
Ensure a facade is used, so that the test suite remains independent
of py.test, but also tailors the functions to the more limited
scope in which we are using them.
Additionally, create a class-based version that works from the
same facade.
Several old polymorphic tests as well as two of the sql test
are refactored to use the new features.
Change-Id: I6ef8af1dafff92534313016944d447f9439856cf
References: #4896
|
| |
|
|
|
|
|
|
|
| |
this seems to be the best place to put this as it is guaranteed
before the module is imported. this is for the benefit of
3rd party dialects that also would have this in their conftest.py,
so that they don't have to do the "bootstrap" loading hack.
Change-Id: Ieae5324240e04a7919df46f4fca03f8db7a2af81
|
| |
|
|
|
|
|
|
| |
Since our various eq_(), ne_() etc. functions use assert, pytest
can rewrite this module using its enhanced string reporting.
very helpful for comparing SQL strings
Change-Id: Ia71328401fd7965bcb14eb1ccea0dc48a8f2c3ea
|
| |
|
|
|
|
|
|
|
|
|
| |
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient. test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.
Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
|
| |
|
|
|
|
|
|
|
| |
We have a custom test collection hook that did not take
node of the actual list of functions in items. By looking
in this list we now support the class/function arguments
passed to the py.test command line.
Change-Id: I1238c7c5796a296037ab9ef3bedf0f619a730481
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
|
| |
|
|
|
| |
Change-Id: I69e39d2368f50b126c369ecc35e01799fd013254
(cherry picked from commit 3fc6f32ddc5fbbf439acff42c2fdae9e910154be)
|
| |
|
|
|
|
| |
supporting custom dburi etc.
Change-Id: Ic0ab0b3b4223e40fd335ee3313fda4dfce942100
|
| |
|
|
|
|
| |
run to prevent race conditions against concurrent runs
Change-Id: I065d1cec346ea7af03792c3cc2f30766f73c2bd3
|
| | |
|
| |
|
|
| |
per server
|
| |
|
|
|
|
|
| |
function patched onto config. nose/pytest backends now fill
in their exception class here only when loaded
- use more public seeming api to get at py.test Skipped
exception
|
| |
|
|
| |
test suite redefining an existing test in test_suite
|
| |
|
|
|
|
|
|
| |
- reestablish the "bootstrap" system of loading the test runners
in testing/plugin; using the updated approach we just came up with for
alembic. Coverage should be fixed now when running either py.test or
nose. fixes #3196
- upgrade tox.ini and start using a .coveragerc file
|
| |
|
|
| |
- guard against some potential pytest snarkiness
|
| |
|
|
| |
are a little more crazy under xdist mode
|
| |
|
|
| |
simplify tox again now that we can exclude tests more easily
|
| | |
|
| | |
|
| |
|
|
| |
sqlalchemy/orm, sqlalchemy/event, sqlalchemy/testing
|
| |
|
|
|
| |
- move the logic to determine "test id" into plugin_base
- update callcounts
|
| | |
|
| |
|
|
|
|
| |
number of tests.
- move out logging tests from test_execute to test_logging
|
| | |
|
|
|
is currently being supported in addition to nose, and will likely
be preferred to nose going forward. The nose plugin system used
by SQLAlchemy has been split out so that it works under pytest as
well. There are no plans to drop support for nose at the moment
and we hope that the test suite itself can continue to remain as
agnostic of testing platform as possible. See the file
README.unittests.rst for updated information on running tests
with pytest.
The test plugin system has also been enhanced to support running
tests against mutiple database URLs at once, by specifying the ``--db``
and/or ``--dburi`` flags multiple times. This does not run the entire test
suite for each database, but instead allows test cases that are specific
to certain backends make use of that backend as the test is run.
When using pytest as the test runner, the system will also run
specific test suites multiple times, once for each database, particularly
those tests within the "dialect suite". The plan is that the enhanced
system will also be used by Alembic, and allow Alembic to run
migration operation tests against multiple backends in one run, including
third-party backends not included within Alembic itself.
Third party dialects and extensions are also encouraged to standardize
on SQLAlchemy's test suite as a basis; see the file README.dialects.rst
for background on building out from SQLAlchemy's test platform.
|
