summaryrefslogtreecommitdiff
path: root/test/engine/test_pool.py
Commit message (Collapse)AuthorAgeFilesLines
* fix test suite warningsMike Bayer2023-05-091-0/+5
| | | | | | | | | | | | | | | | fix a handful of warnings that were emitting but not raising, usually because they were inside an "expect_warnings" block. modify "expect_warnings" to always use "raise_on_any_unexpected" behavior; remove this parameter. Fixed issue in semi-private ``await_only()`` and ``await_fallback()`` concurrency functions where the given awaitable would remain un-awaited if the function threw a ``GreenletError``, which could cause "was not awaited" warnings later on if the program continued. In this case, the given awaitable is now cancelled before the exception is thrown. Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
* do not return asyncio connections to the pool under gcMike Bayer2023-02-061-23/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a regression caused by the fix for :ticket:`8419` which caused asyncpg connections to be reset (i.e. transaction ``rollback()`` called) and returned to the pool normally in the case that the connection were not explicitly returned to the connection pool and was instead being intercepted by Python garbage collection, which would fail if the garbage collection operation were being called outside of the asyncio event loop, leading to a large amount of stack trace activity dumped into logging and standard output. The correct behavior is restored, which is that all asyncio connections that are garbage collected due to not being explicitly returned to the connection pool are detached from the pool and discarded, along with a warning, rather than being returned the pool, as they cannot be reliably reset. In the case of asyncpg connections, the asyncpg-specific ``terminate()`` method will be used to end the connection more gracefully within this process as opposed to just dropping it. This change includes a small behavioral change that is hoped to be useful for debugging asyncio applications, where the warning that's emitted in the case of asyncio connections being unexpectedly garbage collected has been made slightly more aggressive by moving it outside of a ``try/except`` block and into a ``finally:`` block, where it will emit unconditionally regardless of whether the detach/termination operation succeeded or not. It will also have the effect that applications or test suites which promote Python warnings to exceptions will see this as a full exception raise, whereas previously it was not possible for this warning to actually propagate as an exception. Applications and test suites which need to tolerate this warning in the interim should adjust the Python warnings filter to allow these warnings to not raise. The behavior for traditional sync connections remains unchanged, that garbage collected connections continue to be returned to the pool normally without emitting a warning. This will likely be changed in a future major release to at least emit a similar warning as is emitted for asyncio drivers, as it is a usage error for pooled connections to be intercepted by garbage collection without being properly returned to the pool. Fixes: #9237 Change-Id: Ib35cfb2e628f2eb2da6d2b65674702556f55603a
* catch all BaseException in pool and revert failed checkoutsMike Bayer2022-12-131-12/+50
| | | | | | | | | | | | | | | | | | | | | Fixed a long-standing race condition in the connection pool which could occur under eventlet/gevent monkeypatching schemes in conjunction with the use of eventlet/gevent ``Timeout`` conditions, where a connection pool checkout that's interrupted due to the timeout would fail to clean up the failed state, causing the underlying connection record and sometimes the database connection itself to "leak", leaving the pool in an invalid state with unreachable entries. This issue was first identified and fixed in SQLAlchemy 1.2 for :ticket:`4225`, however the failure modes detected in that fix failed to accommodate for ``BaseException``, rather than ``Exception``, which prevented eventlet/gevent ``Timeout`` from being caught. In addition, a block within initial pool connect has also been identified and hardened with a ``BaseException`` -> "clean failed connect" block to accommodate for the same condition in this location. Big thanks to Github user @niklaus for their tenacious efforts in identifying and describing this intricate issue. Fixes: #8974 Change-Id: I95a0e1f080d0cee6f1a66977432a586fdf87f686
* Try running pyupgrade on the codeFederico Caselli2022-11-161-3/+3
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* ensure pool.reset event always called for resetMike Bayer2022-10-301-1/+132
| | | | | | | | | | | | | | | | | | | | | | Added new parameter :paramref:`.PoolEvents.reset.reset_state` parameter to the :meth:`.PoolEvents.reset` event, with deprecation logic in place that will continue to accept event hooks using the previous set of arguments. This indicates various state information about how the reset is taking place and is used to allow custom reset schemes to take place with full context given. Within this change a fix that's also backported to 1.4 is included which re-enables the :meth:`.PoolEvents.reset` event to continue to take place under all circumstances, including when :class:`.Connection` has already "reset" the connection. The two changes together allow custom reset schemes to be implemented using the :meth:`.PoolEvents.reset` event, instead of the :meth:`.PoolEvents.checkin` event (which continues to function as it always has). Change-Id: Ie17c4f55d02beb6f570b9de6b3044baffa7d6df6 Fixes: #8717
* Ignore max_overflow when pool_size is 0Federico Caselli2022-09-151-0/+10
| | | | | | | | The :class:`_pool.QueuePool` now ignores ``max_overflow`` when ``pool_size=0``, properly making the pool unlimited in all cases. Fixes: #8523 Change-Id: Ifc32eb47a281c4b3acf357352f07b9b8a73d1b6f
* Merge "integrate connection.terminate() for supporting dialects" into mainmike bayer2022-08-241-5/+15
|\
| * integrate connection.terminate() for supporting dialectsMike Bayer2022-08-241-5/+15
| | | | | | | | | | | | | | | | | | | | | | | | Integrated support for asyncpg's ``terminate()`` method call for cases where the connection pool is recycling a possibly timed-out connection, where a connection is being garbage collected that wasn't gracefully closed, as well as when the connection has been invalidated. This allows asyncpg to abandon the connection without waiting for a response that may incur long timeouts. Fixes: #8419 Change-Id: Ia575af779d5733b483a72dff3690b8bbbad2bb05
* | fix mock issue in pool testMike Bayer2022-08-241-0/+1
|/ | | | | | | | | | | | | | | Due to the change for #5648 in ad14471bc99c2fb2315d3333a95dd, the mock in test_recycle_pool_no_race needs to implement an empty handle_error dispatcher. the exception was in a thread so did not interrupt the test suite: File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 2059, in _handle_dbapi_exception for fn in self.dialect.dispatch.handle_error: TypeError: 'Mock' object is not iterable Change-Id: I764357c48aa1bf53a572d8ee24c89a7463505092
* deprecate .connection on _ConnectionFairy, _ConnectionRecordMike Bayer2022-05-221-19/+0
| | | | | | | | | | | | These are replaced by the read-only ManagesConnection.dbapi_connection attribute. For some reason both of these objects had "setter" for .connection as well; there's no use case for that at all so just remove setter logic entirely. Fixes: #6981 Change-Id: I6425de4a017f6370e1a7476cd491cabc55e55e67
* pep 484 for typesMike Bayer2022-03-191-1/+1
| | | | | | | strict types type_api.py, including TypeDecorator, NativeForEmulated, etc. Change-Id: Ib2eba26de0981324a83733954cb7044a29bbd7db
* pep-484 for poolMike Bayer2022-02-171-2/+3
| | | | | | | | | | | | | | | | | also extends into some areas of utils, events and others as needed. Formalizes a public hierarchy for pool API, with ManagesConnection -> PoolProxiedConnection / ConnectionPoolEntry for connectionfairy / connectionrecord, which are now what's exposed in the event API and other APIs. all public API docs moved to the new objects. Corrects the mypy plugin's check for sqlalchemy-stubs not being insatlled, which has to be imported using the dash in the name to be effective. Change-Id: I16c2cb43b2e840d28e70a015f370a768e70f3581
* dont use exception catches for warnings; modernize xdist detectionMike Bayer2022-01-221-2/+2
| | | | | | | | | | | | | | | | | Improvements to the test suite's integration with pytest such that the "warnings" plugin, if manually enabled, will not interfere with the test suite, such that third parties can enable the warnings plugin or make use of the ``-W`` parameter and SQLAlchemy's test suite will continue to pass. Additionally, modernized the detection of the "pytest-xdist" plugin so that plugins can be globally disabled using PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 without breaking the test suite if xdist were still installed. Warning filters that promote deprecation warnings to errors are now localized to SQLAlchemy-specific warnings, or within SQLAlchemy-specific sources for general Python deprecation warnings, so that non-SQLAlchemy deprecation warnings emitted from pytest plugins should also not impact the test suite. Fixes: #7599 Change-Id: Ibcf09af25228d39ee5a943fda82d8a9302433726
* remove dbapi_proxy moduleMike Bayer2022-01-141-5/+0
| | | | | | | | | | | This module was not documented nor part of any test suite, and it's unlikely it was working correctly. It's not likely that this module was ever used after the first year or so of SQLAlchemy, and it's stayed around because it is so obscure that I never remembered to remove it. Change-Id: I0ed9030438982e935add87c51abbfff50e7382be References: #7257
* Clean up most py3k compatFederico Caselli2021-11-241-4/+4
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Remove object in class definitionFederico Caselli2021-11-221-1/+1
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* First round of removal of python 2Federico Caselli2021-11-011-2/+2
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* Surface driver connection object when using a proxied dialectFederico Caselli2021-09-171-64/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | Improve the interface used by adapted drivers, like the asyncio ones, to access the actual connection object returned by the driver. The :class:`_engine._ConnectionRecord` and :class:`_engine._ConnectionFairy` now have two new attributes: * ``dbapi_connection`` always represents a DBAPI compatible object. For pep-249 drivers, this is the DBAPI connection as it always has been, previously accessed under the ``.connection`` attribute. For asyncio drivers that SQLAlchemy adapts into a pep-249 interface, the returned object will normally be a SQLAlchemy adaption object called :class:`_engine.AdaptedConnection`. * ``driver_connection`` always represents the actual connection object maintained by the third party pep-249 DBAPI or async driver in use. For standard pep-249 DBAPIs, this will always be the same object as that of the ``dbapi_connection``. For an asyncio driver, it will be the underlying asyncio-only connection object. The ``.connection`` attribute remains available and is now a legacy alias of ``.dbapi_connection``. Fixes: #6832 Change-Id: Ib72f97deefca96dce4e61e7c38ba430068d6a82e
* Modernize testsGord Thompson2021-07-031-1/+2
| | | | | | Eliminate engine.execute() and engine.scalar() Change-Id: I99f76d0e615ddebab2da4fd07a40a0a2796995c7
* Propagate asyncio flag from the dialect to selected pool classesFederico Caselli2021-06-081-1/+35
| | | | | | | | | Fixed an issue that presented itself when using the :class:`_pool.NullPool` or the :class:`_pool.StaticPool` with an async engine. This mostly affected the aiosqlite dialect. Fixes: #6575 Change-Id: Ic1e27d99ffcb20ed4de82ea78f430a0f3b629d86
* Add new "sync once" mode for pool.connectMike Bayer2021-04-211-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed critical regression caused by the change in :ticket`5497` where the connection pool "init" phase no longer occurred within mutexed isolation, allowing other threads to proceed with the dialect uninitialized, which could then impact the compilation of SQL statements. This issue is essentially the same regression which was fixed many years ago in :ticket:`2964` in dd32540dabbee0678530fb1b0868d1eb41572dca, which was missed this time as the test suite fo that issue only tested the pool in isolation, and assumed the "first_connect" event would be used by the Engine. However :ticket:`5497` stopped using "first_connect" and no test detected the lack of mutexing, that has been resolved here through the addition of more tests. This fix also identifies what is probably a bug in earlier versions of SQLAlchemy where the "first_connect" handler would be cancelled if the initializer failed; this is evidenced by test_explode_in_initializer which was doing a reconnect due to c.rollback() yet wasn't hanging. We now solve this issue by preventing the manufactured Connection from ever reconnecting inside the first_connect handler. Also remove the "_sqla_unwrap" test attribute; this is almost not used anymore however we can use a more targeted wrapper supplied by the testing.engines.proxying_engine function. See if we can also open up Oracle for "ad hoc engines" tests now that we have better connection management logic. Fixes: #6337 Change-Id: I4a3476625c4606f1a304dbc940d500325e8adc1a
* Add support for aiosqliteFederico Caselli2021-03-241-1/+16
| | | | | | | | Added support for the aiosqlite database driver for use with the SQLAlchemy asyncio extension. Fixes: #5920 Change-Id: Id11a320516a44e886a6f518d2866a0f992413e55
* Replace reset_agent with direct call from connectionMike Bayer2021-03-061-86/+0
| | | | | | | | | | | | | Fixed a regression where the "reset agent" of the connection pool wasn't really being utilized by the :class:`_engine.Connection` when it were closed, and also leading to a double-rollback scenario that was somewhat wasteful. The newer architecture of the engine has been updated so that the connection pool "reset-on-return" logic will be skipped when the :class:`_engine.Connection` explicitly closes out the transaction before returning the pool to the connection. Fixes: #6004 Change-Id: I5d2ac16cac71aa45a00b4b7481d7268bd828a168
* Fix a couple of bugs in the asyncio implementationFederico Caselli2021-01-211-12/+14
| | | | | | | | | | | | | Log an informative message if a connection is not closed and the gc is reclaiming it when using an async dpapi, that does not support running IO at that stage. The ``AsyncAdaptedQueue`` used by default on async dpapis should instantiate a queue only when it's first used to avoid binding it to a possibly wrong event loop. Fixes: #5823 Change-Id: Ibfc50e209b1937ae3d6599ae7997f028c7a92c33
* Create explicit GC ordering between ConnectionFairy/ConnectionRecordMike Bayer2021-01-151-8/+2
| | | | | | | | | | | | | | Fixed issue where connection pool would not return connections to the pool or otherwise be finalized upon garbage collection under pypy if the checked out connection fell out of scope without being closed. This is a long standing issue due to pypy's difference in GC behavior that does not call weakref finalizers if they are relative to another object that is also being garbage collected. A strong reference to the related record is now maintained so that the weakref has a strong-referenced "base" to trigger off of. Fixes: #5842 Change-Id: Id5448fdacb6cceaac1ea40b2fbc851f052ed8e86
* reinvent xdist hooks in terms of pytest fixturesMike Bayer2021-01-131-10/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To allow the "connection" pytest fixture and others work correctly in conjunction with setup/teardown that expects to be external to the transaction, remove and prevent any usage of "xdist" style names that are hardcoded by pytest to run inside of fixtures, even function level ones. Instead use pytest autouse fixtures to implement our own r"setup|teardown_test(?:_class)?" methods so that we can ensure function-scoped fixtures are run within them. A new more explicit flow is set up within plugin_base and pytestplugin such that the order of setup/teardown steps, which there are now many, is fully documented and controllable. New granularity has been added to the test teardown phase to distinguish between "end of the test" when lock-holding structures on connections should be released to allow for table drops, vs. "end of the test plus its teardown steps" when we can perform final cleanup on connections and run assertions that everything is closed out. From there we can remove most of the defensive "tear down everything" logic inside of engines which for many years would frequently dispose of pools over and over again, creating for a broken and expensive connection flow. A quick test shows that running test/sql/ against a single Postgresql engine with the new approach uses 75% fewer new connections, creating 42 new connections total, vs. 164 new connections total with the previous system. As part of this, the new fixtures metadata/connection/future_connection have been integrated such that they can be combined together effectively. The fixture_session(), provide_metadata() fixtures have been improved, including that fixture_session() now strongly references sessions which are explicitly torn down before table drops occur afer a test. Major changes have been made to the ConnectionKiller such that it now features different "scopes" for testing engines and will limit its cleanup to those testing engines corresponding to end of test, end of test class, or end of test session. The system by which it tracks DBAPI connections has been reworked, is ultimately somewhat similar to how it worked before but is organized more clearly along with the proxy-tracking logic. A "testing_engine" fixture is also added that works as a pytest fixture rather than a standalone function. The connection cleanup logic should now be very robust, as we now can use the same global connection pools for the whole suite without ever disposing them, while also running a query for PostgreSQL locks remaining after every test and assert there are no open transactions leaking between tests at all. Additional steps are added that also accommodate for asyncio connections not explicitly closed, as is the case for legacy sync-style tests as well as the async tests themselves. As always, hundreds of tests are further refined to use the new fixtures where problems with loose connections were identified, largely as a result of the new PostgreSQL assertions, many more tests have moved from legacy patterns into the newest. An unfortunate discovery during the creation of this system is that autouse fixtures (as well as if they are set up by @pytest.mark.usefixtures) are not usable at our current scale with pytest 4.6.11 running under Python 2. It's unclear if this is due to the older version of pytest or how it implements itself for Python 2, as well as if the issue is CPU slowness or just large memory use, but collecting the full span of tests takes over a minute for a single process when any autouse fixtures are in place and on CI the jobs just time out after ten minutes. So at the moment this patch also reinvents a small version of "autouse" fixtures when py2k is running, which skips generating the real fixture and instead uses two global pytest fixtures (which don't seem to impact performance) to invoke the "autouse" fixtures ourselves outside of pytest. This will limit our ability to do more with fixtures until we can remove py2k support. py.test is still observed to be much slower in collection in the 4.6.11 version compared to modern 6.2 versions, so add support for new TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that will run the suite for fewer backends under Python 2. For Python 3 pin pytest to modern 6.2 versions where performance for collection has been improved greatly. Includes the following improvements: Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would be raised rather than :class:`.exc.TimeoutError`. Also repaired the :paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using the async engine, which previously would ignore the timeout and block rather than timing out immediately as is the behavior with regular :class:`.QueuePool`. For asyncio the connection pool will now also not interact at all with an asyncio connection whose ConnectionFairy is being garbage collected; a warning that the connection was not properly closed is emitted and the connection is discarded. Within the test suite the ConnectionKiller is now maintaining strong references to all DBAPI connections and ensuring they are released when tests end, including those whose ConnectionFairy proxies are GCed. Identified cx_Oracle.stmtcachesize as a major factor in Oracle test scalability issues, this can be reset on a per-test basis rather than setting it to zero across the board. the addition of this flag has resolved the long-standing oracle "two task" error problem. For SQL Server, changed the temp table style used by the "suite" tests to be the double-pound-sign, i.e. global, variety, which is much easier to test generically. There are already reflection tests that are more finely tuned to both styles of temp table within the mssql test suite. Additionally, added an extra step to the "dropfirst" mechanism for SQL Server that will remove all foreign key constraints first as some issues were observed when using this flag when multiple schemas had not been torn down. Identified and fixed two subtle failure modes in the engine, when commit/rollback fails in a begin() context manager, the connection is explicitly closed, and when "initialize()" fails on the first new connection of a dialect, the transactional state on that connection is still rolled back. Fixes: #5826 Fixes: #5827 Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
* remove more bound metadataMike Bayer2021-01-051-1/+1
| | | | | | | | | | | | | | in Iae6ab95938a7e92b6d42086aec534af27b5577d3 I missed that the "bind" was being stuck onto the MetaData in TablesTest, which led thousands of ORM tests to still use bound metadata. Keep looking for bound metadata. standardize all ORM tests on a single means of getting a Session when the Session API isn't the thing we are directly testing, using a new function fixture_session() that replaces create_session() and uses modern defaults. Change-Id: Iaf71206e9ee568151496d8bc213a069504bf65ef
* QueuePool: support subsecond timeoutJordan Pittier2020-11-211-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes: #5719 <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description Make it explicit in the documentation and in the default value for the 'timeout' parameter that `timeout` can be a float. Because Python timing is not very accurate, warn about the precision. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #5710 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5710 Pull-request-sha: 5f4eef8b4aba756d32e14ea41f71ef2919c26b84 Change-Id: I462524b1624ca5cc76d083a1d58e5dc89501c1a9
* Support pool.connect() event firing before all elseMike Bayer2020-11-191-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where a connection pool event specified with a keyword, most notably ``insert=True``, would be lost when the event were set up. This would prevent startup events that need to fire before dialect-level events from working correctly. The internal mechanics of the engine connection routine has been altered such that it's now guaranteed that a user-defined event handler for the :meth:`_pool.PoolEvents.connect` handler, when established using ``insert=True``, will allow an event handler to run that is definitely invoked **before** any dialect-specific initialization starts up, most notably when it does things like detect default schema name. Previously, this would occur in most cases but not unconditionally. A new example is added to the schema documentation illustrating how to establish the "default schema name" within an on-connect event (upcoming as part of I882edd5bbe06ee5b4d0a9c148854a57b2bcd4741) Addiional changes to support setting default schema name: The Oracle dialect now uses ``select sys_context( 'userenv', 'current_schema' ) from dual`` to get the default schema name, rather than ``SELECT USER FROM DUAL``, to accommodate for changes to the session-local schema name under Oracle. Added a read/write ``.autocommit`` attribute to the DBAPI-adaptation layer for the asyncpg dialect. This so that when working with DBAPI-specific schemes that need to use "autocommit" directly with the DBAPI connection, the same ``.autocommit`` attribute which works with both psycopg2 as well as pg8000 is available. Fixes: #5716 Fixes: #5708 Change-Id: I7dce56b4345ffc720e25e2aaccb7e42bb29e5671
* Revert "Use monotonic time for pool age measurement"Mike Bayer2020-10-071-1/+1
| | | | | | | | | | This reverts commit 0220b58917b5a979891b5765f6ac5095e0368489. I completely misread https://www.python.org/dev/peps/pep-0418/#rationale and the accuracy of monotonic() is *worse* on windows than time.time(), which is bizarre. Change-Id: I2d571e268a2051bea68736507773d3904403af9e
* Use monotonic time for pool age measurementMike Bayer2020-10-071-1/+1
| | | | | | | | | | | | | The internal clock used by the :class:`_pool.Pool` object is now time.monotonic_time() under Python 3. Under Python 2, time.time() is still used, which is legacy. This clock is used to measure the age of a connection against its starttime, and used in comparisons against the pool_timeout setting as well as the last time the pool was marked as invalid to determine if the connection should be recycled. Previously, time.time() was used which was subject to inaccuracies as a result of system clock changes as well as poor time resolution on windows. Change-Id: I94f90044c1809508e26a5a00134981c2a00d0405
* add --notimingintensive; block from github jobsMike Bayer2020-10-071-2/+1
| | | | | | | | | | | | | | | | | | | | | | this provides a front-end option to disable tests marked as timing_intensive, all of which are in test_pool, which are more fragile and aren't consistent on the github runners. also remove /reduce unnecessary time.sleep() from two other pool tests that are not timing intensive. note that this removes test_hanging_connect_within_overflow from the github runs via the timing_intensive requirement. I've also removed MockReconnectTest from exclusions as those are really important tests and they use mocks so should not have platform dependent issues. Need to see what the windows failures are. Closes: #5633 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5633 Pull-request-sha: 166833e16ec342dfa10edb287d7aa495ddd1b59d Change-Id: Icb3d284a2a952e2495d80fa91e22e0b32a54340f
* Enable pypy tests on github workflowFederico Caselli2020-10-021-0/+1
| | | | | Fixes: #5223 Change-Id: I0952e54ed9af2952ea340be1945311376ffc1ad2
* Pass all pool parameters in recreate()Mike Bayer2020-09-141-1/+56
| | | | | | | | | | | The following pool parameters were not being propagated to the new pool created when :meth:`_engine.Engine.dispose` were called: ``pre_ping``, ``use_lifo``. Additionally the ``recycle`` and ``reset_on_return`` parameters were not propagated for the :class:`_engine.AssertionPool` class. These issues have been fixed. Fixes: #5582 Change-Id: Ifdb703aa7e849652242b9ff8071c854cd1d77e71
* Update select usage to use the new 1.4 formatFederico Caselli2020-09-081-1/+1
| | | | | | | | | | | | | | | | This change includes mainly that the bracketed use within select() is moved to positional, and keyword arguments are removed from calls to the select() function. it does not yet fully address other issues such as keyword arguments passed to the table.select(). Additionally, allows False / None to both be considered as "disable" for all of select.correlate(), select.correlate_except(), query.correlate(), which establishes consistency with passing of ``False`` for the legact select(correlate=False) argument. Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
* internal test framework files for standardization of is_not/not_in;jonathan vanasco2020-08-291-6/+6
| | | | | | this is safe for 1.3.x Change-Id: Icba38fdc20f5d8ac407383a4278ccb346e09af38
* Add an extra step to pool test_syncMike Bayer2020-06-271-0/+16
| | | | | | | | | Have observed CI failure with windows where not all three connections got pulled out at the same time here as the threads got serialized. make sure all three connections get used. Change-Id: Ic2f7c7de1069358d95035f90c725c7dddd4f34d4
* Assert reset agent always set correctly and is activeMike Bayer2020-05-131-0/+4
| | | | | | | | | | | | | | | | | | | | Fixed fairly critical issue where the DBAPI connection could be returned to the connection pool while still in an un-rolled-back state. The reset agent responsible for rolling back the connection could be corrupted in the case that the transaction was "closed" without being rolled back or committed, which can occur in some scenarios when using ORM sessions and emitting .close() in a certain pattern involving savepoints. The fix ensures that the reset agent is always active. note that the reset agent will go away in 2.0 and the only real purpose of it is for logging of ROLLBACK. Apparently with the SQLite singleton engine in the test suite, there are some strucutral mismatches in the test fixtures where the reset agent is getting set differently than the transaction likely due to the same connection being shared in multiple context, though it's unclear. Fixes: #5326 Change-Id: If056870ea70a2d9a1749768988d5e023f3061b31
* Ensure all nested exception throws have a causeMike Bayer2020-03-021-1/+2
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Do away with pool._refsMike Bayer2020-02-011-5/+7
| | | | | | | | | | | | | This collection was added only for the benefit of unit tests and is unnecessary for the pool to function. As SQLAlchemy 2.0 will be removing the automatic handling of connections that are garbage collection, remove this collection so that we ultimately don't need a weakref handler to do anything within the pool. The handler will do nothing other than emit a warning that a connection was dereferenced without being explicitly returned to the pool, invalidated, or detached. Change-Id: I4ca196270d5714efbac44dbf6f034e8c7f0af58a
* Use context managers for threading.Lock()Heckad2020-01-031-4/+1
| | | | | | | | | | | | | (zzzeek:) For some reason I thought that threading.Lock() still did not support context managers, at least in Python 2, however this does not seem to be the case. Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #5069 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5069 Pull-request-sha: efeac06dda5afdbe33abcf9b27c8b5b5725c8444 Change-Id: Ic64fcd99cd587bc70b4ecc5b45d8205b5c76eff2
* Note time passage requirement for pool.invalidate()Mike Bayer2019-12-271-1/+16
| | | | | | | | | | | | | | | | | | For Windows, time.time() may only have 16 millisecond accuracy, so invalidation routines which compare the time.time() of invalidate() to the time.time() when the ConnectionRecord last connected may fail in a unit test environment that does not pause at least this much time since the ConnectionRecord startup. Using >= for comparison instead of > was considered but this only leads to more confusing results as the ConnecitonRecord goes into a re-connect loop as time continues to not pass. Overall, while using routines such as Python 3.7's time_ns() might be helpful, for now make sure tests which rely on this are marked under timing intensive and add small sleeps. Change-Id: I1a7162e67912d22c135fa517b687a073f8fd9151
* Remove threadlocal engine strategy, engine strategies pool threadlocalMike Bayer2019-07-151-2/+0
| | | | | | | | | | | | The "threadlocal" execution strategy, deprecated in 1.3, has been removed for 1.4, as well as the concept of "engine strategies" and the ``Engine.contextual_connect`` method. The "strategy='mock'" keyword argument is still accepted for now with a deprecation warning; use :func:`.create_mock_engine` instead for this use case. Fixes: #4632 Change-Id: I8a351f9fa1f7dfa2a56eec1cd2d1a4b9d65765a2 (cherry picked from commit b368c49b44c5716d93c7428ab22b6761c6ca7cf5)
* Restore use_threadlocal equivalent behavior to SingletonThreadPoolMike Bayer2019-04-031-0/+25
| | | | | | | | | | | | | | | | | | Fixed behavioral regression as a result of deprecating the "use_threadlocal" flag for :class:`.Pool`, where the :class:`.SingletonThreadPool` no longer makes use of this option which causes the "rollback on return" logic to take place when the same :class:`.Engine` is used multiple times in the context of a transaction to connect or implicitly execute, thereby cancelling the transaction. While this is not the recommended way to work with engines and connections, it is nonetheless a confusing behavioral change as when using :class:`.SingletonThreadPool`, the transaction should stay open regardless of what else is done with the same engine in the same thread. The ``use_threadlocal`` flag remains deprecated however the :class:`.SingletonThreadPool` now implements its own version of the same logic. Fixes: #4585 Change-Id: I906293f2d0a5d14ed46cd9e64305a6481505a5a3
* Loosen SingletonThreadPoolTestMike Bayer2019-03-061-1/+4
| | | | | | | | | This test is non-critical as the SingletonThreadPool is not production-level robust under multithreaded scenarios in any case. Fixes: #4527 Change-Id: Ie19ebd69438c97b2d8adb571f8f1b2c56894f7fb
* run a black runMike Bayer2019-03-051-2/+1
| | | | | | fix some pep8s that snuck in Change-Id: Ied282007df30a52d232b1ba88659f2a123ff380f
* Merge "Provide public accessor for Pool.timeout()."mike bayer2019-01-241-0/+6
|\
| * Provide public accessor for Pool.timeout().Irina Delamare2019-01-241-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Added public accessor :meth:`.Pool.timeout` that returns the configured timeout for a :class:`.Pool` object. Pull request courtesy Irina Delamare. Fixes: #3689 Closes: #4447 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4447 Pull-request-sha: 0179b0a829e9609429dc698992670d2e6749c10c Change-Id: I402b065cf9183160f71d9de73e308268356b7deb
* | Add deprecation warnings to all deprecated APIsMike Bayer2019-01-231-424/+0
|/ | | | | | | | | | | | | | | A large change throughout the library has ensured that all objects, parameters, and behaviors which have been noted as deprecated or legacy now emit ``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now defaults to displaying deprecation warnings, as well as that modern test suites based on tools like tox and pytest tend to display deprecation warnings, this change should make it easier to note what API features are obsolete. See the notes added to the changelog and migration notes for further details. Fixes: #4393 Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
* use ..deprecated directive w/ version in all casesMike Bayer2019-01-111-2/+8
| | | | | | | | | These changes should be ported from 1.3 back to 1.0 or possibly 0.9 to the extent they are relevant in each version. In 1.3 we hope to turn all deprecation documentation into warnings. Change-Id: I205186cde161af9389af513a425c62ce90dd54d8
View Lorry Controller Status