summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/engine
Commit message (Collapse)AuthorAgeFilesLines
...
* implement batched INSERT..VALUES () () for executemanyMike Bayer2022-09-244-57/+566
| | | | | | | | | | | | | | | | | | | | the feature is enabled for all built in backends when RETURNING is used, except for Oracle that doesn't need it, and on psycopg2 and mssql+pyodbc it is used for all INSERT statements, not just those that use RETURNING. third party dialects would need to opt in to the new feature by setting use_insertmanyvalues to True. Also adds dialect-level guards against using returning with executemany where we dont have an implementation to suit it. execute single w/ returning still defers to the server without us checking. Fixes: #6047 Fixes: #7907 Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
* Tighten password security by removing `URL.__str__`Yassen Damyanov2022-09-231-3/+0
| | | | | | | | | | | | | | | For improved security, the :class:`_url.URL` object will now use password obfuscation by default when ``str(url)`` is called. To stringify a URL with cleartext password, the :meth:`_url.URL.render_as_string` may be used, passing the :paramref:`_url.URL.render_as_string.hide_password` parameter as ``False``. Thanks to our contributors for this pull request. Fixes: #8567 Closes: #8563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8563 Pull-request-sha: d1f1127f753849eb70b8d6cc64badf34e1b9219b Change-Id: If756c8073ff99ac83876d9833c8fe1d7c76211f9
* Merge "break out text() from TextualSelect for col matching" into mainmike bayer2022-09-202-4/+15
|\
| * break out text() from TextualSelect for col matchingMike Bayer2022-09-192-4/+15
| | | | | | | | | | | | | | | | | | | | Fixed issue where mixing "*" with additional explicitly-named column expressions within the columns clause of a :func:`_sql.select` construct would cause result-column targeting to sometimes consider the label name or other non-repeated names to be an ambiguous target. Fixes: #8536 Change-Id: I3c845eaf571033e54c9208762344f67f4351ac3a
* | change verbiage stating exact compliance with RFC-1738Mike Bayer2022-09-171-14/+19
|/ | | | | | | | | | | | | | As long as we aren't using urlparse() to parse URLs, we are not RFC-1738 compliant. As we accept underscores in the scheme and not dashes or dots, we are not RFC-1738 compliant, so emulate language like that of PostgreSQL [1] that we "generally follow" this scheme but include some exceptions. [1] https://www.postgresql.org/docs/current/libpq-connect.html#id-1.7.3.8.3.6 Fixes: #8519 Change-Id: I2d7e55d9df17aed122cebb2c4c315f56c06a3da5
* integrate connection.terminate() for supporting dialectsMike Bayer2022-08-242-0/+26
| | | | | | | | | | | | Integrated support for asyncpg's ``terminate()`` method call for cases where the connection pool is recycling a possibly timed-out connection, where a connection is being garbage collected that wasn't gracefully closed, as well as when the connection has been invalidated. This allows asyncpg to abandon the connection without waiting for a response that may incur long timeouts. Fixes: #8419 Change-Id: Ia575af779d5733b483a72dff3690b8bbbad2bb05
* restore Connectable and other changes for oslo.dbMike Bayer2022-08-121-0/+2
| | | | | | | | | | | at this point oslo.db should be passing tests however they seem to be importing "Connectable", which we had temporarily removed, it's now a different kind of object but restore it to the engine import space which seems to be where they're pulling it from to see if oslo.db can run some tests now. Change-Id: I52cb5ad82b00eec12004eeeb571c4eb8efa5ced2
* translate joined inheritance cols in UPDATE/DELETEMike Bayer2022-08-052-0/+16
| | | | | | | | | | | | | | | | | | | | | | | Fixed issue in ORM enabled UPDATE when the statement is created against a joined-inheritance subclass, updating only local table columns, where the "fetch" synchronization strategy would not render the correct RETURNING clause for databases that use RETURNING for fetch synchronization. Also adjusts the strategy used for RETURNING in UPDATE FROM and DELETE FROM statements. Also fixes MariaDB which does not support RETURNING with DELETE..USING. this was not caught in tests because "fetch" strategy wasn't tested. so also adjust the ORMDMLState classes to look for "extra froms" first before adding RETURNING, add new parameters to interfaces for "update_returning_multitable" and "delete_returning_multitable". A new execution option is_delete_using=True, described in the changelog message, is added to allow the ORM to know up front if a certain statement should have a SELECT up front for "fetch" strategy. Fixes: #8344 Change-Id: I3dcdb68e6e97ab0807a573c2fdb3d53c16d063ba
* ensure RETURNING renders in stringify w/ no server versionMike Bayer2022-08-031-0/+4
| | | | | | | | | just in my own testing, if I say insert().return_defaults() and stringify, I should see it, so make sure all the dialects default to "insert_returning" etc. , with downgrade on server version check. Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
* fix up SAVEPOINT docsMike Bayer2022-08-021-0/+2
| | | | | | | | | | | these contained a factual error that the entire session is expired, which is no longer the case (I can't find exactly when this was changed). Additionally, added a PostgreSQL specific example w/ IntegrityError as this is the most common case for this. Tried to tighten up other language and make it as clear as possible. Change-Id: I39160e7443964db59d1d5a2e0616084767813eea
* Update to flake8 5.Federico Caselli2022-07-314-6/+12
| | | | Change-Id: I5a241a70efba68bcea9819ddce6aebc25703e68d
* Reflect expression-based indexes on PostgreSQLFederico Caselli2022-07-282-35/+55
| | | | | | | | | | | The PostgreSQL dialect now supports reflection of expression based indexes. The reflection is supported both when using :meth:`_engine.Inspector.get_indexes` and when reflecting a :class:`_schema.Table` using :paramref:`_schema.Table.autoload_with`. Thanks to immerrr and Aidan Kane for the help on this ticket. Fixes: #7442 Change-Id: I3e36d557235286c0f7f6d8276272ff9225058d48
* repair yield_per for non-SS dialects and add new optionsMike Bayer2022-07-015-25/+153
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented new :paramref:`_engine.Connection.execution_options.yield_per` execution option for :class:`_engine.Connection` in Core, to mirror that of the same :ref:`yield_per <orm_queryguide_yield_per>` option available in the ORM. The option sets both the :paramref:`_engine.Connection.execution_options.stream_results` option at the same time as invoking :meth:`_engine.Result.yield_per`, to provide the most common streaming result configuration which also mirrors that of the ORM use case in its usage pattern. Fixed bug in :class:`_engine.Result` where the usage of a buffered result strategy would not be used if the dialect in use did not support an explicit "server side cursor" setting, when using :paramref:`_engine.Connection.execution_options.stream_results`. This is in error as DBAPIs such as that of SQLite and Oracle already use a non-buffered result fetching scheme, which still benefits from usage of partial result fetching. The "buffered" strategy is now used in all cases where :paramref:`_engine.Connection.execution_options.stream_results` is set. Added :meth:`.FilterResult.yield_per` so that result implementations such as :class:`.MappingResult`, :class:`.ScalarResult` and :class:`.AsyncResult` have access to this method. Fixes: #8199 Change-Id: I6dde3cbe483a1bf81e945561b60f4b7d1c434750
* Improve reflection method docsFederico Caselli2022-06-302-162/+84
| | | | | | | Point to the typed dict for the documentation instead of duplicating it in the mothod text. Change-Id: I041abee7dc27b95409950741e702d69101c22c6d
* add more cross-linking / notes for yield_per, partitionsMike Bayer2022-06-301-0/+20
| | | | | Change-Id: I0f8db2532827c76a2751186638d22104230db843 references: #8198
* Comments on (named) constraintscheremnov2022-06-293-18/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667). Supports only PostgreSQL backend. ### Description Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints. Enables setting the comments for named constraints like this: ``` Table( 'example', metadata, Column('id', Integer), Column('data', sa.String(30)), PrimaryKeyConstraint( "id", name="id_pk", comment="id_pk comment" ), CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"), UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"), ) ``` Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` . ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - [ ] A short code fix - [x] A new feature implementation - Solves the issue 5667. - The commit message includes `Fixes: 5667`. - Includes tests based on comment reflection. **Have a nice day!** Fixes: #5667 Closes: #7742 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7742 Pull-request-sha: 42a5d3c3e9ccf9a9d5397fd007aeab0854f66130 Change-Id: Ia60f578595afdbd6089541c9a00e37997ef78ad3
* remove warnings for index/unique skipped due to exclude_colsMike Bayer2022-06-201-8/+0
| | | | | | | | | | | | | | | | The warnings that are emitted regarding reflection of indexes or unique constraints, when the :paramref:`.Table.include_columns` parameter is used to exclude columns that are then found to be part of those constraints, have been removed. When the :paramref:`.Table.include_columns` parameter is used it should be expected that the resulting :class:`.Table` construct will not include constraints that rely upon omitted columns. This change was made in response to :ticket:`8100` which repaired :paramref:`.Table.include_columns` in conjunction with foreign key constraints that rely upon omitted columns, where the use case became clear that omitting such constraints should be expected. Fixes: #8102 Change-Id: Id32f628def2d12499cd49d0b436ed345fe49dc6b
* rearchitect reflection for batched performanceFederico Caselli2022-06-184-288/+1737
| | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* Merge "update cx_Oracle / oracledb LOB handling" into mainmike bayer2022-06-101-0/+3
|\
| * update cx_Oracle / oracledb LOB handlingMike Bayer2022-06-101-0/+3
| | | | | | | | | | | | | | | | | | | | Adjustments made to the BLOB / CLOB / NCLOB datatypes in the cx_Oracle and oracledb dialects, to improve performance based on recommendations from Oracle developers. References: https://github.com/oracle/python-cx_Oracle/issues/596 Fixes: #7494 Change-Id: I0d8cc3579140aa65cacf5b7d3373f7e1929a8f85
* | restore parameter escaping for public methodsMike Bayer2022-06-091-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Adjusted the fix made for :ticket:`8056` which adjusted the escaping of bound parameter names with special characters such that the escaped names were translated after the SQL compilation step, which broke a published recipe on the FAQ illustrating how to merge parameter names into the string output of a compiled SQL string. The change restores the escaped names that come from ``compiled.params`` and adds a conditional parameter to :meth:`.SQLCompiler.construct_params` named ``escape_names`` that defaults to ``True``, restoring the old behavior by default. Fixes: #8113 Change-Id: I9cbedb1080bc06d51f287fd2cbf26aaab1c74653
* | Merge "graceful degrade for FKs not reflectable" into mainmike bayer2022-06-081-1/+11
|\ \ | |/ |/|
| * graceful degrade for FKs not reflectableMike Bayer2022-06-071-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed bugs involving the :paramref:`.Table.include_columns` and the :paramref:`.Table.resolve_fks` parameters on :class:`.Table`; these little-used parameters were apparently not working for columns that refer to foreign key constraints. In the first case, not-included columns that refer to foreign keys would still attempt to create a :class:`.ForeignKey` object, producing errors when attempting to resolve the columns for the foreign key constraint within reflection; foreign key constraints that refer to skipped columns are now omitted from the table reflection process in the same way as occurs for :class:`.Index` and :class:`.UniqueConstraint` objects with the same conditions. No warning is produced however, as we likely want to remove the include_columns warnings for all constraints in 2.0. In the latter case, the production of table aliases or subqueries would fail on an FK related table not found despite the presence of ``resolve_fks=False``; the logic has been repaired so that if a related table is not found, the :class:`.ForeignKey` object is still proxied to the aliased table or subquery (these :class:`.ForeignKey` objects are normally used in the production of join conditions), but it is sent with a flag that it's not resolvable. The aliased table / subquery will then work normally, with the exception that it cannot be used to generate a join condition automatically, as the foreign key information is missing. This was already the behavior for such foreign key constraints produced using non-reflection methods, such as joining :class:`.Table` objects from different :class:`.MetaData` collections. Fixes: #8100 Fixes: #8101 Change-Id: Ifa37a91bd1f1785fca85ef163eec031660d9ea4d
* | Merge "migrate labels to new tutorial" into mainmike bayer2022-06-073-9/+48
|\ \ | |/ |/|
| * migrate labels to new tutorialMike Bayer2022-06-073-9/+48
| | | | | | | | | | | | | | other org changes and some sections from old tutorial ported to new tutorial. Change-Id: Ic0fba60ec82fff481890887beef9ed0fa271875a
* | Generalize RETURNING and suppor for MariaDB / SQLiteDaniel Black2022-06-024-27/+58
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As almost every dialect supports RETURNING now, RETURNING is also made more of a default assumption. * the default compiler generates a RETURNING clause now when specified; CompileError is no longer raised. * The dialect-level implicit_returning parameter now has no effect. It's not fully clear if there are real world cases relying on the dialect-level parameter, so we will see once 2.0 is released. ORM-level RETURNING can be disabled at the table level, and perhaps "implicit returning" should become an ORM-level option at some point as that's where it applies. * Altered ORM update() / delete() to respect table-level implicit returning for fetch. * Since MariaDB doesnt support UPDATE returning, "full_returning" is now split into insert_returning, update_returning, delete_returning * Crazy new thing. Dialects that have *both* cursor.lastrowid *and* returning. so now we can pick between them for SQLite and mariadb. so, we are trying to keep it on .lastrowid for simple inserts with an autoincrement column, this helps with some edge case test scenarios and i bet .lastrowid is faster anyway. any return_defaults() / multiparams etc then we use returning * SQLite decided they dont want to return rows that match in ON CONFLICT. this is flat out wrong, but for now we need to work with it. Fixes: #6195 Fixes: #7011 Closes: #7047 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7047 Pull-request-sha: d25d5ea3abe094f282c53c7dd87f5f53a9e85248 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I9908ce0ff7bdc50bd5b27722081767c31c19a950
* Merge "add backend agnostic UUID datatype" into mainmike bayer2022-06-012-0/+9
|\
| * add backend agnostic UUID datatypeMike Bayer2022-06-012-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new backend-agnostic :class:`_types.Uuid` datatype generalized from the PostgreSQL dialects to now be a core type, as well as migrated :class:`_types.UUID` from the PostgreSQL dialect. Thanks to Trevor Gross for the help on this. also includes: * corrects some missing behaviors in the suite literal fixtures test where row round trips weren't being correctly asserted. * fixes some of the ISO literal date rendering added in 952383f9ee0 for #5052 to truncate datetime strings for date/time datatypes in the same way that drivers typically do for bound parameters; this was not working fully and wasn't caught by the broken test fixture Fixes: #7212 Change-Id: I981ac6d34d278c18281c144430a528764c241b04
* | Support handle_error for pre_pingMike Bayer2022-05-314-158/+205
|/ | | | | | | | | | | | | The :meth:`.DialectEvents.handle_error` event is now moved to the :class:`.DialectEvents` suite from the :class:`.EngineEvents` suite, and now participates in the connection pool "pre ping" event for those dialects that make use of disconnect codes in order to detect if the database is live. This allows end-user code to alter the state of "pre ping". Note that this does not include dialects which contain a native "ping" method such as that of psycopg2 or most MySQL dialects. Fixes: #5648 Change-Id: I353d84a4f66f309d2467b7e67621db6b8c70411e
* move bindparam quote application from compiler to defaultMike Bayer2022-05-291-7/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in 296c84313ab29bf9599634f3 for #5653 we generalized Oracle's parameter escaping feature into the compiler, so that it could also work for PostgreSQL. The compiler used quoted names within parameter dictionaries, which then led to the complexity that all functions which interpreted keys from the compiled_params dict had to also quote the param names to use the dictionary. This extra complexity was not added to the ORM peristence.py however, which led to the versioning id feature being broken as well as other areas where persistence.py relies on naming schemes present in context.compiled_params. It also was not added to the "processors" lookup which led to #8053, that added this escaping to that part of the compiler. To both solve the whole problem as well as simplify the compiler quite a bit, move the actual application of the escaped names to be as late as possible, when default.py builds the final list of parameters. This is more similar to how it worked previously where OracleExecutionContext would be late-applying these escaped names. This re-establishes context.compiled_params as deterministically named regardless of dialect in use and moves out the complexity of the quoted param names to be only at the cursor.execute stage. Fixed bug, likely a regression from 1.3, where usage of column names that require bound parameter escaping, more concretely when using Oracle with column names that require quoting such as those that start with an underscore, or in less common cases with some PostgreSQL drivers when using column names that contain percent signs, would cause the ORM versioning feature to not work correctly if the versioning column itself had such a name, as the ORM assumes certain bound parameter naming conventions that were being interfered with via the quotes. This issue is related to :ticket:`8053` and essentially revises the approach towards fixing this, revising the original issue :ticket:`5653` that created the initial implementation for generalized bound-parameter name quoting. Fixes: #8056 Change-Id: I57b064e8f0d070e328b65789c30076f6a0ca0fef
* fix most sphinx warnings (1.4)Mike Bayer2022-05-162-10/+16
| | | | | | | | | | | | | | | still can't figure out the warnings with some of the older changelog files. this cherry-picks the sphinx fixes from 1.4 and additionally fixes a small number of new issues in the 2.0 docs. However, 2.0 has many more errors to fix, primarily from the removal of the legacy tutorials left behind a lot of labels that need to be re-linked to the new tutorial. Fixes: #7946 Change-Id: Id657ab23008eed0b133fed65b2f9ea75a626215c (cherry picked from commit 9b55a423459236ca8a2ced713c9e93999dd18922)
* revenge of pep 484Mike Bayer2022-05-153-35/+113
| | | | | | trying to get remaining must-haves for ORM Change-Id: I66a3ecbbb8e5ba37c818c8a92737b576ecf012f7
* adjust log stacklevel for py3.11.0b1; enable greenletMike Bayer2022-05-151-4/+4
| | | | | | | | | | | | | | | | Fixed issue where support for logging "stacklevel" implemented in :ticket:`7612` required adjustment to work with recently released Python 3.11.0b1, also repairs the unit tests which tested this feature. Install greenlet from a py311 compat patch. re: the stacklevel thing, this is going to be very inconvenient if we have to keep hardcoding numbers everywhere for every new python version Change-Id: I0c8f7293e98c0ca5cc544538284bfd1d3020cb1f References: https://github.com/python-greenlet/greenlet/issues/288 Fixes: #8019
* update for flake8-future-imports 0.0.5Mike Bayer2022-05-145-12/+17
| | | | | | | | a whole bunch of errors were apparently blocked by 0.0.4 being installed. Fixes: #8020 Change-Id: I22a0faeaabe03de501897893391946d677c2df7e
* explicitly fetch inserted pk for values(pkcol=None)Mike Bayer2022-05-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | Altered the compilation mechanics of the :class:`.Insert` construct such that the "autoincrement primary key" column value will be fetched via ``cursor.lastrowid`` or RETURNING even if present in the parameter set or within the :meth:`.Insert.values` method as a plain bound value, for single-row INSERT statements on specific backends that are known to generate autoincrementing values even when explicit NULL is passed. This restores a behavior that was in the 1.3 series for both the use case of separate parameter set as well as :meth:`.Insert.values`. In 1.4, the parameter set behavior unintentionally changed to no longer do this, but the :meth:`.Insert.values` method would still fetch autoincrement values up until 1.4.21 where :ticket:`6770` changed the behavior yet again again unintentionally as this use case was never covered. The behavior is now defined as "working" to suit the case where databases such as SQLite, MySQL and MariaDB will ignore an explicit NULL primary key value and nonetheless invoke an autoincrement generator. Fixes: #7998 Change-Id: I5d4105a14217945f87fbe9a6f2a3c87f6ef20529
* run zimports to match pref changesMike Bayer2022-05-061-2/+0
| | | | | | | I've turned "remove unused imports" back on so this affects some not-used imports in TYPE_CHECKING blocks Change-Id: I8b64ff4ec63f4cee01c2bf41399b691e1c3fb04a
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-282-0/+2
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* pep484 ORM / SQL result supportMike Bayer2022-04-278-106/+500
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | after some experimentation it seems mypy is more amenable to the generic types being fully integrated rather than having separate spin-off types. so key structures like Result, Row, Select become generic. For DML Insert, Update, Delete, these are spun into type-specific subclasses ReturningInsert, ReturningUpdate, ReturningDelete, which is fine since the "row-ness" of these constructs doesn't happen until returning() is called in any case. a Tuple based model is then integrated so that these objects can carry along information about their return types. Overloads at the .execute() level carry through the Tuple from the invoked object to the result. To suit the issue of AliasedClass generating attributes that are dynamic, experimented with a custom subclass AsAliased, but then just settled on having aliased() lie to the type checker and return `Type[_O]`, essentially. will need some type-related accessors for with_polymorphic() also. Additionally, identified an issue in Update when used "mysql style" against a join(), it basically doesn't work if asked to UPDATE two tables on the same column name. added an error message to the specific condition where it happens with a very non-specific error message that we hit a thing we can't do right now, suggest multi-table update as a possible cause. Change-Id: I5eff7eefe1d6166ee74160b2785c5e6a81fa8b95
* fix result.columns() methodMike Bayer2022-04-211-4/+3
| | | | | | | | | | | | Fixed issue in :meth:`.Result.columns` method where calling upon :meth:`.Result.columns` with a single index could in some cases, particularly ORM result object cases, cause the :class:`.Result` to yield scalar objects rather than :class:`.Row` objects, as though the :meth:`.Result.scalars` method had been called. In SQLAlchemy 1.4, this scenario emits a warning that the behavior will change in SQLAlchemy 2.0. Fixes: #7953 Change-Id: I3c4ca3eecc2bfc85ad1c38000e5990d6dde80d22
* pep-484: ORM public API, constructorsMike Bayer2022-04-202-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for the moment, abandoning using @overload with relationship() and mapped_column(). The overloads are very difficult to get working at all, and the overloads that were there all wouldn't pass on mypy. various techniques of getting them to "work", meaning having right hand side dictate what's legal on the left, have mixed success and wont give consistent results; additionally, it's legal to have Optional / non-optional independent of nullable in any case for columns. relationship cases are less ambiguous but mypy was not going along with things. we have a comprehensive system of allowing left side annotations to drive the right side, in the absense of explicit settings on the right. so type-centric SQLAlchemy will be left-side driven just like dataclasses, and the various flags and switches on the right side will just not be needed very much. in other matters, one surprise, forgot to remove string support from orm.join(A, B, "somename") or do deprecations for it in 1.4. This is a really not-directly-used structure barely mentioned in the docs for many years, the example shows a relationship being used, not a string, so we will just change it to raise the usual error here. Change-Id: Iefbbb8d34548b538023890ab8b7c9a5d9496ec6e
* pep484: schema APIMike Bayer2022-04-153-11/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | implement strict typing for schema.py this module has lots of public API, lots of old decisions and very hard to follow construction sequences in many cases, and is also where we get a lot of new feature requests, so strict typing should help keep things clean. among improvements here, fixed the pool .info getters and also figured out how to get ColumnCollection and related to be covariant so that we may set them up as returning Column or ColumnClause without any conflicts. DDL was affected, noting that superclasses of DDLElement (_DDLCompiles, added recently) can now be passed into "ddl_if" callables; reorganized ddl into ExecutableDDLElement as a new name for DDLElement and _DDLCompiles renamed to BaseDDLElement. setting up strict also located an API use case that is completely broken, which is connection.execute(some_default) returns a scalar value. This case has been deprecated and new paths have been set up so that connection.scalar() may be used. This likely wasn't possible in previous versions because scalar() would assume a CursorResult. The scalar() change also impacts Session as we have explicit support (since someone had reported it as a regression) for session.execute(Sequence()) to work. They will get the same deprecation message (which omits the word "Connection", just uses ".execute()" and ".scalar()") and they can then use Session.scalar() as well. Getting this to type correctly while still supporting ORM use cases required some refactoring, and I also set up a keyword only delimeter for Session.execute() and related as execution_options / bind_arguments should always be keyword only, applied these changes to AsyncSession as well. Additionally simpify Table __init__ now that we are Python 3 only, we can have positional plus explicit kwargs finally. Simplify Column.__init__ as well again taking advantage of kw only arguments. Fill in most/all __init__ methods in sqltypes.py as the constructor for types is most of the API. should likely do this for dialect-specific types as well. Apply _InfoType for all info attributes as should have been done originally and update descriptor decorators. Change-Id: I3f9f8ff3f1c8858471ff4545ac83d68c88107527
* pep-484: asyncioMike Bayer2022-04-114-18/+35
| | | | | | | | | | | | | | | | | | | | | in this patch the asyncio/events.py module, which existed only to raise errors when trying to attach event listeners, is removed, as we were already coding an asyncio-specific workaround in upstream Pool / Session to raise this error, just moved the error out to the target and did the same thing for Engine. We also add an async_sessionmaker class. The initial rationale here is because sessionmaker() is hardcoded to Session subclasses, and there's not a way to get the use case of sessionmaker(class_=AsyncSession) to type correctly without changing the sessionmaker() symbol itself to be a function and not a class, which gets too complicated for what this is. Additionally, _SessionClassMethods has only three methods on it, one of which is not usable with asyncio (close_all()), the others not generally used from the session class. Change-Id: I064a5fa5d91cc8d5bbe9597437536e37b4e801fe
* pep-484: session, instancestate, etcMike Bayer2022-04-122-9/+31
| | | | | | | | Also adds some fixes to annotation-based mapping that have come up, as well as starts to add more pep-484 test cases Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
* use code generation for scoped_sessionMike Bayer2022-04-121-2/+6
| | | | | | | | | | | | | | | | | | | our decorator thing generates code in any case, so point it at the file itself to generate real code for the blocks rather than doing things dynamically. this will allow typing tools to have no problem whatsoever and we also reduce import time overhead. file size will be a lot bigger though, shrugs. syntax / dupe method / etc. checking will be accomplished by our existing linting / typing / formatting tools. As we are also using "from __future__ import annotations", we also no longer have to apply quotes to generated annotations. Change-Id: I20962cb65bda63ff0fb67357ab346e9b1ef4f108
* level up pep-484 on engine resultMike Bayer2022-04-122-129/+186
| | | | | | | | | | Have each result subclass be generic to the kind of row/object it returns. rework things so that a significant number of "type ignores" can go away and also allow asyncio to more cleanly proxy the result objects. Change-Id: Ia3ddb2cb52f5856839bd8c9c46f0289ab4b10405
* update flake8 noqa skips with proper syntaxFederico Caselli2022-04-113-4/+4
| | | | Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
* implement iso date literals for all backendsMike Bayer2022-04-081-28/+2
| | | | | | | | | | | | Added modified ISO-8601 rendering (i.e. ISO-8601 with the T converted to a space) when using ``literal_binds`` with the SQL compilers provided by the PostgreSQL, MySQL, MariaDB, MSSQL, Oracle dialects. For Oracle, the ISO format is wrapped inside of an appropriate TO_DATE() function call. Previously this rendering was not implemented for dialect-specific compilation. Fixes: #5052 Change-Id: I7af15a51fedf5c5a8e76e645f7c3be997ece35f0
* cx_Oracle modernizeMike Bayer2022-04-071-8/+6
| | | | | | | | | | | | | | | | | | | Full "RETURNING" support is implemented for the cx_Oracle dialect, meaning multiple RETURNING rows are now recived for DML statements that produce more than one row for RETURNING. cx_Oracle 7 is now the minimum version for cx_Oracle. Getting Oracle to do multirow returning took about 5 minutes. however, getting Oracle's RETURNING system to integrate with ORM-enabled insert, update, delete, is a big deal because that architecture wasn't really working very robustly, including some recent changes in 1.4 for FromStatement were done in a hurry, so this patch also cleans up the FromStatement situation and begins to establish it more concretely as the base for all ReturnsRows / TextClause ORM scenarios. Fixes: #6245 Change-Id: I2b4e6007affa51ce311d2d5baa3917f356ab961f
* use .fromisoformat() for sqlite datetime, date, time parsingMike Bayer2022-04-031-9/+26
| | | | | | | | | | | | SQLite datetime, date, and time datatypes now use Python standard lib ``fromisoformat()`` methods in order to parse incoming datetime, date, and time string values. This improves performance vs. the previous regular expression-based approach, and also automatically accommodates for datetime and time formats that contain either a six-digit "microseconds" format or a three-digit "milliseconds" format. Fixes: #7029 Change-Id: I67aab4fe5ee3055e5996050cf4564981413cc221
* add close=False parameter to engine.dispose()Mike Bayer2022-03-311-17/+30
| | | | | | | | | | | Added new parameter :paramref:`.Engine.dispose.close`, defaulting to True. When False, the engine disposal does not touch the connections in the old pool at all, simply dropping the pool and replacing it. This use case is so that when the original pool is transferred from a parent process, the parent process may continue to use those connections. Fixes: #7877 Change-Id: I88b0808442381ba5e50674787cdb64f0e77d8b54
View Lorry Controller Status