summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/compiler.py
Commit message (Collapse)AuthorAgeFilesLines
* repair broken truediv test suite; memusageMike Bayer2022-01-201-1/+7
| | | | | | | | | | | | | the truediv test suite didn't have __backend__ so wasn't running for every DB except in the main build. Repaired this as well as truediv support to preserve the right-hand side type when casting to numeric, if the right type is already a numeric type. also fixed a memusage test that relies on savepoints so was not running under gerrit runs. Change-Id: I3be223fdf697af9c1ed61b70d621f57cbbb7a92b
* track item schema names to identify name collisions w/ default schemaMike Bayer2022-01-141-0/+28
| | | | | | | | | | | | | | | | | | Added an additional lookup step to the compiler which will track all FROM clauses which are tables, that may have the same name shared in multiple schemas where one of the schemas is the implicit "default" schema; in this case, the table name when referring to that name without a schema qualification will be rendered with an anonymous alias name at the compiler level in order to disambiguate the two (or more) names. The approach of schema-qualifying the normally unqualified name with the server-detected "default schema name" value was also considered, however this approach doesn't apply to Oracle nor is it accepted by SQL Server, nor would it work with multiple entries in the PostgreSQL search path. The name collision issue resolved here has been identified as affecting at least Oracle, PostgreSQL, SQL Server, MySQL and MariaDB. Fixes: #7471 Change-Id: Id65e7ca8c43fe8d95777084e8d5ec140ebcd784d
* remove internal use of metaclassesMike Bayer2022-01-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All but one metaclass used internally can now be replaced using __init_subclass__(). Within this patch we remove: * events._EventMeta * sql.visitors.TraversibleType * sql.visitors.InternalTraversibleType * testing.fixtures.FindFixture * testing.fixtures.FindFixtureDeclarative * langhelpers.EnsureKWArgType * sql.functions._GenericMeta * sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType and EnsureKWArgType) The remaining internal class is MetaOptions used by the sql.Options object which is in turn currently mostly for ORM internal use, as this type implements class level overrides for the ``+`` operator. For declarative, removing DeclarativeMeta in place of an `__init_subclass__()` class would not be fully feasible as it would break backwards compatibility with applications that refer to this class explicitly, but also DeclarativeMeta intercepts class-level attribute set and delete operations which is a widely used pattern. An option for declarative base to use `__init_subclass__()` should be provided but this is out of scope for this particular change. Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78 References: #6810
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Update Black's target-version to py37Hugo van Kemenade2022-01-051-23/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* replace Variant with direct feature inside of TypeEngineMike Bayer2021-12-291-0/+5
| | | | | | | | | | | | | The :meth:`_sqltypes.TypeEngine.with_variant` method now returns a copy of the original :class:`_sqltypes.TypeEngine` object, rather than wrapping it inside the ``Variant`` class, which is effectively removed (the import symbol remains for backwards compatibility with code that may be testing for this symbol). While the previous approach maintained in-Python behaviors, maintaining the original type allows for clearer type checking and debugging. Fixes: #6980 Change-Id: I158c7e56306b886b5b82b040205c428a5c4a242c
* Replace raise_ with raise fromFederico Caselli2021-12-271-30/+12
| | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* consider truediv as truediv; support floordiv operatorMike Bayer2021-12-261-1/+35
| | | | | | | | | | | | | | | | | | | | | Implemented full support for "truediv" and "floordiv" using the "/" and "//" operators. A "truediv" operation between two expressions using :class:`_types.Integer` now considers the result to be :class:`_types.Numeric`, and the dialect-level compilation will cast the right operand to a numeric type on a dialect-specific basis to ensure truediv is achieved. For floordiv, conversion is also added for those databases that don't already do floordiv by default (MySQL, Oracle) and the ``FLOOR()`` function is rendered in this case, as well as for cases where the right operand is not an integer (needed for PostgreSQL, others). The change resolves issues both with inconsistent behavior of the division operator on different backends and also fixes an issue where integer division on Oracle would fail to be able to fetch a result due to inappropriate outputtypehandlers. Fixes: #4926 Change-Id: Id54cc018c1fb7a49dd3ce1216d68d40f43fe2659
* Merge "propose emulated setinputsizes embedded in the compiler" into mainmike bayer2021-11-251-53/+62
|\
| * propose emulated setinputsizes embedded in the compilerMike Bayer2021-11-231-53/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new system so that PostgreSQL and other dialects have a reliable way to add casts to bound parameters in SQL statements, replacing previous use of setinputsizes() for PG dialects. rationale: 1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes" as asyncpg, so we will be seeing a lot more of this 2. the full rendering that SQLAlchemy's compilation is performing is in the engine log as well as error messages. Without this, we introduce three levels of SQL rendering, the compiler, the hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI driver does. With this new approach, users reporting bugs etc. will be less confused that there are as many as two separate layers of "hidden rendering"; SQLAlchemy's rendering is again fully transparent 3. calling upon a setinputsizes() method for every statement execution is expensive. this way, the work is done behind the caching layer 4. for "fast insertmany()", I also want there to be a fast approach towards setinputsizes. As it was, we were going to be taking a SQL INSERT with thousands of bound parameter placeholders and running a whole second pass on it to apply typecasts. this way, we will at least be able to build the SQL string once without a huge second pass over the whole string 5. psycopg2 can use this same system for its ARRAY casts 6. the general need for PostgreSQL to have lots of type casts is now mostly in the base PostgreSQL dialect and works independently of a DBAPI being present. dependence on DBAPI symbols that aren't complete / consistent / hashable is removed I was originally going to try to build this into bind_expression(), but it was revealed this worked poorly with custom bind_expression() as well as empty sets. the current impl also doesn't need to run a second expression pass over the POSTCOMPILE sections, which came out better than I originally thought it would. Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
* | Clean up most py3k compatFederico Caselli2021-11-241-15/+12
|/ | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Support lightweight compiler column elements w/ slotsMike Bayer2021-11-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the _CompileLabel class included ``__slots__`` but these weren't used as the superclasses included slots. Create a ``__slots__`` superclass for ``ClauseElement``, creating a new class of compilable SQL elements that don't include heavier features like caching, annotations and cloning, which are meant to be used only in an ad-hoc compiler fashion. Create new ``CompilerColumnElement`` from that which serves in column-oriented contexts, but similarly does not include any expression operator support as it is intended to be used only to generate a string. Apply this to both ``_CompileLabel`` as well as PostgreSQL ``_ColonCast``, which does not actually subclass ``ColumnElement`` as this class has memoized attributes that aren't worth changing, and does not include SQL operator capabilities as these are not needed for these compiler-only objects. this allows us to more inexpensively add new ad-hoc labels / casts etc. at compile time, as we will be seeking to expand out the typecasts that are needed for PostgreSQL dialects in a subsequent patch. Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
* Remove object in class definitionFederico Caselli2021-11-221-2/+2
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* Merge "set within_columns_clause=False for all sub-elements of select()" ↵mike bayer2021-11-091-0/+3
|\ | | | | | | into main
| * set within_columns_clause=False for all sub-elements of select()Mike Bayer2021-11-091-0/+3
| | | | | | | | | | | | | | | | | | | | | | Fixed issue where using the feature of using a string label for ordering or grouping described at :ref:`tutorial_order_by_label` would fail to function correctly if used on a :class:`.CTE` construct, when the CTE were embedded inside of an enclosing :class:`_sql.Select` statement that itself was set up as a scalar subquery. Fixes: #7269 Change-Id: Ied6048a1c9a622374a418230c8cfedafa8d3f87e
* | Merge "change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql ↵mike bayer2021-11-091-6/+6
|\ \ | |/ |/| | | quoting" into main
| * change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql quotingMike Bayer2021-11-091-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adjusted the compiler's generation of "post compile" symbols including those used for "expanding IN" as well as for the "schema translate map" to not be based directly on plain bracketed strings with underscores, as this conflicts directly with SQL Server's quoting format of also using brackets, which produces false matches when the compiler replaces "post compile" and "schema translate" symbols. The issue created easy to reproduce examples both with the :meth:`.Inspector.get_schema_names` method when used in conjunction with the :paramref:`_engine.Connection.execution_options.schema_translate_map` feature, as well in the unlikely case that a symbol overlapping with the internal name "POSTCOMPILE" would be used with a feature like "expanding in". Fixes: #7300 Change-Id: I6255c850b140522a4aba95085216d0bca18ce230
* | revert mis-commit from aa026c302c6b188a7e28508f9ecb603809b9e03fMike Bayer2021-11-091-2/+0
|/ | | | | | | A scratch line from #7269 was inadvertently committed here. this needs to be in its own commit w/ tests. Change-Id: I62796e18b05bbbd3b6225e9f27e1e63ab98cf24c
* Merge "fully implement future engine and remove legacy" into mainmike bayer2021-11-071-2/+2
|\
| * fully implement future engine and remove legacyMike Bayer2021-11-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* | use tuple expansion if type._is_tuple, test for Sequence if no typeMike Bayer2021-11-051-6/+15
|/ | | | | | | | | | | | | | | | | | | Fixed regression where the row objects returned for ORM queries, which are now the normal :class:`_sql.Row` objects, would not be interpreted by the :meth:`_sql.ColumnOperators.in_` operator as tuple values to be broken out into individual bound parameters, and would instead pass them as single values to the driver leading to failures. The change to the "expanding IN" system now accommodates for the expression already being of type :class:`.TupleType` and treats values accordingly if so. In the uncommon case of using "tuple-in" with an untyped statement such as a textual statement with no typing information, a tuple value is detected for values that implement ``collections.abc.Sequence``, but that are not ``str`` or ``bytes``, as always when testing for ``Sequence``. Added :class:`.TupleType` to the top level ``sqlalchemy`` import namespace. Fixes: #7292 Change-Id: I8286387e3b3c3752b3bd4ae3560d4f31172acc22
* First round of removal of python 2Federico Caselli2021-11-011-1/+0
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* Revise "literal parameters" FAQ sectionMike Bayer2021-11-011-1/+23
| | | | | | | | | | based on feedback in #7271, the emphasis on TypeDecorator as a solution to this problem is not very practical. illustrate a series of quick recipes that are useful for debugging purposes to print out a repr() or simple stringify of a parameter without the need to construct custom dialects or types. Change-Id: I788ce1b5ea01d88dd0a22d03d06f35aabff5e5c8
* 2.0 removals: LegacyRow, connectionless execution, close_with_resultMike Bayer2021-10-311-0/+2
| | | | | | | | | | | | | | | | | | | in order to remove LegacyRow / LegacyResult, we have to also lose close_with_result, which connectionless execution relies upon. also includes a new profiles.txt file that's all against py310, as that's what CI is on now. some result counts changed by one function call which was enough to fail the low-count result tests. Replaces Connectable as the common interface between Connection and Engine with EngineEventsTarget. Engine is no longer Connectable. Connection and MockConnection still are. References: #7257 Change-Id: Iad5eba0313836d347e65490349a22b061356896a
* support bind expressions w/ expanding IN; apply to psycopg2Mike Bayer2021-10-151-30/+76
| | | | | | | | | | | | | | | | | | | | | | | Fixed issue where "expanding IN" would fail to function correctly with datatypes that use the :meth:`_types.TypeEngine.bind_expression` method, where the method would need to be applied to each element of the IN expression rather than the overall IN expression itself. Fixed issue where IN expressions against a series of array elements, as can be done with PostgreSQL, would fail to function correctly due to multiple issues within the "expanding IN" feature of SQLAlchemy Core that was standardized in version 1.4. The psycopg2 dialect now makes use of the :meth:`_types.TypeEngine.bind_expression` method with :class:`_types.ARRAY` to portably apply the correct casts to elements. The asyncpg dialect was not affected by this issue as it applies bind-level casts at the driver level rather than at the compiler level. as part of this commit the "bind translate" feature has been simplified and also applies to the names in the POSTCOMPILE tag to accommodate for brackets. Fixes: #7177 Change-Id: I08c703adb0a9bd6f5aeee5de3ff6f03cccdccdc5
* Fix recursive CTE to support nestingEric Masseran2021-10-121-34/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | Repaired issue in new :paramref:`_sql.HasCTE.cte.nesting` parameter introduced with :ticket:`4123` where a recursive :class:`_sql.CTE` using :paramref:`_sql.HasCTE.cte.recursive` in typical conjunction with UNION would not compile correctly. Additionally makes some adjustments so that the :class:`_sql.CTE` construct creates a correct cache key. Pull request courtesy Eric Masseran. Fixes: #4123 > This has not been caught by the tests because the nesting recursive queries there did not union against itself, eg there was only the i root clause... - Now tests are real recursive queries - Add tests on aliased nested CTEs (recursive or not) - Adapt the `_restates` attribute to use it as a reference - Add some docs around to explain some variables usage Closes: #7133 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7133 Pull-request-sha: 2633f34f7f5336a4a85bd3f71d07bca33ce27a2c Change-Id: I15512c94e1bc1f52afc619d82057ca647d274e92
* Implement nesting CTEEric Masseran2021-09-171-24/+105
| | | | | | | | | | | | | | | | | | Added new parameter :meth:`_sql.HasCte.cte.nesting` to the :class:`_sql.CTE` constructor and :meth:`_sql.HasCTE.cte` method, which flags the CTE as one which should remain nested within an enclosing CTE, rather than being moved to the top level of the outermost SELECT. While in the vast majority of cases there is no difference in SQL functionality, users have identified various edge-cases where true nesting of CTE constructs is desirable. Much thanks to Eric Masseran for lots of work on this intricate feature. Fixes: #4123 Closes: #6709 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6709 Pull-request-sha: 64ab2f6ea269f2dcf37376a13ea38c48c5226fb6 Change-Id: Ic4dc25ab763af96d96632369e01527d48a654149
* use the stack for insert_from_selectMike Bayer2021-09-161-1/+4
| | | | | | | | | Fixed issue related to new ``add_cte()`` feature where pairing two "INSERT..FROM SELECT" statements simultaneously would lose track of the two independent SELECT statements, leading to the wrong SQL. Fixes: #7036 Change-Id: I90fe47eb203bc5c1ea5810db0edba08250b2b7e6
* Fix various lib / test / examples typos (#7017)Kevin Kirsche2021-09-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fix: lib/sqlalchemy/sql/lambdas.py * fix: lib/sqlalchemy/sql/compiler.py * fix: lib/sqlalchemy/sql/selectable.py * fix: lib/sqlalchemy/orm/relationships.py * fix: lib/sqlalchemy/dialects/mssql/base.py * fix: lib/sql/test_compiler.py * fix: lib/sqlalchemy/testing/requirements.py * fix: lib/sqlalchemy/orm/path_registry.py * fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py * fix: lib/sqlalchemy/cextension/immutabledict.c * fix: lib/sqlalchemy/cextension/resultproxy.c * fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py * fix: examples/versioned_rows/versioned_rows_w_versionid.py * fix: examples/elementtree/optimized_al.py * fix: test/orm/test_attribute.py * fix: test/sql/test_compare.py * fix: test/sql/test_type_expression.py * fix: capitalization in test/dialect/mysql/test_compiler.py * fix: typos in test/dialect/postgresql/test_reflection.py * fix: typo in tox.ini comment * fix: typo in /lib/sqlalchemy/orm/decl_api.py * fix: typo in test/orm/test_update_delete.py * fix: self-induced typo * fix: typo in test/orm/test_query.py * fix: typos in test/dialect/mssql/test_types.py * fix: typo in test/sql/test_types.py
* fix linter JOIN logic; fix PostgreSQL ARRAY op comparisonMike Bayer2021-08-151-1/+5
| | | | | | | | | | | | | | | | | | | Adjusted the "from linter" warning feature to accommodate for a chain of joins more than one level deep where the ON clauses don't explicitly match up the targets, such as an expression such as "ON TRUE". This mode of use is intended to cancel the cartesian product warning simply by the fact that there's a JOIN from "a to b", which was not working for the case where the chain of joins had more than one element. this incurs a bit more compiler overhead that comes out in profiling but is not extensive. Added the "is_comparison" flag to the PostgreSQL "overlaps", "contained_by", "contains" operators, so that they work in relevant ORM contexts as well as in conjunction with the "from linter" feature. Fixes: #6886 Change-Id: I078dc3fe6d4f7871ffe4ebac3e71e62f3f213d12
* Merge "Dispatch independent ctes on compound select"mike bayer2021-08-081-0/+6
|\
| * Dispatch independent ctes on compound selectEric Masseran2021-08-081-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix issue in :class:`_sql.CTE` where new :meth:`_sql.HasCTE.add_cte` method added in version 1.4.21 / :ticket:`6752` failed to function correctly for "compound select" structures such as :func:`_sql.union`, :func:`_sql.union_all`, :func:`_sql.except`, etc. Pull request courtesy Eric Masseran. Fixes: #6752 Closes: #6849 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6849 Pull-request-sha: 1c4b4d72b2789cf89ff5043ca964ebdd6c9a6617 Change-Id: I49a16a4fc2af8299502011f3a02d8a2ad93255e3
* | dont qualify literal_binds with literal_executeMike Bayer2021-08-071-1/+1
|/ | | | | | | | | | | | | | | this appears to be unnecessary and prevents end-user literal_binds case from working. Fixed issue where the ``literal_binds`` compiler flag, as used externally to render bound parameters inline, would fail to work when used with a certain class of parameters known as "literal_execute", which covers things like LIMIT and OFFSET values for dialects where the drivers don't allow a bound parameter, such as SQL Server's "TOP" clause. The issue locally seemed to affect only the MSSQL dialect. Fixes: #6863 Change-Id: Ia74cff5b0107b129a11b9b965883552b2962e449
* accommodate for cloned bindparams w/ maintain_keyMike Bayer2021-07-291-2/+8
| | | | | | | | | | | Fixed issue where a bound parameter object that was "cloned" would cause a name conflict in the compiler, if more than one clone of this parameter were used at the same time in a single statement. This could occur in particular with things like ORM single table inheritance queries that indicated the same "discriminator" value multiple times in one query. Fixes: #6824 Change-Id: Iba7a786fc5a2341ff7d07fc666d24ed790ad4fe8
* Ensure alias traversal block works when adapt_from_selectables presentMike Bayer2021-07-141-1/+0
| | | | | | | | | | | | Fixed regression which appeared in version 1.4.3 due to :ticket:`6060` where rules that limit ORM adaptation of derived selectables interfered with other ORM-adaptation based cases, in this case when applying adaptations for a :func:`_orm.with_polymorphic` against a mapping which uses a :func:`_orm.column_property` which in turn makes use of a scalar select that includes a :func:`_orm.aliased` object of the mapped table. Fixes: #6762 Change-Id: Ice3dc34b97d12b59f044bdc0c5faaefcc4015227
* Merge "Adjust CTE recrusive col list to accommodate dupe col names"mike bayer2021-07-131-6/+35
|\
| * Adjust CTE recrusive col list to accommodate dupe col namesMike Bayer2021-07-131-6/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue in CTE constructs where a recursive CTE that referred to a SELECT that has duplicate column names, which are typically deduplicated using labeling logic in 1.4, would fail to refer to the deduplicated label name correctly within the WITH clause. As part of this change we are also attempting to remove the behavior of SelectStatementGrouping forcing off the "asfrom" contextual flag, which will have the result of additional labeling being applied to some UNION and similar statements when they are interpreted as subqueries. To maintain compatibility with "grouping", the Grouping/SelectStatementGrouping are now broken out into two separate compiler cases, as the "asfrom" logic appears to be tailored towards table valued SELECTS as column expressions. Fixes: #6710 Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
* | Merge "Extract format_constraint truncation rules to ON CONFLICT"mike bayer2021-07-131-13/+34
|\ \ | |/ |/|
| * Extract format_constraint truncation rules to ON CONFLICTMike Bayer2021-07-131-13/+34
| | | | | | | | | | | | | | | | | | | | | | Fixed issue where a too-long constraint name rendered as part of the "ON CONFLICT ON CONSTRAINT" element of the :class:`_postgresql.Insert` construct due to naming convention generation would not correctly truncate the name in the same way that it normally renders within a CREATE TABLE statement, thus producing a non-matching and too-long constraint name. Fixes: #6755 Change-Id: Ib27014a5ecbc9cd5861a396f8bb49fbc60bf49fe
* | Merge "labeling refactor"mike bayer2021-07-131-48/+128
|\ \
| * | labeling refactorMike Bayer2021-07-121-48/+128
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To service #6718 and #6710, the system by which columns are given labels in a SELECT statement as well as the system that gives them keys in a .c or .selected_columns collection have been refactored to provide a single source of truth for both, in constrast to the previous approach that included similar logic repeated in slightly different ways. Main ideas: 1. ColumnElement attributes ._label, ._anon_label, ._key_label are renamed to include the letters "tq", meaning "table-qualified" - these labels are only used when rendering a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its label style; as this label style is primarily legacy, the "tq" names should be isolated so that in a 2.0 style application these aren't being used at all 2. The means by which the "labels" and "proxy keys" for the elements of a SELECT has been centralized to a single source of truth; previously, the three of _generate_columns_plus_names, _generate_fromclause_column_proxies, and _column_naming_convention all had duplicated rules between them, as well as that there were a little bit of labeling rules in compiler._label_select_column as well; by this we mean that the various "anon_label" "anon_key" methods on ColumnElement were called by all four of these methods, where there were many cases where it was necessary that one method comes up with the same answer as another of the methods. This has all been centralized into _generate_columns_plus_names for all the names except the "proxy key", which is generated by _column_naming_convention. 3. compiler._label_select_column has been rewritten to both not make any naming decisions nor any "proxy key" decisions, only whether to label or not to label; the _generate_columns_plus_names method gives it the information, where the proxy keys come from _column_naming_convention; previously, these proxy keys were matched based on restatement of similar (but not really the same) logic in two places. The heuristics of "whether to label or not to label" are also reorganized to be much easier to read and understand. 4. a new method compiler._label_returning_column is added for dialects to use in their "generate returning columns" methods. A github search reveals a small number of third party dialects also doing this using the prior _label_select_column method so we try to make sure _label_select_column continues to work the exact same way for that specific use case; for the "SELECT" use case it now needs 5. After some attempts to do it different ways, for the case where _proxy_key is giving us some kind of anon label, we are hard changing it to "_no_label" right now, as there's not currently a way to fully match anonymized labels from stmt.c or stmt.selected_columns to what will be in the result map. The idea of "_no_label" is to encourage the user to use label('name') for columns they want to be able to target by string name that don't have a natural name. Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
* | Merge "implement independent CTEs"mike bayer2021-07-121-2/+17
|\ \
| * | implement independent CTEsMike Bayer2021-07-121-2/+17
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | Added new method :meth:`_sql.HasCTE.add_cte` to each of the :func:`_sql.select`, :func:`_sql.insert`, :func:`_sql.update` and :func:`_sql.delete` constructs. This method will add the given :class:`_sql.CTE` as an "independent" CTE of the statement, meaning it renders in the WITH clause above the statement unconditionally even if it is not otherwise referenced in the primary statement. This is a popular use case on the PostgreSQL database where a CTE is used for a DML statement that runs against database rows independently of the primary statement. Fixes: #6752 Change-Id: Ibf635763e40269cbd10f4c17e208850d8e8d0188
* | repair schema_translate_map for schema type use casesMike Bayer2021-07-111-0/+1
|/ | | | | | | | | | | | | Fixed issue where the PostgreSQL ``ENUM`` datatype as embedded in the ``ARRAY`` datatype would fail to emit correctly in create/drop when the ``schema_translate_map`` feature were also in use. Additionally repairs a related issue where the same ``schema_translate_map`` feature would not work for the ``ENUM`` datatype in combination with a ``CAST``, that's also intrinsic to how the ``ARRAY(ENUM)`` combination works on the PostgreSQL dialect. Fixes: #6739 Change-Id: I44b1ad4db4af3acbf639aa422c46c22dd3b0d3a6
* Replace all http:// links to https://Federico Caselli2021-07-041-1/+1
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* Ensure compiler uses quote_schema hook for translates rendersMike Bayer2021-06-301-1/+1
| | | | | | | | | Fixed regression where the special dotted-schema name handling for the SQL Server dialect would not function correctly if the dotted schema name were used within the ``schema_translate_map`` feature. Fixes: #6697 Change-Id: Idb610755cbf8122e71223d5dd0a17fcb61b1b98d
* consider "*" col as textual orderedMike Bayer2021-06-231-1/+1
| | | | | | | | | Fixed old issue where a :func:`_sql.select()` made against the token "*", which then yielded exactly one column, would fail to correctly organize the ``cursor.description`` column name into the keys of the result object. Fixes: #6665 Change-Id: Ie8c00f62998972ad4a19a750d2642d00fde006f6
* Export deferred columns but not col props; fix CTE labelingMike Bayer2021-06-221-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refined the behavior of ORM subquery rendering with regards to deferred columns and column properties to be more compatible with that of 1.3 while also providing for 1.4's newer features. As a subquery in 1.4 does not make use of loader options, including :func:`_orm.deferred`, a subquery that is against an ORM entity with deferred attributes will now render those deferred attributes that refer directly to mapped table columns, as these are needed in the outer SELECT if that outer SELECT makes use of these columns; however a deferred attribute that refers to a composed SQL expression as we normally do with :func:`_orm.column_property` will not be part of the subquery, as these can be selected explicitly if needed in the subquery. If the entity is being SELECTed from this subquery, the column expression can still render on "the outside" in terms of the derived subquery columns. This produces essentially the same behavior as when working with 1.3. However in this case the fix has to also make sure that the ``.selected_columns`` collection of an ORM-enabled :func:`_sql.select` also follows these rules, which in particular allows recursive CTEs to render correctly in this scenario, which were previously failing to render correctly due to this issue. As part of this change the _exported_columns_iterator() method has been removed and logic simplified to use ._all_selected_columns from any SelectBase object where _exported_columns_iterator() was used before. Additionally sets up UpdateBase to include ReturnsRows in its hierarchy; the literal point of ReturnsRows was to be a common base for UpdateBase and SelectBase so it was kind of weird it wasn't there. Fixes: #6661 Fixed issue in CTE constructs mostly relevant to ORM use cases where a recursive CTE against "anonymous" labels such as those seen in ORM ``column_property()`` mappings would render in the ``WITH RECURSIVE xyz(...)`` section as their raw internal label and not a cleanly anonymized name. Fixes: #6663 Change-Id: I26219d4d8e6c0915b641426e9885540f74fae4d2
* don't enable "fast insert executemany" for ON CONFLICT etcMike Bayer2021-06-041-1/+8
| | | | | | | | | | | | | Fixed issue where using the PostgreSQL "INSERT..ON CONFLICT" structure would fail to work with the psycopg2 driver if it were used in an "executemany" context along with bound parameters in the "SET" clause, due to the implicit use of the psycopg2 fast execution helpers which are not appropriate for this style of INSERT statement. Additional checks to exclude this kind of statement from that particular extension have been added. Fixes: #6581 Change-Id: I3d6169e7e188dc087d1d1bfba9a42162db183265
* set bindparam.expanding in coercion againMike Bayer2021-05-101-20/+2
| | | | | | | | | | | | | | | | | Adjusted the logic added as part of :ticket:`6397` in 1.4.12 so that internal mutation of the :class:`.BindParameter` object occurs within the clause construction phase as it did before, rather than in the compilation phase. In the latter case, the mutation still produced side effects against the incoming construct and additionally could potentially interfere with other internal mutation routines. In order to solve the issue of the correct operator being present on the BindParameter.expand_op, we necessarily have to expand the BinaryExpression._negate() routine to flip the operator on the BindParameter also. Fixes: #6460 Change-Id: I1e53a9aeee4de4fc11af51d7593431532731561b
View Lorry Controller Status