summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/dialects
Commit message (Collapse)AuthorAgeFilesLines
...
* | Rewrite positional handling, test for "numeric"Federico Caselli2022-12-053-15/+117
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed how the positional compilation is performed. It's rendered by the compiler the same as the pyformat compilation. The string is then processed to replace the placeholders with the correct ones, and to obtain the correct order of the parameters. This vastly simplifies the computation of the order of the parameters, that in case of nested CTE is very hard to compute correctly. Reworked how numeric paramstyle behavers: - added support for repeated parameter, without duplicating them like in normal positional dialects - implement insertmany support. This requires that the dialect supports out of order placehoders, since all parameters that are not part of the VALUES clauses are placed at the beginning of the parameter tuple - support for different identifiers for a numeric parameter. It's for example possible to use postgresql style placeholder $1, $2, etc Added two new dialect based on sqlite to test "numeric" fully using both :1 style and $1 style. Includes a workaround for SQLite's not-really-correct numeric implementation. Changed parmstyle of asyncpg dialect to use numeric, rendering with its native $ identifiers Fixes: #8926 Fixes: #8849 Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
* add spaces, leading underscore to oracle checksMike Bayer2022-12-021-4/+4
| | | | | | | | | | | | Expand the test suite from #8708 which unfortunately did not exercise the bound parameter codepaths completely. Continued fixes for Oracle fix :ticket:`8708` released in 1.4.43 where bound parameter names that start with underscores, which are disallowed by Oracle, were still not being properly escaped in all circumstances. Fixes: #8708 Change-Id: Ic389c09bd7c53b773e5de35f1a18ef20769b92a7
* Merge "Fix positional compiling bugs" into mainmike bayer2022-12-022-20/+15
|\
| * Fix positional compiling bugsFederico Caselli2022-12-012-20/+15
| | | | | | | | | | | | | | | | | | | | | | Fixed a series of issues regarding positionally rendered bound parameters, such as those used for SQLite, asyncpg, MySQL and others. Some compiled forms would not maintain the order of parameters correctly, such as the PostgreSQL ``regexp_replace()`` function as well as within the "nesting" feature of the :class:`.CTE` construct first introduced in :ticket:`4123`. Fixes: #8827 Change-Id: I9813ed7c358cc5c1e26725c48df546b209a442cb
* | [sqlite] Reflect DEFERRABLE and INITIALLY options for foreign keysMichael Gorven2022-11-301-1/+11
|/ | | | | | | | | | | | | Added support for the SQLite backend to reflect the "DEFERRABLE" and "INITIALLY" keywords which may be present on a foreign key construct. Pull request courtesy Michael Gorven. Fixes: #8903 Closes: #8904 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8904 Pull-request-sha: 52aa4cf77482c4051899e21bea75b9830e4c3efa Change-Id: I713906db1a458d8f1be39625841ca3bbc03ec835
* Merge "Add value-level hooks for SQL type detection; apply to Range" into mainmike bayer2022-11-291-0/+20
|\
| * Add value-level hooks for SQL type detection; apply to RangeLele Gaifax2022-11-291-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added additional type-detection for the new PostgreSQL :class:`_postgresql.Range` type, where previous cases that allowed the psycopg2-native range objects to be received directly by the DBAPI without SQLAlchemy intercepting them stopped working, as we now have our own value object. The :class:`_postgresql.Range` object has been enhanced such that SQLAlchemy Core detects it in otherwise ambiguous situations (such as comparison to dates) and applies appropriate bind handlers. Pull request courtesy Lele Gaifax. Fixes: #8884 Closes: #8886 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8886 Pull-request-sha: 6e95e08a30597d3735ab38f2f1a2ccabd968852c Change-Id: I3ca277c826dcf4b5644f44eb251345b439a84ee4
* | Merge "add partial index predicate to SQLiteDialect.get_indexes() result" ↵mike bayer2022-11-291-1/+49
|\ \ | | | | | | | | | into main
| * | add partial index predicate to SQLiteDialect.get_indexes() resultTobias Pfeiffer2022-11-281-1/+49
| |/ | | | | | | | | | | | | | | | | | | | | | | | | Added support for reflection of expression-oriented WHERE criteria included in indexes on the SQLite dialect, in a manner similar to that of the PostgreSQL dialect. Pull request courtesy Tobias Pfeiffer. Fixes: #8804 Closes: #8806 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8806 Pull-request-sha: 539dfcb372360911b69aed2a804698bb1a2220b1 Change-Id: I0e34d47dbe2b9c1da6fce531363084843e5127a3
* | Rectify PG Range.__bool__, inverting previous logicLele Gaifax2022-11-271-1/+1
|/ | | | | | | | | | The boolness of the range was defined to be equal to its emptiness. As this has been identified as a typo rather than the intended, this inverts the logic, to match common Python behaviour as well as how other popular PG drivers do. Closes: #8885 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8885 Pull-request-sha: 5670cdb920692a62f77b7b6ea312784033de83d9 Change-Id: I6f4a40168b2f037c578e84f7550370411bd42160
* Merge "Issue #8765: implement missing methods on PG Range" into mainmike bayer2022-11-171-94/+403
|\
| * Issue #8765: implement missing methods on PG RangeLele Gaifax2022-11-151-94/+403
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description This PR implements missing methods on the PG `Range` class, as described by issue #8765. ### Checklist This pull request is: - [ ] A documentation / typographical error fix - [ ] A short code fix - [x] A new feature implementation Closes: #8766 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8766 Pull-request-sha: 21c0df86cc0d1502855527e29425fbffc3f45d64 Change-Id: I86fabd966ad1f14a3a86132be741df46965b9aa9
* | Merge "Try running pyupgrade on the code" into mainmike bayer2022-11-1732-663/+574
|\ \
| * | Try running pyupgrade on the codeFederico Caselli2022-11-1632-663/+574
| |/ | | | | | | | | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* | accommodate NULL format_type()Mike Bayer2022-11-161-5/+18
|/ | | | | | | | | Made an adjustment to how the PostgreSQL dialect considers column types when it reflects columns from a table, to accommodate for alternative backends which may return NULL from the PG ``format_type()`` function. Fixes: #8748 Change-Id: I6178287aac567210a76afaa5805b825daa7fa4db
* establish consistency for RETURNING column labelsMike Bayer2022-11-111-3/+16
| | | | | | | | | | | | | | | The RETURNING clause now renders columns using the routine as that of the :class:`.Select` to generate labels, which will include disambiguating labels, as well as that a SQL function surrounding a named column will be labeled using the column name itself. This is a more comprehensive change than a similar one made for the 1.4 series that adjusted the function label issue only. includes 1.4's changelog for the backported version which also fixes an Oracle issue independently of the 2.0 series. Fixes: #8770 Change-Id: I2ab078a214a778ffe1720dbd864ae4c105a0691d
* work around Python 3.11 IntEnum issue; update FastIntFlagMike Bayer2022-11-101-1/+1
| | | | | | | | | | | | | | | | | in [1], Python 3.11 seems to have changed the behavior of IntEnum. We didn't notice this because we have our own workaround class already, but typing did. Ensure we remain compatible with IntFlag. This change also modifies FastIntFlag to no longer use global symbols; this is unnecessary as we assign FastIntFlag members explicitly. Use of ``symbol()`` should probably be phased out. [1] https://github.com/python/cpython/issues/99304 Fixes: #8783 Change-Id: I8ae2e871ff1467ae5ca1f63e66b5dae45d4a6c93
* disable sqlite returning on pypyMike Bayer2022-11-051-1/+1
| | | | | | using latest 3.9-v7.3.9 and returning does not work at all. Change-Id: I208c3e1ff10949651ffbebc54beea6ede6af1dd3
* Merge "Implement contains_value(), issubset() and issuperset() on PG Range" ↵mike bayer2022-11-032-0/+208
|\ | | | | | | into main
| * Implement contains_value(), issubset() and issuperset() on PG RangeLele Gaifax2022-11-032-0/+208
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new methods :meth:`_postgresql.Range.contains` and :meth:`_postgresql.Range.contained_by` to the new :class:`.Range` data object, which mirror the behavior of the PostgreSQL ``@>`` and ``<@`` operators, as well as the :meth:`_postgresql.AbstractRange.comparator_factory.contains` and :meth:`_postgresql.AbstractRange.comparator_factory.contained_by` SQL operator methods. Pull request courtesy Lele Gaifax. Fixes: #8706 Closes: #8707 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8707 Pull-request-sha: 3a74a0d93e63032ebee02992977498c717a077ff Change-Id: Ief81ca5c31448640b26dfbc3defd4dde1d51e366
* | Merge "use simple decimal query to detect decimal char" into mainmike bayer2022-11-021-4/+27
|\ \
| * | use simple decimal query to detect decimal charMike Bayer2022-11-011-4/+27
| |/ | | | | | | | | | | | | | | | | | | | | | | Fixed issue where the ``nls_session_parameters`` view queried on first connect in order to get the default decimal point character may not be available depending on Oracle connection modes, and would therefore raise an error. The approach to detecting decimal char has been simplified to test a decimal value directly, instead of reading system views, which works on any backend / driver. Fixes: #8744 Change-Id: I39825131c13513798863197d0c180dd5a18b32dc
* | fix event nameMike Bayer2022-11-021-1/+1
|/ | | | | Change-Id: I26af2326034be07f0ebc91dfbf31d00c40acf585 References: #8717
* ensure pool.reset event always called for resetMike Bayer2022-10-302-0/+119
| | | | | | | | | | | | | | | | | | | | | | Added new parameter :paramref:`.PoolEvents.reset.reset_state` parameter to the :meth:`.PoolEvents.reset` event, with deprecation logic in place that will continue to accept event hooks using the previous set of arguments. This indicates various state information about how the reset is taking place and is used to allow custom reset schemes to take place with full context given. Within this change a fix that's also backported to 1.4 is included which re-enables the :meth:`.PoolEvents.reset` event to continue to take place under all circumstances, including when :class:`.Connection` has already "reset" the connection. The two changes together allow custom reset schemes to be implemented using the :meth:`.PoolEvents.reset` event, instead of the :meth:`.PoolEvents.checkin` event (which continues to function as it always has). Change-Id: Ie17c4f55d02beb6f570b9de6b3044baffa7d6df6 Fixes: #8717
* use only object_id() function for temp tablesMike Barry2022-10-281-20/+5
| | | | | | | | | | | | | | | | | | Fixed issue with :meth:`.Inspector.has_table` when used against a temporary table for the SQL Server dialect would fail an invalid object name error on some Azure variants, due to an unnecessary information schema query that is not supported on those server versions. Pull request courtesy Mike Barry. the patch also fills out test support for has_table() against temp tables, temp views, adding to the has_table() support just added for views in #8700. Fixes: #8714 Closes: #8716 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8716 Pull-request-sha: e2ac7a52e2b09a349a703ba1e1a2911f4d3c0912 Change-Id: Ia73e4e9e977a2d6b7e100abd2f81a8c8777dc9bb
* mssql doc updatesMike Bayer2022-10-281-16/+17
| | | | | | clarify some URL things Change-Id: Ic162834052f06fd3a6c010ce5d091903fdc65cd8
* add Oracle-specific parameter escapes for expanding paramsMike Bayer2022-10-241-0/+36
| | | | | | | | | | | | | | Fixed issue where bound parameter names, including those automatically derived from similarly-named database columns, which contained characters that normally require quoting with Oracle would not be escaped when using "expanding parameters" with the Oracle dialect, causing execution errors. The usual "quoting" for bound parameters used by the Oracle dialect is not used with the "expanding parameters" architecture, so escaping for a large range of characters is used instead, now using a list of characters/escapes that are specific to Oracle. Fixes: #8708 Change-Id: I90c24e48534e1b3a4c222b3022da58159784d91a
* Only convert Range for sqlalchemy Range objectMike Bayer2022-10-213-23/+19
| | | | | | | | | | Refined the new approach to range objects described at :ref:`change_7156` to accommodate driver-specific range and multirange objects, to better accommodate both legacy code as well as when passing results from raw SQL result sets back into new range or multirange expressions. Fixes: #8690 Change-Id: I7e62c47067f695c6380ad0fe2fe19deaf33594d1
* Merge "further qualify pyodbc setinputsizes types for long stirngs" into mainmike bayer2022-10-191-4/+13
|\
| * further qualify pyodbc setinputsizes types for long stirngsMike Bayer2022-10-181-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by SQL Server pyodbc change :ticket:`8177` where we now use ``setinputsizes()`` by default; for VARCHAR, this fails if the character size is greater than 4000 (or 2000, depending on data) characters as the incoming datatype is NVARCHAR, which has a limit of 4000 characters, despite the fact that VARCHAR can handle unlimited characters. Additional pyodbc-specific typing information is now passed to ``setinputsizes()`` when the datatype's size is > 2000 characters. The change is also applied to the :class:`.JSON` type which was also impacted by this issue for large JSON serializations. Fixes: #8661 Change-Id: I07fa873e95dbd2c94f3d286e93e8b3229c3a9807
* | Revert automatic set of sequence start to 1Federico Caselli2022-10-174-10/+43
|/ | | | | | | | | | | | | | | | | The :class:`.Sequence` construct restores itself to the DDL behavior it had prior to the 1.4 series, where creating a :class:`.Sequence` with no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction **without** any additional parameters for "start value". For most backends, this is how things worked previously in any case; **however**, for MS SQL Server, the default value on this database is ``-2**63``; to prevent this generally impractical default from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter should be provided. As usage of :class:`.Sequence` is unusual for SQL Server which for many years has standardized on ``IDENTITY``, it is hoped that this change has minimal impact. Fixes: #7211 Change-Id: I1207ea10c8cb1528a1519a0fb3581d9621c27b31
* enable check same thread for aiosqliteMike Bayer2022-10-111-13/+18
| | | | | | | | | | to do this we have to invent our own isolation level setter based on their current internals. however now we can ensure thread-safe access. we are trying to resolve an issue where test suite on CI seems to fail around the same time each time. Change-Id: I79c8fc04b9afef0876fb446ad40a7621a772cd34
* adjust MySQL view reflection for non-standard MySQL variantsJohn Bodley2022-10-032-1/+6
| | | | | | | | | | | | | | | | Adjusted the regular expression used to match "CREATE VIEW" when testing for views to work more flexibly, no longer requiring the special keyword "ALGORITHM" in the middle, which was intended to be optional but was not working correctly. The change allows view reflection to work more completely on MySQL-compatible variants such as StarRocks. Pull request courtesy John Bodley. Fixes: #8588 Closes: #8589 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8589 Pull-request-sha: d85b2c5b51e45cec543c9ae9d62d6d659b063354 Change-Id: I173137f0bf68639cad0d5c329055475b40ddb5e4
* Merge "Document user-defined functions for sqlite" into mainmike bayer2022-09-292-0/+41
|\
| * Document user-defined functions for sqliteGord Thompson2022-06-292-0/+41
| | | | | | | | Change-Id: I64e4d4dce8c5f5aced3190f9e3682c630462a61e
* | Merge "`aggregate_order_by` now supports cache generation." into mainmike bayer2022-09-261-1/+11
|\ \
| * | `aggregate_order_by` now supports cache generation.Federico Caselli2022-09-251-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | also adjusted CacheKeyFixture to be a general purpose fixture so that sub-components / dialects can run their own cache key tests. Fixes: #8574 Change-Id: I6c66107856aee11e548d357cea77bceee3e316a0
* | | Merge "New ORM Query Guide featuring DML support" into mainmike bayer2022-09-261-0/+1
|\ \ \
| * | | New ORM Query Guide featuring DML supportMike Bayer2022-09-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | reviewers: these docs publish periodically at: https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html See the "last generated" timestamp near the bottom of the page to ensure the latest version is up Change includes some other adjustments: * small typing fixes for end-user benefit * removal of a bunch of old examples for patterns that nobody uses or aren't really what we promote now * modernization of some examples, including inheritance Change-Id: I9929daab7797be9515f71c888b28af1209e789ff
* | | | Merge "ORM bulk insert via execute" into mainmike bayer2022-09-263-7/+4
|\ \ \ \ | |/ / /
| * | | ORM bulk insert via executeMike Bayer2022-09-243-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * ORM Insert now includes "bulk" mode that will run essentially the same process as session.bulk_insert_mappings; interprets the given list of values as ORM attributes for key names * ORM UPDATE has a similar feature, without RETURNING support, for session.bulk_update_mappings * Added support for upserts to do RETURNING ORM objects as well * ORM UPDATE/DELETE with list of parameters + WHERE criteria is a not implemented; use connection * ORM UPDATE/DELETE defaults to "auto" synchronize_session; use fetch if RETURNING is present, evaluate if not, as "fetch" is much more efficient (no expired object SELECT problem) and less error prone if RETURNING is available UPDATE: howver this is inefficient! please continue to use evaluate for simple cases, auto can move to fetch if criteria not evaluable * "Evaluate" criteria will now not preemptively unexpire and SELECT attributes that were individually expired. Instead, if evaluation of the criteria indicates that the necessary attrs were expired, we expire the object completely (delete) or expire the SET attrs unconditionally (update). This keeps the object in the same unloaded state where it will refresh those attrs on the next pass, for this generally unusual case. (originally #5664) * Core change! update/delete rowcount comes from len(rows) if RETURNING was used. SQLite at least otherwise did not support this. adjusted test_rowcount accordingly * ORM DELETE with a list of parameters at all is also a not implemented as this would imply "bulk", and there is no bulk_delete_mappings (could be, but we dont have that) * ORM insert().values() with single or multi-values translates key names based on ORM attribute names * ORM returning() implemented for insert, update, delete; explcit returning clauses now interpret rows in an ORM context, with support for qualifying loader options as well * session.bulk_insert_mappings() assigns polymorphic identity if not set. * explicit RETURNING + synchronize_session='fetch' is now supported with UPDATE and DELETE. * expanded return_defaults() to work with DELETE also. * added support for composite attributes to be present in the dictionaries used by bulk_insert_mappings and bulk_update_mappings, which is also the new ORM bulk insert/update feature, that will expand the composite values into their individual mapped attributes the way they'd be on a mapped instance. * bulk UPDATE supports "synchronize_session=evaluate", is the default. this does not apply to session.bulk_update_mappings, just the new version * both bulk UPDATE and bulk INSERT, the latter with or without RETURNING, support *heterogenous* parameter sets. session.bulk_insert/update_mappings did this, so this feature is maintained. now cursor result can be both horizontally and vertically spliced :) This is now a long story with a lot of options, which in itself is a problem to be able to document all of this in some way that makes sense. raising exceptions for use cases we haven't supported is pretty important here too, the tradition of letting unsupported things just not work is likely not a good idea at this point, though there are still many cases that aren't easily avoidable Fixes: #8360 Fixes: #7864 Fixes: #7865 Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
* | | | Merge "implement batched INSERT..VALUES () () for executemany" into mainmike bayer2022-09-2611-182/+205
|\ \ \ \ | |/ / /
| * | | implement batched INSERT..VALUES () () for executemanyMike Bayer2022-09-2411-182/+205
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the feature is enabled for all built in backends when RETURNING is used, except for Oracle that doesn't need it, and on psycopg2 and mssql+pyodbc it is used for all INSERT statements, not just those that use RETURNING. third party dialects would need to opt in to the new feature by setting use_insertmanyvalues to True. Also adds dialect-level guards against using returning with executemany where we dont have an implementation to suit it. execute single w/ returning still defers to the server without us checking. Fixes: #6047 Fixes: #7907 Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
* | | Use aggregate order by instead of order by in subqueryFederico Caselli2022-09-251-25/+23
|/ / | | | | | | | | Fixes: #8561 Change-Id: I2d9f6bd895061bf8fbc66723930716670791d896
* | auto-cast PG range typesMike Bayer2022-09-204-5/+44
| | | | | | | | | | | | | | | | | | | | Range type handling has been enhanced so that it automatically renders type casts, so that in-place round trips for statements that don't provide the database with any context don't require the :func:`_sql.cast` construct to be explicit for the database to know the desired type. Change-Id: Id630b726f8a23059dd2f4cbc410bf5229d89cbfb References: #8540
* | Merge "Support GROUP BY ROLLUP on MySql/MariaDB" into mainmike bayer2022-09-161-0/+6
|\ \
| * | Support GROUP BY ROLLUP on MySql/MariaDBFederico Caselli2022-09-091-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | The ``ROLLUP`` function will now correctly render ``WITH ROLLUP`` on MySql and MariaDB, allowing the use of group by rollup with these backend. Fixes: #8503 Change-Id: I9289af3a39ca667a2f0f84f73346ebd4b091fedd
* | | Merge "catch exception for system_views also" into mainmike bayer2022-09-161-26/+28
|\ \ \
| * | | catch exception for system_views alsoMike Bayer2022-09-151-26/+28
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed yet another regression in SQL Server isolation level fetch (see :ticket:`8231`, :ticket:`8475`), this time with "Microsoft Dynamics CRM Database via Azure Active Directory", which apparently lacks the ``system_views`` view entirely. Error catching has been extended that under no circumstances will this method ever fail, provided database connectivity is present. Fixes: #8525 Change-Id: I76a429e3329926069a0367d2e77ca1124b9a059d
* | | Use ``;`` instead of ``select 1`` to ping PostgreSQLFederico Caselli2022-09-153-0/+25
|/ / | | | | | | | | Fixes: #8491 Change-Id: I941d2a3cf92e5609e2045a53cec94522340951db
View Lorry Controller Status