summaryrefslogtreecommitdiff
path: root/test/base
Commit message (Collapse)AuthorAgeFilesLines
* fix test suite warningsMike Bayer2023-05-091-3/+10
| | | | | | | | | | | | | | | | fix a handful of warnings that were emitting but not raising, usually because they were inside an "expect_warnings" block. modify "expect_warnings" to always use "raise_on_any_unexpected" behavior; remove this parameter. Fixed issue in semi-private ``await_only()`` and ``await_fallback()`` concurrency functions where the given awaitable would remain un-awaited if the function threw a ``GreenletError``, which could cause "was not awaited" warnings later on if the program continued. In this case, the given awaitable is now cancelled before the exception is thrown. Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
* Performance improvement in RowFederico Caselli2023-04-261-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Various performance improvements to Row instanciation - avoid passing processors if they are all None - improve processor logic in cython - improve tuplegetter using slices when contiguous indexes are used Some timing follow. In particular [base_]row_new_proc that tests using processors has a 25% improvement compared to before in cython. Looking at the [b]row_new_proc_none that test a list of processors all None, this has 50% improvement in cython when passing the none list, but in this patch it would usually be disabled by passing None, so the performance gain is actually 90%, since it would run the case [base_]row_new. Tuplegetter is a bit faster in the single item get and when getting sequential indexes (like indexes 1,2,3,4) at the cost of a bit longer creation time in python, cython is mostly the same. Current times | python | cython | cy / py | base_row_new | 0.639817400 | 0.118265500 | 0.184842582 | row_new | 0.680355100 | 0.129714600 | 0.190657202 | base_row_new_proc | 3.076538900 | 1.488428600 | 0.483799701 | row_new_proc | 3.119700100 | 1.532197500 | 0.491136151 | brow_new_proc_none | 1.917702300 | 0.475511500 | 0.247958977 | row_new_proc_none | 1.956253300 | 0.497803100 | 0.254467609 | tuplegetter_one | 0.152512600 | 0.148523900 | 0.973846751 | tuplegetter_many | 0.184394100 | 0.184511500 | 1.000636680 | tuplegetter_seq | 0.154832800 | 0.156270100 | 1.009282917 | tuplegetter_new_one | 0.523730000 | 0.343402200 | 0.655685563 | tuplegetter_new_many| 0.738924400 | 0.420961400 | 0.569694816 | tuplegetter_new_seq | 1.062036900 | 0.495462000 | 0.466520514 | Parent commit times | python | cython | cy / py | base_row_new | 0.643890800 | 0.113548300 | 0.176347138 | row_new | 0.674885900 | 0.124391800 | 0.184315304 | base_row_new_proc | 3.072020400 | 2.017367000 | 0.656690626 | row_new_proc | 3.109943400 | 2.048359400 | 0.658648450 | brow_new_proc_none | 1.967133700 | 1.006326000 | 0.511569702 | row_new_proc_none | 1.960814900 | 1.025217800 | 0.522852922 | tuplegetter_one | 0.197359900 | 0.205999000 | 1.043773330 | tuplegetter_many | 0.196575900 | 0.194888500 | 0.991416038 | tuplegetter_seq | 0.192723900 | 0.205635000 | 1.066992729 | tuplegetter_new_one | 0.534644500 | 0.414311700 | 0.774929322 | tuplegetter_new_many| 0.479376500 | 0.417448100 | 0.870814694 | tuplegetter_new_seq | 0.481580200 | 0.412697900 | 0.856966088 | Change-Id: I2ca1f49dca2beff625c283f1363c29c8ccc0c3f7
* Prebuild the row string to position lookup for RowsJ. Nick Koston2023-04-261-3/+1
| | | | | | | | | | | | | | | Improved :class:`_engine.Row` implementation to optimize ``__getattr__`` performance. The serialization of a :class:`_engine.Row` to pickle has changed with this change. Pickle saved by older SQLAlchemy versions can still be loaded, but new pickle saved by this version cannot be loaded by older ones. Fixes: #9678 Closes: #9668 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9668 Pull-request-sha: 86b8ccd1959dbd91b1208f7a648a91f217e1f866 Change-Id: Ia85c26a59e1a57ba2bf0d65578c6168f82a559f2
* Add missing methods to OrderedSet.Federico Caselli2023-03-301-0/+128
| | | | | | | Implemented missing method ``copy`` and ``pop`` in OrderedSet class. Fixes: #9487 Change-Id: I1d2278b64939b44422e9d5857ec7d345fff53997
* support DeclarativeBase for versioned history exampleMike Bayer2023-03-271-1/+9
| | | | | | | | | | Fixed issue in "versioned history" example where using a declarative base that is derived from :class:`_orm.DeclarativeBase` would fail to be mapped. Additionally, repaired the given test suite so that the documented instructions for running the example using Python unittest now work again. Change-Id: I164a5b8dbdd01e3d815eb356f7b7cadf226ca296 References: #9546
* Fix regression when deserializing python rows into cythonFederico Caselli2023-03-102-13/+25
| | | | | | | | | | | | | | Fixed regression involving pickling of Python rows between the cython and pure Python implementations of :class:`.Row`, which occurred as part of refactoring code for version 2.0 with typing. A particular constant were turned into a string based ``Enum`` for the pure Python version of :class:`.Row` whereas the cython version continued to use an integer constant, leading to deserialization failures. Regression occurred in a4bb502cf95ea3523e4d383c4377e50f402d7d52 Fixes: #9423 Change-Id: Icbd85cacb2d589cef7c246de7064249926146f2e
* port history meta to 2.0Mike Bayer2023-02-061-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | first change: Reworked the :ref:`examples_versioned_history` to work with version 2.0, while at the same time improving the overall working of this example to use newer APIs, including a newly added hook :meth:`_orm.MapperEvents.after_mapper_constructed`. second change: Added new event hook :meth:`_orm.MapperEvents.after_mapper_constructed`, which supplies an event hook to take place right as the :class:`_orm.Mapper` object has been fully constructed, but before the :meth:`_orm.registry.configure` call has been called. This allows code that can create additional mappings and table structures based on the initial configuration of a :class:`_orm.Mapper`, which also integrates within Declarative configuration. Previously, when using Declarative, where the :class:`_orm.Mapper` object is created within the class creation process, there was no documented means of running code at this point. The change is to immediately benefit custom mapping schemes such as that of the :ref:`examples_versioned_history` example, which generate additional mappers and tables in response to the creation of mapped classes. third change: The infrequently used :attr:`_orm.Mapper.iterate_properties` attribute and :meth:`_orm.Mapper.get_property` method, which are primarily used internally, no longer implicitly invoke the :meth:`_orm.registry.configure` process. Public access to these methods is extremely rare and the only benefit to having :meth:`_orm.registry.configure` would have been allowing "backref" properties be present in these collections. In order to support the new :meth:`_orm.MapperEvents.after_mapper_constructed` event, iteration and access to the internal :class:`_orm.MapperProperty` objects is now possible without triggering an implicit configure of the mapper itself. The more-public facing route to iteration of all mapper attributes, the :attr:`_orm.Mapper.attrs` collection and similar, will still implicitly invoke the :meth:`_orm.registry.configure` step thus making backref attributes available. In all cases, the :meth:`_orm.registry.configure` is always available to be called directly. fourth change: Fixed obscure ORM inheritance issue caused by :ticket:`8705` where some scenarios of inheriting mappers that indicated groups of columns from the local table and the inheriting table together under a :func:`_orm.column_property` would nonetheless warn that properties of the same name were being combined implicitly. Fixes: #9220 Fixes: #9232 Change-Id: Id335b8e8071c8ea509c057c389df9dcd2059437d
* Improve sql formattingFederico Caselli2023-01-111-1/+3
| | | | | | change {opensql} to {printsql} in prints, add missing markers Change-Id: I07b72e6620bb64e329d6b641afa27631e91c4f16
* warn and skip for FKs that refer to invisible cols for OracleMike Bayer2023-01-091-0/+1
| | | | | | | | | | | | | | | | | | | | | Supported use case for foreign key constraints where the local column is marked as "invisible". The errors normally generated when a :class:`.ForeignKeyConstraint` is created that check for the target column are disabled when reflecting, and the constraint is skipped with a warning in the same way which already occurs for an :class:`.Index` with a similar issue. tests are added for indexes, unique constraints, and primary key constraints, which were already working; indexes and uniques warn, primary keys don't which we would assume is because we never see those PK columns in the first place. Constraints now raise an informative ConstraintColumnNotFoundError in the general case for strings in the "pending colargs" collection not being resolvable. Fixes: #9059 Change-Id: I400cf0bff6abba0e0c75f38b07617be1a8ec3453
* disallow same-named columns, unchecked replacement in TableMike Bayer2022-12-041-0/+1
| | | | | | | | | | | | | | | Fixed issue where table reflection using :paramref:`.Table.extend_existing` would fail to deduplicate a same-named column if the existing :class:`.Table` used a separate key. The :paramref:`.Table.autoload_replace` parameter would allow the column to be skipped but under no circumstances should a :class:`.Table` ever have the same-named column twice. Additionally, changed deprecation warnings to exceptions as were implemented in I1d58c8ebe081079cb669e7ead60886ffc1b1a7f5 . Fixes: #8925 Change-Id: I83d0f8658177a7ffbb06e01dbca91377d1a98d49
* Try running pyupgrade on the codeFederico Caselli2022-11-164-78/+74
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* perf improvements related to corresponding_column (2)Mike Bayer2022-11-151-3/+20
| | | | | | | | | | | | | | | | | commit two of two. this reorganizes ColumnCollection to build a new index up front that's used to optimize the corresponding_column() method. Additional performance enhancements within ORM-enabled SQL statements, specifically targeting callcounts within the construction of ORM statements, using combinations of :func:`_orm.aliased` with :func:`_sql.union` and similar "compound" constructs, in addition to direct performance improvements to the ``corresponding_column()`` internal method that is used heavily by the ORM by constructs like :func:`_orm.aliased` and similar. Fixes: #8796 Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
* work around Python 3.11 IntEnum issue; update FastIntFlagMike Bayer2022-11-101-8/+97
| | | | | | | | | | | | | | | | | in [1], Python 3.11 seems to have changed the behavior of IntEnum. We didn't notice this because we have our own workaround class already, but typing did. Ensure we remain compatible with IntFlag. This change also modifies FastIntFlag to no longer use global symbols; this is unnecessary as we assign FastIntFlag members explicitly. Use of ``symbol()`` should probably be phased out. [1] https://github.com/python/cpython/issues/99304 Fixes: #8783 Change-Id: I8ae2e871ff1467ae5ca1f63e66b5dae45d4a6c93
* Support result.close() for all iterator patternsMike Bayer2022-11-031-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change contains new features for 2.0 only as well as some behaviors that will be backported to 1.4. For 1.4 and 2.0: Fixed issue where the underlying DBAPI cursor would not be closed when using :class:`_orm.Query` with :meth:`_orm.Query.yield_per` and direct iteration, if a user-defined exception case were raised within the iteration process, interrupting the iterator. This would lead to the usual MySQL-related issues with server side cursors out of sync. For 1.4 only: A similar scenario can occur when using :term:`2.x` executions with direct use of :class:`.Result`, in that case the end-user code has access to the :class:`.Result` itself and should call :meth:`.Result.close` directly. Version 2.0 will feature context-manager calling patterns to address this use case. However within the 1.4 scope, ensured that ``.close()`` methods are available on all :class:`.Result` implementations including :class:`.ScalarResult`, :class:`.MappingResult`. For 2.0 only: To better support the use case of iterating :class:`.Result` and :class:`.AsyncResult` objects where user-defined exceptions may interrupt the iteration, both objects as well as variants such as :class:`.ScalarResult`, :class:`.MappingResult`, :class:`.AsyncScalarResult`, :class:`.AsyncMappingResult` now support context manager usage, where the result will be closed at the end of iteration. Corrected various typing issues within the engine and async engine packages. Fixes: #8710 Change-Id: I3166328bfd3900957eb33cbf1061d0495c9df670
* Add pep 584 to python immutabledict fallbackFederico Caselli2022-10-221-0/+17
| | | | | Fixes: #8695 Change-Id: Ie0412c3a7b2b1ba5bd5112f204318ff763cbb8f4
* implement write-only colletions, typing for dynamicMike Bayer2022-10-061-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For 2.0, we provide a truly "larger than memory collection" implementation, a write-only collection that will never under any circumstances implicitly load the entire collection, even during flush. This is essentially a much more "strict" version of the "dynamic" loader, which in fact has a lot of scenarios that it loads the full backing collection into memory, mostly defeating its purpose. Typing constructs are added that support both the new feature WriteOnlyMapping as well as the legacy feature DynamicMapping. These have been integrated with "annotion based mapping" so that relationship() uses these annotations to configure the loader strategy as well. additional changes: * the docs triggered a conflict in hybrid's "transformers" section, this section is hard-coded to Query using a pattern that doesnt seem to have any use and isn't part of the current select() interface, so just removed this section * As the docs for WriteOnlyMapping are very long, collections.rst is broken up into two pages now. Fixes: #6229 Fixes: #7123 Change-Id: I6929f3da6e441cad92285e7309030a9bac4e429d
* add disable doctest tag for autodoc test suiteMike Bayer2022-10-011-12/+18
| | | | | | | | | | | ahead of trying to get everything formatted, some more flexibility so that we can use doctest for all python + sql code, while still being able to tell the test suite to not run doctests on a sample. All of the "non-console python with SQL" in the docs is because I was showing an example that I didn't want tested. Change-Id: Iae876ae1ffd93c36b096c6c2d6048843ae9698c8
* adjust tests for sqlites w/o returningMike Bayer2022-09-261-2/+2
| | | | | | | | | the sqlite builds on github actions seem to be very inconsistent about versions and many don't support RETURNING. ensure any tests that depend on RETURNING present are marked as such. Change-Id: I7a60a81fa70b90642448cdd58eda33212c3afebc
* New ORM Query Guide featuring DML supportMike Bayer2022-09-251-8/+71
| | | | | | | | | | | | | | | | | reviewers: these docs publish periodically at: https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html See the "last generated" timestamp near the bottom of the page to ensure the latest version is up Change includes some other adjustments: * small typing fixes for end-user benefit * removal of a bunch of old examples for patterns that nobody uses or aren't really what we promote now * modernization of some examples, including inheritance Change-Id: I9929daab7797be9515f71c888b28af1209e789ff
* Improve compiled extension detectionFederico Caselli2022-09-051-0/+18
| | | | | | | | | Ensure that all cython extension are imported by the compied detection logic. This is required since cython extensions moduels are marked as optional in the install, so it's possible that only some of them are compiled. The extensions are enabled only if all of them are correctly compiled Change-Id: I355cbac06f5c7a47d35661f42ebab3b0156c1965
* run update_subclass anytime we add new clslevel dispatchMike Bayer2022-08-311-0/+29
| | | | | | | | | | | Fixed event listening issue where event listeners added to a superclass would be lost if a subclass were created which then had its own listeners associated. The practical example is that of the :class:`.sessionmaker` class created after events have been associated with the :class:`_orm.Session` class. Fixes: #8467 Change-Id: I9bdba8769147e30110a09900d4a577e833ac3af9
* implement tuple-slices from .c collectionsMike Bayer2022-08-011-2/+25
| | | | | | | | | | | | Added new syntax to the ``.c`` collection on all :class:`.FromClause` objects allowing tuples of keys to be passed to ``__getitem__()``, along with support for ``select()`` handling of ``.c`` collections directly, allowing the syntax ``select(table.c['a', 'b', 'c'])`` to be possible. The sub-collection returned is itself a :class:`.ColumnCollection` which is also directly consumable by :func:`_sql.select` and similar now. Fixes: #8285 Change-Id: I2236662c477ffc50af079310589e213323c960d1
* update ORM declarative docs for new featuresMike Bayer2022-07-161-0/+2
| | | | | | | I screwed up a rebase or something so this was temporarily in Ic51a12de3358f3a451bd7cf3542b375569499fc1 Change-Id: I847ee1336381221c0112b67854df022edf596b25
* try fixing the buildFederico Caselli2022-06-131-0/+2
| | | | Change-Id: Id2e965aa13a6d7134ca1081554cc5b25dbcc9fde
* dont transfer __weakref__ to regenerated classMike Bayer2022-06-091-0/+46
| | | | | | | | | | Repaired a deprecation warning class decorator that was preventing key objects such as :class:`_engine.Connection` from having a proper ``__weakref__`` attribute, causing operations like Python standard library ``inspect.getmembers()`` to fail. Fixes: #8115 Change-Id: Ifd0bc2325fb9dc9e1431998c308b7fc081968373
* Support handle_error for pre_pingMike Bayer2022-05-311-2/+2
| | | | | | | | | | | | | The :meth:`.DialectEvents.handle_error` event is now moved to the :class:`.DialectEvents` suite from the :class:`.EngineEvents` suite, and now participates in the connection pool "pre ping" event for those dialects that make use of disconnect codes in order to detect if the database is live. This allows end-user code to alter the state of "pre ping". Note that this does not include dialects which contain a native "ping" method such as that of psycopg2 or most MySQL dialects. Fixes: #5648 Change-Id: I353d84a4f66f309d2467b7e67621db6b8c70411e
* pep484 ORM / SQL result supportMike Bayer2022-04-271-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | after some experimentation it seems mypy is more amenable to the generic types being fully integrated rather than having separate spin-off types. so key structures like Result, Row, Select become generic. For DML Insert, Update, Delete, these are spun into type-specific subclasses ReturningInsert, ReturningUpdate, ReturningDelete, which is fine since the "row-ness" of these constructs doesn't happen until returning() is called in any case. a Tuple based model is then integrated so that these objects can carry along information about their return types. Overloads at the .execute() level carry through the Tuple from the invoked object to the result. To suit the issue of AliasedClass generating attributes that are dynamic, experimented with a custom subclass AsAliased, but then just settled on having aliased() lie to the type checker and return `Type[_O]`, essentially. will need some type-related accessors for with_polymorphic() also. Additionally, identified an issue in Update when used "mysql style" against a join(), it basically doesn't work if asked to UPDATE two tables on the same column name. added an error message to the specific condition where it happens with a very non-specific error message that we hit a thing we can't do right now, suggest multi-table update as a possible cause. Change-Id: I5eff7eefe1d6166ee74160b2785c5e6a81fa8b95
* fix result.columns() methodMike Bayer2022-04-211-0/+28
| | | | | | | | | | | | Fixed issue in :meth:`.Result.columns` method where calling upon :meth:`.Result.columns` with a single index could in some cases, particularly ORM result object cases, cause the :class:`.Result` to yield scalar objects rather than :class:`.Row` objects, as though the :meth:`.Result.scalars` method had been called. In SQLAlchemy 1.4, this scenario emits a warning that the behavior will change in SQLAlchemy 2.0. Fixes: #7953 Change-Id: I3c4ca3eecc2bfc85ad1c38000e5990d6dde80d22
* Allow contextvars to be set in events when using asyncioFederico Caselli2022-04-171-7/+31
| | | | | | | | | Allow setting contextvar values inside async adapted event handlers. Previously the value set to the contextvar would not be properly propagated. Fixes: #7937 Change-Id: I787aa869f8d057579e13e32c749f05f184ffd02a
* pep-484: session, instancestate, etcMike Bayer2022-04-121-12/+35
| | | | | | | | Also adds some fixes to annotation-based mapping that have come up, as well as starts to add more pep-484 test cases Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
* pep484 - sql.selectableMike Bayer2022-04-041-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the pep484 task becomes more intense as there is mounting pressure to come up with a consistency in how data moves from end-user to instance variable. current thinking is coming into: 1. there are _typing._XYZArgument objects that represent "what the user sent" 2. there's the roles, which represent a kind of "filter" for different kinds of objects. These are mostly important as the argument we pass to coerce(). 3. there's the thing that coerce() returns, which should be what the construct uses as its internal representation of the thing. This is _typing._XYZElement. but there's some controversy over whether or not we should pass actual ClauseElements around by their role or not. I think we shouldn't at the moment, but this makes the "role-ness" of something a little less portable. Like, we have to set DMLTableRole for TableClause, Join, and Alias, but then also we have to repeat those three types in order to set up _DMLTableElement. Other change introduced here, there was a deannotate=True for the left/right of a sql.join(). All tests pass without that. I'd rather not have that there as if we have a join(A, B) where A, B are mapped classes, we want them inside of the _annotations. The rationale seems to be performance, but this performance can be illustrated to be on the compile side which we hope is cached in the normal case. CTEs now accommodate for text selects including recursive. Get typing to accommodate "util.preloaded" cleanly; add "preloaded" as a real module. This seemed like we would have needed pep562 `__getattr__()` but we don't, just set names in globals() as we import them. References: #6810 Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
* pep-484: the pep-484ening, SQL part threeMike Bayer2022-03-301-19/+19
| | | | | | | | | | | | | | | hitting DML which is causing us to open up the ColumnCollection structure a bit, as we do put anonymous column expressions with None here. However, we still want Table /TableClause to have named column collections that don't return None, so parametrize the "key" in this collection also. * rename some "immutable" elements to "readonly". we change the contents of immutablecolumncollection underneath, so it's not "immutable" Change-Id: I2593995a4e5c6eae874bed5bf76117198be8ae97
* ORM quickstartMike Bayer2022-03-111-0/+3
| | | | | | | | | This is done in 1.4 style so it can be backported to 1.4. Will put this up as is, we can work on it. For 2.0, the ORM mapping will be updated to mapped_column() style when we do the full pass. Change-Id: Icfdf81449973844dac244b3a107ce955a7d3b16c
* improve error raise for dialect/pool events w/ async engineMike Bayer2022-03-021-0/+45
| | | | | | | | Fixed issues where a descriptive error message was not raised for some classes of event listening with an async engine, which should instead be a sync engine instance. Change-Id: I00b9f4fe9373ef5fd5464fac10651cc4024f648e
* pep-484 for engineMike Bayer2022-03-011-22/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All modules in sqlalchemy.engine are strictly typed with the exception of cursor, default, and reflection. cursor and default pass with non-strict typing, reflection is waiting on the multi-reflection refactor. Behavioral changes: * create_connect_args() methods return a tuple of list, dict, rather than a list of list, dict * removed allow_chars parameter from pyodbc connector ._get_server_version_info() method * the parameter list passed to do_executemany is now a list in all cases. previously, this was being run through dialect.execute_sequence_format, which defaults to tuple and was only intended for individual tuple params. * broke up dialect.dbapi into dialect.import_dbapi class method and dialect.dbapi module object. added a deprecation path for legacy dialects. it's not really feasible to type a single attr as a classmethod vs. module type. The "type_compiler" attribute also has this problem with greater ability to work around, left that one for now. * lots of constants changing to be Enum, so that we can type them. for fixed tuple-position constants in cursor.py / compiler.py (which are used to avoid the speed overhead of namedtuple), using Literal[value] which seems to work well * some tightening up in Row regarding __getitem__, which we can do since we are on full 2.0 style result use * altered the set_connection_execution_options and set_engine_execution_options event flows so that the dictionary of options may be mutated within the event hook, where it will then take effect as the actual options used. Previously, changing the dict would be silently ignored which seems counter-intuitive and not very useful. * A lot of DefaultDialect/DefaultExecutionContext methods and attributes, including underscored ones, move to interfaces. This is not fully ideal as it means the Dialect/ExecutionContext interfaces aren't publicly subclassable directly, but their current purpose is more of documentation for dialect authors who should (and certainly are) still be subclassing the DefaultXYZ versions in all cases Overall, Result was the most extremely difficult class hierarchy to type here as this hierarchy passes through largely amorphous "row" datatypes throughout, which can in fact by all kinds of different things, like raw DBAPI rows, or Row objects, or "scalar"/Any, but at the same time these types have meaning so I tried still maintaining some level of semantic markings for these, it highlights how complex Result is now, as it's trying to be extremely efficient and inlined while also being very open-ended and extensible. Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
* establish mypy / typing approach for v2.0Mike Bayer2022-02-132-1/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* mypy: sqlalchemy.utilMike Bayer2022-01-241-2/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting to set up practices and conventions to get the library typed. Key goals for typing are: 1. whole library can pass mypy without any strict turned on. 2. we can incrementally turn on some strict flags on a per-package/ module basis, as here we turn on more strictness for sqlalchemy.util, exc, and log 3. mypy ORM plugin tests work fully without sqlalchemy2-stubs installed 4. public facing methods all have return types, major parameter signatures filled in also 5. Foundational elements like util etc. are typed enough so that we can use them in fully typed internals higher up the stack. Conventions set up here: 1. we can use lots of config in setup.cfg to limit where mypy is throwing errors and how detailed it should be in different packages / modules. We can use this to push up gerrits that will pass tests fully without everything being typed. 2. a new tox target pep484 is added. this links to a new jenkins pep484 job that works across all projects (alembic, dogpile, etc.) We've worked around some mypy bugs that will likely be around for awhile, and also set up some core practices for how to deal with certain things such as public_factory modules (mypy won't accept a module from a callable at all, so need to use simple type checking conditionals). References: #6810 Change-Id: I80be58029896a29fd9f491aa3215422a8b705e12
* after all that, use pytest warnings pluginMike Bayer2022-01-231-0/+1
| | | | | | | | | | | | | | | | | The warnings plugin lets us set the filters up in the config, and as our filter requirements are now simple we can just set this up. additionally pytest now recommends pyproject.toml, since we fully include this now, let's move it there. the pytest logging plugin seems to not be any problem either at the moment, so re-enable that. if it becomes apparent whatever the problem was (which was probably that it was just surprising, or something) we can disable it again and comment what the reason was. Change-Id: Ia9715533b01f72aa5fdcf6a27ce75b76f829fa43
* initial reorganize for static typingMike Bayer2022-01-122-36/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | start applying foundational annotations to key elements. two main elements addressed here: 1. removal of public_factory() and replacement with explicit functions. this just works much better with typing. 2. typing support for column expressions and operators. The biggest part of this involves stubbing out all the ColumnOperators methods under ColumnElement in a TYPE_CHECKING section. Took me a while to see this method vs. much more complicated things I thought I needed. Also for this version implementing #7519, ColumnElement types against the Python type and not TypeEngine. it is hoped this leads to easier transferrence between ORM/Core as well as eventual support for result set typing. Not clear yet how well this approach will work and what new issues it may introduce. given the current approach we now get full, rich typing for scenarios like this: from sqlalchemy import column, Integer, String, Boolean c1 = column('a', String) c2 = column('a', Integer) expr1 = c2.in_([1, 2, 3]) expr2 = c2 / 5 expr3 = -c2 expr4_a = ~(c2 == 5) expr4_b = ~column('q', Boolean) expr5 = c1 + 'x' expr6 = c2 + 10 Fixes: #7519 Fixes: #6810 Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
* Merge "Fix various source comment/doc typos" into mainmike bayer2022-01-071-2/+2
|\
| * Fix various source comment/doc typosluz paz2021-12-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description Found via `codespell -q 3 -L ba,crate,datas,froms,gord,hist,inh,nd,selectin,strat,ue` Also added codespell to the pep8 tox env ### Checklist This pull request is: - [x] A documentation / typographical error fix - Good to go, no issue or tests are needed Closes: #7338 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7338 Pull-request-sha: 0deac2219396bc0eba7da53eb3a80932edbf2dd7 Change-Id: Icd61db31c8dc655d4a39d8a304194804d08555fe
* | Remove all remaining removed_in_20 warnings slated for removalMike Bayer2022-01-051-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Finalize all remaining removed-in-2.0 changes so that we can begin doing pep-484 typing without old things getting in the way (we will also have to do public_factory). note there are a few "moved_in_20()" and "became_legacy_in_20()" warnings still in place. The SQLALCHEMY_WARN_20 variable is now removed. Also removed here are the legacy "in place mutators" for Select statements, and some keyword-only argument signatures in Core have been added. Also in the big change department, the ORM mapper() function is removed entirely; the Mapper class is otherwise unchanged, just the public-facing API function. Mappers are now always given a registry in which to participate, however the argument signature of Mapper is not changed. ideally "registry" would be the first positional argument. Fixes: #7257 Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
* | remove 2.0-removed Query elementsMike Bayer2022-01-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * :meth:`_orm.Query.join` no longer accepts the "aliased" and "from_joinpoint" arguments * :meth:`_orm.Query.join` no longer accepts chains of multiple join targets in one method call. * ``Query.from_self()`` and ``Query.with_polymorphic()`` are removed. Change-Id: I534d04b53a538a4fc374966eb2bc8eb98a16497d References: #7257
* | Merge "Replace raise_ with raise from" into mainFederico Caselli2021-12-271-60/+0
|\ \
| * | Replace raise_ with raise fromFederico Caselli2021-12-271-60/+0
| |/ | | | | | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* | Merge "propose concurrency check for SessionTransaction" into mainmike bayer2021-12-271-0/+1
|\ \ | |/ |/|
| * propose concurrency check for SessionTransactionMike Bayer2021-12-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the discussion at #7387 refers to a condition that seems to happen in the wild also, such as [1] [2] [3], it's not entirely clear why this specific spot is how this occurs, however it's maybe that when the connection is being acquired from the pool, under load there might be a wait on the connection pool, leading to more time for another errant thread to be calling .close(), just a theory. in this patch we propose using decorators and context managers along with declarative state declarations to block reentrant or concurrent calls to methods that conflict with expected state changes. The :class:`_orm.Session` (and by extension :class:`.AsyncSession`) now has new state-tracking functionality that will proactively trap any unexpected state changes which occur as a particular transactional method proceeds. This is to allow situations where the :class:`_orm.Session` is being used in a thread-unsafe manner, where event hooks or similar may be calling unexpected methods within operations, as well as potentially under other concurrency situations such as asyncio or gevent to raise an informative message when the illegal access first occurs, rather than passing silently leading to secondary failures due to the :class:`_orm.Session` being in an invalid state. [1] https://stackoverflow.com/questions/25768428/sqlalchemy-connection-errors [2] https://groups.google.com/g/sqlalchemy/c/n5oVX3v4WOw [3] https://github.com/cosmicpython/code/issues/23 Fixes: #7433 Change-Id: I699b935c0ec4e5a63f12cf878af6f7a92a30a3aa
* | factor out UnboundLoad and rearchitect strategy_options.pyMike Bayer2021-12-271-22/+37
|/ | | | | | | | | | | | | | | | | | | | | The architecture of Load is mostly rewritten here. The change includes removal of the "pluggable" aspect of the loader options, which would patch new methods onto Load. This has been replaced by normal methods that respond normally to typing annotations. As part of this change, the bake_loaders() and unbake_loaders() options, which have no effect since 1.4 and were unlikely to be in any common use, have been removed. Additionally, to support annotations for methods that make use of @decorator, @generative etc., modified format_argspec_plus to no longer return "args", instead returns "grouped_args" which is always grouped and allows return annotations to format correctly. Fixes: #6986 Change-Id: I6117c642345cdde65a64389bba6057ddd5374427
* Replace c extension with cython versions.workflow_test_cythonFederico Caselli2021-12-172-0/+81
| | | | | | | | | | | | | | | Re-implement c version immutabledict / processors / resultproxy / utils with cython. Performance is in general in par or better than the c version Added a collection module that has cython version of OrderedSet and IdentitySet Added a new test/perf file to compare the implementations. Run ``python test/perf/compiled_extensions.py all`` to execute the comparison test. See results here: https://docs.google.com/document/d/1nOcDGojHRtXEkuy4vNXcW_XOJd9gqKhSeALGG3kYr6A/edit?usp=sharing Fixes: #7256 Change-Id: I2930ef1894b5048210384728118e586e813f6a76 Signed-off-by: Federico Caselli <cfederico87@gmail.com>
* adapt pytest plugin to support pytest v7Federico Caselli2021-11-291-1/+0
| | | | | | | | | | | | Implemented support for the test suite to run correctly under Pytest 7. Previously, only Pytest 6.x was supported for Python 3, however the version was not pinned on the upper bound in tox.ini. Pytest is not pinned in tox.ini to be lower than version 8 so that SQLAlchemy versions released with the current codebase will be able to be tested under tox without changes to the environment. Much thanks to the Pytest developers for their help with this issue. Change-Id: I3b12166199be2b913ee16e78b3ebbff415654396
View Lorry Controller Status