summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/util/typing.py
Commit message (Collapse)AuthorAgeFilesLines
* adjust fwd_ref logicMike Bayer2023-04-281-1/+1
| | | | | | | | | | Fixed issue where ORM Annotated Declarative would not resolve forward references correctly in all cases; in particular, when using ``from __future__ import annotations`` in combination with Pydantic dataclasses. Change-Id: If643c9a4ac7e217d4cb3a7d09b96cfd49432c44e References: #9717
* check for recursion with container typesMike Bayer2023-03-271-8/+22
| | | | | | | | | | Fixed issue in ORM Annotated Declarative where using a recursive type (e.g. using a nested Dict type) would result in a recursion overflow in the ORM's annotation resolution logic, even if this datatype were not necessary to map the column. Fixes: #9553 Change-Id: Ied99dc0d47276c6e9c23fa9df5fc65f7736d65cf
* apply a fixed locals w/ Mapped to all de-stringifyMike Bayer2023-02-261-6/+29
| | | | | | | | | | | | | | | Continued the fix for :ticket:`8853`, allowing the :class:`_orm.Mapped` name to be fully qualified regardless of whether or not ``from __annotations__ import future`` were present. This issue first fixed in 2.0.0b3 confirmed that this case worked via the test suite, however the test suite apparently was not testing the behavior for the name ``Mapped`` not being locally present at all; string resolution has been updated to ensure the ``Mapped`` symbol is locatable as applies to how the ORM uses these functions. Fixes: #8853 Fixes: #9335 Change-Id: Id82d09aee906165a4d77c7da6a0b4177dd675c10
* Remove `typing.Self` workaroundYurii Karabas2023-02-081-1/+1
| | | | | | | | | | | | Remove ``typing.Self`` workaround, now using :pep:`673` for most methods that return ``Self``. Pull request courtesy Yurii Karabas. Fixes: #9254 Closes: #9255 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9255 Pull-request-sha: 2947df8ada79f5c3afe9c838e65993302199c2f7 Change-Id: Ic32015ad52e95a61f3913d43ea436aa9402804df
* Add support for typing.Literal in MappedFrederik Aalund2023-01-311-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added support for :pep:`586` ``Literal`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. To use custom types such as these, they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Pull request courtesy Frederik Aalund. As part of this change, the support for :class:`.sqltypes.Enum` in the :paramref:`_orm.registry.type_annotation_map` has been expanded to include support for ``Literal[]`` types consisting of string values to be used, in addition to ``enum.Enum`` datatypes. If a ``Literal[]`` datatype is used within ``Mapped[]`` that is not linked in :paramref:`_orm.registry.type_annotation_map` to a specific datatype, a :class:`.sqltypes.Enum` will be used by default. Fixed issue involving the use of :class:`.sqltypes.Enum` within the :paramref:`_orm.registry.type_annotation_map` where the :paramref:`_sqltypes.Enum.native_enum` parameter would not be correctly copied to the mapped column datatype, if it were overridden as stated in the documentation to set this parameter to False. Fixes: #9187 Fixes: #9200 Closes: #9191 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9191 Pull-request-sha: 7d13f705307bf62560fc831f6f049a425d411374 Change-Id: Ife3ba2655f4897f806d6a9cf0041c69fd4f39e9d
* support NewType in type_annotation_mapMike Bayer2023-01-311-11/+19
| | | | | | | | | | | | | | | | | Added support for :pep:`484` ``NewType`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. These types will behave in the same way as custom subclasses of types right now; they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Within this change, the lookup between decl_api._resolve_type and TypeEngine._resolve_for_python_type is streamlined to not inspect the given type multiple times, instead passing in from decl_api to TypeEngine the already "flattened" version of a Generic or NewType type. Fixes: #9175 Change-Id: I227cf84b4b88e4567fa2d1d7da0c05b54e00c562
* dont assume copy_with() on builtins list, dict, etc; improve error msg.Mike Bayer2023-01-161-3/+18
| | | | | | | | | | | | | | | | | | Fixed issue where using an ``Annotated`` type in the ``type_annotation_map`` which itself contained a plain container type (e.g. ``list``, ``dict``) generic type as the target type would produce an internal error where the ORM were trying to interpret the ``Annotated`` instance. Added an error message when a :func:`_orm.relationship` is mapped against an abstract container type, such as ``Mapped[Sequence[B]]``, without providing the :paramref:`_orm.relationship.container_class` parameter which is necessary when the type is abstract. Previously the the abstract container would attempt to be instantiated and fail. Fixes: #9099 Fixes: #9100 Change-Id: I18aa6abd5451c5ac75a9caed8441ff0cd8f44589
* Improve support for enum in mapped classesFederico Caselli2022-11-291-1/+10
| | | | | | | | | | Add a new system by which TypeEngine objects have some say in how the declarative type registry interprets them. The Enum datatype is the primary target for this but it is hoped the system may be useful for other types as well. Fixes: #8859 Change-Id: I15ac3daee770408b5795746f47c1bbd931b7d26d
* Add new script to sync test files adding future annotationFederico Caselli2022-11-261-1/+1
| | | | | | | | | | | Create a new test file test_tm_future_annotations_sync.py that's obtained from test_tm_future_annotations.py by using the new script sync_test_files. This files includes the ``from __future__ import annotations`` It also turns out we had some bugs in there and we can also support some additional typing things Change-Id: Iac005df206d45a55345d9d88d67a80ce799d449f
* Merge "fix optionalized forms for dict[]" into mainmike bayer2022-11-221-9/+61
|\
| * fix optionalized forms for dict[]Mike Bayer2022-11-221-9/+61
| | | | | | | | | | | | | | | | | | | | | | Fixed a suite of issues involving :class:`.Mapped` use with dictionary types, such as ``Mapped[dict[str, str] | None]``, would not be correctly interpreted in Declarative ORM mappings. Support to correctly "de-optionalize" this type including for lookup in type_annotation_map has been fixed. Fixes: #8777 Change-Id: Iaba90791dea341d00eaff788d40b0a4e48dab02e
* | fall back to eval() for names that have dotsMike Bayer2022-11-211-0/+2
|/ | | | | | | | | | Fixed regression in 2.0.0b3 caused by :ticket:`8759` where indicating the :class:`.Mapped` name using a qualified name such as ``sqlalchemy.orm.Mapped`` would fail to be recognized by Declarative as indicating the :class:`.Mapped` construct. Fixes: #8853 Change-Id: Iddb6efaae864d4545e80c54658244670f81ef6cc
* try to support mypy 0.990Mike Bayer2022-11-091-18/+31
| | | | | | | | | | | mypy introduces a crash we need to work around, also some new rules. It also has either a behavioral change regarding how output is rendered in relationship to files being within sys.path or not, so work around that for test_mypy_plugin_py3k.py References: https://github.com/python/mypy/issues/14027 Change-Id: I689c7fe27dc52abee932de9e0fb23b2a2eba76fa
* support renamed symbols in annotation scansMike Bayer2022-11-041-14/+53
| | | | | | | | | | | | | | | | Added support in ORM declarative annotations for class names specified for :func:`_orm.relationship`, as well as the name of the :class:`_orm.Mapped` symbol itself, to be different names than their direct class name, to support scenarios such as where :class:`_orm.Mapped` is imported as ``from sqlalchemy.orm import Mapped as M``, or where related class names are imported with an alternate name in a similar fashion. Additionally, a target class name given as the lead argument for :func:`_orm.relationship` will always supersede the name given in the left hand annotation, so that otherwise un-importable names that also don't match the class name can still be used in annotations. Fixes: #8759 Change-Id: I74a00de7e1a45bf62dad50fd385bb75cf343f9f3
* evaluate types in terms of the class in which they appearMike Bayer2022-10-311-3/+7
| | | | | | | | | | | Fixed issues within the declarative typing resolver (i.e. which resolves ``ForwardRef`` objects) where types that were declared for columns in one particular source file would raise ``NameError`` when the ultimate mapped class were in another source file. The types are now resolved in terms of the module for each class in which the types are used. Fixes: #8742 Change-Id: I236f94484ea79d47392a6201e671eeb89c305fd8
* more many-to-one typingMike Bayer2022-10-191-2/+47
| | | | | | | | | | | | | | | | | | since we are typing centric, note this configuration as we have just supported in #8668. Note also I am just taking "backref" out of the basic version of the docs here totally, this doc is already a lot to read / take in without making it even more confusing; backref still has an entirely dedicated docs page which can have all the additional behaviors of backref() described. Additionally, get other "optional" forms to work including ``cls | None`` and ``Union[cls, None]``. Fixes: #8668 Change-Id: I2b026f496a1710ddebfb4aa6cf8459b4892cbc54
* implement write-only colletions, typing for dynamicMike Bayer2022-10-061-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For 2.0, we provide a truly "larger than memory collection" implementation, a write-only collection that will never under any circumstances implicitly load the entire collection, even during flush. This is essentially a much more "strict" version of the "dynamic" loader, which in fact has a lot of scenarios that it loads the full backing collection into memory, mostly defeating its purpose. Typing constructs are added that support both the new feature WriteOnlyMapping as well as the legacy feature DynamicMapping. These have been integrated with "annotion based mapping" so that relationship() uses these annotations to configure the loader strategy as well. additional changes: * the docs triggered a conflict in hybrid's "transformers" section, this section is hard-coded to Query using a pattern that doesnt seem to have any use and isn't part of the current select() interface, so just removed this section * As the docs for WriteOnlyMapping are very long, collections.rst is broken up into two pages now. Fixes: #6229 Fixes: #7123 Change-Id: I6929f3da6e441cad92285e7309030a9bac4e429d
* additional de-stringify pass for unionsMike Bayer2022-09-081-2/+15
| | | | | | | | | | | | | | the change in c3cfee5b00a40790c18d took out a pass for de-stringify that broke some un-tested cases for Optional with future annotations mode. Adding tests for this revealed that this was a subset of a more general case where Union is presented with ForwardRefs inside of it matching up within the type map, which wasn't working before either, fixed that as well with an additional de-stringify for elements within the Union. Fixes: #8478 Change-Id: I8804cf6c67f14d10804584e1cddd2cfaa2376654
* Detection of PEP 604 union syntax.Peter Schutt2022-09-011-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description Fixes #8478 Handle `UnionType` as arguments to `Mapped`, e.g., `Mapped[str | None]`: - adds `utils.typing.is_optional_union()` used to detect if a column should be nullable. - adds `"UnionType"` to `utils.typing.is_optional()` names. - uses `get_origin()` in `utils.typing.is_origin_of()` as `UnionType` has no `__origin__` attribute. - tests with runtime type and postponed annotations and guard the tests running with `compat.py310`. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #8479 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8479 Pull-request-sha: 12417654822272c5847e684c53677f665553ef0e Change-Id: Ib3248043dd4a97324ac592c048385006536b2d49
* runtime annotation fixes for relationshipMike Bayer2022-07-031-2/+4
| | | | | | | | | | | | | | | | * derive uselist=False when fwd ref passed to relationship This case needs to work whether or not the class name is a forward ref. we dont allow the colleciton to be a forward ref so this will work. * fix issues with MappedCollection When using string annotations or __future__.annotations, we need to do more parsing in order to get the target collection properly Change-Id: I9e5a1358b62d060a8815826f98190801a9cc0b68
* always use typing_extensionFederico Caselli2022-06-251-53/+17
| | | | | | always use typing_extensions to import missing types in old python version Change-Id: Ied03e8597e10bfae06f0e37d8624fbad90e74bcf
* rearchitect reflection for batched performanceFederico Caselli2022-06-181-0/+2
| | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* new features for pep 593 AnnotatedMike Bayer2022-06-151-0/+16
| | | | | | | | | | | | | | | * extract the inner type from Annotated when the outer type isn't present in the type map, to allow for arbitrary Annotated * allow _IntrospectsAnnotations objects to be directly present in an Annotated and resolve the mapper property from that. Currently implemented for mapped_column(), with message for others. Can work for composite() and likely some relationship() as well at some point References: https://twitter.com/zzzeek/status/1536693554621341697 and replies Change-Id: I04657050a8785f194bf8f63291faf3475af88781
* implement dataclass_transformsMike Bayer2022-05-201-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | Implement a new means of creating a mapped dataclass where instead of applying the `@dataclass` decorator distinctly, the declarative process itself can create the dataclass. MapperProperty and MappedColumn objects themselves take the place of the dataclasses.Field object when constructing the class. The overall approach is made possible at the typing level using pep-681 dataclass transforms [1]. This new approach should be able to completely supersede the previous "dataclasses" approach of embedding metadata into Field() objects, which remains a mutually exclusive declarative setup style (mixing them introduces new issues that are not worth solving). [1] https://peps.python.org/pep-0681/#transform-descriptor-types-example Fixes: #7642 Change-Id: I6ba88a87c5df38270317b4faf085904d91c8a63c
* revenge of pep 484Mike Bayer2022-05-151-9/+68
| | | | | | trying to get remaining must-haves for ORM Change-Id: I66a3ecbbb8e5ba37c818c8a92737b576ecf012f7
* update for flake8-future-imports 0.0.5Mike Bayer2022-05-141-4/+4
| | | | | | | | a whole bunch of errors were apparently blocked by 0.0.4 being installed. Fixes: #8020 Change-Id: I22a0faeaabe03de501897893391946d677c2df7e
* pep484: attributes and relatedMike Bayer2022-05-031-0/+39
| | | | | | | also implements __slots__ for QueryableAttribute, InstrumentedAttribute, Relationship.Comparator. Change-Id: I47e823160706fc35a616f1179a06c7864089e5b5
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-281-0/+2
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* pep484 ORM / SQL result supportMike Bayer2022-04-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | after some experimentation it seems mypy is more amenable to the generic types being fully integrated rather than having separate spin-off types. so key structures like Result, Row, Select become generic. For DML Insert, Update, Delete, these are spun into type-specific subclasses ReturningInsert, ReturningUpdate, ReturningDelete, which is fine since the "row-ness" of these constructs doesn't happen until returning() is called in any case. a Tuple based model is then integrated so that these objects can carry along information about their return types. Overloads at the .execute() level carry through the Tuple from the invoked object to the result. To suit the issue of AliasedClass generating attributes that are dynamic, experimented with a custom subclass AsAliased, but then just settled on having aliased() lie to the type checker and return `Type[_O]`, essentially. will need some type-related accessors for with_polymorphic() also. Additionally, identified an issue in Update when used "mysql style" against a join(), it basically doesn't work if asked to UPDATE two tables on the same column name. added an error message to the specific condition where it happens with a very non-specific error message that we hit a thing we can't do right now, suggest multi-table update as a possible cause. Change-Id: I5eff7eefe1d6166ee74160b2785c5e6a81fa8b95
* pep-484: ORM public API, constructorsMike Bayer2022-04-201-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for the moment, abandoning using @overload with relationship() and mapped_column(). The overloads are very difficult to get working at all, and the overloads that were there all wouldn't pass on mypy. various techniques of getting them to "work", meaning having right hand side dictate what's legal on the left, have mixed success and wont give consistent results; additionally, it's legal to have Optional / non-optional independent of nullable in any case for columns. relationship cases are less ambiguous but mypy was not going along with things. we have a comprehensive system of allowing left side annotations to drive the right side, in the absense of explicit settings on the right. so type-centric SQLAlchemy will be left-side driven just like dataclasses, and the various flags and switches on the right side will just not be needed very much. in other matters, one surprise, forgot to remove string support from orm.join(A, B, "somename") or do deprecations for it in 1.4. This is a really not-directly-used structure barely mentioned in the docs for many years, the example shows a relationship being used, not a string, so we will just change it to raise the usual error here. Change-Id: Iefbbb8d34548b538023890ab8b7c9a5d9496ec6e
* pep484: schema APIMike Bayer2022-04-151-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | implement strict typing for schema.py this module has lots of public API, lots of old decisions and very hard to follow construction sequences in many cases, and is also where we get a lot of new feature requests, so strict typing should help keep things clean. among improvements here, fixed the pool .info getters and also figured out how to get ColumnCollection and related to be covariant so that we may set them up as returning Column or ColumnClause without any conflicts. DDL was affected, noting that superclasses of DDLElement (_DDLCompiles, added recently) can now be passed into "ddl_if" callables; reorganized ddl into ExecutableDDLElement as a new name for DDLElement and _DDLCompiles renamed to BaseDDLElement. setting up strict also located an API use case that is completely broken, which is connection.execute(some_default) returns a scalar value. This case has been deprecated and new paths have been set up so that connection.scalar() may be used. This likely wasn't possible in previous versions because scalar() would assume a CursorResult. The scalar() change also impacts Session as we have explicit support (since someone had reported it as a regression) for session.execute(Sequence()) to work. They will get the same deprecation message (which omits the word "Connection", just uses ".execute()" and ".scalar()") and they can then use Session.scalar() as well. Getting this to type correctly while still supporting ORM use cases required some refactoring, and I also set up a keyword only delimeter for Session.execute() and related as execution_options / bind_arguments should always be keyword only, applied these changes to AsyncSession as well. Additionally simpify Table __init__ now that we are Python 3 only, we can have positional plus explicit kwargs finally. Simplify Column.__init__ as well again taking advantage of kw only arguments. Fill in most/all __init__ methods in sqltypes.py as the constructor for types is most of the API. should likely do this for dialect-specific types as well. Apply _InfoType for all info attributes as should have been done originally and update descriptor decorators. Change-Id: I3f9f8ff3f1c8858471ff4545ac83d68c88107527
* pep-484: session, instancestate, etcMike Bayer2022-04-121-1/+9
| | | | | | | | Also adds some fixes to annotation-based mapping that have come up, as well as starts to add more pep-484 test cases Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
* update flake8 noqa skips with proper syntaxFederico Caselli2022-04-111-6/+6
| | | | Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
* pep484 - SQL internalsMike Bayer2022-03-241-0/+2
| | | | | | | non-strict checking for mostly internal or semi-internal code Change-Id: Ib91b47f1a8ccc15e666b94bad1ce78c4ab15b0ec
* pep-484 - SQL column operationsMike Bayer2022-03-151-0/+5
| | | | | | | | | note we are taking out the ColumnOperartors[SQLCoreOperations] thing; not really clear why that was needed and at the moment it seems I was likely confused. Change-Id: I834b75f9b44f91b97e29f2e1a7b1029bd910e0a1
* pep-484: sqlalchemy.sql pass oneMike Bayer2022-03-131-2/+22
| | | | | | | | | | | | | | | | | | sqlalchemy.sql will require many passes to get all modules even gradually typed. Will have to pick and choose what modules can be strictly typed vs. which can be gradual. in this patch, emphasis is on visitors.py, cache_key.py, annotations.py for strict typing, compiler.py is on gradual typing but has much more structure, in particular where it connects with the outside world. The work within compiler.py also reached back out to engine/cursor.py , default.py quite a bit. References: #6810 Change-Id: I6e8a29f6013fd216e43d45091bc193f8be0368fd
* additional mypy strictnessMike Bayer2022-03-121-31/+0
| | | | | | | | | | | | | | | | | | enable type checking within untyped defs. This allowed some more internals to be fixed up with assertions etc. some internals that were unnecessary or not even used at all were removed. BaseCursorResult was no longer necessary since we only have one kind of CursorResult now. The different ResultProxy subclasses that had alternate "strategies" dont appear to be used at all even in 1.4.x, as there's no code that accesses the _cursor_strategy_cls attribute, which is also removed. As these were mostly private constructs that weren't even functioning correctly in any case, it's fine to remove these over the 2.0 boundary. Change-Id: Ifd536987d104b1cd8b546cefdbd5c1e5d1801082
* pep484 + abc bases for assocaitionproxyMike Bayer2022-03-011-0/+7
| | | | | | | | | | went to this one next as it was going to be hard, and also exercises the ORM expression hierarchy a bit. made some adjustments to SQLCoreOperations etc. Change-Id: Ie5dde9218dc1318252826b766d3e70b17dd24ea7 References: #6810 References: #7774
* pep-484 for engineMike Bayer2022-03-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All modules in sqlalchemy.engine are strictly typed with the exception of cursor, default, and reflection. cursor and default pass with non-strict typing, reflection is waiting on the multi-reflection refactor. Behavioral changes: * create_connect_args() methods return a tuple of list, dict, rather than a list of list, dict * removed allow_chars parameter from pyodbc connector ._get_server_version_info() method * the parameter list passed to do_executemany is now a list in all cases. previously, this was being run through dialect.execute_sequence_format, which defaults to tuple and was only intended for individual tuple params. * broke up dialect.dbapi into dialect.import_dbapi class method and dialect.dbapi module object. added a deprecation path for legacy dialects. it's not really feasible to type a single attr as a classmethod vs. module type. The "type_compiler" attribute also has this problem with greater ability to work around, left that one for now. * lots of constants changing to be Enum, so that we can type them. for fixed tuple-position constants in cursor.py / compiler.py (which are used to avoid the speed overhead of namedtuple), using Literal[value] which seems to work well * some tightening up in Row regarding __getitem__, which we can do since we are on full 2.0 style result use * altered the set_connection_execution_options and set_engine_execution_options event flows so that the dictionary of options may be mutated within the event hook, where it will then take effect as the actual options used. Previously, changing the dict would be silently ignored which seems counter-intuitive and not very useful. * A lot of DefaultDialect/DefaultExecutionContext methods and attributes, including underscored ones, move to interfaces. This is not fully ideal as it means the Dialect/ExecutionContext interfaces aren't publicly subclassable directly, but their current purpose is more of documentation for dialect authors who should (and certainly are) still be subclassing the DefaultXYZ versions in all cases Overall, Result was the most extremely difficult class hierarchy to type here as this hierarchy passes through largely amorphous "row" datatypes throughout, which can in fact by all kinds of different things, like raw DBAPI rows, or Row objects, or "scalar"/Any, but at the same time these types have meaning so I tried still maintaining some level of semantic markings for these, it highlights how complex Result is now, as it's trying to be extremely efficient and inlined while also being very open-ended and extensible. Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
* pep-484 for poolMike Bayer2022-02-171-19/+0
| | | | | | | | | | | | | | | | | also extends into some areas of utils, events and others as needed. Formalizes a public hierarchy for pool API, with ManagesConnection -> PoolProxiedConnection / ConnectionPoolEntry for connectionfairy / connectionrecord, which are now what's exposed in the event API and other APIs. all public API docs moved to the new objects. Corrects the mypy plugin's check for sqlalchemy-stubs not being insatlled, which has to be imported using the dash in the name to be effective. Change-Id: I16c2cb43b2e840d28e70a015f370a768e70f3581
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-151-0/+2
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
* establish mypy / typing approach for v2.0Mike Bayer2022-02-131-11/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* mypy: sqlalchemy.utilMike Bayer2022-01-241-14/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting to set up practices and conventions to get the library typed. Key goals for typing are: 1. whole library can pass mypy without any strict turned on. 2. we can incrementally turn on some strict flags on a per-package/ module basis, as here we turn on more strictness for sqlalchemy.util, exc, and log 3. mypy ORM plugin tests work fully without sqlalchemy2-stubs installed 4. public facing methods all have return types, major parameter signatures filled in also 5. Foundational elements like util etc. are typed enough so that we can use them in fully typed internals higher up the stack. Conventions set up here: 1. we can use lots of config in setup.cfg to limit where mypy is throwing errors and how detailed it should be in different packages / modules. We can use this to push up gerrits that will pass tests fully without everything being typed. 2. a new tox target pep484 is added. this links to a new jenkins pep484 job that works across all projects (alembic, dogpile, etc.) We've worked around some mypy bugs that will likely be around for awhile, and also set up some core practices for how to deal with certain things such as public_factory modules (mypy won't accept a module from a callable at all, so need to use simple type checking conditionals). References: #6810 Change-Id: I80be58029896a29fd9f491aa3215422a8b705e12
* repair mapper sortMike Bayer2022-01-201-5/+2
| | | | | | | | | | | | | | | | Fixed issue in :meth:`_orm.Session.bulk_save_mappings` where the sorting that takes place when the ``preserve_order`` parameter is set to False would sort partially on ``Mapper`` objects, which is rejected in Python 3.11. Also uses typing_extensions for NotRequired as this symbol does not seem to be in Python 3.11.0a4 yet. For interim 3.11 support, adds the git main build of greenlet for Python 3.11 Fixes: #7591 Change-Id: I24a62f2322ad7dac5d8e4a00853f8a9408877c9c
* Initial ORM typing layoutMike Bayer2022-01-141-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
* Properly type _generative, decorator, public_factoryFederico Caselli2021-12-301-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656 . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
* provide connectionfairy on initializeMike Bayer2021-11-291-0/+43
This is so that dialect methods that are called within init can assume the same argument structure as when they are called in other places; we can nail down the type of object as well. This change seems to mostly impact the isolation level routines in the dialects, as these are called during initialize() as well as on established connections. these methods can now assume a non-proxied DBAPI connection object in all cases, as it is commonly required that attributes like ".autocommit" are set on the object which don't work well in a proxied situation. Other changes: * adds an interface for the "connectionfairy" concept called PoolProxiedConnection. * Removes ``Connectable`` superclass of Connection. ``Connectable`` was originally meant to provide for the "method which accepts connection or engine" theme. As this pattern is greatly reduced in 2.0 and Engine no longer extends from it, the ``Connectable`` superclass doesnt serve any real purpose. Leading from that, to set this in I also applied pep 484 annotations to the Dialect base, and then in the interests of seeing some of the typing information show up in my IDE did a little bit for Engine, Connection and others. I hope that it's feasible that we can add annotations to specific classes and attributes ahead of when we actually try to mass-populate the whole library. This was the original spirit of pep-484 that we can apply annotations gradually. I do of course want to try to do a mass-populate although i think even in that case we will end up doing a lot of manual work anyway (in particular for the changes here which are distinct from what the stubs have). Fixes: #7122 Change-Id: I5dd7fbff8a7ae520a81c165091af12a6a68826db
View Lorry Controller Status