summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/orm/attributes.py
Commit message (Collapse)AuthorAgeFilesLines
* introduce deferred lambdasMike Bayer2020-07-031-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The coercions system allows us to add in lambdas as arguments to Core and ORM elements without changing them at all. By allowing the lambda to produce a deterministic cache key where we can also cheat and yank out literal parameters means we can move towards having 90% of "baked" functionality in a clearer way right in Core / ORM. As a second step, we can have whole statements inside the lambda, and can then add generation with __add__(), so then we have 100% of "baked" functionality with full support of ad-hoc literal values. Adds some more short_selects tests for the moment for comparison. Other tweaks inside cache key generation as we're trying to approach a certain level of performance such that we can remove the use of "baked" from the loader strategies. As we have not yet closed #4639, however the caching feature has been fully integrated as of b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also add complete caching documentation here and close that issue as well. Closes: #4639 Fixes: #5380 Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
* Turn on caching everywhere, add loggingMike Bayer2020-06-101-5/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A variety of caching issues found by running all tests with statement caching turned on. The cache system now has a more conservative approach where any subclass of a SQL element will by default invalidate the cache key unless it adds the flag inherit_cache=True at the class level, or if it implements its own caching. Add working caching to a few elements that were omitted previously; fix some caching implementations to suit lesser used edge cases such as json casts and array slices. Refine the way BaseCursorResult and CursorMetaData interact with caching; to suit cases like Alembic modifying table structures, don't cache the cursor metadata if it were created against a cursor.description using non-positional matching, e.g. "select *". if a table re-ordered its columns or added/removed, now that data is obsolete. Additionally we have to adapt the cursor metadata _keymap regardless of if we just processed cursor.description, because if we ran against a cached SQLCompiler we won't have the right columns in _keymap. Other refinements to how and when we do this adaption as some weird cases were exposed in the Postgresql dialect, a text() construct that names just one column that is not actually in the statement. Fixed that also as it looks like a cut-and-paste artifact that doesn't actually affect anything. Various issues with re-use of compiled result maps and cursor metadata in conjunction with tables being changed, such as change in order of columns. mappers can be cleared but the class remains, meaning a mapper has to use itself as the cache key not the class. lots of bound parameter / literal issues, due to Alembic creating a straight subclass of bindparam that renders inline directly. While we can update Alembic to not do this, we have to assume other people might be doing this, so bindparam() implements the inherit_cache=True logic as well that was a bit involved. turn on cache stats in logging. Includes a fix to subqueryloader which moves all setup to the create_row_processor() phase and elminates any storage within the compiled context. This includes some changes to create_row_processor() signature and a revising of the technique used to determine if the loader can participate in polymorphic queries, which is also applied to selectinloading. DML update.values() and ordered_values() now coerces the keys as we have tests that pass an arbitrary class here which only includes __clause_element__(), so the key can't be cached unless it is coerced. this in turn changed how composite attributes support bulk update to use the standard approach of ClauseElement with annotations that are parsed in the ORM context. memory profiling successfully caught that the Session from Query was getting passed into _statement_20() so that was a big win for that test suite. Apparently Compiler had .execute() and .scalar() methods stuck on it, these date back to version 0.4 and there was a single test in the PostgreSQL dialect tests that exercised it for no apparent reason. Removed these methods as well as the concept of a Compiler holding onto a "bind". Fixes: #5386 Change-Id: I990b43aab96b42665af1b2187ad6020bee778784
* Convert execution to move through SessionMike Bayer2020-05-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch replaces the ORM execution flow with a single pathway through Session.execute() for all queries, including Core and ORM. Currently included is full support for ORM Query, Query.from_statement(), select(), as well as the baked query and horizontal shard systems. Initial changes have also been made to the dogpile caching example, which like baked query makes use of a new ORM-specific execution hook that replaces the use of both QueryEvents.before_compile() as well as Query._execute_and_instances() as the central ORM interception hooks. select() and Query() constructs alike can be passed to Session.execute() where they will return ORM results in a Results object. This API is currently used internally by Query. Full support for Session.execute()->results to behave in a fully 2.0 fashion will be in later changesets. bulk update/delete with ORM support will also be delivered via the update() and delete() constructs, however these have not yet been adapted to the new system and may follow in a subsequent update. Performance is also beginning to lag as of this commit and some previous ones. It is hoped that a few central functions such as the coercions functions can be rewritten in C to re-gain performance. Additionally, query caching is now available and some subsequent patches will attempt to cache more of the per-execution work from the ORM layer, e.g. column getters and adapters. This patch also contains initial "turn on" of the caching system enginewide via the query_cache_size parameter to create_engine(). Still defaulting at zero for "no caching". The caching system still needs adjustments in order to gain adequate performance. Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
* Unify Query and select() , move all processing to compile phaseMike Bayer2020-05-241-5/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | Convert Query to do virtually all compile state computation in the _compile_context() phase, and organize it all such that a plain select() construct may also be used as the source of information in order to generate ORM query state. This makes it such that Query is not needed except for its additional methods like from_self() which are all to be deprecated. The construction of ORM state will occur beyond the caching boundary when the new execution model is integrated. future select() gains a working join() and filter_by() method. as we continue to rebase and merge each commit in the steps, callcounts continue to bump around. will have to look at the final result when it's all in. References: #5159 References: #4705 References: #4639 References: #4871 References: #5010 Change-Id: I19e05b3424b07114cce6c439b05198ac47f7ac10
* Run search and replace of symbolic module namesMike Bayer2020-04-141-8/+9
| | | | | | | | Replaces a wide array of Sphinx-relative doc references with an abbreviated absolute form now supported by zzzeeksphinx. Change-Id: I94bffcc3f37885ffdde6238767224296339698a2
* Enable zzzeeksphinx module prefixesMike Bayer2020-04-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zzzeeksphinx 1.1.2 in git can now convert short prefix names in a configured lookup to fully qualified module names, so that we can have succinct and portable pyrefs that still resolve absolutely. It also includes a formatter that will format all pyrefs in a fully consistent way regardless of the package path, by unconditionally removing all package tokens but always leaving class names in place including for methods, which means we no longer have to deal with tildes in pyrefs. The most immediate goal of the absolute prefixes is that we have lots of "ambiguous" names that appear in muliple places, like select(), ARRAY, ENUM etc. With the incoming future packages there is going to be lots of name overlap so it is necessary that all names eventually use absolute package paths when Sphinx receives them. In multiple stages, pyrefs will be converted using the zzzeeksphinx tools/fix_xrefs.py tool so that doclinks can be made absolute using symbolic prefixes. For this review, the actual search and replace of symbols is not performed, instead some general cleanup to prepare the docs as well as a lookup file used by the tool to do the conversion. this relatively small patch will be backported with appropriate changes to 1.3, 1.2, 1.1 and the tool can then be run on each branch individually. We are shooting for almost no warnings at all for master (still a handful I can't figure out which don't seem to have any impact) , very few for 1.3, and for 1.2 / 1.1 we hope for a significant reduction in warnings. Overall for all versions pyrefs should always point to the correct target, if they are in fact hyperlinked. it's better for a ref to go nowhere and be plain text than go to the wrong thing. Right now, hundreds of API links are pointing to the wrong thing as they are ambiguous names such as refresh(), insert(), update(), select(), join(), JSON etc. and Sphinx sends these all to essesntially random destinations among as many as five or six possible choices per symbol. A shorthand system that allows us to use absolute refs without having to type out a full blown absoulte module is the only way this is going to work, and we should ultimately seek to abandon any use of prefix dot for lookups. Everything should be on an underscore token so at the very least the module spaces can be reorganized without having to search and replace the entire documentation every time. Change-Id: I484a7329034af275fcdb322b62b6255dfeea9151
* Remove code deprecated before version 1.1Federico Caselli2020-04-091-12/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Remove deprecated method ``get_primary_keys` in the :class:`.Dialect` and :class:`.Inspector` classes. - Remove deprecated event ``dbapi_error`` and the method ``ConnectionEvents.dbapi_error`. - Remove support for deprecated engine URLs of the form ``postgres://``. - Remove deprecated dialect ``mysql+gaerdbms``. - Remove deprecated parameter ``quoting`` from :class:`.mysql.ENUM` and :class:`.mysql.SET` in the ``mysql`` dialect. - Remove deprecated function ``comparable_property``. and function ``comparable_using`` in the declarative extension. - Remove deprecated function ``compile_mappers``. - Remove deprecated method ``collection.linker``. - Remove deprecated method ``Session.prune`` and parameter ``Session.weak_identity_map``. This change also removes the class ``StrongInstanceDict``. - Remove deprecated parameter ``mapper.order_by``. - Remove deprecated parameter ``Session._enable_transaction_accounting`. - Remove deprecated parameter ``Session.is_modified.passive``. - Remove deprecated class ``Binary``. Please use :class:`.LargeBinary`. - Remove deprecated methods ``Compiled.compile``, ``ClauseElement.__and__`` and ``ClauseElement.__or__`` and attribute ``Over.func``. - Remove deprecated ``FromClause.count`` method. - Remove deprecated parameter ``Table.useexisting``. - Remove deprecated parameters ``text.bindparams`` and ``text.typemap``. - Remove boolean support for the ``passive`` parameter in ``get_history``. - Remove deprecated ``adapt_operator`` in ``UserDefinedType.Comparator``. Fixes: #4643 Change-Id: Idcd390c77bf7b0e9957907716993bdaa3f1a1763
* Test instance for matching class hierarchy on get_from_identityMike Bayer2020-03-221-0/+1
| | | | | | | | | | | Fixed issue where a lazyload that uses session-local "get" against a target many-to-one relationship where an object with the correct primary key is present, however it's an instance of a sibling class, does not correctly return None as is the case when the lazy loader actually emits a load for that row. Fixes: #5210 Change-Id: I89f9946cfeba61d89a272435f76a5a082b1da30c
* Ensure all nested exception throws have a causeMike Bayer2020-03-021-32/+46
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Enable F821Mike Bayer2020-01-041-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In Ia63a510f9c1d08b055eef62cf047f1f427f0450c we introduced "lambda combinations" which use a bit of function closure inspection in order to allow for testing combinations that make use of symbols that come from test fixtures, or from the test itself. Two problems. One is that we can't use F821 flake8 rule without either adding lots of noqas, skipping the file, or adding arguments to the lambdas themselves that are then populated, which makes for a very verbose system. The other is that the system is already verbose with all those lambdas and the magic in use is a non-explicit kind, hence F821 reminds us that if we can improve upon this, we should. So let's improve upon it by making it so that the "lambda" is just once and up front for the whole thing, and let it accept the arguments directly. This still requires magic, because these test cases need to resolve at test collection time, not test runtime. But we will instead substitute a namespace up front that can be coerced into its desired form within the tests. Additionally, there's a little bit of py2k compatible type annotations present; f821 is checking these, so we have to add those imports also using the TYPE_CHECKING boolean so they don't take place in py2k. Change-Id: Idb7e7a0c8af86d9ab133f548511306ef68cdba14
* happy new yearMike Bayer2020-01-011-1/+1
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Add anonymizing context to cache keys, comparison; convert traversalMike Bayer2019-11-041-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Created new visitor system called "internal traversal" that applies a data driven approach to the concept of a class that defines its own traversal steps, in contrast to the existing style of traversal now known as "external traversal" where the visitor class defines the traversal, i.e. the SQLCompiler. The internal traversal system now implements get_children(), _copy_internals(), compare() and _cache_key() for most Core elements. Core elements with special needs like Select still implement some of these methods directly however most of these methods are no longer explicitly implemented. The data-driven system is also applied to ORM elements that take part in SQL expressions so that these objects, like mappers, aliasedclass, query options, etc. can all participate in the cache key process. Still not considered is that this approach to defining traversibility will be used to create some kind of generic introspection system that works across Core / ORM. It's also not clear if real statement caching using the _cache_key() method is feasible, if it is shown that running _cache_key() is nearly as expensive as compiling in any case. Because it is data driven, it is more straightforward to optimize using inlined code, as is the case now, as well as potentially using C code to speed it up. In addition, the caching sytem now accommodates for anonymous name labels, which is essential so that constructs which have anonymous labels can be cacheable, that is, their position within a statement in relation to other anonymous names causes them to generate an integer counter relative to that construct which will be the same every time. Gathering of bound parameters from any cache key generation is also now required as there is no use case for a cache key that does not extract bound parameter values. Applies-to: #4639 Change-Id: I0660584def8627cad566719ee98d3be045db4b8d
* Implement raiseload for deferred columnsMike Bayer2019-10-161-7/+11
| | | | | | | | | | | | | | | | | | | | | Added "raiseload" feature for ORM mapped columns. As part of this change, the behavior of "deferred" is now more strict; an attribute that is set up as "deferred" at the mapper level no longer participates in an "unexpire" operation; that is, when an unexpire loads all the expired columns of an object which are not themselves in a deferred group, those which are mapper-level deferred will never be loaded. Deferral options set at query time should always be reset by an expiration operation. Renames deferred_scalar_loader to expired_attribute_loader Unfortunately we can't have raiseload() do this because it would break existing wildcard behavior. Fixes: #4826 Change-Id: I30d9a30236e0b69134e4094fb7c1ad2267f089d1
* Remove deprecated extension and similar classesMike Bayer2019-10-061-23/+0
| | | | | | | | | | All long-deprecated "extension" classes have been removed, including MapperExtension, SessionExtension, PoolListener, ConnectionProxy, AttributExtension. These classes have been deprecated since version 0.7 long superseded by the event listener system. Fixes: #4638 Change-Id: If4156d4956b10847bd93b6408a7c52ff5168db9b
* Simplify _ColumnEntity, relatedMike Bayer2019-09-301-10/+10
| | | | | | | | | | | | | | | In the interests of making Query much more lightweight up front, rework the calculations done at the top when the entities are constructed to be much less inolved. Use the new coercion system for _ColumnEntity and stop accepting plain strings, this will need to emit a deprecation warning in 1.3.x. Use annotations and other techniques to reduce the decisionmaking and complexity of Query. For the use case of subquery(), .statement, etc. we would like to do minimal work in order to get the columns clause. Change-Id: I7e459bbd3bb10ec71235f75ef4f3b0a969bec590
* Remove unnecessary util.callable usageSteven Loria2019-09-091-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes: #4850 <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Removes usage of `util.callable`. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #4851 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4851 Pull-request-sha: a0ccdff2cb74f5e944d8baccc269c382b591c8e2 Change-Id: I79918f44becbc5dbefdc7ff65128695c1cabed1d
* Run eager loaders on unexpireMike Bayer2019-08-231-1/+2
| | | | | | | | | | | | Eager loaders, such as joined loading, SELECT IN loading, etc., when configured on a mapper or via query options will now be invoked during the refresh on an expired object; in the case of selectinload and subqueryload, since the additional load is for a single object only, the "immediateload" scheme is used in these cases which resembles the single-parent query emitted by lazy loading. Change-Id: I7ca2c77bff58dc21015d60093a88c387937376b2 Fixes: #1763
* Ensure discarded collection removed from empty collectionsMike Bayer2019-08-221-5/+37
| | | | | | | | | A bulk replace operation was not attending to the previous list still present in the "_empty_collections" dictionary which was added as part of #4519. Fixes: #4519 Change-Id: I3f99f8647c0fb8140b3dfb03686a5d3b90da633f
* Intercept unresolveable comparator attrbute error for attr accessMike Bayer2019-07-171-10/+21
| | | | | | | | | | | | Fixed bug where a synonym created against a mapped attribute that does not exist yet, as is the case when it refers to backref before mappers are configured, would raise recursion errors when trying to test for attributes on it which ultimately don't exist (as occurs when the classes are run through Sphinx autodoc), as the unconfigured state of the synonym would put it into an attribute not found loop. Fixes: #4767 Change-Id: I9aade8628349fbf538181a0049416cec0a17179c
* Merge "Rework AliasedClass __getattr__ to use top-level getattr()"mike bayer2019-05-311-2/+6
|\
| * Rework AliasedClass __getattr__ to use top-level getattr()Dmytro Starosud2019-05-291-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Reworked the attribute mechanics used by :class:`.AliasedClass` to no longer rely upon calling ``__getattribute__`` on the MRO of the wrapped class, and to instead resolve the attribute normally on the wrapped class using getattr(), and then unwrap/adapt that. This allows a greater range of attribute styles on the mapped class including special ``__getattr__()`` schemes; but it also makes the code simpler and more resilient in general. Fixes: #4694 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I28901e2472d3c21e881fe5cafa3b1d3af704fad8
* | Hold implicitly created collections in a pending areaMike Bayer2019-05-271-12/+15
|/ | | | | | | | | | | | | | | Accessing a collection-oriented attribute on a newly created object no longer mutates ``__dict__``, but still returns an empty collection as has always been the case. This allows collection-oriented attributes to work consistently in comparison to scalar attributes which return ``None``, but also don't mutate ``__dict__``. In order to accommodate for the collection being mutated, the same empty collection is returned each time once initially created, and when it is mutated (e.g. an item appended, added, etc.) it is then moved into ``__dict__``. This removes the last of mutating side-effects on read-only attribute access within the ORM. Fixes: #4519 Change-Id: I06a058d24e6eb24b5c6b6092d3f8b31cf9c244ae
* Unify NO_VALUE and NEVER_SETMike Bayer2019-05-241-29/+32
| | | | | | | | | | | | There's no real difference between these two constants except they are used in different places and therefore allow various codepaths to work largely by accident. These codepaths should be explicit. Assign NO_VALUE and NEVER_SET to the same constant and work towards having just one constant for "we have no value to return right now". Fixes: #4696 Change-Id: I7c324967952c1886bf202074d627323a2ad013cc
* Include active_history when propagating attribute listenersMike Bayer2019-05-231-0/+2
| | | | | | | | | | Fixed issue where the :paramref:`.AttributeEvents.active_history` flag would not be set for an event listener that propgated to a subclass via the :paramref:`.AttributeEvents.propagate` flag. This bug has been present for the full span of the :class:`.AttributeEvents` system. Fixes: #4695 Change-Id: Ie384f4847f37c267d94b6d56e7538438efc1a54c
* Add QueryableAttribute._impl_uses_objects accessor for AssociationProxyMike Bayer2019-05-191-0/+11
| | | | | | | | | | | | | | Fixed regression where new association proxy system was still not proxying hybrid attributes when they made use of the ``@hybrid_property.expression`` decorator to return an alternate SQL expression, or when the hybrid returned an arbitrary :class:`.PropComparator`, at the expression level. This involved futher generalization of the heuristics used to detect the type of object being proxied at the level of :class:`.QueryableAttribute`, to better detect if the descriptor ultimately serves mapped classes or column expressions. Fixes: #4690 Change-Id: I5b5300661291c94a23de53bcf92d747701720aa1
* Implement new ClauseElement role and coercion systemMike Bayer2019-05-181-4/+0
| | | | | | | | | | | | | | | | | | | | A major refactoring of all the functions handle all detection of Core argument types as well as perform coercions into a new class hierarchy based on "roles", each of which identify a syntactical location within a SQL statement. In contrast to the ClauseElement hierarchy that identifies "what" each object is syntactically, the SQLRole hierarchy identifies the "where does it go" of each object syntactically. From this we define a consistent type checking and coercion system that establishes well defined behviors. This is a breakout of the patch that is reorganizing select() constructs to no longer be in the FromClause hierarchy. Also includes a rename of as_scalar() into scalar_subquery(); deprecates automatic coercion to scalar_subquery(). Partially-fixes: #4617 Change-Id: I26f1e78898693c6b99ef7ea2f4e7dfd0e8e1a1bd
* Note for history methods that history is reset per-flush.Mike Bayer2019-02-091-0/+18
| | | | Change-Id: I9bc4d0ddfa93f13e6717b89fa9934f1b8052147f
* Fix many spell glitchesLele Gaifax2019-01-251-2/+2
| | | | | | | | | | | | This affects mostly docstrings, except in orm/events.py::dispose_collection() where one parameter gets renamed: given that the method is empty, it seemed reasonable to me to fix that too. Closes: #4440 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4440 Pull-request-sha: 779ed75acb6142e1f1daac467b5b14134529bb4b Change-Id: Ic0553fe97853054b09c2453af76d96363de6eb0e
* Add deprecation warnings to all deprecated APIsMike Bayer2019-01-231-16/+23
| | | | | | | | | | | | | | | A large change throughout the library has ensured that all objects, parameters, and behaviors which have been noted as deprecated or legacy now emit ``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now defaults to displaying deprecation warnings, as well as that modern test suites based on tools like tox and pytest tend to display deprecation warnings, this change should make it easier to note what API features are obsolete. See the notes added to the changelog and migration notes for further details. Fixes: #4393 Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
* Remove version directives for 0.6, 0.7, 0.8Mike Bayer2019-01-151-2/+0
| | | | | | | | | - fix a few "seealso"s - ComparableProprerty's "superseded in 0.7" becomes deprecated in 0.7 Backport to currently maintained doc versions 1.2, 1.1 Change-Id: Ib1fcb2df8673dbe5c4ffc47f3896a60d1dfcb4b2
* happy new yearMike Bayer2019-01-111-1/+1
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-28/+32
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-274/+507
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Check collection less than two items remaining before firing scalar backref ↵Mike Bayer2018-12-281-6/+14
| | | | | | | | | | | | remove Fixed long-standing issue where duplicate collection members would cause a backref to delete the association between the member and its parent object when one of the duplicates were removed, as occurs as a side effect of swapping two objects in one statement. Fixes: #1103 Change-Id: Ic12877f7bd5a4eb688091725a78410748e7fdf16
* Call __del() before remove()Mike Bayer2018-12-281-0/+8
| | | | | | | | | | | | | | | | | | The "remove" event for collections is now called before the item is removed in the case of the ``collection.remove()`` method, as is consistent with the behavior for most other forms of collection item removal (such as ``__delitem__``, replacement under ``__setitem__``). The ``pop()`` methods are now the only exception as the target item is not available until after the pop operation proceeds. This allows ``remove()`` to be consistent in its behavior with all the other collection operations, allows the "before_delete" hook to be local to "pop()" operations only, and removes some method overhead. We are also looking here to gain some more predictability in terms of the fix for #1103. Change-Id: I4fdea911517d65cc300fae0e9c351a471e52e4ab
* Add new NO_RAISE attribute flag and specify for m2o history loadMike Bayer2018-11-021-4/+7
| | | | | | | | | Added new behavior to the lazy load that takes place when the "old" value of a many-to-one is retrieved, such that exceptions which would be raised due to either ``lazy="raise"`` or a detached session error are skipped. Fixes: #4353 Change-Id: I6c6c77613e93061a909f5062b70b17e8913fc9ee
* Implement __delete__Mike Bayer2018-11-021-10/+33
| | | | | | | | | | A long-standing oversight in the ORM, the ``__delete__`` method for a many- to-one relationship was non-functional, e.g. for an operation such as ``del a.b``. This is now implemented and is equivalent to setting the attribute to ``None``. Fixes: #4354 Change-Id: I60131a84c007b0bf6f20c5cc5f21a3b96e954046
* Unwrap Proxy objects when scanning declared_attrMike Bayer2018-08-231-0/+2
| | | | | | | | | | | | Fixed bug where the declarative scan for attributes would receive the expression proxy delivered by a hybrid attribute at the class level, and not the hybrid attribute itself, when receiving the descriptor via the ``@declared_attr`` callable on a subclass of an already-mapped class. This would lead to an attribute that did not report itself as a hybrid when viewed within :attr:`.Mapper.all_orm_descriptors`. Fixes: #4326 Change-Id: I582d03f05c3768b3344f93e3791240e9e69b9d1e
* Handle association proxy delete and provide for scalar delete cascadeMike Bayer2018-08-011-4/+11
| | | | | | | | | | | Fixed multiple issues regarding de-association of scalar objects with the association proxy. ``del`` now works, and additionally a new flag :paramref:`.AssociationProxy.cascade_scalar_deletes` is added, which when set to True indicates that setting a scalar attribute to ``None`` or deleting via ``del`` will also set the source association to ``None``. Change-Id: I1580d761571d63eb03a7e8df078cef97d265b85c Fixes: #4308
* Add initiator argument to set_attributeMike Bayer2018-02-081-2/+17
| | | | | | | | | Added new argument :paramref:`.attributes.set_attribute.inititator` to the :func:`.attributes.set_attribute` function, allowing an event token received from a listener function to be propagated to subsequent set events. Change-Id: I6ede21e42153026ab46a1d2ec33aa3f999db98e2
* Ensure backrefs accommodate for op_bulk_replaceMike Bayer2018-01-241-56/+60
| | | | | | | | | | | | | | | Fixed 1.2 regression regarding new bulk_replace event where a backref would fail to remove an object from the previous owner when a bulk-assignment assigned the object to a new owner. As this revisits the Event tokens associated with AttributeImpl objects, remove the verbosity of the "inline lazy init" pattern; the Event token is a simple slotted object and should have minimal memory overhead. Change-Id: Id188b4026fc2f3500186548008f4db8cdf7afecc Fixes: #4171
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Support state expiration for with_expression(); rename deferred_expressionMike Bayer2017-06-261-5/+10
| | | | | | | | | | | | | | | The attributeimpl for a deferred_expression does not support a scalar loader, add new configurability so that the impl can have this flag turned off. Document that the with_expression() system currently does not offer any deferred loading. To eliminate confusion over "deferred", which refers to lazy loading of column attributes, and "with_expression", which refers to an attribute that is explicitly at query time only, rename deferred_expression to query_expression. Change-Id: I07c4a050ed68c79ccbde9492e9de1630b7470d74
* Add AttributeEvents.modifiedMike Bayer2017-05-231-3/+12
| | | | | | | | | | Added new event handler :meth:`.AttributeEvents.modified` which is triggered when the func:`.attributes.flag_modified` function is invoked, which is common when using the :mod:`sqlalchemy.ext.mutable` extension module. Change-Id: Ic152f1d5c53087d780b24ed7f1f1571527b9e8fc Fixes: #3303
* Support hybrids/composites with bulk updatesMike Bayer2017-03-221-0/+5
| | | | | | | | | | | The :meth:`.Query.update` method can now accommodate both hybrid attributes as well as composite attributes as a source of the key to be placed in the SET clause. For hybrids, an additional decorator :meth:`.hybrid_property.update_expression` is supplied for which the user supplies a tuple-returning function. Change-Id: I15e97b02381d553f30b3301308155e19128d2cfb Fixes: #3229
* Raise on flag_modified() for non-present attributeMike Bayer2017-03-211-1/+35
| | | | | | | | | | | | The :func:`.attributes.flag_modified` function now raises :class:`.InvalidRequestError` if the named attribute key is not present within the object, as this is assumed to be present in the flush process. To mark an object "dirty" for a flush without referring to any specific attribute, the :func:`.attributes.flag_dirty` function may be used. Change-Id: I6c64e4d253c239e38632f38c27bb16e68fe8dfbe Fixes: #3753
* Add bulk_replace event, integrate with @validatesMike Bayer2017-03-161-7/+14
| | | | | | | | | | | | | | | | | | | | Added new attribute event :meth:`.AttributeEvents.bulk_replace`. This event is triggered when a collection is assigned to a relationship, before the incoming collection is compared with the existing one. This early event allows for conversion of incoming non-ORM objects as well. The event is integrated with the ``@validates`` decorator. The ``@validates`` decorator now allows the decorated method to receive objects from a "bulk collection set" operation that have not yet been compared to the existing collection. This allows incoming values to be converted to compatible ORM objects as is already allowed from an "append" event. Note that this means that the ``@validates`` method is called for **all** values during a collection assignment, rather than just the ones that are new. Change-Id: I27f59db008d9e521d31a3e30143d7cd997e4b7b3 Fixes: #3896
* Support python3.6Mike Bayer2017-01-131-1/+1
| | | | | | | | | | | Corrects some warnings and adds tox config. Adds DeprecationWarning to the error category. Large sweep for string literals w/ backslashes as this is common in docstrings Co-authored-by: Andrii Soldatenko Fixes: #3886 Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Use the "committed" values when extracting many-to-one lazyload valueMike Bayer2016-06-081-2/+6
| | | | | | | | | | | | | | | The scalar object set() method calls upon the lazy loader to get at the "old" value of the attriute, however doesn't ensure that the "committed" value of the foreign key attributes is used. If the user has manipulated these attributes and they themselves have pending, non committed changes, we get the "new" value which these attributes would have set up if they were flushed. "old" vs "new" value is always about how the value has changed since the load, so we always have to use the DB-persisted values for everything when looking for "old". Change-Id: I82bdc40ad0cf033c3a98f3361776cf3161542cd6 Fixes: #3708