diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/build/changelog/changelog_11.rst | 44 | ||||
| -rw-r--r-- | doc/build/changelog/migration_11.rst | 90 | ||||
| -rw-r--r-- | doc/build/glossary.rst | 23 | ||||
| -rw-r--r-- | doc/build/orm/session_events.rst | 237 | ||||
| -rw-r--r-- | doc/build/orm/session_state_management.rst | 117 |
5 files changed, 457 insertions, 54 deletions
diff --git a/doc/build/changelog/changelog_11.rst b/doc/build/changelog/changelog_11.rst index 4fff4ab64..27dc8fd46 100644 --- a/doc/build/changelog/changelog_11.rst +++ b/doc/build/changelog/changelog_11.rst @@ -22,6 +22,50 @@ :version: 1.1.0b1 .. change:: + :tags: feature, orm + :tickets: 2677 + + The :class:`.SessionEvents` suite now includes events to allow + unambiguous tracking of all object lifecycle state transitions + in terms of the :class:`.Session` itself, e.g. pending, + transient, persistent, detached. The state of the object + within each event is also defined. + + .. seealso:: + + :ref:`change_2677` + + .. change:: + :tags: feature, orm + :tickets: 2677 + + Added a new session lifecycle state :term:`deleted`. This new state + represents an object that has been deleted from the :term:`persistent` + state and will move to the :term:`detached` state once the transaction + is committed. This resolves the long-standing issue that objects + which were deleted existed in a gray area between persistent and + detached. The :attr:`.InstanceState.persistent` accessor will + **no longer** report on a deleted object as persistent; the + :attr:`.InstanceState.deleted` accessor will instead be True for + these objects, until they become detached. + + .. seealso:: + + :ref:`change_2677` + + .. change:: + :tags: change, orm + :tickets: 2677 + + The :paramref:`.Session.weak_identity_map` parameter is deprecated. + See the new recipe at :ref:`session_referencing_behavior` for + an event-based approach to maintaining strong identity map behavior. + + .. seealso:: + + :ref:`change_2677` + + .. change:: :tags: bug, sql :tickets: 2919 diff --git a/doc/build/changelog/migration_11.rst b/doc/build/changelog/migration_11.rst index 412f42d27..ff26a00bd 100644 --- a/doc/build/changelog/migration_11.rst +++ b/doc/build/changelog/migration_11.rst @@ -16,7 +16,7 @@ What's New in SQLAlchemy 1.1? some issues may be moved to later milestones in order to allow for a timely release. - Document last updated: August 26, 2015 + Document last updated: September 2, 2015 Introduction ============ @@ -66,6 +66,94 @@ as it relies on deprecated features of setuptools. New Features and Improvements - ORM =================================== +.. _change_2677: + +New Session lifecycle events +---------------------------- + +The :class:`.Session` has long supported events that allow some degree +of tracking of state changes to objects, including +:meth:`.SessionEvents.before_attach`, :meth:`.SessionEvents.after_attach`, +and :meth:`.SessionEvents.before_flush`. The Session documentation also +documents major object states at :ref:`session_object_states`. However, +there has never been system of tracking objects specifically as they +pass through these transitions. Additionally, the status of "deleted" objects +has historically been murky as the objects act somewhere between +the "persistent" and "detached" states. + +To clean up this area and allow the realm of session state transition +to be fully transparent, a new series of events have been added that +are intended to cover every possible way that an object might transition +between states, and additionally the "deleted" status has been given +its own official state name within the realm of session object states. + +New State Transition Events +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Transitions between all states of an object such as :term:`persistent`, +:term:`pending` and others can now be intercepted in terms of a +session-level event intended to cover a specific transition. +Transitions as objects move into a :class:`.Session`, move out of a +:class:`.Session`, and even all the transitions which occur when the +transaction is rolled back using :meth:`.Session.rollback` +are explicitly present in the interface of :class:`.SessionEvents`. + +In total, there are **ten new events**. A summary of these events is in a +newly written documentation section :ref:`session_lifecycle_events`. + + +New Object State "deleted" is added, deleted objects no longer "persistent" +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The :term:`persistent` state of an object in the :class:`.Session` has +always been documented as an object that has a valid database identity; +however in the case of objects that were deleted within a flush, they +have always been in a grey area where they are not really "detached" +from the :class:`.Session` yet, because they can still be restored +within a rollback, but are not really "persistent" because their database +identity has been deleted and they aren't present in the identity map. + +To resolve this grey area given the new events, a new object state +:term:`deleted` is introduced. This state exists between the "persistent" and +"detached" states. An object that is marked for deletion via +:meth:`.Session.delete` remains in the "persistent" state until a flush +proceeds; at that point, it is removed from the identity map, moves +to the "deleted" state, and the :meth:`.SessionEvents.persistent_to_deleted` +hook is invoked. If the :class:`.Session` object's transaction is rolled +back, the object is restored as persistent; the +:meth:`.SessionEvents.deleted_to_persistent` transition is called. Otherwise +if the :class:`.Session` object's transaction is committed, +the :meth:`.SessionEvents.deleted_to_detached` transition is invoked. + +Additionally, the :attr:`.InstanceState.persistent` accessor **no longer returns +True** for an object that is in the new "deleted" state; instead, the +:attr:`.InstanceState.deleted` accessor has been enhanced to reliably +report on this new state. When the object is detached, the :attr:`.InstanceState.deleted` +returns False and the :attr:`.InstanceState.detached` accessor is True +instead. To determine if an object was deleted either in the current +transaction or in a previous transaction, use the +:attr:`.InstanceState.was_deleted` accessor. + +Strong Identity Map is Deprecated +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +One of the inspirations for the new series of transition events was to enable +leak-proof tracking of objects as they move in and out of the identity map, +so that a "strong reference" may be maintained mirroring the object +moving in and out of this map. With this new capability, there is no longer +any need for the :paramref:`.Session.weak_identity_map` parameter and the +corresponding :class:`.StrongIdentityMap` object. This option has remained +in SQLAlchemy for many years as the "strong-referencing" behavior used to be +the only behavior available, and many applications were written to assume +this behavior. It has long been recommended that strong-reference tracking +of objects not be an intrinsic job of the :class:`.Session` and instead +be an application-level construct built as needed by the application; the +new event model allows even the exact behavior of the strong identity map +to be replicated. See :ref:`session_referencing_behavior` for a new +recipe illustrating how to replace the strong identity map. + +:ticket:`2677` + .. _change_3499: Changes regarding "unhashable" types diff --git a/doc/build/glossary.rst b/doc/build/glossary.rst index c0ecee84b..9c1395f14 100644 --- a/doc/build/glossary.rst +++ b/doc/build/glossary.rst @@ -1019,7 +1019,7 @@ Glossary http://en.wikipedia.org/wiki/Unique_key#Defining_unique_keys transient - This describes one of the four major object states which + This describes one of the major object states which an object can have within a :term:`session`; a transient object is a new object that doesn't have any database identity and has not been associated with a session yet. When the @@ -1031,7 +1031,7 @@ Glossary :ref:`session_object_states` pending - This describes one of the four major object states which + This describes one of the major object states which an object can have within a :term:`session`; a pending object is a new object that doesn't have any database identity, but has been recently associated with a session. When @@ -1042,8 +1042,23 @@ Glossary :ref:`session_object_states` + deleted + This describes one of the major object states which + an object can have within a :term:`session`; a deleted object + is an object that was formerly persistent and has had a + DELETE statement emitted to the database within a flush + to delete its row. The object will move to the :term:`detached` + state once the session's transaction is committed; alternatively, + if the session's transaction is rolled back, the DELETE is + reverted and the object moves back to the :term:`persistent` + state. + + .. seealso:: + + :ref:`session_object_states` + persistent - This describes one of the four major object states which + This describes one of the major object states which an object can have within a :term:`session`; a persistent object is an object that has a database identity (i.e. a primary key) and is currently associated with a session. Any object @@ -1058,7 +1073,7 @@ Glossary :ref:`session_object_states` detached - This describes one of the four major object states which + This describes one of the major object states which an object can have within a :term:`session`; a detached object is an object that has a database identity (i.e. a primary key) but is not associated with any session. An object that diff --git a/doc/build/orm/session_events.rst b/doc/build/orm/session_events.rst index 231311aa0..dc7ad39e0 100644 --- a/doc/build/orm/session_events.rst +++ b/doc/build/orm/session_events.rst @@ -148,24 +148,239 @@ Object Lifecycle Events Another use case for events is to track the lifecycle of objects. This refers to the states first introduced at :ref:`session_object_states`. -As of SQLAlchemy 1.0, there is no direct event interface for tracking of -these states. Events that can be used at the moment to track the state of -objects include: +.. versionadded:: 1.1 added a system of events that intercept all possible + state transitions of an object within the :class:`.Session`. -* :meth:`.InstanceEvents.init` +All the states above can be tracked fully with events. Each event +represents a distinct state transition, meaning, the starting state +and the destination state are both part of what are tracked. With the +exception of the initial transient event, all the events are in terms of +the :class:`.Session` object or class, meaning they can be associated either +with a specific :class:`.Session` object:: -* :meth:`.InstanceEvents.load` + from sqlalchemy import event + from sqlalchemy.orm import Session -* :meth:`.SessionEvents.before_attach` + session = Session() -* :meth:`.SessionEvents.after_attach` + @event.listens_for(session, 'transient_to_pending') + def object_is_pending(session, obj): + print("new pending: %s" % obj) -* :meth:`.SessionEvents.before_flush` - by scanning the session's collections +Or with the :class:`.Session` class itself, as well as with a specific +:class:`.sessionmaker`, which is likely the most useful form:: -* :meth:`.SessionEvents.after_flush` - by scanning the session's collections + from sqlalchemy import event + from sqlalchemy.orm import sessionmaker -SQLAlchemy 1.1 will introduce a comprehensive event system to track -the object persistence states fully and unambiguously. + maker = sessionmaker() + + @event.listens_for(maker, 'transient_to_pending') + def object_is_pending(session, obj): + print("new pending: %s" % obj) + +The listeners can of course be stacked on top of one function, as is +likely to be common. For example, to track all objects that are +entering the persistent state:: + + @event.listens_for(maker, "pending_to_persistent") + @event.listens_for(maker, "deleted_to_persistent") + @event.listens_for(maker, "detached_to_persistent") + @event.listens_for(maker, "loaded_as_persistent") + def detect_all_persistent(session, instance): + print("object is now persistent: %s" % instance) + +Transient +^^^^^^^^^ + +All mapped objects when first constructed start out as :term:`transient`. +In this state, the object exists alone and doesn't have an association with +any :class:`.Session`. For this initial state, there's no specific +"transition" event since there is no :class:`.Session`, however if one +wanted to intercept when any transient object is created, the +:meth:`.InstanceEvents.init` method is probably the best event. This +event is applied to a specific class or superclass. For example, to +intercept all new objects for a particular declarative base:: + + from sqlalchemy.ext.declarative import declarative_base + from sqlalchemy import event + + Base = declarative_base() + + @event.listens_for(Base, "init", propagate=True) + def intercept_init(instance, args, kwargs): + print("new transient: %s" % instance) + + +Transient to Pending +^^^^^^^^^^^^^^^^^^^^ + +The transient object becomes :term:`pending` when it is first associated +with a :class:`.Session` via the :meth:`.Session.add` or :meth:`.Session.add_all` +method. An object may also become part of a :class:`.Session` as a result +of a :ref:`"cascade" <unitofwork_cascades>` from a referencing object that was +explicitly added. The transient to pending transition is detectable using +the :meth:`.SessionEvents.transient_to_pending` event:: + + @event.listens_for(sessionmaker, "transient_to_pending") + def intercept_transient_to_pending(session, object_): + print("transient to pending: %s" % object_) + + +Pending to Persistent +^^^^^^^^^^^^^^^^^^^^^ + +The :term:`pending` object becomes :term:`persistent` when a flush +proceeds and an INSERT statement takes place for the instance. The object +now has an identity key. Track pending to persistent with the +:meth:`.SessionEvents.pending_to_persistent` event:: + + @event.listens_for(sessionmaker, "pending_to_persistent") + def intercept_pending_to_persistent(session, object_): + print("pending to persistent: %s" % object_) + +Pending to Transient +^^^^^^^^^^^^^^^^^^^^^^^ + +The :term:`pending` object can revert back to :term:`transient` if the +:meth:`.Session.rollback` method is called before the pending object +has been flushed, or if the :meth:`.Session.expunge` method is called +for the object before it is flushed. Track pending to transient with the +:meth:`.SessionEvents.pending_to_transient` event:: + + @event.listens_for(sessionmaker, "pending_to_transient") + def intercept_pending_to_transient(session, object_): + print("transient to pending: %s" % object_) + +Loaded as Persistent +^^^^^^^^^^^^^^^^^^^^^^^ + +Objects can appear in the :class:`.Session` directly in the :term:`persistent` +state when they are loaded from the database. Tracking this state transition +is synonymous with tracking objects as they are loaded, and is synonomous +with using the :meth:`.InstanceEvents.load` instance-level event. However, the +:meth:`.SessionEvents.loaded_as_persistent` event is provided as a +session-centric hook for intercepting objects as they enter the persistent +state via this particular avenue:: + + @event.listens_for(sessionmaker, "loaded_as_persistent") + def intercept_loaded_as_persistent(session, object_): + print("object loaded into persistent state: %s" % object_) + + +Persistent to Transient +^^^^^^^^^^^^^^^^^^^^^^^ + +The persistent object can revert to the transient state if the +:meth:`.Session.rollback` method is called for a transaction where the +object was first added as pending. In the case of the ROLLBACK, the +INSERT statement that made this object persistent is rolled back, and +the object is evicted from the :class:`.Session` to again become transient. +Track objects that were reverted to transient from +persistent using the :meth:`.SessionEvents.persistent_to_transient` +event hook:: + + @event.listens_for(sessionmaker, "persistent_to_transient") + def intercept_persistent_to_transient(session, object_): + print("persistent to transient: %s" % object_) + +Persistent to Deleted +^^^^^^^^^^^^^^^^^^^^^ + +The persistent object enters the :term:`deleted` state when an object +marked for deletion is deleted from the database within the flush +process. Note that this is **not the same** as when the :meth:`.Session.delete` +method is called for a target object. The :meth:`.Session.delete` +method only **marks** the object for deletion; the actual DELETE statement +is not emitted until the flush proceeds. It is subsequent to the flush +that the "deleted" state is present for the target object. + +Within the "deleted" state, the object is only marginally associated +with the :class:`.Session`. It is not present in the identity map +nor is it present in the :attr:`.Session.deleted` collection that refers +to when it was pending for deletion. + +From the "deleted" state, the object can go either to the detached state +when the transaction is committed, or back to the persistent state +if the transaction is instead rolled back. + +Track the persistent to deleted transition with +:meth:`.SessionEvents.persistent_to_deleted`:: + + @event.listens_for(sessionmaker, "persistent_to_deleted") + def intercept_persistent_to_deleted(session, object_): + print("object was DELETEd, is now in deleted state: %s" % object_) + + +Deleted to Detached +^^^^^^^^^^^^^^^^^^^^ + +The deleted object becomes :term:`detached` when the session's transaction +is committed. After the :meth:`.Session.commit` method is called, the +database transaction is final and the :class:`.Session` now fully discards +the deleted object and removes all associations to it. Track +the deleted to detached transition using :meth:`.SessionEvents.deleted_to_detached`:: + + @event.listens_for(sessionmaker, "deleted_to_detached") + def intercept_deleted_to_detached(session, object_): + print("deleted to detached: %s" % object_) + + +.. note:: + + While the object is in the deleted state, the :attr:`.InstanceState.deleted` + attribute, accessible using ``inspect(object).deleted``, returns True. However + when the object is detached, :attr:`.InstanceState.deleted` will again + return False. To detect that an object was deleted, regardless of whether + or not it is detached, use the :attr:`.InstanceState.was_deleted` + accessor. + + +Persistent to Detached +^^^^^^^^^^^^^^^^^^^^^^^ + +The persistent object becomes :term:`detached` when the object is de-associated +with the :class:`.Session`, via the :meth:`.Session.expunge`, +:meth:`.Session.expunge_all`, or :meth:`.Session.close` methods. + +.. note:: + + An object may also become **implicitly detached** if its owning + :class:`.Session` is dereferenced by the application and discarded due to + garbage collection. In this case, **no event is emitted**. + +Track objects as they move from persistent to detached using the +:meth:`.SessionEvents.persistent_to_detached` event:: + + @event.listens_for(sessionmaker, "persistent_to_detached") + def intecept_persistent_to_detached(session, object_): + print("object became detached: %s" % object_) + +Detached to Persistent +^^^^^^^^^^^^^^^^^^^^^^^ + +The detached object becomes persistent when it is re-associated with a +session using the :meth:`.Session.add` or equivalent method. Track +objects moving back to persistent from detached using the +:meth:`.SessionEvents.detached_to_persistent` event:: + + @event.listens_for(sessionmaker, "detached_to_persistent") + def intecept_detached_to_persistent(session, object_): + print("object became persistent again: %s" % object_) + + +Deleted to Persistent +^^^^^^^^^^^^^^^^^^^^^^^ + +The :term:`deleted` object can be reverted to the :term:`persistent` +state when the transaction in which it was DELETEd was rolled back +using the :meth:`.Session.rollback` method. Track deleted objects +moving back to the persistent state using the +:meth:`.SessionEvents.deleted_to_persistent` event:: + + @event.listens_for(sessionmaker, "transient_to_pending") + def intercept_transient_to_pending(session, object_): + print("transient to pending: %s" % object_) .. _session_transaction_events: diff --git a/doc/build/orm/session_state_management.rst b/doc/build/orm/session_state_management.rst index 18673fde1..090bf7674 100644 --- a/doc/build/orm/session_state_management.rst +++ b/doc/build/orm/session_state_management.rst @@ -23,16 +23,15 @@ It's helpful to know the states which an instance can have within a session: existing instances (or moving persistent instances from other sessions into your local session). - .. note:: +* **Deleted** - An instance which has been deleted within a flush, but + the transaction has not yet completed. Objects in this state are essentially + in the opposite of "pending" state; when the session's transaction is committed, + the object will move to the detached state. Alternatively, when + the session's transaction is rolled back, a deleted object moves + *back* to the persistent state. - An object that is marked as deleted, e.g. via the - :meth:`.Session.delete` method, is still considered persistent. The - object remains in the identity map until the flush proceeds and a DELETE - state is emitted, at which point the object moves to the state that is - for most practical purposes "detached" - after the session's transaction - is committed, the object becomes fully detached. SQLAlchemy 1.1 will - introduce a new object state called "deleted" which represents - this "deleted but not quite detached" state explicitly. + .. versionchanged:: 1.1 The 'deleted' state is a newly added session + object state distinct from the 'persistent' state. * **Detached** - an instance which corresponds, or previously corresponded, to a record in the database, but is not currently in any session. @@ -43,10 +42,9 @@ It's helpful to know the states which an instance can have within a session: load unloaded attributes or attributes that were previously marked as "expired". -Knowing these states is important, since the -:class:`.Session` tries to be strict about ambiguous -operations (such as trying to save the same object to two different sessions -at the same time). +For a deeper dive into all possible state transitions, see the +section :ref:`session_lifecycle_events` which describes each transition +as well as how to programmatically track each one. Getting the Current State of an Object ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -67,6 +65,8 @@ the :func:`.inspect` system:: :attr:`.InstanceState.persistent` + :attr:`.InstanceState.deleted` + :attr:`.InstanceState.detached` .. _session_attributes: @@ -107,7 +107,13 @@ all objects which have had changes since they were last loaded or saved (i.e. (Documentation: :attr:`.Session.new`, :attr:`.Session.dirty`, :attr:`.Session.deleted`, :attr:`.Session.identity_map`). -Note that objects within the session are *weakly referenced*. This + +.. _session_referencing_behavior: + +Session Referencing Behavior +---------------------------- + +Objects within the session are *weakly referenced*. This means that when they are dereferenced in the outside application, they fall out of scope from within the :class:`~sqlalchemy.orm.session.Session` as well and are subject to garbage collection by the Python interpreter. The @@ -116,30 +122,65 @@ as deleted, or persistent objects which have pending changes on them. After a full flush, these collections are all empty, and all objects are again weakly referenced. -.. note:: - - To disable the weak referencing behavior and force all objects - within the session to remain until explicitly expunged, configure - :class:`.sessionmaker` with the ``weak_identity_map=False`` - setting. However note that this option is **deprecated**; - it is present only to allow compatibility with older - applications, typically those that were made back before SQLAlchemy - had the ability to effectively weak-reference all objects. - It is recommended that strong references to objects - be maintained by the calling application externally to the - :class:`.Session` itself, to the extent that is required by the application. - This eliminates the - :class:`.Session` as a possible source of unbounded memory growth in the case - where large numbers of objects are being loaded and/or persisted. - - Simple examples of externally managed strong-referencing behavior - include loading objects into a local dictionary keyed to their primary key, - or into lists or sets for the span of time that they need to remain referenced. - These collections can be associated with a :class:`.Session`, if desired, - by placing them into the :attr:`.Session.info` dictionary. Events such - as the :meth:`.SessionEvents.after_attach` and :meth:`.MapperEvents.load` - event may also be of use for intercepting objects as they are associated - with a :class:`.Session`. +To cause objects in the :class:`.Session` to remain strongly +referenced, usually a simple approach is all that's needed. Examples +of externally managed strong-referencing behavior include loading +objects into a local dictionary keyed to their primary key, or into +lists or sets for the span of time that they need to remain +referenced. These collections can be associated with a +:class:`.Session`, if desired, by placing them into the +:attr:`.Session.info` dictionary. + +An event based approach is also feasable. A simple recipe that provides +"strong referencing" behavior for all objects as they remain within +the :term:`persistent` state is as follows:: + + from sqlalchemy import event + + def strong_reference_session(session): + @event.listens_for(session, "pending_to_persistent") + @event.listens_for(session, "deleted_to_persistent") + @event.listens_for(session, "detached_to_persistent") + @event.listens_for(session, "loaded_as_persistent") + def strong_ref_object(sess, instance): + if 'refs' not in sess.info: + sess.info['refs'] = refs = set() + else: + refs = sess.info['refs'] + + refs.add(instance) + + + @event.listens_for(session, "persistent_to_detached") + @event.listens_for(session, "persistent_to_deleted") + @event.listens_for(session, "persistent_to_transient") + def deref_object(sess, instance): + sess.info['refs'].discard(instance) + +Above, we intercept the :meth:`.SessionEvents.pending_to_persistent`, +:meth:`.SessionEvents.detached_to_persistent`, +:meth:`.SessionEvents.deleted_to_persistent` and +:meth:`.SessionEvents.loaded_as_persistent` event hooks in order to intercept +objects as they enter the :term:`persistent` transition, and the +:meth:`.SessionEvents.persistent_to_detached` and +:meth:`.SessionEvents.persistent_to_deleted` hooks to intercept +objects as they leave the persistent state. + +The above function may be called for any :class:`.Session` in order to +provide strong-referencing behavior on a per-:class:`.Session` basis:: + + from sqlalchemy.orm import Session + + my_session = Session() + strong_reference_session(my_session) + +It may also be called for any :class:`.sessionmaker`:: + + from sqlalchemy.orm import sessionmaker + + maker = sessionmaker() + strong_reference_session(maker) + .. _unitofwork_merging: |