Convert execution to move through Session

This patch replaces the ORM execution flow with a
single pathway through Session.execute() for all queries,
including Core and ORM.

Currently included is full support for ORM Query,
Query.from_statement(), select(), as well as the
baked query and horizontal shard systems.  Initial
changes have also been made to the dogpile caching
example, which like baked query makes use of a
new ORM-specific execution hook that replaces the
use of both QueryEvents.before_compile() as well
as Query._execute_and_instances() as the central
ORM interception hooks.

select() and Query() constructs alike can be passed to
Session.execute() where they will return ORM
results in a Results object.   This API is currently
used internally by Query.   Full support for
Session.execute()->results to behave in a fully
2.0 fashion will be in later changesets.

bulk update/delete with ORM support will also
be delivered via the update() and delete()
constructs, however these have not yet been adapted
to the new system and may follow in a subsequent
update.

Performance is also beginning to lag as of this
commit and some previous ones.   It is hoped that
a few central functions such as the coercions
functions can be rewritten in C to re-gain
performance.  Additionally, query caching
is now available and some subsequent patches
will attempt to cache more of the per-execution
work from the ORM layer, e.g. column getters
and adapters.

This patch also contains initial "turn on" of the
caching system enginewide via the query_cache_size
parameter to create_engine(). Still defaulting at
zero for "no caching".   The caching system still
needs adjustments in order to gain adequate performance.

Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d

1 files changed, 49 insertions, 0 deletions

diff --git a/lib/sqlalchemy/engine/default.py b/lib/sqlalchemy/engine/default.py
index e30daaeb8..b5cb2a1b2 100644
--- a/lib/sqlalchemy/engine/default.py
+++ b/lib/sqlalchemy/engine/default.py
@@ -16,6 +16,7 @@ as the base class for their own corresponding classes.
 import codecs
 import random
 import re
+import time
 import weakref
 
 from . import cursor as _cursor
@@ -226,6 +227,7 @@ class DefaultDialect(interfaces.Dialect):
         supports_native_boolean=None,
         max_identifier_length=None,
         label_length=None,
+        query_cache_size=0,
         # int() is because the @deprecated_params decorator cannot accommodate
         # the direct reference to the "NO_LINTING" object
         compiler_linting=int(compiler.NO_LINTING),
@@ -257,6 +259,10 @@ class DefaultDialect(interfaces.Dialect):
         if supports_native_boolean is not None:
             self.supports_native_boolean = supports_native_boolean
         self.case_sensitive = case_sensitive
+        if query_cache_size != 0:
+            self._compiled_cache = util.LRUCache(query_cache_size)
+        else:
+            self._compiled_cache = None
 
         self._user_defined_max_identifier_length = max_identifier_length
         if self._user_defined_max_identifier_length:
@@ -702,11 +708,17 @@ class DefaultExecutionContext(interfaces.ExecutionContext):
     result_column_struct = None
     returned_defaults = None
     execution_options = util.immutabledict()
+
+    cache_stats = None
+    invoked_statement = None
+
     _is_implicit_returning = False
     _is_explicit_returning = False
     _is_future_result = False
     _is_server_side = False
 
+    _soft_closed = False
+
     # a hook for SQLite's translation of
     # result column names
     # NOTE: pyhive is using this hook, can't remove it :(
@@ -1011,6 +1023,16 @@ class DefaultExecutionContext(interfaces.ExecutionContext):
         self.cursor = self.create_cursor()
         return self
 
+    def _get_cache_stats(self):
+        if self.compiled is None:
+            return "raw SQL"
+
+        now = time.time()
+        if self.compiled.cache_key is None:
+            return "gen %.5fs" % (now - self.compiled._gen_time,)
+        else:
+            return "cached %.5fs" % (now - self.compiled._gen_time,)
+
     @util.memoized_property
     def engine(self):
         return self.root_connection.engine
@@ -1234,6 +1256,33 @@ class DefaultExecutionContext(interfaces.ExecutionContext):
         ):
             self._setup_out_parameters(result)
 
+        if not self._is_future_result:
+            conn = self.root_connection
+            assert not conn._is_future
+
+            if not result._soft_closed and conn.should_close_with_result:
+                result._autoclose_connection = True
+
+        self._soft_closed = result._soft_closed
+
+        # result rewrite/ adapt step.  two translations can occur here.
+        # one is if we are invoked against a cached statement, we want
+        # to rewrite the ResultMetaData to reflect the column objects
+        # that are in our current selectable, not the cached one.  the
+        # other is, the CompileState can return an alternative Result
+        # object.   Finally, CompileState might want to tell us to not
+        # actually do the ResultMetaData adapt step if it in fact has
+        # changed the selected columns in any case.
+        compiled = self.compiled
+        if compiled:
+            adapt_metadata = (
+                result._metadata_from_cache
+                and not compiled._rewrites_selected_columns
+            )
+
+            if adapt_metadata:
+                result._metadata = result._metadata._adapt_to_context(self)
+
         return result
 
     def _setup_out_parameters(self, result):