introduce deferred lambdas

The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all.   By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.

As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.

Adds some more short_selects tests for the moment for comparison.

Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.

As we have not yet closed #4639, however the caching feature
has been fully integrated as of
b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.

Closes: #4639
Fixes: #5380
Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba

1 files changed, 22 insertions, 6 deletions

diff --git a/lib/sqlalchemy/sql/roles.py b/lib/sqlalchemy/sql/roles.py
index 3d94ec9ff..4205d9f0d 100644
--- a/lib/sqlalchemy/sql/roles.py
+++ b/lib/sqlalchemy/sql/roles.py
@@ -19,9 +19,21 @@ class SQLRole(object):
 
     """
 
+    allows_lambda = False
+    uses_inspection = False
+
 
 class UsesInspection(object):
     _post_inspect = None
+    uses_inspection = True
+
+
+class AllowsLambdaRole(object):
+    allows_lambda = True
+
+
+class HasCacheKeyRole(SQLRole):
+    _role_name = "Cacheable Core or ORM object"
 
 
 class ColumnArgumentRole(SQLRole):
@@ -40,7 +52,7 @@ class TruncatedLabelRole(SQLRole):
     _role_name = "String SQL identifier"
 
 
-class ColumnsClauseRole(UsesInspection, ColumnListRole):
+class ColumnsClauseRole(AllowsLambdaRole, UsesInspection, ColumnListRole):
     _role_name = "Column expression or FROM clause"
 
     @property
@@ -56,7 +68,7 @@ class ByOfRole(ColumnListRole):
     _role_name = "GROUP BY / OF / etc. expression"
 
 
-class GroupByRole(UsesInspection, ByOfRole):
+class GroupByRole(AllowsLambdaRole, UsesInspection, ByOfRole):
     # note there's a special case right now where you can pass a whole
     # ORM entity to group_by() and it splits out.   we may not want to keep
     # this around
@@ -64,7 +76,7 @@ class GroupByRole(UsesInspection, ByOfRole):
     _role_name = "GROUP BY expression"
 
 
-class OrderByRole(ByOfRole):
+class OrderByRole(AllowsLambdaRole, ByOfRole):
     _role_name = "ORDER BY expression"
 
 
@@ -76,7 +88,11 @@ class StatementOptionRole(StructuralRole):
     _role_name = "statement sub-expression element"
 
 
-class WhereHavingRole(StructuralRole):
+class OnClauseRole(AllowsLambdaRole, StructuralRole):
+    _role_name = "SQL expression for ON clause"
+
+
+class WhereHavingRole(OnClauseRole):
     _role_name = "SQL expression for WHERE/HAVING role"
 
 
@@ -102,7 +118,7 @@ class InElementRole(SQLRole):
     )
 
 
-class JoinTargetRole(UsesInspection, StructuralRole):
+class JoinTargetRole(AllowsLambdaRole, UsesInspection, StructuralRole):
     _role_name = (
         "Join target, typically a FROM expression, or ORM "
         "relationship attribute"
@@ -176,7 +192,7 @@ class HasCTERole(ReturnsRowsRole):
     pass
 
 
-class CompoundElementRole(SQLRole):
+class CompoundElementRole(AllowsLambdaRole, SQLRole):
     """SELECT statements inside a CompoundSelect, e.g. UNION, EXTRACT, etc."""
 
     _role_name = (