Major revisals to lambdas

1. Improve coercions._deep_is_literal to check sequences
for clause elements, thus allowing a phrase like
lambda: col.in_([literal("x"), literal("y")]) to be handled

2. revise closure variable caching completely.   All variables
entering must be part of a closure cache key or rejected.
only objects that can be resolved to HasCacheKey or FunctionType
are accepted; all other types are rejected.  This adds a high
degree of strictness to lambdas and will make them a little more
awkward to use in some cases, however prevents several classes
of critical issues:

a. previously, a lambda that had an expression derived from
some kind of state, like "self.x", or "execution_context.session.foo"
would produce a closure cache key from "self" or "execution_context",
objects that can very well be per-execution and would therefore
cause a AnalyzedFunction objects to overflow. (memory won't leak
as it looks like an LRUCache is already used for these)

b. a lambda, such as one used within DeferredLamdaElement, that
produces different SQL expressions based on the arguments
(which is in fact what it's supposed to do), however it would
through the use of conditionals produce different bound parameter
combinations, leading to literal parameters not tracked properly.
These are now rejected as uncacheable whereas previously they would
again be part of the closure cache key, causing an overflow of
AnalyizedFunction objects.

3. Ensure non-mapped mixins are handled correctly by
with_loader_criteria().

4. Fixed bug in lambda SQL system where we are not supposed to allow a Python
function to be embedded in the lambda, since we can't predict a bound value
from it.   While there was an error condition added for this, it was not
tested and wasn't working; an informative error is now raised.

5. new docs for lambdas

6. consolidated changelog for all of these

Fixes: #5760
Fixes: #5765
Fixes: #5766
Fixes: #5768
Fixes: #5770
Change-Id: Iedaa636c3225fad496df23b612c516c8ab247ab7

1 files changed, 10 insertions, 4 deletions

diff --git a/lib/sqlalchemy/sql/coercions.py b/lib/sqlalchemy/sql/coercions.py
index bdd807438..43c89ee82 100644
--- a/lib/sqlalchemy/sql/coercions.py
+++ b/lib/sqlalchemy/sql/coercions.py
@@ -7,7 +7,6 @@
 
 import numbers
 import re
-import types
 
 from . import operators
 from . import roles
@@ -56,6 +55,15 @@ def _deep_is_literal(element):
 
     """
 
+    if isinstance(element, collections_abc.Sequence) and not isinstance(
+        element, str
+    ):
+        for elem in element:
+            if not _deep_is_literal(elem):
+                return False
+        else:
+            return True
+
     return (
         not isinstance(
             element,
@@ -66,7 +74,6 @@ def _deep_is_literal(element):
             not isinstance(element, type)
             or not issubclass(element, HasCacheKey)
         )
-        and not isinstance(element, types.FunctionType)
     )
 
 
@@ -109,9 +116,8 @@ def expect(role, element, apply_propagate_attrs=None, argname=None, **kw):
         return lambdas.LambdaElement(
             element,
             role,
+            lambdas.LambdaOptions(**kw),
             apply_propagate_attrs=apply_propagate_attrs,
-            argname=argname,
-            **kw
         )
 
     # major case is that we are given a ClauseElement already, skip more