- Fixed some "double invalidate" situations were detected where

a connection invalidation could occur within an already critical section
like a connection.close(); ultimately, these conditions are caused
by the change in :ticket:`2907`, in that the "reset on return" feature
calls out to the Connection/Transaction in order to handle it, where
"disconnect detection" might be caught.  However, it's possible that
the more recent change in :ticket:`2985` made it more likely for this
to be seen as the "connection invalidate" operation is much quicker,
as the issue is more reproducible on 0.9.4 than 0.9.3.

Checks are now added within any section that
an invalidate might occur to halt further disallowed operations
on the invalidated connection.  This includes two fixes both at the
engine level and at the pool level.   While the issue was observed
with highly concurrent gevent cases, it could in theory occur in
any kind of scenario where a disconnect occurs within the connection
close operation.
fixes #3043
ref #2985
ref #2907

- add some defensive checks during an invalidate situation:
1. _ConnectionRecord.invalidate might be called twice within finalize_fairy
if the _reset() raises an invalidate condition, invalidates, raises and then
goes to invalidate the CR.  so check for this.
2. similarly within Conneciton, anytime we do handle_dbapi_error(), we might become invalidated.
so a following finally must check self.__invalid before dealing with the connection
any futher.

1 files changed, 10 insertions, 3 deletions

diff --git a/lib/sqlalchemy/engine/base.py b/lib/sqlalchemy/engine/base.py
index bb3b82eea..997991b30 100644
--- a/lib/sqlalchemy/engine/base.py
+++ b/lib/sqlalchemy/engine/base.py
@@ -507,7 +507,8 @@ class Connection(Connectable):
             except Exception as e:
                 self._handle_dbapi_exception(e, None, None, None, None)
             finally:
-                if self.connection._reset_agent is self.__transaction:
+                if not self.__invalid and \
+                        self.connection._reset_agent is self.__transaction:
                     self.connection._reset_agent = None
                 self.__transaction = None
         else:
@@ -524,7 +525,8 @@ class Connection(Connectable):
         except Exception as e:
             self._handle_dbapi_exception(e, None, None, None, None)
         finally:
-            if self.connection._reset_agent is self.__transaction:
+            if not self.__invalid and \
+                    self.connection._reset_agent is self.__transaction:
                 self.connection._reset_agent = None
             self.__transaction = None
 
@@ -637,7 +639,12 @@ class Connection(Connectable):
                 conn.close()
             if conn._reset_agent is self.__transaction:
                 conn._reset_agent = None
-            del self.__connection
+
+            # the close() process can end up invalidating us,
+            # as the pool will call our transaction as the "reset_agent"
+            # for rollback(), which can then cause an invalidation
+            if not self.__invalid:
+                del self.__connection
         self.__can_reconnect = False
         self.__transaction = None