Improve how connection pools operate in forked/child proceeses.

Sometimes a process with an active connection to Redis forks and creates
child processes taht also want to talk to Redis. Prior to this change there
were a number of potential conflicts that could cause this to fail.

Retrieving a connection from the pool and releasing a connection back
to the pool check the current proceeses PID. If it's different than the
PID that created the pool, reset() is called to get a fresh set of connections
for the current process. However in doing so, pool.disconnect() was caused
which closes the file descriptors that the parent may still be using. Further
when the available_connections and in_use_connections lists are reset, all of
those connections inherited from the parent are GC'd and the connection's
`__del__` was called, which also closed the socket and file descriptor.

This change prevents pool.disconnect() from being called when a pid is changed.
It also removes the `__del__` destructor from connections. Neither of these
are necessary or practical. Child processes still reset() their copy of the
pool when first accessed causing their own connections to be created.

`ConnectionPool.disconnect()` now checks the current process ID
so that a child or parent can't disconnect the other's connections.

Additionally, `Connection.disconnect()` now checks the current process ID
and only calls `socket.shutdown()` if `disconnect()` is called by the same
process that created the connection. This allows for a child process that
inherited a connection to call `Connection.disconnect()` and not shutdown
the parent's copy of the socket.

Fixes #863
Fixes #784
Fixes #732
Fixes #1085
Fixes #504

1 files changed, 4 insertions, 8 deletions

diff --git a/redis/connection.py b/redis/connection.py
index c81e4c1..ee0b92a 100755
--- a/redis/connection.py
+++ b/redis/connection.py
@@ -471,12 +471,6 @@ class Connection(object):
     def __repr__(self):
         return self.description_format % self._description_args
 
-    def __del__(self):
-        try:
-            self.disconnect()
-        except Exception:
-            pass
-
     def register_connect_callback(self, callback):
         self._connect_callbacks.append(callback)
 
@@ -580,7 +574,8 @@ class Connection(object):
         if self._sock is None:
             return
         try:
-            self._sock.shutdown(socket.SHUT_RDWR)
+            if os.getpid() == self.pid:
+                self._sock.shutdown(socket.SHUT_RDWR)
             self._sock.close()
         except socket.error:
             pass
@@ -973,7 +968,6 @@ class ConnectionPool(object):
                     # another thread already did the work while we waited
                     # on the lock.
                     return
-                self.disconnect()
                 self.reset()
 
     def get_connection(self, command_name, *keys, **options):
@@ -1012,6 +1006,7 @@ class ConnectionPool(object):
 
     def disconnect(self):
         "Disconnects all connections in the pool"
+        self._checkpid()
         all_conns = chain(self._available_connections,
                           self._in_use_connections)
         for connection in all_conns:
@@ -1133,5 +1128,6 @@ class BlockingConnectionPool(ConnectionPool):
 
     def disconnect(self):
         "Disconnects all connections in the pool."
+        self._checkpid()
         for connection in self._connections:
             connection.disconnect()