Fix remap private view memory leak and formalize w->X ugprade process.

Makes the upgrade R_to_W() block until it succeeds.

Replaces "runExclusively" with a new "X" state, which is equivalent to "W", but
can only be reached from "w".  Client code in "w" calls "w_to_X()", which blocks
until all other threads in "w" call w_to_X() or call unlock_w().  At that point,
exactly one thread returns from w_to_X() with return value "true".  This thread
is the "exclusive writer", and may behave like it's in "W" state.  When that
thread calls "X_to_w()", it reverts to "w" state, and releases all the other
threads that called w_to_X() back into "w" state.

Because X_to_w() is effectively a barrier, we use generation counters to make
sure that fast racers don't cause deadlocks or race through the barrier.  The
generation counters are generationX and generationXExit.

Use of w_to_X() is wrapped in the Lock::DBWrite::UpgradeToExclusive() guard
object, which has a "gotUpgrade()" method to check if a particular thread was
the exclusive worker for the upgrade period.

Decision making about which condition variables to notify is more fully
delegated to the notifyWeUnlocked() method of QLock, to eliminate some
inadvertent deadlocks due that surfaced when R_to_W() was made to wait until
success.

May help SERVER-5533, because it fixes a greediness logic bug exercised by
fsync-and-lock.

This patch also introduces a directed test of the w->X functionality,
though it could use the addition of of extra "noise" work.

1 files changed, 11 insertions, 29 deletions

diff --git a/src/mongo/db/dur.cpp b/src/mongo/db/dur.cpp
index ce598955c34..b94ea57c9cc 100644
--- a/src/mongo/db/dur.cpp
+++ b/src/mongo/db/dur.cpp
@@ -74,7 +74,6 @@ using namespace mongoutils;
 namespace mongo {
 
     bool lockedForWriting();
-    void runExclusively(void (*f)(void));
 
     namespace dur {
 
@@ -251,29 +250,12 @@ namespace mongo {
             return commitJob.bytes() > UncommittedBytesLimit;
         }
 
-        static int in_f;
-        void f_commitEarlyInRunExclusively() { 
-            dassert( in_f == 0 );
-            in_f++;
-            try { 
-                assertNothingSpooled();
-                getDur().commitNow();
-                assertNothingSpooled(); // a light sanity check that we truly were exclusive
-            }
-            catch(...) { 
-                in_f--;
-                throw;
-            }
-            in_f--;
-        }
-
         void assertLockedForCommitting() { 
             char t = Lock::isLocked();
             if(  t == 'R' || t == 'W' )
                 return;
-            // 'w' case we use runExclusively
-            fassert( 16110, t == 'w' );
-            fassert( 16111, in_f == 1 );
+            // 'w' case we upgrade to exclusive (X).
+            fassertFailed( 16110 );
         }
 
         bool NOINLINE_DECL DurableImpl::_aCommitIsNeeded() {
@@ -300,8 +282,12 @@ namespace mongo {
                     return false;
                 }
                 else {
-                    log(1) << "commitIfNeeded calling runExclusively" << endl;
-                    runExclusively(f_commitEarlyInRunExclusively);
+                    log(1) << "commitIfNeeded upgrading from shared write to exclusive write state"
+                           << endl;
+                    Lock::DBWrite::UpgradeToExclusive ex;
+                    if (ex.gotUpgrade()) {
+                        commitNow();
+                    }
                 }
             }
             else { 
@@ -627,7 +613,7 @@ namespace mongo {
         static void _groupCommit(Lock::GlobalWrite *lgw) {
             LOG(4) << "_groupCommit " << endl;
 
-            // runExclusively is in 'w', else we are 'R' or 'W'
+            // We are 'R' or 'W'
             assertLockedForCommitting();
 
             unspoolWriteIntents(); // in case we were doing some writing ourself
@@ -682,12 +668,8 @@ namespace mongo {
                 // to be in W the entire time we were committing about (in particular for WRITETOJOURNAL() which takes time).
                 if( lgw ) { 
                     LOG(4) << "_groupCommit upgrade" << endl;
-                    if( lgw->upgrade() ) {
-                        REMAPPRIVATEVIEW();
-                    }
-                    else {
-                        log() << "info timeout on lock upgrade for REMAPPRIVATEVIEW" << endl;
-                    }
+                    lgw->upgrade();
+                    REMAPPRIVATEVIEW();
                 }
             }
             else {