Journal: commits may not include all journaled seqs

At one point, a commit had to drain the FileStore op
queue.  This is no longer the case.  Consequently, the
journal may have to wait more than one commit for the
filestore to create a stable commit point at a particular
sequence.  Handling this requires two changes:

1) We cannot transition to FULL_WAIT until we receive
a commit_start on a seq >= journaled_seq.
2) We cannot remove the journal completion plug until get
a committed_thru on a seq >= header.start_seq at least as
new as the oldest committed item in the journal.  If on
replay, the journal does not include fs_op_seq, we ignore
it, which is fine since we won't have reported those
entries committed!

Signed-off-by: Samuel Just <sam.just@inktank.com>

1 files changed, 13 insertions, 3 deletions

diff --git a/src/os/FileJournal.cc b/src/os/FileJournal.cc
index 8986490e600..c99f1662f33 100644
--- a/src/os/FileJournal.cc
+++ b/src/os/FileJournal.cc
@@ -1490,8 +1490,18 @@ void FileJournal::commit_start(uint64_t seq)
     break; // all good
 
   case FULL_FULL:
-    dout(1) << " FULL_FULL -> FULL_WAIT.  last commit epoch committed, waiting for a new one to start." << dendl;
-    full_state = FULL_WAIT;
+    if (seq >= journaled_seq) {
+      dout(1) << " FULL_FULL -> FULL_WAIT.  commit_start on seq "
+	      << seq << " > journaled_seq " << journaled_seq
+	      << ", moving to FULL_WAIT."
+	      << dendl;
+      full_state = FULL_WAIT;
+    } else {
+      dout(1) << "FULL_FULL commit_start on seq "
+	      << seq << " < journaled_seq " << journaled_seq
+	      << ", remaining in FULL_FULL"
+	      << dendl;
+    }
     break;
 
   case FULL_WAIT:
@@ -1537,7 +1547,7 @@ void FileJournal::committed_thru(uint64_t seq)
     Mutex::Locker locker(finisher_lock);
     // completions!
     queue_completions_thru(seq);
-    if (plug_journal_completions) {
+    if (plug_journal_completions && seq >= header.start_seq) {
       dout(10) << " removing completion plug, queuing completions thru journaled_seq " << journaled_seq << dendl;
       plug_journal_completions = false;
       queue_completions_thru(journaled_seq);