MDEV-15370 Upgrade fails when both insert_undo and update_undo exist

Before MDEV-12288 in MariaDB 10.3.1, InnoDB used to partition
the persistent transaction undo log into insert_undo and update_undo.

MDEV-12288 repurposes the update_undo as the single undo log.
In order to support an upgrade from earlier MariaDB versions,
the insert_undo is recovered in data structures, called old_insert.

An assertion failure occurred in TrxUndoRsegsIterator::set_next()
when an incomplete transaction was recovered with both insert_undo
and update_undo log. This could be easily demonstrated by starting
./mysql-test-run --manual-gdb innodb.read_only_recovery
in MariaDB 10.2, and after the first kill, start up the MariaDB 10.3
server with the same parameters.

The problem is that MariaDB 10.3 would roll back the recovered
transaction, and finally "commit" it twice (with all changes to
data rolled back), both insert_undo and update_undo with the same
commit end identifier (trx->no).

Our fix is to introduce a "commit number" that comprises two components:
(trx->no << 1 | !old_insert). In this way, the assertion in the purge
subsystem can be relaxed so that only the trx->no component must match.

1 files changed, 17 insertions, 12 deletions

diff --git a/storage/innobase/include/trx0purge.h b/storage/innobase/include/trx0purge.h
index f0c9fa436c3..2e7c3828607 100644
--- a/storage/innobase/include/trx0purge.h
+++ b/storage/innobase/include/trx0purge.h
@@ -103,14 +103,16 @@ public:
 	TrxUndoRsegs() {}
 	/** Constructor */
 	TrxUndoRsegs(trx_rseg_t& rseg)
-		: m_trx_no(rseg.last_trx_no), m_rsegs(1, &rseg) {}
+		: m_commit(rseg.last_commit), m_rsegs(1, &rseg) {}
 	/** Constructor */
 	TrxUndoRsegs(trx_id_t trx_no, trx_rseg_t& rseg)
-		: m_trx_no(trx_no), m_rsegs(1, &rseg) {}
+		: m_commit(trx_no << 1), m_rsegs(1, &rseg) {}
 
-	/** Get the commit number */
-	trx_id_t get_trx_no() const { return m_trx_no; }
+	/** @return the transaction commit identifier */
+	trx_id_t trx_no() const { return m_commit >> 1; }
 
+	bool operator!=(const TrxUndoRsegs& other) const
+	{ return m_commit != other.m_commit; }
 	bool empty() const { return m_rsegs.empty(); }
 	void erase(iterator& it) { m_rsegs.erase(it); }
 	iterator begin() { return(m_rsegs.begin()); }
@@ -124,13 +126,12 @@ public:
 	@return true if elem1 > elem2 else false.*/
 	bool operator()(const TrxUndoRsegs& lhs, const TrxUndoRsegs& rhs)
 	{
-		return(lhs.m_trx_no > rhs.m_trx_no);
+		return(lhs.m_commit > rhs.m_commit);
 	}
 
 private:
-	/** Copy trx_rseg_t::last_trx_no */
-	trx_id_t		m_trx_no;
-
+	/** Copy trx_rseg_t::last_commit */
+	trx_id_t		m_commit;
 	/** Rollback segments of a transaction, scheduled for purge. */
 	trx_rsegs_t		m_rsegs;
 };
@@ -438,13 +439,17 @@ public:
 	{
 		bool operator<=(const iterator& other) const
 		{
-			if (trx_no < other.trx_no) return true;
-			if (trx_no > other.trx_no) return false;
+			if (commit < other.commit) return true;
+			if (commit > other.commit) return false;
 			return undo_no <= other.undo_no;
 		}
 
-		/** The trx_t::no of the committed transaction */
-		trx_id_t	trx_no;
+		/** @return the commit number of the transaction */
+		trx_id_t trx_no() const { return commit >> 1; }
+		void reset_trx_no(trx_id_t trx_no) { commit = trx_no << 1; }
+
+		/** 2 * trx_t::no + old_insert of the committed transaction */
+		trx_id_t	commit;
 		/** The record number within the committed transaction's undo
 		log, increasing, purged from from 0 onwards */
 		undo_no_t	undo_no;