MDEV-13495 Crash in rollback of a recovered INSERT transaction after upgrade

If the server is upgraded from a database that was created
before MDEV-12288, and if the undo logs in the database contain
an incomplete transaction that performed an INSERT operation,
the server would crash when rolling back that transaction.

trx_commit_low(): Relax a too strict transaction. This function
will also be called after completing the rollback of a recovered
transaction.

trx_purge_add_undo_to_history(): Merged from the functions
trx_purge_add_update_undo_to_history() and trx_undo_update_cleanup(),
which are removed. Remove the parameter undo_page, and instead call
trx_undo_set_state_at_finish() to obtain it.

trx_write_serialisation_history(): Treat undo and old_insert equally.
That is, after the rollback (or XA COMMIT) of a recovered transaction
before upgrade, move all logs (both insert_undo and update_undo) to
the purge queue.

2 files changed, 6 insertions, 22 deletions

diff --git a/storage/innobase/include/trx0purge.h b/storage/innobase/include/trx0purge.h
index c9f5edb1432..0b42479cc21 100644
--- a/storage/innobase/include/trx0purge.h
+++ b/storage/innobase/include/trx0purge.h
@@ -51,16 +51,13 @@ trx_purge_get_log_from_hist(
 /*========================*/
 	fil_addr_t	node_addr);	/*!< in: file address of the history
 					list node of the log */
-/************************************************************************
-Adds the update undo log as the first log in the history list. Removes the
-update undo log segment from the rseg slot if it is too big for reuse. */
+/** Prepend the history list with an undo log.
+Remove the undo log segment from the rseg slot if it is too big for reuse.
+@param[in]	trx		transaction
+@param[in,out]	undo		undo log
+@param[in,out]	mtr		mini-transaction */
 void
-trx_purge_add_update_undo_to_history(
-/*=================================*/
-	trx_t*		trx,		/*!< in: transaction */
-	page_t*		undo_page,	/*!< in: update undo log header page,
-					x-latched */
-	mtr_t*		mtr);		/*!< in: mtr */
+trx_purge_add_undo_to_history(const trx_t* trx, trx_undo_t*& undo, mtr_t* mtr);
 /*******************************************************************//**
 This function runs a purge batch.
 @return number of undo log pages handled in the batch */
diff --git a/storage/innobase/include/trx0undo.h b/storage/innobase/include/trx0undo.h
index 1302d3fd6df..ba4dc73f73b 100644
--- a/storage/innobase/include/trx0undo.h
+++ b/storage/innobase/include/trx0undo.h
@@ -287,19 +287,6 @@ trx_undo_set_state_at_prepare(
 	bool		rollback,
 	mtr_t*		mtr);
 
-/**********************************************************************//**
-Adds the update undo log header as the first in the history list, and
-frees the memory object, or puts it to the list of cached update undo log
-segments. */
-void
-trx_undo_update_cleanup(
-/*====================*/
-	trx_t*		trx,		/*!< in: trx owning the update
-					undo log */
-	page_t*		undo_page,	/*!< in: update undo log header page,
-					x-latched */
-	mtr_t*		mtr);		/*!< in: mtr */
-
 /** Free an old insert or temporary undo log after commit or rollback.
 The information is not needed after a commit or rollback, therefore
 the data can be discarded.