MDEV-15250 UPSERT during ALTER TABLE results in 'Duplicate entry' error for alter

- InnoDB DDL results in `Duplicate entry' if concurrent DML throws
duplicate key error. The following scenario explains the problem

connection con1:
  ALTER TABLE t1 FORCE;

connection con2:
  INSERT INTO t1(pk, uk) VALUES (2, 2), (3, 2);

In connection con2, InnoDB throws the 'DUPLICATE KEY' error because
of unique index. Alter operation will throw the error when applying
the concurrent DML log.

- Inserting the duplicate key for unique index logs the insert
operation for online ALTER TABLE. When insertion fails,
transaction does rollback and it leads to logging of
delete operation for online ALTER TABLE.
While applying the insert log entries, alter operation
encounters 'DUPLICATE KEY' error.

- To avoid the above fake duplicate scenario, InnoDB should
not write any log for online ALTER TABLE before DML transaction
commit.

- User thread which does DML can apply the online log if
InnoDB ran out of online log and index is marked as completed.
Set online log error if apply phase encountered any error.
It can also clear all other indexes log, marks the newly
added indexes as corrupted.

- Removed the old online code which was a part of DML operations

commit_inplace_alter_table() : Does apply the online log
for the last batch of secondary index log and does frees
the log for the completed index.

trx_t::apply_online_log: Set to true while writing the undo
log if the modified table has active DDL

trx_t::apply_log(): Apply the DML changes to online DDL tables

dict_table_t::is_active_ddl(): Returns true if the table
has an active DDL

dict_index_t::online_log_make_dummy(): Assign dummy value
for clustered index online log to indicate the secondary
indexes are being rebuild.

dict_index_t::online_log_is_dummy(): Check whether the online
log has dummy value

ha_innobase_inplace_ctx::log_failure(): Handle the apply log
failure for online DDL transaction

row_log_mark_other_online_index_abort(): Clear out all other
online index log after encountering the error during
row_log_apply()

row_log_get_error(): Get the error happened during row_log_apply()

row_log_online_op(): Does apply the online log if index is
completed and ran out of memory. Returns false if apply log fails

UndorecApplier: Introduced a class to maintain the undo log
record, latched undo buffer page, parse the undo log record,
maintain the undo record type, info bits and update vector

UndorecApplier::get_old_rec(): Get the correct version of the
clustered index record that was modified by the current undo
log record

UndorecApplier::clear_undo_rec(): Clear the undo log related
information after applying the undo log record

UndorecApplier::log_update(): Handle the update, delete undo
log and apply it on online indexes

UndorecApplier::log_insert(): Handle the insert undo log
and apply it on online indexes

UndorecApplier::is_same(): Check whether the given roll pointer
is generated by the current undo log record information

trx_t::rollback_low(): Set apply_online_log for the transaction
after partially rollbacked transaction has any active DDL

prepare_inplace_alter_table_dict(): After allocating the online
log, InnoDB does create fulltext common tables. Fulltext index
doesn't allow the index to be online. So removed the dead
code of online log removal

Thanks to Marko Mäkelä for providing the initial prototype and
Matthias Leich for testing the issue patiently.

1 files changed, 100 insertions, 1 deletions

diff --git a/storage/innobase/include/trx0undo.h b/storage/innobase/include/trx0undo.h
index 62662ffe221..111369e6a0f 100644
--- a/storage/innobase/include/trx0undo.h
+++ b/storage/innobase/include/trx0undo.h
@@ -1,7 +1,7 @@
 /*****************************************************************************
 
 Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
-Copyright (c) 2017, 2021, MariaDB Corporation.
+Copyright (c) 2017, 2022, MariaDB Corporation.
 
 This program is free software; you can redistribute it and/or modify it under
 the terms of the GNU General Public License as published by the Free Software
@@ -326,6 +326,105 @@ struct trx_undo_t {
 					/*!< undo log objects in the rollback
 					segment are chained into lists */
 };
+
+/** Cache a pointer to an undo record in a latched buffer pool page,
+parse the undo log record and store the record type, update vector
+and compiler information */
+class UndorecApplier
+{
+  /** undo log block which was latched */
+  const buf_block_t *block;
+  /** Undo log record pointer */
+  trx_undo_rec_t *undo_rec;
+  /** Offset of the undo log record within the block */
+  ulint offset;
+  /** Transaction id of the undo log */
+  trx_id_t trx_id;
+  /** Undo log record type */
+  ulint type;
+  /** compiler information */
+  ulint cmpl_info;
+  /** Update vector */
+  upd_t *update;
+  /** memory heap which can be used to build previous version of
+  the index record and its offsets */
+  mem_heap_t *heap;
+  /** mini-transaction for accessing B-tree pages */
+  mtr_t mtr;
+
+public:
+  UndorecApplier(const buf_block_t *block, trx_id_t trx_id)
+                : block(block), trx_id(trx_id)
+  {
+    ut_ad(block->page.lock.have_any());
+    heap= mem_heap_create(100);
+  }
+
+  /** Assign the undo log block */
+  void assign_block(const buf_block_t *undo_block)
+  {
+    block= undo_block;
+  }
+
+  /** Assign the undo log record and offset */
+  void assign_rec(trx_undo_rec_t *rec);
+
+  /** Handle the DML undo log and apply it on online indexes */
+  void apply_undo_rec();
+
+  ~UndorecApplier()
+  {
+    mem_heap_free(heap);
+  }
+
+private:
+  /** Handle the insert undo log and apply it on online indexes
+  @param  tuple		row reference from undo log record
+  @param  clust_index	clustered index */
+  void log_insert(const dtuple_t &tuple, dict_index_t *clust_index);
+
+  /** Handle the update, delete undo log and apply it on online
+  indexes.
+  @param  tuple		row reference from undo log record
+  @param  clust_index	clustered index */
+  void log_update(const dtuple_t &tuple, dict_index_t *clust_index);
+
+  /** Check whether the given roll pointer is generated by
+  the current undo log record information stored.
+  @return true if roll pointer matches with current undo log info */
+  bool is_same(roll_ptr_t roll_ptr) const
+  {
+    uint16_t offset= static_cast<uint16_t>(roll_ptr);
+    uint32_t page_no= static_cast<uint32_t>(roll_ptr >> 16);
+    return page_no == block->page.id().page_no() && offset == this->offset;
+  }
+
+  /** Clear the undo log record information */
+  void clear_undo_rec()
+  {
+    undo_rec= nullptr;
+    cmpl_info= 0;
+    type= 0;
+    update= nullptr;
+    offset= 0;
+    mem_heap_empty(heap);
+  }
+
+  /** Get the correct version of the clustered index record that
+  was modified by the current undo log record. Because there could
+  be the multiple successive updates of the same record within the
+  same transaction.
+  @param	tuple		tuple contains primary key value
+  @param	index		clustered index
+  @param[out]	clust_rec	current clustered index record
+  @param	offsets		offsets points to the record
+  @return clustered index record which was changed by
+  the undo log record or nullptr when there is no clustered
+  index record changed by undo log record */
+  const rec_t* get_old_rec(const dtuple_t &tuple, dict_index_t *index,
+                           const rec_t **clust_rec, rec_offs **offsets);
+};
+
 #endif /* !UNIV_INNOCHECKSUM */
 
 /** The offset of the undo log page header on pages of the undo log */