MDEV-24971 InnoDB access freed virtual column after rollback of secondary index

Problem:
========
 InnoDB fails to clean the index stub if it fails to add the
virtual index which contains new virtual column. But it clears
the newly virtual column from index in clear_added_indexes()
during inplace_alter_table. On commit, InnoDB evicts and
reload the table. In case of rollback, it doesn't happen.
InnoDB clears the ABORTED index while opening the table
or doing the DDL. In the mean time, InnoDB can access
the dropped virtual index columns while creating prebuilt
or rollback of concurrent DML.

Solution:
==========
(1) InnoDB should maintain newly added virtual column while
rollbacking the newly added virtual index.
(2) InnoDB must not defer the index removal
if the alter table is executed with LOCK=EXCLUSIVE.
(3) For LOCK=SHARED, InnoDB should check whether the table
has any other transaction lock other than alter transaction
before deferring the index stub.

Replaced has_new_v_col with dict_add_vcol_info in dict_index_t to
indicate whether the index has any new virtual column.

dict_index_t::has_new_v_col(): Returns whether the index has
newly added virtual column, it doesn't say which columns are
newly added virtual column

ha_innobase_inplace_ctx::is_new_vcol(): Return whether the
given column is added as a part of the current alter.

ha_innobase_inplace_ctx::clean_new_vcol_index(): Copy the newly
added virtual column to new_vcol_info in dict_index_t. Replace
the column in the index fields with virtual column stored
in new_vcol_info.

dict_index_t::assign_new_v_col(): Store the number of virtual
column added in index as a part of alter table.

dict_index_t::get_n_new_vcol(): Get the number of newly added
virtual column

dict_index_t::assign_drop_v_col(): Allocate the memory for
adding new virtual column in new_vcol_info.

dict_index_t::add_drop_v_col(): Add the newly added virtual
column in new_vcol_info.

dict_table_t::has_lock_for_other_trx(): Whether the table has
any other transaction lock than given transaction.

row_merge_drop_indexes(): Add parameter alter_trx and check
whether the table has any other lock than alter transaction.

1 files changed, 17 insertions, 12 deletions

diff --git a/storage/innobase/row/row0merge.cc b/storage/innobase/row/row0merge.cc
index 835d74043fd..e3b3f2c2762 100644
--- a/storage/innobase/row/row0merge.cc
+++ b/storage/innobase/row/row0merge.cc
@@ -3696,17 +3696,20 @@ row_merge_drop_indexes_dict(
 	trx->op_info = "";
 }
 
-/*********************************************************************//**
-Drop indexes that were created before an error occurred.
+/** Drop indexes that were created before an error occurred.
 The data dictionary must have been locked exclusively by the caller,
-because the transaction will not be committed. */
+because the transaction will not be committed.
+@param trx              dictionary transaction
+@param table            table containing the indexes
+@param locked           True if table is locked,
+                        false - may need to do lazy drop
+@param alter_trx        Alter table transaction */
 void
 row_merge_drop_indexes(
-/*===================*/
-	trx_t*		trx,	/*!< in/out: dictionary transaction */
-	dict_table_t*	table,	/*!< in/out: table containing the indexes */
-	ibool		locked)	/*!< in: TRUE=table locked,
-				FALSE=may need to do a lazy drop */
+        trx_t*          trx,
+        dict_table_t*   table,
+        bool            locked,
+        const trx_t*    alter_trx)
 {
 	dict_index_t*	index;
 	dict_index_t*	next_index;
@@ -3732,7 +3735,7 @@ row_merge_drop_indexes(
 	A concurrent purge will be prevented by dict_operation_lock. */
 
 	if (!locked && (table->get_ref_count() > 1
-			|| UT_LIST_GET_FIRST(table->locks))) {
+			|| table->has_lock_other_than(alter_trx))) {
 		/* We will have to drop the indexes later, when the
 		table is guaranteed to be no longer in use.  Mark the
 		indexes as incomplete and corrupted, so that other
@@ -4363,7 +4366,7 @@ row_merge_create_index(
 	dberr_t		err;
 	ulint		n_fields = index_def->n_fields;
 	ulint		i;
-	bool		has_new_v_col = false;
+	ulint		n_add_vcol = 0;
 
 	DBUG_ENTER("row_merge_create_index");
 
@@ -4391,7 +4394,7 @@ row_merge_create_index(
 				ut_ad(ifield->col_no >= table->n_v_def);
 				name = add_v->v_col_name[
 					ifield->col_no - table->n_v_def];
-				has_new_v_col = true;
+				n_add_vcol++;
 			} else {
 				name = dict_table_get_v_col_name(
 					table, ifield->col_no);
@@ -4410,7 +4413,9 @@ row_merge_create_index(
 	if (err == DB_SUCCESS) {
 		ut_ad(index != index_template);
 		index->parser = index_def->parser;
-		index->has_new_v_col = has_new_v_col;
+		if (n_add_vcol) {
+			index->assign_new_v_col(n_add_vcol);
+		}
 		/* Note the id of the transaction that created this
 		index, we use it to restrict readers from accessing
 		this index, to ensure read consistency. */