diff options
| author | Marko Mäkelä <marko.makela@mariadb.com> | 2021-06-09 17:02:55 +0300 |
|---|---|---|
| committer | Marko Mäkelä <marko.makela@mariadb.com> | 2021-06-09 17:06:07 +0300 |
| commit | 1bd681c8b3c5213ce1f7976940a7dc38b48a0d39 (patch) | |
| tree | 1b5cbbac58a1fda7542555065d148cb21a427f47 | |
| parent | 3f78fbc582ed4c1187fa1a42189fd6d26bc041c4 (diff) | |
| download | mariadb-git-1bd681c8b3c5213ce1f7976940a7dc38b48a0d39.tar.gz | |
MDEV-25506 (3 of 3): Do not delete .ibd files before commit
This is a complete rewrite of DROP TABLE, also as part of other DDL,
such as ALTER TABLE, CREATE TABLE...SELECT, TRUNCATE TABLE.
The background DROP TABLE queue hack is removed.
If a transaction needs to drop and create a table by the same name
(like TRUNCATE TABLE does), it must first rename the table to an
internal #sql-ib name. No committed version of the data dictionary
will include any #sql-ib tables, because whenever a transaction
renames a table to a #sql-ib name, it will also drop that table.
Either the rename will be rolled back, or the drop will be committed.
Data files will be unlinked after the transaction has been committed
and a FILE_RENAME record has been durably written. The file will
actually be deleted when the detached file handle returned by
fil_delete_tablespace() will be closed, after the latches have been
released. It is possible that a purge of the delete of the SYS_INDEXES
record for the clustered index will execute fil_delete_tablespace()
concurrently with the DDL transaction. In that case, the thread that
arrives later will wait for the other thread to finish.
HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE: A new handler flag.
ha_innobase::truncate() now requires that all other references to
the table be released in advance. This was implemented by Monty.
ha_innobase::delete_table(): If CREATE TABLE..SELECT is detected,
we will "hijack" the current transaction, drop the table in
the current transaction and commit the current transaction.
This essentially fixes MDEV-21602. There is a FIXME comment about
making the check less failure-prone.
ha_innobase::truncate(), ha_innobase::delete_table():
Implement a fast path for temporary tables. We will no longer allow
temporary tables to use the adaptive hash index.
dict_table_t::mdl_name: The original table name for the purpose of
acquiring MDL in purge, to prevent a race condition between a
DDL transaction that is dropping a table, and purge processing
undo log records of DML that had executed before the DDL operation.
For #sql-backup- tables during ALTER TABLE...ALGORITHM=COPY, the
dict_table_t::mdl_name will differ from dict_table_t::name.
dict_table_t::parse_name(): Use mdl_name instead of name.
dict_table_rename_in_cache(): Update mdl_name.
For the internal FTS_ tables of FULLTEXT INDEX, purge would
acquire MDL on the FTS_ table name, but not on the main table,
and therefore it would be able to run concurrently with a
DDL transaction that is dropping the table. Previously, the
DROP TABLE queue hack prevented a race between purge and DDL.
For now, we introduce purge_sys.stop_FTS() to prevent purge from
opening any table, while a DDL transaction that may drop FTS_
tables is in progress. The function fts_lock_table(), which will
be invoked before the dictionary is locked, will wait for
purge to release any table handles.
trx_t::drop_table_statistics(): Drop statistics for the table.
This replaces dict_stats_drop_index(). We will drop or rename
persistent statistics atomically as part of DDL transactions.
On lock conflict for dropping statistics, we will fail instantly
with DB_LOCK_WAIT_TIMEOUT, because we will be holding the
exclusive data dictionary latch.
trx_t::commit_cleanup(): Separated from trx_t::commit_in_memory().
Relax an assertion around fts_commit() and allow DB_LOCK_WAIT_TIMEOUT
in addition to DB_DUPLICATE_KEY. The call to fts_commit() is
entirely misplaced here and may obviously break the consistency
of transactions that affect FULLTEXT INDEX. It needs to be fixed
separately.
dict_table_t::n_foreign_key_checks_running: Remove (MDEV-21175).
The counter was a work-around for missing meta-data locking (MDL)
on the SQL layer, and not really needed in MariaDB.
ER_TABLE_IN_FK_CHECK: Replaced with ER_UNUSED_28.
HA_ERR_TABLE_IN_FK_CHECK: Remove.
row_ins_check_foreign_constraints(): Do not acquire
dict_sys.latch either. The SQL-layer MDL will protect us.
This was reviewed by Thirunarayanan Balathandayuthapani
and tested by Matthias Leich.
98 files changed, 2184 insertions, 3808 deletions
diff --git a/include/handler_ername.h b/include/handler_ername.h index fe55062e6fb..d03790b8e64 100644 --- a/include/handler_ername.h +++ b/include/handler_ername.h @@ -1,4 +1,4 @@ -/* Copyright (c) 2013 SkySQL Ab +/* Copyright (c) 2013, 2021, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -74,7 +74,6 @@ { "HA_ERR_INDEX_COL_TOO_LONG", HA_ERR_INDEX_COL_TOO_LONG, "" }, { "HA_ERR_INDEX_CORRUPT", HA_ERR_INDEX_CORRUPT, "" }, { "HA_ERR_UNDO_REC_TOO_BIG", HA_ERR_UNDO_REC_TOO_BIG, "" }, -{ "HA_ERR_TABLE_IN_FK_CHECK", HA_ERR_TABLE_IN_FK_CHECK, "" }, { "HA_ERR_ROW_NOT_VISIBLE", HA_ERR_ROW_NOT_VISIBLE, "" }, { "HA_ERR_ABORTED_BY_USER", HA_ERR_ABORTED_BY_USER, "" }, { "HA_ERR_DISK_FULL", HA_ERR_DISK_FULL, "" }, diff --git a/include/my_base.h b/include/my_base.h index 7db8156e9b8..053bf3fbb69 100644 --- a/include/my_base.h +++ b/include/my_base.h @@ -513,7 +513,7 @@ enum ha_base_keytype { #define HA_ERR_INDEX_CORRUPT 180 /* Index corrupted */ #define HA_ERR_UNDO_REC_TOO_BIG 181 /* Undo log record too big */ #define HA_FTS_INVALID_DOCID 182 /* Invalid InnoDB Doc ID */ -#define HA_ERR_TABLE_IN_FK_CHECK 183 /* Table being used in foreign key check */ +/* #define HA_ERR_TABLE_IN_FK_CHECK 183 */ /* Table being used in foreign key check */ #define HA_ERR_TABLESPACE_EXISTS 184 /* The tablespace existed in storage engine */ #define HA_ERR_TOO_MANY_FIELDS 185 /* Table has too many columns */ #define HA_ERR_ROW_IN_WRONG_PARTITION 186 /* Row in wrong partition */ diff --git a/mysql-test/include/check-testcase.test b/mysql-test/include/check-testcase.test index e984c4dc497..60b376a2836 100644 --- a/mysql-test/include/check-testcase.test +++ b/mysql-test/include/check-testcase.test @@ -97,10 +97,7 @@ call mtr.check_testcase(); let $datadir=`select @@datadir`; list_files $datadir mysql_upgrade_info; -list_files_write_file $datadir.tempfiles.txt $datadir/test #sql*; ---replace_regex /#sql-ib[1-9][0-9]*\.ibd\n// -cat_file $datadir.tempfiles.txt; -remove_file $datadir.tempfiles.txt; +list_files $datadir/test #sql*; list_files $datadir/mysql #sql*; # diff --git a/mysql-test/main/partition_alter.test b/mysql-test/main/partition_alter.test index cca25d0989f..804b43dc3c2 100644 --- a/mysql-test/main/partition_alter.test +++ b/mysql-test/main/partition_alter.test @@ -86,11 +86,7 @@ show create table t1; --error ER_CONSTRAINT_FAILED insert t1 values (2, '2020-01-03', 20); drop table t1; ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $datadir.files.txt $datadir/test ---replace_regex $regexp ---cat_file $datadir.files.txt ---remove_file $datadir.files.txt +--list_files $datadir/test # MyISAM, different execution path create table t1(id int, d date not null, b bool not null default 0, primary key(id,d)) @@ -106,10 +102,7 @@ show create table t1; --error ER_CONSTRAINT_FAILED insert t1 values (2, '2020-01-03', 20); drop table t1; ---list_files_write_file $datadir.files.txt $datadir/test ---replace_regex $regexp ---cat_file $datadir.files.txt ---remove_file $datadir.files.txt +--list_files $datadir/test # # MDEV-13097 Online alter of a partitioned MyISAM table with auto_increment diff --git a/mysql-test/suite/gcol/r/innodb_virtual_stats.result b/mysql-test/suite/gcol/r/innodb_virtual_stats.result index c0f595263df..74a0480883d 100644 --- a/mysql-test/suite/gcol/r/innodb_virtual_stats.result +++ b/mysql-test/suite/gcol/r/innodb_virtual_stats.result @@ -38,10 +38,6 @@ idxa n_diff_pfx01 a idxa n_diff_pfx02 a,DB_ROW_ID idxa n_leaf_pages Number of leaf pages in the index idxa size Number of pages in the index -idxb n_diff_pfx01 b -idxb n_diff_pfx02 b,DB_ROW_ID -idxb n_leaf_pages Number of leaf pages in the index -idxb size Number of pages in the index vidxcd n_diff_pfx01 c vidxcd n_diff_pfx02 c,d vidxcd n_diff_pfx03 c,d,DB_ROW_ID @@ -58,14 +54,6 @@ index_name stat_name stat_description GEN_CLUST_INDEX n_diff_pfx01 DB_ROW_ID GEN_CLUST_INDEX n_leaf_pages Number of leaf pages in the index GEN_CLUST_INDEX size Number of pages in the index -idxb n_diff_pfx01 b -idxb n_diff_pfx02 b,DB_ROW_ID -idxb n_leaf_pages Number of leaf pages in the index -idxb size Number of pages in the index -vidxcd n_diff_pfx01 d -vidxcd n_diff_pfx02 d,DB_ROW_ID -vidxcd n_leaf_pages Number of leaf pages in the index -vidxcd size Number of pages in the index ALTER TABLE t ADD INDEX vidxe (e), ALGORITHM=INPLACE; select count(*) from t; count(*) @@ -77,18 +65,6 @@ index_name stat_name stat_description GEN_CLUST_INDEX n_diff_pfx01 DB_ROW_ID GEN_CLUST_INDEX n_leaf_pages Number of leaf pages in the index GEN_CLUST_INDEX size Number of pages in the index -idxb n_diff_pfx01 b -idxb n_diff_pfx02 b,DB_ROW_ID -idxb n_leaf_pages Number of leaf pages in the index -idxb size Number of pages in the index -vidxcd n_diff_pfx01 d -vidxcd n_diff_pfx02 d,DB_ROW_ID -vidxcd n_leaf_pages Number of leaf pages in the index -vidxcd size Number of pages in the index -vidxe n_diff_pfx01 e -vidxe n_diff_pfx02 e,DB_ROW_ID -vidxe n_leaf_pages Number of leaf pages in the index -vidxe size Number of pages in the index ALTER TABLE t ADD COLUMN f INT GENERATED ALWAYS AS(a + a), ADD INDEX vidxf (f), ALGORITHM=INPLACE; select count(*) from t; count(*) @@ -100,22 +76,6 @@ index_name stat_name stat_description GEN_CLUST_INDEX n_diff_pfx01 DB_ROW_ID GEN_CLUST_INDEX n_leaf_pages Number of leaf pages in the index GEN_CLUST_INDEX size Number of pages in the index -idxb n_diff_pfx01 b -idxb n_diff_pfx02 b,DB_ROW_ID -idxb n_leaf_pages Number of leaf pages in the index -idxb size Number of pages in the index -vidxcd n_diff_pfx01 d -vidxcd n_diff_pfx02 d,DB_ROW_ID -vidxcd n_leaf_pages Number of leaf pages in the index -vidxcd size Number of pages in the index -vidxe n_diff_pfx01 e -vidxe n_diff_pfx02 e,DB_ROW_ID -vidxe n_leaf_pages Number of leaf pages in the index -vidxe size Number of pages in the index -vidxf n_diff_pfx01 f -vidxf n_diff_pfx02 f,DB_ROW_ID -vidxf n_leaf_pages Number of leaf pages in the index -vidxf size Number of pages in the index ALTER TABLE t DROP INDEX vidxcd; SELECT index_name, stat_name, stat_description FROM mysql.innodb_index_stats @@ -124,16 +84,4 @@ index_name stat_name stat_description GEN_CLUST_INDEX n_diff_pfx01 DB_ROW_ID GEN_CLUST_INDEX n_leaf_pages Number of leaf pages in the index GEN_CLUST_INDEX size Number of pages in the index -idxb n_diff_pfx01 b -idxb n_diff_pfx02 b,DB_ROW_ID -idxb n_leaf_pages Number of leaf pages in the index -idxb size Number of pages in the index -vidxe n_diff_pfx01 e -vidxe n_diff_pfx02 e,DB_ROW_ID -vidxe n_leaf_pages Number of leaf pages in the index -vidxe size Number of pages in the index -vidxf n_diff_pfx01 f -vidxf n_diff_pfx02 f,DB_ROW_ID -vidxf n_leaf_pages Number of leaf pages in the index -vidxf size Number of pages in the index DROP TABLE t; diff --git a/mysql-test/suite/innodb/include/show_i_s_tables.inc b/mysql-test/suite/innodb/include/show_i_s_tables.inc index 8e4a362de95..5fe34c370c8 100644 --- a/mysql-test/suite/innodb/include/show_i_s_tables.inc +++ b/mysql-test/suite/innodb/include/show_i_s_tables.inc @@ -15,6 +15,5 @@ SELECT t.name 'Table Name', WHERE t.name not like 'SYS_%' AND t.name NOT LIKE 'mysql/%' AND t.name NOT LIKE 'sys/%' - AND t.name NOT LIKE '%/#sql-ib%' ORDER BY t.name; --enable_query_log diff --git a/mysql-test/suite/innodb/include/show_i_s_tablespaces.inc b/mysql-test/suite/innodb/include/show_i_s_tablespaces.inc index 9b108da4115..a85a294860d 100644 --- a/mysql-test/suite/innodb/include/show_i_s_tablespaces.inc +++ b/mysql-test/suite/innodb/include/show_i_s_tablespaces.inc @@ -11,6 +11,6 @@ SELECT name 'Space_Name', filename 'Path' FROM information_schema.innodb_sys_tablespaces WHERE name != 'innodb_system' - AND name NOT LIKE 'mysql/%' AND name NOT LIKE '%/#sql-ib%' + AND name NOT LIKE 'mysql/%' ORDER BY space; --enable_query_log diff --git a/mysql-test/suite/innodb/r/drop_table_background.result b/mysql-test/suite/innodb/r/drop_table_background.result deleted file mode 100644 index 378f3ce00ab..00000000000 --- a/mysql-test/suite/innodb/r/drop_table_background.result +++ /dev/null @@ -1,25 +0,0 @@ -CREATE TABLE t(c0 SERIAL, c1 INT, c2 INT, c3 INT, c4 INT, -KEY(c1), KEY(c2), KEY(c2,c1), -KEY(c3), KEY(c3,c1), KEY(c3,c2), KEY(c3,c2,c1), -KEY(c4), KEY(c4,c1), KEY(c4,c2), KEY(c4,c2,c1), -KEY(c4,c3), KEY(c4,c3,c1), KEY(c4,c3,c2), KEY(c4,c3,c2,c1)) ENGINE=InnoDB; -CREATE TABLE `#mysql50##sql-ib-foo`(a SERIAL) ENGINE=InnoDB; -INSERT INTO t (c1) VALUES (1),(2),(1); -SET DEBUG_DBUG='+d,row_drop_table_add_to_background'; -CREATE TABLE target (PRIMARY KEY(c1)) ENGINE=InnoDB SELECT * FROM t; -ERROR 23000: Duplicate entry '1' for key 'PRIMARY' -SELECT * from target; -ERROR 42S02: Table 'test.target' doesn't exist -DROP TABLE t; -# restart -CREATE TABLE t (a INT) ENGINE=InnoDB; -DROP TABLE t; -DROP TABLE target; -ERROR 42S02: Unknown table 'test.target' -CREATE TABLE target (a INT) ENGINE=InnoDB; -DROP TABLE target; -SELECT * FROM `#mysql50##sql-ib-foo`; -ERROR 42S02: Table 'test.#mysql50##sql-ib-foo' doesn't exist in engine -DROP TABLE `#mysql50##sql-ib-foo`; -Warnings: -Warning 1932 Table 'test.#mysql50##sql-ib-foo' doesn't exist in engine diff --git a/mysql-test/suite/innodb/r/innodb-index-online.result b/mysql-test/suite/innodb/r/innodb-index-online.result index 6ac5e897731..92a2f57397c 100644 --- a/mysql-test/suite/innodb/r/innodb-index-online.result +++ b/mysql-test/suite/innodb/r/innodb-index-online.result @@ -8,7 +8,6 @@ SET GLOBAL innodb_monitor_enable = module_ddl; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -27,7 +26,6 @@ ERROR 23000: Duplicate entry '1' for key 'PRIMARY' SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -81,7 +79,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR scanned'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -99,7 +96,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR created'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -113,7 +109,6 @@ ALTER TABLE t1 ADD UNIQUE INDEX(c2); SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -163,7 +158,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2d_created'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -175,7 +169,6 @@ ERROR 70100: Query execution was interrupted SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -243,7 +236,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2e_created'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -283,7 +275,6 @@ ROLLBACK; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -316,7 +307,6 @@ ERROR HY000: Creating index 'c2e' required more than 'innodb_online_alter_log_ma SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 1 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -328,7 +318,6 @@ name pos SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 1 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -338,7 +327,6 @@ ALTER TABLE t1 COMMENT 'testing if c2e will be dropped'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -363,7 +351,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2f_created'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -387,7 +374,6 @@ ROLLBACK; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -400,7 +386,6 @@ ALTER TABLE t1 CHANGE c2 c22f INT; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -464,7 +449,6 @@ name pos SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 1 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -473,7 +457,6 @@ connection default; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 1 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -493,7 +476,6 @@ ALTER TABLE t1 DROP INDEX c2d, DROP INDEX c2f; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 diff --git a/mysql-test/suite/innodb/r/innodb-table-online.result b/mysql-test/suite/innodb/r/innodb-table-online.result index 63313d7d7eb..8f06576f2ed 100644 --- a/mysql-test/suite/innodb/r/innodb-table-online.result +++ b/mysql-test/suite/innodb/r/innodb-table-online.result @@ -10,7 +10,6 @@ SET GLOBAL innodb_monitor_enable = module_ddl; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -29,7 +28,6 @@ ERROR 23000: Duplicate entry '1' for key 'PRIMARY' SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -95,7 +93,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR scanned'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -123,7 +120,6 @@ ALTER TABLE t1 DROP INDEX c2, ADD PRIMARY KEY(c1), ALGORITHM = INPLACE; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -148,7 +144,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR rebuilt'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -164,7 +159,6 @@ ERROR 70100: Query execution was interrupted SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -215,7 +209,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR rebuilt2'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -247,7 +240,6 @@ ROLLBACK; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 0 @@ -274,7 +266,6 @@ ERROR HY000: Creating index 'PRIMARY' required more than 'innodb_online_alter_lo SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 0 @@ -304,7 +295,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR rebuilt3'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 2 @@ -320,7 +310,6 @@ ROLLBACK; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 1 ddl_pending_alter_table 1 ddl_sort_file_alter_table 2 @@ -330,7 +319,6 @@ connection con1; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 2 @@ -410,7 +398,6 @@ SET DEBUG_SYNC = 'now SIGNAL ddl_timed_out'; SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl'; name count ddl_background_drop_indexes 0 -ddl_background_drop_tables 0 ddl_online_create_index 0 ddl_pending_alter_table 0 ddl_sort_file_alter_table 6 diff --git a/mysql-test/suite/innodb/r/innodb-wl5522-debug.result b/mysql-test/suite/innodb/r/innodb-wl5522-debug.result index 94d181ce448..2109d47bf4b 100644 --- a/mysql-test/suite/innodb/r/innodb-wl5522-debug.result +++ b/mysql-test/suite/innodb/r/innodb-wl5522-debug.result @@ -476,6 +476,7 @@ ALTER TABLE t1 IMPORT TABLESPACE; ERROR HY000: Index for table 't1' is corrupt; try to repair it SET SESSION debug_dbug=@saved_debug_dbug; restore: t1 .ibd and .cfg files +ALTER TABLE t1 IMPORT TABLESPACE; DROP TABLE t1; CREATE TABLE t1 ( c1 BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, diff --git a/mysql-test/suite/innodb/r/innodb_defrag_stats.result b/mysql-test/suite/innodb/r/innodb_defrag_stats.result index 598124e4ccb..c2fd378cb4b 100644 --- a/mysql-test/suite/innodb/r/innodb_defrag_stats.result +++ b/mysql-test/suite/innodb/r/innodb_defrag_stats.result @@ -1,41 +1,22 @@ -DROP TABLE if exists t1; select @@global.innodb_stats_persistent; @@global.innodb_stats_persistent 0 set global innodb_defragment_stats_accuracy = 20; -# Create table. -CREATE TABLE t1 (a INT NOT NULL PRIMARY KEY AUTO_INCREMENT, b VARCHAR(256), KEY SECOND(a, b)) ENGINE=INNODB; -# Populate data -INSERT INTO t1 VALUES(1, REPEAT('A', 256)); -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; +CREATE TABLE t1(a INT PRIMARY KEY, b VARCHAR(256), KEY SECOND(a, b)) +ENGINE=INNODB; +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_1_to_1024; # Not enough page splits to trigger persistent stats write yet. -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); -count(stat_value) = 0 -1 -INSERT INTO t1 (b) SELECT b from t1; +select * from mysql.innodb_index_stats where table_name='t1' +and stat_name in ('n_page_split','n_pages_freed,n_leaf_pages_defrag'); +database_name table_name index_name last_update stat_name stat_value sample_size stat_description +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_1025_to_2048; # Persistent stats recorded. -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select * from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +database_name table_name index_name last_update stat_name stat_value sample_size stat_description +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 # Delete some rows. @@ -62,13 +43,12 @@ delete from t1 where a between 100 * 1 and 100 * 1 + 30; # restart # Server Restarted # Confirm persistent stats still there after restart. -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select * from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +database_name table_name index_name last_update stat_name stat_value sample_size stat_description +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 optimize table t1; @@ -77,101 +57,65 @@ test.t1 optimize status OK select sleep(2); sleep(2) 0 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 set global innodb_defragment_stats_accuracy = 40; -INSERT INTO t1 (b) SELECT b from t1; -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_2049_to_4096; +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 -INSERT INTO t1 (b) SELECT b from t1; -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_4097_to_8192; +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 # Table rename should cause stats rename. rename table t1 to t2; -select sleep(1); -sleep(1) -0 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); -count(stat_value) = 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); +select * from mysql.innodb_index_stats where table_name = 't1'; +database_name table_name index_name last_update stat_name stat_value sample_size stat_description +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_pages_freed'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 # Drop index should cause stats drop. drop index SECOND on t2; -select sleep(3); -sleep(3) -0 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_page_split'); -count(stat_value) > 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_pages_freed'); -count(stat_value) > 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_leaf_pages_defrag'); -count(stat_value) > 0 -1 +select * from mysql.innodb_index_stats where table_name = 't2' and index_name = 'SECOND'; +database_name table_name index_name last_update stat_name stat_value sample_size stat_description # restart Server Restarted -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); -count(stat_value) = 0 -1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_page_split'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_pages_freed'); count(stat_value) > 0 1 -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_leaf_pages_defrag'); count(stat_value) > 0 1 # Clean up DROP TABLE t2; -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); -count(stat_value) = 0 -1 -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); -count(stat_value) = 0 -1 +select * from mysql.innodb_index_stats where table_name = 't2'; +database_name table_name index_name last_update stat_name stat_value sample_size stat_description diff --git a/mysql-test/suite/innodb/r/innodb_skip_innodb_is_tables.result b/mysql-test/suite/innodb/r/innodb_skip_innodb_is_tables.result index 1f3b0750e56..49fe8e629e3 100644 --- a/mysql-test/suite/innodb/r/innodb_skip_innodb_is_tables.result +++ b/mysql-test/suite/innodb/r/innodb_skip_innodb_is_tables.result @@ -228,7 +228,6 @@ innodb_master_thread_sleeps server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL N innodb_activity_count server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Current server activity count innodb_master_active_loops server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times master thread performs its tasks when server is active innodb_master_idle_loops server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times master thread performs its tasks when server is idle -innodb_background_drop_table_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to process drop table list innodb_log_flush_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to flush log records innodb_dict_lru_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to process DICT LRU list innodb_dict_lru_count_active server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of tables evicted from DICT LRU list in the active loop @@ -245,7 +244,6 @@ dml_system_inserts dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 s dml_system_deletes dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Number of system rows deleted dml_system_updates dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Number of system rows updated ddl_background_drop_indexes ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of indexes waiting to be dropped after failed index creation -ddl_background_drop_tables ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of tables in background drop table list ddl_online_create_index ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of indexes being created online ddl_pending_alter_table ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of ALTER TABLE, CREATE INDEX, DROP INDEX in progress ddl_sort_file_alter_table ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of sort files created during alter table diff --git a/mysql-test/suite/innodb/r/innodb_stats_drop_locked.result b/mysql-test/suite/innodb/r/innodb_stats_drop_locked.result index 3ae06c48ecb..7a6d5c46aab 100644 --- a/mysql-test/suite/innodb/r/innodb_stats_drop_locked.result +++ b/mysql-test/suite/innodb/r/innodb_stats_drop_locked.result @@ -1,7 +1,14 @@ +CREATE DATABASE unlocked; +CREATE TABLE unlocked.t1(a INT PRIMARY KEY) ENGINE=INNODB STATS_PERSISTENT=0; +CREATE DATABASE locked; +CREATE TABLE locked.t1(a INT PRIMARY KEY) ENGINE=INNODB STATS_PERSISTENT=1; +CREATE TABLE innodb_stats_drop_locked (c INT, KEY c_key (c)) +ENGINE=INNODB STATS_PERSISTENT=1; +ANALYZE TABLE innodb_stats_drop_locked; Table Op Msg_type Msg_text test.innodb_stats_drop_locked analyze status Engine-independent statistics collected test.innodb_stats_drop_locked analyze status OK -SET autocommit=0; +BEGIN; SELECT table_name FROM mysql.innodb_table_stats WHERE table_name='innodb_stats_drop_locked' FOR UPDATE; @@ -19,21 +26,23 @@ innodb_stats_drop_locked innodb_stats_drop_locked innodb_stats_drop_locked connect con1,localhost,root,,; -connection con1; ALTER TABLE innodb_stats_drop_locked DROP INDEX c_key; -Warnings: -Warning 1205 Unable to delete statistics for index c_key from mysql.innodb_index_stats because the rows are locked: Lock wait timeout. They can be deleted later using DELETE FROM mysql.innodb_index_stats WHERE database_name = 'test' AND table_name = 'innodb_stats_drop_locked' AND index_name = 'c_key'; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction SHOW CREATE TABLE innodb_stats_drop_locked; Table Create Table innodb_stats_drop_locked CREATE TABLE `innodb_stats_drop_locked` ( - `c` int(11) DEFAULT NULL + `c` int(11) DEFAULT NULL, + KEY `c_key` (`c`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 STATS_PERSISTENT=1 DROP TABLE innodb_stats_drop_locked; -SHOW TABLES; -Tables_in_test -connection default; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +DROP DATABASE unlocked; +DROP DATABASE locked; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction disconnect con1; +connection default; COMMIT; +DROP DATABASE locked; SELECT table_name FROM mysql.innodb_table_stats WHERE table_name='innodb_stats_drop_locked'; table_name @@ -48,5 +57,17 @@ innodb_stats_drop_locked innodb_stats_drop_locked innodb_stats_drop_locked innodb_stats_drop_locked -DELETE FROM mysql.innodb_index_stats WHERE database_name='test' AND table_name='innodb_stats_drop_locked'; -DELETE FROM mysql.innodb_table_stats WHERE database_name='test' AND table_name='innodb_stats_drop_locked'; +ALTER TABLE innodb_stats_drop_locked DROP INDEX c_key; +SELECT table_name FROM mysql.innodb_index_stats +WHERE table_name='innodb_stats_drop_locked'; +table_name +innodb_stats_drop_locked +innodb_stats_drop_locked +innodb_stats_drop_locked +DROP TABLE innodb_stats_drop_locked; +SELECT table_name FROM mysql.innodb_table_stats +WHERE table_name='innodb_stats_drop_locked'; +table_name +SELECT table_name FROM mysql.innodb_index_stats +WHERE table_name='innodb_stats_drop_locked'; +table_name diff --git a/mysql-test/suite/innodb/r/innodb_stats_rename_table_if_exists.result b/mysql-test/suite/innodb/r/innodb_stats_rename_table_if_exists.result index 5614b4ba490..a966f629d7e 100644 --- a/mysql-test/suite/innodb/r/innodb_stats_rename_table_if_exists.result +++ b/mysql-test/suite/innodb/r/innodb_stats_rename_table_if_exists.result @@ -1,5 +1,6 @@ -CREATE TABLE stats_rename1 (a INT, PRIMARY KEY (a)) +CREATE TABLE stats_rename1 (a INT PRIMARY KEY, b INT UNIQUE) ENGINE=INNODB STATS_PERSISTENT=1; +BEGIN; INSERT INTO mysql.innodb_table_stats SELECT database_name, @@ -10,7 +11,7 @@ clustered_index_size, sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE table_name = 'stats_rename1'; -INSERT INTO mysql.innodb_index_stats +INSERT INTO mysql.innodb_index_stats SELECT database_name, 'stats_rename2' AS table_name, @@ -22,6 +23,7 @@ sample_size, stat_description FROM mysql.innodb_index_stats WHERE table_name = 'stats_rename1'; +COMMIT; SELECT table_name, n_rows FROM mysql.innodb_table_stats WHERE table_name IN ('stats_rename1', 'stats_rename2'); @@ -44,6 +46,18 @@ table_name stats_rename1 index_name PRIMARY stat_name size stat_value 1 +table_name stats_rename1 +index_name b +stat_name n_diff_pfx01 +stat_value 0 +table_name stats_rename1 +index_name b +stat_name n_leaf_pages +stat_value 1 +table_name stats_rename1 +index_name b +stat_name size +stat_value 1 table_name stats_rename2 index_name PRIMARY stat_name n_diff_pfx01 @@ -56,7 +70,32 @@ table_name stats_rename2 index_name PRIMARY stat_name size stat_value 567 +table_name stats_rename2 +index_name b +stat_name n_diff_pfx01 +stat_value 567 +table_name stats_rename2 +index_name b +stat_name n_leaf_pages +stat_value 567 +table_name stats_rename2 +index_name b +stat_name size +stat_value 567 RENAME TABLE stats_rename1 TO stats_rename2; +ERROR 23000: Can't write; duplicate key in table 'mysql.innodb_table_stats' +BEGIN; +DELETE FROM mysql.innodb_table_stats WHERE table_name='stats_rename2'; +DELETE FROM mysql.innodb_index_stats WHERE table_name='stats_rename2'; +COMMIT; +RENAME TABLE stats_rename1 TO stats_rename2; +UPDATE mysql.innodb_index_stats SET index_name='c' +WHERE table_name='stats_rename2' AND index_name='PRIMARY'; +ALTER TABLE stats_rename2 CHANGE b d INT, RENAME INDEX b TO c; +ERROR 23000: Can't write; duplicate key in table 'mysql.innodb_index_stats' +UPDATE mysql.innodb_index_stats SET index_name='PRIMARY' +WHERE table_name='stats_rename2' AND index_name='c'; +ALTER TABLE stats_rename2 CHANGE b d INT, RENAME INDEX b TO c; SELECT table_name, n_rows FROM mysql.innodb_table_stats WHERE table_name IN ('stats_rename1', 'stats_rename2'); @@ -77,4 +116,16 @@ table_name stats_rename2 index_name PRIMARY stat_name size stat_value 1 +table_name stats_rename2 +index_name c +stat_name n_diff_pfx01 +stat_value 0 +table_name stats_rename2 +index_name c +stat_name n_leaf_pages +stat_value 1 +table_name stats_rename2 +index_name c +stat_name size +stat_value 1 DROP TABLE stats_rename2; diff --git a/mysql-test/suite/innodb/r/log_file_name.result b/mysql-test/suite/innodb/r/log_file_name.result index 8a22615eae0..766122959ab 100644 --- a/mysql-test/suite/innodb/r/log_file_name.result +++ b/mysql-test/suite/innodb/r/log_file_name.result @@ -74,6 +74,8 @@ DROP TABLE t2,t3; CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; ERROR HY000: Can't create table `test`.`t0` (errno: 184 "Tablespace already exists") CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; +ERROR HY000: Can't create table `test`.`t0` (errno: 184 "Tablespace already exists") +CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; DROP TABLE t0; CREATE TABLE u1(a INT PRIMARY KEY) ENGINE=InnoDB; CREATE TABLE u2(a INT PRIMARY KEY) ENGINE=InnoDB; diff --git a/mysql-test/suite/innodb/r/monitor.result b/mysql-test/suite/innodb/r/monitor.result index 767ed7edcc5..0b2b0049f68 100644 --- a/mysql-test/suite/innodb/r/monitor.result +++ b/mysql-test/suite/innodb/r/monitor.result @@ -194,7 +194,6 @@ innodb_master_thread_sleeps disabled innodb_activity_count disabled innodb_master_active_loops disabled innodb_master_idle_loops disabled -innodb_background_drop_table_usec disabled innodb_log_flush_usec disabled innodb_dict_lru_usec disabled innodb_dict_lru_count_active disabled @@ -211,7 +210,6 @@ dml_system_inserts disabled dml_system_deletes disabled dml_system_updates disabled ddl_background_drop_indexes disabled -ddl_background_drop_tables disabled ddl_online_create_index disabled ddl_pending_alter_table disabled ddl_sort_file_alter_table disabled diff --git a/mysql-test/suite/innodb/r/rename_table_debug.result b/mysql-test/suite/innodb/r/rename_table_debug.result deleted file mode 100644 index 48211ad0aca..00000000000 --- a/mysql-test/suite/innodb/r/rename_table_debug.result +++ /dev/null @@ -1,31 +0,0 @@ -FLUSH TABLES; -CREATE TABLE t1 (a SERIAL, b INT, c INT, d INT) ENGINE=InnoDB; -INSERT INTO t1 () VALUES (); -connect con1,localhost,root,,test; -SET DEBUG_SYNC='before_rename_table_commit SIGNAL renamed WAIT_FOR ever'; -RENAME TABLE t1 TO t2; -connection default; -SET DEBUG_SYNC='now WAIT_FOR renamed'; -# restart -disconnect con1; -SELECT * FROM t1; -a b c d -1 NULL NULL NULL -CREATE TABLE t2 (a INT PRIMARY KEY) ENGINE=InnoDB; -BEGIN; -INSERT INTO t2 VALUES(1); -connect con1,localhost,root,,test; -SET DEBUG_SYNC='innodb_rename_in_cache SIGNAL committed WAIT_FOR ever'; -RENAME TABLE t1 TO t3; -connection default; -SET DEBUG_SYNC='now WAIT_FOR committed'; -COMMIT; -# restart -disconnect con1; -SELECT * FROM t1; -a b c d -1 NULL NULL NULL -SELECT * FROM t2; -a -1 -DROP TABLE t1,t2; diff --git a/mysql-test/suite/innodb/r/truncate.result b/mysql-test/suite/innodb/r/truncate.result index e12a401017f..ae1ca0f474d 100644 --- a/mysql-test/suite/innodb/r/truncate.result +++ b/mysql-test/suite/innodb/r/truncate.result @@ -50,3 +50,13 @@ Warnings: Warning 1814 Tablespace has been discarded for table `u` TRUNCATE u; DROP TABLE u; +# +# Test for a regression found during MDEV-25506 rewrite of DROP +# +CREATE TEMPORARY TABLE t1 (a INT) ENGINE=InnoDB; +LOCK TABLE t1 READ; +TRUNCATE TABLE t1; +TRUNCATE TABLE t1; +UNLOCK TABLES; +DROP TEMPORARY TABLE t1; +# End of 10.6 tests diff --git a/mysql-test/suite/innodb/r/xa_recovery.result b/mysql-test/suite/innodb/r/xa_recovery.result index ed9f19b7eb3..f61e29be20e 100644 --- a/mysql-test/suite/innodb/r/xa_recovery.result +++ b/mysql-test/suite/innodb/r/xa_recovery.result @@ -18,8 +18,9 @@ disconnect con2; connect con1,localhost,root; SELECT * FROM t1 LOCK IN SHARE MODE; connection default; +SET innodb_lock_wait_timeout=1; DROP TABLE t2; -# restart +ERROR HY000: Lock wait timeout exceeded; try restarting transaction disconnect con1; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; SELECT * FROM t1; @@ -30,6 +31,9 @@ SELECT * FROM t1; a 1 DROP TABLE t1; +DROP TABLE t2; +ERROR HY000: Lock wait timeout exceeded; try restarting transaction +XA ROLLBACK 'y'; +DROP TABLE t2; SET GLOBAL innodb_fast_shutdown=0; # restart -XA ROLLBACK 'y'; diff --git a/mysql-test/suite/innodb/t/default_row_format_compatibility.test b/mysql-test/suite/innodb/t/default_row_format_compatibility.test index 41abc2e67bc..d256fa7946a 100644 --- a/mysql-test/suite/innodb/t/default_row_format_compatibility.test +++ b/mysql-test/suite/innodb/t/default_row_format_compatibility.test @@ -46,16 +46,15 @@ CREATE TABLE tab(a INT) ENGINE=InnoDB; # Remove the *.ibd file ALTER TABLE tab DISCARD TABLESPACE; -# Move the *.ibd,*.cfg file into orginal location +# Move the *.ibd,*.cfg file into original location --move_file $MYSQLD_DATADIR/tab.cfg $MYSQLD_DATADIR/test/tab.cfg --move_file $MYSQLD_DATADIR/tab.ibd $MYSQLD_DATADIR/test/tab.ibd --error ER_TABLE_SCHEMA_MISMATCH ALTER TABLE tab IMPORT TABLESPACE; -# Take the backup of the ibd and cfg files ---copy_file $MYSQLD_DATADIR/test/tab.cfg $MYSQLD_DATADIR/tab.cfg ---copy_file $MYSQLD_DATADIR/test/tab.ibd $MYSQLD_DATADIR/tab.ibd +--move_file $MYSQLD_DATADIR/test/tab.cfg $MYSQLD_DATADIR/tab.cfg +--move_file $MYSQLD_DATADIR/test/tab.ibd $MYSQLD_DATADIR/tab.ibd # Cleanup DROP TABLE tab; diff --git a/mysql-test/suite/innodb/t/drop_table_background.test b/mysql-test/suite/innodb/t/drop_table_background.test deleted file mode 100644 index 20101dada84..00000000000 --- a/mysql-test/suite/innodb/t/drop_table_background.test +++ /dev/null @@ -1,48 +0,0 @@ ---source include/have_innodb.inc ---source include/have_debug.inc -# Embedded server does not support restarting ---source include/not_embedded.inc - -let $MYSQLD_DATADIR=`select @@datadir`; - -CREATE TABLE t(c0 SERIAL, c1 INT, c2 INT, c3 INT, c4 INT, -KEY(c1), KEY(c2), KEY(c2,c1), -KEY(c3), KEY(c3,c1), KEY(c3,c2), KEY(c3,c2,c1), -KEY(c4), KEY(c4,c1), KEY(c4,c2), KEY(c4,c2,c1), -KEY(c4,c3), KEY(c4,c3,c1), KEY(c4,c3,c2), KEY(c4,c3,c2,c1)) ENGINE=InnoDB; - -CREATE TABLE `#mysql50##sql-ib-foo`(a SERIAL) ENGINE=InnoDB; -INSERT INTO t (c1) VALUES (1),(2),(1); - -let $n= 10; - -SET DEBUG_DBUG='+d,row_drop_table_add_to_background'; ---disable_query_log -let $i= $n; -while ($i) { - eval CREATE TABLE t$i LIKE t; - dec $i; -} -let $i= $n; -while ($i) { - eval DROP TABLE t$i; - dec $i; -} ---enable_query_log ---error ER_DUP_ENTRY -CREATE TABLE target (PRIMARY KEY(c1)) ENGINE=InnoDB SELECT * FROM t; ---error ER_NO_SUCH_TABLE -SELECT * from target; -DROP TABLE t; ---source include/shutdown_mysqld.inc ---remove_files_wildcard $MYSQLD_DATADIR/test #sql-*.ibd ---source include/start_mysqld.inc -CREATE TABLE t (a INT) ENGINE=InnoDB; -DROP TABLE t; ---error ER_BAD_TABLE_ERROR -DROP TABLE target; -CREATE TABLE target (a INT) ENGINE=InnoDB; -DROP TABLE target; ---error ER_NO_SUCH_TABLE_IN_ENGINE -SELECT * FROM `#mysql50##sql-ib-foo`; -DROP TABLE `#mysql50##sql-ib-foo`; diff --git a/mysql-test/suite/innodb/t/innodb-wl5522-debug.test b/mysql-test/suite/innodb/t/innodb-wl5522-debug.test index 165264041fb..7256e2f23e0 100644 --- a/mysql-test/suite/innodb/t/innodb-wl5522-debug.test +++ b/mysql-test/suite/innodb/t/innodb-wl5522-debug.test @@ -1014,6 +1014,7 @@ do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl"; ib_restore_tablespaces("test", "t1"); EOF +ALTER TABLE t1 IMPORT TABLESPACE; DROP TABLE t1; # diff --git a/mysql-test/suite/innodb/t/innodb-wl5522.test b/mysql-test/suite/innodb/t/innodb-wl5522.test index 7675a1305a4..9b18b29fc20 100644 --- a/mysql-test/suite/innodb/t/innodb-wl5522.test +++ b/mysql-test/suite/innodb/t/innodb-wl5522.test @@ -96,6 +96,10 @@ if ($checksum_algorithm == "strict_full_crc32") { ALTER TABLE t2 IMPORT TABLESPACE; DROP TABLE t2; +if ($error_code) { +--remove_file $MYSQLD_DATADIR/test/t2.ibd +} + SET GLOBAL innodb_file_per_table = 1; SELECT @@innodb_file_per_table; diff --git a/mysql-test/suite/innodb/t/innodb-wl5980-alter.test b/mysql-test/suite/innodb/t/innodb-wl5980-alter.test index 1a32b94c140..f57d263fb3b 100644 --- a/mysql-test/suite/innodb/t/innodb-wl5980-alter.test +++ b/mysql-test/suite/innodb/t/innodb-wl5980-alter.test @@ -6,7 +6,7 @@ --source include/have_innodb.inc SET @innodb_file_per_table_orig=@@GLOBAL.innodb_file_per_table; -LET $regexp=/FTS_[0-9a-f_]+([A-Z0-9_]+)\.([islbd]{3})/FTS_AUX_\1.\2/ /#sql-ib[1-9][0-9]*\.ibd\n//; +LET $regexp=/FTS_[0-9a-f_]+([A-Z0-9_]+)\.([islbd]{3})/FTS_AUX_\1.\2/; # Set up some variables LET $MYSQL_DATA_DIR = `select @@datadir`; diff --git a/mysql-test/suite/innodb/t/innodb_defrag_stats.test b/mysql-test/suite/innodb/t/innodb_defrag_stats.test index 2a5026a68e5..248fd24f0c8 100644 --- a/mysql-test/suite/innodb/t/innodb_defrag_stats.test +++ b/mysql-test/suite/innodb/t/innodb_defrag_stats.test @@ -1,46 +1,26 @@ --source include/have_innodb.inc +--source include/have_sequence.inc --source include/big_test.inc --source include/not_valgrind.inc --source include/not_embedded.inc ---disable_warnings -DROP TABLE if exists t1; ---enable_warnings - ---disable_query_log -let $innodb_defragment_stats_accuracy_orig=`select @@innodb_defragment_stats_accuracy`; ---enable_query_log - select @@global.innodb_stats_persistent; set global innodb_defragment_stats_accuracy = 20; ---echo # Create table. -CREATE TABLE t1 (a INT NOT NULL PRIMARY KEY AUTO_INCREMENT, b VARCHAR(256), KEY SECOND(a, b)) ENGINE=INNODB; - ---echo # Populate data -INSERT INTO t1 VALUES(1, REPEAT('A', 256)); -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; -INSERT INTO t1 (b) SELECT b from t1; +CREATE TABLE t1(a INT PRIMARY KEY, b VARCHAR(256), KEY SECOND(a, b)) +ENGINE=INNODB; +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_1_to_1024; --echo # Not enough page splits to trigger persistent stats write yet. -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select * from mysql.innodb_index_stats where table_name='t1' +and stat_name in ('n_page_split','n_pages_freed,n_leaf_pages_defrag'); -INSERT INTO t1 (b) SELECT b from t1; +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_1025_to_2048; --echo # Persistent stats recorded. -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); +select * from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); --echo # Delete some rows. let $num_delete = 20; @@ -55,71 +35,53 @@ while ($num_delete) --echo # Server Restarted --echo # Confirm persistent stats still there after restart. -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); +select * from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); optimize table t1; select sleep(2); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); set global innodb_defragment_stats_accuracy = 40; -INSERT INTO t1 (b) SELECT b from t1; - -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_2049_to_4096; +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); -INSERT INTO t1 (b) SELECT b from t1; +INSERT INTO t1 SELECT seq, REPEAT('A', 256) FROM seq_4097_to_8192; -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't1' and stat_name in ('n_leaf_pages_defrag'); --echo # Table rename should cause stats rename. rename table t1 to t2; -select sleep(1); +select * from mysql.innodb_index_stats where table_name = 't1'; -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); - -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_leaf_pages_defrag'); --echo # Drop index should cause stats drop. drop index SECOND on t2; -select sleep(3); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and index_name = 'SECOND' and stat_name in ('n_leaf_pages_defrag'); +select * from mysql.innodb_index_stats where table_name = 't2' and index_name = 'SECOND'; --source include/restart_mysqld.inc --echo Server Restarted -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_pages_freed'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t1%' and stat_name in ('n_leaf_pages_defrag'); - -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); -select count(stat_value) > 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_page_split'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_pages_freed'); +select count(stat_value) > 0 from mysql.innodb_index_stats where table_name = 't2' and stat_name in ('n_leaf_pages_defrag'); --echo # Clean up DROP TABLE t2; -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_page_split'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_pages_freed'); -select count(stat_value) = 0 from mysql.innodb_index_stats where table_name like '%t2%' and stat_name in ('n_leaf_pages_defrag'); - ---disable_query_log -EVAL SET GLOBAL innodb_defragment_stats_accuracy = $innodb_defragment_stats_accuracy_orig; ---enable_query_log +select * from mysql.innodb_index_stats where table_name = 't2'; diff --git a/mysql-test/suite/innodb/t/innodb_stats_drop_locked.test b/mysql-test/suite/innodb/t/innodb_stats_drop_locked.test index 07c77299451..ab4cc78b337 100644 --- a/mysql-test/suite/innodb/t/innodb_stats_drop_locked.test +++ b/mysql-test/suite/innodb/t/innodb_stats_drop_locked.test @@ -5,49 +5,42 @@ -- source include/have_innodb.inc --- disable_warnings --- disable_query_log - -DROP TABLE IF EXISTS innodb_stats_drop_locked; +CREATE DATABASE unlocked; +CREATE TABLE unlocked.t1(a INT PRIMARY KEY) ENGINE=INNODB STATS_PERSISTENT=0; +CREATE DATABASE locked; +CREATE TABLE locked.t1(a INT PRIMARY KEY) ENGINE=INNODB STATS_PERSISTENT=1; CREATE TABLE innodb_stats_drop_locked (c INT, KEY c_key (c)) ENGINE=INNODB STATS_PERSISTENT=1; - ANALYZE TABLE innodb_stats_drop_locked; --- enable_warnings --- enable_query_log - -SET autocommit=0; - +BEGIN; SELECT table_name FROM mysql.innodb_table_stats WHERE table_name='innodb_stats_drop_locked' FOR UPDATE; - SELECT table_name FROM mysql.innodb_index_stats WHERE table_name='innodb_stats_drop_locked' FOR UPDATE; -- connect (con1,localhost,root,,) - --- connection con1 - +--error ER_LOCK_WAIT_TIMEOUT ALTER TABLE innodb_stats_drop_locked DROP INDEX c_key; # the index should be gone SHOW CREATE TABLE innodb_stats_drop_locked; +--error ER_LOCK_WAIT_TIMEOUT DROP TABLE innodb_stats_drop_locked; -# the table should be gone -SHOW TABLES; - --- connection default - +DROP DATABASE unlocked; +--error ER_LOCK_WAIT_TIMEOUT +DROP DATABASE locked; -- disconnect con1 - +-- connection default COMMIT; +DROP DATABASE locked; + # the stats should be there SELECT table_name FROM mysql.innodb_table_stats @@ -56,8 +49,15 @@ WHERE table_name='innodb_stats_drop_locked'; SELECT table_name FROM mysql.innodb_index_stats WHERE table_name='innodb_stats_drop_locked'; -DELETE FROM mysql.innodb_index_stats WHERE database_name='test' AND table_name='innodb_stats_drop_locked'; -DELETE FROM mysql.innodb_table_stats WHERE database_name='test' AND table_name='innodb_stats_drop_locked'; ---disable_query_log -call mtr.add_suppression("Unable to delete statistics for table test.innodb_stats_drop_locked: Lock wait timeout. They can be deleted later using DELETE FROM mysql.innodb_index_stats WHERE database_name"); ---enable_query_log +ALTER TABLE innodb_stats_drop_locked DROP INDEX c_key; + +SELECT table_name FROM mysql.innodb_index_stats +WHERE table_name='innodb_stats_drop_locked'; + +DROP TABLE innodb_stats_drop_locked; + +SELECT table_name FROM mysql.innodb_table_stats +WHERE table_name='innodb_stats_drop_locked'; + +SELECT table_name FROM mysql.innodb_index_stats +WHERE table_name='innodb_stats_drop_locked'; diff --git a/mysql-test/suite/innodb/t/innodb_stats_rename_table_if_exists.test b/mysql-test/suite/innodb/t/innodb_stats_rename_table_if_exists.test index e5b5d1814c0..3f0b5c03b3d 100644 --- a/mysql-test/suite/innodb/t/innodb_stats_rename_table_if_exists.test +++ b/mysql-test/suite/innodb/t/innodb_stats_rename_table_if_exists.test @@ -10,9 +10,10 @@ -- vertical_results -CREATE TABLE stats_rename1 (a INT, PRIMARY KEY (a)) +CREATE TABLE stats_rename1 (a INT PRIMARY KEY, b INT UNIQUE) ENGINE=INNODB STATS_PERSISTENT=1; +BEGIN; INSERT INTO mysql.innodb_table_stats SELECT database_name, @@ -24,7 +25,7 @@ sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE table_name = 'stats_rename1'; -INSERT INTO mysql.innodb_index_stats +INSERT INTO mysql.innodb_index_stats SELECT database_name, 'stats_rename2' AS table_name, @@ -36,6 +37,7 @@ sample_size, stat_description FROM mysql.innodb_index_stats WHERE table_name = 'stats_rename1'; +COMMIT; SELECT table_name, n_rows FROM mysql.innodb_table_stats @@ -45,7 +47,22 @@ SELECT table_name, index_name, stat_name, stat_value FROM mysql.innodb_index_stats WHERE table_name IN ('stats_rename1', 'stats_rename2'); +--error ER_DUP_KEY RENAME TABLE stats_rename1 TO stats_rename2; +BEGIN; +DELETE FROM mysql.innodb_table_stats WHERE table_name='stats_rename2'; +DELETE FROM mysql.innodb_index_stats WHERE table_name='stats_rename2'; +COMMIT; +RENAME TABLE stats_rename1 TO stats_rename2; + +UPDATE mysql.innodb_index_stats SET index_name='c' +WHERE table_name='stats_rename2' AND index_name='PRIMARY'; +--error ER_DUP_KEY +ALTER TABLE stats_rename2 CHANGE b d INT, RENAME INDEX b TO c; +UPDATE mysql.innodb_index_stats SET index_name='PRIMARY' +WHERE table_name='stats_rename2' AND index_name='c'; + +ALTER TABLE stats_rename2 CHANGE b d INT, RENAME INDEX b TO c; SELECT table_name, n_rows FROM mysql.innodb_table_stats diff --git a/mysql-test/suite/innodb/t/instant_alter_import.test b/mysql-test/suite/innodb/t/instant_alter_import.test index fb187debb51..208bc423a11 100644 --- a/mysql-test/suite/innodb/t/instant_alter_import.test +++ b/mysql-test/suite/innodb/t/instant_alter_import.test @@ -79,6 +79,7 @@ select * from t1; alter table t1 import tablespace; --error ER_TABLESPACE_DISCARDED select * from t1; +--remove_file $MYSQLD_DATADIR/test/t1.ibd drop table t2; drop table t1; diff --git a/mysql-test/suite/innodb/t/log_file_name.test b/mysql-test/suite/innodb/t/log_file_name.test index 534c4e6984f..2a7ed7b494d 100644 --- a/mysql-test/suite/innodb/t/log_file_name.test +++ b/mysql-test/suite/innodb/t/log_file_name.test @@ -137,7 +137,9 @@ DROP TABLE t2,t3; --error ER_CANT_CREATE_TABLE CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; - +--error ER_CANT_CREATE_TABLE +CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; +--remove_file $MYSQLD_DATADIR/test/t0.ibd CREATE TABLE t0(a INT PRIMARY KEY) ENGINE=InnoDB; DROP TABLE t0; diff --git a/mysql-test/suite/innodb/t/rename_table_debug.test b/mysql-test/suite/innodb/t/rename_table_debug.test deleted file mode 100644 index 3e2de242d49..00000000000 --- a/mysql-test/suite/innodb/t/rename_table_debug.test +++ /dev/null @@ -1,41 +0,0 @@ ---source include/have_innodb.inc ---source include/have_debug.inc ---source include/have_debug_sync.inc ---source include/not_embedded.inc - -FLUSH TABLES; -LET $datadir= `SELECT @@datadir`; - -CREATE TABLE t1 (a SERIAL, b INT, c INT, d INT) ENGINE=InnoDB; -INSERT INTO t1 () VALUES (); - ---connect (con1,localhost,root,,test) -SET DEBUG_SYNC='before_rename_table_commit SIGNAL renamed WAIT_FOR ever'; ---send -RENAME TABLE t1 TO t2; ---connection default -SET DEBUG_SYNC='now WAIT_FOR renamed'; ---let $shutdown_timeout=0 ---source include/restart_mysqld.inc ---disconnect con1 -SELECT * FROM t1; - -CREATE TABLE t2 (a INT PRIMARY KEY) ENGINE=InnoDB; -BEGIN; -INSERT INTO t2 VALUES(1); - ---connect (con1,localhost,root,,test) -SET DEBUG_SYNC='innodb_rename_in_cache SIGNAL committed WAIT_FOR ever'; ---send -RENAME TABLE t1 TO t3; ---connection default -SET DEBUG_SYNC='now WAIT_FOR committed'; -COMMIT; - ---let $shutdown_timeout=0 ---source include/restart_mysqld.inc ---disconnect con1 -SELECT * FROM t1; -SELECT * FROM t2; - -DROP TABLE t1,t2; diff --git a/mysql-test/suite/innodb/t/table_flags.test b/mysql-test/suite/innodb/t/table_flags.test index f3a2c856af2..380052b9db6 100644 --- a/mysql-test/suite/innodb/t/table_flags.test +++ b/mysql-test/suite/innodb/t/table_flags.test @@ -1,7 +1,6 @@ --source include/innodb_page_size.inc # Embedded server tests do not support restarting --source include/not_embedded.inc ---source include/maybe_debug.inc --disable_query_log call mtr.add_suppression("InnoDB: Table `mysql`\\.`innodb_table_stats` not found"); @@ -31,9 +30,6 @@ let bugdir= $MYSQLTEST_VARDIR/tmp/table_flags; --let $d=$d --innodb-undo-tablespaces=0 --let $d=$d --innodb-purge-rseg-truncate-frequency=1 --let $d=$d --skip-innodb-fast-shutdown -if ($have_debug) { ---let $d=$d --debug=d,create_and_drop_garbage -} --let $restart_noprint=1 --let $restart_parameters=$d --innodb-stats-persistent=0 --source include/restart_mysqld.inc diff --git a/mysql-test/suite/innodb/t/truncate.test b/mysql-test/suite/innodb/t/truncate.test index dc2b2a81484..436fc01b5a1 100644 --- a/mysql-test/suite/innodb/t/truncate.test +++ b/mysql-test/suite/innodb/t/truncate.test @@ -67,3 +67,15 @@ RENAME TABLE t TO u; TRUNCATE u; TRUNCATE u; DROP TABLE u; + +--echo # +--echo # Test for a regression found during MDEV-25506 rewrite of DROP +--echo # +CREATE TEMPORARY TABLE t1 (a INT) ENGINE=InnoDB; +LOCK TABLE t1 READ; +TRUNCATE TABLE t1; +TRUNCATE TABLE t1; +UNLOCK TABLES; +DROP TEMPORARY TABLE t1; + +--echo # End of 10.6 tests diff --git a/mysql-test/suite/innodb/t/xa_recovery.test b/mysql-test/suite/innodb/t/xa_recovery.test index bb8e3316860..7a67cd8e8b7 100644 --- a/mysql-test/suite/innodb/t/xa_recovery.test +++ b/mysql-test/suite/innodb/t/xa_recovery.test @@ -39,10 +39,10 @@ let $wait_condition= info = 'SELECT * FROM t1 LOCK IN SHARE MODE'; --source include/wait_condition.inc +SET innodb_lock_wait_timeout=1; +--error ER_LOCK_WAIT_TIMEOUT DROP TABLE t2; ---source include/restart_mysqld.inc - disconnect con1; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; @@ -51,8 +51,10 @@ XA ROLLBACK 'x'; SELECT * FROM t1; DROP TABLE t1; +--error ER_LOCK_WAIT_TIMEOUT +DROP TABLE t2; +XA ROLLBACK 'y'; +DROP TABLE t2; SET GLOBAL innodb_fast_shutdown=0; --source include/restart_mysqld.inc - -XA ROLLBACK 'y'; diff --git a/mysql-test/suite/innodb_fts/r/crash_recovery.result b/mysql-test/suite/innodb_fts/r/crash_recovery.result index 518e5007048..74287f509b7 100644 --- a/mysql-test/suite/innodb_fts/r/crash_recovery.result +++ b/mysql-test/suite/innodb_fts/r/crash_recovery.result @@ -33,6 +33,10 @@ connection default; disconnect ddl1; disconnect ddl2; disconnect ddl3; +SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency; +SET GLOBAL innodb_purge_rseg_truncate_frequency = 1; +InnoDB 0 transactions not purged +SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency; CHECK TABLE t1,t2,t3; Table Op Msg_type Msg_text test.t1 check status OK @@ -151,6 +155,5 @@ id title body 1 MySQL Tutorial DBMS stands for Database... 2 MariaDB Tutorial DB means Database ... DROP TABLE mdev19073, mdev19073_2; -SELECT * FROM information_schema.innodb_sys_tables -WHERE name LIKE 'test/%' AND name NOT LIKE 'test/#sql-ib%'; +SELECT * FROM information_schema.innodb_sys_tables WHERE name LIKE 'test/%'; TABLE_ID NAME FLAG N_COLS SPACE ROW_FORMAT ZIP_PAGE_SIZE SPACE_TYPE diff --git a/mysql-test/suite/innodb_fts/r/misc_debug.result b/mysql-test/suite/innodb_fts/r/misc_debug.result index 41f8d08ea7d..9143d3f48f0 100644 --- a/mysql-test/suite/innodb_fts/r/misc_debug.result +++ b/mysql-test/suite/innodb_fts/r/misc_debug.result @@ -20,12 +20,13 @@ DROP TABLE t; CREATE TABLE t1 (pk INT, a VARCHAR(8), PRIMARY KEY(pk), FULLTEXT KEY(a)) ENGINE=InnoDB; CREATE TABLE t2 (b INT, FOREIGN KEY(b) REFERENCES t1(pk)) ENGINE=InnoDB; -DROP TABLE t1; +DROP TABLE/*foo*/ t1; ERROR 23000: Cannot delete or update a parent row: a foreign key constraint fails SET DEBUG_DBUG="+d,fts_instrument_sync"; INSERT INTO t1 VALUES(1, "mariadb"); ALTER TABLE t1 FORCE; DROP TABLE t2, t1; +SET SESSION debug_dbug=@saved_debug_dbug; # # MDEV-25200 Index count mismatch due to aborted FULLTEXT INDEX # @@ -56,12 +57,10 @@ DROP TABLE t1; # MDEV-25663 Double free of transaction during TRUNCATE # call mtr.add_suppression("InnoDB: \\(Too many concurrent transactions\\)"); -SET DEBUG_DBUG='-d,ib_create_table_fail_too_many_trx'; CREATE TABLE t1 (b CHAR(12), FULLTEXT KEY(b)) engine=InnoDB; -SET @save_dbug= @@debug_dbug; SET debug_dbug='+d,ib_create_table_fail_too_many_trx'; TRUNCATE t1; ERROR HY000: Got error -1 "Internal error < 0 (Not system error)" from storage engine InnoDB -SET debug_dbug=@save_dbug; +SET debug_dbug=@saved_debug_dbug; DROP TABLE t1; # End of 10.3 tests diff --git a/mysql-test/suite/innodb_fts/t/crash_recovery.test b/mysql-test/suite/innodb_fts/t/crash_recovery.test index d5518b91590..229f5affe87 100644 --- a/mysql-test/suite/innodb_fts/t/crash_recovery.test +++ b/mysql-test/suite/innodb_fts/t/crash_recovery.test @@ -100,6 +100,14 @@ disconnect ddl1; disconnect ddl2; disconnect ddl3; +# Ensure that the history list length will actually be decremented by purge. +SET @saved_frequency = @@GLOBAL.innodb_purge_rseg_truncate_frequency; +SET GLOBAL innodb_purge_rseg_truncate_frequency = 1; +# Wait for purge, so that any #sql-ib.ibd files from the previous kill +# will be deleted. +source ../../innodb/include/wait_all_purged.inc; +SET GLOBAL innodb_purge_rseg_truncate_frequency = @saved_frequency; + CHECK TABLE t1,t2,t3; DROP TABLE t1,t2,t3; @@ -245,17 +253,4 @@ SELECT * FROM mdev19073_2 WHERE MATCH (title, body) AGAINST ('Database' IN NATURAL LANGUAGE MODE); DROP TABLE mdev19073, mdev19073_2; -if (!$have_debug) -{ ---disable_query_log -# Some errors are reported despite the MDEV-24626 fix. -call mtr.add_suppression("InnoDB: Cannot (read first page of|open datafile for read-only:) '\\./test/(FTS_|#sql-(alter|backup)-).*\\.ibd'"); -call mtr.add_suppression("InnoDB: Datafile '\\./test/(FTS_|#sql-(alter|backup)-).*\\.ibd' is corrupted"); -call mtr.add_suppression("InnoDB: (The error means|Operating system error)"); -call mtr.add_suppression("InnoDB: Ignoring tablespace for test/(FTS_|#sql-(backup|alter)-).* because it could not be opened\\."); -call mtr.add_suppression("InnoDB: Expected tablespace id [1-9][0-9]* but found 0 in the file .*/test/(FTS_|#sql-(alter|backup)-).*\\.ibd"); ---enable_query_log -} - -SELECT * FROM information_schema.innodb_sys_tables -WHERE name LIKE 'test/%' AND name NOT LIKE 'test/#sql-ib%'; +SELECT * FROM information_schema.innodb_sys_tables WHERE name LIKE 'test/%'; diff --git a/mysql-test/suite/innodb_fts/t/misc_debug.test b/mysql-test/suite/innodb_fts/t/misc_debug.test index 90cb84976ce..b9b0f54e3e8 100644 --- a/mysql-test/suite/innodb_fts/t/misc_debug.test +++ b/mysql-test/suite/innodb_fts/t/misc_debug.test @@ -48,12 +48,13 @@ CREATE TABLE t1 (pk INT, a VARCHAR(8), PRIMARY KEY(pk), FULLTEXT KEY(a)) ENGINE=InnoDB; CREATE TABLE t2 (b INT, FOREIGN KEY(b) REFERENCES t1(pk)) ENGINE=InnoDB; --error ER_ROW_IS_REFERENCED_2 -DROP TABLE t1; +DROP TABLE/*foo*/ t1; SET DEBUG_DBUG="+d,fts_instrument_sync"; INSERT INTO t1 VALUES(1, "mariadb"); ALTER TABLE t1 FORCE; # Cleanup DROP TABLE t2, t1; +SET SESSION debug_dbug=@saved_debug_dbug; --echo # --echo # MDEV-25200 Index count mismatch due to aborted FULLTEXT INDEX @@ -88,13 +89,11 @@ DROP TABLE t1; --echo # MDEV-25663 Double free of transaction during TRUNCATE --echo # call mtr.add_suppression("InnoDB: \\(Too many concurrent transactions\\)"); -SET DEBUG_DBUG='-d,ib_create_table_fail_too_many_trx'; CREATE TABLE t1 (b CHAR(12), FULLTEXT KEY(b)) engine=InnoDB; -SET @save_dbug= @@debug_dbug; SET debug_dbug='+d,ib_create_table_fail_too_many_trx'; --error ER_GET_ERRNO TRUNCATE t1; -SET debug_dbug=@save_dbug; +SET debug_dbug=@saved_debug_dbug; DROP TABLE t1; --echo # End of 10.3 tests diff --git a/mysql-test/suite/innodb_zip/r/wl5522_debug_zip.result b/mysql-test/suite/innodb_zip/r/wl5522_debug_zip.result index 8708f26a653..7221358efee 100644 --- a/mysql-test/suite/innodb_zip/r/wl5522_debug_zip.result +++ b/mysql-test/suite/innodb_zip/r/wl5522_debug_zip.result @@ -111,6 +111,7 @@ ALTER TABLE t1 IMPORT TABLESPACE; ERROR HY000: Index for table 't1' is corrupt; try to repair it SET SESSION debug_dbug=@saved_debug_dbug; restore: t1 .ibd and .cfg files +ALTER TABLE t1 IMPORT TABLESPACE; DROP TABLE t1; CREATE TABLE t1 ( c1 BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, diff --git a/mysql-test/suite/innodb_zip/t/restart.test b/mysql-test/suite/innodb_zip/t/restart.test index c01b4071b6c..baef95a66d3 100644 --- a/mysql-test/suite/innodb_zip/t/restart.test +++ b/mysql-test/suite/innodb_zip/t/restart.test @@ -162,11 +162,8 @@ SELECT count(*) FROM t7_restart; --echo # --source include/shutdown_mysqld.inc ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// - --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -174,7 +171,6 @@ SELECT count(*) FROM t7_restart; --list_files $MYSQL_TMP_DIR/alt_dir --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -263,13 +259,11 @@ SHOW CREATE TABLE t7_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -320,13 +314,11 @@ RENAME TABLE t5_restart TO t55_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -358,13 +350,11 @@ SHOW CREATE TABLE t77_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -404,19 +394,16 @@ SHOW CREATE TABLE t77_restart; --mkdir $MYSQL_TMP_DIR/new_dir/test --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/new_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/new_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -466,19 +453,16 @@ SHOW CREATE TABLE t77_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/alt_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/alt_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/new_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/new_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -522,13 +506,11 @@ SHOW CREATE TABLE t77_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/new_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/new_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt @@ -570,13 +552,11 @@ SHOW CREATE TABLE t77_restart; --echo ---- MYSQL_DATA_DIR/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_DATA_DIR/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt --echo ---- MYSQL_TMP_DIR/new_dir/test --list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQL_TMP_DIR/new_dir/test ---replace_regex $regexp --replace_result #P# #p# #SP# #sp# --cat_file $MYSQLD_DATADIR.files.txt --remove_file $MYSQLD_DATADIR.files.txt diff --git a/mysql-test/suite/innodb_zip/t/wl5522_debug_zip.test b/mysql-test/suite/innodb_zip/t/wl5522_debug_zip.test index 0c9bcb6cf9d..42f76a78ed9 100644 --- a/mysql-test/suite/innodb_zip/t/wl5522_debug_zip.test +++ b/mysql-test/suite/innodb_zip/t/wl5522_debug_zip.test @@ -280,6 +280,7 @@ do "$ENV{MTR_SUITE_DIR}/../innodb/include/innodb-util.pl"; ib_restore_tablespaces("test", "t1"); EOF +ALTER TABLE t1 IMPORT TABLESPACE; DROP TABLE t1; # diff --git a/mysql-test/suite/parts/inc/partition_crash.inc b/mysql-test/suite/parts/inc/partition_crash.inc index 32bf5c10423..c657ba880c9 100644 --- a/mysql-test/suite/parts/inc/partition_crash.inc +++ b/mysql-test/suite/parts/inc/partition_crash.inc @@ -6,7 +6,6 @@ --list_files_write_file $DATADIR.files.txt $DATADIR/test --replace_result #p# #P# #sp# #SP# ---replace_regex /#sql-ib[1-9][0-9]*\.ibd\n// --cat_file $DATADIR.files.txt --remove_file $DATADIR.files.txt SHOW CREATE TABLE t1; @@ -20,7 +19,7 @@ SELECT * FROM t1; --echo # State after crash (before recovery) --list_files_write_file $DATADIR.files.txt $DATADIR/test --replace_result #p# #P# #sp# #SP# #tmp# #TMP# ---replace_regex /sql-exchange.*\./sql-exchange./ /sql-shadow-[0-9a-f]*-/sql-shadow-/ /#sql-ib[1-9][0-9]*\.ibd\n// +--replace_regex /sql-exchange.*\./sql-exchange./ /sql-shadow-[0-9a-f]*-/sql-shadow-/ --cat_file $DATADIR.files.txt --remove_file $DATADIR.files.txt --exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect @@ -29,7 +28,6 @@ SELECT * FROM t1; --echo # State after crash recovery --list_files_write_file $DATADIR.files.txt $DATADIR/test --replace_result #p# #P# #sp# #SP# ---replace_regex /#sql-ib[1-9][0-9]*\.ibd\n// --cat_file $DATADIR.files.txt --remove_file $DATADIR.files.txt SHOW CREATE TABLE t1; diff --git a/mysql-test/suite/parts/t/partition_basic_symlink_innodb.test b/mysql-test/suite/parts/t/partition_basic_symlink_innodb.test index e6b2f4300cc..c0e3c21a12d 100644 --- a/mysql-test/suite/parts/t/partition_basic_symlink_innodb.test +++ b/mysql-test/suite/parts/t/partition_basic_symlink_innodb.test @@ -93,17 +93,9 @@ SHOW WARNINGS; --echo # Verifying .frm, .par, .isl & .ibd files --echo ---- MYSQLD_DATADIR/test ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQLD_DATADIR/test ---replace_regex $regexp ---cat_file $MYSQLD_DATADIR.files.txt ---remove_file $MYSQLD_DATADIR.files.txt +--list_files $MYSQLD_DATADIR/test --echo ---- MYSQLTEST_VARDIR/mysql-test-data-dir/test ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLTEST_VARDIR/files.txt $MYSQLTEST_VARDIR/mysql-test-data-dir/test ---replace_regex $regexp ---cat_file $MYSQLTEST_VARDIR/files.txt ---remove_file $MYSQLTEST_VARDIR/files.txt +--list_files $MYSQLTEST_VARDIR/mysql-test-data-dir/test --echo # The ibd tablespaces should not be directly under the DATA DIRECTORY --echo ---- MYSQLTEST_VARDIR/mysql-test-data-dir --list_files $MYSQLTEST_VARDIR/mysql-test-data-dir @@ -122,11 +114,7 @@ ALTER TABLE t1 engine=MyISAM; --replace_result $MYSQLTEST_VARDIR MYSQLTEST_VARDIR SHOW CREATE TABLE t1; --echo # Verifying .frm, .par and MyISAM files (.MYD, MYI) ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQLD_DATADIR/test ---replace_regex $regexp ---cat_file $MYSQLD_DATADIR.files.txt ---remove_file $MYSQLD_DATADIR.files.txt +--list_files $MYSQLD_DATADIR/test --echo ---- MYSQLTEST_VARDIR/mysql-test-data-dir --list_files $MYSQLTEST_VARDIR/mysql-test-data-dir --echo ---- MYSQLTEST_VARDIR/mysql-test-idx-dir @@ -142,21 +130,13 @@ ALTER TABLE t1 engine=InnoDB; SHOW CREATE TABLE t1; --echo # Verifying .frm, .par, .isl and InnoDB .ibd files --echo ---- MYSQLD_DATADIR/test ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQLD_DATADIR/test ---replace_regex $regexp ---cat_file $MYSQLD_DATADIR.files.txt ---remove_file $MYSQLD_DATADIR.files.txt +--list_files $MYSQLD_DATADIR/test --echo ---- MYSQLTEST_VARDIR/mysql-test-data-dir --list_files $MYSQLTEST_VARDIR/mysql-test-data-dir --echo ---- MYSQLTEST_VARDIR/mysql-test-idx-dir --list_files $MYSQLTEST_VARDIR/mysql-test-idx-dir --echo ---- MYSQLTEST_VARDIR/mysql-test-data-dir/test ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLTEST_VARDIR/files.txt $MYSQLTEST_VARDIR/mysql-test-data-dir/test ---replace_regex $regexp ---cat_file $MYSQLTEST_VARDIR/files.txt ---remove_file $MYSQLTEST_VARDIR/files.txt +--list_files $MYSQLTEST_VARDIR/mysql-test-data-dir/test DROP TABLE t1; diff --git a/mysql-test/suite/parts/t/partition_debug_sync_innodb.test b/mysql-test/suite/parts/t/partition_debug_sync_innodb.test index 61eb48faa11..d73333c3b8d 100644 --- a/mysql-test/suite/parts/t/partition_debug_sync_innodb.test +++ b/mysql-test/suite/parts/t/partition_debug_sync_innodb.test @@ -52,11 +52,7 @@ insert into t1 values (1), (11), (21), (33); SELECT * FROM t1; SHOW CREATE TABLE t1; --replace_result #p# #P# #sp# #SP# ---let $regexp=/#sql-ib[0-9a-f]+\.ibd\n// ---list_files_write_file $MYSQLD_DATADIR.files.txt $MYSQLD_DATADIR/test ---replace_regex $regexp ---cat_file $MYSQLD_DATADIR.files.txt ---remove_file $MYSQLD_DATADIR.files.txt +--list_files $MYSQLD_DATADIR/test SET DEBUG_SYNC='before_open_in_get_all_tables SIGNAL parked WAIT_FOR open'; SET DEBUG_SYNC='partition_open_error SIGNAL alter WAIT_FOR finish'; diff --git a/mysql-test/suite/sys_vars/r/sysvars_innodb.result b/mysql-test/suite/sys_vars/r/sysvars_innodb.result index ad359c29ee1..1a88710de94 100644 --- a/mysql-test/suite/sys_vars/r/sysvars_innodb.result +++ b/mysql-test/suite/sys_vars/r/sysvars_innodb.result @@ -81,18 +81,6 @@ NUMERIC_BLOCK_SIZE 0 ENUM_VALUE_LIST NULL READ_ONLY YES COMMAND_LINE_ARGUMENT REQUIRED -VARIABLE_NAME INNODB_BACKGROUND_DROP_LIST_EMPTY -SESSION_VALUE NULL -DEFAULT_VALUE OFF -VARIABLE_SCOPE GLOBAL -VARIABLE_TYPE BOOLEAN -VARIABLE_COMMENT Wait for the background drop list to become empty -NUMERIC_MIN_VALUE NULL -NUMERIC_MAX_VALUE NULL -NUMERIC_BLOCK_SIZE NULL -ENUM_VALUE_LIST OFF,ON -READ_ONLY NO -COMMAND_LINE_ARGUMENT OPTIONAL VARIABLE_NAME INNODB_BUFFER_POOL_CHUNK_SIZE SESSION_VALUE NULL DEFAULT_VALUE 134217728 diff --git a/sql/handler.cc b/sql/handler.cc index dbfdae5e6d0..fc2c46395c3 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -493,7 +493,6 @@ int ha_init_errors(void) SETMSG(HA_ERR_INDEX_COL_TOO_LONG, ER_DEFAULT(ER_INDEX_COLUMN_TOO_LONG)); SETMSG(HA_ERR_INDEX_CORRUPT, ER_DEFAULT(ER_INDEX_CORRUPT)); SETMSG(HA_FTS_INVALID_DOCID, "Invalid InnoDB FTS Doc ID"); - SETMSG(HA_ERR_TABLE_IN_FK_CHECK, ER_DEFAULT(ER_TABLE_IN_FK_CHECK)); SETMSG(HA_ERR_DISK_FULL, ER_DEFAULT(ER_DISK_FULL)); SETMSG(HA_ERR_FTS_TOO_MANY_WORDS_IN_PHRASE, "Too many words in a FTS phrase or proximity search"); SETMSG(HA_ERR_FK_DEPTH_EXCEEDED, "Foreign key cascade delete/update exceeds"); @@ -4259,9 +4258,6 @@ void handler::print_error(int error, myf errflag) case HA_ERR_UNDO_REC_TOO_BIG: textno= ER_UNDO_RECORD_TOO_BIG; break; - case HA_ERR_TABLE_IN_FK_CHECK: - textno= ER_TABLE_IN_FK_CHECK; - break; default: { /* The error was "unknown" to this function. diff --git a/sql/handler.h b/sql/handler.h index 71804a4e98c..a7c455ae7c9 100644 --- a/sql/handler.h +++ b/sql/handler.h @@ -1853,11 +1853,13 @@ handlerton *ha_default_tmp_handlerton(THD *thd); */ #define HTON_REQUIRES_CLOSE_AFTER_TRUNCATE (1 << 18) +/* Truncate requires that all other handlers are closed */ +#define HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE (1 << 19) /* Used by mysql_inplace_alter_table() to decide if we should call hton->notify_tabledef_changed() before commit (MyRocks) or after (InnoDB). */ -#define HTON_REQUIRES_NOTIFY_TABLEDEF_CHANGED_AFTER_COMMIT (1 << 19) +#define HTON_REQUIRES_NOTIFY_TABLEDEF_CHANGED_AFTER_COMMIT (1 << 20) class Ha_trx_info; diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt index ed68576b5b5..205f42f50a0 100644 --- a/sql/share/errmsg-utf8.txt +++ b/sql/share/errmsg-utf8.txt @@ -6636,8 +6636,8 @@ ER_BINLOG_UNSAFE_CREATE_SELECT_AUTOINC ER_BINLOG_UNSAFE_INSERT_TWO_KEYS eng "INSERT... ON DUPLICATE KEY UPDATE on a table with more than one UNIQUE KEY is unsafe" -ER_TABLE_IN_FK_CHECK - eng "Table is being used in foreign key check" +ER_UNUSED_28 + eng "You should never see it" ER_UNUSED_1 eng "You should never see it" diff --git a/sql/sql_base.cc b/sql/sql_base.cc index 6845fe80445..68524cbb1f2 100644 --- a/sql/sql_base.cc +++ b/sql/sql_base.cc @@ -2580,10 +2580,15 @@ void Locked_tables_list::mark_table_for_reopen(THD *thd, TABLE *table) { TABLE_SHARE *share= table->s; - for (TABLE_LIST *table_list= m_locked_tables; + for (TABLE_LIST *table_list= m_locked_tables; table_list; table_list= table_list->next_global) { - if (table_list->table->s == share) + /* + table_list->table can be NULL in the case of TRUNCATE TABLE where + the table was locked twice and one instance closed in + close_all_tables_for_name(). + */ + if (table_list->table && table_list->table->s == share) table_list->table->internal_set_needs_reopen(true); } /* This is needed in the case where lock tables where not used */ diff --git a/sql/sql_truncate.cc b/sql/sql_truncate.cc index 0f88a159d63..01e95d5f6b9 100644 --- a/sql/sql_truncate.cc +++ b/sql/sql_truncate.cc @@ -237,6 +237,21 @@ Sql_cmd_truncate_table::handler_truncate(THD *thd, TABLE_LIST *table_ref, DBUG_RETURN(TRUNCATE_FAILED_SKIP_BINLOG); table= table_ref->table; + + if ((table->file->ht->flags & HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE) && + !is_tmp_table) + { + if (wait_while_table_is_used(thd, table, HA_EXTRA_FORCE_REOPEN)) + DBUG_RETURN(TRUNCATE_FAILED_SKIP_BINLOG); + /* + Get rid of all TABLE instances belonging to this thread + except one to be used for TRUNCATE + */ + close_all_tables_for_name(thd, table->s, + HA_EXTRA_NOT_USED, + table); + } + error= table->file->ha_truncate(); if (!is_tmp_table && !error) @@ -366,8 +381,8 @@ bool Sql_cmd_truncate_table::lock_table(THD *thd, TABLE_LIST *table_ref, } } - *hton_can_recreate= !sequence - && ha_check_storage_engine_flag(hton, HTON_CAN_RECREATE); + *hton_can_recreate= (!sequence && + ha_check_storage_engine_flag(hton, HTON_CAN_RECREATE)); if (versioned) { @@ -496,10 +511,11 @@ bool Sql_cmd_truncate_table::truncate_table(THD *thd, TABLE_LIST *table_ref) if (error == TRUNCATE_OK && thd->locked_tables_mode && (table_ref->table->file->ht->flags & - HTON_REQUIRES_CLOSE_AFTER_TRUNCATE)) + (HTON_REQUIRES_CLOSE_AFTER_TRUNCATE | + HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE))) { thd->locked_tables_list.mark_table_for_reopen(thd, table_ref->table); - if (unlikely(thd->locked_tables_list.reopen_tables(thd, true))) + if (unlikely(thd->locked_tables_list.reopen_tables(thd, false))) thd->locked_tables_list.unlink_all_closed_tables(thd, NULL, 0); } diff --git a/storage/innobase/CMakeLists.txt b/storage/innobase/CMakeLists.txt index 222e671a900..e6e1c98f87d 100644 --- a/storage/innobase/CMakeLists.txt +++ b/storage/innobase/CMakeLists.txt @@ -48,6 +48,7 @@ SET(INNOBASE_SOURCES dict/dict0stats.cc dict/dict0stats_bg.cc dict/dict0defrag_bg.cc + dict/drop.cc eval/eval0eval.cc eval/eval0proc.cc fil/fil0fil.cc diff --git a/storage/innobase/btr/btr0btr.cc b/storage/innobase/btr/btr0btr.cc index 9b7ed30c3cd..07f2ee974c0 100644 --- a/storage/innobase/btr/btr0btr.cc +++ b/storage/innobase/btr/btr0btr.cc @@ -1196,21 +1196,29 @@ void btr_free_if_exists(fil_space_t *space, uint32_t page, } } -/** Free an index tree in a temporary tablespace. -@param[in] page_id root page id */ -void btr_free(const page_id_t page_id) +/** Drop a temporary table +@param table temporary table */ +void btr_drop_temporary_table(const dict_table_t &table) { - mtr_t mtr; - mtr.start(); - mtr.set_log_mode(MTR_LOG_NO_REDO); - - buf_block_t* block = buf_page_get(page_id, 0, RW_X_LATCH, &mtr); - - if (block) { - btr_free_but_not_root(block, MTR_LOG_NO_REDO); - btr_free_root(block, &mtr); - } - mtr.commit(); + ut_ad(table.is_temporary()); + ut_ad(table.space == fil_system.temp_space); + mtr_t mtr; + mtr.start(); + for (const dict_index_t *index= table.indexes.start; index; + index= dict_table_get_next_index(index)) + { + if (buf_block_t *block= buf_page_get_low({SRV_TMP_SPACE_ID, index->page}, 0, + RW_X_LATCH, nullptr, BUF_GET, &mtr, + nullptr, false)) + { + btr_free_but_not_root(block, MTR_LOG_NO_REDO); + mtr.set_log_mode(MTR_LOG_NO_REDO); + btr_free_root(block, &mtr); + mtr.commit(); + mtr.start(); + } + } + mtr.commit(); } /** Read the last used AUTO_INCREMENT value from PAGE_ROOT_AUTO_INC. diff --git a/storage/innobase/btr/btr0cur.cc b/storage/innobase/btr/btr0cur.cc index c02fb72a102..21b51f2410b 100644 --- a/storage/innobase/btr/btr0cur.cc +++ b/storage/innobase/btr/btr0cur.cc @@ -1404,22 +1404,19 @@ btr_cur_search_to_nth_level_func( info->n_searches++; # endif if (autoinc == 0 - && latch_mode <= BTR_MODIFY_LEAF - && info->last_hash_succ -# ifdef MYSQL_INDEX_DISABLE_AHI - && !index->disable_ahi -# endif && !estimate -# ifdef PAGE_CUR_LE_OR_EXTENDS - && mode != PAGE_CUR_LE_OR_EXTENDS -# endif /* PAGE_CUR_LE_OR_EXTENDS */ - && !dict_index_is_spatial(index) + && latch_mode <= BTR_MODIFY_LEAF + && !modify_external /* If !ahi_latch, we do a dirty read of btr_search_enabled below, and btr_search_guess_on_hash() will have to check it again. */ && btr_search_enabled - && !modify_external +# ifdef PAGE_CUR_LE_OR_EXTENDS + && mode != PAGE_CUR_LE_OR_EXTENDS +# endif /* PAGE_CUR_LE_OR_EXTENDS */ + && info->last_hash_succ && !(tuple->info_bits & REC_INFO_MIN_REC_FLAG) + && !index->is_spatial() && !index->table->is_temporary() && btr_search_guess_on_hash(index, info, tuple, mode, latch_mode, cursor, ahi_latch, mtr)) { @@ -2443,9 +2440,6 @@ need_opposite_intention: btr_search_build_page_hash_index() before building a page hash index, while holding search latch. */ if (!btr_search_enabled) { -# ifdef MYSQL_INDEX_DISABLE_AHI - } else if (index->disable_ahi) { -# endif } else if (tuple->info_bits & REC_INFO_MIN_REC_FLAG) { ut_ad(index->is_instant()); /* This may be a search tuple for @@ -2453,6 +2447,8 @@ need_opposite_intention: ut_ad(tuple->is_metadata() || (tuple->is_metadata(tuple->info_bits ^ REC_STATUS_INSTANT))); + } else if (index->is_spatial()) { + } else if (index->table->is_temporary()) { } else if (rec_is_metadata(btr_cur_get_rec(cursor), *index)) { /* Only user records belong in the adaptive hash index. */ @@ -3602,13 +3598,11 @@ fail_err: #ifdef BTR_CUR_HASH_ADAPT if (!leaf) { -# ifdef MYSQL_INDEX_DISABLE_AHI - } else if (index->disable_ahi) { -# endif } else if (entry->info_bits & REC_INFO_MIN_REC_FLAG) { ut_ad(entry->is_metadata()); ut_ad(index->is_instant()); ut_ad(flags == BTR_NO_LOCKING_FLAG); + } else if (index->table->is_temporary()) { } else { srw_lock* ahi_latch = btr_search_sys.get_latch(*index); if (!reorg && cursor->flag == BTR_CUR_HASH) { @@ -3811,14 +3805,12 @@ btr_cur_pessimistic_insert( ut_ad(!big_rec_vec); } else { #ifdef BTR_CUR_HASH_ADAPT -# ifdef MYSQL_INDEX_DISABLE_AHI - if (index->disable_ahi); else -# endif if (entry->info_bits & REC_INFO_MIN_REC_FLAG) { ut_ad(entry->is_metadata()); ut_ad(index->is_instant()); ut_ad(flags & BTR_NO_LOCKING_FLAG); ut_ad(!(flags & BTR_CREATE_FLAG)); + } else if (index->table->is_temporary()) { } else { btr_search_update_hash_on_insert( cursor, btr_search_sys.get_latch(*index)); diff --git a/storage/innobase/btr/btr0sea.cc b/storage/innobase/btr/btr0sea.cc index ccf4992d335..7f110541fee 100644 --- a/storage/innobase/btr/btr0sea.cc +++ b/storage/innobase/btr/btr0sea.cc @@ -293,11 +293,7 @@ is NOT protected by any semaphore, to save CPU time! Do not assume its fields are consistent. @param[in,out] info search info @param[in] cursor cursor which was just positioned */ -static -void -btr_search_info_update_hash( - btr_search_t* info, - btr_cur_t* cursor) +static void btr_search_info_update_hash(btr_search_t *info, btr_cur_t *cursor) { dict_index_t* index = cursor->index; int cmp; @@ -1280,7 +1276,6 @@ retry: assert_block_ahi_valid(block); - if (!index || !btr_search_enabled) { if (is_freed) { part->latch.wr_unlock(); @@ -1290,9 +1285,7 @@ retry: return; } -#ifdef MYSQL_INDEX_DISABLE_AHI - ut_ad(!index->disable_ahi); -#endif + ut_ad(!index->table->is_temporary()); ut_ad(btr_search_enabled); ut_ad(block->page.id().space() == index->table->space_id); @@ -1479,9 +1472,8 @@ btr_search_build_page_hash_index( rec_offs offsets_[REC_OFFS_NORMAL_SIZE]; rec_offs* offsets = offsets_; -#ifdef MYSQL_INDEX_DISABLE_AHI - if (index->disable_ahi) return; -#endif + ut_ad(!index->table->is_temporary()); + if (!btr_search_enabled) { return; } @@ -1661,8 +1653,7 @@ exit_func: /** Updates the search info. @param[in,out] info search info @param[in,out] cursor cursor which was just positioned */ -void -btr_search_info_update_slow(btr_search_t* info, btr_cur_t* cursor) +void btr_search_info_update_slow(btr_search_t *info, btr_cur_t *cursor) { srw_lock* ahi_latch = &btr_search_sys.get_part(*cursor->index) ->latch; @@ -1779,7 +1770,7 @@ drop_exit: /** Updates the page hash index when a single record is deleted from a page. @param[in] cursor cursor which was positioned on the record to delete using btr_cur_search_, the record is not yet deleted.*/ -void btr_search_update_hash_on_delete(btr_cur_t* cursor) +void btr_search_update_hash_on_delete(btr_cur_t *cursor) { buf_block_t* block; const rec_t* rec; @@ -1790,9 +1781,6 @@ void btr_search_update_hash_on_delete(btr_cur_t* cursor) rec_offs_init(offsets_); ut_ad(page_is_leaf(btr_cur_get_page(cursor))); -#ifdef MYSQL_INDEX_DISABLE_AHI - if (cursor->index->disable_ahi) return; -#endif if (!btr_search_enabled) { return; @@ -1810,6 +1798,8 @@ void btr_search_update_hash_on_delete(btr_cur_t* cursor) return; } + ut_ad(!cursor->index->table->is_temporary()); + if (index != cursor->index) { btr_search_drop_page_hash_index(block); return; @@ -1864,9 +1854,7 @@ void btr_search_update_hash_node_on_insert(btr_cur_t *cursor, rec_t* rec; ut_ad(ahi_latch == &btr_search_sys.get_part(*cursor->index)->latch); -#ifdef MYSQL_INDEX_DISABLE_AHI - if (cursor->index->disable_ahi) return; -#endif + if (!btr_search_enabled) { return; } @@ -1884,6 +1872,8 @@ void btr_search_update_hash_node_on_insert(btr_cur_t *cursor, return; } + ut_ad(!cursor->index->table->is_temporary()); + if (index != cursor->index) { ut_ad(index->id == cursor->index->id); btr_search_drop_page_hash_index(block); @@ -1949,9 +1939,7 @@ void btr_search_update_hash_on_insert(btr_cur_t *cursor, ut_ad(ahi_latch == &btr_search_sys.get_part(*cursor->index)->latch); ut_ad(page_is_leaf(btr_cur_get_page(cursor))); -#ifdef MYSQL_INDEX_DISABLE_AHI - if (cursor->index->disable_ahi) return; -#endif + if (!btr_search_enabled) { return; } @@ -1973,9 +1961,8 @@ void btr_search_update_hash_on_insert(btr_cur_t *cursor, rec = btr_cur_get_rec(cursor); -#ifdef MYSQL_INDEX_DISABLE_AHI - ut_a(!index->disable_ahi); -#endif + ut_ad(!cursor->index->table->is_temporary()); + if (index != cursor->index) { ut_ad(index->id == cursor->index->id); btr_search_drop_page_hash_index(block); diff --git a/storage/innobase/dict/dict0crea.cc b/storage/innobase/dict/dict0crea.cc index 4782268aa44..70d9e2bb022 100644 --- a/storage/innobase/dict/dict0crea.cc +++ b/storage/innobase/dict/dict0crea.cc @@ -897,17 +897,6 @@ rec_corrupted: return 0; } -/** @return whether SYS_TABLES.NAME is for a '#sql-ib' table */ -bool dict_table_t::is_garbage_name(const void *data, size_t size) -{ - constexpr size_t suffix= sizeof TEMP_FILE_PREFIX_INNODB; - if (size <= suffix) - return false; - const char *f= static_cast<const char*>(memchr(data, '/', size - suffix)); - return f && !memcmp(f + 1, TEMP_FILE_PREFIX_INNODB, - (sizeof TEMP_FILE_PREFIX_INNODB) - 1); -} - /*********************************************************************//** Creates a table create graph. @return own: table create node */ @@ -1388,34 +1377,15 @@ dberr_t dict_sys_t::create_or_check_sys_tables() return DB_SUCCESS; trx_t *trx= trx_create(); - trx->dict_operation = true; + trx->dict_operation= true; row_mysql_lock_data_dictionary(trx); - DBUG_EXECUTE_IF("create_and_drop_garbage", - ut_ad(DB_SUCCESS == que_eval_sql( - nullptr, - "PROCEDURE CREATE_GARBAGE_TABLE_PROC () IS\n" - "BEGIN\n" - "CREATE TABLE\n" - "\"test/" TEMP_FILE_PREFIX_INNODB "-garbage\"" - "(ID CHAR);\n" - "CREATE UNIQUE CLUSTERED INDEX PRIMARY ON " - "\"test/" TEMP_FILE_PREFIX_INNODB - "-garbage\"(ID);\n" - "END;\n", false, trx)); - row_drop_table_for_mysql("test/" - TEMP_FILE_PREFIX_INNODB "-garbage", - trx, SQLCOM_DROP_DB, true);); - /* NOTE: when designing InnoDB's foreign key support in 2001, Heikki Tuuri - made a mistake defined table names and the foreign key id to be of type - 'CHAR' (internally, really a VARCHAR). - The type should have been VARBINARY. */ + made a mistake and defined table names and the foreign key id to be of type + CHAR (internally, really VARCHAR). The type should have been VARBINARY. */ + /* System tables are always created inside the system tablespace. */ const auto srv_file_per_table_backup= srv_file_per_table; - - /* We always want SYSTEM tables to be created inside the system - tablespace. */ srv_file_per_table= 0; dberr_t error; const char *tablename; diff --git a/storage/innobase/dict/dict0dict.cc b/storage/innobase/dict/dict0dict.cc index a1d68ecbc3e..3997989233b 100644 --- a/storage/innobase/dict/dict0dict.cc +++ b/storage/innobase/dict/dict0dict.cc @@ -716,18 +716,18 @@ bool dict_table_t::parse_name(char (&db_name)[NAME_LEN + 1], const size_t db_len= name.dblen(); ut_ad(db_len <= MAX_DATABASE_NAME_LEN); - memcpy(db_buf, name.m_name, db_len); + memcpy(db_buf, mdl_name.m_name, db_len); db_buf[db_len]= 0; - size_t tbl_len= strlen(name.m_name + db_len + 1); - const bool is_temp= name.is_temporary(); + size_t tbl_len= strlen(mdl_name.m_name + db_len + 1); + const bool is_temp= mdl_name.is_temporary(); if (is_temp); else if (const char *is_part= static_cast<const char*> - (memchr(name.m_name + db_len + 1, '#', tbl_len))) - tbl_len= static_cast<size_t>(is_part - &name.m_name[db_len + 1]); + (memchr(mdl_name.m_name + db_len + 1, '#', tbl_len))) + tbl_len= static_cast<size_t>(is_part - &mdl_name.m_name[db_len + 1]); - memcpy(tbl_buf, name.m_name + db_len + 1, tbl_len); + memcpy(tbl_buf, mdl_name.m_name + db_len + 1, tbl_len); tbl_buf[tbl_len]= 0; if (!dict_locked) @@ -1019,13 +1019,13 @@ void dict_sys_t::create() /** Acquire a reference to a cached table. */ -inline void dict_sys_t::acquire(dict_table_t* table) +inline void dict_sys_t::acquire(dict_table_t *table) { ut_ad(dict_sys.find(table)); if (table->can_be_evicted) { - UT_LIST_REMOVE(dict_sys.table_LRU, table); - UT_LIST_ADD_FIRST(dict_sys.table_LRU, table); + UT_LIST_REMOVE(table_LRU, table); + UT_LIST_ADD_FIRST(table_LRU, table); } table->acquire(); @@ -1490,25 +1490,7 @@ dict_table_t::rename_tablespace(const char *new_name, bool replace) const ut_ad(!is_temporary()); if (!space) - { - const char *data_dir= DICT_TF_HAS_DATA_DIR(flags) - ? data_dir_path : nullptr; - ut_ad(data_dir || !DICT_TF_HAS_DATA_DIR(flags)); - - if (char *filepath= fil_make_filepath(data_dir, name, IBD, - data_dir != nullptr)) - { - fil_delete_tablespace(space_id, true); - os_file_type_t ftype; - bool exists; - /* Delete any temp file hanging around. */ - if (os_file_status(filepath, &exists, &ftype) && exists && - !os_file_delete_if_exists(innodb_temp_file_key, filepath, nullptr)) - ib::info() << "Delete of " << filepath << " failed."; - ut_free(filepath); - } return DB_SUCCESS; - } const char *old_path= UT_LIST_GET_FIRST(space->chain)->name; fil_space_t::name_type space_name{new_name, strlen(new_name)}; @@ -1594,17 +1576,33 @@ dict_table_rename_in_cache( HASH_DELETE(dict_table_t, name_hash, &dict_sys.table_hash, ut_fold_string(old_name), table); - if (strlen(new_name) > strlen(table->name.m_name)) { + const bool keep_mdl_name = dict_table_t::is_temporary_name(new_name) + && !table->name.is_temporary(); + + if (keep_mdl_name) { + /* Preserve the original table name for + dict_table_t::parse_name() and dict_acquire_mdl_shared(). */ + table->mdl_name.m_name = mem_heap_strdup(table->heap, + table->name.m_name); + } + + const size_t new_len = strlen(new_name); + + if (new_len > strlen(table->name.m_name)) { /* We allocate MAX_FULL_NAME_LEN + 1 bytes here to avoid memory fragmentation, we assume a repeated calls of ut_realloc() with the same size do not cause fragmentation */ - ut_a(strlen(new_name) <= MAX_FULL_NAME_LEN); + ut_a(new_len <= MAX_FULL_NAME_LEN); table->name.m_name = static_cast<char*>( ut_realloc(table->name.m_name, MAX_FULL_NAME_LEN + 1)); } strcpy(table->name.m_name, new_name); + if (!keep_mdl_name) { + table->mdl_name.m_name = table->name.m_name; + } + /* Add table to hash table of tables */ HASH_INSERT(dict_table_t, name_hash, &dict_sys.table_hash, fold, table); @@ -2065,9 +2063,6 @@ dict_index_add_to_cache( new_index->trx_id = index->trx_id; new_index->set_committed(index->is_committed()); new_index->nulls_equal = index->nulls_equal; -#ifdef MYSQL_INDEX_DISABLE_AHI - new_index->disable_ahi = index->disable_ahi; -#endif n_ord = new_index->n_uniq; /* Flag the ordering columns and also set column max_prefix */ diff --git a/storage/innobase/dict/dict0load.cc b/storage/innobase/dict/dict0load.cc index 3fe0104f484..2b9078a1c85 100644 --- a/storage/innobase/dict/dict0load.cc +++ b/storage/innobase/dict/dict0load.cc @@ -853,14 +853,6 @@ static ulint dict_check_sys_tables() rec_get_nth_field_old(rec, DICT_FLD__SYS_TABLES__NAME, &len)); - if (len == UNIV_SQL_NULL - || dict_table_t::is_garbage_name(field, len)) { - /* This table will be dropped by - dict_table_t::drop_garbage(). - We do not care if the file exists. */ - continue; - } - DBUG_PRINT("dict_check_sys_tables", ("name: %*.s", static_cast<int>(len), field)); @@ -2451,7 +2443,7 @@ corrupted: << " failed, the table has missing" " foreign key indexes. Turn off" " 'foreign_key_checks' and try again."; - +evict: dict_sys.remove(table); table = NULL; } else { @@ -2468,8 +2460,9 @@ corrupted: if (!srv_force_recovery || !index || !index->is_primary()) { - dict_sys.remove(table); - table = NULL; + ib::warn() << "Failed to load table " << table->name + << ":" << err; + goto evict; } else if (index->is_corrupted() && table->is_readable()) { /* It is possible we force to load a corrupted diff --git a/storage/innobase/dict/dict0mem.cc b/storage/innobase/dict/dict0mem.cc index 0631bd24bc0..5f273a60be8 100644 --- a/storage/innobase/dict/dict0mem.cc +++ b/storage/innobase/dict/dict0mem.cc @@ -159,6 +159,7 @@ dict_table_t *dict_table_t::create(const span<const char> &name, table->flags= static_cast<unsigned>(flags) & ((1U << DICT_TF_BITS) - 1); table->flags2= static_cast<unsigned>(flags2) & ((1U << DICT_TF2_BITS) - 1); table->name.m_name= mem_strdupl(name.data(), name.size()); + table->mdl_name.m_name= table->name.m_name; table->is_system_db= dict_mem_table_is_system(table->name.m_name); table->space= space; table->space_id= space ? space->id : ULINT_UNDEFINED; @@ -221,7 +222,6 @@ dict_mem_table_free( table->referenced_set.~dict_foreign_set(); ut_free(table->name.m_name); - table->name.m_name = NULL; /* Clean up virtual index info structures that are registered with virtual columns */ diff --git a/storage/innobase/dict/dict0stats.cc b/storage/innobase/dict/dict0stats.cc index e9e91f730d5..3bc9ed79318 100644 --- a/storage/innobase/dict/dict0stats.cc +++ b/storage/innobase/dict/dict0stats.cc @@ -529,61 +529,35 @@ free the trx object. If it is not NULL then it will be rolled back only in the case of error, but not freed. @return DB_SUCCESS or error code */ static -dberr_t -dict_stats_exec_sql( - pars_info_t* pinfo, - const char* sql, - trx_t* trx) +dberr_t dict_stats_exec_sql(pars_info_t *pinfo, const char* sql, trx_t *trx) { - dberr_t err; - bool trx_started = false; - - ut_d(dict_sys.assert_locked()); - - if (!dict_stats_persistent_storage_check(true)) { - pars_info_free(pinfo); - return(DB_STATS_DO_NOT_EXIST); - } - - if (trx == NULL) { - trx = trx_create(); - trx_started = true; + ut_d(dict_sys.assert_locked()); - if (srv_read_only_mode) { - trx_start_internal_read_only(trx); - } else { - trx_start_internal(trx); - } - } - - err = que_eval_sql(pinfo, sql, FALSE, trx); /* pinfo is freed here */ - - DBUG_EXECUTE_IF("stats_index_error", - if (!trx_started) { - err = DB_STATS_DO_NOT_EXIST; - trx->error_state = DB_STATS_DO_NOT_EXIST; - }); + if (!dict_stats_persistent_storage_check(true)) + { + pars_info_free(pinfo); + return DB_STATS_DO_NOT_EXIST; + } - if (!trx_started && err == DB_SUCCESS) { - return(DB_SUCCESS); - } + if (trx) + return que_eval_sql(pinfo, sql, FALSE, trx); - if (err == DB_SUCCESS) { - trx_commit_for_mysql(trx); - } else { - trx->op_info = "rollback of internal trx on stats tables"; - trx->dict_operation_lock_mode = RW_X_LATCH; - trx->rollback(); - trx->dict_operation_lock_mode = 0; - trx->op_info = ""; - ut_a(trx->error_state == DB_SUCCESS); - } + trx= trx_create(); + if (srv_read_only_mode) + trx_start_internal_read_only(trx); + else + trx_start_internal(trx); - if (trx_started) { - trx->free(); - } + trx->dict_operation_lock_mode= RW_X_LATCH; + dberr_t err= que_eval_sql(pinfo, sql, FALSE, trx); - return(err); + if (err == DB_SUCCESS) + trx->commit(); + else + trx->rollback(); + trx->dict_operation_lock_mode= 0; + trx->free(); + return err; } /*********************************************************************//** @@ -671,6 +645,7 @@ dict_stats_table_clone_create( t->heap = heap; t->name.m_name = mem_heap_strdup(heap, table->name.m_name); + t->mdl_name.m_name = t->name.m_name; t->corrupted = table->corrupted; @@ -2749,6 +2724,9 @@ dict_stats_save( table_utf8, sizeof(table_utf8)); const time_t now = time(NULL); + trx_t* trx = trx_create(); + trx_start_internal(trx); + trx->dict_operation_lock_mode = RW_X_LATCH; dict_sys_lock(); pinfo = pars_info_create(); @@ -2782,20 +2760,21 @@ dict_stats_save( ":clustered_index_size,\n" ":sum_of_other_index_sizes\n" ");\n" - "END;", NULL); + "END;", trx); if (UNIV_UNLIKELY(ret != DB_SUCCESS)) { ib::error() << "Cannot save table statistics for table " << table->name << ": " << ret; -func_exit: +rollback_and_exit: + trx->rollback(); +free_and_exit: + trx->dict_operation_lock_mode = 0; dict_sys_unlock(); + trx->free(); dict_stats_snapshot_free(table); return ret; } - trx_t* trx = trx_create(); - trx_start_internal(trx); - dict_index_t* index; index_map_t indexes( (ut_strcmp_functor()), @@ -2864,7 +2843,7 @@ func_exit: stat_description, trx); if (ret != DB_SUCCESS) { - goto end; + goto rollback_and_exit; } } @@ -2874,7 +2853,7 @@ func_exit: "Number of leaf pages " "in the index", trx); if (ret != DB_SUCCESS) { - goto end; + goto rollback_and_exit; } ret = dict_stats_save_index_stat(index, now, "size", @@ -2883,15 +2862,12 @@ func_exit: "Number of pages " "in the index", trx); if (ret != DB_SUCCESS) { - goto end; + goto rollback_and_exit; } } - trx_commit_for_mysql(trx); - -end: - trx->free(); - goto func_exit; + trx->commit(); + goto free_and_exit; } /*********************************************************************//** @@ -3657,112 +3633,15 @@ transient: return(DB_SUCCESS); } -/** Remove the information for a particular index's stats from the persistent -storage if it exists and if there is data stored for this index. -This function creates its own trx and commits it. - -We must modify system tables in a separate transaction in order to -adhere to the InnoDB design constraint that dict_sys.latch prevents -lock waits on system tables. If we modified system and user tables in -the same transaction, we should exclusively hold dict_sys.latch until -the transaction is committed, and effectively block other transactions -that will attempt to open any InnoDB tables. Because we have no -guarantee that user transactions will be committed fast, we cannot -afford to keep the system tables locked in a user transaction. +/** Execute DELETE FROM mysql.innodb_table_stats +@param database_name database name +@param table_name table name +@param trx transaction (nullptr=start and commit a new one) @return DB_SUCCESS or error code */ -dberr_t -dict_stats_drop_index( -/*==================*/ - const char* db_and_table,/*!< in: db and table, e.g. 'db/table' */ - const char* iname, /*!< in: index name */ - char* errstr, /*!< out: error message if != DB_SUCCESS - is returned */ - ulint errstr_sz)/*!< in: size of the errstr buffer */ +dberr_t dict_stats_delete_from_table_stats(const char *database_name, + const char *table_name, trx_t *trx) { - char db_utf8[MAX_DB_UTF8_LEN]; - char table_utf8[MAX_TABLE_UTF8_LEN]; pars_info_t* pinfo; - dberr_t ret; - - dict_sys.assert_not_locked(); - - /* skip indexes whose table names do not contain a database name - e.g. if we are dropping an index from SYS_TABLES */ - if (strchr(db_and_table, '/') == NULL) { - - return(DB_SUCCESS); - } - - dict_fs2utf8(db_and_table, db_utf8, sizeof(db_utf8), - table_utf8, sizeof(table_utf8)); - - pinfo = pars_info_create(); - - pars_info_add_str_literal(pinfo, "database_name", db_utf8); - - pars_info_add_str_literal(pinfo, "table_name", table_utf8); - - pars_info_add_str_literal(pinfo, "index_name", iname); - - dict_sys_lock(); - - ret = dict_stats_exec_sql( - pinfo, - "PROCEDURE DROP_INDEX_STATS () IS\n" - "BEGIN\n" - "DELETE FROM \"" INDEX_STATS_NAME "\" WHERE\n" - "database_name = :database_name AND\n" - "table_name = :table_name AND\n" - "index_name = :index_name;\n" - "END;\n", NULL); - - dict_sys_unlock(); - - if (ret == DB_STATS_DO_NOT_EXIST) { - ret = DB_SUCCESS; - } - - if (ret != DB_SUCCESS) { - snprintf(errstr, errstr_sz, - "Unable to delete statistics for index %s" - " from %s%s: %s. They can be deleted later using" - " DELETE FROM %s WHERE" - " database_name = '%s' AND" - " table_name = '%s' AND" - " index_name = '%s';", - iname, - INDEX_STATS_NAME_PRINT, - (ret == DB_LOCK_WAIT_TIMEOUT - ? " because the rows are locked" - : ""), - ut_strerr(ret), - INDEX_STATS_NAME_PRINT, - db_utf8, - table_utf8, - iname); - - ut_print_timestamp(stderr); - fprintf(stderr, " InnoDB: %s\n", errstr); - } - - return(ret); -} - -/*********************************************************************//** -Executes -DELETE FROM mysql.innodb_table_stats -WHERE database_name = '...' AND table_name = '...'; -Creates its own transaction and commits it. -@return DB_SUCCESS or error code */ -UNIV_INLINE -dberr_t -dict_stats_delete_from_table_stats( -/*===============================*/ - const char* database_name, /*!< in: database name, e.g. 'db' */ - const char* table_name) /*!< in: table name, e.g. 'table' */ -{ - pars_info_t* pinfo; - dberr_t ret; ut_d(dict_sys.assert_locked()); @@ -3771,33 +3650,25 @@ dict_stats_delete_from_table_stats( pars_info_add_str_literal(pinfo, "database_name", database_name); pars_info_add_str_literal(pinfo, "table_name", table_name); - ret = dict_stats_exec_sql( + return dict_stats_exec_sql( pinfo, "PROCEDURE DELETE_FROM_TABLE_STATS () IS\n" "BEGIN\n" "DELETE FROM \"" TABLE_STATS_NAME "\" WHERE\n" "database_name = :database_name AND\n" "table_name = :table_name;\n" - "END;\n", NULL); - - return(ret); + "END;\n", trx); } -/*********************************************************************//** -Executes -DELETE FROM mysql.innodb_index_stats -WHERE database_name = '...' AND table_name = '...'; -Creates its own transaction and commits it. +/** Execute DELETE FROM mysql.innodb_index_stats +@param database_name database name +@param table_name table name +@param trx transaction (nullptr=start and commit a new one) @return DB_SUCCESS or error code */ -UNIV_INLINE -dberr_t -dict_stats_delete_from_index_stats( -/*===============================*/ - const char* database_name, /*!< in: database name, e.g. 'db' */ - const char* table_name) /*!< in: table name, e.g. 'table' */ +dberr_t dict_stats_delete_from_index_stats(const char *database_name, + const char *table_name, trx_t *trx) { pars_info_t* pinfo; - dberr_t ret; ut_d(dict_sys.assert_locked()); @@ -3806,375 +3677,144 @@ dict_stats_delete_from_index_stats( pars_info_add_str_literal(pinfo, "database_name", database_name); pars_info_add_str_literal(pinfo, "table_name", table_name); - ret = dict_stats_exec_sql( + return dict_stats_exec_sql( pinfo, "PROCEDURE DELETE_FROM_INDEX_STATS () IS\n" "BEGIN\n" "DELETE FROM \"" INDEX_STATS_NAME "\" WHERE\n" "database_name = :database_name AND\n" "table_name = :table_name;\n" - "END;\n", NULL); - - return(ret); -} - -/*********************************************************************//** -Removes the statistics for a table and all of its indexes from the -persistent statistics storage if it exists and if there is data stored for -the table. This function creates its own transaction and commits it. -@return DB_SUCCESS or error code */ -dberr_t -dict_stats_drop_table( -/*==================*/ - const char* db_and_table, /*!< in: db and table, e.g. 'db/table' */ - char* errstr, /*!< out: error message - if != DB_SUCCESS is returned */ - ulint errstr_sz) /*!< in: size of errstr buffer */ -{ - char db_utf8[MAX_DB_UTF8_LEN]; - char table_utf8[MAX_TABLE_UTF8_LEN]; - dberr_t ret; - - ut_d(dict_sys.assert_locked()); - - /* skip tables that do not contain a database name - e.g. if we are dropping SYS_TABLES */ - if (strchr(db_and_table, '/') == NULL) { - - return(DB_SUCCESS); - } - - /* skip innodb_table_stats and innodb_index_stats themselves */ - if (strcmp(db_and_table, TABLE_STATS_NAME) == 0 - || strcmp(db_and_table, INDEX_STATS_NAME) == 0) { - - return(DB_SUCCESS); - } - - dict_fs2utf8(db_and_table, db_utf8, sizeof(db_utf8), - table_utf8, sizeof(table_utf8)); - - ret = dict_stats_delete_from_table_stats(db_utf8, table_utf8); - - if (ret == DB_SUCCESS) { - ret = dict_stats_delete_from_index_stats(db_utf8, table_utf8); - } - - if (ret == DB_STATS_DO_NOT_EXIST) { - ret = DB_SUCCESS; - } - - if (ret != DB_SUCCESS) { - - snprintf(errstr, errstr_sz, - "Unable to delete statistics for table %s.%s: %s." - " They can be deleted later using" - - " DELETE FROM %s WHERE" - " database_name = '%s' AND" - " table_name = '%s';" - - " DELETE FROM %s WHERE" - " database_name = '%s' AND" - " table_name = '%s';", - - db_utf8, table_utf8, - ut_strerr(ret), - - INDEX_STATS_NAME_PRINT, - db_utf8, table_utf8, - - TABLE_STATS_NAME_PRINT, - db_utf8, table_utf8); - } - - return(ret); + "END;\n", trx); } -/*********************************************************************//** -Executes -UPDATE mysql.innodb_table_stats SET -database_name = '...', table_name = '...' -WHERE database_name = '...' AND table_name = '...'; -Creates its own transaction and commits it. +/** Execute DELETE FROM mysql.innodb_index_stats +@param database_name database name +@param table_name table name +@param index_name name of the index +@param trx transaction (nullptr=start and commit a new one) @return DB_SUCCESS or error code */ -UNIV_INLINE -dberr_t -dict_stats_rename_table_in_table_stats( -/*===================================*/ - const char* old_dbname_utf8,/*!< in: database name, e.g. 'olddb' */ - const char* old_tablename_utf8,/*!< in: table name, e.g. 'oldtable' */ - const char* new_dbname_utf8,/*!< in: database name, e.g. 'newdb' */ - const char* new_tablename_utf8)/*!< in: table name, e.g. 'newtable' */ +dberr_t dict_stats_delete_from_index_stats(const char *database_name, + const char *table_name, + const char *index_name, trx_t *trx) { pars_info_t* pinfo; - dberr_t ret; ut_d(dict_sys.assert_locked()); pinfo = pars_info_create(); - pars_info_add_str_literal(pinfo, "old_dbname_utf8", old_dbname_utf8); - pars_info_add_str_literal(pinfo, "old_tablename_utf8", old_tablename_utf8); - pars_info_add_str_literal(pinfo, "new_dbname_utf8", new_dbname_utf8); - pars_info_add_str_literal(pinfo, "new_tablename_utf8", new_tablename_utf8); + pars_info_add_str_literal(pinfo, "database_name", database_name); + pars_info_add_str_literal(pinfo, "table_name", table_name); + pars_info_add_str_literal(pinfo, "index_name", index_name); - ret = dict_stats_exec_sql( + return dict_stats_exec_sql( pinfo, - "PROCEDURE RENAME_TABLE_IN_TABLE_STATS () IS\n" + "PROCEDURE DELETE_FROM_INDEX_STATS () IS\n" "BEGIN\n" - "UPDATE \"" TABLE_STATS_NAME "\" SET\n" - "database_name = :new_dbname_utf8,\n" - "table_name = :new_tablename_utf8\n" - "WHERE\n" - "database_name = :old_dbname_utf8 AND\n" - "table_name = :old_tablename_utf8;\n" - "END;\n", NULL); - - return(ret); + "DELETE FROM \"" INDEX_STATS_NAME "\" WHERE\n" + "database_name = :database_name AND\n" + "table_name = :table_name AND\n" + "index_name = :index_name;\n" + "END;\n", trx); } -/*********************************************************************//** -Executes -UPDATE mysql.innodb_index_stats SET -database_name = '...', table_name = '...' -WHERE database_name = '...' AND table_name = '...'; -Creates its own transaction and commits it. +/** Rename a table in InnoDB persistent stats storage. +@param old_name old table name +@param new_name new table name +@param trx transaction @return DB_SUCCESS or error code */ -UNIV_INLINE -dberr_t -dict_stats_rename_table_in_index_stats( -/*===================================*/ - const char* old_dbname_utf8,/*!< in: database name, e.g. 'olddb' */ - const char* old_tablename_utf8,/*!< in: table name, e.g. 'oldtable' */ - const char* new_dbname_utf8,/*!< in: database name, e.g. 'newdb' */ - const char* new_tablename_utf8)/*!< in: table name, e.g. 'newtable' */ +dberr_t dict_stats_rename_table(const char *old_name, const char *new_name, + trx_t *trx) { - pars_info_t* pinfo; - dberr_t ret; - - ut_d(dict_sys.assert_locked()); - - pinfo = pars_info_create(); - - pars_info_add_str_literal(pinfo, "old_dbname_utf8", old_dbname_utf8); - pars_info_add_str_literal(pinfo, "old_tablename_utf8", old_tablename_utf8); - pars_info_add_str_literal(pinfo, "new_dbname_utf8", new_dbname_utf8); - pars_info_add_str_literal(pinfo, "new_tablename_utf8", new_tablename_utf8); - - ret = dict_stats_exec_sql( - pinfo, - "PROCEDURE RENAME_TABLE_IN_INDEX_STATS () IS\n" - "BEGIN\n" - "UPDATE \"" INDEX_STATS_NAME "\" SET\n" - "database_name = :new_dbname_utf8,\n" - "table_name = :new_tablename_utf8\n" - "WHERE\n" - "database_name = :old_dbname_utf8 AND\n" - "table_name = :old_tablename_utf8;\n" - "END;\n", NULL); + /* skip the statistics tables themselves */ + if (!strcmp(old_name, TABLE_STATS_NAME) || + !strcmp(old_name, INDEX_STATS_NAME) || + !strcmp(new_name, TABLE_STATS_NAME) || + !strcmp(new_name, INDEX_STATS_NAME)) + return DB_SUCCESS; + + char old_db[MAX_DB_UTF8_LEN]; + char new_db[MAX_DB_UTF8_LEN]; + char old_table[MAX_TABLE_UTF8_LEN]; + char new_table[MAX_TABLE_UTF8_LEN]; + + dict_fs2utf8(old_name, old_db, sizeof old_db, old_table, sizeof old_table); + dict_fs2utf8(new_name, new_db, sizeof new_db, new_table, sizeof new_table); + + if (dict_table_t::is_temporary_name(old_name) || + dict_table_t::is_temporary_name(new_name)) + { + if (dberr_t e= dict_stats_delete_from_table_stats(old_db, old_table, trx)) + return e; + return dict_stats_delete_from_index_stats(old_db, old_table, trx); + } - return(ret); + pars_info_t *pinfo= pars_info_create(); + pars_info_add_str_literal(pinfo, "old_db", old_db); + pars_info_add_str_literal(pinfo, "old_table", old_table); + pars_info_add_str_literal(pinfo, "new_db", new_db); + pars_info_add_str_literal(pinfo, "new_table", new_table); + + static const char sql[]= + "PROCEDURE RENAME_TABLE_IN_STATS() IS\n" + "BEGIN\n" + "UPDATE \"" TABLE_STATS_NAME "\" SET\n" + "database_name=:new_db, table_name=:new_table\n" + "WHERE database_name=:old_db AND table_name=:old_table;\n" + "UPDATE \"" INDEX_STATS_NAME "\" SET\n" + "database_name=:new_db, table_name=:new_table\n" + "WHERE database_name=:old_db AND table_name=:old_table;\n" + "END;\n"; + + return dict_stats_exec_sql(pinfo, sql, trx); } -/*********************************************************************//** -Renames a table in InnoDB persistent stats storage. -This function creates its own transaction and commits it. +/** Rename an index in InnoDB persistent statistics. +@param db database name +@param table table name +@param old_name old table name +@param new_name new table name +@param trx transaction @return DB_SUCCESS or error code */ -dberr_t -dict_stats_rename_table( -/*====================*/ - const char* old_name, /*!< in: old name, e.g. 'db/table' */ - const char* new_name, /*!< in: new name, e.g. 'db/table' */ - char* errstr, /*!< out: error string if != DB_SUCCESS - is returned */ - size_t errstr_sz) /*!< in: errstr size */ +dberr_t dict_stats_rename_index(const char *db, const char *table, + const char *old_name, const char *new_name, + trx_t *trx) { - char old_db_utf8[MAX_DB_UTF8_LEN]; - char new_db_utf8[MAX_DB_UTF8_LEN]; - char old_table_utf8[MAX_TABLE_UTF8_LEN]; - char new_table_utf8[MAX_TABLE_UTF8_LEN]; - dberr_t ret; - - /* skip innodb_table_stats and innodb_index_stats themselves */ - if (strcmp(old_name, TABLE_STATS_NAME) == 0 - || strcmp(old_name, INDEX_STATS_NAME) == 0 - || strcmp(new_name, TABLE_STATS_NAME) == 0 - || strcmp(new_name, INDEX_STATS_NAME) == 0) { - - return(DB_SUCCESS); - } - - dict_fs2utf8(old_name, old_db_utf8, sizeof(old_db_utf8), - old_table_utf8, sizeof(old_table_utf8)); - - dict_fs2utf8(new_name, new_db_utf8, sizeof(new_db_utf8), - new_table_utf8, sizeof(new_table_utf8)); - - dict_sys_lock(); - - ulint n_attempts = 0; - do { - n_attempts++; - - ret = dict_stats_rename_table_in_table_stats( - old_db_utf8, old_table_utf8, - new_db_utf8, new_table_utf8); - - if (ret == DB_DUPLICATE_KEY) { - dict_stats_delete_from_table_stats( - new_db_utf8, new_table_utf8); - } - - if (ret == DB_STATS_DO_NOT_EXIST) { - ret = DB_SUCCESS; - } - - if (ret != DB_SUCCESS) { - dict_sys_unlock(); - std::this_thread::sleep_for( - std::chrono::milliseconds(200)); - dict_sys_lock(); - } - } while ((ret == DB_DEADLOCK - || ret == DB_DUPLICATE_KEY - || ret == DB_LOCK_WAIT_TIMEOUT) - && n_attempts < 5); - - if (ret != DB_SUCCESS) { - snprintf(errstr, errstr_sz, - "Unable to rename statistics from" - " %s.%s to %s.%s in %s: %s." - " They can be renamed later using" - - " UPDATE %s SET" - " database_name = '%s'," - " table_name = '%s'" - " WHERE" - " database_name = '%s' AND" - " table_name = '%s';", - - old_db_utf8, old_table_utf8, - new_db_utf8, new_table_utf8, - TABLE_STATS_NAME_PRINT, - ut_strerr(ret), - - TABLE_STATS_NAME_PRINT, - new_db_utf8, new_table_utf8, - old_db_utf8, old_table_utf8); - dict_sys_unlock(); - return(ret); - } - /* else */ - - n_attempts = 0; - do { - n_attempts++; - - ret = dict_stats_rename_table_in_index_stats( - old_db_utf8, old_table_utf8, - new_db_utf8, new_table_utf8); - - if (ret == DB_DUPLICATE_KEY) { - dict_stats_delete_from_index_stats( - new_db_utf8, new_table_utf8); - } - - if (ret == DB_STATS_DO_NOT_EXIST) { - ret = DB_SUCCESS; - } - - if (ret != DB_SUCCESS) { - dict_sys_unlock(); - std::this_thread::sleep_for( - std::chrono::milliseconds(200)); - dict_sys_lock(); - } - } while ((ret == DB_DEADLOCK - || ret == DB_DUPLICATE_KEY - || ret == DB_LOCK_WAIT_TIMEOUT) - && n_attempts < 5); - - dict_sys_unlock(); - - if (ret != DB_SUCCESS) { - snprintf(errstr, errstr_sz, - "Unable to rename statistics from" - " %s.%s to %s.%s in %s: %s." - " They can be renamed later using" - - " UPDATE %s SET" - " database_name = '%s'," - " table_name = '%s'" - " WHERE" - " database_name = '%s' AND" - " table_name = '%s';", - - old_db_utf8, old_table_utf8, - new_db_utf8, new_table_utf8, - INDEX_STATS_NAME_PRINT, - ut_strerr(ret), - - INDEX_STATS_NAME_PRINT, - new_db_utf8, new_table_utf8, - old_db_utf8, old_table_utf8); - } - - return(ret); + if (!dict_stats_persistent_storage_check(true)) + return DB_STATS_DO_NOT_EXIST; + pars_info_t *pinfo= pars_info_create(); + + pars_info_add_str_literal(pinfo, "db", db); + pars_info_add_str_literal(pinfo, "table", table); + pars_info_add_str_literal(pinfo, "old", old_name); + pars_info_add_str_literal(pinfo, "new", new_name); + + static const char sql[]= + "PROCEDURE RENAME_INDEX_IN_STATS() IS\n" + "BEGIN\n" + "UPDATE \"" INDEX_STATS_NAME "\" SET index_name=:new\n" + "WHERE database_name=:db AND table_name=:table AND index_name=:old;\n" + "END;\n"; + + return dict_stats_exec_sql(pinfo, sql, trx); } -/*********************************************************************//** -Renames an index in InnoDB persistent stats storage. -This function creates its own transaction and commits it. -@return DB_SUCCESS or error code. DB_STATS_DO_NOT_EXIST will be returned -if the persistent stats do not exist. */ -dberr_t -dict_stats_rename_index( -/*====================*/ - const dict_table_t* table, /*!< in: table whose index - is renamed */ - const char* old_index_name, /*!< in: old index name */ - const char* new_index_name) /*!< in: new index name */ +/** Delete all persistent statistics for a database. +@param db database name +@param trx transaction +@return DB_SUCCESS or error code */ +dberr_t dict_stats_delete(const char *db, trx_t *trx) { - dict_sys_lock(); - - if (!dict_stats_persistent_storage_check(true)) { - dict_sys_unlock(); - return(DB_STATS_DO_NOT_EXIST); - } - - char dbname_utf8[MAX_DB_UTF8_LEN]; - char tablename_utf8[MAX_TABLE_UTF8_LEN]; - - dict_fs2utf8(table->name.m_name, dbname_utf8, sizeof(dbname_utf8), - tablename_utf8, sizeof(tablename_utf8)); - - pars_info_t* pinfo; - - pinfo = pars_info_create(); - - pars_info_add_str_literal(pinfo, "dbname_utf8", dbname_utf8); - pars_info_add_str_literal(pinfo, "tablename_utf8", tablename_utf8); - pars_info_add_str_literal(pinfo, "new_index_name", new_index_name); - pars_info_add_str_literal(pinfo, "old_index_name", old_index_name); - - dberr_t ret; - - ret = dict_stats_exec_sql( - pinfo, - "PROCEDURE RENAME_INDEX_IN_INDEX_STATS () IS\n" - "BEGIN\n" - "UPDATE \"" INDEX_STATS_NAME "\" SET\n" - "index_name = :new_index_name\n" - "WHERE\n" - "database_name = :dbname_utf8 AND\n" - "table_name = :tablename_utf8 AND\n" - "index_name = :old_index_name;\n" - "END;\n", NULL); - - dict_sys_unlock(); - - return(ret); + static const char sql[] = + "PROCEDURE DROP_DATABASE_STATS () IS\n" + "BEGIN\n" + "DELETE FROM \"" TABLE_STATS_NAME "\" WHERE database_name=:db;\n" + "DELETE FROM \"" INDEX_STATS_NAME "\" WHERE database_name=:db;\n" + "END;\n"; + + pars_info_t *pinfo= pars_info_create(); + pars_info_add_str_literal(pinfo, "db", db); + return dict_stats_exec_sql(pinfo, sql, trx); } /* tests @{ */ diff --git a/storage/innobase/dict/drop.cc b/storage/innobase/dict/drop.cc new file mode 100644 index 00000000000..7f936aaf051 --- /dev/null +++ b/storage/innobase/dict/drop.cc @@ -0,0 +1,256 @@ +/***************************************************************************** + +Copyright (c) 2021, MariaDB Corporation. + +This program is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free Software +Foundation; version 2 of the License. + +This program is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along with +this program; if not, write to the Free Software Foundation, Inc., +51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA + +*****************************************************************************/ + +/** +@file dict/drop.cc +Data Dictionary Language operations that delete .ibd files */ + +/* We implement atomic data dictionary operations as follows. + +1. A data dictionary transaction is started. +2. We acquire exclusive lock on all the tables that are to be dropped +during the execution of the transaction. +3. We lock the data dictionary cache. +4. All metadata tables will be updated within the single DDL transaction, +including deleting or renaming InnoDB persistent statistics. +4b. If any lock wait would occur while we are holding the dict_sys latches, +we will instantly report a timeout error and roll back the transaction. +5. The transaction metadata is marked as committed. +6. If any files were deleted, we will durably write FILE_DELETE +to the redo log and start deleting the files. +6b. Also purge after a commit may perform file deletion. This is also the +recovery mechanism if the server was killed between step 5 and 6. +7. We unlock the data dictionary cache. +8. The file handles of the unlinked files will be closed. This will actually +reclaim the space in the file system (delete-on-close semantics). + +Notes: + +(a) Purge will be locked out by MDL. For internal tables related to +FULLTEXT INDEX, purge will not acquire MDL on the user table name, +and therefore, when we are dropping any FTS_ tables, we must suspend +and resume purge to prevent a race condition. + +(b) If a transaction needs to both drop and create a table by some +name, it must rename the table in between. This is used by +ha_innobase::truncate() and fts_drop_common_tables(). + +(c) No data is ever destroyed before the transaction is committed, +so we can trivially roll back the transaction at any time. +Lock waits during a DDL operation are no longer a fatal error +that would cause the InnoDB to hang or to intentionally crash. +(Only ALTER TABLE...DISCARD TABLESPACE may discard data before commit.) + +(d) The only changes to the data dictionary cache that are performed +before transaction commit and must be rolled back explicitly are as follows: +(d1) fts_optimize_add_table() to undo fts_optimize_remove_table() +(d2) stats_bg_flag= BG_STAT_NONE to undo dict_stats_stop_bg() +*/ + +#include "trx0purge.h" +#include "dict0dict.h" +#include "dict0stats.h" +#include "dict0stats_bg.h" + +#include "dict0defrag_bg.h" +#include "btr0defragment.h" + +#include "que0que.h" +#include "pars0pars.h" + +/** Try to drop the foreign key constraints for a persistent table. +@param name name of persistent table +@return error code */ +dberr_t trx_t::drop_table_foreign(const table_name_t &name) +{ + ut_d(dict_sys.assert_locked()); + ut_ad(state == TRX_STATE_ACTIVE); + ut_ad(dict_operation); + ut_ad(dict_operation_lock_mode == RW_X_LATCH); + + if (!dict_sys.sys_foreign || !dict_sys.sys_foreign_cols) + return DB_SUCCESS; + + pars_info_t *info= pars_info_create(); + pars_info_add_str_literal(info, "name", name.m_name); + return que_eval_sql(info, + "PROCEDURE DROP_FOREIGN() IS\n" + "fid CHAR;\n" + + "DECLARE CURSOR fk IS\n" + "SELECT ID FROM SYS_FOREIGN\n" + "WHERE FOR_NAME=:name\n" + "AND TO_BINARY(FOR_NAME)=TO_BINARY(:name)\n" + "FOR UPDATE;\n" + + "BEGIN\n" + "OPEN fk;\n" + "WHILE 1=1 LOOP\n" + " FETCH fk INTO fid;\n" + " IF (SQL % NOTFOUND)THEN RETURN;END IF;\n" + " DELETE FROM SYS_FOREIGN_COLS" + " WHERE ID=fid;\n" + " DELETE FROM SYS_FOREIGN WHERE ID=fid;\n" + "END LOOP;\n" + "CLOSE fk;\n" + "END;\n", FALSE, this); +} + +/** Try to drop the statistics for a persistent table. +@param name name of persistent table +@return error code */ +dberr_t trx_t::drop_table_statistics(const table_name_t &name) +{ + ut_d(dict_sys.assert_locked()); + ut_ad(dict_operation_lock_mode == RW_X_LATCH); + + if (strstr(name.m_name, "/" TEMP_FILE_PREFIX_INNODB) || + !strcmp(name.m_name, TABLE_STATS_NAME) || + !strcmp(name.m_name, INDEX_STATS_NAME)) + return DB_SUCCESS; + + char db[MAX_DB_UTF8_LEN], table[MAX_TABLE_UTF8_LEN]; + dict_fs2utf8(name.m_name, db, sizeof db, table, sizeof table); + + dberr_t err= dict_stats_delete_from_table_stats(db, table, this); + if (err == DB_SUCCESS || err == DB_STATS_DO_NOT_EXIST) + { + err= dict_stats_delete_from_index_stats(db, table, this); + if (err == DB_STATS_DO_NOT_EXIST) + err= DB_SUCCESS; + } + return err; +} + +/** Try to drop a persistent table. +@param table persistent table +@param fk whether to drop FOREIGN KEY metadata +@return error code */ +dberr_t trx_t::drop_table(const dict_table_t &table) +{ + ut_d(dict_sys.assert_locked()); + ut_ad(state == TRX_STATE_ACTIVE); + ut_ad(dict_operation); + ut_ad(dict_operation_lock_mode == RW_X_LATCH); + ut_ad(!table.is_temporary()); + ut_ad(!(table.stats_bg_flag & BG_STAT_IN_PROGRESS)); + /* The table must be exclusively locked by this transaction. */ + ut_ad(table.get_ref_count() <= 1); + ut_ad(!table.n_waiting_or_granted_auto_inc_locks); + ut_ad(table.n_lock_x_or_s == 1); + ut_ad(UT_LIST_GET_LEN(table.locks) >= 1); +#ifdef UNIV_DEBUG + bool found_x; + for (lock_t *lock= UT_LIST_GET_FIRST(table.locks); lock; + lock= UT_LIST_GET_NEXT(un_member.tab_lock.locks, lock)) + { + ut_ad(lock->trx == this); + if (lock->type_mode == (LOCK_X | LOCK_TABLE)) + found_x= true; + else + ut_ad(lock->type_mode == (LOCK_IX | LOCK_TABLE)); + } + ut_ad(found_x); +#endif + + if (dict_sys.sys_virtual) + { + pars_info_t *info= pars_info_create(); + pars_info_add_ull_literal(info, "id", table.id); + if (dberr_t err= que_eval_sql(info, + "PROCEDURE DROP_VIRTUAL() IS\n" + "BEGIN\n" + "DELETE FROM SYS_VIRTUAL" + " WHERE TABLE_ID=:id;\n" + "END;\n", FALSE, this)) + return err; + } + + /* Once DELETE FROM SYS_INDEXES is committed, purge may invoke + dict_drop_index_tree(). */ + + if (!(table.flags2 & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS))); + else if (dberr_t err= fts_drop_tables(this, table)) + { + ib::error() << "Unable to remove FTS tables for " + << table.name << ": " << err; + return err; + } + + mod_tables.emplace(const_cast<dict_table_t*>(&table), undo_no). + first->second.set_dropped(); + + pars_info_t *info= pars_info_create(); + pars_info_add_ull_literal(info, "id", table.id); + return que_eval_sql(info, + "PROCEDURE DROP_TABLE() IS\n" + "iid CHAR;\n" + + "DECLARE CURSOR idx IS\n" + "SELECT ID FROM SYS_INDEXES\n" + "WHERE TABLE_ID=:id FOR UPDATE;\n" + + "BEGIN\n" + "OPEN idx;\n" + "WHILE 1 = 1 LOOP\n" + " FETCH idx INTO iid;\n" + " IF (SQL % NOTFOUND) THEN EXIT; END IF;\n" + " DELETE FROM SYS_FIELDS WHERE INDEX_ID=iid;\n" + " DELETE FROM SYS_INDEXES WHERE CURRENT OF idx;\n" + "END LOOP;\n" + "CLOSE idx;\n" + + "DELETE FROM SYS_COLUMNS WHERE TABLE_ID=:id;\n" + "DELETE FROM SYS_TABLES WHERE ID=:id;\n" + + "END;\n", FALSE, this); +} + +/** Commit the transaction, possibly after drop_table(). +@param deleted handles of data files that were deleted */ +void trx_t::commit(std::vector<pfs_os_file_t> &deleted) +{ + ut_ad(dict_operation); + commit_persist(); + if (dict_operation) + { + dict_sys.assert_locked(); + for (const auto &p : mod_tables) + { + if (p.second.is_dropped()) + { + dict_table_t *table= p.first; + dict_stats_recalc_pool_del(table); + dict_stats_defrag_pool_del(table, nullptr); + if (btr_defragment_active) + btr_defragment_remove_table(table); + const fil_space_t *space= table->space; + ut_ad(!strstr(table->name.m_name, "/FTS_") || + purge_sys.must_wait_FTS()); + dict_sys.remove(table); + if (const auto id= space ? space->id : 0) + { + pfs_os_file_t d= fil_delete_tablespace(id); + if (d != OS_FILE_CLOSED) + deleted.emplace_back(d); + } + } + } + } + commit_cleanup(); +} diff --git a/storage/innobase/fil/fil0crypt.cc b/storage/innobase/fil/fil0crypt.cc index f7f39337f90..1d2447a64ea 100644 --- a/storage/innobase/fil/fil0crypt.cc +++ b/storage/innobase/fil/fil0crypt.cc @@ -2312,9 +2312,6 @@ void fil_space_crypt_close_tablespace(const fil_space_t *space) while (crypt_data->rotate_state.active_threads || crypt_data->rotate_state.flushing) { mysql_mutex_unlock(&crypt_data->mutex); - /* release dict mutex so that scrub threads can release their - * table references */ - dict_sys.mutex_unlock(); /* wakeup throttle (all) sleepers */ mysql_mutex_lock(&fil_crypt_threads_mutex); @@ -2323,8 +2320,6 @@ void fil_space_crypt_close_tablespace(const fil_space_t *space) mysql_mutex_unlock(&fil_crypt_threads_mutex); std::this_thread::sleep_for(std::chrono::milliseconds(20)); - dict_sys.mutex_lock(); - mysql_mutex_lock(&crypt_data->mutex); time_t now = time(0); @@ -2339,6 +2334,8 @@ void fil_space_crypt_close_tablespace(const fil_space_t *space) << crypt_data->rotate_state.flushing << "."; last = now; } + + mysql_mutex_lock(&crypt_data->mutex); } mysql_mutex_unlock(&crypt_data->mutex); diff --git a/storage/innobase/fil/fil0fil.cc b/storage/innobase/fil/fil0fil.cc index 3072b5497e8..5f6b9e5578d 100644 --- a/storage/innobase/fil/fil0fil.cc +++ b/storage/innobase/fil/fil0fil.cc @@ -1566,39 +1566,61 @@ fil_name_write( fil_space_t *fil_space_t::check_pending_operations(ulint id) { ut_a(!is_system_tablespace(id)); + bool being_deleted= false; mysql_mutex_lock(&fil_system.mutex); fil_space_t *space= fil_space_get_by_id(id); if (!space); else if (space->pending() & STOPPING) - space= nullptr; + being_deleted= true; else { - space->reacquire(); if (space->crypt_data) { + space->reacquire(); mysql_mutex_unlock(&fil_system.mutex); fil_space_crypt_close_tablespace(space); mysql_mutex_lock(&fil_system.mutex); + space->release(); } - space->set_stopping(true); - space->release(); + being_deleted= space->set_stopping(); } mysql_mutex_unlock(&fil_system.mutex); if (!space) return nullptr; + if (being_deleted) + { + /* A thread executing DDL and another thread executing purge may + be executing fil_delete_tablespace() concurrently for the same + tablespace. Wait for the other thread to complete the operation. */ + for (ulint count= 0;; count++) + { + mysql_mutex_lock(&fil_system.mutex); + space= fil_space_get_by_id(id); + ut_ad(!space || space->is_stopping()); + mysql_mutex_unlock(&fil_system.mutex); + if (!space) + return nullptr; + /* Issue a warning every 10.24 seconds, starting after 2.56 seconds */ + if ((count & 511) == 128) + sql_print_warning("InnoDB: Waiting for tablespace " ULINTPF + " to be deleted", id); + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + } + } + for (ulint count= 0;; count++) { - auto pending= space->referenced(); + const unsigned pending= space->referenced(); if (!pending) return space; - /* Give a warning every 10 second, starting after 1 second */ - if ((count % 500) == 50) - ib::warn() << "Trying to delete tablespace '" - << space->chain.start->name << "' but there are " - << pending << " pending operations on it."; + /* Issue a warning every 10.24 seconds, starting after 2.56 seconds */ + if ((count & 511) == 128) + sql_print_warning("InnoDB: Trying to delete tablespace '%s' " + "but there are %u pending operations", + space->chain.start->name, id); std::this_thread::sleep_for(std::chrono::milliseconds(20)); } } @@ -1645,117 +1667,60 @@ void fil_close_tablespace(ulint id) } /** Delete a tablespace and associated .ibd file. -@param[in] id tablespace identifier -@param[in] if_exists whether to ignore missing tablespace -@param[out] detached deatched file handle (if closing is not wanted) -@return DB_SUCCESS or error */ -dberr_t fil_delete_tablespace(ulint id, bool if_exists, - pfs_os_file_t *detached) +@param id tablespace identifier +@return detached file handle (to be closed by the caller) +@return OS_FILE_CLOSED if no file existed */ +pfs_os_file_t fil_delete_tablespace(ulint id) { - ut_ad(!is_system_tablespace(id)); - ut_ad(!detached || *detached == OS_FILE_CLOSED); - - dberr_t err; - fil_space_t *space = fil_space_t::check_pending_operations(id); - - if (!space) { - err = DB_TABLESPACE_NOT_FOUND; - if (!if_exists) { - ib::error() << "Cannot delete tablespace " << id - << " because it is not found" - " in the tablespace memory cache."; - } -func_exit: - ibuf_delete_for_discarded_space(id); - return err; - } - - /* IMPORTANT: Because we have set space::stop_new_ops there - can't be any new reads or flushes. We are here - because node::n_pending was zero above. However, it is still - possible to have pending read and write requests: - - A read request can happen because the reader thread has - gone through the ::stop_new_ops check in buf_page_init_for_read() - before the flag was set and has not yet incremented ::n_pending - when we checked it above. - - A write request can be issued any time because we don't check - fil_space_t::is_stopping() when queueing a block for write. - - We deal with pending write requests in the following function - where we'd minimally evict all dirty pages belonging to this - space from the flush_list. Note that if a block is IO-fixed - we'll wait for IO to complete. - - To deal with potential read requests, we will check the - is_stopping() in fil_space_t::io(). */ - - err = DB_SUCCESS; - buf_flush_remove_pages(id); - - /* If it is a delete then also delete any generated files, otherwise - when we drop the database the remove directory will fail. */ - { - /* Before deleting the file, write a log record about - it, so that InnoDB crash recovery will expect the file - to be gone. */ - mtr_t mtr; - - mtr.start(); - mtr.log_file_op(FILE_DELETE, id, space->chain.start->name); - mtr.commit(); - /* Even if we got killed shortly after deleting the - tablespace file, the record must have already been - written to the redo log. */ - log_write_up_to(mtr.commit_lsn(), true); - - if (char* cfg_name = fil_make_filepath( - space->chain.start->name, - fil_space_t::name_type{}, CFG, false)) { - os_file_delete_if_exists(innodb_data_file_key, - cfg_name, nullptr); - ut_free(cfg_name); - } - } - - /* Delete the link file pointing to the ibd file we are deleting. */ - if (FSP_FLAGS_HAS_DATA_DIR(space->flags)) { - RemoteDatafile::delete_link_file(space->name()); - } - - mysql_mutex_lock(&fil_system.mutex); - - /* Double check the sanity of pending ops after reacquiring - the fil_system::mutex. */ - ut_a(space == fil_space_get_by_id(id)); - ut_a(!space->referenced()); - ut_a(UT_LIST_GET_LEN(space->chain) == 1); - pfs_os_file_t handle = fil_system.detach(space, detached != nullptr); - if (detached) { - *detached = handle; - } - mysql_mutex_unlock(&fil_system.mutex); + ut_ad(!is_system_tablespace(id)); + pfs_os_file_t handle= OS_FILE_CLOSED; + if (fil_space_t *space= fil_space_t::check_pending_operations(id)) + { + /* Before deleting the file(s), persistently write a log record. */ + mtr_t mtr; + mtr.start(); + mtr.log_file_op(FILE_DELETE, id, space->chain.start->name); + mtr.commit(); + log_write_up_to(mtr.commit_lsn(), true); + + /* Remove any additional files. */ + if (char *cfg_name= fil_make_filepath(space->chain.start->name, + fil_space_t::name_type{}, CFG, + false)) + { + os_file_delete_if_exists(innodb_data_file_key, cfg_name, nullptr); + ut_free(cfg_name); + } + if (FSP_FLAGS_HAS_DATA_DIR(space->flags)) + RemoteDatafile::delete_link_file(space->name()); - mysql_mutex_lock(&log_sys.mutex); + /* Remove the directory entry. The file will actually be deleted + when our caller closes the handle. */ + os_file_delete(innodb_data_file_key, space->chain.start->name); - if (space->max_lsn != 0) { - ut_d(space->max_lsn = 0); - fil_system.named_spaces.remove(*space); - } + mysql_mutex_lock(&fil_system.mutex); + /* Sanity checks after reacquiring fil_system.mutex */ + ut_ad(space == fil_space_get_by_id(id)); + ut_ad(!space->referenced()); + ut_ad(space->is_stopping()); + ut_ad(UT_LIST_GET_LEN(space->chain) == 1); + /* Detach the file handle. */ + handle= fil_system.detach(space, true); + mysql_mutex_unlock(&fil_system.mutex); - mysql_mutex_unlock(&log_sys.mutex); + mysql_mutex_lock(&log_sys.mutex); + if (space->max_lsn) + { + ut_d(space->max_lsn = 0); + fil_system.named_spaces.remove(*space); + } + mysql_mutex_unlock(&log_sys.mutex); - if (!os_file_delete(innodb_data_file_key, space->chain.start->name) - && !os_file_delete_if_exists(innodb_data_file_key, - space->chain.start->name, NULL)) { - /* Note: This is because we have removed the - tablespace instance from the cache. */ - err = DB_IO_ERROR; - } + fil_space_free_low(space); + } - fil_space_free_low(space); - goto func_exit; + ibuf_delete_for_discarded_space(id); + return handle; } /*******************************************************************//** diff --git a/storage/innobase/fsp/fsp0file.cc b/storage/innobase/fsp/fsp0file.cc index 0c159a8ccb4..aebe6c9ce23 100644 --- a/storage/innobase/fsp/fsp0file.cc +++ b/storage/innobase/fsp/fsp0file.cc @@ -267,20 +267,20 @@ Datafile::read_first_page(bool read_only_mode) IORequestReadPartial, m_handle, m_first_page, 0, page_size, &n_read); - if (err == DB_IO_ERROR && n_read >= UNIV_PAGE_SIZE_MIN) { - - page_size >>= 1; - - } else if (err == DB_SUCCESS) { - + if (err == DB_SUCCESS) { ut_a(n_read == page_size); - break; + } + if (err == DB_IO_ERROR && n_read == 0) { + break; + } + if (err == DB_IO_ERROR && n_read >= UNIV_PAGE_SIZE_MIN) { + page_size >>= 1; } else if (srv_operation == SRV_OPERATION_BACKUP) { break; } else { - ib::error() << "Cannot read first page of '" + ib::info() << "Cannot read first page of '" << m_filepath << "': " << err; break; } diff --git a/storage/innobase/fts/fts0fts.cc b/storage/innobase/fts/fts0fts.cc index 050c8eb7d23..d1b71f3e72e 100644 --- a/storage/innobase/fts/fts0fts.cc +++ b/storage/innobase/fts/fts0fts.cc @@ -23,6 +23,9 @@ Full Text Search interface ***********************************************************************/ #include "trx0roll.h" +#ifdef UNIV_DEBUG +# include "trx0purge.h" +#endif #include "row0mysql.h" #include "row0upd.h" #include "dict0types.h" @@ -871,7 +874,7 @@ fts_drop_index( mysql_mutex_unlock(&cache->init_lock); } - err = fts_drop_index_tables(trx, index); + err = fts_drop_index_tables(trx, *index); ib_vector_remove(indexes, (const void*) index); @@ -1400,46 +1403,43 @@ fts_cache_add_doc( } } -/****************************************************************//** -Drops a table. If the table can't be found we return a SUCCESS code. -@return DB_SUCCESS or error code */ -static MY_ATTRIBUTE((nonnull, warn_unused_result)) -dberr_t -fts_drop_table( -/*===========*/ - trx_t* trx, /*!< in: transaction */ - const char* table_name) /*!< in: table to drop */ +/** Drop a table. +@param trx transaction +@param table_name FTS_ table name +@param rename whether to rename before dropping +@return error code +@retval DB_SUCCESS if the table was dropped +@retval DB_FAIL if the table did not exist */ +static dberr_t fts_drop_table(trx_t *trx, const char *table_name, bool rename) { - dict_table_t* table; - dberr_t error = DB_SUCCESS; - - /* Check that the table exists in our data dictionary. - Similar to regular drop table case, we will open table with - DICT_ERR_IGNORE_INDEX_ROOT and DICT_ERR_IGNORE_CORRUPT option */ - table = dict_table_open_on_name( - table_name, TRUE, FALSE, - static_cast<dict_err_ignore_t>( - DICT_ERR_IGNORE_INDEX_ROOT | DICT_ERR_IGNORE_CORRUPT)); - - if (table != 0) { - - dict_table_close(table, TRUE, FALSE); - - /* Pass nonatomic=false (don't allow data dict unlock), - because the transaction may hold locks on SYS_* tables from - previous calls to fts_drop_table(). */ - error = row_drop_table_for_mysql(table_name, trx, - SQLCOM_DROP_DB, false, false); + if (dict_table_t *table= dict_table_open_on_name(table_name, TRUE, FALSE, + DICT_ERR_IGNORE_DROP)) + { + table->release(); + if (rename) + { + mem_heap_t *heap= mem_heap_create(FN_REFLEN); + char *tmp= dict_mem_create_temporary_tablename(heap, table->name.m_name, + table->id); + dberr_t err= row_rename_table_for_mysql(table->name.m_name, tmp, trx, + false); + mem_heap_free(heap); + if (err != DB_SUCCESS) + { + ib::error() << "Unable to rename table " << table_name << ": " << err; + return err; + } + } + if (dberr_t err= trx->drop_table(*table)) + { + ib::error() << "Unable to drop table " << table->name << ": " << err; + return err; + } - if (UNIV_UNLIKELY(error != DB_SUCCESS)) { - ib::error() << "Unable to drop FTS index aux table " - << table_name << ": " << error; - } - } else { - error = DB_FAIL; - } + return DB_SUCCESS; + } - return(error); + return DB_FAIL; } /****************************************************************//** @@ -1473,7 +1473,7 @@ fts_rename_one_aux_table( fts_table_new_name[table_new_name_len] = 0; return row_rename_table_for_mysql( - fts_table_old_name, fts_table_new_name, trx, false, false); + fts_table_old_name, fts_table_new_name, trx, false); } /****************************************************************//** @@ -1539,25 +1539,115 @@ fts_rename_aux_tables( return(DB_SUCCESS); } +/** Lock an internal FTS_ table, before fts_drop_table() */ +static dberr_t fts_lock_table(trx_t *trx, const char *table_name) +{ + ut_ad(purge_sys.must_wait_FTS()); + + if (dict_table_t *table= dict_table_open_on_name(table_name, false, false, + DICT_ERR_IGNORE_DROP)) + { + dberr_t err= lock_table_for_trx(table, trx, LOCK_X); + /* Wait for purge threads to stop using the table. */ + for (uint n= 15; table->get_ref_count() > 1; ) + { + if (!--n) + { + err= DB_LOCK_WAIT_TIMEOUT; + break; + } + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + } + table->release(); + return err; + } + return DB_SUCCESS; +} + +/** Lock the internal FTS_ tables for an index, before fts_drop_index_tables(). +@param trx transaction +@param index fulltext index */ +dberr_t fts_lock_index_tables(trx_t *trx, const dict_index_t &index) +{ + ut_ad(index.type & DICT_FTS); + fts_table_t fts_table; + char table_name[MAX_FULL_NAME_LEN]; + FTS_INIT_INDEX_TABLE(&fts_table, nullptr, FTS_INDEX_TABLE, (&index)); + for (const fts_index_selector_t *s= fts_index_selector; s->suffix; s++) + { + fts_table.suffix= s->suffix; + fts_get_table_name(&fts_table, table_name, false); + if (dberr_t err= fts_lock_table(trx, table_name)) + return err; + } + return DB_SUCCESS; +} + +/** Lock the internal common FTS_ tables, before fts_drop_common_tables(). +@param trx transaction +@param table table containing FULLTEXT INDEX +@return DB_SUCCESS or error code */ +dberr_t fts_lock_common_tables(trx_t *trx, const dict_table_t &table) +{ + fts_table_t fts_table; + char table_name[MAX_FULL_NAME_LEN]; + + FTS_INIT_FTS_TABLE(&fts_table, nullptr, FTS_COMMON_TABLE, (&table)); + + for (const char **suffix= fts_common_tables; *suffix; suffix++) + { + fts_table.suffix= *suffix; + fts_get_table_name(&fts_table, table_name, false); + if (dberr_t err= fts_lock_table(trx, table_name)) + return err; + } + return DB_SUCCESS; +} + +/** Lock the internal FTS_ tables for table, before fts_drop_tables(). +@param trx transaction +@param table table containing FULLTEXT INDEX +@return DB_SUCCESS or error code */ +dberr_t fts_lock_tables(trx_t *trx, const dict_table_t &table) +{ + if (dberr_t err= fts_lock_common_tables(trx, table)) + return err; + + if (!table.fts) + return DB_SUCCESS; + + auto indexes= table.fts->indexes; + if (!indexes) + return DB_SUCCESS; + + for (ulint i= 0; i < ib_vector_size(indexes); ++i) + if (dberr_t err= + fts_lock_index_tables(trx, *static_cast<const dict_index_t*> + (ib_vector_getp(indexes, i)))) + return err; + return DB_SUCCESS; +} + /** Drops the common ancillary tables needed for supporting an FTS index on the given table. row_mysql_lock_data_dictionary must have been called before this. -@param[in] trx transaction to drop fts common table -@param[in] fts_table table with an FTS index +@param trx transaction to drop fts common table +@param fts_table table with an FTS index +@param rename whether to rename before dropping @return DB_SUCCESS or error code */ -static dberr_t fts_drop_common_tables(trx_t *trx, fts_table_t *fts_table) +static dberr_t fts_drop_common_tables(trx_t *trx, fts_table_t *fts_table, + bool rename) { - ulint i; dberr_t error = DB_SUCCESS; - for (i = 0; fts_common_tables[i] != NULL; ++i) { + for (ulint i = 0; fts_common_tables[i] != NULL; ++i) { dberr_t err; char table_name[MAX_FULL_NAME_LEN]; fts_table->suffix = fts_common_tables[i]; fts_get_table_name(fts_table, table_name, true); - err = fts_drop_table(trx, table_name); + err = fts_drop_table(trx, table_name, rename); /* We only return the status of the last error. */ if (err != DB_SUCCESS && err != DB_FAIL) { @@ -1571,17 +1661,13 @@ static dberr_t fts_drop_common_tables(trx_t *trx, fts_table_t *fts_table) /****************************************************************//** Drops FTS auxiliary tables for an FTS index @return DB_SUCCESS or error code */ -dberr_t -fts_drop_index_tables( - trx_t* trx, /*!< in: transaction */ - dict_index_t* index) /*!< in: fts instance */ - +dberr_t fts_drop_index_tables(trx_t *trx, const dict_index_t &index) { ulint i; fts_table_t fts_table; dberr_t error = DB_SUCCESS; - FTS_INIT_INDEX_TABLE(&fts_table, NULL, FTS_INDEX_TABLE, index); + FTS_INIT_INDEX_TABLE(&fts_table, nullptr, FTS_INDEX_TABLE, (&index)); for (i = 0; i < FTS_NUM_AUX_INDEX; ++i) { dberr_t err; @@ -1590,7 +1676,7 @@ fts_drop_index_tables( fts_table.suffix = fts_get_suffix(i); fts_get_table_name(&fts_table, table_name, true); - err = fts_drop_table(trx, table_name); + err = fts_drop_table(trx, table_name, false); /* We only return the status of the last error. */ if (err != DB_SUCCESS && err != DB_FAIL) { @@ -1611,52 +1697,36 @@ dberr_t fts_drop_all_index_tables( /*======================*/ trx_t* trx, /*!< in: transaction */ - fts_t* fts) /*!< in: fts instance */ + const fts_t* fts) /*!< in: fts instance */ { - dberr_t error = DB_SUCCESS; - - for (ulint i = 0; - fts->indexes != 0 && i < ib_vector_size(fts->indexes); - ++i) { - - dberr_t err; - dict_index_t* index; - - index = static_cast<dict_index_t*>( - ib_vector_getp(fts->indexes, i)); - - err = fts_drop_index_tables(trx, index); - - if (err != DB_SUCCESS) { - error = err; - } - } - - return(error); + dberr_t error= DB_SUCCESS; + auto indexes= fts->indexes; + if (!indexes) + return DB_SUCCESS; + + for (ulint i= 0; i < ib_vector_size(indexes); ++i) + if (dberr_t err= fts_drop_index_tables(trx, + *static_cast<const dict_index_t*> + (ib_vector_getp(indexes, i)))) + error= err; + return error; } -/*********************************************************************//** -Drops the ancillary tables needed for supporting an FTS index on a -given table. row_mysql_lock_data_dictionary must have been called before -this. +/** Drop the internal FTS_ tables for table. +@param trx transaction +@param table table containing FULLTEXT INDEX @return DB_SUCCESS or error code */ -dberr_t -fts_drop_tables( -/*============*/ - trx_t* trx, /*!< in: transaction */ - dict_table_t* table) /*!< in: table has the FTS index */ +dberr_t fts_drop_tables(trx_t *trx, const dict_table_t &table) { dberr_t error; fts_table_t fts_table; - FTS_INIT_FTS_TABLE(&fts_table, NULL, FTS_COMMON_TABLE, table); - - /* TODO: This is not atomic and can cause problems during recovery. */ + FTS_INIT_FTS_TABLE(&fts_table, NULL, FTS_COMMON_TABLE, (&table)); - error = fts_drop_common_tables(trx, &fts_table); + error = fts_drop_common_tables(trx, &fts_table, false); - if (error == DB_SUCCESS && table->fts) { - error = fts_drop_all_index_tables(trx, table->fts); + if (error == DB_SUCCESS && table.fts) { + error = fts_drop_all_index_tables(trx, table.fts); } return(error); @@ -1797,7 +1867,7 @@ fts_create_common_tables( FTS_INIT_FTS_TABLE(&fts_table, NULL, FTS_COMMON_TABLE, table); - error = fts_drop_common_tables(trx, &fts_table); + error = fts_drop_common_tables(trx, &fts_table, true); if (error != DB_SUCCESS) { @@ -4172,8 +4242,7 @@ begin_sync: index_cache = static_cast<fts_index_cache_t*>( ib_vector_get(cache->indexes, i)); - if (index_cache->index->to_be_dropped - || index_cache->index->table->to_be_dropped) { + if (index_cache->index->to_be_dropped) { continue; } @@ -4201,7 +4270,6 @@ begin_sync: ib_vector_get(cache->indexes, i)); if (index_cache->index->to_be_dropped - || index_cache->index->table->to_be_dropped || fts_sync_index_check(index_cache)) { continue; } diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index 78abc11d1dc..7199fcfb547 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -94,7 +94,6 @@ this program; if not, write to the Free Software Foundation, Inc., #include "mtr0mtr.h" #include "os0file.h" #include "page0zip.h" -#include "pars0pars.h" #include "rem0types.h" #include "row0import.h" #include "row0ins.h" @@ -186,7 +185,7 @@ static char* innobase_reset_all_monitor_counter; stopword table to be used */ static char* innobase_server_stopword_table; -static my_bool innobase_rollback_on_timeout; +my_bool innobase_rollback_on_timeout; static my_bool innobase_create_status_file; my_bool innobase_stats_on_metadata; static my_bool innodb_optimize_fulltext_only; @@ -1312,7 +1311,8 @@ static ibool innodb_drop_database_fk(void *node, void *report) return true; } -/** Remove all tables in the named database inside InnoDB. +/** After DROP DATABASE executed ha_innobase::delete_table() on all +tables that it was aware of, drop any leftover tables inside InnoDB. @param path database path */ static void innodb_drop_database(handlerton*, char *path) { @@ -1375,15 +1375,12 @@ retry: continue; const auto n_handles= table->get_ref_count(); const bool locks= !n_handles && lock_table_has_locks(table); - const auto n_fk_checks= table->n_foreign_key_checks_running; - if (n_fk_checks || n_handles || locks) + if (n_handles || locks) { err= DB_ERROR; ib::error errmsg; errmsg << "DROP DATABASE: cannot DROP TABLE " << table->name; - if (n_fk_checks) - errmsg << " due to " << n_fk_checks << " FOREIGN KEY checks"; - else if (n_handles) + if (n_handles) errmsg << " due to " << n_handles << " open handles"; else errmsg << " due to locks"; @@ -1393,6 +1390,20 @@ retry: } } + trx_start_for_ddl(trx); + uint errors= 0; + char db[NAME_LEN + 1]; + strconvert(&my_charset_filename, namebuf, len, system_charset_info, db, + sizeof db, &errors); + if (errors); + else if (dict_stats_delete(db, trx)) + { + /* Ignore this error. Leaving garbage statistics behind is a + lesser evil. Carry on to try to remove any garbage tables. */ + trx->rollback(); + trx_start_for_ddl(trx); + } + static const char drop_database[] = "PROCEDURE DROP_DATABASE_PROC () IS\n" "fk CHAR;\n" @@ -1453,7 +1464,6 @@ retry: "END;\n"; innodb_drop_database_fk_report report{{namebuf, len + 1}, false}; - trx_start_for_ddl(trx); if (err == DB_SUCCESS) { @@ -1539,8 +1549,7 @@ retry: ut_ad("corrupted SYS_TABLES.SPACE" == 0); else if (uint32_t space_id= mach_read_from_4(s)) { - pfs_os_file_t detached= OS_FILE_CLOSED; - fil_delete_tablespace(space_id, true, &detached); + pfs_os_file_t detached= fil_delete_tablespace(space_id); if (detached != OS_FILE_CLOSED) to_close.emplace_back(detached); } @@ -1900,6 +1909,9 @@ static int innobase_wsrep_set_checkpoint(handlerton* hton, const XID* xid); static int innobase_wsrep_get_checkpoint(handlerton* hton, XID* xid); #endif /* WITH_WSREP */ +#define normalize_table_name(a,b) \ + normalize_table_name_c_low(a,b,IF_WIN(true,false)) + ulonglong ha_innobase::table_version() const { /* This is either "garbage" or something that was assigned @@ -2030,7 +2042,7 @@ convert_error_code_to_mysql( if (thd) { thd_mark_transaction_to_rollback( - thd, (bool) row_rollback_on_timeout); + thd, innobase_rollback_on_timeout); } return(HA_ERR_LOCK_WAIT_TIMEOUT); @@ -2066,9 +2078,6 @@ convert_error_code_to_mysql( "InnoDB"); return(HA_ERR_INTERNAL_ERROR); - case DB_TABLE_IN_FK_CHECK: - return(HA_ERR_TABLE_IN_FK_CHECK); - case DB_TABLE_NOT_FOUND: return(HA_ERR_NO_SUCH_TABLE); @@ -3809,8 +3818,6 @@ static int innodb_init_params() srv_buf_pool_size = ulint(innobase_buffer_pool_size); - row_rollback_on_timeout = (ibool) innobase_rollback_on_timeout; - if (innobase_open_files < 10) { innobase_open_files = 300; if (srv_file_per_table && tc_size > 300 && tc_size < open_files_limit) { @@ -3943,6 +3950,7 @@ static int innodb_init(void* p) HTON_NATIVE_SYS_VERSIONING | HTON_WSREP_REPLICATION | HTON_REQUIRES_CLOSE_AFTER_TRUNCATE | + HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE | HTON_REQUIRES_NOTIFY_TABLEDEF_CHANGED_AFTER_COMMIT; #ifdef WITH_WSREP @@ -4129,9 +4137,8 @@ innobase_commit_low( #ifdef WITH_WSREP const char* tmp = 0; const bool is_wsrep = trx->is_wsrep(); - THD* thd = trx->mysql_thd; if (is_wsrep) { - tmp = thd_proc_info(thd, "innobase_commit_low()"); + tmp = thd_proc_info(trx->mysql_thd, "innobase_commit_low()"); } #endif /* WITH_WSREP */ if (trx_is_started(trx)) { @@ -4145,7 +4152,7 @@ innobase_commit_low( #ifdef WITH_WSREP if (is_wsrep) { - thd_proc_info(thd, tmp); + thd_proc_info(trx->mysql_thd, tmp); } #endif /* WITH_WSREP */ } @@ -5009,10 +5016,6 @@ ha_innobase::table_cache_type() return(HA_CACHE_TBL_ASKTRANSACT); } -/****************************************************************//** -Determines if the primary key is clustered index. -@return true */ - /** Normalizes a table name string. A normalized name consists of the database name catenated to '/' and table name. For example: test/mytable. @@ -5027,7 +5030,7 @@ normalize_table_name_c_low( char* norm_name, /* out: normalized name as a null-terminated string */ const char* name, /* in: table name string */ - ibool set_lower_case) /* in: TRUE if we want to set + bool set_lower_case) /* in: TRUE if we want to set name to lower case */ { char* name_ptr; @@ -5095,29 +5098,11 @@ create_table_info_t::create_table_info_t( m_default_row_format(innodb_default_row_format), m_create_info(create_info), m_table_name(table_name), m_table(NULL), - m_drop_before_rollback(false), m_remote_path(remote_path), m_innodb_file_per_table(file_per_table) { } -/** Normalizes a table name string. -A normalized name consists of the database name catenated to '/' -and table name. For example: test/mytable. -On Windows, normalization puts both the database name and the -table name always to lower case if "set_lower_case" is set to TRUE. -@param[out] norm_name Normalized name, null-terminated. -@param[in] name Name to normalize. -@param[in] set_lower_case True if we also should fold to lower case. */ -void -create_table_info_t::normalize_table_name_low( - char* norm_name, - const char* name, - ibool set_lower_case) -{ - normalize_table_name_c_low(norm_name, name, set_lower_case); -} - #if !defined(DBUG_OFF) /********************************************************************* Test normalize_table_name_low(). */ @@ -5172,7 +5157,7 @@ test_normalize_table_name_low() " testing \"%s\", expected \"%s\"... ", test_data[i][0], test_data[i][1]); - create_table_info_t::normalize_table_name_low( + normalize_table_name_c_low( norm_name, test_data[i][0], FALSE); if (strcmp(norm_name, test_data[i][1]) == 0) { @@ -6040,10 +6025,8 @@ ha_innobase::open_dict_table( whether there exists table name in system table whose name is not being normalized to lower case */ - create_table_info_t:: - normalize_table_name_low( - par_case_name, - table_name, FALSE); + normalize_table_name_c_low( + par_case_name, table_name, false); #endif ib_table = dict_table_open_on_name( par_case_name, FALSE, TRUE, @@ -10395,7 +10378,6 @@ create_table_info_t::create_table_def() DBUG_PRINT("enter", ("table_name: %s", m_table_name)); DBUG_ASSERT(m_trx->mysql_thd == m_thd); - DBUG_ASSERT(!m_drop_before_rollback); /* MySQL does the name length check. But we do additional check on the name length here */ @@ -10672,7 +10654,6 @@ err_col: } else { if (err == DB_SUCCESS) { err = row_create_table_for_mysql(table, m_trx); - m_drop_before_rollback = (err == DB_SUCCESS); } DBUG_EXECUTE_IF("ib_crash_during_create_for_encryption", @@ -12493,9 +12474,6 @@ int create_table_info_t::create_table(bool create_fk) DBUG_RETURN(error); } - DBUG_ASSERT(m_drop_before_rollback - == !(m_flags2 & DICT_TF2_TEMPORARY)); - /* Create the keys */ if (m_form->s->keys == 0 || primary_key_no == -1) { @@ -13044,18 +13022,15 @@ ha_innobase::create( /* Drop the being-created table before rollback, so that rollback can possibly rename back a table that could have been renamed before the failed creation. */ - if (info.drop_before_rollback()) { - trx->error_state = DB_SUCCESS; - row_drop_table_for_mysql(info.table_name(), - trx, SQLCOM_TRUNCATE, true, - false); - } trx_rollback_for_mysql(trx); row_mysql_unlock_data_dictionary(trx); } else { - innobase_commit_low(trx); + /* When this is invoked as part of ha_innobase::truncate(), + the old copy of the table will be deleted here. */ + std::vector<pfs_os_file_t> deleted; + trx->commit(deleted); row_mysql_unlock_data_dictionary(trx); - ut_ad(!srv_read_only_mode); + for (pfs_os_file_t d : deleted) os_file_close(d); error = info.create_table_update_dict(); } @@ -13142,7 +13117,7 @@ ha_innobase::discard_or_import_tablespace( } err = row_discard_tablespace_for_mysql( - m_prebuilt->table->name.m_name, m_prebuilt->trx); + m_prebuilt->table, m_prebuilt->trx); } else if (m_prebuilt->table->is_readable()) { /* Commit the transaction in order to @@ -13222,141 +13197,210 @@ ha_innobase::discard_or_import_tablespace( } -/** -Drops a table from an InnoDB database. Before calling this function, -MySQL calls innobase_commit to commit the transaction of the current user. -Then the current user cannot have locks set on the table. Drop table -operation inside InnoDB will remove all locks any user has on the table -inside InnoDB. -@param[in] name table name -@param[in] sqlcom SQLCOM_DROP_DB, SQLCOM_TRUNCATE, ... +/** DROP TABLE (possibly as part of DROP DATABASE, CREATE/ALTER TABLE) +@param name table name @return error number */ -inline int ha_innobase::delete_table(const char* name, enum_sql_command sqlcom) +int ha_innobase::delete_table(const char *name) { - dberr_t err; - THD* thd = ha_thd(); - char norm_name[FN_REFLEN]; - - DBUG_ENTER("ha_innobase::delete_table"); - - DBUG_EXECUTE_IF( - "test_normalize_table_name_low", - test_normalize_table_name_low(); - ); - DBUG_EXECUTE_IF( - "test_ut_format_name", - test_ut_format_name(); - ); - - /* Strangely, MySQL passes the table name without the '.frm' - extension, in contrast to ::create */ - normalize_table_name(norm_name, name); - - if (high_level_read_only) { - DBUG_RETURN(HA_ERR_TABLE_READONLY); - } - - trx_t* parent_trx = check_trx_exists(thd); + DBUG_ENTER("ha_innobase::delete_table"); + if (high_level_read_only) + DBUG_RETURN(HA_ERR_TABLE_READONLY); - /* Remove the to-be-dropped table from the list of modified tables - by parent_trx. Otherwise we may end up with an orphaned pointer to - the table object from parent_trx::mod_tables. This could happen in: - SET AUTOCOMMIT=0; - CREATE TABLE t (PRIMARY KEY (a)) ENGINE=INNODB SELECT 1 AS a UNION - ALL SELECT 1 AS a; */ - for (auto iter = parent_trx->mod_tables.begin(); - iter != parent_trx->mod_tables.end(); - ++iter) { + THD *thd= ha_thd(); - dict_table_t* table_to_drop = iter->first; + DBUG_EXECUTE_IF("test_normalize_table_name_low", + test_normalize_table_name_low();); + DBUG_EXECUTE_IF("test_ut_format_name", test_ut_format_name();); - if (strcmp(norm_name, table_to_drop->name.m_name) == 0) { - parent_trx->mod_tables.erase(iter); - break; - } - } + trx_t *parent_trx= check_trx_exists(thd); + dict_table_t *table; - trx_t* trx = innobase_trx_allocate(thd); + for (;;) + { + char norm_name[FN_REFLEN]; + normalize_table_name(norm_name, name); + span<const char> n{norm_name, strlen(norm_name)}; + + dict_sys.lock(SRW_LOCK_CALL); + table= dict_sys.load_table(n, DICT_ERR_IGNORE_DROP); +#ifdef WITH_PARTITION_STORAGE_ENGINE + if (!table && lower_case_table_names == 1 && is_partition(norm_name)) + { + IF_WIN(normalize_table_name_c_low(norm_name, name, false), + innobase_casedn_str(norm_name)); + table= dict_sys.load_table(n, DICT_ERR_IGNORE_DROP); + } +#endif + if (!table) + { + dict_sys.unlock(); + DBUG_RETURN(HA_ERR_NO_SUCH_TABLE); + } - ulint name_len = strlen(name); + if (dict_stats_stop_bg(table)) + break; + dict_sys.unlock(); + } - ut_a(name_len < 1000); + if (table->is_temporary()) + { + dict_sys.remove(table, false, true); + dict_sys.unlock(); + parent_trx->mod_tables.erase(table); /* CREATE...SELECT error handling */ + btr_drop_temporary_table(*table); + dict_mem_table_free(table); + DBUG_RETURN(0); + } - trx->will_lock = true; + table->acquire(); + dict_sys.unlock(); - /* Drop the table in InnoDB */ + trx_t *trx= parent_trx; + if (!trx->lock.table_locks.empty() && + thd_sql_command(trx->mysql_thd) == SQLCOM_CREATE_TABLE) + { +#if 0 // MDEV-21602 FIXME: this fails for innodb.innodb and some others + for (const lock_t *l : trx->lock.table_locks) + if (l && l->type_mode == (LOCK_IX | LOCK_TABLE) && + l->un_member.tab_lock.table == table) + goto create_select; + sql_print_warning("InnoDB: CREATE...SELECT did not hold expected locks"); +create_select: +#endif + /* CREATE TABLE...PRIMARY KEY...SELECT ought to be dropping the + table because a duplicate key was detected. We shall hijack the + existing transaction to drop the table and commit the transaction. + If this is a partitioned table, one partition will use this hijacked + transaction; others will use a separate transaction, one per partition. */ + ut_ad(!trx->dict_operation_lock_mode); + ut_ad(trx->will_lock); + ut_ad(trx->state == TRX_STATE_ACTIVE); + trx->dict_operation= true; + trx->internal= true; + } + else + { + trx= innobase_trx_allocate(thd); + trx_start_for_ddl(trx); + } - err = row_drop_table_for_mysql(norm_name, trx, sqlcom); + dberr_t err= lock_table_for_trx(table, trx, LOCK_X); + const bool fts= err == DB_SUCCESS && + (table->flags2 & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS)); - if (err == DB_TABLE_NOT_FOUND && lower_case_table_names == 1) { - char* is_part = is_partition(norm_name); + if (fts) + { + fts_optimize_remove_table(table); + purge_sys.stop_FTS(); + err= fts_lock_tables(trx, *table); + } - if (is_part) { - char par_case_name[FN_REFLEN]; + dict_sys.lock(SRW_LOCK_CALL); + trx->dict_operation_lock_mode= RW_X_LATCH; + if (!table->release() && err == DB_SUCCESS) + { + /* Wait for purge threads to stop using the table. */ + for (uint n= 15;;) + { + row_mysql_unlock_data_dictionary(trx); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + row_mysql_lock_data_dictionary(trx); -#ifndef _WIN32 - /* Check for the table using lower - case name, including the partition - separator "P" */ - strcpy(par_case_name, norm_name); - innobase_casedn_str(par_case_name); -#else - /* On Windows platfrom, check - whether there exists table name in - system table whose name is - not being normalized to lower case */ - normalize_table_name_c_low( - par_case_name, name, FALSE); -#endif - err = row_drop_table_for_mysql( - par_case_name, trx, sqlcom); - } - } + if (!--n) + { + err= DB_LOCK_WAIT_TIMEOUT; + break; + } + if (!table->get_ref_count()) + break; + } + } - ut_ad(!srv_read_only_mode); + if (err != DB_SUCCESS) + { +err_exit: + trx->rollback(); + trx->dict_operation_lock_mode= 0; + switch (err) { + case DB_CANNOT_DROP_CONSTRAINT: + case DB_LOCK_WAIT_TIMEOUT: + break; + default: + ib::error() << "DROP TABLE " << table->name << ": " << err; + } + table->stats_bg_flag= BG_STAT_NONE; + if (fts) + { + fts_optimize_add_table(table); + purge_sys.resume_FTS(); + } + dict_sys.unlock(); + if (trx != parent_trx) + trx->free(); + DBUG_RETURN(convert_error_code_to_mysql(err, 0, NULL)); + } - innobase_commit_low(trx); + if (!table->no_rollback() && trx->check_foreigns) + { + const bool drop_db= thd_sql_command(thd) == SQLCOM_DROP_DB; + for (auto foreign : table->referenced_set) + { + /* We should allow dropping a referenced table if creating + that referenced table has failed for some reason. For example + if referenced table is created but it column types that are + referenced do not match. */ + if (foreign->foreign_table == table || + (drop_db && + dict_tables_have_same_db(table->name.m_name, + foreign->foreign_table_name_lookup))) + continue; + mysql_mutex_lock(&dict_foreign_err_mutex); + rewind(dict_foreign_err_file); + ut_print_timestamp(dict_foreign_err_file); + fputs(" Cannot drop table ", dict_foreign_err_file); + ut_print_name(dict_foreign_err_file, trx, table->name.m_name); + fputs("\nbecause it is referenced by ", dict_foreign_err_file); + ut_print_name(dict_foreign_err_file, trx, foreign->foreign_table_name); + putc('\n', dict_foreign_err_file); + mysql_mutex_unlock(&dict_foreign_err_mutex); + err= DB_CANNOT_DROP_CONSTRAINT; + goto err_exit; + } + } - trx->free(); + if (!table->no_rollback()) + { + err= trx->drop_table_foreign(table->name); + if (err == DB_SUCCESS) + err= trx->drop_table_statistics(table->name); + if (err != DB_SUCCESS) + goto err_exit; + } - DBUG_RETURN(convert_error_code_to_mysql(err, 0, NULL)); -} + err= trx->drop_table(*table); + if (err != DB_SUCCESS) + goto err_exit; -/** Drop an InnoDB table. -@param[in] name table name -@return error number */ -int ha_innobase::delete_table(const char* name) -{ - enum_sql_command sqlcom = enum_sql_command(thd_sql_command(ha_thd())); - /* SQLCOM_TRUNCATE should be passed via ha_innobase::truncate() only. - - On client disconnect, when dropping temporary tables, the - previous sqlcom would not be overwritten. In such a case, we - will have thd_kill_level() != NOT_KILLED, !m_prebuilt can - hold, and sqlcom could be anything, including TRUNCATE. - - The sqlcom only matters for persistent tables; no persistent - metadata or FOREIGN KEY metadata is kept for temporary - tables. Therefore, we relax the assertion. If there is a bug - that slips through this assertion due to !m_prebuilt, the - worst impact should be that on DROP TABLE of a persistent - table, FOREIGN KEY constraints will be ignored and their - metadata will not be removed. */ - DBUG_ASSERT(sqlcom != SQLCOM_TRUNCATE - || (thd_kill_level(ha_thd()) != THD_IS_NOT_KILLED - && (!m_prebuilt - || m_prebuilt->table->is_temporary()))); - return delete_table(name, sqlcom); + std::vector<pfs_os_file_t> deleted; + trx->commit(deleted); + row_mysql_unlock_data_dictionary(trx); + if (trx != parent_trx) + trx->free(); + for (pfs_os_file_t d : deleted) + os_file_close(d); + if (fts) + purge_sys.resume_FTS(); + DBUG_RETURN(0); } /** Rename an InnoDB table. @param[in,out] trx InnoDB data dictionary transaction @param[in] from old table name @param[in] to new table name -@param[in] commit whether to commit trx (and to enforce FOREIGN KEY) +@param[in] use_fk whether to enforce FOREIGN KEY @return DB_SUCCESS or error code */ inline dberr_t innobase_rename_table(trx_t *trx, const char *from, - const char *to, bool commit) + const char *to, bool use_fk) { dberr_t error; char norm_to[FN_REFLEN]; @@ -13375,14 +13419,7 @@ inline dberr_t innobase_rename_table(trx_t *trx, const char *from, trx_start_if_not_started(trx, true); ut_ad(trx->will_lock); - if (commit) { - /* Serialize data dictionary operations with dictionary mutex: - no deadlocks can occur then in these operations. */ - row_mysql_lock_data_dictionary(trx); - } - - error = row_rename_table_for_mysql(norm_from, norm_to, trx, commit, - commit); + error = row_rename_table_for_mysql(norm_from, norm_to, trx, use_fk); if (error != DB_SUCCESS) { if (error == DB_TABLE_NOT_FOUND @@ -13402,13 +13439,12 @@ inline dberr_t innobase_rename_table(trx_t *trx, const char *from, whether there exists table name in system table whose name is not being normalized to lower case */ - create_table_info_t::normalize_table_name_low( - par_case_name, from, FALSE); + normalize_table_name_c_low( + par_case_name, from, false); #endif /* _WIN32 */ trx_start_if_not_started(trx, true); error = row_rename_table_for_mysql( - par_case_name, norm_to, trx, - true, false); + par_case_name, norm_to, trx, false); } } @@ -13432,10 +13468,6 @@ inline dberr_t innobase_rename_table(trx_t *trx, const char *from, } } - if (commit) { - row_mysql_unlock_data_dictionary(trx); - } - DBUG_RETURN(error); } @@ -13453,92 +13485,158 @@ int ha_innobase::truncate() } HA_CREATE_INFO info; - mem_heap_t* heap = mem_heap_create(1000); dict_table_t* ib_table = m_prebuilt->table; - const auto update_time = ib_table->update_time; - const auto stored_lock = m_prebuilt->stored_select_lock_type; info.init(); update_create_info_from_table(&info, table); + switch (dict_tf_get_rec_format(ib_table->flags)) { + case REC_FORMAT_REDUNDANT: + info.row_type = ROW_TYPE_REDUNDANT; + break; + case REC_FORMAT_COMPACT: + info.row_type = ROW_TYPE_COMPACT; + break; + case REC_FORMAT_COMPRESSED: + info.row_type = ROW_TYPE_COMPRESSED; + break; + case REC_FORMAT_DYNAMIC: + info.row_type = ROW_TYPE_DYNAMIC; + break; + } + + const auto stored_lock = m_prebuilt->stored_select_lock_type; + trx_t* trx = innobase_trx_allocate(m_user_thd); + trx_start_for_ddl(trx); if (ib_table->is_temporary()) { info.options|= HA_LEX_CREATE_TMP_TABLE; - } else { - dict_get_and_save_data_dir_path(ib_table, false); - } + btr_drop_temporary_table(*ib_table); + m_prebuilt->table = nullptr; + row_prebuilt_free(m_prebuilt, false); + m_prebuilt = nullptr; + my_free(m_upd_buf); + m_upd_buf = nullptr; + m_upd_buf_size = 0; + + row_mysql_lock_data_dictionary(trx); + ib_table->release(); + dict_sys.remove(ib_table, false, true); + + int err = create(ib_table->name.m_name, table, &info, true, + trx); + if (!err) { + err = open(ib_table->name.m_name, 0, 0); + m_prebuilt->stored_select_lock_type = stored_lock; + } - char* data_file_name = ib_table->data_dir_path; + trx->free(); - if (data_file_name) { - info.data_file_name = data_file_name - = mem_heap_strdup(heap, data_file_name); +#ifdef BTR_CUR_HASH_ADAPT + if (UT_LIST_GET_LEN(ib_table->freed_indexes)) { + ib_table->vc_templ = nullptr; + ib_table->id = 0; + DBUG_RETURN(err); + } +#endif /* BTR_CUR_HASH_ADAPT */ + + dict_mem_table_free(ib_table); + DBUG_RETURN(err); } + mem_heap_t* heap = mem_heap_create(1000); + + dict_get_and_save_data_dir_path(ib_table, false); + info.data_file_name = ib_table->data_dir_path; const char* temp_name = dict_mem_create_temporary_tablename( heap, ib_table->name.m_name, ib_table->id); const char* name = mem_heap_strdup(heap, ib_table->name.m_name); - trx_t* trx = innobase_trx_allocate(m_user_thd); - trx->will_lock = true; - trx->dict_operation = true; + dberr_t error = lock_table_for_trx(ib_table, trx, LOCK_X); + const bool fts = error == DB_SUCCESS + && ib_table->flags2 & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS); + + if (fts) { + fts_optimize_remove_table(ib_table); + purge_sys.stop_FTS(); + error = fts_lock_tables(trx, *ib_table); + } + row_mysql_lock_data_dictionary(trx); dict_stats_wait_bg_to_stop_using_table(ib_table, trx); + /* Wait for purge threads to stop using the table. */ + for (uint n = 15; ib_table->get_ref_count() > 1; ) { + if (!--n) { + error = DB_LOCK_WAIT_TIMEOUT; + break; + } + + row_mysql_unlock_data_dictionary(trx); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + row_mysql_lock_data_dictionary(trx); + } - int err = convert_error_code_to_mysql( - innobase_rename_table(trx, ib_table->name.m_name, temp_name, - false), - ib_table->flags, m_user_thd); + if (error == DB_SUCCESS) { + error = innobase_rename_table(trx, ib_table->name.m_name, + temp_name, false); + + if (error == DB_SUCCESS) { + error = trx->drop_table(*ib_table); + } + } + + int err = convert_error_code_to_mysql(error, ib_table->flags, + m_user_thd); if (err) { trx_rollback_for_mysql(trx); + if (fts) { + fts_optimize_add_table(ib_table); + purge_sys.resume_FTS(); + } row_mysql_unlock_data_dictionary(trx); } else { - switch (dict_tf_get_rec_format(ib_table->flags)) { - case REC_FORMAT_REDUNDANT: - info.row_type = ROW_TYPE_REDUNDANT; - break; - case REC_FORMAT_COMPACT: - info.row_type = ROW_TYPE_COMPACT; - break; - case REC_FORMAT_COMPRESSED: - info.row_type = ROW_TYPE_COMPRESSED; - break; - case REC_FORMAT_DYNAMIC: - info.row_type = ROW_TYPE_DYNAMIC; - break; - } + const auto update_time = ib_table->update_time; + const auto stored_lock = m_prebuilt->stored_select_lock_type; + ib_table->release(); + m_prebuilt->table = nullptr; err = create(name, table, &info, - ib_table->is_temporary() - || dict_table_is_file_per_table(ib_table), trx); - } + dict_table_is_file_per_table(ib_table), trx); + if (fts) { + purge_sys.resume_FTS(); + } - trx->free(); + if (err) { +reload: + m_prebuilt->table = dict_table_open_on_name( + name, false, false, DICT_ERR_IGNORE_NONE); + } else { + row_prebuilt_t* prebuilt = m_prebuilt; + uchar* upd_buf = m_upd_buf; + ulint upd_buf_size = m_upd_buf_size; + /* Mimic ha_innobase::close(). */ + m_prebuilt = nullptr; + m_upd_buf = nullptr; + m_upd_buf_size = 0; - if (!err) { - /* Reopen the newly created table, and drop the - original table that was renamed to temp_name. */ + err = open(name, 0, 0); - row_prebuilt_t* prebuilt = m_prebuilt; - uchar* upd_buf = m_upd_buf; - ulint upd_buf_size = m_upd_buf_size; - /* Mimic ha_innobase::close(). */ - m_prebuilt = NULL; - m_upd_buf = NULL; - m_upd_buf_size = 0; - err = open(name, 0, 0); - if (!err) { - m_prebuilt->stored_select_lock_type = stored_lock; - m_prebuilt->table->update_time = update_time; - row_prebuilt_free(prebuilt, FALSE); - delete_table(temp_name, SQLCOM_TRUNCATE); - my_free(upd_buf); - } else { - /* Revert to the old table before truncation. */ - m_prebuilt = prebuilt; - m_upd_buf = upd_buf; - m_upd_buf_size = upd_buf_size; + if (!err) { + m_prebuilt->stored_select_lock_type + = stored_lock; + m_prebuilt->table->update_time = update_time; + row_prebuilt_free(prebuilt, false); + my_free(upd_buf); + } else { + /* Revert to the old table. */ + m_prebuilt = prebuilt; + m_upd_buf = upd_buf; + m_upd_buf_size = upd_buf_size; + goto reload; + } } } + trx->free(); + mem_heap_free(heap); DBUG_RETURN(err); } @@ -13567,54 +13665,46 @@ ha_innobase::rename_table( /* We are doing a DDL operation. */ trx->will_lock = true; trx->dict_operation = true; + row_mysql_lock_data_dictionary(trx); dberr_t error = innobase_rename_table(trx, from, to, true); DEBUG_SYNC(thd, "after_innobase_rename_table"); - innobase_commit_low(trx); - - trx->free(); - if (error == DB_SUCCESS) { char norm_from[MAX_FULL_NAME_LEN]; char norm_to[MAX_FULL_NAME_LEN]; - char errstr[512]; - dberr_t ret; normalize_table_name(norm_from, from); normalize_table_name(norm_to, to); - ret = dict_stats_rename_table(norm_from, norm_to, - errstr, sizeof(errstr)); - - if (ret != DB_SUCCESS) { - ib::error() << errstr; - - push_warning(thd, Sql_condition::WARN_LEVEL_WARN, - ER_LOCK_WAIT_TIMEOUT, errstr); + error = dict_stats_rename_table(norm_from, norm_to, trx); + if (error == DB_DUPLICATE_KEY) { + /* The duplicate may also occur in + mysql.innodb_index_stats. */ + my_error(ER_DUP_KEY, MYF(0), + "mysql.innodb_table_stats"); + error = DB_ERROR; } } - /* Add a special case to handle the Duplicated Key error - and return DB_ERROR instead. - This is to avoid a possible SIGSEGV error from mysql error - handling code. Currently, mysql handles the Duplicated Key - error by re-entering the storage layer and getting dup key - info by calling get_dup_key(). This operation requires a valid - table handle ('row_prebuilt_t' structure) which could no - longer be available in the error handling stage. The suggested - solution is to report a 'table exists' error message (since - the dup key error here is due to an existing table whose name - is the one we are trying to rename to) and return the generic - error code. */ + if (error == DB_SUCCESS) { + innobase_commit_low(trx); + } else { + trx->rollback(); + } + + row_mysql_unlock_data_dictionary(trx); + trx->free(); + if (error == DB_DUPLICATE_KEY) { + /* We are not able to deal with handler::get_dup_key() + during DDL operations, because the duplicate key would + exist in metadata tables, not in the user table. */ my_error(ER_TABLE_EXISTS_ERROR, MYF(0), to); - error = DB_ERROR; } else if (error == DB_LOCK_WAIT_TIMEOUT) { my_error(ER_LOCK_WAIT_TIMEOUT, MYF(0), to); - error = DB_LOCK_WAIT; } @@ -17784,20 +17874,11 @@ innobase_fts_find_ranking(FT_INFO* fts_hdl, uchar*, uint) } #ifdef UNIV_DEBUG -static my_bool innodb_background_drop_list_empty = TRUE; static my_bool innodb_log_checkpoint_now = TRUE; static my_bool innodb_buf_flush_list_now = TRUE; static uint innodb_merge_threshold_set_all_debug = DICT_INDEX_MERGE_THRESHOLD_DEFAULT; -/** Wait for the background drop list to become empty. */ -static -void -wait_background_drop_list_empty(THD*, st_mysql_sys_var*, void*, const void*) -{ - row_wait_for_background_drop_list_empty(); -} - /****************************************************************//** Force innodb to checkpoint. */ static @@ -18341,12 +18422,6 @@ static MYSQL_SYSVAR_ULONG(io_capacity_max, srv_max_io_capacity, SRV_MAX_IO_CAPACITY_LIMIT, 0); #ifdef UNIV_DEBUG -static MYSQL_SYSVAR_BOOL(background_drop_list_empty, - innodb_background_drop_list_empty, - PLUGIN_VAR_OPCMDARG, - "Wait for the background drop list to become empty", - NULL, wait_background_drop_list_empty, FALSE); - static MYSQL_SYSVAR_BOOL(log_checkpoint_now, innodb_log_checkpoint_now, PLUGIN_VAR_OPCMDARG, "Force checkpoint now", @@ -19382,7 +19457,6 @@ static struct st_mysql_sys_var* innobase_system_variables[]= { MYSQL_SYSVAR(purge_threads), MYSQL_SYSVAR(purge_batch_size), #ifdef UNIV_DEBUG - MYSQL_SYSVAR(background_drop_list_empty), MYSQL_SYSVAR(log_checkpoint_now), MYSQL_SYSVAR(buf_flush_list_now), MYSQL_SYSVAR(merge_threshold_set_all_debug), diff --git a/storage/innobase/handler/ha_innodb.h b/storage/innobase/handler/ha_innodb.h index 431e93f7021..4185f7a68cd 100644 --- a/storage/innobase/handler/ha_innodb.h +++ b/storage/innobase/handler/ha_innodb.h @@ -204,8 +204,6 @@ public: TABLE* form, HA_CREATE_INFO* create_info) override; - inline int delete_table(const char* name, enum_sql_command sqlcom); - int truncate() override; int delete_table(const char *name) override; @@ -708,26 +706,9 @@ public: const char* table_name() const { return(m_table_name); } - /** @return whether the table needs to be dropped on rollback */ - bool drop_before_rollback() const { return m_drop_before_rollback; } - THD* thd() const { return(m_thd); } - /** Normalizes a table name string. - A normalized name consists of the database name catenated to '/' and - table name. An example: test/mytable. On Windows normalization puts - both the database name and the table name always to lower case if - "set_lower_case" is set to true. - @param[in,out] norm_name Buffer to return the normalized name in. - @param[in] name Table name string. - @param[in] set_lower_case True if we want to set name to lower - case. */ - static void normalize_table_name_low( - char* norm_name, - const char* name, - ibool set_lower_case); - private: /** Parses the table name into normal name and either temp path or remote path if needed.*/ @@ -757,8 +738,6 @@ private: char* m_table_name; /** Table */ dict_table_t* m_table; - /** Whether the table needs to be dropped before rollback */ - bool m_drop_before_rollback; /** Remote path (DATA DIRECTORY) or zero length-string */ char* m_remote_path; @@ -861,15 +840,6 @@ innodb_base_col_setup_for_stored( /** whether this is a stored generated column */ #define innobase_is_s_fld(field) ((field)->vcol_info && (field)->stored_in_db()) -/** Always normalize table name to lower case on Windows */ -#ifdef _WIN32 -#define normalize_table_name(norm_name, name) \ - create_table_info_t::normalize_table_name_low(norm_name, name, TRUE) -#else -#define normalize_table_name(norm_name, name) \ - create_table_info_t::normalize_table_name_low(norm_name, name, FALSE) -#endif /* _WIN32 */ - /** Converts a search mode flag understood by MySQL to a flag understood by InnoDB. @param[in] find_flag MySQL search mode flag. diff --git a/storage/innobase/handler/handler0alter.cc b/storage/innobase/handler/handler0alter.cc index 471ff1574d6..fce6c06b6c6 100644 --- a/storage/innobase/handler/handler0alter.cc +++ b/storage/innobase/handler/handler0alter.cc @@ -1164,10 +1164,14 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx } } -private: - // Disable copying - ha_innobase_inplace_ctx(const ha_innobase_inplace_ctx&); - ha_innobase_inplace_ctx& operator=(const ha_innobase_inplace_ctx&); + /** @return whether a FULLTEXT INDEX is being added */ + bool adding_fulltext_index() const + { + for (ulint a= 0; a < num_to_add_index; a++) + if (add_index[a]->type & DICT_FTS) + return true; + return false; + } }; /********************************************************************//** @@ -4515,38 +4519,6 @@ found_col: DBUG_RETURN(col_map); } -/** Drop newly create FTS index related auxiliary table during -FIC create index process, before fts_add_index is called -@param table table that was being rebuilt online -@param trx transaction -@return DB_SUCCESS if successful, otherwise last error code -*/ -static -dberr_t -innobase_drop_fts_index_table( -/*==========================*/ - dict_table_t* table, - trx_t* trx) -{ - dberr_t ret_err = DB_SUCCESS; - - for (dict_index_t* index = dict_table_get_first_index(table); - index != NULL; - index = dict_table_get_next_index(index)) { - if (index->type & DICT_FTS) { - dberr_t err; - - err = fts_drop_index_tables(trx, index); - - if (err != DB_SUCCESS) { - ret_err = err; - } - } - } - - return(ret_err); -} - /** Get the new non-virtual column names if any columns were renamed @param ha_alter_info Data used during in-place alter @param altered_table MySQL table that is being altered @@ -6122,6 +6094,24 @@ create_index_dict( DBUG_RETURN(index); } +/** After releasing the data dictionary latch, close deleted files +@param deleted handles of deleted files */ +static void close_unlinked_files(const std::vector<pfs_os_file_t> &deleted) +{ + dict_sys.assert_not_locked(); + for (pfs_os_file_t d : deleted) + os_file_close(d); +} + +/** Commit a DDL transaction and unlink any deleted files. */ +static void commit_unlock_and_unlink(trx_t *trx) +{ + std::vector<pfs_os_file_t> deleted; + trx->commit(deleted); + row_mysql_unlock_data_dictionary(trx); + close_unlinked_files(deleted); +} + /** Update internal structures with concurrent writes blocked, while preparing ALTER TABLE. @@ -6277,6 +6267,8 @@ prepare_inplace_alter_table_dict( mem_heap_alloc(ctx->heap, ctx->num_to_add_index * sizeof *ctx->add_key_numbers)); + const bool fts_exist = ctx->new_table->flags2 + & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS); /* Acquire a lock on the table before creating any indexes. */ bool table_lock_failed = false; @@ -6284,15 +6276,21 @@ prepare_inplace_alter_table_dict( error = DB_SUCCESS; } else { ctx->prebuilt->trx->op_info = "acquiring table lock"; - error = lock_table_for_trx(ctx->new_table, - ctx->prebuilt->trx, LOCK_S); + error = lock_table_for_trx(ctx->new_table, ctx->trx, LOCK_S); + } - if (error != DB_SUCCESS) { - table_lock_failed = true; - goto error_handling; + if (fts_exist) { + purge_sys.stop_FTS(); + if (error == DB_SUCCESS) { + error = fts_lock_tables(ctx->trx, *ctx->new_table); } } + if (error != DB_SUCCESS) { + table_lock_failed = true; + goto error_handling; + } + /* Latch the InnoDB data dictionary exclusively so that no deadlocks or lock waits can happen in it during an index create operation. */ @@ -7086,22 +7084,28 @@ error_handling_drop_uncached: DBUG_ASSERT(error == DB_SUCCESS); - if (UT_LIST_GET_LEN(ctx->trx->lock.trx_locks)) { + { /* Commit the data dictionary transaction in order to release the table locks on the system tables. This means that if MariaDB is killed while rebuilding the table inside row_merge_build_indexes(), ctx->new_table will not be dropped by trx_rollback_active(). */ - trx_commit_for_mysql(ctx->trx); - trx_start_for_ddl(ctx->trx); - + ut_d(dict_table_check_for_dup_indexes(user_table, + CHECK_PARTIAL_OK)); if (ctx->need_rebuild()) { ctx->new_table->acquire(); } - } - ut_d(dict_table_check_for_dup_indexes(user_table, CHECK_PARTIAL_OK)); - row_mysql_unlock_data_dictionary(ctx->trx); + /* fts_create_common_tables() may drop old common tables, + whose files would be deleted here. */ + commit_unlock_and_unlink(ctx->trx); + if (fts_exist) { + purge_sys.resume_FTS(); + } + + trx_start_for_ddl(ctx->trx); + ctx->prebuilt->trx_id = ctx->trx->id; + } if (ctx->old_table->fts) { fts_sync_during_ddl(ctx->old_table); @@ -7190,6 +7194,9 @@ err_exit: ctx->trx->free(); } trx_commit_for_mysql(ctx->prebuilt->trx); + if (fts_exist) { + purge_sys.resume_FTS(); + } for (uint i = 0; i < ctx->num_to_add_fk; i++) { if (ctx->add_fk[i]) { @@ -8654,26 +8661,7 @@ innobase_rollback_sec_index( } } -/* Get the number of uncommitted fts index during rollback -operation. -@param[in] table table which undergoes rollback for alter -@return number of uncommitted fts indexes. */ -static -ulint innobase_get_uncommitted_fts_indexes(const dict_table_t* table) -{ - dict_sys.assert_locked(); - dict_index_t* index = dict_table_get_first_index(table); - ulint n_uncommitted_fts = 0; - - for (; index ; index = dict_table_get_next_index(index)) - { - if (index->type & DICT_FTS && !index->is_committed()) - n_uncommitted_fts++; - } - - return n_uncommitted_fts; -} - +MY_ATTRIBUTE((nonnull, warn_unused_result)) /** Roll back the changes made during prepare_inplace_alter_table() and inplace_alter_table() inside the storage engine. Note that the allowed level of concurrency during this operation will be the same as @@ -8687,162 +8675,150 @@ during prepare, but might not be during commit). @retval true Failure @retval false Success */ -inline MY_ATTRIBUTE((nonnull, warn_unused_result)) -bool -rollback_inplace_alter_table( -/*=========================*/ - Alter_inplace_info* ha_alter_info, - const TABLE* table, - row_prebuilt_t* prebuilt) +inline bool rollback_inplace_alter_table(Alter_inplace_info *ha_alter_info, + const TABLE *table, + row_prebuilt_t *prebuilt) { - bool fail = false; - - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*> - (ha_alter_info->handler_ctx); - - DBUG_ENTER("rollback_inplace_alter_table"); - - if (!ctx) { - /* If we have not started a transaction yet, - (almost) nothing has been or needs to be done. */ - goto func_exit; - } - - row_mysql_lock_data_dictionary(ctx->trx); - ctx->trx->dict_operation = true; - - if (!ctx->new_table) { - } else if (ctx->need_rebuild()) { - /* DML threads can access ctx->new_table via the - online rebuild log. Free it first. */ - innobase_online_rebuild_log_free(prebuilt->table); - dberr_t err= DB_SUCCESS; - ulint flags = ctx->new_table->flags; - - /* Since the FTS index specific auxiliary tables has - not yet registered with "table->fts" by fts_add_index(), - we will need explicitly delete them here */ - if (dict_table_has_fts_index(ctx->new_table)) { - - err = innobase_drop_fts_index_table( - ctx->new_table, ctx->trx); - - if (err != DB_SUCCESS) { - my_error_innodb( - err, table->s->table_name.str, - flags); - fail = true; - } - } - - ut_d(const bool last_handle=) ctx->new_table->release(); - ut_ad(last_handle); - err = row_drop_table_for_mysql(ctx->new_table->name.m_name, - ctx->trx, SQLCOM_DROP_TABLE, - false, false); - if (err != DB_SUCCESS) { - my_error_innodb(err, table->s->table_name.str, - flags); - fail = true; - } - } else { - DBUG_ASSERT(!(ha_alter_info->handler_flags - & ALTER_ADD_PK_INDEX)); - DBUG_ASSERT(ctx->new_table == prebuilt->table); - - /* Remove the fts table from fts_optimize_wq if - there is only one fts index exist. */ - if (prebuilt->table->fts - && innobase_get_uncommitted_fts_indexes( - prebuilt->table) == 1 - && (ib_vector_is_empty(prebuilt->table->fts->indexes) - || ib_vector_size(prebuilt->table->fts->indexes) - == 1)) { - row_mysql_unlock_data_dictionary(ctx->trx); - fts_optimize_remove_table(prebuilt->table); - row_mysql_lock_data_dictionary(ctx->trx); - } - - innobase_rollback_sec_index( - prebuilt->table, table, - (ha_alter_info->alter_info->requested_lock - == Alter_info::ALTER_TABLE_LOCK_EXCLUSIVE), - ctx->trx, prebuilt->trx); - - ctx->clean_new_vcol_index(); - } + bool fail= false; + ha_innobase_inplace_ctx *ctx= static_cast<ha_innobase_inplace_ctx*> + (ha_alter_info->handler_ctx); - trx_commit_for_mysql(ctx->trx); - row_mysql_unlock_data_dictionary(ctx->trx); - ctx->trx->free(); - ctx->trx = NULL; + DBUG_ENTER("rollback_inplace_alter_table"); -func_exit: -#ifndef DBUG_OFF - dict_index_t* clust_index = dict_table_get_first_index( - prebuilt->table); - DBUG_ASSERT(!clust_index->online_log); - DBUG_ASSERT(dict_index_get_online_status(clust_index) - == ONLINE_INDEX_COMPLETE); -#endif /* !DBUG_OFF */ - - if (ctx) { - DBUG_ASSERT(ctx->prebuilt == prebuilt); - - if (ctx->num_to_add_fk) { - for (ulint i = 0; i < ctx->num_to_add_fk; i++) { - dict_foreign_free(ctx->add_fk[i]); - } - } - - if (ctx->num_to_drop_index) { - row_mysql_lock_data_dictionary(prebuilt->trx); + if (!ctx) + /* If we have not started a transaction yet, + (almost) nothing has been or needs to be done. */ + dict_sys.lock(SRW_LOCK_CALL); + else if (ctx->new_table) + { + ut_ad(ctx->trx->state == TRX_STATE_ACTIVE); + const bool fts_exist= (ctx->new_table->flags2 & + (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS)) || + ctx->adding_fulltext_index(); + if (fts_exist) + { + fts_optimize_remove_table(ctx->new_table); + purge_sys.stop_FTS(); + } + if (ctx->need_rebuild()) + { + dberr_t err= lock_table_for_trx(ctx->new_table, ctx->trx, LOCK_X); + if (fts_exist) + { + if (err == DB_SUCCESS) + err= fts_lock_common_tables(ctx->trx, *ctx->new_table); + for (const dict_index_t* index= ctx->new_table->indexes.start; + err == DB_SUCCESS && index; index= index->indexes.next) + if (index->type & DICT_FTS) + err= fts_lock_index_tables(ctx->trx, *index); + } + row_mysql_lock_data_dictionary(ctx->trx); + /* Detach ctx->new_table from dict_index_t::online_log. */ + innobase_online_rebuild_log_free(ctx->old_table); + + ut_d(const bool last_handle=) ctx->new_table->release(); + ut_ad(last_handle); + if (err == DB_SUCCESS) + err= ctx->trx->drop_table(*ctx->new_table); + + if (err == DB_SUCCESS) + for (const dict_index_t* index= ctx->new_table->indexes.start; index; + index= index->indexes.next) + if (index->type & DICT_FTS) + if (dberr_t err2= fts_drop_index_tables(ctx->trx, *index)) + err= err2; + + if (err != DB_SUCCESS) + { + my_error_innodb(err, table->s->table_name.str, ctx->new_table->flags); + fail= true; + } + } + else + { + DBUG_ASSERT(!(ha_alter_info->handler_flags & ALTER_ADD_PK_INDEX)); + DBUG_ASSERT(ctx->old_table == prebuilt->table); + if (fts_exist) + { + for (ulint a= 0; a < ctx->num_to_add_index; a++) + { + const dict_index_t *index = ctx->add_index[a]; + // FIXME: skip fts_drop_index_tables() if we failed to acquire locks + if (index->type & DICT_FTS) + fts_lock_index_tables(ctx->trx, *index); + } + // FIXME: skip fts_drop_tables() if we failed to acquire locks + fts_lock_common_tables(ctx->trx, *ctx->new_table); + } + row_mysql_lock_data_dictionary(ctx->trx); + ctx->rollback_instant(); + innobase_rollback_sec_index(ctx->old_table, table, + ha_alter_info->alter_info->requested_lock == + Alter_info::ALTER_TABLE_LOCK_EXCLUSIVE, + ctx->trx, prebuilt->trx); + ctx->clean_new_vcol_index(); + ut_d(dict_table_check_for_dup_indexes(ctx->old_table, CHECK_ABORTED_OK)); + } - /* Clear the to_be_dropped flags - in the data dictionary cache. - The flags may already have been cleared, - in case an error was detected in - commit_inplace_alter_table(). */ - for (ulint i = 0; i < ctx->num_to_drop_index; i++) { - dict_index_t* index = ctx->drop_index[i]; - DBUG_ASSERT(index->is_committed()); - index->to_be_dropped = 0; - } + commit_unlock_and_unlink(ctx->trx); + if (fts_exist) + purge_sys.resume_FTS(); + if (ctx->old_table->fts) + { + dict_sys.mutex_lock(); + ut_ad(fts_check_cached_index(ctx->old_table)); + fts_optimize_add_table(ctx->old_table); + dict_sys.mutex_unlock(); + } + goto free_and_exit; + } + else + { +free_and_exit: + DBUG_ASSERT(ctx->prebuilt == prebuilt); + ctx->trx->free(); + ctx->trx= nullptr; + + dict_sys.lock(SRW_LOCK_CALL); + + for (ulint i= 0; i < ctx->num_to_add_fk; i++) + dict_foreign_free(ctx->add_fk[i]); + /* Clear the to_be_dropped flags in the data dictionary cache. + The flags may already have been cleared, in case an error was + detected in commit_inplace_alter_table(). */ + for (ulint i= 0; i < ctx->num_to_drop_index; i++) + { + dict_index_t *index= ctx->drop_index[i]; + DBUG_ASSERT(index->is_committed()); + index->to_be_dropped= 0; + } + } - row_mysql_unlock_data_dictionary(prebuilt->trx); - } - } + DBUG_ASSERT(!prebuilt->table->indexes.start->online_log); + DBUG_ASSERT(prebuilt->table->indexes.start->online_status == + ONLINE_INDEX_COMPLETE); - /* Reset dict_col_t::ord_part for those columns fail to be indexed, - we do this by checking every existing column, if any current - index would index them */ - for (ulint i = 0; i < dict_table_get_n_cols(prebuilt->table); i++) { - dict_col_t& col = prebuilt->table->cols[i]; - if (!col.ord_part) { - continue; - } - if (!check_col_exists_in_indexes(prebuilt->table, i, false, - true)) { - col.ord_part = 0; - } - } - - for (ulint i = 0; i < dict_table_get_n_v_cols(prebuilt->table); i++) { - dict_col_t& col = prebuilt->table->v_cols[i].m_col; - if (!col.ord_part) { - continue; - } - if (!check_col_exists_in_indexes(prebuilt->table, i, true, - true)) { - col.ord_part = 0; - } - } + /* Reset dict_col_t::ord_part for unindexed columns */ + for (ulint i= 0; i < dict_table_get_n_cols(prebuilt->table); i++) + { + dict_col_t &col= prebuilt->table->cols[i]; + if (col.ord_part && !check_col_exists_in_indexes(prebuilt->table, i, false, + true)) + col.ord_part= 0; + } - trx_commit_for_mysql(prebuilt->trx); - prebuilt->trx_id = 0; - MONITOR_ATOMIC_DEC(MONITOR_PENDING_ALTER_TABLE); - DBUG_RETURN(fail); + for (ulint i = 0; i < dict_table_get_n_v_cols(prebuilt->table); i++) + { + dict_col_t &col = prebuilt->table->v_cols[i].m_col; + if (col.ord_part && !check_col_exists_in_indexes(prebuilt->table, i, true, + true)) + col.ord_part= 0; + } + dict_sys.unlock(); + trx_commit_for_mysql(prebuilt->trx); + prebuilt->trx_id = 0; + MONITOR_ATOMIC_DEC(MONITOR_PENDING_ALTER_TABLE); + DBUG_RETURN(fail); } /** Drop a FOREIGN KEY constraint from the data dictionary tables. @@ -9679,8 +9655,6 @@ innobase_update_foreign_cache( column names. No need to pass col_names or to drop constraints from the data dictionary cache. */ DBUG_ASSERT(!ctx->col_names); - DBUG_ASSERT(user_table->foreign_set.empty()); - DBUG_ASSERT(user_table->referenced_set.empty()); user_table = ctx->new_table; } else { /* Drop the foreign key constraints if the @@ -9924,6 +9898,8 @@ commit_try_rebuild( DBUG_ASSERT(ctx->num_to_drop_fk <= ha_alter_info->alter_info->drop_list.elements); + innobase_online_rebuild_log_free(user_table); + for (dict_index_t* index = dict_table_get_first_index(rebuilt_table); index; index = dict_table_get_next_index(index)) { @@ -9957,8 +9933,6 @@ commit_try_rebuild( DBUG_RETURN(true); } - DBUG_EXECUTE_IF("ib_ddl_crash_before_rename", DBUG_SUICIDE();); - /* The new table must inherit the flag from the "parent" table. */ if (!user_table->space) { @@ -9966,31 +9940,27 @@ commit_try_rebuild( rebuilt_table->flags2 |= DICT_TF2_DISCARDED; } - dberr_t error = (ctx->old_table->flags2 & DICT_TF2_FTS) - ? fts_drop_tables(trx, ctx->old_table) - : DB_SUCCESS; + /* We can now rename the old table as a temporary table, + rename the new temporary table as the old table and drop the + old table. */ + char* old_name= mem_heap_strdup(ctx->heap, user_table->name.m_name); + dberr_t error = row_rename_table_for_mysql(user_table->name.m_name, + ctx->tmp_name, trx, false); if (error == DB_SUCCESS) { - /* We can now rename the old table as a temporary table, - rename the new temporary table as the old table and drop the - old table. */ - char* old_name= mem_heap_strdup(ctx->heap, - user_table->name.m_name); - - error = row_rename_table_for_mysql(user_table->name.m_name, - ctx->tmp_name, trx, - false, false); + error = row_rename_table_for_mysql( + rebuilt_table->name.m_name, old_name, trx, false); if (error == DB_SUCCESS) { - error = row_rename_table_for_mysql( - rebuilt_table->name.m_name, old_name, trx, - false, false); + error = trx->drop_table_statistics( + ctx->old_table->name); + if (error == DB_SUCCESS) { + error = trx->drop_table(*ctx->old_table); + } } } /* We must be still holding a table handle. */ DBUG_ASSERT(user_table->get_ref_count() == 1); - - DBUG_EXECUTE_IF("ib_ddl_crash_after_rename", DBUG_SUICIDE();); DBUG_EXECUTE_IF("ib_rebuild_cannot_rename", error = DB_ERROR;); switch (error) { @@ -10139,22 +10109,6 @@ innobase_page_compression_try( DBUG_RETURN(false); } -static -void -dict_stats_try_drop_table(THD *thd, const table_name_t &name, - const LEX_CSTRING &table_name) -{ - char errstr[1024]; - if (dict_stats_drop_table(name.m_name, errstr, sizeof(errstr)) != DB_SUCCESS) - { - push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, ER_ALTER_INFO, - "Deleting persistent statistics" - " for table '%s' in InnoDB failed: %s", - table_name.str, - errstr); - } -} - /** Evict the table from cache and reopen it. Drop outdated statistics. @param thd mariadb THD entity @param table innodb table @@ -10165,26 +10119,19 @@ static dict_table_t *innobase_reload_table(THD *thd, dict_table_t *table, const LEX_CSTRING &table_name, ha_innobase_inplace_ctx &ctx) { - char *tb_name= strdup(table->name.m_name); - dict_table_close(table, true, false); - if (ctx.is_instant()) { - for (auto i = ctx.old_n_v_cols; i--; ) + for (auto i= ctx.old_n_v_cols; i--; ) { ctx.old_v_cols[i].~dict_v_col_t(); - const_cast<unsigned&>(ctx.old_n_v_cols) = 0; + const_cast<unsigned&>(ctx.old_n_v_cols)= 0; } } + const table_id_t id= table->id; + table->release(); dict_sys.remove(table); - table= dict_table_open_on_name(tb_name, TRUE, TRUE, - DICT_ERR_IGNORE_FK_NOKEY); - - /* Drop outdated table stats. */ - dict_stats_try_drop_table(thd, table->name, table_name); - free(tb_name); - return table; + return dict_table_open_on_id(id, true, DICT_TABLE_OP_NORMAL); } /** Commit the changes made during prepare_inplace_alter_table() @@ -10259,7 +10206,7 @@ commit_try_norebuild( DBUG_RETURN(true); } - dberr_t error; + dberr_t error = DB_SUCCESS; dict_index_t* index; const char *op = "rename index to add"; ulint num_fts_index = 0; @@ -10284,6 +10231,12 @@ commit_try_norebuild( } } + char db[MAX_DB_UTF8_LEN], table[MAX_TABLE_UTF8_LEN]; + if (ctx->num_to_drop_index) { + dict_fs2utf8(ctx->old_table->name.m_name, + db, sizeof db, table, sizeof table); + } + for (ulint i = 0; i < ctx->num_to_drop_index; i++) { index = ctx->drop_index[i]; DBUG_ASSERT(index->is_committed()); @@ -10313,16 +10266,66 @@ commit_try_norebuild( if (error != DB_SUCCESS) { goto handle_error; } + + error = dict_stats_delete_from_index_stats(db, table, + index->name, trx); + switch (error) { + case DB_SUCCESS: + case DB_STATS_DO_NOT_EXIST: + continue; + default: + goto handle_error; + } + } + + if (const size_t size = ha_alter_info->rename_keys.size()) { + char tmp_name[5]; + char db[MAX_DB_UTF8_LEN], table[MAX_TABLE_UTF8_LEN]; + + dict_fs2utf8(ctx->new_table->name.m_name, db, sizeof db, + table, sizeof table); + + for (size_t i = 0; error == DB_SUCCESS && i < size; i++) { + snprintf(tmp_name, sizeof tmp_name, "\xff%zu", i); + error = dict_stats_rename_index(db, table, + ha_alter_info-> + rename_keys[i]. + old_key->name.str, + tmp_name, trx); + } + for (size_t i = 0; error == DB_SUCCESS && i < size; i++) { + snprintf(tmp_name, sizeof tmp_name, "\xff%zu", i); + error = dict_stats_rename_index(db, table, tmp_name, + ha_alter_info + ->rename_keys[i]. + new_key->name.str, + trx); + } + + switch (error) { + case DB_SUCCESS: + case DB_STATS_DO_NOT_EXIST: + break; + case DB_DUPLICATE_KEY: + my_error(ER_DUP_KEY, MYF(0), + "mysql.innodb_index_stats"); + DBUG_RETURN(true); + default: + goto handle_error; + } } if ((ctx->old_table->flags2 & DICT_TF2_FTS) && !num_fts_index) { - error = fts_drop_tables(trx, ctx->old_table); + error = fts_drop_tables(trx, *ctx->old_table); if (error != DB_SUCCESS) { handle_error: switch (error) { case DB_TOO_MANY_CONCURRENT_TRXS: my_error(ER_TOO_MANY_CONCURRENT_TRXS, MYF(0)); break; + case DB_LOCK_WAIT_TIMEOUT: + my_error(ER_LOCK_WAIT_TIMEOUT, MYF(0)); + break; default: sql_print_error("InnoDB: %s: %s\n", op, ut_strerr(error)); @@ -10598,8 +10601,6 @@ alter_stats_norebuild( ha_innobase_inplace_ctx* ctx, THD* thd) { - ulint i; - DBUG_ENTER("alter_stats_norebuild"); DBUG_ASSERT(!ctx->need_rebuild()); @@ -10607,98 +10608,7 @@ alter_stats_norebuild( DBUG_VOID_RETURN; } - /* Delete corresponding rows from the stats table. We do this - in a separate transaction from trx, because lock waits are not - allowed in a data dictionary transaction. (Lock waits are possible - on the statistics table, because it is directly accessible by users, - not covered by the dict_sys.latch.) - - Because the data dictionary changes were already committed, orphaned - rows may be left in the statistics table if the system crashes. - - FIXME: each change to the statistics tables is being committed in a - separate transaction, meaning that the operation is not atomic - - FIXME: This will not drop the (unused) statistics for - FTS_DOC_ID_INDEX if it was a hidden index, dropped together - with the last renamining FULLTEXT index. */ - for (i = 0; i < ha_alter_info->index_drop_count; i++) { - const KEY* key = ha_alter_info->index_drop_buffer[i]; - - if (key->flags & HA_FULLTEXT) { - /* There are no index cardinality - statistics for FULLTEXT indexes. */ - continue; - } - - char errstr[1024]; - - if (dict_stats_drop_index( - ctx->new_table->name.m_name, key->name.str, - errstr, sizeof errstr) != DB_SUCCESS) { - push_warning(thd, - Sql_condition::WARN_LEVEL_WARN, - ER_LOCK_WAIT_TIMEOUT, errstr); - } - } - - for (size_t i = 0; i < ha_alter_info->rename_keys.size(); i++) { - const Alter_inplace_info::Rename_key_pair& pair - = ha_alter_info->rename_keys[i]; - - std::stringstream ss; - ss << TEMP_FILE_PREFIX_INNODB << std::this_thread::get_id() - << i; - auto tmp_name = ss.str(); - - dberr_t err = dict_stats_rename_index(ctx->new_table, - pair.old_key->name.str, - tmp_name.c_str()); - - if (err != DB_SUCCESS) { - push_warning_printf( - thd, - Sql_condition::WARN_LEVEL_WARN, - ER_ERROR_ON_RENAME, - "Error renaming an index of table '%s'" - " from '%s' to '%s' in InnoDB persistent" - " statistics storage: %s", - ctx->new_table->name.m_name, - pair.old_key->name.str, - tmp_name.c_str(), - ut_strerr(err)); - } - } - - for (size_t i = 0; i < ha_alter_info->rename_keys.size(); i++) { - const Alter_inplace_info::Rename_key_pair& pair - = ha_alter_info->rename_keys[i]; - - std::stringstream ss; - ss << TEMP_FILE_PREFIX_INNODB << std::this_thread::get_id() - << i; - auto tmp_name = ss.str(); - - dberr_t err = dict_stats_rename_index(ctx->new_table, - tmp_name.c_str(), - pair.new_key->name.str); - - if (err != DB_SUCCESS) { - push_warning_printf( - thd, - Sql_condition::WARN_LEVEL_WARN, - ER_ERROR_ON_RENAME, - "Error renaming an index of table '%s'" - " from '%s' to '%s' in InnoDB persistent" - " statistics storage: %s", - ctx->new_table->name.m_name, - tmp_name.c_str(), - pair.new_key->name.str, - ut_strerr(err)); - } - } - - for (i = 0; i < ctx->num_to_add_index; i++) { + for (ulint i = 0; i < ctx->num_to_add_index; i++) { dict_index_t* index = ctx->add_index[i]; DBUG_ASSERT(index->table == ctx->new_table); @@ -10748,17 +10658,6 @@ alter_stats_rebuild( DBUG_VOID_RETURN; } -#ifndef DBUG_OFF -# define DBUG_INJECT_CRASH(prefix, count) \ -do { \ - char buf[32]; \ - snprintf(buf, sizeof buf, prefix "_%u", count); \ - DBUG_EXECUTE_IF(buf, DBUG_SUICIDE();); \ -} while (0) -#else -# define DBUG_INJECT_CRASH(prefix, count) -#endif - /** Apply the log for the table rebuild operation. @param[in] ctx Inplace Alter table context @param[in] altered_table MySQL table that is being altered @@ -10872,8 +10771,6 @@ ha_innobase::commit_inplace_alter_table( (ha_alter_info->handler_ctx); #ifndef DBUG_OFF - uint crash_inject_count = 1; - uint crash_fail_inject_count = 1; uint failure_inject_count = 1; #endif /* DBUG_OFF */ @@ -10892,12 +10789,10 @@ ha_innobase::commit_inplace_alter_table( if (!commit) { /* A rollback is being requested. So far we may at - most have created some indexes. If any indexes were to - be dropped, they would actually be dropped in this - method if commit=true. */ - const bool ret = rollback_inplace_alter_table( - ha_alter_info, table, m_prebuilt); - DBUG_RETURN(ret); + most have created stubs for ADD INDEX or a copy of the + table for rebuild. */ + DBUG_RETURN(rollback_inplace_alter_table( + ha_alter_info, table, m_prebuilt)); } if (!(ha_alter_info->handler_flags & ~INNOBASE_INPLACE_IGNORE)) { @@ -10924,15 +10819,15 @@ ha_innobase::commit_inplace_alter_table( ut_ad(m_prebuilt->table == ctx0->old_table); ha_alter_info->group_commit_ctx = NULL; - trx_start_if_not_started_xa(m_prebuilt->trx, true); const bool new_clustered = ctx0->need_rebuild(); - + trx_t* const trx = ctx0->trx; + trx->op_info = "acquiring table lock"; + bool fts_exist = false; for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; pctx++) { - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*>(*pctx); + auto ctx = static_cast<ha_innobase_inplace_ctx*>(*pctx); DBUG_ASSERT(ctx->prebuilt->trx == m_prebuilt->trx); + ut_ad(m_prebuilt != ctx->prebuilt || ctx == ctx0); DBUG_ASSERT(new_clustered == ctx->need_rebuild()); - /* If decryption failed for old table or new table fail here. */ if ((!ctx->old_table->is_readable() @@ -10945,72 +10840,79 @@ ha_innobase::commit_inplace_alter_table( my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_DECRYPTION_FAILED, str.c_ptr(), engine); DBUG_RETURN(true); } + if ((ctx->old_table->flags2 | ctx->new_table->flags2) + & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS)) { + fts_exist = true; + } + } - if (new_clustered) { - continue; + if (fts_exist) { + purge_sys.stop_FTS(); + } + + for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; pctx++) { + auto ctx = static_cast<ha_innobase_inplace_ctx*>(*pctx); + + if (new_clustered && ctx->old_table->fts) { + ut_ad(!ctx->old_table->fts->add_wq); + fts_optimize_remove_table(ctx->old_table); + } + + if (ctx->new_table->fts) { + ut_ad(!ctx->new_table->fts->add_wq); + fts_optimize_remove_table(ctx->new_table); + fts_sync_during_ddl(ctx->new_table); } /* Exclusively lock the table, to ensure that no other transaction is holding locks on the table while we change the table definition. Any recovered incomplete transactions would be holding InnoDB locks only, not MDL. */ - ctx->prebuilt->trx->op_info = "acquiring table lock"; - if (dberr_t error = lock_table_for_trx(ctx->new_table, - ctx->prebuilt->trx, + if (dberr_t error = lock_table_for_trx(ctx->new_table, trx, LOCK_X)) { +lock_fail: my_error_innodb( error, table_share->table_name.str, 0); DBUG_RETURN(true); + } else if ((ctx->new_table->flags2 + & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS)) + && (error = fts_lock_tables(trx, *ctx->new_table)) + != DB_SUCCESS) { + goto lock_fail; + } else if (!new_clustered) { + } else if ((error = lock_table_for_trx(ctx->old_table, trx, + LOCK_X)) + != DB_SUCCESS) { + goto lock_fail; + } else if ((ctx->old_table->flags2 + & (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS)) + && (error = fts_lock_tables(trx, *ctx->old_table)) + != DB_SUCCESS) { + goto lock_fail; } } DEBUG_SYNC(m_user_thd, "innodb_alter_commit_after_lock_table"); - trx_t* trx = ctx0->trx; - bool fail = false; - - /* Stop background FTS operations. */ - for (inplace_alter_handler_ctx** pctx = ctx_array; - *pctx; pctx++) { - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*>(*pctx); - - DBUG_ASSERT(new_clustered == ctx->need_rebuild()); - - if (new_clustered) { - if (ctx->old_table->fts) { - ut_ad(!ctx->old_table->fts->add_wq); - fts_optimize_remove_table(ctx->old_table); + if (new_clustered) { + /* We are holding MDL_EXCLUSIVE as well as exclusive + InnoDB table locks. Let us apply any table rebuild log + before locking dict_sys. */ + for (inplace_alter_handler_ctx** pctx= ctx_array; *pctx; + pctx++) { + auto ctx= static_cast<ha_innobase_inplace_ctx*>(*pctx); + DBUG_ASSERT(ctx->need_rebuild()); + if (alter_rebuild_apply_log(ctx, ha_alter_info, + altered_table)) { + DBUG_RETURN(true); } } - - if (ctx->new_table->fts) { - ut_ad(!ctx->new_table->fts->add_wq); - fts_optimize_remove_table(ctx->new_table); - fts_sync_during_ddl(ctx->new_table); - } - - /* Apply the online log of the table before acquiring - data dictionary latches. Here alter thread already acquired - MDL_EXCLUSIVE on the table. So there can't be anymore DDLs, DMLs - for the altered table. By applying the log here, InnoDB - makes sure that concurrent DDLs, purge thread or any other - background thread doesn't wait for the dict_operation_lock - for longer time. */ - if (new_clustered && commit - && alter_rebuild_apply_log( - ctx, ha_alter_info, altered_table)) { - DBUG_RETURN(true); - } } /* Latch the InnoDB data dictionary exclusively so that no deadlocks or lock waits can happen in it during the data dictionary operation. */ row_mysql_lock_data_dictionary(trx); - if (trx->state != TRX_STATE_ACTIVE) { - trx_start_for_ddl(trx); - } /* Prevent the background statistics collection from accessing the tables. */ @@ -11043,38 +10945,45 @@ ha_innobase::commit_inplace_alter_table( /* Apply the changes to the data dictionary tables, for all partitions. */ - - for (inplace_alter_handler_ctx** pctx = ctx_array; - *pctx && !fail; pctx++) { - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*>(*pctx); + for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; pctx++) { + auto ctx = static_cast<ha_innobase_inplace_ctx*>(*pctx); DBUG_ASSERT(new_clustered == ctx->need_rebuild()); if (ctx->need_rebuild() && !ctx->old_table->space) { my_error(ER_TABLESPACE_DISCARDED, MYF(0), table->s->table_name.str); - fail = true; - } else { - fail = commit_set_autoinc(ha_alter_info, ctx, - altered_table, table); +fail: + trx->rollback(); + ut_ad(!trx->fts_trx); + row_mysql_unlock_data_dictionary(trx); + if (fts_exist) { + purge_sys.resume_FTS(); + } + trx_start_for_ddl(trx); + DBUG_RETURN(true); + } + + if (commit_set_autoinc(ha_alter_info, ctx, + altered_table, table)) { + goto fail; } - if (fail) { - } else if (ctx->need_rebuild()) { + if (ctx->need_rebuild()) { ctx->tmp_name = dict_mem_create_temporary_tablename( ctx->heap, ctx->new_table->name.m_name, ctx->new_table->id); - fail = commit_try_rebuild( - ha_alter_info, ctx, altered_table, table, - trx, table_share->table_name.str); - } else { - fail = commit_try_norebuild( - ha_alter_info, ctx, altered_table, table, trx, - table_share->table_name.str); + if (commit_try_rebuild(ha_alter_info, ctx, + altered_table, table, + trx, + table_share->table_name.str)) { + goto fail; + } + } else if (commit_try_norebuild(ha_alter_info, ctx, + altered_table, table, trx, + table_share->table_name.str)) { + goto fail; } - DBUG_INJECT_CRASH("ib_commit_inplace_crash", - crash_inject_count++); #ifndef DBUG_OFF { /* Generate a dynamic dbug text. */ @@ -11087,7 +10996,7 @@ ha_innobase::commit_inplace_alter_table( DBUG_EXECUTE_IF(buf, my_error(ER_INTERNAL_ERROR, MYF(0), "Injected error!"); - fail = true; + goto fail; ); } #endif @@ -11096,52 +11005,56 @@ ha_innobase::commit_inplace_alter_table( /* Commit or roll back the changes to the data dictionary. */ DEBUG_SYNC(m_user_thd, "innodb_alter_inplace_before_commit"); - if (fail) { - trx_rollback_for_mysql(trx); - for (inplace_alter_handler_ctx** pctx = ctx_array; - *pctx; pctx++) { - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*>(*pctx); - ctx->rollback_instant(); + if (new_clustered) { + ut_ad(trx->has_logged()); + for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; + pctx++) { + auto ctx= static_cast<ha_innobase_inplace_ctx*>(*pctx); + ut_ad(!strcmp(ctx->old_table->name.m_name, + ctx->tmp_name)); + ut_ad(ctx->new_table->get_ref_count() == 1); + const bool own = m_prebuilt == ctx->prebuilt; + trx_t* const user_trx = m_prebuilt->trx; + row_prebuilt_free(ctx->prebuilt, true); + /* Rebuild the prebuilt object. */ + ctx->prebuilt = row_create_prebuilt( + ctx->new_table, altered_table->s->reclength); + if (own) { + m_prebuilt = ctx->prebuilt; + } + trx_start_if_not_started(user_trx, true); + m_prebuilt->trx = user_trx; } - } else { - /* Test what happens on crash if the redo logs - are flushed to disk here. The log records - about the rename should not be committed, and - the data dictionary transaction should be - rolled back, restoring the old table. */ - DBUG_EXECUTE_IF("innodb_alter_commit_crash_before_commit", - log_buffer_flush_to_disk(); - DBUG_SUICIDE();); - ut_ad(!trx->fts_trx); - - ut_ad(trx_state_eq(trx, TRX_STATE_ACTIVE)); - ut_ad(!new_clustered || trx->has_logged()); - /* The SQL layer recovery of ALTER TABLE will invoke - innodb_check_version() to know whether our trx->id, which we - reported via ha_innobase::table_version() after - ha_innobase::prepare_inplace_alter_table(), was committed. - - If this trx was committed (the log write below completed), - we will be able to recover our trx->id to - dict_table_t::def_trx_id from the data dictionary tables. - - For this logic to work, purge_sys.stop_SYS() and - purge_sys.resume_SYS() will ensure that the DB_TRX_ID that we - wrote to the SYS_ tables will be preserved until the SQL layer - has durably marked the ALTER TABLE operation as completed. - - During recovery, the purge of InnoDB transaction history will - not start until innodb_ddl_recovery_done(). */ - ha_alter_info->inplace_alter_table_committed = - purge_sys.resume_SYS; - purge_sys.stop_SYS(); - trx->commit(); - log_write_up_to(trx->commit_lsn, true); - DBUG_EXECUTE_IF("innodb_alter_commit_crash_after_commit", - DBUG_SUICIDE();); } + ut_ad(!trx->fts_trx); + + std::vector<pfs_os_file_t> deleted; + DBUG_EXECUTE_IF("innodb_alter_commit_crash_before_commit", + log_buffer_flush_to_disk(); DBUG_SUICIDE();); + /* The SQL layer recovery of ALTER TABLE will invoke + innodb_check_version() to know whether our trx->id, which we + reported via ha_innobase::table_version() after + ha_innobase::prepare_inplace_alter_table(), was committed. + + If this trx was committed (the log write below completed), + we will be able to recover our trx->id to + dict_table_t::def_trx_id from the data dictionary tables. + + For this logic to work, purge_sys.stop_SYS() and + purge_sys.resume_SYS() will ensure that the DB_TRX_ID that we + wrote to the SYS_ tables will be preserved until the SQL layer + has durably marked the ALTER TABLE operation as completed. + + During recovery, the purge of InnoDB transaction history will + not start until innodb_ddl_recovery_done(). */ + ha_alter_info->inplace_alter_table_committed = purge_sys.resume_SYS; + purge_sys.stop_SYS(); + trx->commit(deleted); + log_write_up_to(trx->commit_lsn, true); + DBUG_EXECUTE_IF("innodb_alter_commit_crash_after_commit", + DBUG_SUICIDE();); + /* At this point, the changes to the persistent storage have been committed or rolled back. What remains to be done is to update the in-memory structures, close some handles, release @@ -11154,55 +11067,10 @@ ha_innobase::commit_inplace_alter_table( DBUG_ASSERT(ctx->need_rebuild() == new_clustered); - if (new_clustered) { - innobase_online_rebuild_log_free(ctx->old_table); - } - - if (fail) { - if (new_clustered) { - trx_start_for_ddl(trx); - - ut_d(const bool last_handle=) - ctx->new_table->release(); - ut_ad(last_handle); - row_drop_table_for_mysql( - ctx->new_table->name.m_name, - trx, SQLCOM_DROP_TABLE, false, false); - - trx_commit_for_mysql(trx); - ctx->new_table = NULL; - } else { - /* We failed, but did not rebuild the table. - Roll back any ADD INDEX, or get rid of garbage - ADD INDEX that was left over from a previous - ALTER TABLE statement. */ - trx_start_for_ddl(trx); - innobase_rollback_sec_index( - ctx->new_table, table, TRUE, trx); - trx_commit_for_mysql(trx); - } - DBUG_INJECT_CRASH("ib_commit_inplace_crash_fail", - crash_fail_inject_count++); - - continue; - } - innobase_copy_frm_flags_from_table_share( ctx->new_table, altered_table->s); if (new_clustered) { - /* We will reload and refresh the - in-memory foreign key constraint - metadata. This is a rename operation - in preparing for dropping the old - table. Set the table to_be_dropped bit - here, so to make sure DML foreign key - constraint check does not use the - stale dict_foreign_t. This is done - because WL#6049 (FK MDL) has not been - implemented yet. */ - ctx->old_table->to_be_dropped = true; - DBUG_PRINT("to_be_dropped", ("table: %s", ctx->old_table->name.m_name)); @@ -11234,42 +11102,10 @@ foreign_fail: dict_mem_table_free_foreign_vcol_set(ctx->new_table); dict_mem_table_fill_foreign_vcol_set(ctx->new_table); - - DBUG_INJECT_CRASH("ib_commit_inplace_crash", - crash_inject_count++); } - if (fail) { - for (inplace_alter_handler_ctx** pctx = ctx_array; - *pctx; pctx++) { - ha_innobase_inplace_ctx* ctx - = static_cast<ha_innobase_inplace_ctx*> - (*pctx); - DBUG_ASSERT(ctx->need_rebuild() == new_clustered); - - ut_d(dict_table_check_for_dup_indexes( - ctx->old_table, - CHECK_ABORTED_OK)); - ut_a(fts_check_cached_index(ctx->old_table)); - DBUG_INJECT_CRASH("ib_commit_inplace_crash_fail", - crash_fail_inject_count++); - - /* Restart the FTS background operations. */ - if (ctx->old_table->fts) { - fts_optimize_add_table(ctx->old_table); - } - } - - row_mysql_unlock_data_dictionary(trx); - if (trx != ctx0->trx) { - trx->free(); - } - DBUG_RETURN(true); - } - - if (trx == ctx0->trx) { - ctx0->trx = NULL; - } + ut_ad(trx == ctx0->trx); + ctx0->trx = nullptr; /* Free the ctx->trx of other partitions, if any. We will only use the ctx0->trx here. Others may have been allocated in @@ -11299,7 +11135,6 @@ foreign_fail: && ha_alter_info->handler_flags & ALTER_STORED_COLUMN_ORDER)) { DBUG_ASSERT(ctx0->old_table->get_ref_count() == 1); ut_ad(ctx0->prebuilt == m_prebuilt); - trx_commit_for_mysql(m_prebuilt->trx); for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; pctx++) { @@ -11313,15 +11148,14 @@ foreign_fail: row_mysql_unlock_data_dictionary(trx); trx->free(); + close_unlinked_files(deleted); + if (fts_exist) { + purge_sys.resume_FTS(); + } MONITOR_ATOMIC_DEC(MONITOR_PENDING_ALTER_TABLE); DBUG_RETURN(false); } - /* Release the table locks. */ - trx_commit_for_mysql(m_prebuilt->trx); - - DBUG_EXECUTE_IF("ib_ddl_crash_after_user_trx_commit", DBUG_SUICIDE();); - for (inplace_alter_handler_ctx** pctx = ctx_array; *pctx; pctx++) { ha_innobase_inplace_ctx* ctx @@ -11363,63 +11197,14 @@ foreign_fail: ut_a(fts_check_cached_index(ctx->new_table)); } #endif - if (new_clustered) { - /* Since the table has been rebuilt, we remove - all persistent statistics corresponding to the - old copy of the table (which was renamed to - ctx->tmp_name). */ - - DBUG_ASSERT(0 == strcmp(ctx->old_table->name.m_name, - ctx->tmp_name)); - - dict_stats_try_drop_table(m_user_thd, - ctx->new_table->name, - table->s->table_name); - - DBUG_EXECUTE_IF("ib_ddl_crash_before_commit", - DBUG_SUICIDE();); - - ut_ad(m_prebuilt != ctx->prebuilt - || ctx == ctx0); - bool update_own_prebuilt = - (m_prebuilt == ctx->prebuilt); - trx_t* const user_trx = m_prebuilt->trx; - - row_prebuilt_free(ctx->prebuilt, TRUE); - - /* Drop the copy of the old table, which was - renamed to ctx->tmp_name at the atomic DDL - transaction commit. If the system crashes - before this is completed, some orphan tables - with ctx->tmp_name may be recovered. */ - trx_start_for_ddl(trx); - if (dberr_t error = row_drop_table_for_mysql( - ctx->old_table->name.m_name, - trx, SQLCOM_DROP_TABLE, false, false)) { - ib::error() << "Inplace alter table " << ctx->old_table->name - << " dropping copy of the old table failed error " - << error - << ". tmp_name " << (ctx->tmp_name ? ctx->tmp_name : "N/A") - << " new_table " << ctx->new_table->name; - } - - trx_commit_for_mysql(trx); - - /* Rebuild the prebuilt object. */ - ctx->prebuilt = row_create_prebuilt( - ctx->new_table, altered_table->s->reclength); - if (update_own_prebuilt) { - m_prebuilt = ctx->prebuilt; - } - trx_start_if_not_started(user_trx, true); - m_prebuilt->trx = user_trx; - } - DBUG_INJECT_CRASH("ib_commit_inplace_crash", - crash_inject_count++); } row_mysql_unlock_data_dictionary(trx); trx->free(); + close_unlinked_files(deleted); + if (fts_exist) { + purge_sys.resume_FTS(); + } /* TODO: The following code could be executed while allowing concurrent access to the table @@ -11436,8 +11221,6 @@ foreign_fail: alter_stats_rebuild( ctx->new_table, table->s->table_name.str, m_user_thd); - DBUG_INJECT_CRASH("ib_commit_inplace_crash", - crash_inject_count++); } } else { for (inplace_alter_handler_ctx** pctx = ctx_array; @@ -11448,8 +11231,6 @@ foreign_fail: DBUG_ASSERT(!ctx->need_rebuild()); alter_stats_norebuild(ha_alter_info, ctx, m_user_thd); - DBUG_INJECT_CRASH("ib_commit_inplace_crash", - crash_inject_count++); } } diff --git a/storage/innobase/include/btr0btr.h b/storage/innobase/include/btr0btr.h index 28cb80ba65c..1c5533ecd59 100644 --- a/storage/innobase/include/btr0btr.h +++ b/storage/innobase/include/btr0btr.h @@ -337,9 +337,9 @@ btr_create( void btr_free_if_exists(fil_space_t *space, uint32_t page, index_id_t index_id, mtr_t *mtr); -/** Free an index tree in a temporary tablespace. -@param[in] page_id root page id */ -void btr_free(const page_id_t page_id); +/** Drop a temporary table +@param table temporary table */ +void btr_drop_temporary_table(const dict_table_t &table); /** Read the last used AUTO_INCREMENT value from PAGE_ROOT_AUTO_INC. @param[in,out] index clustered index diff --git a/storage/innobase/include/btr0sea.h b/storage/innobase/include/btr0sea.h index 2f27395155f..da7bbac1f83 100644 --- a/storage/innobase/include/btr0sea.h +++ b/storage/innobase/include/btr0sea.h @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2017, 2020, MariaDB Corporation. +Copyright (c) 2017, 2021, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -128,7 +128,7 @@ void btr_search_update_hash_on_insert(btr_cur_t *cursor, /** Updates the page hash index when a single record is deleted from a page. @param[in] cursor cursor which was positioned on the record to delete using btr_cur_search_, the record is not yet deleted.*/ -void btr_search_update_hash_on_delete(btr_cur_t* cursor); +void btr_search_update_hash_on_delete(btr_cur_t *cursor); /** Validates the search system. @return true if ok */ diff --git a/storage/innobase/include/btr0sea.ic b/storage/innobase/include/btr0sea.ic index 0f972528ce2..5a8d648029a 100644 --- a/storage/innobase/include/btr0sea.ic +++ b/storage/innobase/include/btr0sea.ic @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 1996, 2015, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2018, 2020, MariaDB Corporation. +Copyright (c) 2018, 2021, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -47,8 +47,7 @@ static inline btr_search_t* btr_search_info_create(mem_heap_t* heap) /** Updates the search info. @param[in,out] info search info @param[in,out] cursor cursor which was just positioned */ -void -btr_search_info_update_slow(btr_search_t* info, btr_cur_t* cursor); +void btr_search_info_update_slow(btr_search_t *info, btr_cur_t *cursor); /*********************************************************************//** Updates the search info. */ @@ -59,7 +58,10 @@ btr_search_info_update( dict_index_t* index, /*!< in: index of the cursor */ btr_cur_t* cursor) /*!< in: cursor which was just positioned */ { - if (dict_index_is_spatial(index) || !btr_search_enabled) { + ut_ad(!index->is_spatial()); + ut_ad(!index->table->is_temporary()); + + if (!btr_search_enabled) { return; } diff --git a/storage/innobase/include/db0err.h b/storage/innobase/include/db0err.h index 98d02e3a767..bd5eb0688cb 100644 --- a/storage/innobase/include/db0err.h +++ b/storage/innobase/include/db0err.h @@ -118,8 +118,6 @@ enum dberr_t { DB_READ_ONLY, /*!< Update operation attempted in a read-only transaction */ DB_FTS_INVALID_DOCID, /* FTS Doc ID cannot be zero */ - DB_TABLE_IN_FK_CHECK, /* table is being used in foreign - key check */ DB_ONLINE_LOG_TOO_BIG, /*!< Modification log grew too big during online index creation */ diff --git a/storage/innobase/include/dict0mem.h b/storage/innobase/include/dict0mem.h index 867826f92d6..5424fce9ba3 100644 --- a/storage/innobase/include/dict0mem.h +++ b/storage/innobase/include/dict0mem.h @@ -1000,15 +1000,6 @@ struct dict_index_t { representation we add more columns */ unsigned nulls_equal:1; /*!< if true, SQL NULL == SQL NULL */ -#ifdef BTR_CUR_HASH_ADAPT -#ifdef MYSQL_INDEX_DISABLE_AHI - unsigned disable_ahi:1; - /*!< whether to disable the - adaptive hash index. - Maybe this could be disabled for - temporary tables? */ -#endif -#endif /* BTR_CUR_HASH_ADAPT */ unsigned n_uniq:10;/*!< number of fields from the beginning which are enough to determine an index entry uniquely */ @@ -1958,23 +1949,6 @@ struct dict_table_t { return versioned() && cols[vers_start].mtype == DATA_INT; } - void inc_fk_checks() - { -#ifdef UNIV_DEBUG - int32_t fk_checks= -#endif - n_foreign_key_checks_running++; - ut_ad(fk_checks >= 0); - } - void dec_fk_checks() - { -#ifdef UNIV_DEBUG - int32_t fk_checks= -#endif - n_foreign_key_checks_running--; - ut_ad(fk_checks > 0); - } - /** For overflow fields returns potential max length stored inline */ inline size_t get_overflow_field_local_len() const; @@ -2049,7 +2023,7 @@ public: table_id_t id; /** dict_sys.id_hash chain node */ dict_table_t* id_hash; - /** Table name. */ + /** Table name in name_hash */ table_name_t name; /** dict_sys.name_hash chain node */ dict_table_t* name_hash; @@ -2100,12 +2074,6 @@ public: /** TRUE if the table object has been added to the dictionary cache. */ unsigned cached:1; - /** TRUE if the table is to be dropped, but not yet actually dropped - (could in the background drop list). It is turned on at the beginning - of row_drop_table_for_mysql() and turned off just before we start to - update system tables for the drop. It is protected by dict_sys.latch. */ - unsigned to_be_dropped:1; - /** Number of non-virtual columns defined so far. */ unsigned n_def:10; @@ -2202,11 +2170,6 @@ public: loading child table into memory along with its parent table. */ byte fk_max_recusive_level; - /** Count of how many foreign key check operations are currently being - performed on the table. We cannot drop the table while there are - foreign key checks running on it. */ - Atomic_counter<int32_t> n_foreign_key_checks_running; - /** DDL transaction that last touched the table definition, or 0 if no history is available. This includes possible changes in ha_innobase::prepare_inplace_alter_table() and @@ -2219,6 +2182,11 @@ public: an empty leaf page), and an ahi_latch (if btr_search_enabled). */ Atomic_relaxed<trx_id_t> bulk_trx_id; + /** Original table name, for MDL acquisition in purge. Normally, + this points to the same as name. When is_temporary_name(name.m_name) holds, + this should be a copy of the original table name, allocated from heap. */ + table_name_t mdl_name; + /*!< set of foreign key constraints in the table; these refer to columns in other tables */ dict_foreign_set foreign_set; @@ -2439,9 +2407,6 @@ public: static dict_table_t *create(const span<const char> &name, fil_space_t *space, ulint n_cols, ulint n_v_cols, ulint flags, ulint flags2); - - /** @return whether SYS_TABLES.NAME is for a '#sql-ib' table */ - static bool is_garbage_name(const void *data, size_t size); }; inline void dict_index_t::set_modified(mtr_t& mtr) const diff --git a/storage/innobase/include/dict0mem.ic b/storage/innobase/include/dict0mem.ic index 0a554a54dbd..d60ee5d9bf4 100644 --- a/storage/innobase/include/dict0mem.ic +++ b/storage/innobase/include/dict0mem.ic @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 1996, 2015, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2017, MariaDB Corporation. +Copyright (c) 2017, 2021, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -64,10 +64,5 @@ dict_mem_fill_index_struct( /* The '1 +' above prevents allocation of an empty mem block */ index->nulls_equal = false; -#ifdef BTR_CUR_HASH_ADAPT -#ifdef MYSQL_INDEX_DISABLE_AHI - index->disable_ahi = false; -#endif -#endif /* BTR_CUR_HASH_ADAPT */ ut_d(index->magic_n = DICT_INDEX_MAGIC_N); } diff --git a/storage/innobase/include/dict0stats.h b/storage/innobase/include/dict0stats.h index cf0e2adab76..7112238c9b6 100644 --- a/storage/innobase/include/dict0stats.h +++ b/storage/innobase/include/dict0stats.h @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 2009, 2018, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2017, 2020, MariaDB Corporation. +Copyright (c) 2017, 2021, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -30,9 +30,6 @@ Created Jan 06, 2010 Vasil Dimov #include "dict0types.h" #include "trx0types.h" -#define TABLE_STATS_NAME "mysql/innodb_table_stats" -#define INDEX_STATS_NAME "mysql/innodb_index_stats" - enum dict_stats_upd_option_t { DICT_STATS_RECALC_PERSISTENT,/* (re) calculate the statistics using a precise and slow @@ -140,40 +137,31 @@ dict_stats_update( the stats or to fetch them from the persistent storage */ -/** Remove the information for a particular index's stats from the persistent -storage if it exists and if there is data stored for this index. -This function creates its own trx and commits it. - -We must modify system tables in a separate transaction in order to -adhere to the InnoDB design constraint that dict_sys.latch prevents -lock waits on system tables. If we modified system and user tables in -the same transaction, we should exclusively hold dict_sys.latch until -the transaction is committed, and effectively block other transactions -that will attempt to open any InnoDB tables. Because we have no -guarantee that user transactions will be committed fast, we cannot -afford to keep the system tables locked in a user transaction. +/** Execute DELETE FROM mysql.innodb_table_stats +@param database_name database name +@param table_name table name +@param trx transaction (nullptr=start and commit a new one) @return DB_SUCCESS or error code */ -dberr_t -dict_stats_drop_index( -/*==================*/ - const char* tname, /*!< in: table name */ - const char* iname, /*!< in: index name */ - char* errstr, /*!< out: error message if != DB_SUCCESS - is returned */ - ulint errstr_sz);/*!< in: size of the errstr buffer */ - -/*********************************************************************//** -Removes the statistics for a table and all of its indexes from the -persistent storage if it exists and if there is data stored for the table. -This function creates its own transaction and commits it. +dberr_t dict_stats_delete_from_table_stats(const char *database_name, + const char *table_name, + trx_t *trx= nullptr); +/** Execute DELETE FROM mysql.innodb_index_stats +@param database_name database name +@param table_name table name +@param trx transaction (nullptr=start and commit a new one) @return DB_SUCCESS or error code */ -dberr_t -dict_stats_drop_table( -/*==================*/ - const char* table_name, /*!< in: table name */ - char* errstr, /*!< out: error message - if != DB_SUCCESS is returned */ - ulint errstr_sz); /*!< in: size of errstr buffer */ +dberr_t dict_stats_delete_from_index_stats(const char *database_name, + const char *table_name, + trx_t *trx= nullptr); +/** Execute DELETE FROM mysql.innodb_index_stats +@param database_name database name +@param table_name table name +@param index_name name of the index +@param trx transaction (nullptr=start and commit a new one) +@return DB_SUCCESS or error code */ +dberr_t dict_stats_delete_from_index_stats(const char *database_name, + const char *table_name, + const char *index_name, trx_t *trx); /*********************************************************************//** Fetches or calculates new estimates for index statistics. */ @@ -183,31 +171,29 @@ dict_stats_update_for_index( dict_index_t* index) /*!< in/out: index */ MY_ATTRIBUTE((nonnull)); -/*********************************************************************//** -Renames a table in InnoDB persistent stats storage. -This function creates its own transaction and commits it. +/** Rename a table in InnoDB persistent stats storage. +@param old_name old table name +@param new_name new table name +@param trx transaction @return DB_SUCCESS or error code */ -dberr_t -dict_stats_rename_table( -/*====================*/ - const char* old_name, /*!< in: old table name */ - const char* new_name, /*!< in: new table name */ - char* errstr, /*!< out: error string if != DB_SUCCESS - is returned */ - size_t errstr_sz); /*!< in: errstr size */ -/*********************************************************************//** -Renames an index in InnoDB persistent stats storage. -This function creates its own transaction and commits it. -@return DB_SUCCESS or error code. DB_STATS_DO_NOT_EXIST will be returned -if the persistent stats do not exist. */ -dberr_t -dict_stats_rename_index( -/*====================*/ - const dict_table_t* table, /*!< in: table whose index - is renamed */ - const char* old_index_name, /*!< in: old index name */ - const char* new_index_name) /*!< in: new index name */ - __attribute__((warn_unused_result)); +dberr_t dict_stats_rename_table(const char *old_name, const char *new_name, + trx_t *trx); +/** Rename an index in InnoDB persistent statistics. +@param db database name +@param table table name +@param old_name old table name +@param new_name new table name +@param trx transaction +@return DB_SUCCESS or error code */ +dberr_t dict_stats_rename_index(const char *db, const char *table, + const char *old_name, const char *new_name, + trx_t *trx); + +/** Delete all persistent statistics for a database. +@param db database name +@param trx transaction +@return DB_SUCCESS or error code */ +dberr_t dict_stats_delete(const char *db, trx_t *trx); /** Save an individual index's statistic into the persistent statistics storage. diff --git a/storage/innobase/include/dict0types.h b/storage/innobase/include/dict0types.h index cfd0ff98912..9a53ecdab8f 100644 --- a/storage/innobase/include/dict0types.h +++ b/storage/innobase/include/dict0types.h @@ -168,4 +168,7 @@ enum spatial_status_t { SPATIAL_ONLY = 3 }; +#define TABLE_STATS_NAME "mysql/innodb_table_stats" +#define INDEX_STATS_NAME "mysql/innodb_index_stats" + #endif diff --git a/storage/innobase/include/fil0fil.h b/storage/innobase/include/fil0fil.h index cf770ccce37..dfee49a500c 100644 --- a/storage/innobase/include/fil0fil.h +++ b/storage/innobase/include/fil0fil.h @@ -518,8 +518,12 @@ public: /** Close each file. Only invoked on fil_system.temp_space. */ void close(); - /** Note that operations on the tablespace must stop or can resume */ - inline void set_stopping(bool stopping); + /** Note that operations on the tablespace must stop. + @return whether the operations were already stopped */ + inline bool set_stopping(); + + /** Note that operations on the tablespace can resume after truncation */ + inline void clear_stopping(); /** Look up the tablespace and wait for pending operations to cease @param id tablespace identifier @@ -1508,12 +1512,19 @@ inline void fil_space_t::reacquire() #endif /* SAFE_MUTEX */ } -/** Note that operations on the tablespace must stop or can resume */ -inline void fil_space_t::set_stopping(bool stopping) +/** Note that operations on the tablespace must stop. +@return whether the operations were already stopped */ +inline bool fil_space_t::set_stopping() +{ + mysql_mutex_assert_owner(&fil_system.mutex); + return n_pending.fetch_or(STOPPING, std::memory_order_relaxed) & STOPPING; +} + +inline void fil_space_t::clear_stopping() { mysql_mutex_assert_owner(&fil_system.mutex); - ut_d(auto n=) n_pending.fetch_xor(STOPPING, std::memory_order_relaxed); - ut_ad(!(n & STOPPING) == stopping); + ut_d(auto n=) n_pending.fetch_and(~STOPPING, std::memory_order_relaxed); + ut_ad(n & STOPPING); } /** Flush pending writes from the file system cache to the file. */ @@ -1595,13 +1606,12 @@ fil_write_flushed_lsn( lsn_t lsn) MY_ATTRIBUTE((warn_unused_result)); +MY_ATTRIBUTE((warn_unused_result)) /** Delete a tablespace and associated .ibd file. -@param[in] id tablespace identifier -@param[in] if_exists whether to ignore missing tablespace -@param[out] detached deatched file handle (if closing is not wanted) -@return DB_SUCCESS or error */ -dberr_t fil_delete_tablespace(ulint id, bool if_exists= false, - pfs_os_file_t *detached= nullptr); +@param id tablespace identifier +@return detached file handle (to be closed by the caller) +@return OS_FILE_CLOSED if no file existed */ +pfs_os_file_t fil_delete_tablespace(ulint id); /** Close a single-table tablespace on failed IMPORT TABLESPACE. The tablespace must be cached in the memory cache. diff --git a/storage/innobase/include/fts0fts.h b/storage/innobase/include/fts0fts.h index e360e3227b1..bb61eae43f2 100644 --- a/storage/innobase/include/fts0fts.h +++ b/storage/innobase/include/fts0fts.h @@ -500,17 +500,29 @@ fts_add_doc_id_column( dict_table_t* table, /*!< in/out: Table with FTS index */ mem_heap_t* heap); /*!< in: temporary memory heap, or NULL */ -/*********************************************************************//** -Drops the ancillary tables needed for supporting an FTS index on the -given table. row_mysql_lock_data_dictionary must have been called before -this. +/** Lock the internal FTS_ tables for an index, before fts_drop_index_tables(). +@param trx transaction +@param index fulltext index */ +dberr_t fts_lock_index_tables(trx_t *trx, const dict_index_t &index); + +/** Lock the internal common FTS_ tables, before fts_drop_common_tables(). +@param trx transaction +@param table table containing FULLTEXT INDEX @return DB_SUCCESS or error code */ -dberr_t -fts_drop_tables( -/*============*/ - trx_t* trx, /*!< in: transaction */ - dict_table_t* table); /*!< in: table has the FTS - index */ +dberr_t fts_lock_common_tables(trx_t *trx, const dict_table_t &table); + +/** Lock the internal FTS_ tables for table, before fts_drop_tables(). +@param trx transaction +@param table table containing FULLTEXT INDEX +@return DB_SUCCESS or error code */ +dberr_t fts_lock_tables(trx_t *trx, const dict_table_t &table); + +/** Drop the internal FTS_ tables for table. +@param trx transaction +@param table table containing FULLTEXT INDEX +@return DB_SUCCESS or error code */ +dberr_t fts_drop_tables(trx_t *trx, const dict_table_t &table); + /******************************************************************//** The given transaction is about to be committed; do whatever is necessary from the FTS system's POV. @@ -643,11 +655,7 @@ fts_optimize_init(void); /****************************************************************//** Drops index ancillary tables for a FTS index @return DB_SUCCESS or error code */ -dberr_t -fts_drop_index_tables( -/*==================*/ - trx_t* trx, /*!< in: transaction */ - dict_index_t* index) /*!< in: Index to drop */ +dberr_t fts_drop_index_tables(trx_t *trx, const dict_index_t &index) MY_ATTRIBUTE((warn_unused_result)); /** Add the table to add to the OPTIMIZER's list. diff --git a/storage/innobase/include/ha_prototypes.h b/storage/innobase/include/ha_prototypes.h index 4acee09715a..b65a874177c 100644 --- a/storage/innobase/include/ha_prototypes.h +++ b/storage/innobase/include/ha_prototypes.h @@ -453,7 +453,7 @@ normalize_table_name_c_low( char* norm_name, /*!< out: normalized name as a null-terminated string */ const char* name, /*!< in: table name string */ - ibool set_lower_case); /*!< in: TRUE if we want to set + bool set_lower_case); /*!< in: true if we want to set name to lower case */ /** Create a MYSQL_THD for a background thread and mark it as such. diff --git a/storage/innobase/include/row0mysql.h b/storage/innobase/include/row0mysql.h index 0b4590b67bc..6c783ec3495 100644 --- a/storage/innobase/include/row0mysql.h +++ b/storage/innobase/include/row0mysql.h @@ -37,11 +37,6 @@ Created 9/17/2000 Heikki Tuuri #include "fts0fts.h" #include "gis0type.h" -#include "sql_list.h" -#include "sql_cmd.h" - -extern ibool row_rollback_on_timeout; - struct row_prebuilt_t; class ha_innobase; @@ -378,17 +373,6 @@ row_create_index_for_mysql( uint32_t key_id) /*!< in: encryption key_id */ MY_ATTRIBUTE((warn_unused_result)); -/** The master task calls this regularly to drop tables which -we must drop in background after queries to them have ended. -@return how many tables dropped + remaining tables in list */ -ulint row_drop_tables_for_mysql_in_background(); - -/** @return number of tables in the background drop list */ -ulint row_get_background_drop_list_len_low(); - -/** Drop garbage tables during recovery. */ -void row_mysql_drop_garbage_tables(); - /*********************************************************************//** Sets an exclusive lock on a table. @return error code or DB_SUCCESS */ @@ -401,36 +385,12 @@ row_mysql_lock_table( const char* op_info) /*!< in: string for trx->op_info */ MY_ATTRIBUTE((nonnull, warn_unused_result)); -/** Drop a table. -If the data dictionary was not already locked by the transaction, -the transaction will be committed. Otherwise, the data dictionary -will remain locked. -@param[in] name Table name -@param[in,out] trx Transaction handle -@param[in] sqlcom type of SQL operation -@param[in] create_failed true=create table failed - because e.g. foreign key column -@param[in] nonatomic Whether it is permitted to release - and reacquire dict_sys.latch -@return error code */ -dberr_t -row_drop_table_for_mysql( - const char* name, - trx_t* trx, - enum_sql_command sqlcom, - bool create_failed = false, - bool nonatomic = true); - /*********************************************************************//** Discards the tablespace of a table which stored in an .ibd file. Discarding means that this function deletes the .ibd file and assigns a new table id for the table. Also the file_unreadable flag is set. @return error code or DB_SUCCESS */ -dberr_t -row_discard_tablespace_for_mysql( -/*=============================*/ - const char* name, /*!< in: table name */ - trx_t* trx) /*!< in: transaction handle */ +dberr_t row_discard_tablespace_for_mysql(dict_table_t *table, trx_t *trx) MY_ATTRIBUTE((nonnull, warn_unused_result)); /*****************************************************************//** Imports a tablespace. The space id in the .ibd file must match the space id @@ -452,7 +412,6 @@ row_rename_table_for_mysql( const char* old_name, /*!< in: old table name */ const char* new_name, /*!< in: new table name */ trx_t* trx, /*!< in/out: transaction */ - bool commit, /*!< in: whether to commit trx */ bool use_fk) /*!< in: whether to parse and enforce FOREIGN KEY constraints */ MY_ATTRIBUTE((nonnull, warn_unused_result)); @@ -472,17 +431,6 @@ row_scan_index_for_mysql( ulint* n_rows) /*!< out: number of entries seen in the consistent read */ MY_ATTRIBUTE((warn_unused_result)); -/*********************************************************************//** -Initialize this module */ -void -row_mysql_init(void); -/*================*/ - -/*********************************************************************//** -Close this module */ -void -row_mysql_close(void); -/*=================*/ /* A struct describing a place for an individual column in the MySQL row format which is presented to the table handler in ha_innobase. @@ -946,10 +894,4 @@ innobase_rename_vc_templ( #define ROW_READ_TRY_SEMI_CONSISTENT 1 #define ROW_READ_DID_SEMI_CONSISTENT 2 -#ifdef UNIV_DEBUG -/** Wait for the background drop list to become empty. */ -void -row_wait_for_background_drop_list_empty(); -#endif /* UNIV_DEBUG */ - #endif /* row0mysql.h */ diff --git a/storage/innobase/include/srv0mon.h b/storage/innobase/include/srv0mon.h index a7aa6e99307..56b56a18bdb 100644 --- a/storage/innobase/include/srv0mon.h +++ b/storage/innobase/include/srv0mon.h @@ -377,7 +377,6 @@ enum monitor_id_t { MONITOR_OVLD_SERVER_ACTIVITY, MONITOR_MASTER_ACTIVE_LOOPS, MONITOR_MASTER_IDLE_LOOPS, - MONITOR_SRV_BACKGROUND_DROP_TABLE_MICROSECOND, MONITOR_SRV_LOG_FLUSH_MICROSECOND, MONITOR_SRV_DICT_LRU_MICROSECOND, MONITOR_SRV_DICT_LRU_EVICT_COUNT_ACTIVE, @@ -400,7 +399,6 @@ enum monitor_id_t { /* Data DDL related counters */ MONITOR_MODULE_DDL_STATS, MONITOR_BACKGROUND_DROP_INDEX, - MONITOR_BACKGROUND_DROP_TABLE, MONITOR_ONLINE_CREATE_INDEX, MONITOR_PENDING_ALTER_TABLE, MONITOR_ALTER_TABLE_SORT_FILES, diff --git a/storage/innobase/include/trx0purge.h b/storage/innobase/include/trx0purge.h index db11f882968..559e38b73b6 100644 --- a/storage/innobase/include/trx0purge.h +++ b/storage/innobase/include/trx0purge.h @@ -138,6 +138,8 @@ private: Atomic_counter<uint32_t> m_paused; /** number of stop_SYS() calls without resume_SYS() */ Atomic_counter<uint32_t> m_SYS_paused; + /** number of stop_FTS() calls without resume_FTS() */ + Atomic_counter<uint32_t> m_FTS_paused; public: que_t* query; /*!< The query graph which will do the parallelized purge operation */ @@ -243,13 +245,14 @@ public: /** @return whether the purge tasks are active */ bool running() const; - /** Stop purge during FLUSH TABLES FOR EXPORT */ + /** Stop purge during FLUSH TABLES FOR EXPORT. */ void stop(); /** Resume purge at UNLOCK TABLES after FLUSH TABLES FOR EXPORT */ void resume(); private: void wait_SYS(); + void wait_FTS(); public: /** Suspend purge in data dictionary tables */ void stop_SYS(); @@ -260,6 +263,15 @@ public: /** check stop_SYS() */ void check_stop_SYS() { if (must_wait_SYS()) wait_SYS(); } + /** Pause purge during a DDL operation that could drop FTS_ tables. */ + void stop_FTS() { m_FTS_paused++; } + /** Resume purge after stop_FTS(). */ + void resume_FTS() { ut_d(const auto p=) m_FTS_paused--; ut_ad(p); } + /** @return whether stop_SYS() is in effect */ + bool must_wait_FTS() const { return m_FTS_paused; } + /** check stop_SYS() */ + void check_stop_FTS() { if (must_wait_FTS()) wait_FTS(); } + /** A wrapper around ReadView::changes_visible(). */ bool changes_visible(trx_id_t id, const table_name_t &name) const { diff --git a/storage/innobase/include/trx0trx.h b/storage/innobase/include/trx0trx.h index a570f3cb5e0..7c98db62c67 100644 --- a/storage/innobase/include/trx0trx.h +++ b/storage/innobase/include/trx0trx.h @@ -38,7 +38,6 @@ Created 3/26/1996 Heikki Tuuri #include "ilist.h" #include <vector> -#include <set> // Forward declaration struct mtr_t; @@ -412,7 +411,8 @@ class trx_mod_table_time_t /** First modification of the table, possibly ORed with BULK */ undo_no_t first; - /** First modification of a system versioned column (or NONE) */ + /** First modification of a system versioned column + (NONE= no versioning, BULK= the table was dropped) */ undo_no_t first_versioned= NONE; public: /** Constructor @@ -427,17 +427,26 @@ public: { auto f= first & LIMIT; return f <= first_versioned && f <= rows; } #endif /* UNIV_DEBUG */ /** @return if versioned columns were modified */ - bool is_versioned() const { return first_versioned != NONE; } + bool is_versioned() const { return (~first_versioned & LIMIT) != 0; } + /** @return if the table was dropped */ + bool is_dropped() const { return first_versioned == BULK; } /** After writing an undo log record, set is_versioned() if needed @param rows number of modified rows so far */ void set_versioned(undo_no_t rows) { - ut_ad(!is_versioned()); + ut_ad(first_versioned == NONE); first_versioned= rows; ut_ad(valid(rows)); } + /** After writing an undo log record, note that the table will be dropped */ + void set_dropped() + { + ut_ad(first_versioned == NONE); + first_versioned= BULK; + } + /** Notify the start of a bulk insert operation */ void start_bulk_insert() { first|= BULK; } @@ -923,6 +932,10 @@ private: inline void commit_tables(); /** Mark a transaction committed in the main memory data structures. */ inline void commit_in_memory(const mtr_t *mtr); + /** Write log for committing the transaction. */ + void commit_persist(); + /** Clean up the transaction after commit_in_memory() */ + void commit_cleanup(); /** Commit the transaction in a mini-transaction. @param mtr mini-transaction (if there are any persistent modifications) */ void commit_low(mtr_t *mtr= nullptr); @@ -931,6 +944,24 @@ public: void commit(); + /** Try to drop a persistent table. + @param table persistent table + @param fk whether to drop FOREIGN KEY metadata + @return error code */ + dberr_t drop_table(const dict_table_t &table); + /** Try to drop the foreign key constraints for a persistent table. + @param name name of persistent table + @return error code */ + dberr_t drop_table_foreign(const table_name_t &name); + /** Try to drop the statistics for a persistent table. + @param name name of persistent table + @return error code */ + dberr_t drop_table_statistics(const table_name_t &name); + /** Commit the transaction, possibly after drop_table(). + @param deleted handles of data files that were deleted */ + void commit(std::vector<pfs_os_file_t> &deleted); + + /** Discard all savepoints */ void savepoints_discard() { savepoints_discard(UT_LIST_GET_FIRST(trx_savepoints)); } diff --git a/storage/innobase/lock/lock0lock.cc b/storage/innobase/lock/lock0lock.cc index 209a9eb6fc5..199df390582 100644 --- a/storage/innobase/lock/lock0lock.cc +++ b/storage/innobase/lock/lock0lock.cc @@ -1265,19 +1265,16 @@ lock_rec_enqueue_waiting( trx_t* trx = thr_get_trx(thr); ut_ad(trx->mutex_is_owner()); - if (UNIV_UNLIKELY(trx->dict_operation)) { - ib::error() << "A record lock wait happens in a dictionary" - " operation. index " - << index->name - << " of table " - << index->table->name - << ". " << BUG_REPORT_MSG; - ut_ad(0); + if (UNIV_UNLIKELY(trx->dict_operation_lock_mode == RW_X_LATCH)) { + ut_ad(!strcmp(index->table->name.m_name, TABLE_STATS_NAME) + || !strcmp(index->table->name.m_name, INDEX_STATS_NAME)); +instant_timeout: + trx->error_state = DB_LOCK_WAIT_TIMEOUT; + return DB_LOCK_WAIT_TIMEOUT; } if (trx->mysql_thd && thd_lock_wait_timeout(trx->mysql_thd) == 0) { - trx->error_state = DB_LOCK_WAIT_TIMEOUT; - return DB_LOCK_WAIT_TIMEOUT; + goto instant_timeout; } /* Enqueue the lock request that will wait to be granted, note that @@ -3314,11 +3311,11 @@ lock_table_enqueue_waiting( trx_t* trx = thr_get_trx(thr); ut_ad(trx->mutex_is_owner()); - if (UNIV_UNLIKELY(trx->dict_operation)) { - ib::error() << "A table lock wait happens in a dictionary" - " operation. Table " << table->name - << ". " << BUG_REPORT_MSG; - ut_ad(0); + if (UNIV_UNLIKELY(trx->dict_operation_lock_mode == RW_X_LATCH)) { + ut_ad(!strcmp(table->name.m_name, TABLE_STATS_NAME) + || !strcmp(table->name.m_name, INDEX_STATS_NAME)); + trx->error_state = DB_LOCK_WAIT_TIMEOUT; + return DB_LOCK_WAIT_TIMEOUT; } #ifdef WITH_WSREP diff --git a/storage/innobase/os/os0file.cc b/storage/innobase/os/os0file.cc index e197b1cf81f..6b8b11605af 100644 --- a/storage/innobase/os/os0file.cc +++ b/storage/innobase/os/os0file.cc @@ -1482,8 +1482,7 @@ os_file_rename_func( /* New path must not exist. */ ut_ad(os_file_status(newpath, &exists, &type)); - /* MDEV-25506 FIXME: Remove the strstr() */ - ut_ad(!exists || strstr(oldpath, "/" TEMP_FILE_PREFIX_INNODB)); + ut_ad(!exists); /* Old path must exist. */ ut_ad(os_file_status(oldpath, &exists, &type)); @@ -2535,8 +2534,7 @@ os_file_rename_func( /* New path must not exist. */ ut_ad(os_file_status(newpath, &exists, &type)); - /* MDEV-25506 FIXME: Remove the strstr() */ - ut_ad(!exists || strstr(oldpath, "/" TEMP_FILE_PREFIX_INNODB)); + ut_ad(!exists); /* Old path must exist. */ ut_ad(os_file_status(oldpath, &exists, &type)); diff --git a/storage/innobase/row/row0ins.cc b/storage/innobase/row/row0ins.cc index 6cb1ad64c88..08bc40a5da2 100644 --- a/storage/innobase/row/row0ins.cc +++ b/storage/innobase/row/row0ins.cc @@ -1830,16 +1830,12 @@ do_possible_lock_wait: thr->lock_state = QUE_THR_LOCK_ROW; - check_table->inc_fk_checks(); - err = lock_wait(thr); thr->lock_state = QUE_THR_LOCK_NOLOCK; - check_table->dec_fk_checks(); - if (err != DB_SUCCESS) { - } else if (check_table->to_be_dropped) { + } else if (check_table->name.is_temporary()) { err = DB_LOCK_WAIT_TIMEOUT; } else { err = DB_LOCK_WAIT; @@ -1912,14 +1908,10 @@ row_ins_check_foreign_constraints( { dict_foreign_t* foreign; dberr_t err = DB_SUCCESS; - trx_t* trx; - ibool got_s_lock = FALSE; mem_heap_t* heap = NULL; DBUG_ASSERT(index->is_primary() == pk); - trx = thr_get_trx(thr); - DEBUG_SYNC_C_IF_THD(thr_get_trx(thr)->mysql_thd, "foreign_constraint_check_for_ins"); @@ -1965,32 +1957,9 @@ row_ins_check_foreign_constraints( FALSE, FALSE, DICT_ERR_IGNORE_NONE); } - if (0 == trx->dict_operation_lock_mode) { - got_s_lock = TRUE; - - row_mysql_freeze_data_dictionary(trx); - } - - if (referenced_table) { - foreign->foreign_table->inc_fk_checks(); - } - - /* NOTE that if the thread ends up waiting for a lock - we will release dict_sys.latch temporarily! - But the counter on the table protects the referenced - table from being dropped while the check is running. */ - err = row_ins_check_foreign_constraint( TRUE, foreign, table, ref_tuple, thr); - if (referenced_table) { - foreign->foreign_table->dec_fk_checks(); - } - - if (got_s_lock) { - row_mysql_unfreeze_data_dictionary(trx); - } - if (ref_table != NULL) { dict_table_close(ref_table, FALSE, FALSE); } diff --git a/storage/innobase/row/row0merge.cc b/storage/innobase/row/row0merge.cc index 485af26dd32..715a49cd3ae 100644 --- a/storage/innobase/row/row0merge.cc +++ b/storage/innobase/row/row0merge.cc @@ -3787,7 +3787,7 @@ static void row_merge_drop_fulltext_indexes(trx_t *trx, dict_table_t *table) return; fts_optimize_remove_table(table); - fts_drop_tables(trx, table); + fts_drop_tables(trx, *table); fts_free(table); DICT_TF2_FLAG_UNSET(table, DICT_TF2_FTS); } diff --git a/storage/innobase/row/row0mysql.cc b/storage/innobase/row/row0mysql.cc index 3b365bd86a2..4adc279fb32 100644 --- a/storage/innobase/row/row0mysql.cc +++ b/storage/innobase/row/row0mysql.cc @@ -38,8 +38,6 @@ Created 9/17/2000 Heikki Tuuri #include "dict0load.h" #include "dict0stats.h" #include "dict0stats_bg.h" -#include "dict0defrag_bg.h" -#include "btr0defragment.h" #include "fil0fil.h" #include "fil0crypt.h" #include "fsp0file.h" @@ -62,6 +60,7 @@ Created 9/17/2000 Heikki Tuuri #include "trx0undo.h" #include "srv0mon.h" #include "srv0start.h" +#include "log.h" #include <algorithm> #include <vector> @@ -70,33 +69,9 @@ Created 9/17/2000 Heikki Tuuri #ifdef WITH_WSREP #include "mysql/service_wsrep.h" #include "wsrep.h" -#include "log.h" #include "wsrep_mysqld.h" #endif -/** Provide optional 4.x backwards compatibility for 5.0 and above */ -ibool row_rollback_on_timeout = FALSE; - -/** Chain node of the list of tables to drop in the background. */ -struct row_mysql_drop_t{ - table_id_t table_id; /*!< table id */ - UT_LIST_NODE_T(row_mysql_drop_t)row_mysql_drop_list; - /*!< list chain node */ -}; - -/** @brief List of tables we should drop in background. - -ALTER TABLE in MySQL requires that the table handler can drop the -table in background when there are no queries to it any -more. Protected by row_drop_list_mutex. */ -static UT_LIST_BASE_NODE_T(row_mysql_drop_t) row_mysql_drop_list; - -/** Mutex protecting the background table drop list. */ -static mysql_mutex_t row_drop_list_mutex; - -/** Flag: has row_mysql_drop_list been initialized? */ -static bool row_mysql_drop_list_inited; - /*******************************************************************//** Determine if the given name is a name reserved for MySQL system tables. @return TRUE if name is a MySQL system table name */ @@ -116,21 +91,6 @@ row_mysql_is_system_table( || 0 == strcmp(name + 6, "db")); } -#ifdef UNIV_DEBUG -/** Wait for the background drop list to become empty. */ -void -row_wait_for_background_drop_list_empty() -{ - bool empty = false; - while (!empty) { - mysql_mutex_lock(&row_drop_list_mutex); - empty = (UT_LIST_GET_LEN(row_mysql_drop_list) == 0); - mysql_mutex_unlock(&row_drop_list_mutex); - std::this_thread::sleep_for(std::chrono::milliseconds(100)); - } -} -#endif /* UNIV_DEBUG */ - /*******************************************************************//** Delays an INSERT, DELETE or UPDATE operation if the purge is lagging. */ static @@ -701,7 +661,8 @@ handle_new_error: switch (err) { case DB_LOCK_WAIT_TIMEOUT: - if (row_rollback_on_timeout) { + extern my_bool innobase_rollback_on_timeout; + if (innobase_rollback_on_timeout) { goto rollback; } /* fall through */ @@ -2476,258 +2437,6 @@ row_create_index_for_mysql( return(err); } -/*********************************************************************//** -Drops a table for MySQL as a background operation. MySQL relies on Unix -in ALTER TABLE to the fact that the table handler does not remove the -table before all handles to it has been removed. Furhermore, the MySQL's -call to drop table must be non-blocking. Therefore we do the drop table -as a background operation, which is taken care of by the master thread -in srv0srv.cc. -@return error code or DB_SUCCESS */ -static -dberr_t -row_drop_table_for_mysql_in_background( -/*===================================*/ - const char* name) /*!< in: table name */ -{ - dberr_t error; - trx_t* trx; - - trx = trx_create(); - - /* If the original transaction was dropping a table referenced by - foreign keys, we must set the following to be able to drop the - table: */ - - trx->check_foreigns = false; - - /* Try to drop the table in InnoDB */ - - error = row_drop_table_for_mysql(name, trx, SQLCOM_TRUNCATE); - - trx_commit_for_mysql(trx); - - trx->free(); - - return(error); -} - -/*********************************************************************//** -The master thread in srv0srv.cc calls this regularly to drop tables which -we must drop in background after queries to them have ended. Such lazy -dropping of tables is needed in ALTER TABLE on Unix. -@return how many tables dropped + remaining tables in list */ -ulint -row_drop_tables_for_mysql_in_background(void) -/*=========================================*/ -{ - row_mysql_drop_t* drop; - dict_table_t* table; - ulint n_tables; - ulint n_tables_dropped = 0; -loop: - mysql_mutex_lock(&row_drop_list_mutex); - - ut_a(row_mysql_drop_list_inited); -next: - drop = UT_LIST_GET_FIRST(row_mysql_drop_list); - - n_tables = UT_LIST_GET_LEN(row_mysql_drop_list); - - mysql_mutex_unlock(&row_drop_list_mutex); - - if (drop == NULL) { - /* All tables dropped */ - - return(n_tables + n_tables_dropped); - } - - /* On fast shutdown, just empty the list without dropping tables. */ - table = srv_shutdown_state == SRV_SHUTDOWN_NONE || !srv_fast_shutdown - ? dict_table_open_on_id(drop->table_id, FALSE, - DICT_TABLE_OP_OPEN_ONLY_IF_CACHED) - : NULL; - - if (!table) { - n_tables_dropped++; - mysql_mutex_lock(&row_drop_list_mutex); - UT_LIST_REMOVE(row_mysql_drop_list, drop); - MONITOR_DEC(MONITOR_BACKGROUND_DROP_TABLE); - ut_free(drop); - goto next; - } - - ut_a(!table->can_be_evicted); - - bool skip = false; - - if (!table->to_be_dropped) { -skip: - dict_table_close(table, FALSE, FALSE); - - mysql_mutex_lock(&row_drop_list_mutex); - UT_LIST_REMOVE(row_mysql_drop_list, drop); - if (!skip) { - UT_LIST_ADD_LAST(row_mysql_drop_list, drop); - } else { - ut_free(drop); - } - goto next; - } - - if (!srv_fast_shutdown && !trx_sys.any_active_transactions()) { - table->lock_mutex_lock(); - skip = UT_LIST_GET_LEN(table->locks) != 0; - table->lock_mutex_unlock(); - if (skip) { - /* We cannot drop tables that are locked by XA - PREPARE transactions. */ - goto skip; - } - } - - char* name = mem_strdup(table->name.m_name); - - dict_table_close(table, FALSE, FALSE); - - dberr_t err = row_drop_table_for_mysql_in_background(name); - - ut_free(name); - - if (err != DB_SUCCESS) { - /* If the DROP fails for some table, we return, and let the - main thread retry later */ - return(n_tables + n_tables_dropped); - } - - goto loop; -} - -/*********************************************************************//** -Get the background drop list length. NOTE: the caller must own the -drop list mutex! -@return how many tables in list */ -ulint -row_get_background_drop_list_len_low(void) -/*======================================*/ -{ - ulint len; - - mysql_mutex_lock(&row_drop_list_mutex); - - ut_a(row_mysql_drop_list_inited); - - len = UT_LIST_GET_LEN(row_mysql_drop_list); - - mysql_mutex_unlock(&row_drop_list_mutex); - - return(len); -} - -/** Drop garbage tables during recovery. */ -void row_mysql_drop_garbage_tables() -{ - btr_pcur_t pcur; - mtr_t mtr; - trx_t* trx = trx_create(); - trx->op_info = "dropping garbage tables"; - row_mysql_lock_data_dictionary(trx); - - mtr.start(); - btr_pcur_open_at_index_side( - true, dict_table_get_first_index(dict_sys.sys_tables), - BTR_SEARCH_LEAF, &pcur, true, 0, &mtr); - - for (;;) { - const rec_t* rec; - const byte* field; - ulint len; - - btr_pcur_move_to_next_user_rec(&pcur, &mtr); - - if (!btr_pcur_is_on_user_rec(&pcur)) { - break; - } - - rec = btr_pcur_get_rec(&pcur); - if (rec_get_deleted_flag(rec, 0)) { - continue; - } - - field = rec_get_nth_field_old(rec, 0/*NAME*/, &len); - if (len == UNIV_SQL_NULL) { - /* Corrupted SYS_TABLES.NAME */ - continue; - } - - if (!dict_table_t::is_garbage_name(field, len)) { - continue; - } - - btr_pcur_store_position(&pcur, &mtr); - btr_pcur_commit_specify_mtr(&pcur, &mtr); - - const span<const char> name = { - reinterpret_cast<const char*>(pcur.old_rec), len - }; - if (dict_sys.load_table(name, DICT_ERR_IGNORE_DROP)) { - char* table_name = mem_strdupl(name.data(), len); - row_drop_table_for_mysql(table_name, trx, - SQLCOM_DROP_TABLE); - ut_free(table_name); - trx_commit_for_mysql(trx); - } - - mtr.start(); - btr_pcur_restore_position(BTR_SEARCH_LEAF, &pcur, &mtr); - } - - btr_pcur_close(&pcur); - mtr.commit(); - row_mysql_unlock_data_dictionary(trx); - trx->free(); -} - -/*********************************************************************//** -If a table is not yet in the drop list, adds the table to the list of tables -which the master thread drops in background. We need this on Unix because in -ALTER TABLE MySQL may call drop table even if the table has running queries on -it. Also, if there are running foreign key checks on the table, we drop the -table lazily. -@return whether background DROP TABLE was scheduled for the first time */ -static -bool -row_add_table_to_background_drop_list(table_id_t table_id) -{ - row_mysql_drop_t* drop; - bool added = true; - - mysql_mutex_lock(&row_drop_list_mutex); - - ut_a(row_mysql_drop_list_inited); - - /* Look if the table already is in the drop list */ - for (drop = UT_LIST_GET_FIRST(row_mysql_drop_list); - drop != NULL; - drop = UT_LIST_GET_NEXT(row_mysql_drop_list, drop)) { - - if (drop->table_id == table_id) { - added = false; - goto func_exit; - } - } - - drop = static_cast<row_mysql_drop_t*>(ut_malloc_nokey(sizeof *drop)); - drop->table_id = table_id; - - UT_LIST_ADD_LAST(row_mysql_drop_list, drop); - - MONITOR_INC(MONITOR_BACKGROUND_DROP_TABLE); -func_exit: - mysql_mutex_unlock(&row_drop_list_mutex); - return added; -} - /** Reassigns the table identifier of a table. @param[in,out] table table @param[in,out] trx transaction @@ -2770,42 +2479,6 @@ row_mysql_table_id_reassign( } /*********************************************************************//** -Setup the pre-requisites for DISCARD TABLESPACE. It will start the transaction, -acquire the data dictionary lock in X mode and open the table. -@return table instance or 0 if not found. */ -static -dict_table_t* -row_discard_tablespace_begin( -/*=========================*/ - const char* name, /*!< in: table name */ - trx_t* trx) /*!< in: transaction handle */ -{ - trx->op_info = "discarding tablespace"; - - trx->dict_operation = true; - - trx_start_if_not_started_xa(trx, true); - - /* Serialize data dictionary operations with dictionary mutex: - this is to avoid deadlocks during data dictionary operations */ - - row_mysql_lock_data_dictionary(trx); - - dict_table_t* table; - - table = dict_table_open_on_name( - name, TRUE, FALSE, DICT_ERR_IGNORE_FK_NOKEY); - - if (table) { - dict_stats_wait_bg_to_stop_using_table(table, trx); - ut_a(!is_system_tablespace(table->space_id)); - ut_ad(!table->n_foreign_key_checks_running); - } - - return(table); -} - -/*********************************************************************//** Do the foreign key constraint checks. @return DB_SUCCESS or error code. */ static @@ -2859,38 +2532,6 @@ row_discard_tablespace_foreign_key_checks( } /*********************************************************************//** -Cleanup after the DISCARD TABLESPACE operation. -@return error code. */ -static -dberr_t -row_discard_tablespace_end( -/*=======================*/ - trx_t* trx, /*!< in/out: transaction handle */ - dict_table_t* table, /*!< in/out: table to be discarded */ - dberr_t err) /*!< in: error code */ -{ - if (table != 0) { - dict_table_close(table, TRUE, FALSE); - } - - DBUG_EXECUTE_IF("ib_discard_before_commit_crash", - log_write_up_to(LSN_MAX, true); - DBUG_SUICIDE();); - - trx_commit_for_mysql(trx); - - DBUG_EXECUTE_IF("ib_discard_after_commit_crash", - log_write_up_to(LSN_MAX, true); - DBUG_SUICIDE();); - - row_mysql_unlock_data_dictionary(trx); - - trx->op_info = ""; - - return(err); -} - -/*********************************************************************//** Do the DISCARD TABLESPACE operation. @return DB_SUCCESS or error code. */ static @@ -2900,17 +2541,17 @@ row_discard_tablespace( trx_t* trx, /*!< in/out: transaction handle */ dict_table_t* table) /*!< in/out: table to be discarded */ { - dberr_t err; + dberr_t err; /* How do we prevent crashes caused by ongoing operations on the table? Old operations could try to access non-existent - pages. MySQL will block all DML on the table using MDL and a + pages. The SQL layer will block all DML on the table using MDL and a DISCARD will not start unless all existing operations on the table to be discarded are completed. - 1) Acquire the data dictionary latch in X mode. To prevent any - internal operations that MySQL is not aware off and also for - the internal SQL parser. + 1) Acquire the data dictionary latch in X mode. This will + prevent any internal operations that are not covered by + MDL or InnoDB table locks. 2) Purge and rollback: we assign a new table id for the table. Since purge and rollback look for the table based on @@ -2943,7 +2584,7 @@ row_discard_tablespace( if (dict_table_has_fts_index(table) || DICT_TF2_FLAG_IS_SET(table, DICT_TF2_FTS_HAS_DOC_ID)) { - fts_drop_tables(trx, table); + fts_drop_tables(trx, *table); } /* Assign a new space ID to the table definition so that purge @@ -2955,23 +2596,6 @@ row_discard_tablespace( return(err); } - /* Discard the physical file that is used for the tablespace. */ - err = fil_delete_tablespace(table->space_id); - switch (err) { - case DB_IO_ERROR: - ib::warn() << "ALTER TABLE " << table->name - << " DISCARD TABLESPACE failed to delete file"; - break; - case DB_TABLESPACE_NOT_FOUND: - ib::warn() << "ALTER TABLE " << table->name - << " DISCARD TABLESPACE failed to find tablespace"; - break; - case DB_SUCCESS: - break; - default: - ut_error; - } - /* All persistent operations successful, update the data dictionary memory cache. */ @@ -2999,73 +2623,83 @@ Discards the tablespace of a table which stored in an .ibd file. Discarding means that this function renames the .ibd file and assigns a new table id for the table. Also the file_unreadable flag is set. @return error code or DB_SUCCESS */ -dberr_t -row_discard_tablespace_for_mysql( -/*=============================*/ - const char* name, /*!< in: table name */ - trx_t* trx) /*!< in: transaction handle */ +dberr_t row_discard_tablespace_for_mysql(dict_table_t *table, trx_t *trx) { - dberr_t err; - dict_table_t* table; - - /* Open the table and start the transaction if not started. */ - - table = row_discard_tablespace_begin(name, trx); - - if (table == 0) { - err = DB_TABLE_NOT_FOUND; - } else if (table->is_temporary()) { + ut_ad(!is_system_tablespace(table->space_id)); + ut_ad(!table->is_temporary()); - ib_senderrf(trx->mysql_thd, IB_LOG_LEVEL_ERROR, - ER_CANNOT_DISCARD_TEMPORARY_TABLE); + const auto fts_exist = table->flags2 & + (DICT_TF2_FTS_HAS_DOC_ID | DICT_TF2_FTS); - err = DB_ERROR; - - } else if (table->space_id == TRX_SYS_SPACE) { - char table_name[MAX_FULL_NAME_LEN + 1]; - - innobase_format_name( - table_name, sizeof(table_name), - table->name.m_name); - - ib_senderrf(trx->mysql_thd, IB_LOG_LEVEL_ERROR, - ER_TABLE_IN_SYSTEM_TABLESPACE, table_name); - - err = DB_ERROR; - - } else { - ut_ad(!table->n_foreign_key_checks_running); + dberr_t err; - bool fts_exist = (dict_table_has_fts_index(table) - || DICT_TF2_FLAG_IS_SET( - table, DICT_TF2_FTS_HAS_DOC_ID)); - - if (fts_exist) { - row_mysql_unlock_data_dictionary(trx); - fts_optimize_remove_table(table); - row_mysql_lock_data_dictionary(trx); - } + if (fts_exist) + { + fts_optimize_remove_table(table); + purge_sys.stop_FTS(); + err= fts_lock_tables(trx, *table); + if (err != DB_SUCCESS) + { +rollback: + table->stats_bg_flag= BG_STAT_NONE; + if (fts_exist) + { + purge_sys.resume_FTS(); + fts_optimize_add_table(table); + } + trx->rollback(); + return err; + } + } - /* Do foreign key constraint checks. */ + row_mysql_lock_data_dictionary(trx); + dict_stats_wait_bg_to_stop_using_table(table, trx); - err = row_discard_tablespace_foreign_key_checks(trx, table); + trx->op_info = "discarding tablespace"; + trx->dict_operation= true; - if (err == DB_SUCCESS) { - /* Note: This cannot be rolled back. - Rollback would see the UPDATE SYS_INDEXES - as two operations: DELETE and INSERT. - It would invoke btr_free_if_exists() - when rolling back the INSERT, effectively - dropping all indexes of the table. */ - err = row_discard_tablespace(trx, table); - } + /* Serialize data dictionary operations with dictionary mutex: + this is to avoid deadlocks during data dictionary operations */ - if (fts_exist && err != DB_SUCCESS) { - fts_optimize_add_table(table); - } - } + err= row_discard_tablespace_foreign_key_checks(trx, table); + if (err != DB_SUCCESS) + { + row_mysql_unlock_data_dictionary(trx); + goto rollback; + } - return(row_discard_tablespace_end(trx, table, err)); + /* Note: The following cannot be rolled back. Rollback would see the + UPDATE of SYS_INDEXES.TABLE_ID as two operations: DELETE and INSERT. + It would invoke btr_free_if_exists() when rolling back the INSERT, + effectively dropping all indexes of the table. Furthermore, calls like + ibuf_delete_for_discarded_space() are already discarding data + before the transaction is committed. + + It would be better to remove the integrity-breaking + ALTER TABLE...DISCARD TABLESPACE operation altogether. */ + err= row_discard_tablespace(trx, table); + DBUG_EXECUTE_IF("ib_discard_before_commit_crash", + log_write_up_to(LSN_MAX, true); DBUG_SUICIDE();); + /* FTS_ tables may be deleted */ + std::vector<pfs_os_file_t> deleted; + trx->commit(deleted); + const auto space_id= table->space_id; + pfs_os_file_t d= fil_delete_tablespace(space_id); + table->space= nullptr; + DBUG_EXECUTE_IF("ib_discard_after_commit_crash", DBUG_SUICIDE();); + row_mysql_unlock_data_dictionary(trx); + + if (d != OS_FILE_CLOSED) + os_file_close(d); + for (pfs_os_file_t d : deleted) + os_file_close(d); + + if (fts_exist) + purge_sys.resume_FTS(); + + buf_flush_remove_pages(space_id); + trx->op_info= ""; + return err; } /*********************************************************************//** @@ -3116,532 +2750,6 @@ row_mysql_lock_table( return(err); } -/** Drop ancillary FTS tables as part of dropping a table. -@param[in,out] table Table cache entry -@param[in,out] trx Transaction handle -@return error code or DB_SUCCESS */ -UNIV_INLINE -dberr_t -row_drop_ancillary_fts_tables( - dict_table_t* table, - trx_t* trx) -{ - /* Drop ancillary FTS tables */ - if (dict_table_has_fts_index(table) - || DICT_TF2_FLAG_IS_SET(table, DICT_TF2_FTS_HAS_DOC_ID)) { - - ut_ad(table->get_ref_count() == 0); - ut_ad(trx_is_started(trx)); - - dberr_t err = fts_drop_tables(trx, table); - - if (UNIV_UNLIKELY(err != DB_SUCCESS)) { - ib::error() << " Unable to remove ancillary FTS" - " tables for table " - << table->name << " : " << err; - - return(err); - } - } - - /* The table->fts flag can be set on the table for which - the cluster index is being rebuilt. Such table might not have - DICT_TF2_FTS flag set. So keep this out of above - dict_table_has_fts_index condition */ - if (table->fts != NULL) { - /* fts_que_graph_free_check_lock would try to acquire - dict mutex lock */ - table->fts->dict_locked = true; - - fts_free(table); - } - - return(DB_SUCCESS); -} - -/** Drop a table for MySQL. -If the data dictionary was not already locked by the transaction, -the transaction will be committed. Otherwise, the data dictionary -will remain locked. -@param[in] name Table name -@param[in,out] trx Transaction handle -@param[in] sqlcom type of SQL operation -@param[in] create_failed true=create table failed - because e.g. foreign key column -@param[in] nonatomic Whether it is permitted to release - and reacquire dict_sys.latch -@return error code or DB_SUCCESS */ -dberr_t -row_drop_table_for_mysql( - const char* name, - trx_t* trx, - enum_sql_command sqlcom, - bool create_failed, - bool nonatomic) -{ - dberr_t err; - dict_foreign_t* foreign; - dict_table_t* table; - char* tablename = NULL; - bool locked_dictionary = false; - pars_info_t* info = NULL; - mem_heap_t* heap = NULL; - - - DBUG_ENTER("row_drop_table_for_mysql"); - DBUG_PRINT("row_drop_table_for_mysql", ("table: '%s'", name)); - - ut_a(name != NULL); - - /* Serialize data dictionary operations with dictionary mutex: - no deadlocks can occur then in these operations */ - - trx->op_info = "dropping table"; - - if (trx->dict_operation_lock_mode != RW_X_LATCH) { - /* Prevent foreign key checks etc. while we are - dropping the table */ - - row_mysql_lock_data_dictionary(trx); - - locked_dictionary = true; - nonatomic = true; - } - - ut_d(dict_sys.assert_locked()); - - table = dict_table_open_on_name( - name, TRUE, FALSE, - static_cast<dict_err_ignore_t>( - DICT_ERR_IGNORE_INDEX_ROOT - | DICT_ERR_IGNORE_CORRUPT)); - - if (!table) { - if (locked_dictionary) { - row_mysql_unlock_data_dictionary(trx); - } - trx->op_info = ""; - DBUG_RETURN(DB_TABLE_NOT_FOUND); - } - - pfs_os_file_t detached_handle = OS_FILE_CLOSED; - - const bool is_temp_name = strstr(table->name.m_name, - "/" TEMP_FILE_PREFIX_INNODB); - - if (table->is_temporary()) { - ut_ad(table->space == fil_system.temp_space); - for (dict_index_t* index = dict_table_get_first_index(table); - index != NULL; - index = dict_table_get_next_index(index)) { - btr_free(page_id_t(SRV_TMP_SPACE_ID, index->page)); - } - /* Remove the pointer to this table object from the list - of modified tables by the transaction because the object - is going to be destroyed below. */ - trx->mod_tables.erase(table); - table->release(); - dict_sys.remove(table); - err = DB_SUCCESS; - goto funct_exit_all_freed; - } - - /* This function is called recursively via fts_drop_tables(). */ - if (!trx_is_started(trx)) { - trx_start_for_ddl(trx); - } - - /* Turn on this drop bit before we could release the dictionary - latch */ - table->to_be_dropped = true; - - if (nonatomic) { - /* This trx did not acquire any locks on dictionary - table records yet. Thus it is safe to release and - reacquire the data dictionary latches. */ - if (table->fts) { - row_mysql_unlock_data_dictionary(trx); - fts_optimize_remove_table(table); - row_mysql_lock_data_dictionary(trx); - } - - dict_stats_wait_bg_to_stop_using_table(table, trx); - } - - /* make sure background stats thread is not running on the table */ - ut_ad(!(table->stats_bg_flag & BG_STAT_IN_PROGRESS)); - if (!table->no_rollback()) { - if (table->space != fil_system.sys_space) { - /* Delete the link file if used. */ - if (DICT_TF_HAS_DATA_DIR(table->flags)) { - RemoteDatafile::delete_link_file( - {table->name.m_name, - strlen(table->name.m_name)}); - } - } - - dict_stats_recalc_pool_del(table); - dict_stats_defrag_pool_del(table, NULL); - if (btr_defragment_active) { - btr_defragment_remove_table(table); - } - - if (UNIV_LIKELY(!strstr(name, "/" TEMP_FILE_PREFIX_INNODB))) { - /* Remove any persistent statistics for this table, - in a separate transaction. */ - char errstr[1024]; - err = dict_stats_drop_table(name, errstr, - sizeof errstr); - if (err != DB_SUCCESS) { - ib::warn() << errstr; - } - } - } - - dict_sys.prevent_eviction(table); - dict_table_close(table, TRUE, FALSE); - - /* Check if the table is referenced by foreign key constraints from - some other table (not the table itself) */ - - if (!srv_read_only_mode && trx->check_foreigns) { - - for (dict_foreign_set::iterator it - = table->referenced_set.begin(); - it != table->referenced_set.end(); - ++it) { - - foreign = *it; - - const bool ref_ok = sqlcom == SQLCOM_DROP_DB - && dict_tables_have_same_db( - name, - foreign->foreign_table_name_lookup); - - /* We should allow dropping a referenced table if creating - that referenced table has failed for some reason. For example - if referenced table is created but it column types that are - referenced do not match. */ - if (foreign->foreign_table != table && - !create_failed && !ref_ok) { - - FILE* ef = dict_foreign_err_file; - - /* We only allow dropping a referenced table - if FOREIGN_KEY_CHECKS is set to 0 */ - - err = DB_CANNOT_DROP_CONSTRAINT; - - mysql_mutex_lock(&dict_foreign_err_mutex); - rewind(ef); - ut_print_timestamp(ef); - - fputs(" Cannot drop table ", ef); - ut_print_name(ef, trx, name); - fputs("\n" - "because it is referenced by ", ef); - ut_print_name(ef, trx, - foreign->foreign_table_name); - putc('\n', ef); - mysql_mutex_unlock(&dict_foreign_err_mutex); - - goto funct_exit; - } - } - } - - DBUG_EXECUTE_IF("row_drop_table_add_to_background", goto defer;); - - /* TODO: could we replace the counter n_foreign_key_checks_running - with lock checks on the table? Acquire here an exclusive lock on the - table, and rewrite lock0lock.cc and the lock wait in srv0srv.cc so that - they can cope with the table having been dropped here? Foreign key - checks take an IS or IX lock on the table. */ - - if (table->n_foreign_key_checks_running > 0) { -defer: - /* Rename the table to #sql-ib prefix. This scenario can - occur also for #sql tables when purge is waiting for - dict_sys.mutex so that it could close the table. But - DROP TABLE acquires dict_sys.mutex. */ - if (!is_temp_name) { - heap = mem_heap_create(FN_REFLEN); - const char* tmp_name - = dict_mem_create_temporary_tablename( - heap, table->name.m_name, table->id); - ib::info() << "Deferring DROP TABLE " << table->name - << "; renaming to " << tmp_name; - err = row_rename_table_for_mysql( - table->name.m_name, tmp_name, trx, - false, false); - } else { - err = DB_SUCCESS; - } - if (err == DB_SUCCESS) { - row_add_table_to_background_drop_list(table->id); - } - goto funct_exit; - } - - /* Remove all locks that are on the table or its records, if there - are no references to the table but it has record locks, we release - the record locks unconditionally. One use case is: - - CREATE TABLE t2 (PRIMARY KEY (a)) SELECT * FROM t1; - - If after the user transaction has done the SELECT and there is a - problem in completing the CREATE TABLE operation, MySQL will drop - the table. InnoDB will create a new background transaction to do the - actual drop, the trx instance that is passed to this function. To - preserve existing behaviour we remove the locks but ideally we - shouldn't have to. There should never be record locks on a table - that is going to be dropped. */ - - if (table->get_ref_count() > 0 || lock_table_has_locks(table)) { - goto defer; - } - - /* The "to_be_dropped" marks table that is to be dropped, but - has not been dropped, instead, was put in the background drop - list due to being used by concurrent DML operations. Clear it - here since there are no longer any concurrent activities on it, - and it is free to be dropped */ - table->to_be_dropped = false; - - trx->dict_operation = true; - - /* Mark all indexes unavailable in the data dictionary cache - before starting to drop the table. */ - - unsigned* page_no; - unsigned* page_nos; - heap = mem_heap_create( - 200 + UT_LIST_GET_LEN(table->indexes) * sizeof *page_nos); - tablename = mem_heap_strdup(heap, name); - - page_no = page_nos = static_cast<unsigned*>( - mem_heap_alloc( - heap, - UT_LIST_GET_LEN(table->indexes) * sizeof *page_no)); - - for (dict_index_t* index = dict_table_get_first_index(table); - index != NULL; - index = dict_table_get_next_index(index)) { - index->lock.x_lock(SRW_LOCK_CALL); - /* Save the page numbers so that we can restore them - if the operation fails. */ - *page_no++ = index->page; - /* Mark the index unusable. */ - index->page = FIL_NULL; - index->lock.x_unlock(); - } - - /* Deleting a row from SYS_INDEXES table will invoke - dict_drop_index_tree(). */ - info = pars_info_create(); - - pars_info_add_str_literal(info, "name", name); - - if (sqlcom != SQLCOM_TRUNCATE && strchr(name, '/') - && dict_sys.sys_foreign && dict_sys.sys_foreign_cols) { - err = que_eval_sql( - info, - "PROCEDURE DROP_FOREIGN_PROC () IS\n" - "fid CHAR;\n" - - "DECLARE CURSOR fk IS\n" - "SELECT ID FROM SYS_FOREIGN\n" - "WHERE FOR_NAME = :name\n" - "AND TO_BINARY(FOR_NAME) = TO_BINARY(:name)\n" - "FOR UPDATE;\n" - - "BEGIN\n" - "OPEN fk;\n" - "WHILE 1 = 1 LOOP\n" - " FETCH fk INTO fid;\n" - " IF (SQL % NOTFOUND) THEN RETURN; END IF;\n" - " DELETE FROM SYS_FOREIGN_COLS WHERE ID=fid;\n" - " DELETE FROM SYS_FOREIGN WHERE ID=fid;\n" - "END LOOP;\n" - "CLOSE fk;\n" - "END;\n", FALSE, trx); - if (err == DB_SUCCESS) { - info = pars_info_create(); - pars_info_add_str_literal(info, "name", name); - goto do_drop; - } - } else { -do_drop: - if (dict_sys.sys_virtual) { - err = que_eval_sql( - info, - "PROCEDURE DROP_VIRTUAL_PROC () IS\n" - "tid CHAR;\n" - - "BEGIN\n" - "SELECT ID INTO tid FROM SYS_TABLES\n" - "WHERE NAME = :name FOR UPDATE;\n" - "IF (SQL % NOTFOUND) THEN RETURN;" - " END IF;\n" - "DELETE FROM SYS_VIRTUAL" - " WHERE TABLE_ID = tid;\n" - "END;\n", FALSE, trx); - if (err == DB_SUCCESS) { - info = pars_info_create(); - pars_info_add_str_literal( - info, "name", name); - } - } else { - err = DB_SUCCESS; - } - - err = err == DB_SUCCESS ? que_eval_sql( - info, - "PROCEDURE DROP_TABLE_PROC () IS\n" - "tid CHAR;\n" - "iid CHAR;\n" - - "DECLARE CURSOR cur_idx IS\n" - "SELECT ID FROM SYS_INDEXES\n" - "WHERE TABLE_ID = tid FOR UPDATE;\n" - - "BEGIN\n" - "SELECT ID INTO tid FROM SYS_TABLES\n" - "WHERE NAME = :name FOR UPDATE;\n" - "IF (SQL % NOTFOUND) THEN RETURN; END IF;\n" - - "OPEN cur_idx;\n" - "WHILE 1 = 1 LOOP\n" - " FETCH cur_idx INTO iid;\n" - " IF (SQL % NOTFOUND) THEN EXIT; END IF;\n" - " DELETE FROM SYS_FIELDS\n" - " WHERE INDEX_ID = iid;\n" - " DELETE FROM SYS_INDEXES\n" - " WHERE ID = iid AND TABLE_ID = tid;\n" - "END LOOP;\n" - "CLOSE cur_idx;\n" - - "DELETE FROM SYS_COLUMNS WHERE TABLE_ID=tid;\n" - "DELETE FROM SYS_TABLES WHERE NAME=:name;\n" - - "END;\n", FALSE, trx) : err; - } - - switch (err) { - fil_space_t* space; - char* filepath; - case DB_SUCCESS: - if (!table->no_rollback()) { - err = row_drop_ancillary_fts_tables(table, trx); - if (err != DB_SUCCESS) { - break; - } - } - - space = table->space; - ut_ad(!space || space->id == table->space_id); - /* Determine the tablespace filename before we drop - dict_table_t. */ - if (DICT_TF_HAS_DATA_DIR(table->flags)) { - dict_get_and_save_data_dir_path(table, true); - ut_ad(table->data_dir_path || !space); - } - - filepath = space - ? nullptr - : fil_make_filepath(table->data_dir_path, table->name, - IBD, - table->data_dir_path != nullptr); - - trx->mod_tables.erase(table); - dict_sys.remove(table); - - /* Do not attempt to drop known-to-be-missing tablespaces, - nor the system tablespace. */ - if (!space) { - fil_delete_file(filepath); - ut_free(filepath); - break; - } - - ut_ad(!filepath); - - if (space->id != TRX_SYS_SPACE) { - err = fil_delete_tablespace(space->id, false, - &detached_handle); - } - break; - - default: - /* This is some error we do not expect. Print - the error number and rollback the transaction */ - ib::error() << "Unknown error code " << err << " while" - " dropping table: " - << ut_get_name(trx, tablename) << "."; - - trx->error_state = DB_SUCCESS; - trx->rollback(); - trx->error_state = DB_SUCCESS; - - /* Mark all indexes available in the data dictionary - cache again. */ - - page_no = page_nos; - - for (dict_index_t* index = dict_table_get_first_index(table); - index != NULL; - index = dict_table_get_next_index(index)) { - index->lock.x_lock(SRW_LOCK_CALL); - ut_a(index->page == FIL_NULL); - index->page = *page_no++; - index->lock.x_unlock(); - } - } - - if (err != DB_SUCCESS && table != NULL) { - /* Drop table has failed with error but as drop table is not - transaction safe we should mark the table as corrupted to avoid - unwarranted follow-up action on this table that can result - in more serious issues. */ - - table->corrupted = true; - for (dict_index_t* index = UT_LIST_GET_FIRST(table->indexes); - index != NULL; - index = UT_LIST_GET_NEXT(indexes, index)) { - dict_set_corrupted(index, trx, "DROP TABLE"); - } - } - -funct_exit: - if (heap) { - mem_heap_free(heap); - } - -funct_exit_all_freed: - if (locked_dictionary) { - - if (trx_is_started(trx)) { - - trx_commit_for_mysql(trx); - } - - /* Add the table to fts queue if drop table fails */ - if (err != DB_SUCCESS && table->fts) { - fts_optimize_add_table(table); - } - - row_mysql_unlock_data_dictionary(trx); - } - - if (detached_handle != OS_FILE_CLOSED) { - os_file_close(detached_handle); - } - - trx->op_info = ""; - - DBUG_RETURN(err); -} - /****************************************************************//** Delete a single constraint. @return error code or DB_SUCCESS */ @@ -3707,7 +2815,6 @@ row_rename_table_for_mysql( const char* old_name, /*!< in: old table name */ const char* new_name, /*!< in: new table name */ trx_t* trx, /*!< in/out: transaction */ - bool commit, /*!< in: whether to commit trx */ bool use_fk) /*!< in: whether to parse and enforce FOREIGN KEY constraints */ { @@ -3718,14 +2825,12 @@ row_rename_table_for_mysql( ulint n_constraints_to_drop = 0; ibool old_is_tmp, new_is_tmp; pars_info_t* info = NULL; - int retry; char* is_part = NULL; ut_a(old_name != NULL); ut_a(new_name != NULL); ut_ad(trx->state == TRX_STATE_ACTIVE); - const bool dict_locked = trx->dict_operation_lock_mode == RW_X_LATCH; - ut_ad(!commit || dict_locked); + ut_ad(trx->dict_operation_lock_mode == RW_X_LATCH); if (high_level_read_only) { return(DB_READ_ONLY); @@ -3744,7 +2849,7 @@ row_rename_table_for_mysql( old_is_tmp = dict_table_t::is_temporary_name(old_name); new_is_tmp = dict_table_t::is_temporary_name(new_name); - table = dict_table_open_on_name(old_name, dict_locked, FALSE, + table = dict_table_open_on_name(old_name, true, false, DICT_ERR_IGNORE_FK_NOKEY); /* We look for pattern #P# to see if the table is partitioned @@ -3789,16 +2894,19 @@ row_rename_table_for_mysql( normalize_table_name_c_low( par_case_name, old_name, FALSE); #endif - table = dict_table_open_on_name(par_case_name, dict_locked, FALSE, + table = dict_table_open_on_name(par_case_name, true, false, DICT_ERR_IGNORE_FK_NOKEY); } if (!table) { err = DB_TABLE_NOT_FOUND; goto funct_exit; + } + + ut_ad(!table->is_temporary()); - } else if (!table->is_readable() && !table->space - && !(table->flags2 & DICT_TF2_DISCARDED)) { + if (!table->is_readable() && !table->space + && !(table->flags2 & DICT_TF2_DISCARDED)) { err = DB_TABLE_NOT_FOUND; @@ -3826,35 +2934,12 @@ row_rename_table_for_mysql( } } - /* Is a foreign key check running on this table? */ - for (retry = 0; retry < 100 - && table->n_foreign_key_checks_running > 0; ++retry) { - row_mysql_unlock_data_dictionary(trx); - std::this_thread::yield(); - row_mysql_lock_data_dictionary(trx); - } + err = trx_undo_report_rename(trx, table); - if (table->n_foreign_key_checks_running > 0) { - ib::error() << "In ALTER TABLE " - << ut_get_name(trx, old_name) - << " a FOREIGN KEY check is running. Cannot rename" - " table."; - err = DB_TABLE_IN_FK_CHECK; + if (err != DB_SUCCESS) { goto funct_exit; } - if (!table->is_temporary()) { - if (commit) { - dict_stats_wait_bg_to_stop_using_table(table, trx); - } - - err = trx_undo_report_rename(trx, table); - - if (err != DB_SUCCESS) { - goto funct_exit; - } - } - /* We use the private SQL parser of Innobase to generate the query graphs needed in updating the dictionary data from system tables. */ @@ -4106,16 +3191,8 @@ row_rename_table_for_mysql( } funct_exit: - if (table != NULL) { - if (commit && !table->is_temporary()) { - table->stats_bg_flag &= byte(~BG_STAT_SHOULD_QUIT); - } - dict_table_close(table, dict_locked, FALSE); - } - - if (commit) { - DEBUG_SYNC(trx->mysql_thd, "before_rename_table_commit"); - trx_commit_for_mysql(trx); + if (table) { + table->release(); } if (UNIV_LIKELY_NULL(heap)) { @@ -4305,28 +3382,3 @@ not_ok: goto loop; } - -/** Initialize this module */ -void row_mysql_init() -{ - mysql_mutex_init(row_drop_list_mutex_key, &row_drop_list_mutex, nullptr); - UT_LIST_INIT(row_mysql_drop_list, &row_mysql_drop_t::row_mysql_drop_list); - row_mysql_drop_list_inited= true; -} - -void row_mysql_close() -{ - ut_ad(!UT_LIST_GET_LEN(row_mysql_drop_list) || - srv_force_recovery >= SRV_FORCE_NO_BACKGROUND); - if (row_mysql_drop_list_inited) - { - row_mysql_drop_list_inited= false; - mysql_mutex_destroy(&row_drop_list_mutex); - - while (row_mysql_drop_t *drop= UT_LIST_GET_FIRST(row_mysql_drop_list)) - { - UT_LIST_REMOVE(row_mysql_drop_list, drop); - ut_free(drop); - } - } -} diff --git a/storage/innobase/row/row0purge.cc b/storage/innobase/row/row0purge.cc index db895b9a184..1269c7ae86e 100644 --- a/storage/innobase/row/row0purge.cc +++ b/storage/innobase/row/row0purge.cc @@ -111,8 +111,10 @@ row_purge_remove_clust_if_poss_low( MDL_ticket* mdl_ticket = nullptr; dict_table_t *table = nullptr; pfs_os_file_t f = OS_FILE_CLOSED; -retry: + if (table_id) { +retry: + purge_sys.check_stop_FTS(); dict_sys.mutex_lock(); table = dict_table_open_on_id( table_id, true, DICT_TABLE_OP_OPEN_ONLY_IF_CACHED, @@ -184,7 +186,7 @@ close_and_exit: mdl_ticket = nullptr; } } - fil_delete_tablespace(space_id, true, &f); + f = fil_delete_tablespace(space_id); } mtr.commit(); @@ -739,6 +741,12 @@ void purge_sys_t::wait_SYS() std::this_thread::sleep_for(std::chrono::seconds(1)); } +void purge_sys_t::wait_FTS() +{ + while (must_wait_FTS()) + std::this_thread::sleep_for(std::chrono::seconds(1)); +} + /** Reset DB_TRX_ID, DB_ROLL_PTR of a clustered index record whose old history can no longer be observed. @param[in,out] node purge node @@ -1034,11 +1042,13 @@ row_purge_parse_undo_rec( } try_again: + purge_sys.check_stop_FTS(); + node->table = dict_table_open_on_id( table_id, false, DICT_TABLE_OP_NORMAL, node->purge_thd, &node->mdl_ticket); - if (node->table == NULL || node->table->name.is_temporary()) { + if (!node->table) { /* The table has been dropped: no need to do purge and release mdl happened as a part of open process itself */ goto err_exit; diff --git a/storage/innobase/row/row0uins.cc b/storage/innobase/row/row0uins.cc index 51a5d30e7d3..fce9575aa9d 100644 --- a/storage/innobase/row/row0uins.cc +++ b/storage/innobase/row/row0uins.cc @@ -140,6 +140,27 @@ restart: mem_heap_free(heap); } else { switch (node->table->id) { + case DICT_COLUMNS_ID: + /* This is rolling back an INSERT into SYS_COLUMNS. + If it was part of an instant ALTER TABLE operation, we + must evict the table definition, so that it can be + reloaded after the dictionary operation has been + completed. At this point, any corresponding operation + to the metadata record will have been rolled back. */ + ut_ad(!online); + ut_ad(node->trx->dict_operation_lock_mode + == RW_X_LATCH); + ut_ad(node->rec_type == TRX_UNDO_INSERT_REC); + if (rec_get_n_fields_old(rec) + != DICT_NUM_FIELDS__SYS_COLUMNS + || (rec_get_1byte_offs_flag(rec) + ? rec_1_get_field_end_info(rec, 0) != 8 + : rec_2_get_field_end_info(rec, 0) != 8)) { + break; + } + static_assert(!DICT_FLD__SYS_COLUMNS__TABLE_ID, ""); + node->trx->evict_table(mach_read_from_8(rec)); + break; case DICT_INDEXES_ID: ut_ad(!online); ut_ad(node->trx->dict_operation_lock_mode @@ -154,6 +175,8 @@ restart: ut_ad("corrupted SYS_INDEXES record" == 0); } + pfs_os_file_t d = OS_FILE_CLOSED; + if (const uint32_t space_id = dict_drop_index_tree( &node->pcur, node->trx, &mtr)) { if (table) { @@ -185,36 +208,19 @@ restart: } } - fil_delete_tablespace(space_id, true); + d = fil_delete_tablespace(space_id); } mtr.commit(); + if (d != OS_FILE_CLOSED) { + os_file_close(d); + } + mtr.start(); success = btr_pcur_restore_position( BTR_MODIFY_LEAF, &node->pcur, &mtr); ut_a(success); - break; - case DICT_COLUMNS_ID: - /* This is rolling back an INSERT into SYS_COLUMNS. - If it was part of an instant ALTER TABLE operation, we - must evict the table definition, so that it can be - reloaded after the dictionary operation has been - completed. At this point, any corresponding operation - to the metadata record will have been rolled back. */ - ut_ad(!online); - ut_ad(node->trx->dict_operation_lock_mode - == RW_X_LATCH); - ut_ad(node->rec_type == TRX_UNDO_INSERT_REC); - if (rec_get_n_fields_old(rec) - != DICT_NUM_FIELDS__SYS_COLUMNS - || (rec_get_1byte_offs_flag(rec) - ? rec_1_get_field_end_info(rec, 0) != 8 - : rec_2_get_field_end_info(rec, 0) != 8)) { - break; - } - static_assert(!DICT_FLD__SYS_COLUMNS__TABLE_ID, ""); - node->trx->evict_table(mach_read_from_8(rec)); } } diff --git a/storage/innobase/row/row0upd.cc b/storage/innobase/row/row0upd.cc index 0a259726198..d68280cbeb3 100644 --- a/storage/innobase/row/row0upd.cc +++ b/storage/innobase/row/row0upd.cc @@ -281,21 +281,9 @@ row_upd_check_references_constraints( FALSE, FALSE, DICT_ERR_IGNORE_NONE); } - if (foreign_table) { - foreign_table->inc_fk_checks(); - } - - /* NOTE that if the thread ends up waiting for a lock - we will release dict_sys.latch temporarily! - But the inc_fk_checks() protects foreign_table from - being dropped while the check is running. */ - err = row_ins_check_foreign_constraint( FALSE, foreign, table, entry, thr); - if (foreign_table) { - foreign_table->dec_fk_checks(); - } if (ref_table != NULL) { dict_table_close(ref_table, FALSE, FALSE); } diff --git a/storage/innobase/srv/srv0mon.cc b/storage/innobase/srv/srv0mon.cc index eea86c2cae6..cd7e5c526d8 100644 --- a/storage/innobase/srv/srv0mon.cc +++ b/storage/innobase/srv/srv0mon.cc @@ -1090,11 +1090,6 @@ static monitor_info_t innodb_counter_info[] = MONITOR_NONE, MONITOR_DEFAULT_START, MONITOR_MASTER_IDLE_LOOPS}, - {"innodb_background_drop_table_usec", "server", - "Time (in microseconds) spent to process drop table list", - MONITOR_NONE, - MONITOR_DEFAULT_START, MONITOR_SRV_BACKGROUND_DROP_TABLE_MICROSECOND}, - {"innodb_log_flush_usec", "server", "Time (in microseconds) spent to flush log records", MONITOR_NONE, @@ -1189,11 +1184,6 @@ static monitor_info_t innodb_counter_info[] = MONITOR_NONE, MONITOR_DEFAULT_START, MONITOR_BACKGROUND_DROP_INDEX}, - {"ddl_background_drop_tables", "ddl", - "Number of tables in background drop table list", - MONITOR_NONE, - MONITOR_DEFAULT_START, MONITOR_BACKGROUND_DROP_TABLE}, - {"ddl_online_create_index", "ddl", "Number of indexes being created online", MONITOR_NONE, diff --git a/storage/innobase/srv/srv0srv.cc b/storage/innobase/srv/srv0srv.cc index 09f997d3d6d..74da09b605a 100644 --- a/storage/innobase/srv/srv0srv.cc +++ b/storage/innobase/srv/srv0srv.cc @@ -677,7 +677,6 @@ srv_boot(void) { srv_thread_pool_init(); trx_pool_init(); - row_mysql_init(); srv_init(); } @@ -1374,6 +1373,8 @@ void purge_sys_t::stop_SYS() /** Stop purge during FLUSH TABLES FOR EXPORT */ void purge_sys_t::stop() { + dict_sys.assert_not_locked(); + for (;;) { latch.wr_lock(SRW_LOCK_CALL); @@ -1471,10 +1472,7 @@ The master thread is tasked to ensure that flush of log file happens once every second in the background. This is to ensure that not more than one second of trxs are lost in case of crash when innodb_flush_logs_at_trx_commit != 1 */ -static -void -srv_sync_log_buffer_in_background(void) -/*===================================*/ +static void srv_sync_log_buffer_in_background() { time_t current_time = time(NULL); @@ -1496,8 +1494,6 @@ srv_shutdown_print_master_pending( /*==============================*/ time_t* last_print_time, /*!< last time the function print the message */ - ulint n_tables_to_drop, /*!< number of tables to - be dropped */ ulint n_bytes_merged) /*!< number of change buffer just merged */ { @@ -1506,11 +1502,6 @@ srv_shutdown_print_master_pending( if (difftime(current_time, *last_print_time) > 60) { *last_print_time = current_time; - if (n_tables_to_drop) { - ib::info() << "Waiting for " << n_tables_to_drop - << " table(s) to be dropped"; - } - /* Check change buffer merge, we only wait for change buffer merge if it is a slow shutdown */ if (!srv_fast_shutdown && n_bytes_merged) { @@ -1585,14 +1576,6 @@ srv_master_do_active_tasks(void) MONITOR_INC(MONITOR_MASTER_ACTIVE_LOOPS); - /* ALTER TABLE in MySQL requires on Unix that the table handler - can drop tables lazily after there no longer are SELECT - queries to them. */ - srv_main_thread_op_info = "doing background drop tables"; - row_drop_tables_for_mysql_in_background(); - MONITOR_INC_TIME_IN_MICRO_SECS( - MONITOR_SRV_BACKGROUND_DROP_TABLE_MICROSECOND, counter_time); - ut_d(srv_master_do_disabled_loop()); if (srv_shutdown_state > SRV_SHUTDOWN_INITIATED) { @@ -1646,17 +1629,6 @@ srv_master_do_idle_tasks(void) MONITOR_INC(MONITOR_MASTER_IDLE_LOOPS); - - /* ALTER TABLE in MySQL requires on Unix that the table handler - can drop tables lazily after there no longer are SELECT - queries to them. */ - ulonglong counter_time = microsecond_interval_timer(); - srv_main_thread_op_info = "doing background drop tables"; - row_drop_tables_for_mysql_in_background(); - MONITOR_INC_TIME_IN_MICRO_SECS( - MONITOR_SRV_BACKGROUND_DROP_TABLE_MICROSECOND, - counter_time); - ut_d(srv_master_do_disabled_loop()); if (srv_shutdown_state > SRV_SHUTDOWN_INITIATED) { @@ -1672,6 +1644,7 @@ srv_master_do_idle_tasks(void) return; } + ulonglong counter_time = microsecond_interval_timer(); srv_main_thread_op_info = "enforcing dict cache limit"; if (ulint n_evicted = dict_sys.evict_table_LRU(false)) { MONITOR_INC_VALUE( @@ -1692,7 +1665,6 @@ and optionally change buffer merge (on innodb_fast_shutdown=0). */ void srv_shutdown(bool ibuf_merge) { ulint n_bytes_merged = 0; - ulint n_tables_to_drop; time_t now = time(NULL); do { @@ -1700,11 +1672,6 @@ void srv_shutdown(bool ibuf_merge) ut_ad(srv_shutdown_state == SRV_SHUTDOWN_CLEANUP); ++srv_main_shutdown_loops; - /* FIXME: Remove the background DROP TABLE queue; it is not - crash-safe and breaks ACID. */ - srv_main_thread_op_info = "doing background drop tables"; - n_tables_to_drop = row_drop_tables_for_mysql_in_background(); - if (ibuf_merge) { srv_main_thread_op_info = "checking free log space"; log_free_check(); @@ -1717,10 +1684,10 @@ void srv_shutdown(bool ibuf_merge) /* Print progress message every 60 seconds during shutdown */ if (srv_print_verbose_log) { - srv_shutdown_print_master_pending( - &now, n_tables_to_drop, n_bytes_merged); + srv_shutdown_print_master_pending(&now, + n_bytes_merged); } - } while (n_bytes_merged || n_tables_to_drop); + } while (n_bytes_merged); } /** The periodic master task controlling the server. */ diff --git a/storage/innobase/srv/srv0start.cc b/storage/innobase/srv/srv0start.cc index 27b56d84563..9b51d225318 100644 --- a/storage/innobase/srv/srv0start.cc +++ b/storage/innobase/srv/srv0start.cc @@ -1764,9 +1764,6 @@ file_checked: && !srv_read_only_mode) { /* Drop partially created indexes. */ row_merge_drop_temp_indexes(); - /* Drop garbage tables. */ - row_mysql_drop_garbage_tables(); - /* Rollback incomplete non-DDL transactions */ trx_rollback_is_active = true; srv_thread_pool->submit_task(&rollback_all_recovered_task); @@ -1898,10 +1895,6 @@ void srv_shutdown_bg_undo_sources() ut_ad(!srv_read_only_mode); fts_optimize_shutdown(); dict_stats_shutdown(); - while (row_get_background_drop_list_len_low()) { - srv_inc_activity_count(); - std::this_thread::yield(); - } srv_undo_sources = false; } } @@ -2028,7 +2021,6 @@ void innodb_shutdown() dict_sys.close(); btr_search_sys_free(); - row_mysql_close(); srv_free(); fil_system.close(); pars_lexer_close(); diff --git a/storage/innobase/trx/trx0purge.cc b/storage/innobase/trx/trx0purge.cc index f1dce39c21d..300e664bf5a 100644 --- a/storage/innobase/trx/trx0purge.cc +++ b/storage/innobase/trx/trx0purge.cc @@ -694,7 +694,8 @@ not_free: /* Adjust the tablespace metadata. */ mysql_mutex_lock(&fil_system.mutex); - space.set_stopping(true); + ut_d(bool stopped=) space.set_stopping(); + ut_ad(!stopped); space.is_being_truncated = true; if (space.crypt_data) { space.reacquire(); @@ -806,7 +807,7 @@ not_free: mysql_mutex_lock(&fil_system.mutex); ut_ad(&space == purge_sys.truncate.current); ut_ad(space.is_being_truncated); - purge_sys.truncate.current->set_stopping(false); + purge_sys.truncate.current->clear_stopping(); purge_sys.truncate.current->is_being_truncated = false; mysql_mutex_unlock(&fil_system.mutex); diff --git a/storage/innobase/trx/trx0roll.cc b/storage/innobase/trx/trx0roll.cc index 4ccacaff683..662b62539c8 100644 --- a/storage/innobase/trx/trx0roll.cc +++ b/storage/innobase/trx/trx0roll.cc @@ -59,6 +59,8 @@ const trx_t* trx_roll_crash_recv_trx; @retval false if the rollback was aborted by shutdown */ inline bool trx_t::rollback_finish() { + mod_tables.clear(); + if (UNIV_LIKELY(error_state == DB_SUCCESS)) { commit(); @@ -89,6 +91,7 @@ inline bool trx_t::rollback_finish() undo= nullptr; } commit_low(); + commit_cleanup(); return false; } diff --git a/storage/innobase/trx/trx0trx.cc b/storage/innobase/trx/trx0trx.cc index 916d3e0bd2e..6b4436927c4 100644 --- a/storage/innobase/trx/trx0trx.cc +++ b/storage/innobase/trx/trx0trx.cc @@ -1427,6 +1427,10 @@ inline void trx_t::commit_in_memory(const mtr_t *mtr) ut_ad(!(lock.was_chosen_as_deadlock_victim & byte(~2U))); lock.was_chosen_as_deadlock_victim= false; #endif /* WITH_WSREP */ +} + +void trx_t::commit_cleanup() +{ mutex.wr_lock(); dict_operation= false; @@ -1452,16 +1456,19 @@ void trx_t::commit_low(mtr_t *mtr) ut_ad(!mtr == (aborted || !has_logged_or_recovered())); ut_ad(!mtr || !aborted); - /* undo_no is non-zero if we're doing the final commit. */ if (fts_trx && undo_no) { ut_a(!is_autocommit_non_locking()); - /* FTS-FIXME: Temporarily tolerate DB_DUPLICATE_KEY instead of - dying. This is a possible scenario if there is a crash between + /* MDEV-24088 FIXME: Invoke fts_commit() earlier (before possible + XA PREPARE), so that we will be able to return an error and rollback + the transaction, instead of violating consistency! + + The original claim about DB_DUPLICATE KEY was: + This is a possible scenario if there is a crash between insert to DELETED table committing and transaction committing. The fix would be able to return error from this function */ - if (dberr_t error= fts_commit(this)) - ut_a(error == DB_DUPLICATE_KEY); + if (ut_d(dberr_t error=) fts_commit(this)) + ut_ad(error == DB_DUPLICATE_KEY || error == DB_LOCK_WAIT_TIMEOUT); } #ifndef DBUG_OFF @@ -1498,7 +1505,7 @@ void trx_t::commit_low(mtr_t *mtr) } -void trx_t::commit() +void trx_t::commit_persist() { mtr_t *mtr= nullptr; mtr_t local_mtr; @@ -1511,6 +1518,15 @@ void trx_t::commit() commit_low(mtr); } + +void trx_t::commit() +{ + commit_persist(); + ut_d(for (const auto &p : mod_tables) ut_ad(!p.second.is_dropped())); + commit_cleanup(); +} + + /****************************************************************//** Prepares a transaction for commit/rollback. */ void diff --git a/storage/innobase/ut/ut0ut.cc b/storage/innobase/ut/ut0ut.cc index 41ba54d9a8d..d49fa9fb16c 100644 --- a/storage/innobase/ut/ut0ut.cc +++ b/storage/innobase/ut/ut0ut.cc @@ -428,8 +428,6 @@ ut_strerr( return("End of index"); case DB_IO_ERROR: return("I/O error"); - case DB_TABLE_IN_FK_CHECK: - return("Table is being used in foreign key check"); case DB_NOT_FOUND: return("not found"); case DB_ONLINE_LOG_TOO_BIG: diff --git a/storage/rocksdb/mysql-test/rocksdb/r/innodb_i_s_tables_disabled.result b/storage/rocksdb/mysql-test/rocksdb/r/innodb_i_s_tables_disabled.result index 7cedb9283a6..8f0357a8954 100644 --- a/storage/rocksdb/mysql-test/rocksdb/r/innodb_i_s_tables_disabled.result +++ b/storage/rocksdb/mysql-test/rocksdb/r/innodb_i_s_tables_disabled.result @@ -210,7 +210,6 @@ innodb_master_thread_sleeps server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL N innodb_activity_count server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Current server activity count innodb_master_active_loops server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times master thread performs its tasks when server is active innodb_master_idle_loops server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of times master thread performs its tasks when server is idle -innodb_background_drop_table_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to process drop table list innodb_log_flush_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to flush log records innodb_dict_lru_usec server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Time (in microseconds) spent to process DICT LRU list innodb_dict_lru_count_active server 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of tables evicted from DICT LRU list in the active loop @@ -227,7 +226,6 @@ dml_system_inserts dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 s dml_system_deletes dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Number of system rows deleted dml_system_updates dml 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 status_counter Number of system rows updated ddl_background_drop_indexes ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of indexes waiting to be dropped after failed index creation -ddl_background_drop_tables ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of tables in background drop table list ddl_online_create_index ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of indexes being created online ddl_pending_alter_table ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of ALTER TABLE, CREATE INDEX, DROP INDEX in progress ddl_sort_file_alter_table ddl 0 NULL NULL NULL 0 NULL NULL NULL NULL NULL NULL NULL 0 counter Number of sort files created during alter table |