| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
High level operation
Two DDL chains are used for CREATE OR REPLACE execution:
ddl_log_state_create (C) and ddl_log_state_rm (D).
1. (C) Write DDL_LOG_RENAME_TABLE_ACTION from normal to backup;
2. Rename table from normal to backup;
3. D is replayed only if C is closed (D is linked as a slave to C);
4. (D) Write DDL_LOG_DROP_ACTION of backup;
5. (C) Write DDL_LOG_CREATE_TABLE_ACTION of new table;
6. Create new table;
7. (C) update XID;
(In case of success)
8. (C) close chain;
9. (D) replay chain.
(In case of error)
8. (C) replay chain;
9. (D) close chain.
Backup table for CREATE OR REPLACE
- Instead of dropping the table, CREATE OR REPLACE renames table to
a backup name. The rename is written into ddl_log_state_create.
- CREATE TABLE without REPLACE is not affected by this commit.
- Engines having HTON_EXPENSIVE_RENAME flag set are not affected by
this commit.
- The backup is done inside mysql_rm_table_no_locks() to be able to
support ATOMIC DROP TABLE of multiple tables.
- The ddl logging of CREATE and DROP are linked together. CREATE
entries are executed before DROP entries.
- In case of failed CREATE OR REPLACE ddl_log_state_create is
replayed which restores original table.
- In case of succeeded CREATE OR REPLACE ddl_log_state_rm is
replayed which drops the backup (original table).
- CREATE TABLE .. SELECT XID usage is fixed and now there is no need
to log DROP TABLE via DDL_CREATE_TABLE_PHASE_LOG (see comments in
do_postlock()). XID is now correctly updated so it disables
DDL_LOG_DROP_TABLE_ACTION. Note that binary log is flushed at the
final stage when the table is ready. So if we have XID in the
binary log we don't need to drop the table.
- Three variations of CREATE OR REPLACE handled:
1. CREATE OR REPLACE TABLE t1 (..);
2. CREATE OR REPLACE TABLE t1 LIKE t2;
3. CREATE OR REPLACE TABLE t1 SELECT ..;
- Test case uses 5 combinations for engines (aria, aria_notrans,
myisam, ib, expensive_rename) and 2 combinations for binlog types
(row, stmt). Combinations help to check differences between the
results. Error failures are tested for the above three variations.
- Triggers mechanism is unaffected by this change. This is tested in
create_replace.test.
- Triggers mechanism is unaffected which is tested.
- LOCK TABLES is affected because RENAME TABLE cannot work on locked
table. mysql_do_rename() (original do_rename()) can now work under
locked_tables_mode.
On XID usage
Like with all other atomic DDL operations XID is used to avoid
inconsistency between master and slave in the case of a crash after
binary log is written and before ddl_log_state_create is closed. On
recovery XIDs are taken from binary log and corresponding DDL log
events get disabled. That is done by
ddl_log_close_binlogged_events().
On linking two chains together
Chains are executed in the ascending order of entry_pos of execute
entries. But entry_pos assignment order is undefined: it may assign
bigger number for the first chain and then smaller number for the
second chain. So the execution order in that case will be reverse:
second chain will be executed first.
|
| |
|
|
| |
Improves readability of DDL log debug traces.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
The issue was that we sent two different signals to different threads
after each other. The DEBUG_SYNC functionality cannot handle this (as
the signal is stored in a global variable) and the first one can get
lost.
Fixed by using the same signal for both threads.
|
| |
|
|
|
|
|
|
|
| |
When doing CREATE TABLE ... ENGINE=MyISAM SELECT ...
the MyISAM table was not completely flushed at commit
which gave some warnings about table not closed in the
atomic test suite.
Fixed by flushing the table in select_insert::prepare_eof().
|
| |
|
|
|
|
|
| |
- Rename DDL_IGNORE_LOG_ENTRY_CODE to DDL_LOG_IGNORE_ENTRY_CODE
This makes it similar to all other ddl_log_entry_code's.
- Added some new comments
- ddl_log_revert() now returns != 0 if revert didn't succeed
|
| | |
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
Do not execute user callback just after pwrite. Instead, submit user
function as task into thread pool. This way, the IO thread would not hog
aiocb, which is a limited (in Innodb) resource
|
| | |
| |
| |
| |
| |
| |
| | |
Add fixed for tests mysqld--help,aix.rdiff and sysvars_server_notembedded,aix.rdiff
AIX couldn't compile in embedded mode so leaving sysvars_server_embedded
for later (if required).
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When doing a ALTER TABLE ... RENAME, MariaDB doesn't rename
original table to #sql-backup, which it does in other cases,
but insteads drops the original table directly. However
this optimization doesn't work in case of InnoDB table
with a foreign key constraint.
During copy algorithm, InnoDB fails to rename the foreign key
constraint(MDEV-25855). With this optimisation, InnoDB also
fails to drop the original table because the table has
FOREIGN Key constraint exist in INNODB_SYS_FOREIGN table.
This leads to orphan .ibd file in InnoDB dictionary.
so disabling this optimization when FK is involved.
Reviewer: monty@mariadb.org
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Not all environments have 'diff' installed. Most notably CentOS 8
does not have diff out-of-the-box. Thus users running 'cmake .' and
'make' would fail to build MariaDB, and they would think the error
was in ABI incompatibilities due to the error message emitted by CMake
when in reality simply 'diff' was missing.
This fixes it and makes the developer experience better by simply skipping
the diffing if 'diff' is not found.
## Proof
Running in a clean CentOS 8 container.
### Before
```
$ cmake .
...
-- Looking for cpuid.h
-- Looking for cpuid.h - found
-- Looking for x86intrin.h
-- Looking for x86intrin.h - found
-- Looking for event.h
-- Looking for event.h - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /build
$ make
Scanning dependencies of target abi_check
CMake Error at cmake/do_abi_check.cmake:82 (MESSAGE):
ABI check found difference between /build/include/mysql/plugin_audit.h.pp
and /build/abi_check.out
make[2]: *** [CMakeFiles/abi_check.dir/build.make:57: CMakeFiles/abi_check] Error 1
make[1]: *** [CMakeFiles/Makefile2:168: CMakeFiles/abi_check.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
$ yum install -y diffutils
...
Installed:
diffutils-3.6-6.el8.x86_64
Complete!
$ make
[ 0%] Built target abi_check
Scanning dependencies of target INFO_BIN
[ 0%] Built target INFO_BIN
Scanning dependencies of target INFO_SRC
[ 0%] Built target INFO_SRC
Scanning dependencies of target wsrep_api_v26
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_dummy.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_gtid.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_loader.c.o
[ 0%] Building C object wsrep-lib/wsrep-API/CMakeFiles/wsrep_api_v26.dir/v26/wsrep_uuid.c.o
[ 0%] Linking C static library libwsrep_api_v26.a
[ 0%] Built target wsrep_api_v26
```
### After
```
$ make
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_audit.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_ftparser.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/psi/psi_abi_v1.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/psi/psi_abi_v2.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/client_plugin.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_auth.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_password_validation.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_encryption.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_data_type.h skipped.
Command 'diff' not found. ABI check for /build/server/include/mysql/plugin_function.h skipped.
[ 0%] Built target abi_check
[ 0%] Built target INFO_SRC
[ 0%] Built target INFO_BIN
[ 0%] Built target wsrep_api_v26
[ 0%] Building CXX object wsrep-lib/src/CMakeFiles/wsrep-lib.dir/server_state.cpp.o
```
If diff is installed, those warnings are simply not shown. Builds pass
without the need to install 'diff'.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This test failed in case it was run in PS mode and for embedded server
The reason of test failure was that the following fields
affected_row, server_status, insert_id
of the structure MYSQL_STMT weren't update on calling
mysql_stmt_next_result() in case combination of binary protocol
and embedded server used.
|
| | |
| |
| |
| |
| |
| | |
fixes a bunch of tests failures in --ps --embed
also, don't use cli_advanced_command().
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
do not try to detect metadata change (and reprepare) for
internal short-lived TABLE_LIST objects that couldn't have
possibly lived long enough to see prepare and cache the metadata.
|
| | | |
|
| | |
| |
| |
| | |
--enable_prepared_warnings/--disable_prepared_warnings)
|
| | | |
|
| | |
| |
| |
| |
| | |
' Access denied; you need (at least one of) the SUPER privilege(s) for this operation
on executing SET system_variable=.... in PS mode
|
| | |
| |
| |
| | |
prepare response packet
|
| | |
| |
| |
| | |
--enable-prepared-warnings in mysqltest
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
on response to COM_STMT_EXECUTE
The test cases like the following one
delimiter |;
CREATE PROCEDURE SP001()
BEGIN
DECLARE C1 CURSOR FOR SELECT 1;
OPEN C1;
SELECT 1;
CLOSE C1;
CLOSE C1;
END|
delimiter ;|
--error 1326
call SP001();
are failed since processing of ERR packet was missed by mysqltest
in case it is run with --ps-protocol
Additionally, the test sp-error was changed to don't run multi-statements
since they are not supported by PS protocol
|
| | |
| |
| |
| |
| |
| |
| | |
Added the directive --disable_ps_protocol before running tests
that checking correct handling of ER_STACK_OVERRUN_NEED_MORE.
These tests rely on estimation of stack consumption that differ
in case statements run in regular and in ps mode.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This test has been disable by the reason that it contains multi-statements
that not supported in PS mode. Unfortunately, some of these multi-statements
can't be converted to a sequence of single-statements by the reason that
multi-statementness is a requirement for tests (e.g. tests for
the MySQL bugs #5307, #8762) therefore the whole test file will be
skipped in case it is run with --ps-protocol
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
Fixed the bug in handling queries where select list contains subqueries
with items referencing to items in outer query
|
| | |
| |
| |
| | |
Fixed the test main.view
|
| | |
| |
| |
| |
| |
| | |
prepared statement produces different results for UPDATE with subquery
10.6 cleanup
|
| | |
| |
| |
| | |
Fixed failures of the tests main.compound, main.union
|
| | |
| |
| |
| |
| | |
Fixed test failures caused by missing output of warnings that
got on prepare phase
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Write a log event to the general log before executing a statemment in PS mode
This change fixes failure of the test main.log_tables when it is run in PS mode
Updated the test main.events_logs_tests to query mysql.general_log
for the command_type 'Query' and 'Execute' in order to have consistent
test result both in case the test is run with --ps-protocol and without it
|
| | |
| |
| |
| |
| |
| | |
Disable running of the statements
PREPARE FROM/EXECUTE/EXECUTE IMMEDIATE/DEALLOCATE
in PS mode. Adjust tests main.ps, main.ps_1general
|
| | |
| |
| |
| |
| | |
Extended a set of commands that can be executed via binary protocol
by a user with expired password
|
| | |
| |
| |
| |
| | |
Extended a set of commands that can be executed as prepared statements
by a user with expired password
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Withing this task the following changes were made:
- Added sending of metadata info in prepare phase for the admin related
command (check table, checksum table, repair, optimize, analyze).
- Refactored implmentation of HELP command to support its execution in
PS mode
- Added support for execution of LOAD INTO and XA- related statements
in PS mode
- Modified mysqltest.cc to run statements in PS mode unconditionally
in case the option --ps-protocol is set. Formerly, only those statements
were executed using PS protocol that matched the hard-coded regular expression
- Fixed the following issues:
The statement
explain select (select 2)
executed in regular and PS mode produces different results:
MariaDB [test]> prepare stmt from "explain select (select 2)";
Query OK, 0 rows affected (0,000 sec)
Statement prepared
MariaDB [test]> execute stmt;
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
| 2 | SUBQUERY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
2 rows in set (0,000 sec)
MariaDB [test]> explain select (select 2);
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+------+-------------+-------+------+---------------+------+---------+------+------+----------------+
1 row in set, 1 warning (0,000 sec)
In case the statement
CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
is run in PS mode it fails with the error
ERROR 1054 (42S22): Unknown column 'a' in 'field list'.
- Uniform handling of read-only variables both in case the SET var=val
statement is executed as regular or prepared statememt.
- Fixed assertion firing on handling LOAD DATA statement for temporary tables
- Relaxed assert condition in the function lex_end_stage1() by adding
the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
SQLCOM_CREATE_PACKAGE_BODY to a list of supported command
- Removed raising of the error ER_UNSUPPORTED_PS in the function
check_prepared_statement() for the ALTER VIEW command
- Added initialization of the data memember st_select_lex_unit::last_procedure
(assign NULL value) in the constructor
Without this change the test case main.ctype_utf8 fails with the following
report in case it is run with the optoin --ps-protocol.
mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection
- The following bug reports were fixed:
MDEV-24460: Multiple rows result set returned from stored
routine over prepared statement binary protocol is
handled incorrectly
CONC-519: mariadb client library doesn't handle server_status and
warnign_count fields received in the packet
COM_STMT_EXECUTE_RESPONSE.
Reasons for these bug reports have the same nature and caused by
missing loop iteration on results sent by server in response to
COM_STMT_EXECUTE packet.
Enclosing of statements for processing of COM_STMT_EXECUTE response
in the construct like
do
{
...
} while (!mysql_stmt_next_result());
fixes the above mentioned bug reports.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
In commit 1c5ae9919463ee0fc1d65fdfc7ec39534495ecaf (MDEV-25666)
we had changed Mariabackup so that it would no longer skip files
whose names start with #sql. This turned out to be wrong.
Because operations on such named files are not protected by any
locks in the server, it is not safe to copy them.
Not copying the files may make the InnoDB data dictionary
inconsistent with the file system. So, we must do something
in InnoDB to adjust for that.
If InnoDB is being started up without the redo log (ib_logfile0)
or with a zero-length log file, we will assume that the server
was restored from a backup, and adjust things as follows:
dict_check_sys_tables(), fil_ibd_open(): Do not complain about
missing #sql files if they would be dropped a little later.
dict_stats_update_if_needed(): Never add #sql tables to
the recomputing queue. This avoids a potential race condition when
dropping the garbage tables.
drop_garbage_tables_after_restore(): Try to drop any garbage tables.
innodb_ddl_recovery_done(): Invoke drop_garbage_tables_after_restore()
if srv_start_after_restore (a new flag) was set and we are not in
read-only mode (innodb_read_only=ON or innodb_force_recovery>3).
The tests and dbug_mariabackup_event() instrumentation
were developed by Vladislav Vaintroub, who also reviewed this.
|
| | |
| |
| |
| |
| |
| |
| | |
Due to wsrep uses its own xid format for its recovery,
the xid hashing has to be refined.
When a xid object is not in the server "mysql" format,
the hash record made to contain the xid also in the full format.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
In commit 1bd681c8b3c5213ce1f7976940a7dc38b48a0d39 we introduced
a work-around for the missing MDL protection when the internal
tables of FULLTEXT INDEX are dropped during DDL operations.
That work-around suffered from a race condition. A purge thread
could have narrowly passed purge_sys.check_stop_FTS() and then
(while holding dict_sys.mutex) acquire a table reference right
after fts_lock_table() determined that no references were being
held.
fts_lock_table(): Protect the reference check with dict_sys.mutex.
Thanks to Thirunarayanan Balathandayuthapani for repeating the
failure and testing the fix.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Before any committed DDL operation returns control to the caller,
we must ensure that it will have been durably committed before the
ddl_log state may be changed, no matter if
innodb_flush_log_at_trx_commit=0 is being used.
Operations that would involve deleting files were already safe,
because the durable write of the FILE_DELETE record that would precede
the file deletion would also have made the commit durable.
Furthermore, when we clean up ADD INDEX stubs that were left behind
by a previous ha_innobase::commit_inplace_alter_table(commit=false)
when MDL could not be acquired, we will use the same interface as
for dropping indexes. This safety measure should be dead code,
because ADD FULLTEXT INDEX is not supported online, and dropping indexes
only involves file deletion for FULLTEXT INDEX.
|
| | |
| |
| |
| |
| |
| |
| | |
ha_innobase::extra(HA_EXTRA_END_ALTER_COPY): Flush the redo log
because the main transaction will not have written any log and
thus will not initiate any log write either. But the DDL recovery
layer expects that the operation will have been committed.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
1. work around MDEV-25912 to not apply assert
at wsrep running time;
2. handle wsrep mode of the server recovery
3. convert hton calls to static binlog_commit ones.
4. satisfy MSAN complain on uninitialized std::pair
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
- Replace strncpy() with strmake() to ensure things are \0 terminated
- Initialized some variables that caused compiler warnings
- Increased buffer that caused warnings from strncpy
One warning from InnoDB, others from Connect.
|
