From 1864d9596d9531427f1d63f86ada9e0cb21b51db Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Fri, 28 Sep 2012 02:06:56 +0300 Subject: Implementation of Multi-source replication (MDEV:253) Documentation of the feature can be found at: http://kb.askmonty.org/en/multi-source-replication/ This code is based on code from Taobao, developed by Plinux BUILD/SETUP.sh: Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it) client/mysqltest.cc: Added support for error names starting with 'W' Added connection_name support to --sync_with_master cmake/maintainer.cmake: Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it) mysql-test/r/mysqltest.result: Updated results mysql-test/r/parser.result: Updated results mysql-test/suite/multi_source/my.cnf: Setup of multi-master tests mysql-test/suite/multi_source/simple.result: Simple basic test of multi-source functionality mysql-test/suite/multi_source/simple.test: Simple basic test of multi-source functionality mysql-test/suite/multi_source/syntax.result: Test of multi-source syntax mysql-test/suite/multi_source/syntax.test: Test of multi-source syntax mysql-test/suite/rpl/r/rpl_rotate_logs.result: Updated results because of new error messages mysql-test/t/parser.test: Updated test as master_pos_wait() now takes more arguments than before sql/event_scheduler.cc: No reason to initialize slave_thread (it's guaranteed to be zero here) sql/item_create.cc: Added connection_name argument to master_pos_wait() Simplified code sql/item_func.cc: Added connection_name argument to master_pos_wait() sql/item_func.h: Added connection_name argument to master_pos_wait() sql/log.cc: Added tag "Master 'connection_name'" to slave errors that has a connection name. sql/mysqld.cc: Added variable mysqld_server_initialized so that other functions can test if server is fully initialized. Free all slave data in one place (fewer ifdef's) Removed not needed call to close_active_mi() Initialize slaves() later in startup to ensure that everthing is really initialized when slaves start. Made status variable slave_running multi-source safe sql/mysqld.h: Added mysqld_server_initialized sql/rpl_mi.cc: Store connection name and cmp_connection_name (only used for show full slave status) in Master_info Added code for Master_info_index, which handles storage of multi-master information Don't write the empty "" connection_name to multi-master.info file. This is handled by the original code. sql/rpl_mi.h: Added connection_name and Master_info_index sql/rpl_rli.cc: Added connection_name to relay log files. sql/rpl_rli.h: Fixed type of slave_skip_counter as we now access it directly in sys_vars.cc, so it must be uint sql/share/errmsg-utf8.txt: Added new error messages needed for multi-source Added multi-source name to error ER_MASTER_INFO and WARN_NO_MASTER_INFO sql/slave.cc: Moved things a bit around to make it easier to handle error conditions. Create a global master_info_index and add the "" connection to it Ensure that new Master_info doesn't fail. Don't call terminate_slave_threads(active_mi..) on end_slave() as this is now done automaticly when deleting master_info_index. Delete not needed function close_active_mi(). One can achive same thing by calling end_slave(). Added support for SHOW FULL SLAVE STATUS (show status for all master connections with connection_name as first column) sql/slave.h: Added new prototypes sql/sql_base.cc: More DBUG_PRINT sql/sql_class.cc: Reset thd->connection_name and thd-->default_master_connection sql/sql_class.h: Added thd->connection_name and thd-->default_master_connection Added slave_skip_count to variables to make changing the @@sql_slave_skip_count variable thread safe sql/sql_const.h: Added MAX_CONNECTION_NAME sql/sql_lex.cc: Reset 'lex->verbose' (to simplify some sql_yacc.yy code) sql/sql_lex.h: Added connection_name sql/sql_parse.cc: Added support for connection_name to all SLAVE commands. - Instead of using active_mi, we now get the current Master_info from master_info_index. - Create new replication threads with CHANGE MASTER - Added support for show_all_master_info() sql/sql_reload.cc: Made reset/full slave use master_info_index->get_master_info() instead of active_mi. If one uses 'RESET SLAVE "connection_name" all' the connection is removed from master_info_index. sql/sql_repl.cc: sql_slave_skip_counter is moved to thd->variables to make it thread safe and fix some bugs with it Add connection name to relay log files. Added connection name to errors. Added some logging for multi-master if log_warnings > 1 stop_slave(): - Don't check if thd is set. It's guaranteed to always be set. change_master(): - Check for duplicate connection names in change_master() - Check for wrong arguments first in file (to simplify error handling) - Register new connections in master_info_index sql/sql_yacc.yy: Added optional connection_name to a all relevant master/slave commands sql/strfunc.cc: my_global.h shoud always be included first. sql/sys_vars.cc: Added variable default_master_connection Made variable sql_slave_skip_counter multi-source safe sql/sys_vars.h: Added Sys_var_session_lexstring (needed for default_master_connection) Added Sys_var_multi_source_uint (needed for sql_slave_skip_counter). --- mysql-test/suite/multi_source/skip_counter.test | 126 ++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 mysql-test/suite/multi_source/skip_counter.test (limited to 'mysql-test/suite/multi_source/skip_counter.test') diff --git a/mysql-test/suite/multi_source/skip_counter.test b/mysql-test/suite/multi_source/skip_counter.test new file mode 100644 index 00000000000..fe5df69097a --- /dev/null +++ b/mysql-test/suite/multi_source/skip_counter.test @@ -0,0 +1,126 @@ +# +# Test of sql_slave_skip_counter +# + +--enable_connect_log + +# Create a schema and a table i +# on the 1st master + +--connect (master1,127.0.0.1,root,,,$SERVER_MYPORT_1) + +--disable_warnings +drop database if exists db; +--enable_warnings +create database db; +create table db.t1 (i int) engine=MyISAM; +--save_master_pos + + +# Create the same schema and another table +# on the 2nd master + +--connect (master2,127.0.0.1,root,,,$SERVER_MYPORT_2) + +--disable_warnings +drop database if exists db; +--enable_warnings +create database db; +create table db.t2 (i int) engine=MyISAM; + +--connect (slave,127.0.0.1,root,,,$SERVER_MYPORT_3) +--disable_connect_log + +# Start replication from the first master + +--replace_result $SERVER_MYPORT_1 MYPORT_1 +eval change master 'master1' to +master_port=$SERVER_MYPORT_1, +master_host='127.0.0.1', +master_user='root'; + +start slave 'master1'; +set default_master_connection = 'master1'; +--source include/wait_for_slave_to_start.inc +--sync_with_master 0,'master1' + +# Start replication from the second master + +set default_master_connection = 'master2'; + +--replace_result $SERVER_MYPORT_2 MYPORT_2 +eval change master 'master2' to +master_port=$SERVER_MYPORT_2, +master_host='127.0.0.1', +master_user='root'; + +# the schema creation will be replicated from the 1st master, +# so we want to skip it in the second replication connection. + +# Normally it should have been 2 events, but +# currently Binlog_checkpoint also counts. Maybe we'll need +# to modify the test later + +--let $skip_counter_saved = `select @@global.sql_slave_skip_counter` +set global sql_slave_skip_counter = 2; +select @@global.sql_slave_skip_counter; +select @@session.sql_slave_skip_counter; +set session sql_slave_skip_counter = 3; +select @@global.sql_slave_skip_counter; +select @@session.sql_slave_skip_counter; +set global sql_slave_skip_counter= default; +select @@global.sql_slave_skip_counter; +select @@session.sql_slave_skip_counter; +set global sql_slave_skip_counter= 3; +set default_master_connection = 'master1'; +select @@session.sql_slave_skip_counter; +set default_master_connection = 'qqq'; +select @@session.sql_slave_skip_counter; +set default_master_connection = 'master2'; +select @@session.sql_slave_skip_counter; + +start slave 'master2'; +--source include/wait_for_slave_to_start.inc +set default_master_connection = ''; + +--enable_connect_log + +--connection master2 +--save_master_pos + +--connection slave + +--disable_connect_log +--sync_with_master 0,'master2' + +# If the skip_counter worked as expected, we should +# get here (replication shouldn't have broken) +# and should see both tables here +# (drop database which came from master2 shoudn't have been executed +# so t1 should still exist) + +show tables in db; + +# Cleanup + +drop database db; +set default_master_connection = 'master1'; +stop slave; + +--source include/wait_for_slave_to_stop.inc +set default_master_connection = 'master2'; +stop slave; + +--source include/wait_for_slave_to_stop.inc +--eval set global sql_slave_skip_counter = $skip_counter_saved + +--enable_connect_log +--disconnect slave + +--connection master1 +drop database db; +--disconnect master1 + +--connection master2 +drop database db; +--disconnect master2 -- cgit v1.2.1