diff options
author | Rohit Kalhans <rohit.kalhans@oracle.com> | 2011-09-29 14:47:27 +0530 |
---|---|---|
committer | Rohit Kalhans <rohit.kalhans@oracle.com> | 2011-09-29 14:47:27 +0530 |
commit | 586c0c0ef6e23c4ce3c63f796187178f15e7010a (patch) | |
tree | 8e4e3ce5c98f1dc025cbdf4b892d3103536f40d0 | |
parent | af400ee256fe94f6a4294041105ea83059c9c1bc (diff) | |
download | mariadb-git-586c0c0ef6e23c4ce3c63f796187178f15e7010a.tar.gz |
BUG#11758262 - 50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL
Problem: The following statements can cause the slave to go out of sync
if logged in statement format:
INSERT IGNORE...SELECT
INSERT ... SELECT ... ON DUPLICATE KEY UPDATE
REPLACE ... SELECT
UPDATE IGNORE :
CREATE ... IGNORE SELECT
CREATE ... REPLACE SELECT
Background: Since the order of the rows returned by the SELECT
statement or otherwise may differ on master and slave, therefore
the above statements may cuase the salve to go out of sync with
the master.
Fix:
Issue a warning when statements like the above are exectued and
the bin-logging format is statement. If the logging format is mixed,
use row based logging. Marking a statement as unsafe has been
done in the sql/sql_parse.cc instead of sql/sql_yacc.cc, because while
parsing for a token has been done we cannot be sure if the parsing
of the other tokens has been done as well.
Six new warning messages has been added for each unsafe statement.
binlog.binlog_unsafe.test has been updated to incoporate these additional unsafe statments.
******
BUG#11758262 - 50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL
Problem: The following statements can cause the slave to go out of sync
if logged in statement format:
INSERT IGNORE...SELECT
INSERT ... SELECT ... ON DUPLICATE KEY UPDATE
REPLACE ... SELECT
UPDATE IGNORE :
CREATE ... IGNORE SELECT
CREATE ... REPLACE SELECT
Background: Since the order of the rows returned by the SELECT
statement or otherwise may differ on master and slave, therefore
the above statements may cuase the salve to go out of sync with
the master.
Fix:
Issue a warning when statements like the above are exectued and
the bin-logging format is statement. If the logging format is mixed,
use row based logging. Marking a statement as unsafe has been
done in the sql/sql_parse.cc instead of sql/sql_yacc.cc, because while
parsing for a token has been done we cannot be sure if the parsing
of the other tokens has been done as well.
Six new warning messages has been added for each unsafe statement.
binlog.binlog_unsafe.test has been updated to incoporate these additional unsafe statments.
mysql-test/extra/rpl_tests/rpl_insert_duplicate.test:
Test removed: Added the test to rpl.rpl_insert_ignore.test
******
Test removed: the test is redundant as the same is being tested in rpl.rpl_insert_ignore.
mysql-test/extra/rpl_tests/rpl_insert_id.test:
Warnings disabled for the unsafe statements.
mysql-test/extra/rpl_tests/rpl_insert_ignore.test:
1. Disabled warnings while for unsafe statements
2. As INSERT...IGNORE is an unsafe statement, an insert ignore not changing any rows,
will not be logged in the binary log, in the ROW and MIXED modes. It will however be logged
in STATEMENT mode.
mysql-test/r/commit_1innodb.result:
updated result file
******
updated result file
mysql-test/suite/binlog/r/binlog_stm_blackhole.result:
Updated result file.
mysql-test/suite/binlog/r/binlog_unsafe.result:
updated result file
mysql-test/suite/binlog/t/binlog_unsafe.test:
added tests for the statements marked as unsafe.
mysql-test/suite/rpl/r/rpl_insert_duplicate.result:
File Removed :Result file of rpl_insert_duplicate, which has been removed.
mysql-test/suite/rpl/r/rpl_insert_ignore.result:
Added the content of rpl.rpl_insert_duplicate here.
mysql-test/suite/rpl/r/rpl_insert_select.result:
Result file removed as the corresponding test has beenn removed.
mysql-test/suite/rpl/r/rpl_known_bugs_detection.result:
Updated result file.
mysql-test/suite/rpl/t/rpl_insert_duplicate.test:
File Removed: this was a wrapper for rpl.rpl_insert_duplicate.test, which has been removed.
mysql-test/suite/rpl/t/rpl_insert_select.test:
File Removed: This test became redundant after this fix, This test showed how INSERT IGNORE...SELECT break replication, which has been handled in this fix.
mysql-test/suite/rpl/t/rpl_known_bugs_detection.test:
Since all the tests are statement based bugs are being tested, having mixed format
forces the event to be written in row format. When the statement and causes the
test to fail as certain known bugs do not occur when the even is logged in row format.
sql/share/errmsg-utf8.txt:
added 6 new Warning messages.
******
added 6 new Warning messages.
sql/sql_lex.cc:
Added 6 new error Identifier [ER_BINLOG_STMT_UNSAFEE_*]
sql/sql_lex.h:
Added 6 new BINLOG_STMT_UNSAFE_* enums to identify the type of unsafe statement dealt with in this bug.
******
Added 6 new BINLOG_STMT_UNSAFE_* enums to identify the type of unsafe statement dealt with in this bug.
sql/sql_parse.cc:
added check for specific queries and marked them as unsafe.
******
added check for specific queries and marked them as unsafe.
-rw-r--r-- | mysql-test/extra/rpl_tests/rpl_insert_duplicate.test | 59 | ||||
-rw-r--r-- | mysql-test/extra/rpl_tests/rpl_insert_id.test | 29 | ||||
-rw-r--r-- | mysql-test/extra/rpl_tests/rpl_insert_ignore.test | 51 | ||||
-rw-r--r-- | mysql-test/r/commit_1innodb.result | 4 | ||||
-rw-r--r-- | mysql-test/suite/binlog/r/binlog_stm_blackhole.result | 2 | ||||
-rw-r--r-- | mysql-test/suite/binlog/r/binlog_unsafe.result | 38 | ||||
-rw-r--r-- | mysql-test/suite/binlog/t/binlog_unsafe.test | 48 | ||||
-rw-r--r-- | mysql-test/suite/rpl/r/rpl_insert_duplicate.result | 29 | ||||
-rw-r--r-- | mysql-test/suite/rpl/r/rpl_insert_ignore.result | 26 | ||||
-rw-r--r-- | mysql-test/suite/rpl/r/rpl_insert_select.result | 14 | ||||
-rw-r--r-- | mysql-test/suite/rpl/r/rpl_known_bugs_detection.result | 4 | ||||
-rw-r--r-- | mysql-test/suite/rpl/t/rpl_insert_duplicate.test | 14 | ||||
-rw-r--r-- | mysql-test/suite/rpl/t/rpl_insert_select.test | 20 | ||||
-rw-r--r-- | mysql-test/suite/rpl/t/rpl_known_bugs_detection.test | 2 | ||||
-rw-r--r-- | sql/share/errmsg-utf8.txt | 18 | ||||
-rw-r--r-- | sql/sql_lex.cc | 6 | ||||
-rw-r--r-- | sql/sql_lex.h | 42 | ||||
-rw-r--r-- | sql/sql_parse.cc | 40 |
18 files changed, 294 insertions, 152 deletions
diff --git a/mysql-test/extra/rpl_tests/rpl_insert_duplicate.test b/mysql-test/extra/rpl_tests/rpl_insert_duplicate.test deleted file mode 100644 index a81eeba3231..00000000000 --- a/mysql-test/extra/rpl_tests/rpl_insert_duplicate.test +++ /dev/null @@ -1,59 +0,0 @@ -# BUG#59338 Inconsistency in binlog for statements that don't change any rows STATEMENT SBR -# In SBR, if a statement does not fail, it is always written to the binary log, -# regardless if rows are changed or not. If there is a failure, a statement is -# only written to the binary log if a non-transactional (.e.g. MyIsam) engine -# is updated. INSERT ON DUPLICATE KEY UPDATE was not following the rule above -# and was not written to the binary log, if then engine was Innodb. -# -# In this test case, we check if INSERT ON DUPLICATE KEY UPDATE that does not -# change anything is still written to the binary log. - -# Prepare environment ---connection master - -eval CREATE TABLE t1 ( - a INT UNSIGNED NOT NULL PRIMARY KEY -) ENGINE=$engine_type; - -eval CREATE TABLE t2 ( - a INT UNSIGNED -) ENGINE=$engine_type; - -INSERT INTO t1 VALUES (1); -INSERT INTO t2 VALUES (1); - -# An insert duplicate that does not update anything must be written to the binary -# log in SBR and MIXED modes. We check this property by summing a before and after -# the update and comparing the binlog positions. The sum should be the same at both -# points and the statement should be in the binary log. ---let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1) ---let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1) ---let $statement_file=INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a ---eval $statement_file - ---let $assert_cond= SUM(a) = 1 FROM t1 ---let $assert_text= Sum of elements in t1 should be 1. ---source include/assert.inc - -if (`SELECT @@BINLOG_FORMAT = 'ROW'`) -{ - --let $binlog_position_cmp= = - --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start - --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. -} -if (`SELECT @@BINLOG_FORMAT != 'ROW'`) -{ - --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 1, 1\', Info, 1]\' LIKE \'%$statement_file\' - --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. -} ---source include/assert.inc - -# Compare master and slave ---sync_slave_with_master ---let $diff_tables= master:test.t1 , slave:test.t1 ---source include/diff_tables.inc - -# Clean up ---connection master -drop table t1, t2; ---sync_slave_with_master diff --git a/mysql-test/extra/rpl_tests/rpl_insert_id.test b/mysql-test/extra/rpl_tests/rpl_insert_id.test index 565ab4a67ad..32d861bd45a 100644 --- a/mysql-test/extra/rpl_tests/rpl_insert_id.test +++ b/mysql-test/extra/rpl_tests/rpl_insert_id.test @@ -77,6 +77,7 @@ eval create table t2(b int auto_increment, c int, key(b)) engine=$engine_type; insert into t1 values (10); insert into t1 values (null),(null),(null); insert into t2 values (5,0); +--disable_warnings ONCE insert into t2 (c) select * from t1 ORDER BY a; select * from t2 ORDER BY b; sync_slave_with_master; @@ -113,8 +114,10 @@ set @@session.sql_auto_is_null=1; eval create table t1(a int auto_increment, key(a)) engine=$engine_type; eval create table t2(a int) engine=$engine_type; insert into t1 (a) values (null); +--disable_warnings insert into t2 (a) select a from t1 where a is null; insert into t2 (a) select a from t1 where a is null; +--enable_warnings select * from t2; sync_slave_with_master; connection slave; @@ -172,17 +175,15 @@ begin end| delimiter ;| ---disable_warnings +--disable_warnings ONCE insert into t1 (last_id) values (0); ---enable_warnings drop trigger t1_bi; # Check that nested call doesn't affect outer context. select last_insert_id(); ---disable_warnings +--disable_warnings ONCE select bug15728_insert(); ---enable_warnings select last_insert_id(); insert into t1 (last_id) values (bug15728()); # This should be exactly one greater than in the previous call. @@ -190,9 +191,8 @@ select last_insert_id(); # BUG#20339 - stored procedure using LAST_INSERT_ID() does not # replicate statement-based ---disable_warnings +--disable_warnings ONCE drop procedure if exists foo; ---enable_warnings delimiter |; create procedure foo() begin @@ -252,6 +252,7 @@ select * from t1 order by n; # table's counter, the counter for next row is bigger than the # after-value of the updated row. connection master; +--disable_warnings ONCE insert into t1 values (NULL,400),(3,500),(NULL,600) on duplicate key UPDATE n=1000; select * from t1 order by n; sync_slave_with_master; @@ -270,6 +271,7 @@ delete from t1 where b <> 100; select * from t1 order by n; connection master; +--disable_warnings ONCE insert into t1 values(null,100),(null,350) on duplicate key update n=2; select * from t1 order by n; sync_slave_with_master; @@ -287,6 +289,7 @@ connection master; # testcase with INSERT VALUES eval CREATE TABLE t1 (a INT NOT NULL PRIMARY KEY AUTO_INCREMENT, b INT, UNIQUE(b)) ENGINE=$engine_type; +--disable_warnings ONCE INSERT INTO t1(b) VALUES(1),(1),(2) ON DUPLICATE KEY UPDATE t1.b=10; SELECT * FROM t1 ORDER BY a; sync_slave_with_master; @@ -314,19 +317,23 @@ INSERT INTO t2 (field_a, field_b, field_c) VALUES (3, 'c', '3c'); INSERT INTO t2 (field_a, field_b, field_c) VALUES (4, 'd', '4d'); INSERT INTO t2 (field_a, field_b, field_c) VALUES (5, 'e', '5e'); # Updating table t1 based on values from table t2 +--disable_warnings INSERT INTO t1 (field_1, field_2, field_3) SELECT t2.field_a, t2.field_b, t2.field_c FROM t2 ON DUPLICATE KEY UPDATE t1.field_3 = t2.field_c; +--enable_warnings # Inserting new record into t2 INSERT INTO t2 (field_a, field_b, field_c) VALUES (6, 'f', '6f'); # Updating t1 again +--disable_warnings INSERT INTO t1 (field_1, field_2, field_3) SELECT t2.field_a, t2.field_b, t2.field_c FROM t2 ON DUPLICATE KEY UPDATE t1.field_3 = t2.field_c; +--enable_warnings SELECT * FROM t1 ORDER BY id; sync_slave_with_master; SELECT * FROM t1 ORDER BY id; @@ -433,9 +440,8 @@ delimiter ;| INSERT INTO t1 VALUES (NULL, -1); CALL p1(); ---disable_warnings +--disable_warnings ONCE SELECT f1(); ---enable_warnings INSERT INTO t1 VALUES (NULL, f2()), (NULL, LAST_INSERT_ID()), (NULL, LAST_INSERT_ID()), (NULL, f2()), (NULL, f2()); INSERT INTO t1 VALUES (NULL, f2()); @@ -504,16 +510,14 @@ insert into t2 (id) values(1),(2),(3); delete from t2; set sql_log_bin=1; #inside SELECT, then inside INSERT ---disable_warnings +--disable_warnings ONCE select insid(); ---enable_warnings set sql_log_bin=0; insert into t2 (id) values(5),(6),(7); delete from t2 where id>=5; set sql_log_bin=1; ---disable_warnings +--disable_warnings ONCE insert into t1 select insid(); ---enable_warnings select * from t1 order by id; select * from t2 order by id; @@ -537,6 +541,7 @@ begin insert into t2 values(null,3); end| delimiter ;| +--disable_warnings ONCE call foo(); select * from t1 order by n; select * from t2 order by id; diff --git a/mysql-test/extra/rpl_tests/rpl_insert_ignore.test b/mysql-test/extra/rpl_tests/rpl_insert_ignore.test index 270dde7675c..f422ef35f8c 100644 --- a/mysql-test/extra/rpl_tests/rpl_insert_ignore.test +++ b/mysql-test/extra/rpl_tests/rpl_insert_ignore.test @@ -31,7 +31,7 @@ INSERT INTO t2 VALUES (3, 5); INSERT INTO t2 VALUES (4, 3); INSERT INTO t2 VALUES (5, 4); INSERT INTO t2 VALUES (6, 6); - +--disable_warnings ONCE INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a; --let $assert_cond= COUNT(*) = 6 FROM t1 --let $assert_text= Count of elements in t1 should be 6. @@ -51,25 +51,70 @@ INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a; --let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1) --let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1) --let $statement_file=INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a +--disable_warnings ONCE --eval $statement_file --let $assert_cond= COUNT(*) = 6 FROM t1 --let $assert_text= Count of elements in t1 should be 6. --source include/assert.inc -if (`SELECT @@BINLOG_FORMAT = 'ROW'`) +if (`SELECT @@BINLOG_FORMAT != 'STATEMENT'`) { --let $binlog_position_cmp= = --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. } -if (`SELECT @@BINLOG_FORMAT != 'ROW'`) +if (`SELECT @@BINLOG_FORMAT = 'STATEMENT'`) { --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 2, 1\', Info, 1]\' LIKE \'%$statement_file\' --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. } + +--source include/assert.inc + +# An insert duplicate that does not update anything must be written to the binary +# log in SBR and MIXED modes. We check this property by summing a before and after +# the update and comparing the binlog positions. The sum should be the same at both +# points and the statement should be in the binary log. +--disable_warnings +DROP TABLE t1; +DROP TABLE t2; +--enable_warnings +eval CREATE TABLE t1 ( + a INT UNSIGNED NOT NULL PRIMARY KEY +) ENGINE=$engine_type; + +eval CREATE TABLE t2 ( + a INT UNSIGNED +) ENGINE=$engine_type; + +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); + +--let $binlog_file= query_get_value("SHOW MASTER STATUS", File, 1) +--let $binlog_start= query_get_value("SHOW MASTER STATUS", Position, 1) +--let $statement_file=INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a +--disable_warnings ONCE +--eval $statement_file + +--let $assert_cond= SUM(a) = 1 FROM t1 +--let $assert_text= Sum of elements in t1 should be 1. +--source include/assert.inc + +if (`SELECT @@BINLOG_FORMAT != 'STATEMENT'`) +{ + --let $binlog_position_cmp= = + --let $assert_cond= [SHOW MASTER STATUS, Position, 1] $binlog_position_cmp $binlog_start + --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. +} +if (`SELECT @@BINLOG_FORMAT = 'STATEMENT'`) +{ + --let $assert_cond= \'[\'SHOW BINLOG EVENTS IN "$binlog_file" FROM $binlog_start LIMIT 1, 1\', Info, 1]\' LIKE \'%$statement_file\' + --let $assert_text= In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged. +} --source include/assert.inc + # Clean up --connection master drop table t1, t2; diff --git a/mysql-test/r/commit_1innodb.result b/mysql-test/r/commit_1innodb.result index fb1552b6c28..33fad9a0146 100644 --- a/mysql-test/r/commit_1innodb.result +++ b/mysql-test/r/commit_1innodb.result @@ -549,6 +549,8 @@ SUCCESS # 15. Read-write statement: UPDATE IGNORE, change 0 rows. # update ignore t1 set a=2 where a=1; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave. call p_verify_status_increment(2, 2, 1, 0); SUCCESS @@ -814,6 +816,8 @@ SUCCESS insert into t2 select a from t1; commit; replace into t2 select a from t1; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. commit; call p_verify_status_increment(8, 8, 8, 8); SUCCESS diff --git a/mysql-test/suite/binlog/r/binlog_stm_blackhole.result b/mysql-test/suite/binlog/r/binlog_stm_blackhole.result index 7e79186e7bc..e76f6b494f9 100644 --- a/mysql-test/suite/binlog/r/binlog_stm_blackhole.result +++ b/mysql-test/suite/binlog/r/binlog_stm_blackhole.result @@ -99,6 +99,8 @@ alter table t1 drop b; create table t3 like t1; insert into t1 select * from t3; replace into t1 select * from t3; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. select * from t1; a select * from t2; diff --git a/mysql-test/suite/binlog/r/binlog_unsafe.result b/mysql-test/suite/binlog/r/binlog_unsafe.result index 9f35f09e220..a307b765097 100644 --- a/mysql-test/suite/binlog/r/binlog_unsafe.result +++ b/mysql-test/suite/binlog/r/binlog_unsafe.result @@ -2352,6 +2352,7 @@ Note 1592 Unsafe statement written to the binary log using statement format sinc REPLACE INTO t1 SELECT * FROM t1 LIMIT 1; Warnings: Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted. +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. UPDATE t1 SET a=1 LIMIT 1; Warnings: Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted. @@ -2368,6 +2369,7 @@ END| CALL p1(); Warnings: Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted. +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. DROP PROCEDURE p1; DROP TABLE t1; DROP TABLE IF EXISTS t1; @@ -2651,4 +2653,40 @@ a 13:46:40 1970-01-12 13:46:40 DROP TABLE t1; +CREATE TABLE filler_table (a INT, b INT); +INSERT INTO filler_table values (1,1),(1,2); +CREATE TABLE insert_table (a INT, b INT, PRIMARY KEY(a)); +CREATE TABLE replace_table (a INT, b INT, PRIMARY KEY(a)); +INSERT INTO replace_table values (1,1),(2,2); +CREATE TABLE update_table (a INT, b INT, PRIMARY KEY(a)); +INSERT INTO update_table values (1,1),(2,2); +INSERT IGNORE INTO insert_table SELECT * FROM filler_table; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave. +TRUNCATE TABLE insert_table; +INSERT INTO insert_table SELECT * FROM filler_table ON DUPLICATE KEY UPDATE a = 1; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave. +TRUNCATE TABLE insert_table; +REPLACE INTO replace_table SELECT * FROM filler_table; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. +UPDATE IGNORE update_table SET a=2; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave. +CREATE TABLE create_ignore_test (a INT, b INT, PRIMARY KEY(b)) IGNORE SELECT * FROM filler_table; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... IGNORE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave. +CREATE TABLE create_replace_test (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. +CREATE TEMPORARY TABLE temp1 (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave. +DROP TABLE filler_table; +DROP TABLE insert_table; +DROP TABLE update_table; +DROP TABLE replace_table; +DROP TABLE create_ignore_test; +DROP TABLE create_replace_test; "End of tests" diff --git a/mysql-test/suite/binlog/t/binlog_unsafe.test b/mysql-test/suite/binlog/t/binlog_unsafe.test index 0dcf965112b..040f57597c2 100644 --- a/mysql-test/suite/binlog/t/binlog_unsafe.test +++ b/mysql-test/suite/binlog/t/binlog_unsafe.test @@ -12,6 +12,11 @@ # - insert into two autoinc columns; # - statements using UDF's. # - statements reading from log tables in the mysql database. +# - INSERT ... SELECT ... ON DUPLICATE KEY UPDATE +# - REPLACE ... SELECT +# - CREATE TABLE [IGNORE/REPLACE] SELECT +# - INSERT IGNORE...SELECT +# - UPDATE IGNORE # # Note that statements that use stored functions, stored procedures, # triggers, views, or prepared statements that invoke unsafe @@ -79,6 +84,7 @@ # BUG#45785: LIMIT in SP does not cause RBL if binlog_format=MIXED # BUG#47995: Mark user functions as unsafe # BUG#49222: Mark RAND() unsafe +# BUG#11758262: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE...SEL,CREATE...[IGN|REPL] SEL # # ==== Related test cases ==== # @@ -699,5 +705,47 @@ INSERT INTO t1 VALUES SELECT * FROM t1; DROP TABLE t1; +# +#BUG#11758262-50439: MARK INSERT...SEL...ON DUP KEY UPD,REPLACE.. +#The following statement may be unsafe when logged in statement format. +#INSERT IGNORE...SELECT +#INSERT ... SELECT ... ON DUPLICATE KEY UPDATE +#REPLACE ... SELECT +#UPDATE IGNORE +#CREATE TABLE... IGNORE SELECT +#CREATE TABLE... REPLACE SELECT + +#setup tables +CREATE TABLE filler_table (a INT, b INT); +INSERT INTO filler_table values (1,1),(1,2); +CREATE TABLE insert_table (a INT, b INT, PRIMARY KEY(a)); +CREATE TABLE replace_table (a INT, b INT, PRIMARY KEY(a)); +INSERT INTO replace_table values (1,1),(2,2); +CREATE TABLE update_table (a INT, b INT, PRIMARY KEY(a)); +INSERT INTO update_table values (1,1),(2,2); + +#INSERT IGNORE... SELECT +INSERT IGNORE INTO insert_table SELECT * FROM filler_table; +TRUNCATE TABLE insert_table; +#INSERT ... SELECT ... ON DUPLICATE KEY UPDATE +INSERT INTO insert_table SELECT * FROM filler_table ON DUPLICATE KEY UPDATE a = 1; +TRUNCATE TABLE insert_table; +#REPLACE...SELECT +REPLACE INTO replace_table SELECT * FROM filler_table; +#UPDATE IGNORE +UPDATE IGNORE update_table SET a=2; +#CREATE TABLE [IGNORE/REPLACE] SELECT +CREATE TABLE create_ignore_test (a INT, b INT, PRIMARY KEY(b)) IGNORE SELECT * FROM filler_table; +CREATE TABLE create_replace_test (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table; +#temporary tables should not throw the warning. +CREATE TEMPORARY TABLE temp1 (a INT, b INT, PRIMARY KEY(b)) REPLACE SELECT * FROM filler_table; + +###clean up +DROP TABLE filler_table; +DROP TABLE insert_table; +DROP TABLE update_table; +DROP TABLE replace_table; +DROP TABLE create_ignore_test; +DROP TABLE create_replace_test; --echo "End of tests" diff --git a/mysql-test/suite/rpl/r/rpl_insert_duplicate.result b/mysql-test/suite/rpl/r/rpl_insert_duplicate.result deleted file mode 100644 index 61ebbaaa5a9..00000000000 --- a/mysql-test/suite/rpl/r/rpl_insert_duplicate.result +++ /dev/null @@ -1,29 +0,0 @@ -include/master-slave.inc -[connection master] -CREATE TABLE t1 ( -a INT UNSIGNED NOT NULL PRIMARY KEY -) ENGINE=innodb; -CREATE TABLE t2 ( -a INT UNSIGNED -) ENGINE=innodb; -INSERT INTO t1 VALUES (1); -INSERT INTO t2 VALUES (1); -INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a; -include/assert.inc [Sum of elements in t1 should be 1.] -include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] -include/diff_tables.inc [master:test.t1 , slave:test.t1] -drop table t1, t2; -CREATE TABLE t1 ( -a INT UNSIGNED NOT NULL PRIMARY KEY -) ENGINE=myisam; -CREATE TABLE t2 ( -a INT UNSIGNED -) ENGINE=myisam; -INSERT INTO t1 VALUES (1); -INSERT INTO t2 VALUES (1); -INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a; -include/assert.inc [Sum of elements in t1 should be 1.] -include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] -include/diff_tables.inc [master:test.t1 , slave:test.t1] -drop table t1, t2; -include/rpl_end.inc diff --git a/mysql-test/suite/rpl/r/rpl_insert_ignore.result b/mysql-test/suite/rpl/r/rpl_insert_ignore.result index 04b64359126..1598b3c059f 100644 --- a/mysql-test/suite/rpl/r/rpl_insert_ignore.result +++ b/mysql-test/suite/rpl/r/rpl_insert_ignore.result @@ -26,6 +26,19 @@ include/diff_tables.inc [master:test.t1 , slave:test.t1] INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a; include/assert.inc [Count of elements in t1 should be 6.] include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] +DROP TABLE t1; +DROP TABLE t2; +CREATE TABLE t1 ( +a INT UNSIGNED NOT NULL PRIMARY KEY +) ENGINE=innodb; +CREATE TABLE t2 ( +a INT UNSIGNED +) ENGINE=innodb; +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); +INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a; +include/assert.inc [Sum of elements in t1 should be 1.] +include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] drop table t1, t2; CREATE TABLE t1 ( a int unsigned not null auto_increment primary key, @@ -52,5 +65,18 @@ include/diff_tables.inc [master:test.t1 , slave:test.t1] INSERT IGNORE INTO t1 SELECT NULL, t2.b FROM t2 ORDER BY t2.a; include/assert.inc [Count of elements in t1 should be 6.] include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] +DROP TABLE t1; +DROP TABLE t2; +CREATE TABLE t1 ( +a INT UNSIGNED NOT NULL PRIMARY KEY +) ENGINE=myisam; +CREATE TABLE t2 ( +a INT UNSIGNED +) ENGINE=myisam; +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (1); +INSERT INTO t1 SELECT t2.a FROM t2 ORDER BY t2.a ON DUPLICATE KEY UPDATE t1.a= t1.a; +include/assert.inc [Sum of elements in t1 should be 1.] +include/assert.inc [In SBR or MIXED modes, the event in the binlog should be the same that was executed. In RBR mode, binlog position should stay unchanged.] drop table t1, t2; include/rpl_end.inc diff --git a/mysql-test/suite/rpl/r/rpl_insert_select.result b/mysql-test/suite/rpl/r/rpl_insert_select.result deleted file mode 100644 index d98ae2538fa..00000000000 --- a/mysql-test/suite/rpl/r/rpl_insert_select.result +++ /dev/null @@ -1,14 +0,0 @@ -include/master-slave.inc -[connection master] -create table t1 (n int not null primary key); -insert into t1 values (1); -create table t2 (n int); -insert into t2 values (1); -insert ignore into t1 select * from t2; -insert into t1 values (2); -select * from t1; -n -1 -2 -drop table t1,t2; -include/rpl_end.inc diff --git a/mysql-test/suite/rpl/r/rpl_known_bugs_detection.result b/mysql-test/suite/rpl/r/rpl_known_bugs_detection.result index 43f5c082dea..92ccc78aaca 100644 --- a/mysql-test/suite/rpl/r/rpl_known_bugs_detection.result +++ b/mysql-test/suite/rpl/r/rpl_known_bugs_detection.result @@ -42,12 +42,16 @@ SELECT t2.field_a, t2.field_b, t2.field_c FROM t2 ON DUPLICATE KEY UPDATE t1.field_3 = t2.field_c; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave. INSERT INTO t2 (field_a, field_b, field_c) VALUES (6, 'f', '6f'); INSERT INTO t1 (field_1, field_2, field_3) SELECT t2.field_a, t2.field_b, t2.field_c FROM t2 ON DUPLICATE KEY UPDATE t1.field_3 = t2.field_c; +Warnings: +Note 1592 Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave. SELECT * FROM t1; id field_1 field_2 field_3 1 1 a 1a diff --git a/mysql-test/suite/rpl/t/rpl_insert_duplicate.test b/mysql-test/suite/rpl/t/rpl_insert_duplicate.test deleted file mode 100644 index 7dd38a696ae..00000000000 --- a/mysql-test/suite/rpl/t/rpl_insert_duplicate.test +++ /dev/null @@ -1,14 +0,0 @@ -######################################### -# Wrapper for rpl_insert_duplicate.test # -######################################### --- source include/master-slave.inc --- source include/have_innodb.inc -#-- source include/have_binlog_format_mixed_or_statement.inc - -let $engine_type=innodb; --- source extra/rpl_tests/rpl_insert_duplicate.test - -let $engine_type=myisam; --- source extra/rpl_tests/rpl_insert_duplicate.test - ---source include/rpl_end.inc diff --git a/mysql-test/suite/rpl/t/rpl_insert_select.test b/mysql-test/suite/rpl/t/rpl_insert_select.test deleted file mode 100644 index 23bc7ecd167..00000000000 --- a/mysql-test/suite/rpl/t/rpl_insert_select.test +++ /dev/null @@ -1,20 +0,0 @@ -# Testcase for BUG#10456 - INSERT INTO ... SELECT violating a primary key -# breaks replication - --- source include/master-slave.inc -connection master; - -create table t1 (n int not null primary key); -insert into t1 values (1); -create table t2 (n int); -insert into t2 values (1); -insert ignore into t1 select * from t2; -insert into t1 values (2); -sync_slave_with_master; -connection slave; -select * from t1; - -connection master; -drop table t1,t2; -sync_slave_with_master; ---source include/rpl_end.inc diff --git a/mysql-test/suite/rpl/t/rpl_known_bugs_detection.test b/mysql-test/suite/rpl/t/rpl_known_bugs_detection.test index f4b854eff87..99871b695a6 100644 --- a/mysql-test/suite/rpl/t/rpl_known_bugs_detection.test +++ b/mysql-test/suite/rpl/t/rpl_known_bugs_detection.test @@ -7,7 +7,7 @@ source include/have_debug.inc; source include/master-slave.inc; # Currently only statement-based-specific bugs are here --- source include/have_binlog_format_mixed_or_statement.inc +-- source include/have_binlog_format_statement.inc # # This is to test that slave properly detects if diff --git a/sql/share/errmsg-utf8.txt b/sql/share/errmsg-utf8.txt index c82215f41fb..da176bd5233 100644 --- a/sql/share/errmsg-utf8.txt +++ b/sql/share/errmsg-utf8.txt @@ -6421,6 +6421,24 @@ ER_INDEX_CORRUPT ER_UNDO_RECORD_TOO_BIG eng "Undo log record is too big." +ER_BINLOG_UNSAFE_INSERT_IGNORE_SELECT + eng "INSERT IGNORE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave." + +ER_BINLOG_UNSAFE_INSERT_SELECT_UPDATE + eng "INSERT... SELECT... ON DUPLICATE KEY UPDATE is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are updated. This order cannot be predicted and may differ on master and the slave." + +ER_BINLOG_UNSAFE_REPLACE_SELECT + eng "REPLACE... SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave." + +ER_BINLOG_UNSAFE_CREATE_IGNORE_SELECT + eng "CREATE... IGNORE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave." + +ER_BINLOG_UNSAFE_CREATE_REPLACE_SELECT + eng "CREATE... REPLACE SELECT is unsafe because the order in which rows are retrieved by the SELECT determines which (if any) rows are replaced. This order cannot be predicted and may differ on master and the slave." + +ER_BINLOG_UNSAFE_UPDATE_IGNORE + eng "UPDATE IGNORE is unsafe because the order in which rows are updated determines which (if any) rows are ignored. This order cannot be predicted and may differ on master and the slave." + ER_PLUGIN_NO_UNINSTALL eng "Plugin '%s' is marked as not dynamically uninstallable. You have to stop the server to uninstall it." diff --git a/sql/sql_lex.cc b/sql/sql_lex.cc index 14cf57af141..df27362633f 100644 --- a/sql/sql_lex.cc +++ b/sql/sql_lex.cc @@ -59,6 +59,12 @@ Query_tables_list::binlog_stmt_unsafe_errcode[BINLOG_STMT_UNSAFE_COUNT] = ER_BINLOG_UNSAFE_NONTRANS_AFTER_TRANS, ER_BINLOG_UNSAFE_MULTIPLE_ENGINES_AND_SELF_LOGGING_ENGINE, ER_BINLOG_UNSAFE_MIXED_STATEMENT, + ER_BINLOG_UNSAFE_INSERT_IGNORE_SELECT, + ER_BINLOG_UNSAFE_INSERT_SELECT_UPDATE, + ER_BINLOG_UNSAFE_REPLACE_SELECT, + ER_BINLOG_UNSAFE_CREATE_IGNORE_SELECT, + ER_BINLOG_UNSAFE_CREATE_REPLACE_SELECT, + ER_BINLOG_UNSAFE_UPDATE_IGNORE }; diff --git a/sql/sql_lex.h b/sql/sql_lex.h index 8794006fef7..542e8b42ae2 100644 --- a/sql/sql_lex.h +++ b/sql/sql_lex.h @@ -1253,6 +1253,48 @@ public: */ BINLOG_STMT_UNSAFE_MIXED_STATEMENT, + /** + INSERT...IGNORE SELECT is unsafe because which rows are ignored depends + on the order that rows are retrieved by SELECT. This order cannot be + predicted and may differ on master and the slave. + */ + BINLOG_STMT_UNSAFE_INSERT_IGNORE_SELECT, + + /** + INSERT...SELECT...UPDATE is unsafe because which rows are updated depends + on the order that rows are retrieved by SELECT. This order cannot be + predicted and may differ on master and the slave. + */ + BINLOG_STMT_UNSAFE_INSERT_SELECT_UPDATE, + + /** + INSERT...REPLACE SELECT is unsafe because which rows are replaced depends + on the order that rows are retrieved by SELECT. This order cannot be + predicted and may differ on master and the slave. + */ + BINLOG_STMT_UNSAFE_REPLACE_SELECT, + + /** + CREATE TABLE... IGNORE... SELECT is unsafe because which rows are ignored + depends on the order that rows are retrieved by SELECT. This order cannot + be predicted and may differ on master and the slave. + */ + BINLOG_STMT_UNSAFE_CREATE_IGNORE_SELECT, + + /** + CREATE TABLE...REPLACE... SELECT is unsafe because which rows are replaced + depends on the order that rows are retrieved from SELECT. This order + cannot be predicted and may differ on master and the slave + */ + BINLOG_STMT_UNSAFE_CREATE_REPLACE_SELECT, + + /** + UPDATE...IGNORE is unsafe because which rows are ignored depends on the + order that rows are updated. This order cannot be predicted and may differ + on master and the slave. + */ + BINLOG_STMT_UNSAFE_UPDATE_IGNORE, + /* The last element of this enumeration type. */ BINLOG_STMT_UNSAFE_COUNT }; diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index 4df77b5b15b..4fac64749e9 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -2387,6 +2387,19 @@ case SQLCOM_PREPARE: select_result *result; /* + CREATE TABLE...IGNORE/REPLACE SELECT... can be unsafe, unless + ORDER BY PRIMARY KEY clause is used in SELECT statement. We therefore + use row based logging if mixed or row based logging is available. + TODO: Check if the order of the output of the select statement is + deterministic. Waiting for BUG#42415 + */ + if(lex->ignore) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_CREATE_IGNORE_SELECT); + + if(lex->duplicates == DUP_REPLACE) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_CREATE_REPLACE_SELECT); + + /* If: a) we inside an SP and there was NAME_CONST substitution, b) binlogging is on (STMT mode), @@ -2720,6 +2733,16 @@ end_with_restore_list: DBUG_ASSERT(first_table == all_tables && first_table != 0); if (update_precheck(thd, all_tables)) break; + + /* + UPDATE IGNORE can be unsafe. We therefore use row based + logging if mixed or row based logging is available. + TODO: Check if the order of the output of the select statement is + deterministic. Waiting for BUG#42415 + */ + if (lex->ignore) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_UPDATE_IGNORE); + DBUG_ASSERT(select_lex->offset_limit == 0); unit->set_limit(select_lex); MYSQL_UPDATE_START(thd->query()); @@ -2886,6 +2909,23 @@ end_with_restore_list: DBUG_ASSERT(first_table == all_tables && first_table != 0); if ((res= insert_precheck(thd, all_tables))) break; + /* + INSERT...SELECT...ON DUPLICATE KEY UPDATE/REPLACE SELECT/ + INSERT...IGNORE...SELECT can be unsafe, unless ORDER BY PRIMARY KEY + clause is used in SELECT statement. We therefore use row based + logging if mixed or row based logging is available. + TODO: Check if the order of the output of the select statement is + deterministic. Waiting for BUG#42415 + */ + if (lex->sql_command == SQLCOM_INSERT_SELECT && + lex->duplicates == DUP_UPDATE) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_INSERT_SELECT_UPDATE); + + if (lex->sql_command == SQLCOM_INSERT_SELECT && lex->ignore) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_INSERT_IGNORE_SELECT); + + if (lex->sql_command == SQLCOM_REPLACE_SELECT) + lex->set_stmt_unsafe(LEX::BINLOG_STMT_UNSAFE_REPLACE_SELECT); /* Fix lock for first table */ if (first_table->lock_type == TL_WRITE_DELAYED) |