MDEV-9501: rpl.rpl_binlog_index, rpl.rpl_gtid_crash, rpl.rpl_stm_multi_query fail sporadically in buildbot with Master command COM_REGISTER_SLAVE failed

Analysis:
========
Slave server will send COM_REGISTER_SLAVE command at the time of establishing
a connection to master. If master is down, then the command will fail and
COM_REGISTER_SLAVE failed warning is reported.

'rpl_binlog_index.test' shutsdown the master and it relocates binary logs to a
new location and attempts to start master by pointing 'log-bin' to new
location. During this process the slave threads are active. IO thread actively
checks for the presence of master when it finds that the connection is lost it
attempts a reconnect, as master is down COM_REGISTER_SLAVE command fails.

As part of fix, stop the slave threads and then shutdown the master and do the
binlog relocation. Once master is restarted start the slave threads and sync
them with the master. In test binary logs and index files on master are
relocated to /tmpdir but during master restart only --log-bin option is
provided, this is incorrect. Even --log-bin-index also should be pointed to
/tmpdir otherwise upon master server restart two index files will be created.
One master-bin.index in /tmpdir and a new master-bin.index as per log_basename
in datadir. Due to this slave will fail to connect to master.

'rpl_gtid_crash.test' tests following scenario "crashing master, causing slave
IO thread to reconnect while SQL thread is running". When IO thread tries to
connect to crashed master on slow platforms COM_REGISTER_SLAVE command fails.
This is expected hence the warning should be added to suppression list.

4 files changed, 23 insertions, 5 deletions

diff --git a/mysql-test/suite/rpl/r/rpl_binlog_index.result b/mysql-test/suite/rpl/r/rpl_binlog_index.result
index 6611a9ef2c0..0a53c6f76ee 100644
--- a/mysql-test/suite/rpl/r/rpl_binlog_index.result
+++ b/mysql-test/suite/rpl/r/rpl_binlog_index.result
@@ -3,18 +3,22 @@ include/master-slave.inc
 CREATE TABLE t1 (a INT);
 FLUSH BINARY LOGS;
 INSERT INTO t1 VALUES (1);
+include/stop_slave.inc
 # Shutdown master
 include/rpl_stop_server.inc [server_number=1]
 # Move the master binlog files and the index file to a new place
 # Restart master with log-bin option set to the new path
 # Master has restarted successfully
+include/start_slave.inc
 # Create the master-bin.index file with the old format
+include/stop_slave.inc
 # Shutdown master
 include/rpl_stop_server.inc [server_number=1]
 # Move back the master binlog files
 # Remove the unneeded master-bin.index file
 # Restart master with log-bin option set to default
 # Master has restarted successfully
+include/start_slave.inc
 # stop slave
 include/stop_slave.inc
 include/rpl_stop_server.inc [server_number=2]
diff --git a/mysql-test/suite/rpl/r/rpl_gtid_crash.result b/mysql-test/suite/rpl/r/rpl_gtid_crash.result
index 5f67901b7f5..630098b18c9 100644
--- a/mysql-test/suite/rpl/r/rpl_gtid_crash.result
+++ b/mysql-test/suite/rpl/r/rpl_gtid_crash.result
@@ -7,6 +7,9 @@ flush tables;
 ALTER TABLE mysql.gtid_slave_pos ENGINE=InnoDB;
 CREATE TABLE t1 (a INT PRIMARY KEY, b INT) ENGINE=InnoDB;
 INSERT INTO t1 VALUES (1, 0);
+SET sql_log_bin=0;
+call mtr.add_suppression('Master command COM_REGISTER_SLAVE failed: failed registering on master, reconnecting to try again');
+SET sql_log_bin=1;
 include/stop_slave.inc
 CHANGE MASTER TO master_host = '127.0.0.1', master_port = MASTER_PORT,
 MASTER_USE_GTID=CURRENT_POS;
diff --git a/mysql-test/suite/rpl/t/rpl_binlog_index.test b/mysql-test/suite/rpl/t/rpl_binlog_index.test
index 8586b1d7489..95c49c3d574 100644
--- a/mysql-test/suite/rpl/t/rpl_binlog_index.test
+++ b/mysql-test/suite/rpl/t/rpl_binlog_index.test
@@ -43,7 +43,7 @@ FLUSH BINARY LOGS;
 INSERT INTO t1 VALUES (1);
 
 sync_slave_with_master;
-
+--source include/stop_slave.inc
 #
 # Test on master
 #
@@ -58,13 +58,15 @@ source include/rpl_stop_server.inc;
 --move_file $master_datadir/master-bin.index  $tmpdir/master-bin.index
 
 --echo # Restart master with log-bin option set to the new path
---let $rpl_server_parameters=--log-bin=$tmpdir/master-bin
+--let $rpl_server_parameters=--log-bin=$tmpdir/master-bin --log-bin-index=$tmpdir/master-bin
 --let $keep_include_silent=1
 source include/rpl_start_server.inc;
 --let $keep_include_silent=0
 
 --echo # Master has restarted successfully
-
+--connection slave
+--source include/start_slave.inc
+--connection master
 #
 # Test master can handle old format with directory path in index file
 #
@@ -85,7 +87,10 @@ if (!$is_windows)
 --disable_query_log
 source include/write_var_to_file.inc;
 --enable_query_log
+--sync_slave_with_master
+--source include/stop_slave.inc
 
+--connection master
 --echo # Shutdown master
 --let $rpl_server_number=1
 source include/rpl_stop_server.inc;
@@ -99,14 +104,17 @@ source include/rpl_stop_server.inc;
 --remove_file $tmpdir/master-bin.index
 
 --echo # Restart master with log-bin option set to default
---let $rpl_server_parameters=--log-bin=$master_datadir/master-bin
+--let $rpl_server_parameters=--log-bin=$master_datadir/master-bin --log-bin-index=$master_datadir/master-bin
 --let $keep_include_silent=1
 source include/rpl_start_server.inc;
 --let $keep_include_silent=0
 
 --echo # Master has restarted successfully
+--connection slave
+--source include/start_slave.inc
 
-connection slave;
+--connection master
+--sync_slave_with_master
 --echo # stop slave
 --source include/stop_slave.inc
 --let $rpl_server_number= 2
diff --git a/mysql-test/suite/rpl/t/rpl_gtid_crash.test b/mysql-test/suite/rpl/t/rpl_gtid_crash.test
index b81cbd38cd3..84bf76501d8 100644
--- a/mysql-test/suite/rpl/t/rpl_gtid_crash.test
+++ b/mysql-test/suite/rpl/t/rpl_gtid_crash.test
@@ -21,6 +21,9 @@ INSERT INTO t1 VALUES (1, 0);
 
 --connection server_2
 --sync_with_master
+SET sql_log_bin=0;
+call mtr.add_suppression('Master command COM_REGISTER_SLAVE failed: failed registering on master, reconnecting to try again');
+SET sql_log_bin=1;
 --source include/stop_slave.inc
 --replace_result $MASTER_MYPORT MASTER_PORT
 eval CHANGE MASTER TO master_host = '127.0.0.1', master_port = $MASTER_MYPORT,