summaryrefslogtreecommitdiff
path: root/sql/sql_repl.cc
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-18553: MDEV-16327 pre-requisits part 2: uniform of LIMIT/OFFSET handlingOleksandr Byelkin2019-10-131-1/+1
| | | | | | | Now both offset and limit are stored and do not chenged during execution (offset is decreased during processing in versions before 10.5). (Big part of this changes made by Monty)
* MDEV-18553: MDEV-16327 pre-requisits part 1: isolation of LIMIT/OFFSET handlingOleksandr Byelkin2019-10-131-6/+4
|
* Merge 10.3 into 10.4Marko Mäkelä2019-10-101-19/+44
|\
| * Merge 10.2 into 10.3Marko Mäkelä2019-10-091-19/+44
| |\
| | * MDEV-20574 Position of events reported by mysqlbinlog is wrong with ↵Sachin Setiya2019-10-081-19/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | encrypted binlogs, SHOW BINLOG EVENTS reports the correct one. Analysis Mysqlbinlog output for encrypted binary log #Q> insert into tab1 values (3,'row 003') #190912 17:36:35 server id 10221 end_log_pos 980 CRC32 0x53bcb3d3 Table_map: `test`.`tab1` mapped to number 19 # at 940 #190912 17:36:35 server id 10221 end_log_pos 1026 CRC32 0xf2ae5136 Write_rows: table id 19 flags: STMT_END_F Here we can see Table_map_log_event ends at 980 but Next event starts at 940. And the reason for that is we do not send START_ENCRYPTION_EVENT to the slave Solution:- Send Start_encryption_log_event as Ignorable_log_event to slave(mysqlbinlog), So that mysqlbinlog can update its log_pos. Since Slave can request multiple FORMAT_DESCRIPTION_EVENT while master does not have so We only update slave master pos when master actually have the FORMAT_DESCRIPTION_EVENT. Similar logic should be applied for START_ENCRYPTION_EVENT. Also added the test case when new server reads the data from old server which does not send START_ENCRYPTION_EVENT to slave. Master Slave Upgrade Scenario. When Slave is updated first, Slave will have extra logic of handling START_ENCRYPTION_EVENT But master willnot be sending START_ENCRYPTION_EVENT. So there will be no issue. When Master is updated first, It will send START_ENCRYPTION_EVENT to slave , But slave will ignore this event in queue_event.
* | | Merge branch '10.3' into 10.4Oleksandr Byelkin2019-05-191-1/+1
|\ \ \ | |/ /
| * | Merge 10.2 into 10.3Marko Mäkelä2019-05-141-1/+1
| |\ \ | | |/
| | * Merge 10.1 into 10.2Marko Mäkelä2019-05-131-1/+1
| | |\
| | | * Merge branch '5.5' into 10.1Vicențiu Ciorbaru2019-05-111-1/+1
| | | |\
| | | | * Update FSF AddressVicențiu Ciorbaru2019-05-111-1/+1
| | | | | | | | | | | | | | | | | | | | * Update wrong zip-code
* | | | | MDEV-19117 Don't keep binary log index file locked during show binary logsMonty2019-04-011-53/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On some systems with 10,000+ binlogs, show binary logs could block log rotation for more than 10 seconds. This patch fixes this by first caching all binary log names and releases all mutexes while calculating the sizes of the binary logs. Other things: - Ensure that reinit_io_cache() sets end_of_file when moving to read_cache. This ensures that external changes of the underlying file is known to the cache. - get_binlog_list() is made more efficent and show_binlogs() is changed to call get_binlog_list() Reviewed by Andrei Elkin
* | | | | Fixed valgrind warning: Wrong usage of c_ptr()Monty2019-04-011-1/+1
| | | | |
* | | | | MDEV-18450 Slaves wait shutdownSergey Vojtovich2019-03-121-10/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The patches features an optional shutdown behavior to hold on until after all connected slaves have been sent the last binlogged event. The connected slave is one whose START SLAVE has been acknowledged and that was not stopped since that though it could be technically reconnecting in background. The solution therefore disallows killing the dump thread until is has found EOF of the latest binlog file. It is up to the shutdown requester (DBA) to set up a sufficiently large shutdown timeout value for shudown to wait patiently until lagging behind slaves have been synchronized. On the other hand if a specific slave needs exclusion from synchronization the DBA would have to stop it manually which would terminate its dump thread. `mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query to enable the feature on the client side. The patch also performs a small refactoring of the server shutdown around close_connections() to introduce kill thread phases which are two as of current.
* | | | | Merge branch '10.3' into 10.4Oleksandr Byelkin2019-02-211-3/+3
|\ \ \ \ \ | |/ / / /
| * | | | dirty mergeOleksandr Byelkin2019-02-071-3/+3
| |\ \ \ \ | | |/ / /
| | * | | Merge 10.1 into 10.2Marko Mäkelä2019-02-031-3/+3
| | |\ \ \ | | | |/ / | | | | | | | | | | | | | | | | | | | | Temporarily disable a test for commit 2175bfce3e9da8332f10ab0e0286dc93915533a2 because fixing it in 10.2 requires updating libmariadb.
| | | * | Merge 10.1 into 10.1Marko Mäkelä2019-02-021-3/+3
| | | |\ \ | | | | | | | | | | | | | | | | | | This is joint work with Oleksandr Byelkin.
| | | | * \ Merge branch '5.5' into 10.0Oleksandr Byelkin2019-01-281-3/+3
| | | | |\ \ | | | | | |/
| | | | | * Bug #28178776 COMPARISON OF UNINITAILIZED MEMORY IN LOG_IN_USEAditya A2019-01-231-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROBLEM ------- Memory sanitizer reports uninitialized comparisons in log_in_use(), because strings are compared with memcmp() instead of strncmp. FIX --- Use strncmp() to compare strings
| | | | | * MDEV-13459 Warnings, when compiling with gcc-7.xSergei Golubchik2017-10-171-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | mostly caused by -Wimplicit-fallthrough
| | | * | | MDEV-14014 Multi-Slave Replication Fail: bogus data in log eventAndrei Elkin2018-06-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MDEV-7257 made a dump thread to read from binlog concurrently with writers as long as the read bytes are below a water-mark (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a dump thread reader reach out for bytes past the water mark through a feature of IO_CACHE that fills in the internal buffer and while doing so it could read what the reader is not supposed to see (the bytes above MYSQL_BIN_LOG::binlog_end_pos). The issue is fixed with constraining the IO_CACHE buffer fill to respect the watermark. An added unit test proves reading from file is bound to an external parameter passed to {IO_CACHE::end_of_file} cache member.
| | | * | | MDEV-14721 Big transaction events get lost on semisync master whenAndrei Elkin2018-03-121-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | replicate_events_marked_for_skip=FILTER_ON_MASTER [Note this is a cherry-pick from 10.2 branch.] When events of a big transaction are binlogged offsetting over 2GB from the beginning of the log the semisync master's dump thread lost such events. The events were skipped by the Dump thread that found their skipping status erroneously. The current fixes make sure the skipping status is computed correctly. The test verifies them simulating the 2GB offset.
* | | | | | Fix for galera_3nodes.galera_gtid_2_clustermkaruza2019-02-191-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | Temporary disable WSREP while executing RESET MASTER. In situation when 2 nodes are both master/slave first stop slave on both and than reset master. Enforce stricter causality check with wsrep_sync_wait.
* | | | | | Move THD list handling to THD_listSergey Vojtovich2019-01-281-56/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented and integrated THD_list as a replacement for the global thread list. It uses own mutex instead of LOCK_thread_count for THD list protection. Removed unused first_global_thread() and next_global_thread(). delayed_insert_threads is now protected by LOCK_delayed_insert. Although this patch doesn't fix very wrong synchronization of this variable. After this patch there are only 2 legitimate uses of LOCK_thread_count left, both in mysqld.cc: thread_count and ready_to_exit. Aim is to reduce usage of LOCK_thread_count and COND_thread_count. Part of MDEV-15135.
* | | | | | Simplified THD::current_linfo lockingSergey Vojtovich2019-01-281-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LOG_INFO::lock was useless. It could've only protect against concurrent iterators execution, which was already protected by LOCK_thread_count. Use LOCK_thd_data instead of LOCK_thread_count as a protection against THD::current_linfo reset. Aim is to reduce usage of LOCK_thread_count and COND_thread_count. Part of MDEV-15135.
* | | | | | Galera4Brave Galera Crew2019-01-231-0/+11
| | | | | |
* | | | | | Merge remote-tracking branch 'origin/10.3' into 10.4Alexander Barkov2018-07-031-0/+1
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Merge branch '10.2' into 10.3Sergei Golubchik2018-06-301-0/+1
| |\ \ \ \ \ | | |/ / / /
| | * | | | MDEV-14014 Multi-Slave Replication Fail: bogus data in log eventAndrei Elkin2018-06-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MDEV-7257 made a dump thread to read from binlog concurrently with writers as long as the read bytes are below a water-mark (MYSQL_BIN_LOG::binlog_end_pos). However it appeared to be possible a dump thread reader reach out for bytes past the water mark through a feature of IO_CACHE that fills in the internal buffer and while doing so it could read what the reader is not supposed to see (the bytes above MYSQL_BIN_LOG::binlog_end_pos). The issue is fixed with constraining the IO_CACHE buffer fill to respect the watermark. An added unit test proves reading from file is bound to an external parameter passed to {IO_CACHE::end_of_file} cache member.
* | | | | | mysys: remove dead ME_xxx flagsSergei Golubchik2018-06-041-2/+1
|/ / / / /
* | | | | Add likely/unlikely to speed up executionMonty2018-05-071-38/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added to: - if (error) - Lex - sql_yacc.yy and sql_yacc_ora.yy - In header files to alloc() calls - Added thd argument to thd_net_is_killed()
* | | | | MDEV-15091 : Windows, 64bit: reenable and fix warning C4267 (conversion from ↵Vladislav Vaintroub2018-02-061-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'size_t' to 'type', possible loss of data) Handle string length as size_t, consistently (almost always:)) Change function prototypes to accept size_t, where in the past ulong or uint were used. change local/member variables to size_t when appropriate. This fix excludes rocksdb, spider,spider, sphinx and connect for now.
* | | | | Merge remote-tracking branch 'origin/bb-10.2-ext' into 10.3Alexander Barkov2018-02-041-2/+3
|\ \ \ \ \
| * | | | | Added name to MEM_ROOT for esier debuggingMonty2018-02-021-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will make it easier to how memory allocation is done when debugging with either DBUG or gdb. Will especially help when debugging stored procedures Main change is a name argument as second argument to init_alloc_root() init_sql_alloc() Other things: - Added DBUG_ENTER/EXIT to some Virtual_tmp_table functions
* | | | | | Change C_STRING_WITH_LEN to STRING_WITH_LENMonty2018-01-301-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This preserves const str for constant strings Other things - A few variables where changed from LEX_STRING to LEX_CSTRING - Incident_log_event::Incident_log_event and record_incident where changed to take LEX_CSTRING* as an argument instead of LEX_STRING
* | | | | | Changed database, tablename and alias to be LEX_CSTRINGMonty2018-01-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was done in, among other things: - thd->db and thd->db_length - TABLE_LIST tablename, db, alias and schema_name - Audit plugin database name - lex->db - All db and table names in Alter_table_ctx - st_select_lex db Other things: - Changed a lot of functions to take const LEX_CSTRING* as argument for db, table_name and alias. See init_one_table() as an example. - Changed some function arguments from LEX_CSTRING to const LEX_CSTRING - Changed some lists from LEX_STRING to LEX_CSTRING - threads_mysql.result changed because process list_db wasn't always correctly updated - New append_identifier() function that takes LEX_CSTRING* as arguments - Added new element tmp_buff to Alter_table_ctx to separate temp name handling from temporary space - Ensure we store the length after my_casedn_str() of table/db names - Removed not used version of rename_table_in_stat_tables() - Changed Natural_join_column::table_name and db_name() to never return NULL (used for print) - thd->get_db() now returns db as a printable string (thd->db.str or "")
* | | | | | Merge remote-tracking branch 'origin/bb-10.2-ext' into 10.3Alexander Barkov2018-01-291-5/+2
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Merge remote-tracking branch 'origin/10.2' into bb-10.2-extMonty2018-01-271-5/+2
| |\ \ \ \ \ | | |/ / / / | | | | | | | | | | | | | | | | | | Conflicts: sql/table.cc
| | * | | | MDEV-14721 Big transaction events get lost on semisync master whenAndrei Elkin2018-01-271-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | replicate_events_marked_for_skip=FILTER_ON_MASTER When events of a big transaction are binlogged offsetting over 2GB from the beginning of the log the semisync master's dump thread lost such events. The events were skipped by the Dump thread that found their skipping status erroneously. The current fixes make sure the skipping status is computed correctly. The test verifies them simulating the 2GB offset.
* | | | | | Merge bb-10.2-ext into 10.3Marko Mäkelä2018-01-051-1/+2
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Merge remote-tracking branch 'origin/10.2' into bb-10.2-extMonty2018-01-051-1/+2
| |\ \ \ \ \ | | |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: mysql-test/r/cte_nonrecursive.result mysql-test/suite/galera/r/galera_bf_abort.result mysql-test/suite/galera/r/galera_bf_abort_get_lock.result mysql-test/suite/galera/r/galera_bf_abort_sleep.result mysql-test/suite/galera/r/galera_enum.result mysql-test/suite/galera/r/galera_fk_conflict.result mysql-test/suite/galera/r/galera_insert_multi.result mysql-test/suite/galera/r/galera_many_indexes.result mysql-test/suite/galera/r/galera_mdl_race.result mysql-test/suite/galera/r/galera_nopk_bit.result mysql-test/suite/galera/r/galera_nopk_blob.result mysql-test/suite/galera/r/galera_nopk_large_varchar.result mysql-test/suite/galera/r/galera_nopk_unicode.result mysql-test/suite/galera/r/galera_pk_bigint_signed.result mysql-test/suite/galera/r/galera_pk_bigint_unsigned.result mysql-test/suite/galera/r/galera_serializable.result mysql-test/suite/galera/r/galera_toi_drop_database.result mysql-test/suite/galera/r/galera_toi_lock_exclusive.result mysql-test/suite/galera/r/galera_toi_truncate.result mysql-test/suite/galera/r/galera_unicode_pk.result mysql-test/suite/galera/r/galera_var_auto_inc_control_off.result mysql-test/suite/galera/r/galera_wsrep_log_conficts.result sql/field.cc sql/rpl_gtid.cc sql/share/errmsg-utf8.txt sql/sql_acl.cc sql/sql_parse.cc sql/sql_partition_admin.cc sql/sql_prepare.cc sql/sql_repl.cc sql/sql_table.cc sql/sql_yacc.yy
| | * | | | Fixed wrong arguments to printf and related functionsMonty2018-01-041-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Other things, mainly to get create_mysqld_error_find_printf_error tool to work: - Added protection to not include mysqld_error.h twice - Include "unireg.h" instead of "mysqld_error.h" in server - Added protection if ER_XX messages are already defined - Removed wrong calls to my_error(ER_OUTOFMEMORY) as my_malloc() and my_alloc will do this automatically - Added missing %s to ER_DUP_QUERY_NAME - Removed old and wrong calls to my_strerror() when using MY_ERROR_ON_RENAME (wrong merge) - Fixed deadlock error message from Galera. Before the extra information given to ER_LOCK_DEADLOCK was missing because ER_LOCK_DEADLOCK doesn't provide any extra information. I kept #ifdef mysqld_error_find_printf_error_used in sql_acl.h to make it easy to do this kind of check again in the future
* | | | | | MDEV-13073. This part converts the Ali patch`s identifiers to the MariaDB ↵Andrei Elkin2017-12-181-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | standard. Also some renaming is done as well as white spaces removal.
* | | | | | MDEV-13073. This part patch weeds out RUN_HOOK from the server as semisyncAndrei Elkin2017-12-181-31/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | is defined statically. Consequently the observer interfaces are removed as well.
* | | | | | MDEV-13073 This part merges the Ali semisync related changesAndrei Elkin2017-12-181-77/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and specifically the ack receiving functionality. Semisync is turned to be static instead of plugin so its functions are invoked at the same points as RUN_HOOKS. The RUN_HOOKS and the observer interface remain to be removed by later patch. Todo: React on killed status by repl_semisync_master.wait_after_sync(). Currently Repl_semi_sync_master::commit_trx does not check the killed status. There were few bugfixes found that are present in mysql and its unclear whether/how they are covered. Those include: Bug#15985893: GTID SKIPPED EVENTS ON MASTER CAUSE SEMI SYNC TIME-OUTS Bug#17932935 CALLING IS_SEMI_SYNC_SLAVE() IN EACH FUNCTION CALL HAS BAD PERFORMANCE Bug#20574628: SEMI-SYNC REPLICATION PERFORMANCE DEGRADES WITH A HIGH NUMBER OF THREADS
* | | | | | Changed from using LOCK_log to LOCK_binlog_end_pos for binary logMonty2017-12-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Part of MDEV-13073 AliSQL Optimize performance of semisync The idea it to use a dedicated lock detecting if there is new data in the master's binary log instead of the overused LOCK_log. Changes: - Use dedicated COND variables for the relay and binary log signaling. This was needed as we where the old 'update_cond' variable was used with different mutex's, which could cause deadlocks. - Relay log uses now COND_relay_log_updated and LOCK_log - Binary log uses now COND_bin_log_updated and LOCK_binlog_end_pos - Renamed signal_cnt to relay_signal_cnt (as we now have two signals) - Added some missing error handling in MYSQL_BIN_LOG::new_file_impl() - Reformatted some comments with old style - Renamed m_key_LOCK_binlog_end_pos to key_LOCK_binlog_end_pos - Changed 'signal_update()' to update_binlog_end_pos() which works for both relay and binary log
* | | | | | Removed not used lock argument from read_log_eventMonty2017-12-181-2/+69
| | | | | |
* | | | | | Merge bb-10.2-ext into 10.3Marko Mäkelä2017-12-121-3/+3
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Move all kill mutex protection to LOCK_thd_killMonty2017-12-081-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LOCK_thd_data was used to protect both THD data and ensure that the THD is not deleted while it was in use This patch moves the THD delete protection to LOCK_thd_kill, which already protects the THD for kill. The benefits are: - More well defined what LOCK_thd_data protects - LOCK_thd_data usage is now much simpler and easier to verify - Less chance of deadlocks in SHOW PROCESS LIST as there is less chance of interactions between mutexes - Remove not needed LOCK_thread_count from thd_get_error_context_description() - Fewer mutex taken for thd->awake() Other things: - Don't take mysys->var mutex in show processlist to check if thread is kill marked - thd->awake() now automatically takes the LOCK_thd_kill mutex (Simplifies code) - Apc uses LOCK_thd_kill instead of LOCK_thd_data
* | | | | | Merge bb-10.2-ext into 10.3Marko Mäkelä2017-11-301-67/+1
|\ \ \ \ \ \ | |/ / / / /
View Lorry Controller Status