BUG#46364 MyISAM transbuffer problems (NTM problem)

      
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
                    
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
            
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
            
So, in this patch we introduce the option
      - "--binlog-direct-non-transactional-updates" that can be used to bypass
      the current behavior in order to write directly to binary log statements
      that change non-transactional tables.

Besides, it is used to enable the WL#2687 which is disabled by default.

mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
  The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode.
mysql-test/include/default_mysqld.cnf:
  Makes binlog-direct-non-transactional-updates "TRUE" by default in the test
  cases.
mysql-test/r/mysqld--help-notwin.result:
  Updates the result file with the new option.
mysql-test/r/mysqld--help-win.result:
  Updates the result file with the new option.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing
  the test case and its result file.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
sql/log.cc:
  Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code.
sql/log.h:
  Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed: 
          bool trans_has_updated_trans_table(const THD* thd);
          bool stmt_has_updated_trans_table(const THD *thd);
          bool use_trans_cache(const THD*, bool is_transactional);
sql/share/errmsg-utf8.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/share/errmsg.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/sql_class.h:
  Adds the new option binlog_direct_non_transactional_updates.
sql/sys_vars.cc:
  Adds the new option binlog_direct_non_transactional_updates.
support-files/my-small.cnf.sh:
  Adds binlog-direct-non-transactional-updates to the example file. By default
  the option is disabled.

1 files changed, 42 insertions, 0 deletions

diff --git a/sql/sys_vars.cc b/sql/sys_vars.cc
index 0d6c3327f24..fe3114c9d50 100644
--- a/sql/sys_vars.cc
+++ b/sql/sys_vars.cc
@@ -299,6 +299,48 @@ static Sys_var_enum Sys_binlog_format(
        NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(binlog_format_check),
        ON_UPDATE(fix_binlog_format_after_update));
 
+static bool binlog_direct_check(sys_var *self, THD *thd, set_var *var)
+{
+   /*
+     Makes the session variable 'binlog_direct_non_transactional_updates'
+     read-only inside a transaction.
+   */
+   if (thd->active_transaction() && (var->type == OPT_SESSION))
+   {
+     my_error(ER_INSIDE_TRANSACTION_PREVENTS_SWITCH_BINLOG_DIRECT, MYF(0));
+     return 1;
+   }
+   /*
+     Makes the session variable 'binlog_direct_non_transactional_updates'
+     read-only if within a procedure, trigger or function.
+   */
+   if (thd->in_sub_stmt)
+   {
+     my_error(ER_STORED_FUNCTION_PREVENTS_SWITCH_BINLOG_DIRECT, MYF(0));
+     return 1;
+   }
+
+  if (check_has_super(self, thd, var))
+    return true;
+  if (var->type == OPT_GLOBAL ||
+      (thd->variables.binlog_direct_non_trans_update ==
+       var->save_result.ulonglong_value))
+    return false;
+
+  return false;
+}
+
+static Sys_var_mybool Sys_binlog_direct(
+       "binlog_direct_non_transactional_updates",
+       "Causes updates to non-transactional engines using statement format to "
+       "be written directly to binary log. Before using this option make sure "
+       "that there are no dependencies between transactional and "
+       "non-transactional tables such as in the statement INSERT INTO t_myisam "
+       "SELECT * FROM t_innodb; otherwise, slaves may diverge from the master.",
+       SESSION_VAR(binlog_direct_non_trans_update),
+       CMD_LINE(OPT_ARG), DEFAULT(FALSE),
+       NO_MUTEX_GUARD, NOT_IN_BINLOG, ON_CHECK(binlog_direct_check));
+
 static Sys_var_ulong Sys_bulk_insert_buff_size(
        "bulk_insert_buffer_size", "Size of tree cache used in bulk "
        "insert optimisation. Note that this is a limit per thread!",