diff options
author | unknown <guilhem@mysql.com> | 2005-02-03 16:22:16 +0100 |
---|---|---|
committer | unknown <guilhem@mysql.com> | 2005-02-03 16:22:16 +0100 |
commit | 7636b12f6cc71e209b5189c94b06416beccaf2eb (patch) | |
tree | 7a4b93f6301c5bdc29371dda2344050ce305ba4c /sql/log_event.cc | |
parent | 9aad0ae359881bbe39431714f30e49bbed29aa97 (diff) | |
download | mariadb-git-7636b12f6cc71e209b5189c94b06416beccaf2eb.tar.gz |
WL#1062 "log charset info into all Query_log_event":
we store 7 bytes (1 + 2*3) in every Query_log_event.
In the future if users want binlog optimized for small size and less safe,
we could add --binlog-no-charset (and binlog-no-sql-mode etc): charset info
is something by design optional (even if for now we don't offer possibility to disable it):
it's not a binlog format change.
We try to reduce the number of get_charset() calls in the slave SQL thread to a minimum
by caching the charset read from the previous event (which will often be equal to the one of the current event).
We don't use SET ONE_SHOT for charset-aware repl (we still do for timezones, will be fixed later).
No more errors if one changes the global value of charset vars on master or slave
(as we log charset info in all Query_log_event).
Not fixing Load_log_event as it will be rewritten soon by Dmitri.
Testing how mysqlbinlog behaves in rpl_charset.test.
mysqlbinlog needs to know where charset file is (to be able to convert a charset number found
in binlog (e.g. in User_var_log_event) to a charset name); mysql-test-run needs to pass
the correct value for this option to mysqlbinlog.
Many result udpates (adding charset info into every event shifts log_pos in SHOW BINLOG EVENTS).
Roughly the same job is to be done for timezones :)
client/mysqlbinlog.cc:
mysqlbinlog needs charsets knowledge, to be able to convert a charset
number found in binlog to a charset name (to be able to print things
like this:
SET @`a`:=_cp850 0x4DFC6C6C6572 COLLATE `cp850_general_ci`;
mysql-test/mysql-test-run.sh:
tell mysqlbinlog about charsets dir
mysql-test/r/ctype_ucs.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/drop_temp_table.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/insert_select.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/mix_innodb_myisam_binlog.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/mysqlbinlog.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/mysqlbinlog2.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
The log_pos shift is why the SET INSERT_ID=4 event changes position in the result.
mysql-test/r/rpl_charset.result:
Running mysqlbinlog to check how it behaves on charset stuff.
SET ONE_SHOT is now gone.
Repl of LOAD DATA INFILE is not yet charset-aware (will soon be, when WL#874 is pushed)
and, anyway result has a dependency on the temp filename (SQL-LOAD-*-[0-9] which is not constant).
No more errors if one changes global character sets.
mysql-test/r/rpl_error_ignored_table.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_flush_log_loop.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_flush_tables.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_loaddata.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_loaddata_rule_m.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_log.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_max_relay_size.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_relayrotate.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_replicate_do.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_rotate_logs.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_temporary.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_timezone.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/rpl_user_variables.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/r/user_var.result:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
mysql-test/t/rpl_charset.test:
Running mysqlbinlog to check how it behaves on charset stuff (so, need fixed timestamp).
SET ONE_SHOT is not printed to binlog anymore, so no need to test if ::exec_event() works ok.
Repl of LOAD DATA INFILE is not yet charset-aware (will soon be, when WL#874 is pushed)
and, anyway result has a dependency on the temp filename (SQL-LOAD-*-[0-9] which is not constant).
No more errors if one changes global character sets.
mysql-test/t/rpl_user_variables.test:
different binlogging of charsets results in shifted log_pos and
one added SET @@CHARACTER_SET... per mysqlbinlog run.
sql/log.cc:
No more SET ONE_SHOT for charsets (remains for TZ until solved with Dmitri).
sql/log_event.cc:
We now log charset info in each Query_log_event in binlog. It's 2*3 = 6 bytes:
session character_set_client, session collation_connection, session collation_server.
Now we would need only one byte per variable, but Bar said 2 is safer for the future.
When slave or mysqlbinlog reads that info, it needs to get_charset() on these numbers (so, 3 get_charset() calls),
as most of the time the 6-byte charset info will be equal to the previous event's,
we cache the previous event's charset and if equal, no need to get_charset().
As "flags2", SQL_MODE, catalog, autoinc variables, charset info is not a permanent addition:
in the future we can add options to the master to not log any of these, old 5.0 should be able
to parse these.
A little bit of cleanup on autoinc stuff in replication.
Fixing a bug in Start_log_event_v3::exec_event() where we used rli->relay_log.description_event_for_exec->binlog_version
while we should use binlog_version (if it's a 3.23 master, that's all that counts; not the fact that the relay log is
in 5.0 format).
sql/log_event.h:
binlogging of charset info in each Query_log_event.
sql/mysql_priv.h:
comment
sql/set_var.cc:
checks to refuse change of global charset variables are removed: they were needed for 4.1->4.1
but not for 5.0.3->5.0.3.
Yes this opens a breach if one does 4.1->5.0.3, where the checks would still be needed. But these checks would need
reading relay_log.description_event_for_queue, which is currently an object used in many places by the I/O
thread and only it. So, currently we don't take mutexes for this object, and if we read the object in set_var.cc
(client thread) we need to add mutexes everywhere, but the replication code is already too broken with mutexes
now (no consistent use of mutexes); mutex usage in replication should be fixed but preferrably during/after
multimaster coding as it's going to shuffle mutexes already.
sql/set_var.h:
Since we don't forbid global change of charset vars for replication/binlogging,
don't need specific ::check() methods anymore
sql/slave.cc:
Some little debug info which has nothing to do with charsets.
Disabling master's charset check when slave I/O thread connects.
Functions for charset caching/invalidating in the slave SQL thread.
sql/slave.h:
Cached charset in the slave SQL thread.
Diffstat (limited to 'sql/log_event.cc')
-rw-r--r-- | sql/log_event.cc | 161 |
1 files changed, 121 insertions, 40 deletions
diff --git a/sql/log_event.cc b/sql/log_event.cc index 5ee034d785e..d09b2b3dc03 100644 --- a/sql/log_event.cc +++ b/sql/log_event.cc @@ -953,7 +953,13 @@ void Query_log_event::pack_info(Protocol *protocol) bool Query_log_event::write(IO_CACHE* file) { - uchar buf[QUERY_HEADER_LEN+1+4+1+8+1+1+FN_REFLEN+5], *start, *start_of_status; + uchar buf[QUERY_HEADER_LEN+ + 1+4+ // code of flags2 and flags2 + 1+8+ // code of sql_mode and sql_mode + 1+1+FN_REFLEN+ // code of catalog and catalog length and catalog + 1+4+ // code of autoinc and the 2 autoinc variables + 1+6 // code of charset and charset + ], *start, *start_of_status; ulong event_length; if (!query) @@ -1048,9 +1054,15 @@ bool Query_log_event::write(IO_CACHE* file) int2store(start+2, auto_increment_offset); start+= 4; } + if (charset_inited) + { + *(start++)= Q_CHARSET_CODE; + memcpy(start, charset, 6); + start+= 6; + } /* Here there could be code like - if (command-line-option-which-says-"log_this_variable") + if (command-line-option-which-says-"log_this_variable" && inited) { *(start++)= Q_THIS_VARIABLE_CODE; int4store(start, this_variable); @@ -1095,7 +1107,7 @@ Query_log_event::Query_log_event(THD* thd_arg, const char* query_arg, thread_id(thd_arg->thread_id), /* save the original thread id; we already know the server id */ slave_proxy_id(thd_arg->variables.pseudo_thread_id), - flags2_inited(1), sql_mode_inited(1), flags2(0), + flags2_inited(1), sql_mode_inited(1), charset_inited(1), sql_mode(thd_arg->variables.sql_mode), auto_increment_increment(thd_arg->variables.auto_increment_increment), auto_increment_offset(thd_arg->variables.auto_increment_offset) @@ -1104,7 +1116,7 @@ Query_log_event::Query_log_event(THD* thd_arg, const char* query_arg, time(&end_time); exec_time = (ulong) (end_time - thd->start_time); catalog_len = (catalog) ? (uint32) strlen(catalog) : 0; - status_vars_len= 1+4+1+8+1+1+catalog_len+1; + /* status_vars_len is set just before writing the event */ db_len = (db) ? (uint32) strlen(db) : 0; /* If we don't use flags2 for anything else than options contained in @@ -1114,7 +1126,12 @@ Query_log_event::Query_log_event(THD* thd_arg, const char* query_arg, we will probably want to reclaim the 29 bits. So we need the &. */ flags2= thd_arg->options & OPTIONS_WRITTEN_TO_BIN_LOG; - + DBUG_ASSERT(thd->variables.character_set_client->number < 256*256); + DBUG_ASSERT(thd->variables.collation_connection->number < 256*256); + DBUG_ASSERT(thd->variables.collation_server->number < 256*256); + int2store(charset, thd_arg->variables.character_set_client->number); + int2store(charset+2, thd_arg->variables.collation_connection->number); + int2store(charset+4, thd_arg->variables.collation_server->number); DBUG_PRINT("info",("Query_log_event has flags2=%lu sql_mode=%lu",flags2,sql_mode)); } #endif /* MYSQL_CLIENT */ @@ -1129,7 +1146,8 @@ Query_log_event::Query_log_event(const char* buf, uint event_len, const Format_description_log_event *description_event) :Log_event(buf, description_event), data_buf(0), query(NullS), catalog(NullS), db(NullS), catalog_len(0), status_vars_len(0), - flags2_inited(0), sql_mode_inited(0) + flags2_inited(0), sql_mode_inited(0), charset_inited(0), + auto_increment_increment(1), auto_increment_offset(1) { ulong data_len; uint32 tmp; @@ -1156,8 +1174,6 @@ Query_log_event::Query_log_event(const char* buf, uint event_len, exec_time = uint4korr(buf + Q_EXEC_TIME_OFFSET); db_len = (uint)buf[Q_DB_LEN_OFFSET]; error_code = uint2korr(buf + Q_ERR_CODE_OFFSET); - /* If auto_increment is not set by query_event, they should not be used */ - auto_increment_increment= auto_increment_offset= 1; /* 5.0 format starts here. @@ -1216,6 +1232,13 @@ Query_log_event::Query_log_event(const char* buf, uint event_len, auto_increment_offset= uint2korr(pos+2); pos+= 4; break; + case Q_CHARSET_CODE: + { + charset_inited= 1; + memcpy(charset, pos, 6); + pos+= 6; + break; + } default: /* That's why you must write status vars in growing order of code */ DBUG_PRINT("info",("Query_log_event has unknown status vars (first has\ @@ -1348,6 +1371,27 @@ void Query_log_event::print(FILE* file, bool short_form, last_event_info->auto_increment_offset= auto_increment_offset; } + if (likely(charset_inited)) + { + if (unlikely(!last_event_info->charset_inited)) /* first Query event */ + { + last_event_info->charset_inited= 1; + last_event_info->charset[0]= ~charset[0]; // force a difference to force write + } + if (unlikely(bcmp(last_event_info->charset, charset, 6))) + { + fprintf(file,"SET " + "@@session.character_set_client=%d," + "@@session.collation_connection=%d," + "@@session.collation_server=%d" + ";\n", + uint2korr(charset), + uint2korr(charset+2), + uint2korr(charset+4)); + memcpy(last_event_info->charset, charset, 6); + } + } + my_fwrite(file, (byte*) query, q_len, MYF(MY_NABP | MY_WME)); fputs(";\n", file); } @@ -1400,34 +1444,64 @@ int Query_log_event::exec_event(struct st_relay_log_info* rli) thd->variables.pseudo_thread_id= thread_id; // for temp tables mysql_log.write(thd,COM_QUERY,"%s",thd->query); DBUG_PRINT("query",("%s",thd->query)); - - if (flags2_inited) - /* - all bits of thd->options which are 1 in OPTIONS_WRITTEN_TO_BIN_LOG must - take their value from flags2. - */ - thd->options= flags2|(thd->options & ~(ulong)OPTIONS_WRITTEN_TO_BIN_LOG); - /* - else, we are in a 3.23/4.0 binlog; we previously received a - Rotate_log_event which reset thd->options and sql_mode, so nothing to do. - */ - - /* - We do not replicate IGNORE_DIR_IN_CREATE. That is, if the master is a - slave which runs with SQL_MODE=IGNORE_DIR_IN_CREATE, this should not - force us to ignore the dir too. Imagine you are a ring of machines, and - one has a disk problem so that you temporarily need IGNORE_DIR_IN_CREATE - on this machine; you don't want it to propagate elsewhere (you don't want - all slaves to start ignoring the dirs). - */ - if (sql_mode_inited) - thd->variables.sql_mode= - (ulong) ((thd->variables.sql_mode & MODE_NO_DIR_IN_CREATE) | - (sql_mode & ~(ulong) MODE_NO_DIR_IN_CREATE)); if (ignored_error_code((expected_error= error_code)) || !check_expected_error(thd,rli,expected_error)) + { + if (flags2_inited) + /* + all bits of thd->options which are 1 in OPTIONS_WRITTEN_TO_BIN_LOG must + take their value from flags2. + */ + thd->options= flags2|(thd->options & ~(ulong)OPTIONS_WRITTEN_TO_BIN_LOG); + /* + else, we are in a 3.23/4.0 binlog; we previously received a + Rotate_log_event which reset thd->options and sql_mode etc, so nothing to do. + */ + /* + We do not replicate IGNORE_DIR_IN_CREATE. That is, if the master is a + slave which runs with SQL_MODE=IGNORE_DIR_IN_CREATE, this should not + force us to ignore the dir too. Imagine you are a ring of machines, and + one has a disk problem so that you temporarily need IGNORE_DIR_IN_CREATE + on this machine; you don't want it to propagate elsewhere (you don't want + all slaves to start ignoring the dirs). + */ + if (sql_mode_inited) + thd->variables.sql_mode= + (ulong) ((thd->variables.sql_mode & MODE_NO_DIR_IN_CREATE) | + (sql_mode & ~(ulong) MODE_NO_DIR_IN_CREATE)); + if (charset_inited) + { + if (rli->cached_charset_compare(charset)) + { + /* Verify that we support the charsets found in the event. */ + if (!(thd->variables.character_set_client= + get_charset(uint2korr(charset), MYF(MY_WME))) || + !(thd->variables.collation_connection= + get_charset(uint2korr(charset+2), MYF(MY_WME))) || + !(thd->variables.collation_server= + get_charset(uint2korr(charset+4), MYF(MY_WME)))) + { + /* + We updated the thd->variables with nonsensical values (0), and the + thread is not guaranteed to terminate now (as it may be configured + to ignore EE_UNKNOWN_CHARSET);if we're going to execute a next + statement we'll have a new charset info with it, so no problem to + have stored 0 in thd->variables. But we invalidate cached + charset to force a check next time (otherwise if next time + charset is unknown again we won't detect it). + */ + rli->cached_charset_invalidate(); + goto compare_errors; + } + thd->update_charset(); // for the charset change to take effect + } + } + + /* Execute the query (note that we bypass dispatch_command()) */ mysql_parse(thd, thd->query, q_len); + + } else { /* @@ -1452,6 +1526,8 @@ START SLAVE; . Query: '%s'", expected_error, thd->query); } goto end; } + +compare_errors: /* If we expected a non-zero error code, and we don't get the same error @@ -1666,12 +1742,7 @@ bool Start_log_event_v3::write(IO_CACHE* file) int Start_log_event_v3::exec_event(struct st_relay_log_info* rli) { DBUG_ENTER("Start_log_event_v3::exec_event"); - /* - If the I/O thread has not started, mi->old_format is BINLOG_FORMAT_CURRENT - (that's what the MASTER_INFO constructor does), so the test below is not - perfect at all. - */ - switch (rli->relay_log.description_event_for_exec->binlog_version) + switch (binlog_version) { case 3: case 4: @@ -2789,14 +2860,24 @@ int Rotate_log_event::exec_event(struct st_relay_log_info* rli) rli->group_master_log_name, (ulong) rli->group_master_log_pos)); /* - Reset thd->options and sql_mode, because this could be the signal of a - master's downgrade from 5.0 to 4.0. + Reset thd->options and sql_mode etc, because this could be the signal of + a master's downgrade from 5.0 to 4.0. However, no need to reset description_event_for_exec: indeed, if the next master is 5.0 (even 5.0.1) we will soon get a Format_desc; if the next master is 4.0 then the events are in the slave's format (conversion). */ set_slave_thread_options(thd); thd->variables.sql_mode= global_system_variables.sql_mode; + thd->variables.auto_increment_increment= + thd->variables.auto_increment_offset= 1; + thd->variables.character_set_client= + global_system_variables.character_set_client; + thd->variables.collation_connection= + global_system_variables.collation_connection; + thd->variables.collation_server= + global_system_variables.collation_server; + thd->update_charset(); + rli->cached_charset_invalidate(); } pthread_mutex_unlock(&rli->data_lock); pthread_cond_broadcast(&rli->data_cond); |