Patch for the following bugs:

  - BUG#11986: Stored routines and triggers can fail if the code
    has a non-ascii symbol
  - BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
  - BUG#19443: INFORMATION_SCHEMA does not support charsets properly
  - BUG#21249: Character set of SP-var can be ignored
  - BUG#25212: Character set of string constant is ignored (stored routines)
  - BUG#25221: Character set of string constant is ignored (triggers)

There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
   triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
   inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
   definition;

1. No query-definition-character set.

In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.

The context contains the following data:
  - client character set;
  - connection collation (character set and collation);
  - collation of the owner database;

The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).

2. Wrong mysqldump-output.

The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.

Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).

The solution is
  - to store definition queries in the original character set;
  - to change SHOW CREATE statement to output definition query in the
    binary character set (i.e. without any conversion);
  - introduce SHOW CREATE TRIGGER statement;
  - to dump special statements to switch the context to the original one
    before dumping and restore it afterwards.

Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.

3. INFORMATION_SCHEMA showed non-UTF8 strings

The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.

Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.

This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object.  Specialized SHOW CREATE statements should be
used for this.

The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).

Example:

  - original query:
    CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;

  - UTF8 query (for INFORMATION_SCHEMA):
    CREATE VIEW v1 AS SELECT 'Hello' AS c1;


client/mysqldump.c:
  Set original character set and collation before dumping definition query.
include/my_sys.h:
  Move out-parameter to the end of list.
mysql-test/lib/mtr_report.pl:
  Ignore server-warnings during the test case.
mysql-test/r/create.result:
  Update result file.
mysql-test/r/ctype_cp932_binlog_stm.result:
  Update result file.
mysql-test/r/events.result:
  Update result file.
mysql-test/r/events_bugs.result:
  Update result file.
mysql-test/r/events_grant.result:
  Update result file.
mysql-test/r/func_in.result:
  Update result file.
mysql-test/r/gis.result:
  Update result file.
mysql-test/r/grant.result:
  Update result file.
mysql-test/r/information_schema.result:
  Update result file.
mysql-test/r/information_schema_db.result:
  Update result file.
mysql-test/r/lowercase_view.result:
  Update result file.
mysql-test/r/mysqldump.result:
  Update result file.
mysql-test/r/ndb_sp.result:
  Update result file.
mysql-test/r/ps.result:
  Update result file.
mysql-test/r/rpl_replicate_do.result:
  Update result file.
mysql-test/r/rpl_sp.result:
  Update result file.
mysql-test/r/rpl_trigger.result:
  Update result file.
mysql-test/r/rpl_view.result:
  Update result file.
mysql-test/r/show_check.result:
  Update result file.
mysql-test/r/skip_grants.result:
  Update result file.
mysql-test/r/sp-destruct.result:
  Update result file.
mysql-test/r/sp-error.result:
  Update result file.
mysql-test/r/sp-security.result:
  Update result file.
mysql-test/r/sp.result:
  Update result file.
mysql-test/r/sql_mode.result:
  Update result file.
mysql-test/r/system_mysql_db.result:
  Update result file.
mysql-test/r/temp_table.result:
  Update result file.
mysql-test/r/trigger-compat.result:
  Update result file.
mysql-test/r/trigger-grant.result:
  Update result file.
mysql-test/r/trigger.result:
  Update result file.
mysql-test/r/view.result:
  Update result file.
mysql-test/r/view_grant.result:
  Update result file.
mysql-test/t/events.test:
  Update test case (new columns added).
mysql-test/t/information_schema.test:
  Update test case (new columns added).
mysql-test/t/show_check.test:
  Test case for SHOW CREATE TRIGGER in prepared statements and
  stored routines.
mysql-test/t/sp-destruct.test:
  Update test case (new columns added).
mysql-test/t/sp.test:
  Update test case (new columns added).
mysql-test/t/view.test:
  Update test.
mysys/charset.c:
  Move out-parameter to the end of list.
scripts/mysql_system_tables.sql:
  Add new columns to mysql.proc and mysql.event.
scripts/mysql_system_tables_fix.sql:
  Add new columns to mysql.proc and mysql.event.
sql/event_data_objects.cc:
  Support new attributes for events.
sql/event_data_objects.h:
  Support new attributes for events.
sql/event_db_repository.cc:
  Support new attributes for events.
sql/event_db_repository.h:
  Support new attributes for events.
sql/events.cc:
  Add new columns to SHOW CREATE event resultset.
sql/mysql_priv.h:
  1. Introduce Object_creation_ctx;
  2. Introduce SHOW CREATE TRIGGER;
  3. Introduce auxilary functions.
sql/sp.cc:
  Add support for new store routines attributes.
sql/sp_head.cc:
  Add support for new store routines attributes.
sql/sp_head.h:
  Add support for new store routines attributes.
sql/sql_lex.cc:
  Generate UTF8-body on parsing/lexing.
sql/sql_lex.h:
  1. Generate UTF8-body on parsing/lexing.
  2. Introduce SHOW CREATE TRIGGER.
sql/sql_parse.cc:
  Introduce SHOW CREATE TRIGGER.
sql/sql_partition.cc:
  Update parse_sql().
sql/sql_prepare.cc:
  Update parse_sql().
sql/sql_show.cc:
  Support new attributes for views
sql/sql_trigger.cc:
  Support new attributes for views
sql/sql_trigger.h:
  Support new attributes for views
sql/sql_view.cc:
  Support new attributes for views
sql/sql_yacc.yy:
  1. Add SHOW CREATE TRIGGER statement.
  2. Generate UTF8-body for views, stored routines, triggers and events.
sql/table.cc:
  Introduce Object_creation_ctx.
sql/table.h:
  Introduce Object_creation_ctx.
sql/share/errmsg.txt:
  Add new errors.
mysql-test/include/ddl_i18n.check_events.inc:
  Aux file for test suite.
mysql-test/include/ddl_i18n.check_sp.inc:
  Aux file for test suite.
mysql-test/include/ddl_i18n.check_triggers.inc:
  Aux file for test suite.
mysql-test/include/ddl_i18n.check_views.inc:
  Aux file for test suite.
mysql-test/include/have_cp1251.inc:
  Aux file for test suite.
mysql-test/include/have_cp866.inc:
  Aux file for test suite.
mysql-test/include/have_koi8r.inc:
  Aux file for test suite.
mysql-test/include/have_utf8.inc:
  Aux file for test suite.
mysql-test/r/ddl_i18n_koi8r.result:
  Result file.
mysql-test/r/ddl_i18n_utf8.result:
  Result file.
mysql-test/r/have_cp1251.require:
  Aux file for test suite.
mysql-test/r/have_cp866.require:
  Aux file for test suite.
mysql-test/r/have_koi8r.require:
  Aux file for test suite.
mysql-test/r/have_utf8.require:
  Aux file for test suite.
mysql-test/t/ddl_i18n_koi8r.test:
  Complete koi8r test case for the CS patch.
mysql-test/t/ddl_i18n_utf8.test:
  Complete utf8 test case for the CS patch.

1 files changed, 71 insertions, 29 deletions

diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy
index 97e35d8c6ae..81dd8aebf65 100644
--- a/sql/sql_yacc.yy
+++ b/sql/sql_yacc.yy
@@ -1089,7 +1089,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize);
 %type <lex_str>
         IDENT IDENT_QUOTED TEXT_STRING DECIMAL_NUM FLOAT_NUM NUM LONG_NUM HEX_NUM
 	LEX_HOSTNAME ULONGLONG_NUM field_ident select_alias ident ident_or_text
-        UNDERSCORE_CHARSET IDENT_sys TEXT_STRING_sys TEXT_STRING_literal
+        IDENT_sys TEXT_STRING_sys TEXT_STRING_literal
 	NCHAR_STRING opt_component key_cache_name
         sp_opt_label BIN_NUM label_ident TEXT_STRING_filesystem ident_or_empty
 
@@ -1205,6 +1205,7 @@ bool my_yyoverflow(short **a, YYSTYPE **b, ulong *yystacksize);
 	collation_name
 	collation_name_or_default
 	opt_load_data_charset
+        UNDERSCORE_CHARSET
 
 %type <variable> internal_variable_name
 
@@ -1716,21 +1717,24 @@ event_tail:
              YYTHD->client_capabilities is set back to original value
           */
           {
-            Lex->create_info.options= $2;
+            THD *thd= YYTHD;
+            LEX *lex=Lex;
 
-            if (!(Lex->event_parse_data= Event_parse_data::new_instance(YYTHD)))
+            lex->create_info.options= $2;
+
+            if (!(lex->event_parse_data= Event_parse_data::new_instance(thd)))
               MYSQL_YYABORT;
-            Lex->event_parse_data->identifier= $3;
+            lex->event_parse_data->identifier= $3;
 
             /*
               We have to turn of CLIENT_MULTI_QUERIES while parsing a
               stored procedure, otherwise yylex will chop it into pieces
               at each ';'.
             */
-            $<ulong_num>$= YYTHD->client_capabilities & CLIENT_MULTI_QUERIES;
-            YYTHD->client_capabilities &= (~CLIENT_MULTI_QUERIES);
+            $<ulong_num>$= thd->client_capabilities & CLIENT_MULTI_QUERIES;
+            thd->client_capabilities &= (~CLIENT_MULTI_QUERIES);
 
-            Lex->sql_command= SQLCOM_CREATE_EVENT;
+            lex->sql_command= SQLCOM_CREATE_EVENT;
             /* We need that for disallowing subqueries */
           }
           ON SCHEDULE_SYM ev_schedule_time
@@ -1863,16 +1867,16 @@ ev_sql_stmt:
             if (!(lex->sphead= new sp_head()))
               MYSQL_YYABORT;
 
-            lex->sphead->reset_thd_mem_root(YYTHD);
+            lex->sphead->reset_thd_mem_root(thd);
             lex->sphead->init(lex);
-            lex->sphead->init_sp_name(YYTHD, Lex->event_parse_data->identifier);
+	    lex->sphead->init_sp_name(thd, lex->event_parse_data->identifier);
 
             lex->sphead->m_type= TYPE_ENUM_PROCEDURE;
 
             bzero((char *)&lex->sp_chistics, sizeof(st_sp_chistics));
             lex->sphead->m_chistics= &lex->sp_chistics;
 
-            lex->sphead->set_body_begin_ptr(lip, lip->get_cpp_ptr());
+            lex->sphead->set_body_start(thd, lip->get_cpp_ptr());
           }
           ev_sql_stmt_inner
           {
@@ -1880,7 +1884,7 @@ ev_sql_stmt:
             LEX *lex= thd->lex;
 
             /* return back to the original memory root ASAP */
-            lex->sphead->init_strings(thd, lex);
+            lex->sphead->set_stmt_end(thd);
             lex->sphead->restore_thd_mem_root(thd);
 
             lex->sp_chistics.suid= SP_IS_SUID;  //always the definer!
@@ -1946,7 +1950,7 @@ sp_name:
               MYSQL_YYABORT;
 	    $$= new sp_name(db, $1, false);
             if ($$)
-	      $$->init_qname(YYTHD);
+	      $$->init_qname(thd);
 	  }
 	;
 
@@ -2066,7 +2070,7 @@ create_function_tail:
             Lex_input_stream *lip= thd->m_lip;
 
 	    lex->sphead->m_chistics= &lex->sp_chistics;
-            lex->sphead->set_body_begin_ptr(lip, lip->get_cpp_tok_start());
+            lex->sphead->set_body_start(thd, lip->get_cpp_tok_start());
 	  }
 	  sp_proc_stmt
 	  {
@@ -2078,7 +2082,7 @@ create_function_tail:
               MYSQL_YYABORT;
 
 	    lex->sql_command= SQLCOM_CREATE_SPFUNCTION;
-	    sp->init_strings(thd, lex);
+	    sp->set_stmt_end(thd);
             if (!(sp->m_flags & sp_head::HAS_RETURN))
             {
               my_error(ER_SP_NORETURN, MYF(0), sp->m_qname.str);
@@ -3604,14 +3608,20 @@ create2:
           create3 {}
         | LIKE table_ident
           {
-            Lex->create_info.options|= HA_LEX_CREATE_TABLE_LIKE;
-            if (!Lex->select_lex.add_table_to_list(YYTHD, $2, NULL, 0, TL_READ))
+            THD *thd= YYTHD;
+            LEX *lex= thd->lex;
+
+            lex->create_info.options|= HA_LEX_CREATE_TABLE_LIKE;
+            if (!lex->select_lex.add_table_to_list(thd, $2, NULL, 0, TL_READ))
               MYSQL_YYABORT;
           }
         | '(' LIKE table_ident ')'
           {
-            Lex->create_info.options|= HA_LEX_CREATE_TABLE_LIKE;
-            if (!Lex->select_lex.add_table_to_list(YYTHD, $3, NULL, 0, TL_READ))
+            THD *thd= YYTHD;
+            LEX *lex= thd->lex;
+
+            lex->create_info.options|= HA_LEX_CREATE_TABLE_LIKE;
+            if (!lex->select_lex.add_table_to_list(thd, $3, NULL, 0, TL_READ))
               MYSQL_YYABORT;
           }
         ;
@@ -8669,7 +8679,7 @@ show_param:
 	  { Lex->create_info.db_type= NULL; }
 	| opt_full COLUMNS from_or_in table_ident opt_db wild_and_where
 	  {
-            LEX *lex= Lex;
+	    LEX *lex= Lex;
 	    lex->sql_command= SQLCOM_SHOW_FIELDS;
 	    if ($5)
 	      $4->change_db($5);
@@ -8867,6 +8877,12 @@ show_param:
 	    lex->sql_command = SQLCOM_SHOW_CREATE_FUNC;
 	    lex->spname= $3;
 	  }
+	| CREATE TRIGGER_SYM sp_name
+	  {
+	    LEX *lex= Lex;
+	    lex->sql_command= SQLCOM_SHOW_CREATE_TRIGGER;
+	    lex->spname= $3;
+	  }
 	| PROCEDURE STATUS_SYM wild_and_where
 	  {
             LEX *lex= Lex;
@@ -9307,7 +9323,9 @@ text_literal:
 	| NCHAR_STRING
 	{ $$=  new Item_string($1.str,$1.length,national_charset_info); }
 	| UNDERSCORE_CHARSET TEXT_STRING
-	  { $$ = new Item_string($2.str,$2.length,Lex->underscore_charset); }
+	  {
+	    $$ = new Item_string($2.str, $2.length, $1);
+	  }
 	| text_literal TEXT_STRING_literal
 	  { ((Item_string*) $1)->append($2.str,$2.length); }
 	;
@@ -9394,7 +9412,7 @@ literal:
 	      (String*) 0;
 	    $$= new Item_string(str ? str->ptr() : "",
 				str ? str->length() : 0,
-				Lex->underscore_charset);
+				$1);
 	  }
 	| UNDERSCORE_CHARSET BIN_NUM
           {
@@ -9669,6 +9687,7 @@ IDENT_sys:
 	| IDENT_QUOTED
 	  {
 	    THD *thd= YYTHD;
+
 	    if (thd->charset_is_system_charset)
             {
               CHARSET_INFO *cs= system_charset_info;
@@ -9694,6 +9713,7 @@ TEXT_STRING_sys:
 	TEXT_STRING
 	{
 	  THD *thd= YYTHD;
+
 	  if (thd->charset_is_system_charset)
 	    $$= $1;
 	  else
@@ -9706,6 +9726,7 @@ TEXT_STRING_literal:
 	TEXT_STRING
 	{
 	  THD *thd= YYTHD;
+
 	  if (thd->charset_is_collation_connection)
 	    $$= $1;
 	  else
@@ -9719,6 +9740,7 @@ TEXT_STRING_filesystem:
 	TEXT_STRING
 	{
 	  THD *thd= YYTHD;
+
 	  if (thd->charset_is_character_set_filesystem)
 	    $$= $1;
 	  else
@@ -10408,7 +10430,8 @@ option_value:
 internal_variable_name:
 	ident
 	{
-	  LEX *lex= Lex;
+          THD *thd= YYTHD;
+          LEX *lex= thd->lex;
           sp_pcontext *spc= lex->spcont;
 	  sp_variable_t *spv;
 
@@ -10416,7 +10439,7 @@ internal_variable_name:
 	  if (!spc || !(spv = spc->find_variable(&$1)))
 	  {
             /* Not an SP local variable */
-	    sys_var *tmp=find_sys_var(YYTHD, $1.str, $1.length);
+	    sys_var *tmp=find_sys_var(thd, $1.str, $1.length);
 	    if (!tmp)
 	      MYSQL_YYABORT;
 	    $$.var= tmp;
@@ -11376,8 +11399,27 @@ view_tail:
 	  if (!lex->select_lex.add_table_to_list(thd, $3, NULL, TL_OPTION_UPDATING))
 	    MYSQL_YYABORT;
 	}
-	view_list_opt AS view_select
-	{}
+	view_list_opt AS
+	{
+	  THD *thd= YYTHD;
+	  Lex_input_stream *lip= thd->m_lip;
+
+	  lip->body_utf8_start(thd, lip->get_cpp_ptr());
+	}
+	view_select
+	{
+	  THD *thd= YYTHD;
+          LEX *lex= thd->lex;
+	  Lex_input_stream *lip= thd->m_lip;
+
+	  lip->body_utf8_append(lip->get_cpp_ptr());
+
+          lex->view_body_utf8.str= thd->strmake(lip->get_body_utf8_str(),
+                                                lip->get_body_utf8_length());
+          lex->view_body_utf8.length= lip->get_body_utf8_length();
+
+          trim_whitespace(&my_charset_utf8_general_ci, &lex->view_body_utf8);
+	}
 	;
 
 view_list_opt:
@@ -11504,7 +11546,7 @@ trigger_tail:
 
 	  bzero((char *)&lex->sp_chistics, sizeof(st_sp_chistics));
 	  lex->sphead->m_chistics= &lex->sp_chistics;
-          lex->sphead->set_body_begin_ptr(lip, lip->get_cpp_ptr());
+          lex->sphead->set_body_start(thd, lip->get_cpp_ptr());
 	}
         sp_proc_stmt /* $16 */
         { /* $17 */
@@ -11512,7 +11554,7 @@ trigger_tail:
 	  sp_head *sp= lex->sphead;
 
 	  lex->sql_command= SQLCOM_CREATE_TRIGGER;
-	  sp->init_strings(YYTHD, lex);
+	  sp->set_stmt_end(YYTHD);
 	  /* Restore flag if it was cleared above */
 
 	  YYTHD->client_capabilities |= $<ulong_num>15;
@@ -11607,14 +11649,14 @@ sp_tail:
           Lex_input_stream *lip= thd->m_lip;
 
 	  lex->sphead->m_chistics= &lex->sp_chistics;
-          lex->sphead->set_body_begin_ptr(lip, lip->get_cpp_tok_start());
+          lex->sphead->set_body_start(thd, lip->get_cpp_tok_start());
 	}
 	sp_proc_stmt
 	{
 	  LEX *lex= Lex;
 	  sp_head *sp= lex->sphead;
 
-	  sp->init_strings(YYTHD, lex);
+	  sp->set_stmt_end(YYTHD);
 	  lex->sql_command= SQLCOM_CREATE_PROCEDURE;
           /*
             Restore flag if it was cleared above