Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1,

2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29,
2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and
some other minor revisions.

This patch implements: 

WL#4264 "Backup: Stabilize Service Interface" -- all the
server prerequisites except si_objects.{h,cc} themselves (they can
be just copied over, when needed).

WL#4435: Support OUT-parameters in prepared statements.

(and all issues in the initial patches for these two
tasks, that were discovered in pushbuild and during testing).

Bug#39519: mysql_stmt_close() should flush all data
associated with the statement.

After execution of a prepared statement, send OUT parameters of the invoked
stored procedure, if any, to the client.

When using the binary protocol, send the parameters in an additional result
set over the wire.  When using the text protocol, assign out parameters to
the user variables from the CALL(@var1, @var2, ...) specification.

The following refactoring has been made:
  - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata();
  - A new Protocol::send_result_set_row() was introduced to incapsulate
    common functionality for sending row data.
  - Signature of Protocol::prepare_for_send() was changed: this operation
    does not need a list of items, the number of items is fully sufficient.

The following backward incompatible changes have been made:
  - CLIENT_MULTI_RESULTS is now enabled by default in the client;
  - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.

include/mysql.h:
  Add a new flag to MYSQL_METHODS::flush_use_result
  function pointer. This flag determines if all results
  should be flushed or only the first one:
      
  - if flush_all_results is TRUE, then cli_flush_use_result()
    will read/flush all pending results. I.e. it will read
    all packets while server status attribute indicates that
    there are more results. This is a new semantic, required
    to fix the bug.
              
  - if flush_all_results is FALSE, the old sematic
    is preserved -- i.e. cli_flush_use_result() reads data
    until first EOF-packet.
include/mysql.h.pp:
  Update the ABI with new calls (compatible changes).
include/mysql_com.h:
  Add CLIENT_PS_OUT_PARAMS -- a client capability indicating that the client supportsю
libmysql/libmysql.c:
  Add mysql_stmt_next_result() -- analogue of mysql_next_result() for binary protocol.
  Fix a minor bug in alloc_fields() -- not all members were copied over,
  and some only shallow-copied (catalog).
  Flush all results in mysql_stmt_close() (Bug#39519).
libmysqld/lib_sql.cc:
  Rename send_fields() -> send_result_set_metadata().
  Refactoring: change prepare_for_send() so that it accepts only 
  what it really needs -- a number of elements in the list.
mysql-test/r/ps.result:
  Update results: WL#4435.
mysql-test/t/ps.test:
  WL#4435: A test case for an SQL-part of the problem.
sql-common/client.c:
  Bug#39519.
  Implement new functionality in cli_flush_use_result():
  if flush_all_delete is TRUE, then it should read/flush
  all pending results.
sql/Makefile.am:
  Add a new header sql_prepare.h to the list
  of build headers.
sql/events.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/handler.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/mysql_priv.h:
  Move sql_prepare.cc-specific declarations to a new
  header - sql_prepare.h.
sql/procedure.h:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/protocol.cc:
  Move the logic responsible for sending of one result
  set row to the Protocol class. Define a template
  for end-of-statement action. 
  Refactoring: change prepare_for_send() so that it accepts 
  only what it really needs -- a number of elements in the list.
  Rename send_fields() to send_result_set_metadata().
sql/protocol.h:
  Update with new declarations (WL#4435).
  Rename send_fields() -> send_result_set_metadata().
  prepare_for_send() only needs the number of columns to send,
  and doesn't use the item list - update signature to require
  only what's needed.
  Add a new protocol type -- Protocol_local.
sql/repl_failsafe.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/slave.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_acl.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_base.cc:
  Include sql_prepare.h (for Reprepare_observer).
sql/sql_cache.cc:
  Extend the query cache flags block to be able
  to store a numeric id for the result format,
  not just a flag binary/non-binary.
sql/sql_class.cc:
  Update to use the rename of Protocol::send_fields()
  to Protocol::send_result_set_metadata().
  Use Protocol::send_one_result_set_row().
sql/sql_class.h:
  Move the declaration of Reprepare_observer to the 
  new header - sql_prepare.h.
  Update to the new signature of class Protocol::send_fields().
sql/sql_connect.cc:
  Use a protocol template method instead of
  raw NET layer API at the end of a statement.
sql/sql_cursor.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_error.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_handler.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
  Use new method Protocol::send_one_result_set_row().
sql/sql_help.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_lex.cc:
  Initialize multi_statements variable.
  Add a handy constant for empty lex
  string.
sql/sql_lex.h:
  Add a separate member for a standalone
  parsing option - multi-statements support.
sql/sql_list.cc:
  sql_list.h is a standalone header now, 
  no need to include mysql_priv.h.
sql/sql_list.h:
  Make sql_list.h a stand-alone header.
sql/sql_parse.cc:
  Include sql_prepare.h for prepared
  statements- related declarations.
  Use a new Protocol template method to end
  each statement (send OK, EOF or ERROR to
  the client).
sql/sql_prepare.cc:
  Implement Execute Direct API (WL#4264), 
  currently unused. It will be used by the service
  interface (Backup).
  Use a new header - sql_prepare.h.
  Add support for OUT parameters in the 
  binary and text protocol (prepared statements 
  only).
sql/sql_prepare.h:
  Add a new header to contain (for now)
  all prepared statement- external
  related declarations.
sql/sql_profile.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_repl.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_select.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_show.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_string.h:
  Add a way to convert a String to LEX_STRING.
sql/sql_table.cc:
  Rename: Protocol::send_fields() -> 
  Protocol::send_result_set_metadata().
sql/sql_update.cc:
  Remove an extraneous my_error(). The error
  is already reported in update_non_unique_table_error().
sql/sql_yacc.yy:
  Support for multi-statements is an independent
  property of parsing, not derived from 
  the protocol type.
tests/mysql_client_test.c:
  Add tests for WL#4435 (binary protocol).

2 files changed, 97 insertions, 40 deletions

diff --git a/libmysql/client_settings.h b/libmysql/client_settings.h
index f87e625771f..ec63f495552 100644
--- a/libmysql/client_settings.h
+++ b/libmysql/client_settings.h
@@ -16,9 +16,13 @@
 extern uint		mysql_port;
 extern char *	mysql_unix_port;
 
-#define CLIENT_CAPABILITIES (CLIENT_LONG_PASSWORD | CLIENT_LONG_FLAG |	  \
-                             CLIENT_TRANSACTIONS | \
-			     CLIENT_PROTOCOL_41 | CLIENT_SECURE_CONNECTION)
+#define CLIENT_CAPABILITIES (CLIENT_LONG_PASSWORD | \
+                             CLIENT_LONG_FLAG |     \
+                             CLIENT_TRANSACTIONS |  \
+                             CLIENT_PROTOCOL_41 | \
+                             CLIENT_SECURE_CONNECTION | \
+                             CLIENT_MULTI_RESULTS | \
+                             CLIENT_PS_MULTI_RESULTS)
 
 sig_handler my_pipe_sig_handler(int sig);
 void read_user_name(char *name);
diff --git a/libmysql/libmysql.c b/libmysql/libmysql.c
index f922c98de3a..56a2e492226 100644
--- a/libmysql/libmysql.c
+++ b/libmysql/libmysql.c
@@ -1722,6 +1722,8 @@ static void alloc_stmt_fields(MYSQL_STMT *stmt)
   MEM_ROOT *alloc= &stmt->mem_root;
   MYSQL *mysql= stmt->mysql;
 
+  DBUG_ASSERT(mysql->field_count);
+
   stmt->field_count= mysql->field_count;
 
   /*
@@ -1743,18 +1745,21 @@ static void alloc_stmt_fields(MYSQL_STMT *stmt)
 	 field= stmt->fields;
        field && fields < end; fields++, field++)
   {
-    field->db       = strdup_root(alloc,fields->db);
-    field->table    = strdup_root(alloc,fields->table);
-    field->org_table= strdup_root(alloc,fields->org_table);
-    field->name     = strdup_root(alloc,fields->name);
-    field->org_name = strdup_root(alloc,fields->org_name);
-    field->charsetnr= fields->charsetnr;
-    field->length   = fields->length;
-    field->type     = fields->type;
-    field->flags    = fields->flags;
-    field->decimals = fields->decimals;
-    field->def      = fields->def ? strdup_root(alloc,fields->def): 0;
-    field->max_length= 0;
+    *field= *fields; /* To copy all numeric parts. */
+    field->catalog=   strmake_root(alloc, fields->catalog,
+                                   fields->catalog_length);
+    field->db=        strmake_root(alloc, fields->db, fields->db_length);
+    field->table=     strmake_root(alloc, fields->table, fields->table_length);
+    field->org_table= strmake_root(alloc, fields->org_table,
+                                   fields->org_table_length);
+    field->name=      strmake_root(alloc, fields->name, fields->name_length);
+    field->org_name=  strmake_root(alloc, fields->org_name,
+                                   fields->org_name_length);
+    field->def=       fields->def ? strmake_root(alloc, fields->def,
+                                                 fields->def_length) : 0;
+    field->def_length= field->def ? fields->def_length : 0;
+    field->extension= 0; /* Avoid dangling links. */
+    field->max_length= 0; /* max_length is set in mysql_stmt_store_result() */
   }
 }
 
@@ -2481,6 +2486,33 @@ static void reinit_result_set_metadata(MYSQL_STMT *stmt)
 }
 
 
+static void prepare_to_fetch_result(MYSQL_STMT *stmt)
+{
+  if (stmt->server_status & SERVER_STATUS_CURSOR_EXISTS)
+  {
+    stmt->mysql->status= MYSQL_STATUS_READY;
+    stmt->read_row_func= stmt_read_row_from_cursor;
+  }
+  else if (stmt->flags & CURSOR_TYPE_READ_ONLY)
+  {
+    /*
+      This is a single-row result set, a result set with no rows, EXPLAIN,
+      SHOW VARIABLES, or some other command which either a) bypasses the
+      cursors framework in the server and writes rows directly to the
+      network or b) is more efficient if all (few) result set rows are
+      precached on client and server's resources are freed.
+    */
+    mysql_stmt_store_result(stmt);
+  }
+  else
+  {
+    stmt->mysql->unbuffered_fetch_owner= &stmt->unbuffered_fetch_cancelled;
+    stmt->unbuffered_fetch_cancelled= FALSE;
+    stmt->read_row_func= stmt_read_row_unbuffered;
+  }
+}
+
+
 /*
   Send placeholders data to server (if there are placeholders)
   and execute prepared statement.
@@ -2548,28 +2580,7 @@ int STDCALL mysql_stmt_execute(MYSQL_STMT *stmt)
   if (mysql->field_count)
   {
     reinit_result_set_metadata(stmt);
-    if (stmt->server_status & SERVER_STATUS_CURSOR_EXISTS)
-    {
-      mysql->status= MYSQL_STATUS_READY;
-      stmt->read_row_func= stmt_read_row_from_cursor;
-    }
-    else if (stmt->flags & CURSOR_TYPE_READ_ONLY)
-    {
-      /*
-        This is a single-row result set, a result set with no rows, EXPLAIN,
-        SHOW VARIABLES, or some other command which either a) bypasses the
-        cursors framework in the server and writes rows directly to the
-        network or b) is more efficient if all (few) result set rows are
-        precached on client and server's resources are freed.
-      */
-      mysql_stmt_store_result(stmt);
-    }
-    else
-    {
-      stmt->mysql->unbuffered_fetch_owner= &stmt->unbuffered_fetch_cancelled;
-      stmt->unbuffered_fetch_cancelled= FALSE;
-      stmt->read_row_func= stmt_read_row_unbuffered;
-    }
+    prepare_to_fetch_result(stmt);
   }
   DBUG_RETURN(test(stmt->last_errno));
 }
@@ -4034,7 +4045,6 @@ static my_bool setup_one_fetch_function(MYSQL_BIND *param, MYSQL_FIELD *field)
     field->max_length= 10;                    /* 2003-11-11 */
     param->skip_result= skip_result_with_length;
     break;
-    break;
   case MYSQL_TYPE_DATETIME:
   case MYSQL_TYPE_TIMESTAMP:
     param->skip_result= skip_result_with_length;
@@ -4614,7 +4624,7 @@ static my_bool reset_stmt_handle(MYSQL_STMT *stmt, uint flags)
         if (stmt->field_count && mysql->status != MYSQL_STATUS_READY)
         {
           /* There is a result set and it belongs to this statement */
-          (*mysql->methods->flush_use_result)(mysql);
+          (*mysql->methods->flush_use_result)(mysql, FALSE);
           if (mysql->unbuffered_fetch_owner)
             *mysql->unbuffered_fetch_owner= TRUE;
           mysql->status= MYSQL_STATUS_READY;
@@ -4698,7 +4708,7 @@ my_bool STDCALL mysql_stmt_close(MYSQL_STMT *stmt)
           Flush result set of the connection. If it does not belong
           to this statement, set a warning.
         */
-        (*mysql->methods->flush_use_result)(mysql);
+        (*mysql->methods->flush_use_result)(mysql, TRUE);
         if (mysql->unbuffered_fetch_owner)
           *mysql->unbuffered_fetch_owner= TRUE;
         mysql->status= MYSQL_STATUS_READY;
@@ -4846,6 +4856,49 @@ int STDCALL mysql_next_result(MYSQL *mysql)
 }
 
 
+int STDCALL mysql_stmt_next_result(MYSQL_STMT *stmt)
+{
+  MYSQL *mysql= stmt->mysql;
+  int rc;
+  DBUG_ENTER("mysql_stmt_next_result");
+
+  if (!mysql)
+    DBUG_RETURN(1);
+
+  if (stmt->last_errno)
+    DBUG_RETURN(stmt->last_errno);
+
+  if (mysql->server_status & SERVER_MORE_RESULTS_EXISTS)
+  {
+    if (reset_stmt_handle(stmt, RESET_STORE_RESULT))
+      DBUG_RETURN(1);
+  }
+
+  rc= mysql_next_result(mysql);
+
+  if (rc)
+  {
+    set_stmt_errmsg(stmt, &mysql->net);
+    DBUG_RETURN(rc);
+  }
+
+  stmt->state= MYSQL_STMT_EXECUTE_DONE;
+  stmt->bind_result_done= FALSE;
+
+  if (mysql->field_count)
+  {
+    alloc_stmt_fields(stmt);
+    prepare_to_fetch_result(stmt);
+  }
+  else
+  {
+    stmt->field_count= mysql->field_count;
+  }
+
+  DBUG_RETURN(0);
+}
+
+
 MYSQL_RES * STDCALL mysql_use_result(MYSQL *mysql)
 {
   return (*mysql->methods->use_result)(mysql);