Added progress reporting for alter table, LOAD DATA INFILE and for aria tables: check table, repair table, analyze table.

- The client gets a progress report message that triggers a callback function if requested with mysql_options(MYSQL_PROGRESS_CALLBACK, function)
- Added Progress field last to 'show processlist'
- Stage, Max_stage and Progress field added to information_schema.progresslist
- The 'mysql' client by defaults enables progress reports when the output is a tty.
- Added progress_report_time time variable to configure how often progress reports is sent to client
Added read only system variable 'in_transaction' which is 1 if we have executed a BEGIN statement.


client/client_priv.h:
  Added OPT_REPORT_PROGRESS
client/mysql.cc:
  Added option --progress-reports (on by default if not batch mode)
  Progress reports is written to stdout for long running commands
include/Makefile.am:
  Added mysql/service_progress_report.h
include/myisamchk.h:
  Added variables to be able to do progress reporting in Aria and later in MyISAM
include/mysql.h:
  Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK
include/mysql.h.pp:
  Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK
include/mysql/plugin.h:
  Added functions for reporting progress.
include/mysql/plugin_auth.h.pp:
  Added functions for reporting progress.
include/mysql_com.h:
  Added CLIENT_PROGRESS mysql_real_connect() flag.
include/sql_common.h:
  Added callback function for reporting progress
mysql-test/r/old-mode.result:
  Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode.
mysql-test/suite/funcs_1/datadict/datadict_priv.inc:
  Added new column
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Test all new PROCESSLIST columns
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Updated results
mysql-test/suite/funcs_1/r/is_columns_is_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/is_tables_is_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_priv_ps.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_ps.result:
  Updated results
mysql-test/suite/pbxt/r/pbxt_locking.result:
  Updated results
mysql-test/suite/pbxt/r/skip_name_resolve.result:
  Updated results
mysql-test/t/old-mode.test:
  Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode.
plugin/handler_socket/handlersocket/Makefile.am:
  Added -lmysqlservices
scripts/mytop.sh:
  Made 'State' field width dynamic.
  Added 'Progress' to process list display.
sql-common/client.c:
  Added handling of progress messages.
  Removed check_license() function.
sql/mysql_priv.h:
  Added opt_progress_report_time
sql/mysqld.cc:
  Added progress_report_time time variable to configure how often progress reports is sent to client
sql/protocol.cc:
  Added net_send_progress_packet()
sql/protocol.h:
  New prototypes
sql/set_var.cc:
  Added variables progress_report_time and in_transaction
sql/sql_acl.cc:
  Safety fix: Made client_capabilities ulonglong
sql/sql_class.cc:
  Added interface functions for progress reporting
sql/sql_class.h:
  Added varibles in THD for progress reporting.
  Added CF_REPORT_PROGRESS
sql/sql_load.cc:
  Added progress reporting for LOAD DATA INFILE
sql/sql_parse.cc:
  Added CF_REPORT_PROGRESS for top level commands for which it's safe to send progress reports to client
sql/sql_show.cc:
  Added Progress field last to 'show processlist'
  Stage, Max_stage and Progress field added to information_schema.progresslist
sql/sql_table.cc:
  Added progress reporting for ALTER TABLE
  Added THD as argument to copy_data_between_tables()
storage/maria/ha_maria.cc:
  Added progress reporting for check table, repair table, analyze table
  Fixed a bug in start_bulk_insert() that caused alter table to always run with all keys enabled.
storage/maria/ma_check.c:
  Added progress reporting
  Remember old state before starting repair. This removes some warnings from optimize_table if create-with-sort fails.
storage/maria/ma_check_standalone.h:
  Added dummy reporting function for standalone Aria programs.
storage/maria/ma_sort.c:
  Added progress reporting
storage/maria/maria_chk.c:
  Updated version
storage/maria/maria_def.h:
  Added new prototypes
tests/mysql_client_test.c:
  Added test case for progress reporting

11 files changed, 125 insertions, 9 deletions

diff --git a/include/Makefile.am b/include/Makefile.am
index 99a21d686a3..2607afb938e 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -22,6 +22,7 @@ HEADERS_ABI =		mysql.h mysql_com.h mysql_time.h \
 			my_list.h my_alloc.h typelib.h mysql/plugin.h
 pkginclude_HEADERS =	$(HEADERS_ABI) my_dbug.h m_string.h my_sys.h \
 			my_xml.h mysql_embed.h mysql/services.h \
+			mysql/service_progress_report.h
 			mysql/service_my_snprintf.h mysql/service_thd_alloc.h \
 		  	my_pthread.h my_no_pthread.h \
 			mysql/plugin_auth.h mysql/client_plugin.h \
diff --git a/include/myisamchk.h b/include/myisamchk.h
index 7d7100bdd71..48e4a8d1b9c 100644
--- a/include/myisamchk.h
+++ b/include/myisamchk.h
@@ -155,6 +155,11 @@ typedef struct st_handler_check_param
   char temp_filename[FN_REFLEN];
   IO_CACHE read_cache;
   enum_handler_stats_method stats_method;
+  /* For reporting progress */
+  uint stage, max_stage;
+  uint progress_counter;             /* How often to call _report_progress() */
+  ulonglong progress, max_progress;
+
 #ifdef THREAD
   pthread_mutex_t print_msg_mutex;
   my_bool need_print_msg_lock;
diff --git a/include/mysql.h b/include/mysql.h
index 19aab89283b..2e412468bbe 100644
--- a/include/mysql.h
+++ b/include/mysql.h
@@ -169,7 +169,8 @@ enum mysql_option
   MYSQL_OPT_USE_REMOTE_CONNECTION, MYSQL_OPT_USE_EMBEDDED_CONNECTION,
   MYSQL_OPT_GUESS_CONNECTION, MYSQL_SET_CLIENT_IP, MYSQL_SECURE_AUTH,
   MYSQL_REPORT_DATA_TRUNCATION, MYSQL_OPT_RECONNECT,
-  MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH
+  MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH,
+  MYSQL_PROGRESS_CALLBACK
 };
 
 /**
diff --git a/include/mysql.h.pp b/include/mysql.h.pp
index 7b29c1f2ce9..2ad38bbf82e 100644
--- a/include/mysql.h.pp
+++ b/include/mysql.h.pp
@@ -258,7 +258,8 @@ enum mysql_option
   MYSQL_OPT_USE_REMOTE_CONNECTION, MYSQL_OPT_USE_EMBEDDED_CONNECTION,
   MYSQL_OPT_GUESS_CONNECTION, MYSQL_SET_CLIENT_IP, MYSQL_SECURE_AUTH,
   MYSQL_REPORT_DATA_TRUNCATION, MYSQL_OPT_RECONNECT,
-  MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH
+  MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH,
+  MYSQL_PROGRESS_CALLBACK
 };
 struct st_mysql_options_extention;
 struct st_mysql_options {
diff --git a/include/mysql/plugin.h b/include/mysql/plugin.h
index 77d141d5282..4437f5e94ef 100644
--- a/include/mysql/plugin.h
+++ b/include/mysql/plugin.h
@@ -1,4 +1,5 @@
 /* Copyright (C) 2005 MySQL AB, 2009 Sun Microsystems, Inc.
+   Copyright (C) 2009-2011 Monty Program Ab
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -60,7 +61,7 @@ typedef struct st_mysql_xid MYSQL_XID;
 /* MySQL plugin interface version */
 #define MYSQL_PLUGIN_INTERFACE_VERSION 0x0101
 /* MariaDB plugin interface version */
-#define MARIA_PLUGIN_INTERFACE_VERSION 0x0100
+#define MARIA_PLUGIN_INTERFACE_VERSION 0x0101
 
 /*
   The allowable types of plugins
@@ -747,10 +748,6 @@ char *thd_security_context(MYSQL_THD thd, char *buffer, unsigned int length,
 /* Increments the row counter, see THD::row_count */
 void thd_inc_row_count(MYSQL_THD thd);
 
-#define thd_proc_info(thd, msg)  set_thd_proc_info(thd, msg, __func__, __FILE__, __LINE__)
-const char *set_thd_proc_info(MYSQL_THD, const char * info, const char *func,
-                              const char *file, const unsigned int line);
-
 /**
   Create a temporary file.
 
diff --git a/include/mysql/plugin_auth.h.pp b/include/mysql/plugin_auth.h.pp
index b0d5daf4c64..e4878d28f94 100644
--- a/include/mysql/plugin_auth.h.pp
+++ b/include/mysql/plugin_auth.h.pp
@@ -31,6 +31,27 @@ void *thd_memdup(void* thd, const void* str, unsigned int size);
 MYSQL_LEX_STRING *thd_make_lex_string(void* thd, MYSQL_LEX_STRING *lex_str,
                                       const char *str, unsigned int size,
                                       int allocate_lex_string);
+#include <mysql/service_progress_report.h>
+extern struct progress_report_service_st {
+  void (*thd_progress_init_func)(void* thd, unsigned int max_stage);
+  void (*thd_progress_report_func)(void* thd,
+                                   unsigned long long progress,
+                                   unsigned long long max_progress);
+  void (*thd_progress_next_stage_func)(void* thd);
+  void (*thd_progress_end_func)(void* thd);
+  const char *(*set_thd_proc_info_func)(void*, const char *info,
+                                        const char *func,
+                                        const char *file,
+                                        unsigned int line);
+} *progress_report_service;
+void thd_progress_init(void* thd, unsigned int max_stage);
+void thd_progress_report(void* thd,
+                         unsigned long long progress,
+                         unsigned long long max_progress);
+void thd_progress_next_stage(void* thd);
+void thd_progress_end(void* thd);
+const char *set_thd_proc_info(void*, const char * info, const char *func,
+                              const char *file, unsigned int line);
 struct st_mysql_xid {
   long formatID;
   long gtrid_length;
@@ -166,8 +187,6 @@ int thd_tx_isolation(const void* thd);
 char *thd_security_context(void* thd, char *buffer, unsigned int length,
                            unsigned int max_query_len);
 void thd_inc_row_count(void* thd);
-const char *set_thd_proc_info(void*, const char * info, const char *func,
-                              const char *file, const unsigned int line);
 int mysql_tmpfile(const char *prefix);
 int thd_killed(const void* thd);
 unsigned long thd_get_thread_id(const void* thd);
diff --git a/include/mysql/service_progress_report.h b/include/mysql/service_progress_report.h
new file mode 100644
index 00000000000..670b1c37630
--- /dev/null
+++ b/include/mysql/service_progress_report.h
@@ -0,0 +1,82 @@
+#ifndef MYSQL_SERVICE_PROGRESS_REPORT_INCLUDED
+/* Copyright (C) 2011 Monty Program Ab
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; version 2 of the License.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA */
+
+/**
+  @file
+  This service allows plugins to report progress of long running operations
+  to the server. The progress report is visible in SHOW PROCESSLIST,
+  INFORMATION_SCHEMA.PROCESSLIST, and is sent to the client
+  if requested.
+
+  The functions are documented at
+  http://kb.askmonty.org/en/progress-reporting#how-to-add-support-for-progress-reporting-to-a-storage-engine
+*/
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define thd_proc_info(thd, msg)  set_thd_proc_info(thd, msg, \
+                                                   __func__, __FILE__, __LINE__)
+
+extern struct progress_report_service_st {
+  void (*thd_progress_init_func)(MYSQL_THD thd, unsigned int max_stage);
+  void (*thd_progress_report_func)(MYSQL_THD thd,
+                                   unsigned long long progress,
+                                   unsigned long long max_progress);
+  void (*thd_progress_next_stage_func)(MYSQL_THD thd);
+  void (*thd_progress_end_func)(MYSQL_THD thd);
+  const char *(*set_thd_proc_info_func)(MYSQL_THD, const char *info,
+                                        const char *func,
+                                        const char *file,
+                                        unsigned int line);
+} *progress_report_service;
+
+#ifdef MYSQL_DYNAMIC_PLUGIN
+
+#define thd_progress_init(thd,max_stage) (progress_report_service->thd_progress_init_func((thd),(max_stage)))
+#define thd_progress_report(thd, progress, max_progress) (progress_report_service->thd_progress_report_func((thd), (progress), (max_progress)))
+#define thd_progress_next_stage(thd) (progress_report_service->thd_progress_next_stage_func(thd))
+#define thd_progress_end(thd) (progress_report_service->thd_progress_end_func(thd))
+#define set_thd_proc_info(thd,info,func,file,line) (progress_report_service->set_thd_proc_info_func((thd),(info),(func),(file),(line)))
+
+#else
+
+/**
+   Report progress for long running operations 
+
+   @param thd            User thread connection handle
+   @param progress       Where we are now
+   @param max_progress   Progress will continue up to this
+*/
+void thd_progress_init(MYSQL_THD thd, unsigned int max_stage);
+void thd_progress_report(MYSQL_THD thd,
+                         unsigned long long progress,
+                         unsigned long long max_progress);
+void thd_progress_next_stage(MYSQL_THD thd);
+void thd_progress_end(MYSQL_THD thd);
+const char *set_thd_proc_info(MYSQL_THD, const char * info, const char *func,
+                              const char *file, unsigned int line);
+
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#define MYSQL_SERVICE_PROGRESS_REPORT_INCLUDED
+#endif
+
diff --git a/include/mysql/services.h b/include/mysql/services.h
index 19003e66b96..b8cdfc3510d 100644
--- a/include/mysql/services.h
+++ b/include/mysql/services.h
@@ -20,6 +20,7 @@ extern "C" {
 
 #include <mysql/service_my_snprintf.h>
 #include <mysql/service_thd_alloc.h>
+#include <mysql/service_progress_report.h>
 
 #ifdef __cplusplus
 }
diff --git a/include/mysql_com.h b/include/mysql_com.h
index 96d9afce5e6..119676df5ce 100644
--- a/include/mysql_com.h
+++ b/include/mysql_com.h
@@ -158,6 +158,7 @@ enum enum_server_command
 #define CLIENT_MULTI_RESULTS    (1UL << 17) /* Enable/disable multi-results */
 
 #define CLIENT_PLUGIN_AUTH  (1UL << 19) /* Client supports plugin authentication */
+#define CLIENT_PROGRESS  (1UL << 29)   /* Client support progress indicator */
 
 #define CLIENT_SSL_VERIFY_SERVER_CERT (1UL << 30)
 #define CLIENT_REMEMBER_OPTIONS (1UL << 31)
@@ -189,6 +190,7 @@ enum enum_server_command
                            CLIENT_MULTI_RESULTS | \
                            CLIENT_SSL_VERIFY_SERVER_CERT | \
                            CLIENT_REMEMBER_OPTIONS | \
+                           CLIENT_PROGRESS | \
                            CLIENT_PLUGIN_AUTH)
 
 /*
diff --git a/include/service_versions.h b/include/service_versions.h
index 114957cdd86..6f06b609e61 100644
--- a/include/service_versions.h
+++ b/include/service_versions.h
@@ -21,4 +21,5 @@
 
 #define VERSION_my_snprintf     0x0100
 #define VERSION_thd_alloc       0x0100
+#define VERSION_progress_report 0x0100
 
diff --git a/include/sql_common.h b/include/sql_common.h
index 6b66ae2fd81..20e06275653 100644
--- a/include/sql_common.h
+++ b/include/sql_common.h
@@ -29,6 +29,12 @@ extern const char	*not_error_sqlstate;
 struct st_mysql_options_extention {
   char *plugin_dir;
   char *default_auth;
+  void (*report_progress)(const MYSQL *mysql,
+                          unsigned int stage,
+                          unsigned int max_stage,
+                          double progress,
+                          const char *proc_info,
+                          uint proc_info_length);
 };
 
 typedef struct st_mysql_methods