MDEV-4011 Added per thread memory counting and usage

Base code and idea from a patch from by plinux at Taobao.

The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.

Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.

The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array()  has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.

Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)


client/completion_hash.cc:
  Updated call to init_alloc_root()
client/mysql.cc:
  Updated call to init_alloc_root()
client/mysqlbinlog.cc:
  init_dynamic_array() -> my_init_dynamic_array()
  Updated call to init_alloc_root()
client/mysqlcheck.c:
  Updated call to my_init_dynamic_array()
client/mysqldump.c:
  Updated call to init_alloc_root()
client/mysqltest.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Fixed compiler warnings
extra/comp_err.c:
  Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
  Updated call to my_init_dynamic_array()
include/hash.h:
  Added HASH_THREAD_SPECIFIC
include/heap.h:
  Added flag is internal temporary table.
include/my_dir.h:
  Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
  Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
  Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
  Added malloc_flags to DYNAMIC_ARRAY
  Added extra mysys flag argument to my_init_dynamic_array()
  Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
  Updated paramaters for init_alloc_root()
include/my_tree.h:
  Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
  Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
  Updated parameters to init_tree()
include/myisamchk.h:
  Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
  Added MYSQL_THREAD_SPECIFIC_MALLOC
  Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
  Updated file
include/mysql_com.h:
  Used 'unused1' to mark memory as thread specific.
  Updated parameters for my_net_init()
libmysql/libmysql.c:
  Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
  Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
  Updated call to init_alloc_root()
mysql-test/r/create.result:
  Updated results
mysql-test/r/user_var.result:
  Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/t/show_explain.test:
  Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
  Added test of memory_usage variable.
mysys/array.c:
  Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
  All allocated memory is marked with the given my_flags.
  Removed obsolete function init_dynamic_array()
mysys/default.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
mysys/hash.c:
  Updated call to my_init_dynamic_array_ci().
  Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
  init_dynamic_array() -> my_init_dynamic_array()
  Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
  Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
  Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
  Extend MEM_ROOT with a flag if memory is thread specific.
  This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
  Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
  Store at start of each allocated memory block the size of the block and if the block is thread specific.
  Call malloc_size_cb_func, if set, with the memory allocated/freed.
  Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
  Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
  Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
  Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
  Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
  Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
  Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
  Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
  Added flags to init_tree() so that one can mark tree to be thread specific.
  Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
  Updated call to init_alloc_root()
  All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
  Updated call to my_init_dynamic_array()
sql-common/client.c:
  Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
  Updated call to my_init_dynamic_array().
  Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
  Updated call to init_alloc_root()
sql/debug_sync.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
  Removed calls to net_end() as this is now done in ~THD()
  Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
  Updated call to init_alloc_root()
  Updated call to my_net_init()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
  Updated call to my_net_init()
  Updated call to init_sql_alloc()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
  Updated call to init_alloc_root()
sql/handler.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added missing call to my_dir_end()
sql/item_func.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
  More DBUG
  Updated call to init_alloc_root()
sql/mdl.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
  Added total_memory_used
  Updated call to init_alloc_root()
  Move mysql_cond_broadcast() before my_thread_end()
  Added mariadb_dbug_id() to count memory per THD instead of per thread.
  Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
  Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
  Updated call to my_init_dynamic_array().
  Updated call to my_net_init().
  Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
  Added status variable 'memory_used'.
  Updated call to init_alloc_root()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
  Added set_current_thd()
sql/net_serv.cc:
  Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
  Store in net->thread_specific_malloc if memory is thread specific.
  Mark memory to be thread specific if requested.
sql/opt_range.cc:
  Updated call to my_init_dynamic_array()
  Updated call to init_sql_alloc()
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
  Fixed compiler warning
sql/records.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
  Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
  Updated call to init_sql_alloc()
sql/rpl_mi.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
  Updated call to init_alloc_root()
sql/rpl_utility.cc:
  Updated call to my_init_dynamic_array()
sql/slave.cc:
  Initialize things properly before calling functions that allocate memory.
  Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
  Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
  Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
  Updated call to my_init_dynamic_array()
sql/sql_base.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
  Updated comment
sql/sql_class.cc:
  Added parameter to warning_info() that not initialize it until THD is fully created.
  Updated call to init_sql_alloc()
  Mark THD::user_vars has to be thread specific.
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by THD is assigned to the THD.
  More DBUG
  Always acll net_end() in ~THD()
  Assert that all memory signed to this THD is really deleted at ~THD.
  Fixed set_status_var_init() to not reset memory_used.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
  Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
  This is used by THD::THD() to not allocate memory until THD is ready.
  Added Warning_info::free_memory()
sql/sql_error.h:
  Updated Warning_info() class.
sql/sql_handler.cc:
  Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
  More DBUG
sql/sql_join_cache.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
  Updated call to my_init_dynamic_array()
sql/sql_lex.h:
  Updated call to my_init_dynamic_array()
sql/sql_load.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
  Ensure that examined_row_count() is reset before query.
  Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
  Added parameter to warning_info() that it should be fully initialized.
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
  Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
  Added MY_THREAD_SPECIFIC to allocated memory.
  More DBUG
sql/sql_servers.cc:
  Updated call to init_sql_alloc() to mark memory some memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
  Updated call to my_init_dynamic_array()
  Mark my_dir() memory thread specific.
  Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
  More DBUG.
  Added malloc_size and examined_row_count to SHOW PROCESSLIST.
  Added MY_THREAD_SPECIFIC to allocated memory.
  Updated call to init_sql_alloc()
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
  Fixed compiler warning
sql/sql_string.cc:
  String elements can now be marked as thread specific.
sql/sql_string.h:
  String elements can now be marked as thread specific.
sql/sql_table.cc:
  Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Fixed compiler warning
sql/sql_test.cc:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
  Updated call to init_sql_alloc()
sql/sql_udf.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
  Updated call to init_sql_alloc().
  Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
  Updated call to init_sql_alloc()
sql/thr_malloc.cc:
  Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
  Updated prototype for init_sql_alloc()
sql/tztime.cc:
  Updated call to init_sql_alloc()
  Updated call to init_alloc_root() to mark memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
  Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
  Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
  Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
  - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
  - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
  Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
  Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
  Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
  Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
  Updated call to init_tree()
storage/maria/ma_ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
  Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
  Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
  Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
  Updated call to init_tree()
storage/myisam/ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
  Updated call to init_tree()
storage/myisam/mi_check.c:
  Updated call to init_alloc_root()
storage/myisam/mi_write.c:
  Updated call to my_init_dynamic_array()
  Updated call to init_tree()
storage/myisam/myisamlog.c:
  Updated call to init_tree()
storage/myisam/myisampack.c:
  Updated call to init_tree()
storage/myisam/sort.c:
  Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
  Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
  Rest current_thd
storage/perfschema/pfs_instr.cc:
  Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
  Disable compiler warning from offsetof macro.

1 files changed, 99 insertions, 20 deletions

diff --git a/sql/mysqld.cc b/sql/mysqld.cc
index 6d3c5980280..ca68f8c9ae2 100644
--- a/sql/mysqld.cc
+++ b/sql/mysqld.cc
@@ -1121,7 +1121,7 @@ private:
 
 void Buffered_logs::init()
 {
-  init_alloc_root(&m_root, 1024, 0);
+  init_alloc_root(&m_root, 1024, 0, 0);
 }
 
 void Buffered_logs::cleanup()
@@ -1765,6 +1765,7 @@ extern "C" void unireg_abort(int exit_code)
 
 static void mysqld_exit(int exit_code)
 {
+  DBUG_ENTER("mysqld_exit");
   /*
     Important note: we wait for the signal thread to end,
     but if a kill -15 signal was sent, the signal thread did
@@ -1776,6 +1777,7 @@ static void mysqld_exit(int exit_code)
   clean_up_error_log_mutex();
   my_end((opt_endinfo ? MY_CHECK_ERROR | MY_GIVE_INFO : 0));
   shutdown_performance_schema();        // we do it as late as possible
+  DBUG_LEAVE;
   exit(exit_code); /* purecov: inspected */
 }
 
@@ -2485,9 +2487,13 @@ void unlink_thd(THD *thd)
     sync feature has been shut down at this point.
   */
   DBUG_EXECUTE_IF("sleep_after_lock_thread_count_before_delete_thd", sleep(5););
+  /*
+    We must delete thd inside the lock to ensure that we don't start cleanup
+    before THD is deleted
+  */
+  delete thd;
   mysql_mutex_unlock(&LOCK_thread_count);
 
-  delete thd;
   DBUG_VOID_RETURN;
 }
 
@@ -2594,7 +2600,7 @@ bool one_thread_per_connection_end(THD *thd, bool put_in_cache)
   DBUG_ENTER("one_thread_per_connection_end");
   unlink_thd(thd);
   /* Mark that current_thd is not valid anymore */
-  my_pthread_setspecific_ptr(THR_THD,  0);
+  set_current_thd(0);
   if (put_in_cache)
   {
     mysql_mutex_lock(&LOCK_thread_count);
@@ -2606,9 +2612,10 @@ bool one_thread_per_connection_end(THD *thd, bool put_in_cache)
 
   /* It's safe to broadcast outside a lock (COND... is not deleted here) */
   DBUG_PRINT("signal", ("Broadcasting COND_thread_count"));
+  mysql_cond_broadcast(&COND_thread_count);
+
   DBUG_LEAVE;                                   // Must match DBUG_ENTER()
   my_thread_end();
-  mysql_cond_broadcast(&COND_thread_count);
 
   pthread_exit(0);
   return 0;                                     // Avoid compiler warnings
@@ -3142,9 +3149,9 @@ void my_message_sql(uint error, const char *str, myf MyFlags)
   THD *thd= current_thd;
   MYSQL_ERROR::enum_warning_level level;
   sql_print_message_func func;
-
   DBUG_ENTER("my_message_sql");
-  DBUG_PRINT("error", ("error: %u  message: '%s'  Flag: %d", error, str, MyFlags));
+  DBUG_PRINT("error", ("error: %u  message: '%s'  Flag: %lu", error, str,
+                       MyFlags));
 
   DBUG_ASSERT(str != NULL);
   DBUG_ASSERT(error != 0);
@@ -3430,6 +3437,28 @@ SHOW_VAR com_status_vars[]= {
   {NullS, NullS, SHOW_LONG}
 };
 
+#ifdef SAFEMALLOC
+/*
+  Return the id for the current THD, to allow safemalloc to associate
+  the memory with the right id.
+*/
+
+extern "C" my_thread_id mariadb_dbug_id()
+{
+  THD *thd;
+  if ((thd= current_thd))
+  {
+    return thd->thread_id;
+  }
+  return my_thread_dbug_id();
+}
+#endif /* SAFEMALLOC */
+
+
+/*
+  Init common variables
+*/
+
 static int init_common_variables()
 {
   umask(((~my_umask) & 0666));
@@ -3438,6 +3467,9 @@ static int init_common_variables()
   tzset();			// Set tzname
 
   sf_leaking_memory= 0; // no memory leaks from now on
+#ifdef SAFEMALLOC
+  sf_malloc_dbug_id= mariadb_dbug_id;
+#endif
 
   max_system_variables.pseudo_thread_id= (ulong)~0;
   server_start_time= flush_status_time= my_time(0);
@@ -3897,6 +3929,7 @@ You should consider changing lower_case_table_names to 1 or 2",
 
 static int init_thread_environment()
 {
+  DBUG_ENTER("init_thread_environment");
   mysql_mutex_init(key_LOCK_thread_count, &LOCK_thread_count, MY_MUTEX_INIT_FAST);
   mysql_mutex_init(key_LOCK_status, &LOCK_status, MY_MUTEX_INIT_FAST);
   mysql_mutex_init(key_LOCK_delayed_insert,
@@ -3975,9 +4008,9 @@ static int init_thread_environment()
       pthread_key_create(&THR_MALLOC,NULL))
   {
     sql_print_error("Can't create thread-keys");
-    return 1;
+    DBUG_RETURN(1);
   }
-  return 0;
+  DBUG_RETURN(0);
 }
 
 
@@ -4641,6 +4674,37 @@ static void test_lc_time_sz()
 }
 #endif//DBUG_OFF
 
+
+/* Thread Mem Usage By P.Linux */
+extern "C"
+void my_malloc_size_cb_func(long long size, myf my_flags)
+{
+  /* If thread specific memory */
+  if (my_flags)
+  {
+    THD *thd= current_thd;
+    if (mysqld_server_initialized || thd)
+    {
+      /*
+        THD may not be set if we are called from my_net_init() before THD
+        thread has started.
+        However, this should never happen, so better to assert and
+        fix this.
+      */
+      DBUG_ASSERT(thd);
+      if (thd)
+      {
+        DBUG_PRINT("info", ("memory_used: %lld  size: %lld",
+                            (longlong) thd->status_var.memory_used, size));
+        thd->status_var.memory_used+= size;
+        DBUG_ASSERT((longlong) thd->status_var.memory_used >= 0);
+      }
+    }
+  }
+  my_atomic_add64(&global_status_var.memory_used, size);
+}
+
+
 #ifdef __WIN__
 int win_main(int argc, char **argv)
 #else
@@ -4653,6 +4717,9 @@ int mysqld_main(int argc, char **argv)
   */
   my_progname= argv[0];
   sf_leaking_memory= 1; // no safemalloc memory leak reports if we exit early
+  set_malloc_size_cb(my_malloc_size_cb_func);
+  mysqld_server_started= mysqld_server_initialized= 0;
+
 #ifdef HAVE_NPTL
   ld_assume_kernel_is_set= (getenv("LD_ASSUME_KERNEL") != 0);
 #endif
@@ -4665,7 +4732,6 @@ int mysqld_main(int argc, char **argv)
   }
 #endif
 
-  mysqld_server_started= mysqld_server_initialized= 0;
   orig_argc= argc;
   orig_argv= argv;
   my_getopt_use_args_separator= TRUE;
@@ -4695,7 +4761,7 @@ int mysqld_main(int argc, char **argv)
   my_getopt_skip_unknown= TRUE;
 
   /* prepare all_early_options array */
-  my_init_dynamic_array(&all_early_options, sizeof(my_option), 100, 25);
+  my_init_dynamic_array(&all_early_options, sizeof(my_option), 100, 25, 0);
   sys_var_add_options(&all_early_options, sys_var::PARSE_EARLY);
   add_terminator(&all_early_options);
 
@@ -4956,8 +5022,6 @@ int mysqld_main(int argc, char **argv)
   if (Events::init(opt_noacl || opt_bootstrap))
     unireg_abort(1);
 
-  mysqld_server_initialized= 1;
-
   if (opt_bootstrap)
   {
     select_thread_in_use= 0;                    // Allow 'kill' to work
@@ -4971,6 +5035,9 @@ int mysqld_main(int argc, char **argv)
     }
   }
 
+  /* It's now safe to use thread specific memory */
+  mysqld_server_initialized= 1;
+
   create_shutdown_thread();
   start_handle_manager();
 
@@ -5000,7 +5067,6 @@ int mysqld_main(int argc, char **argv)
   Service.SetRunning();
 #endif
 
-
   /* Signal threads waiting for server to be started */
   mysql_mutex_lock(&LOCK_server_started);
   mysqld_server_started= 1;
@@ -5180,6 +5246,7 @@ int mysqld_main(int argc, char **argv)
 
   /* Must be initialized early for comparison of service name */
   system_charset_info= &my_charset_utf8_general_ci;
+  set_malloc_size_cb(my_malloc_size_cb_func);
 
   if (my_init())
   {
@@ -5288,7 +5355,7 @@ static void bootstrap(MYSQL_FILE *file)
 
   THD *thd= new THD;
   thd->bootstrap=1;
-  my_net_init(&thd->net,(st_vio*) 0);
+  my_net_init(&thd->net,(st_vio*) 0, 0);
   thd->max_client_packet_length= thd->net.max_packet;
   thd->security_ctx->master_access= ~(ulong)0;
   thd->thread_id= thd->variables.pseudo_thread_id= thread_id++;
@@ -5705,17 +5772,20 @@ void handle_connections_sockets()
     ** Don't allow too many connections
     */
 
+    DBUG_PRINT("info", ("Creating THD for new connection"));
     if (!(thd= new THD))
     {
       (void) mysql_socket_shutdown(new_sock, SHUT_RDWR);
       (void) closesocket(new_sock);
       continue;
     }
+    /* Set to get io buffers to be part of THD */
+    set_current_thd(thd);
     if (!(vio_tmp=vio_new(new_sock,
 			  sock == unix_sock ? VIO_TYPE_SOCKET :
 			  VIO_TYPE_TCPIP,
 			  sock == unix_sock ? VIO_LOCALHOST: 0)) ||
-	my_net_init(&thd->net,vio_tmp))
+	my_net_init(&thd->net, vio_tmp, MYF(MY_THREAD_SPECIFIC)))
     {
       /*
         Only delete the temporary vio if we didn't already attach it to the
@@ -5730,6 +5800,7 @@ void handle_connections_sockets()
 	(void) closesocket(new_sock);
       }
       delete thd;
+      set_current_thd(0);
       continue;
     }
     if (sock == unix_sock)
@@ -5741,6 +5812,7 @@ void handle_connections_sockets()
       thd->scheduler= extra_thread_scheduler;
     }
     create_new_thread(thd);
+    set_current_thd(0);
   }
   DBUG_VOID_RETURN;
 }
@@ -5834,16 +5906,19 @@ pthread_handler_t handle_connections_namedpipes(void *arg)
       CloseHandle(hConnectedPipe);
       continue;
     }
+    set_current_thd(thd);
     if (!(thd->net.vio= vio_new_win32pipe(hConnectedPipe)) ||
-	my_net_init(&thd->net, thd->net.vio))
+	my_net_init(&thd->net, thd->net.vio, MYF(MY_THREAD_SPECIFIC)))
     {
       close_connection(thd, ER_OUT_OF_RESOURCES);
       delete thd;
+      set_current_thd(0);
       continue;
     }
     /* Host is unknown */
     thd->security_ctx->host= my_strdup(my_localhost, MYF(0));
     create_new_thread(thd);
+    set_current_thd(0);
   }
   CloseHandle(connectOverlapped.hEvent);
   DBUG_LEAVE;
@@ -6023,6 +6098,7 @@ pthread_handler_t handle_connections_shared_memory(void *arg)
       errmsg= "Could not set client to read mode";
       goto errorconn;
     }
+    set_current_thd(thd);
     if (!(thd->net.vio= vio_new_win32shared_memory(handle_client_file_map,
                                                    handle_client_map,
                                                    event_client_wrote,
@@ -6030,7 +6106,7 @@ pthread_handler_t handle_connections_shared_memory(void *arg)
                                                    event_server_wrote,
                                                    event_server_read,
                                                    event_conn_closed)) ||
-                        my_net_init(&thd->net, thd->net.vio))
+        my_net_init(&thd->net, thd->net.vio, MYF(MY_THREAD_SPECIFIC)))
     {
       close_connection(thd, ER_OUT_OF_RESOURCES);
       errmsg= 0;
@@ -6039,6 +6115,7 @@ pthread_handler_t handle_connections_shared_memory(void *arg)
     thd->security_ctx->host= my_strdup(my_localhost, MYF(0)); /* Host is unknown */
     create_new_thread(thd);
     connect_number++;
+    set_current_thd(thd);
     continue;
 
 errorconn:
@@ -6066,6 +6143,7 @@ errorconn:
       CloseHandle(event_conn_closed);
     delete thd;
   }
+  set_current_thd(0);
 
   /* End shared memory handling */
 error:
@@ -7024,6 +7102,7 @@ SHOW_VAR status_vars[]= {
   {"Key",                      (char*) &show_default_keycache, SHOW_FUNC},
   {"Last_query_cost",          (char*) offsetof(STATUS_VAR, last_query_cost), SHOW_DOUBLE_STATUS},
   {"Max_used_connections",     (char*) &max_used_connections,  SHOW_LONG},
+  {"Memory_used",              (char*) offsetof(STATUS_VAR, memory_used), SHOW_LONGLONG_STATUS},
   {"Not_flushed_delayed_rows", (char*) &delayed_rows_in_use,    SHOW_LONG_NOFLUSH},
   {"Open_files",               (char*) &my_file_opened,         SHOW_LONG_NOFLUSH},
   {"Open_streams",             (char*) &my_stream_opened,       SHOW_LONG_NOFLUSH},
@@ -7170,7 +7249,7 @@ static int option_cmp(my_option *a, my_option *b)
 static void print_help()
 {
   MEM_ROOT mem_root;
-  init_alloc_root(&mem_root, 4096, 4096);
+  init_alloc_root(&mem_root, 4096, 4096, 0);
 
   pop_dynamic(&all_options);
   sys_var_add_options(&all_options, sys_var::PARSE_EARLY);
@@ -7862,7 +7941,7 @@ static int get_options(int *argc_ptr, char ***argv_ptr)
   /* prepare all_options array */
   my_init_dynamic_array(&all_options, sizeof(my_option),
                         array_elements(my_long_options),
-                        array_elements(my_long_options)/4);
+                        array_elements(my_long_options)/4, 0);
   for (my_option *opt= my_long_options;
        opt < my_long_options + array_elements(my_long_options) - 1;
        opt++)
@@ -8364,7 +8443,7 @@ void refresh_status(THD *thd)
   add_to_status(&global_status_var, &thd->status_var);
 
   /* Reset thread's status variables */
-  bzero((uchar*) &thd->status_var, sizeof(thd->status_var));
+  thd->set_status_var_init();
   bzero((uchar*) &thd->org_status_var, sizeof(thd->org_status_var)); 
   thd->start_bytes_received= 0;