MDEV-4011 Added per thread memory counting and usage

Base code and idea from a patch from by plinux at Taobao.

The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC.
Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc().
There are plenty of new asserts to ensure that for a debug server the counting is correct.

Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS.
- The later is there mainly to show how query is progressing.

The following changes in interfaces was needed to get this to work:
- init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root()
- my_init_dynamic_array()  has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC
- Added flag for hash_init() so that one can mark hash table to be thread specific.
- Added flags to init_tree() so that one can mark tree to be thread specific.
- Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
- Added flag to Warning_info::Warning_info() if the structure should be fully initialized.
- String elements can now be marked as thread specific.
- Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
- Changed type of myf from int to ulong, as this is always a set of bit flags.

Other things:
- Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
- We now also show EXAMINED_ROWS in SHOW PROCESSLIST
- Added new variable 'memory_used'
- Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
- Removed calls to the obsoleted function init_dynamic_array()
- Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...)


client/completion_hash.cc:
  Updated call to init_alloc_root()
client/mysql.cc:
  Updated call to init_alloc_root()
client/mysqlbinlog.cc:
  init_dynamic_array() -> my_init_dynamic_array()
  Updated call to init_alloc_root()
client/mysqlcheck.c:
  Updated call to my_init_dynamic_array()
client/mysqldump.c:
  Updated call to init_alloc_root()
client/mysqltest.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Fixed compiler warnings
extra/comp_err.c:
  Updated call to my_init_dynamic_array()
extra/resolve_stack_dump.c:
  Updated call to my_init_dynamic_array()
include/hash.h:
  Added HASH_THREAD_SPECIFIC
include/heap.h:
  Added flag is internal temporary table.
include/my_dir.h:
  Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags
include/my_global.h:
  Changed type of myf from int to ulong, as this is always a set of bit flags.
include/my_sys.h:
  Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE
  Added malloc_flags to DYNAMIC_ARRAY
  Added extra mysys flag argument to my_init_dynamic_array()
  Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci
  Updated paramaters for init_alloc_root()
include/my_tree.h:
  Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables.
  Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG
  Updated parameters to init_tree()
include/myisamchk.h:
  Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks.
include/mysql.h:
  Added MYSQL_THREAD_SPECIFIC_MALLOC
  Used 'unused1' to mark memory as thread specific.
include/mysql.h.pp:
  Updated file
include/mysql_com.h:
  Used 'unused1' to mark memory as thread specific.
  Updated parameters for my_net_init()
libmysql/libmysql.c:
  Updated call to init_alloc_root() to mark memory thread specific.
libmysqld/emb_qcache.cc:
  Updated call to init_alloc_root()
libmysqld/lib_sql.cc:
  Updated call to init_alloc_root()
mysql-test/r/create.result:
  Updated results
mysql-test/r/user_var.result:
  Updated results
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/datadict/processlist_val.inc:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Update to handle new format of SHOW PROCESSLIST
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/t/show_explain.test:
  Fixed usage of debug variable so that one can run test with --debug
mysql-test/t/user_var.test:
  Added test of memory_usage variable.
mysys/array.c:
  Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC
  All allocated memory is marked with the given my_flags.
  Removed obsolete function init_dynamic_array()
mysys/default.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
mysys/hash.c:
  Updated call to my_init_dynamic_array_ci().
  Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used.
mysys/ma_dyncol.c:
  init_dynamic_array() -> my_init_dynamic_array()
  Added #if to get rid of compiler warnings
mysys/mf_tempdir.c:
  Updated call to my_init_dynamic_array()
mysys/my_alloc.c:
  Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC
  Extend MEM_ROOT with a flag if memory is thread specific.
  This is stored in block_size, to keep the size of the MEM_ROOT object identical as before.
  Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root()
mysys/my_chmod.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_chsize.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_copy.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_create.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_delete.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_error.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fopen.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_fstream.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_getwd.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_lib.c:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Updated DBUG_PRINT because of change of myf type
mysys/my_lock.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_malloc.c:
  Store at start of each allocated memory block the size of the block and if the block is thread specific.
  Call malloc_size_cb_func, if set, with the memory allocated/freed.
  Updated DBUG_PRINT because of change of myf type
mysys/my_open.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_pread.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_read.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_redel.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_rename.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_seek.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_sync.c:
  Updated DBUG_PRINT because of change of myf type
mysys/my_thr_init.c:
  Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized.
mysys/my_write.c:
  Updated DBUG_PRINT because of change of myf type
mysys/mysys_priv.h:
  Updated parameters to sf_malloc and sf_realloc()
mysys/safemalloc.c:
  Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory.
  Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory.
  Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC.
  Added sf_report_leaked_memory() to get list of memory not freed by a thread.
mysys/tree.c:
  Added flags to init_tree() so that one can mark tree to be thread specific.
  Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG.
  Updated call to init_alloc_root()
  All allocated memory is marked with the given malloc flags
mysys/waiting_threads.c:
  Updated call to my_init_dynamic_array()
sql-common/client.c:
  Updated call to init_alloc_root() and my_net_init() to mark memory thread specific.
  Updated call to my_init_dynamic_array().
  Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC.
sql-common/client_plugin.c:
  Updated call to init_alloc_root()
sql/debug_sync.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/event_scheduler.cc:
  Removed calls to net_end() as this is now done in ~THD()
  Call set_current_thd() to ensure that memory is assigned to right thread.
sql/events.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/filesort.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/filesort_utils.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/ha_ndbcluster.cc:
  Updated call to init_alloc_root()
  Updated call to my_net_init()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_ndbcluster_binlog.cc:
  Updated call to my_net_init()
  Updated call to init_sql_alloc()
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
sql/ha_partition.cc:
  Updated call to init_alloc_root()
sql/handler.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added missing call to my_dir_end()
sql/item_func.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_subselect.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/item_sum.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/log.cc:
  More DBUG
  Updated call to init_alloc_root()
sql/mdl.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/mysqld.cc:
  Added total_memory_used
  Updated call to init_alloc_root()
  Move mysql_cond_broadcast() before my_thread_end()
  Added mariadb_dbug_id() to count memory per THD instead of per thread.
  Added my_malloc_size_cb_func() callback function for my_malloc() to count memory.
  Move initialization of mysqld_server_started and mysqld_server_initialized earlier.
  Updated call to my_init_dynamic_array().
  Updated call to my_net_init().
  Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread.
  Added status variable 'memory_used'.
  Updated call to init_alloc_root()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/mysqld.h:
  Added set_current_thd()
sql/net_serv.cc:
  Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC.
  Store in net->thread_specific_malloc if memory is thread specific.
  Mark memory to be thread specific if requested.
sql/opt_range.cc:
  Updated call to my_init_dynamic_array()
  Updated call to init_sql_alloc()
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/opt_subselect.cc:
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/protocol.cc:
  Fixed compiler warning
sql/records.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/rpl_filter.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_handler.cc:
  Updated call to my_init_dynamic_array2()
sql/rpl_handler.h:
  Updated call to init_sql_alloc()
sql/rpl_mi.cc:
  Updated call to my_init_dynamic_array()
sql/rpl_tblmap.cc:
  Updated call to init_alloc_root()
sql/rpl_utility.cc:
  Updated call to my_init_dynamic_array()
sql/slave.cc:
  Initialize things properly before calling functions that allocate memory.
  Removed calls to net_end() as this is now done in ~THD()
sql/sp_head.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  Added parameter to warning_info() that it should be fully initialized.
sql/sp_pcontext.cc:
  Updated call to my_init_dynamic_array()
sql/sql_acl.cc:
  Updated call to init_sql_alloc()
  Updated call to my_init_dynamic_array()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_admin.cc:
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_analyse.h:
  Updated call to init_tree() to mark memory thread specific.
sql/sql_array.h:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_audit.cc:
  Updated call to my_init_dynamic_array()
sql/sql_base.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_cache.cc:
  Updated comment
sql/sql_class.cc:
  Added parameter to warning_info() that not initialize it until THD is fully created.
  Updated call to init_sql_alloc()
  Mark THD::user_vars has to be thread specific.
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by THD is assigned to the THD.
  More DBUG
  Always acll net_end() in ~THD()
  Assert that all memory signed to this THD is really deleted at ~THD.
  Fixed set_status_var_init() to not reset memory_used.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_class.h:
  Added MY_THREAD_SPECIFIC to allocated memory.
  Added malloc_size to THD to record allocated memory per THD.
sql/sql_delete.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_error.cc:
  Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures.
  This is used by THD::THD() to not allocate memory until THD is ready.
  Added Warning_info::free_memory()
sql/sql_error.h:
  Updated Warning_info() class.
sql/sql_handler.cc:
  Updated call to init_alloc_root() to mark memory thread specific.
sql/sql_insert.cc:
  More DBUG
sql/sql_join_cache.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_lex.cc:
  Updated call to my_init_dynamic_array()
sql/sql_lex.h:
  Updated call to my_init_dynamic_array()
sql/sql_load.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/sql_parse.cc:
  Removed calls to net_end() and thd->cleanup() as these are now done in ~THD()
  Ensure that examined_row_count() is reset before query.
  Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Don't restore thd->status_var.memory_used when restoring thd->status_var
sql/sql_plugin.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_prepare.cc:
  Added parameter to warning_info() that it should be fully initialized.
  Updated call to init_sql_alloc() to mark memory thread specific.
sql/sql_reload.cc:
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_select.cc:
  Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific.
  Added MY_THREAD_SPECIFIC to allocated memory.
  More DBUG
sql/sql_servers.cc:
  Updated call to init_sql_alloc() to mark memory some memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_show.cc:
  Updated call to my_init_dynamic_array()
  Mark my_dir() memory thread specific.
  Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread.
  More DBUG.
  Added malloc_size and examined_row_count to SHOW PROCESSLIST.
  Added MY_THREAD_SPECIFIC to allocated memory.
  Updated call to init_sql_alloc()
  Added parameter to warning_info() that it should be fully initialized.
sql/sql_statistics.cc:
  Fixed compiler warning
sql/sql_string.cc:
  String elements can now be marked as thread specific.
sql/sql_string.h:
  String elements can now be marked as thread specific.
sql/sql_table.cc:
  Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
  Fixed compiler warning
sql/sql_test.cc:
  Updated call to my_init_dynamic_array() to mark memory thread specific.
sql/sql_trigger.cc:
  Updated call to init_sql_alloc()
sql/sql_udf.cc:
  Updated call to init_sql_alloc()
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/sql_update.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
sql/table.cc:
  Updated call to init_sql_alloc().
  Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC
  Updated call to init_sql_alloc()
sql/thr_malloc.cc:
  Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC.
sql/thr_malloc.h:
  Updated prototype for init_sql_alloc()
sql/tztime.cc:
  Updated call to init_sql_alloc()
  Updated call to init_alloc_root() to mark memory thread specific.
  my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd()
sql/uniques.cc:
  Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific.
sql/unireg.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/csv/ha_tina.cc:
  Updated call to init_alloc_root()
storage/federated/ha_federated.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
  Ensure that memory allocated by fedarated is registered for the system, not for the thread.
storage/federatedx/federatedx_io_mysql.cc:
  Updated call to my_init_dynamic_array()
storage/federatedx/ha_federatedx.cc:
  Updated call to init_alloc_root()
  Updated call to my_init_dynamic_array()
storage/heap/ha_heap.cc:
  Added MY_THREAD_SPECIFIC to allocated memory.
storage/heap/heapdef.h:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_block.c:
  Added parameter to hp_get_new_block() to be able to do thread specific memory tagging.
storage/heap/hp_create.c:
  - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
  - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'.
storage/heap/hp_open.c:
  Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC.
storage/heap/hp_write.c:
  Added new parameter to hp_get_new_block()
storage/maria/ma_bitmap.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_blockrec.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_check.c:
  Updated call to init_alloc_root()
storage/maria/ma_ft_boolean_search.c:
  Updated calls to init_tree() and init_alloc_root()
storage/maria/ma_ft_nlq_search.c:
  Updated call to init_tree()
storage/maria/ma_ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/maria/ma_loghandler.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_open.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_sort.c:
  Updated call to my_init_dynamic_array()
storage/maria/ma_write.c:
  Updated calls to my_init_dynamic_array() and init_tree()
storage/maria/maria_pack.c:
  Updated call to init_tree()
storage/maria/unittest/sequence_storage.c:
  Updated call to my_init_dynamic_array()
storage/myisam/ft_boolean_search.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_nlq_search.c:
  Updated call to init_tree()
storage/myisam/ft_parser.c:
  Updated call to init_tree()
  Updated call to init_alloc_root()
storage/myisam/ft_stopwords.c:
  Updated call to init_tree()
storage/myisam/mi_check.c:
  Updated call to init_alloc_root()
storage/myisam/mi_write.c:
  Updated call to my_init_dynamic_array()
  Updated call to init_tree()
storage/myisam/myisamlog.c:
  Updated call to init_tree()
storage/myisam/myisampack.c:
  Updated call to init_tree()
storage/myisam/sort.c:
  Updated call to my_init_dynamic_array()
storage/myisammrg/ha_myisammrg.cc:
  Updated call to init_sql_alloc()
storage/perfschema/pfs_check.cc:
  Rest current_thd
storage/perfschema/pfs_instr.cc:
  Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems.
support-files/compiler_warnings.supp:
  Disable compiler warning from offsetof macro.

10 files changed, 41 insertions, 23 deletions

diff --git a/include/hash.h b/include/hash.h
index 98bcf911db2..2ecfe0891d7 100644
--- a/include/hash.h
+++ b/include/hash.h
@@ -39,6 +39,7 @@ extern "C" {
 
 /* flags for hash_init */
 #define HASH_UNIQUE     1       /* hash_insert fails on duplicate key */
+#define HASH_THREAD_SPECIFIC 2  /* Mark allocated memory THREAD_SPECIFIC */
 
 typedef uint my_hash_value_type;
 typedef uchar *(*my_hash_get_key)(const uchar *,size_t*,my_bool);
diff --git a/include/heap.h b/include/heap.h
index 10b165d89c6..149f9ed5f56 100644
--- a/include/heap.h
+++ b/include/heap.h
@@ -153,6 +153,7 @@ typedef struct st_heap_share
   THR_LOCK lock;
   mysql_mutex_t intern_lock;            /* Locking for use with _locking */
   my_bool delete_on_close;
+  my_bool internal;                     /* Internal temporary table */
   LIST open_list;
   uint auto_key;
   uint auto_key_type;			/* real type of the auto key segment */
diff --git a/include/my_dir.h b/include/my_dir.h
index 1ee002cc380..6a941b37d79 100644
--- a/include/my_dir.h
+++ b/include/my_dir.h
@@ -45,8 +45,9 @@ extern "C" {
 #define MY_S_ISREG(m)	(((m) & MY_S_IFMT) == MY_S_IFREG)
 #define MY_S_ISFIFO(m)	(((m) & MY_S_IFMT) == MY_S_IFIFO)
 
-#define MY_DONT_SORT	512	/* my_lib; Don't sort files */
-#define MY_WANT_STAT	1024	/* my_lib; stat files */
+/* Ensure these dosn't clash with anything in my_sys.h */
+#define MY_DONT_SORT	8192	/* my_lib; Don't sort files */
+#define MY_WANT_STAT	16384	/* my_lib; stat files */
 
 	/* typedefs for my_dir & my_stat */
 
diff --git a/include/my_global.h b/include/my_global.h
index 09a3a0a6efd..a99823b94d8 100644
--- a/include/my_global.h
+++ b/include/my_global.h
@@ -1003,7 +1003,7 @@ typedef struct st_mysql_lex_string LEX_STRING;
 #define SOCKET_EMFILE	EMFILE
 #endif
 
-typedef int		myf;	/* Type of MyFlags in my_funcs */
+typedef ulong		myf;	/* Type of MyFlags in my_funcs */
 typedef char		my_bool; /* Small bool */
 
 /* Macros for converting *constants* to the right type */
diff --git a/include/my_sys.h b/include/my_sys.h
index 2b0ae41a9b0..8f19293deb0 100644
--- a/include/my_sys.h
+++ b/include/my_sys.h
@@ -86,6 +86,10 @@ typedef struct my_aio_result {
 #define MY_SYNC       4096      /* my_copy(): sync dst file */
 #define MY_SYNC_DIR   32768     /* my_create/delete/rename: sync directory */
 #define MY_SYNC_FILESIZE 65536  /* my_sync(): safe sync when file is extended */
+#define MY_THREAD_SPECIFIC 0x10000  /* my_malloc(): thread specific */
+#define MY_THREAD_MOVE     0x20000  /* realloc(); Memory can move */
+/* Tree that should delete things automaticly */ 
+#define MY_TREE_WITH_DELETE 0x40000
 
 #define MY_CHECK_ERROR	1	/* Params to my_end; Check open-close */
 #define MY_GIVE_INFO	2	/* Give time info about process*/
@@ -148,6 +152,18 @@ typedef struct my_aio_result {
 /* Extra length needed for filename if one calls my_create_backup_name */
 #define MY_BACKUP_NAME_EXTRA_LENGTH 17
 
+/* If we have our own safemalloc (for debugging) */
+#if defined(SAFEMALLOC)
+void sf_report_leaked_memory(my_thread_id id);
+extern my_thread_id (*sf_malloc_dbug_id)(void);
+#define SAFEMALLOC_REPORT_MEMORY(X) sf_report_leaked_memory(X)
+#else
+#define SAFEMALLOC_REPORT_MEMORY(X) do {} while(0)
+#endif
+
+typedef void (*MALLOC_SIZE_CB) (long long size, myf my_flags); 
+extern void set_malloc_size_cb(MALLOC_SIZE_CB func);
+
 	/* defines when allocating data */
 extern void *my_malloc(size_t Size,myf MyFlags);
 extern void *my_multi_malloc(myf MyFlags, ...);
@@ -323,6 +339,7 @@ typedef struct st_dynamic_array
   uint elements,max_element;
   uint alloc_increment;
   uint size_of_element;
+  myf malloc_flags;
 } DYNAMIC_ARRAY;
 
 typedef struct st_my_tmpdir
@@ -768,16 +785,11 @@ extern my_bool real_open_cached_file(IO_CACHE *cache);
 extern void close_cached_file(IO_CACHE *cache);
 File create_temp_file(char *to, const char *dir, const char *pfx,
 		      int mode, myf MyFlags);
-#define my_init_dynamic_array(A,B,C,D) init_dynamic_array2(A,B,NULL,C,D)
-#define my_init_dynamic_array_ci(A,B,C,D) init_dynamic_array2(A,B,NULL,C,D)
-#define my_init_dynamic_array2(A,B,C,D,E) init_dynamic_array2(A,B,C,D,E)
-#define my_init_dynamic_array2_ci(A,B,C,D,E) init_dynamic_array2(A,B,C,D,E)
+#define my_init_dynamic_array(A,B,C,D,E) init_dynamic_array2(A,B,NULL,C,D,E)
+#define my_init_dynamic_array2(A,B,C,D,E,F) init_dynamic_array2(A,B,C,D,E,F)
 extern my_bool init_dynamic_array2(DYNAMIC_ARRAY *array, uint element_size,
                                    void *init_buffer, uint init_alloc,
-                                   uint alloc_increment);
-/* init_dynamic_array() function is deprecated */
-extern my_bool init_dynamic_array(DYNAMIC_ARRAY *array, uint element_size,
-                                  uint init_alloc, uint alloc_increment);
+                                   uint alloc_increment, myf my_flags);
 extern my_bool insert_dynamic(DYNAMIC_ARRAY *array, const uchar * element);
 extern uchar *alloc_dynamic(DYNAMIC_ARRAY *array);
 extern uchar *pop_dynamic(DYNAMIC_ARRAY*);
@@ -829,7 +841,7 @@ extern void my_free_lock(void *ptr);
 #define ALLOC_ROOT_MIN_BLOCK_SIZE (MALLOC_OVERHEAD + sizeof(USED_MEM) + 8)
 #define clear_alloc_root(A) do { (A)->free= (A)->used= (A)->pre_alloc= 0; (A)->min_malloc=0;} while(0)
 extern void init_alloc_root(MEM_ROOT *mem_root, size_t block_size,
-			    size_t pre_alloc_size);
+			    size_t pre_alloc_size, myf my_flags);
 extern void *alloc_root(MEM_ROOT *mem_root, size_t Size);
 extern void *multi_alloc_root(MEM_ROOT *mem_root, ...);
 extern void free_root(MEM_ROOT *root, myf MyFLAGS);
diff --git a/include/my_tree.h b/include/my_tree.h
index 3dd92712af2..f8be55f84b2 100644
--- a/include/my_tree.h
+++ b/include/my_tree.h
@@ -68,13 +68,15 @@ typedef struct st_tree {
   MEM_ROOT mem_root;
   my_bool with_delete;
   tree_element_free free;
+  myf my_flags;
   uint flag;
 } TREE;
 
 	/* Functions on whole tree */
 void init_tree(TREE *tree, size_t default_alloc_size, size_t memory_limit,
-               int size, qsort_cmp2 compare, my_bool with_delete,
-	       tree_element_free free_element, void *custom_arg);
+               int size, qsort_cmp2 compare,
+	       tree_element_free free_element, void *custom_arg,
+               myf my_flags);
 void delete_tree(TREE*);
 void reset_tree(TREE*);
 
diff --git a/include/myisamchk.h b/include/myisamchk.h
index 789e95572b3..7e7b685a88a 100644
--- a/include/myisamchk.h
+++ b/include/myisamchk.h
@@ -164,6 +164,7 @@ typedef struct st_handler_check_param
 
   mysql_mutex_t print_msg_mutex;
   my_bool need_print_msg_lock;
+  myf malloc_flags;
 } HA_CHECK;
 
 
diff --git a/include/mysql.h b/include/mysql.h
index fa62026b44a..1268280395e 100644
--- a/include/mysql.h
+++ b/include/mysql.h
@@ -167,7 +167,7 @@ enum mysql_option
   MYSQL_OPT_GUESS_CONNECTION, MYSQL_SET_CLIENT_IP, MYSQL_SECURE_AUTH,
   MYSQL_REPORT_DATA_TRUNCATION, MYSQL_OPT_RECONNECT,
   MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH,
-  MYSQL_PROGRESS_CALLBACK,
+  MYSQL_PROGRESS_CALLBACK, MYSQL_THREAD_SPECIFIC_MALLOC,
   /* MariaDB options */
   MYSQL_OPT_NONBLOCK=6000
 };
@@ -194,7 +194,7 @@ struct st_mysql_options {
   unsigned long max_allowed_packet;
   my_bool use_ssl;				/* if to use SSL or not */
   my_bool compress,named_pipe;
-  my_bool unused1;
+  my_bool thread_specific_malloc;
   my_bool unused2;
   my_bool unused3;
   my_bool unused4;
diff --git a/include/mysql.h.pp b/include/mysql.h.pp
index 48ce79046ff..84510a69edb 100644
--- a/include/mysql.h.pp
+++ b/include/mysql.h.pp
@@ -28,7 +28,7 @@ typedef struct st_net {
   unsigned char reading_or_writing;
   char save_char;
   char net_skip_rest_factor;
-  my_bool unused1;
+  my_bool thread_specific_malloc;
   my_bool compress;
   my_bool unused3;
   unsigned char *unused;
@@ -80,7 +80,7 @@ enum enum_mysql_set_option
   MYSQL_OPTION_MULTI_STATEMENTS_ON,
   MYSQL_OPTION_MULTI_STATEMENTS_OFF
 };
-my_bool my_net_init(NET *net, Vio* vio);
+my_bool my_net_init(NET *net, Vio* vio, unsigned int my_flags);
 void my_net_local_init(NET *net);
 void net_end(NET *net);
   void net_clear(NET *net, my_bool clear_buffer);
@@ -262,7 +262,7 @@ enum mysql_option
   MYSQL_OPT_GUESS_CONNECTION, MYSQL_SET_CLIENT_IP, MYSQL_SECURE_AUTH,
   MYSQL_REPORT_DATA_TRUNCATION, MYSQL_OPT_RECONNECT,
   MYSQL_OPT_SSL_VERIFY_SERVER_CERT, MYSQL_PLUGIN_DIR, MYSQL_DEFAULT_AUTH,
-  MYSQL_PROGRESS_CALLBACK,
+  MYSQL_PROGRESS_CALLBACK, MYSQL_THREAD_SPECIFIC_MALLOC,
   MYSQL_OPT_NONBLOCK=6000
 };
 struct st_mysql_options_extention;
@@ -282,7 +282,7 @@ struct st_mysql_options {
   unsigned long max_allowed_packet;
   my_bool use_ssl;
   my_bool compress,named_pipe;
-  my_bool unused1;
+  my_bool thread_specific_malloc;
   my_bool unused2;
   my_bool unused3;
   my_bool unused4;
diff --git a/include/mysql_com.h b/include/mysql_com.h
index 46a55e42e50..dbcc7125302 100644
--- a/include/mysql_com.h
+++ b/include/mysql_com.h
@@ -330,7 +330,7 @@ typedef struct st_net {
   unsigned char reading_or_writing;
   char save_char;
   char net_skip_rest_factor;
-  my_bool unused1; /* Please remove with the next incompatible ABI change */
+  my_bool thread_specific_malloc;
   my_bool compress;
   my_bool unused3; /* Please remove with the next incompatible ABI change. */
   /*
@@ -465,10 +465,10 @@ enum enum_mysql_set_option
 extern "C" {
 #endif
 
-my_bool	my_net_init(NET *net, Vio* vio);
+my_bool	my_net_init(NET *net, Vio* vio, unsigned int my_flags);
 void	my_net_local_init(NET *net);
 void	net_end(NET *net);
-  void	net_clear(NET *net, my_bool clear_buffer);
+void	net_clear(NET *net, my_bool clear_buffer);
 my_bool net_realloc(NET *net, size_t length);
 my_bool	net_flush(NET *net);
 my_bool	my_net_write(NET *net,const unsigned char *packet, size_t len);