summaryrefslogtreecommitdiff
path: root/sql/sql_statistics.cc
Commit message (Collapse)AuthorAgeFilesLines
* Removed TABLE->sort to make it possible to have multiple active calls toMonty2016-03-221-0/+1
| | | | | | | | | | | | | | | | | | filesort and init_read_record() for the same table. This will simplify code for WINDOW FUNCTIONS (MDEV-6115) - Filesort_info renamed to SORT_INFO and moved to filesort.h - filesort now returns SORT_INFO - init_read_record() now takes a SORT_INFO parameter. - unique declaration is moved to uniques.h - subselect caching of buffers is now more explicit than before - filesort_buffer is now reusable even if rec_length has changed. - filsort_free_buffers() and free_io_cache() calls are removed - Remove one malloc() when using get_addon_fields() Other things: - Added --debug-assert-on-not-freed-memory option to make it easier to debug some not-freed-memory issues.
* MDEV-8715 - Obsolete sql_alloc() in favor of THD::alloc() and thd_alloc()Sergey Vojtovich2015-11-261-1/+1
| | | | | | | | | | | | | | The following left in semi-improved state to keep patch size reasonable: - Field operator new: left thd_alloc(current_thd) - Sql_alloc operator new: left thd_alloc(thd_get_current_thd()) - Item_args constructors: left thd_alloc(thd) - Item_func_interval::fix_length_and_dec(): no THD arg, have to call current_thd - Item_func_dyncol_exists::val_int(): same - Item_dyncol_get::val_str(): same - Item_dyncol_get::val_int(): same - Item_dyncol_get::val_real(): same - Item_dyncol_get::val_decimal(): same - Item_singlerow_subselect::fix_length_and_dec(): same
* MDEV-8716 - Obsolete sql_calloc() in favor of THD::calloc() and thd_calloc()Sergey Vojtovich2015-11-261-3/+4
|
* Merge branch '10.0' into 10.1Sergei Golubchik2015-11-191-10/+132
|\
| * MDEV-7370: Server deadlocks on renaming a table for which persistent ↵Sergei Petrunia2015-11-181-2/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | statistics exists RENAME TABLE code tries to update EITS statistics. It hung, because it used an index on (db_name,table_name) to find the table, and attempted to update these values at the same time. The fix is do what SQL UPDATE statement does when updating index that it's used for scanning: - First, buffer the rowids of rows to be updated, - then make the second pass to actually update the rows Also fixed the call to rename_table_in_stat_tables() in sql_rename.cc to pass the correct new database (before, it passed old db_name so cross- database renames were not handled correctly). Variant #2, with review feedback addressed.
| * MDEV#7383: engine-independent-stats column_stats has limited values for ↵Sergei Petrunia2015-11-091-8/+8
| | | | | | | | | | | | | | | | | | | | max/min values Patch from Daniel Black: - Change the charset of mysql.column_stats.{min_value, max_value} from utf8_bin varchar to varbinary - Adjust the code that saves/reads the data accordingly. - Also provide upgrade statement in mysql_system_tables_fix.sql
* | MDEV-6650 - LINT_INIT emits code in non-debug buildsSergey Vojtovich2015-03-161-1/+1
|/ | | | | Replaced all references to LINT_INIT with UNINIT_VAR and LINT_INIT_STRUCT. Removed LINT_INIT macro.
* MDEV-7362: ANALYZE TABLES crash with table-independent-statistics gatheringVicențiu Ciorbaru2015-01-171-2/+8
| | | | | | | | | | | FULLTEXT indexes do not permit index first lookups. By calling: ha_index_first() with a garbage parameter, random data gets overwritten that causes the table->field array to be corrupted. Subsequently, when the field array is accessed, a segfault occurs. By not allowing index statistics for FULLTEXT indexes, the problem is resolved.
* MDEV-7087 main.stat_tables-enospc fails in buildbot on a valgrind buildSergei Golubchik2014-11-181-0/+2
| | | | | | when reading data into the record buffer, the tail of the VARCHAR (between real and max varchar length) is not written to. initialize the record buffer to avoid writing uninitialized memory to disk.
* Fixed bug mdev-6843.Igor Babaev2014-10-281-1/+6
| | | | | | The function get_column_range_cardinality() returned a wrong result for any column containing only null values.
* MDEV-6738: use_stat_table + histograms crashing optimizerSergey Petrunya2014-10-101-3/+3
| | | | | | - When EITS code calls store_key_image_to_rec(), it should follow its calling convention (which is counter-intuitive)
* MDEV-6442: Assertion `join->best_read < double(...)' failed with ↵Sergey Petrunya2014-10-061-1/+15
| | | | | | | | | | | | | optimizer_use_condition_selectivity >=3 - Fix the crash by making get_column_range_cardinality() to handle the special case where Column_stats objects is an all-zeros object (the question of what is the point of having Field::read_stats point to such object remains a mystery) - Added a few comments. Learning the code still.
* MDEV-5120 Test suite test maria-no-logging failsMichael Widenius2014-09-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reason for the failure was a bug in an include file on debian that causes 'struct stat' to have different sized depending on the environment. This patch fixes so that we always include my_global.h or my_config.h before we include any other files. Other things: - Removed #include <my_global.h> in some include files; Better to always do this at the top level to have as few "always-include-this-file-first' files as possible. - Removed usage of some include files that where already included by my_global.h or by other files. client/mysql_plugin.c: Use my_global.h first client/mysqlslap.c: Remove duplicated include files extra/comp_err.c: Remove duplicated include files include/m_string.h: Remove duplicated include files include/maria.h: Remove duplicated include files libmysqld/emb_qcache.cc: Use my_global.h first plugin/semisync/semisync.h: Use my_pthread.h first sql/datadict.cc: Use my_global.h first sql/debug_sync.cc: Use my_global.h first sql/derror.cc: Use my_global.h first sql/des_key_file.cc: Use my_global.h first sql/discover.cc: Use my_global.h first sql/event_data_objects.cc: Use my_global.h first sql/event_db_repository.cc: Use my_global.h first sql/event_parse_data.cc: Use my_global.h first sql/event_queue.cc: Use my_global.h first sql/event_scheduler.cc: Use my_global.h first sql/events.cc: Use my_global.h first sql/field.cc: Use my_global.h first Remove duplicated include files sql/field_conv.cc: Use my_global.h first sql/filesort.cc: Use my_global.h first Remove duplicated include files sql/gstream.cc: Use my_global.h first sql/ha_ndbcluster.cc: Use my_global.h first sql/ha_ndbcluster_binlog.cc: Use my_global.h first sql/ha_ndbcluster_cond.cc: Use my_global.h first sql/ha_partition.cc: Use my_global.h first sql/handler.cc: Use my_global.h first sql/hash_filo.cc: Use my_global.h first sql/hostname.cc: Use my_global.h first sql/init.cc: Use my_global.h first sql/item.cc: Use my_global.h first sql/item_buff.cc: Use my_global.h first sql/item_cmpfunc.cc: Use my_global.h first sql/item_create.cc: Use my_global.h first sql/item_geofunc.cc: Use my_global.h first sql/item_inetfunc.cc: Use my_global.h first sql/item_row.cc: Use my_global.h first sql/item_strfunc.cc: Use my_global.h first sql/item_subselect.cc: Use my_global.h first sql/item_sum.cc: Use my_global.h first sql/item_timefunc.cc: Use my_global.h first sql/item_xmlfunc.cc: Use my_global.h first sql/key.cc: Use my_global.h first sql/lock.cc: Use my_global.h first sql/log.cc: Use my_global.h first sql/log_event.cc: Use my_global.h first sql/log_event_old.cc: Use my_global.h first sql/mf_iocache.cc: Use my_global.h first sql/mysql_install_db.cc: Remove duplicated include files sql/mysqld.cc: Remove duplicated include files sql/net_serv.cc: Remove duplicated include files sql/opt_range.cc: Use my_global.h first sql/opt_subselect.cc: Use my_global.h first sql/opt_sum.cc: Use my_global.h first sql/parse_file.cc: Use my_global.h first sql/partition_info.cc: Use my_global.h first sql/procedure.cc: Use my_global.h first sql/protocol.cc: Use my_global.h first sql/records.cc: Use my_global.h first sql/records.h: Don't include my_global.h Better to do this at the upper level sql/repl_failsafe.cc: Use my_global.h first sql/rpl_filter.cc: Use my_global.h first sql/rpl_gtid.cc: Use my_global.h first sql/rpl_handler.cc: Use my_global.h first sql/rpl_injector.cc: Use my_global.h first sql/rpl_record.cc: Use my_global.h first sql/rpl_record_old.cc: Use my_global.h first sql/rpl_reporting.cc: Use my_global.h first sql/rpl_rli.cc: Use my_global.h first sql/rpl_tblmap.cc: Use my_global.h first sql/rpl_utility.cc: Use my_global.h first sql/set_var.cc: Added comment sql/slave.cc: Use my_global.h first sql/sp.cc: Use my_global.h first sql/sp_cache.cc: Use my_global.h first sql/sp_head.cc: Use my_global.h first sql/sp_pcontext.cc: Use my_global.h first sql/sp_rcontext.cc: Use my_global.h first sql/spatial.cc: Use my_global.h first sql/sql_acl.cc: Use my_global.h first sql/sql_admin.cc: Use my_global.h first sql/sql_analyse.cc: Use my_global.h first sql/sql_audit.cc: Use my_global.h first sql/sql_base.cc: Use my_global.h first sql/sql_binlog.cc: Use my_global.h first sql/sql_bootstrap.cc: Use my_global.h first Use my_global.h first sql/sql_cache.cc: Use my_global.h first sql/sql_class.cc: Use my_global.h first sql/sql_client.cc: Use my_global.h first sql/sql_connect.cc: Use my_global.h first sql/sql_crypt.cc: Use my_global.h first sql/sql_cursor.cc: Use my_global.h first sql/sql_db.cc: Use my_global.h first sql/sql_delete.cc: Use my_global.h first sql/sql_derived.cc: Use my_global.h first sql/sql_do.cc: Use my_global.h first sql/sql_error.cc: Use my_global.h first sql/sql_explain.cc: Use my_global.h first sql/sql_expression_cache.cc: Use my_global.h first sql/sql_handler.cc: Use my_global.h first sql/sql_help.cc: Use my_global.h first sql/sql_insert.cc: Use my_global.h first sql/sql_lex.cc: Use my_global.h first sql/sql_load.cc: Use my_global.h first sql/sql_locale.cc: Use my_global.h first sql/sql_manager.cc: Use my_global.h first sql/sql_parse.cc: Use my_global.h first sql/sql_partition.cc: Use my_global.h first sql/sql_plugin.cc: Added comment sql/sql_prepare.cc: Use my_global.h first sql/sql_priv.h: Added error if we use this before including my_global.h This check is here becasue so many files includes sql_priv.h first. sql/sql_profile.cc: Use my_global.h first sql/sql_reload.cc: Use my_global.h first sql/sql_rename.cc: Use my_global.h first sql/sql_repl.cc: Use my_global.h first sql/sql_select.cc: Use my_global.h first sql/sql_servers.cc: Use my_global.h first sql/sql_show.cc: Added comment sql/sql_signal.cc: Use my_global.h first sql/sql_statistics.cc: Use my_global.h first sql/sql_table.cc: Use my_global.h first sql/sql_tablespace.cc: Use my_global.h first sql/sql_test.cc: Use my_global.h first sql/sql_time.cc: Use my_global.h first sql/sql_trigger.cc: Use my_global.h first sql/sql_udf.cc: Use my_global.h first sql/sql_union.cc: Use my_global.h first sql/sql_update.cc: Use my_global.h first sql/sql_view.cc: Use my_global.h first sql/sys_vars.cc: Added comment sql/table.cc: Use my_global.h first sql/thr_malloc.cc: Use my_global.h first sql/transaction.cc: Use my_global.h first sql/uniques.cc: Use my_global.h first sql/unireg.cc: Use my_global.h first sql/unireg.h: Removed inclusion of my_global.h storage/archive/ha_archive.cc: Added comment storage/blackhole/ha_blackhole.cc: Use my_global.h first storage/csv/ha_tina.cc: Use my_global.h first storage/csv/transparent_file.cc: Use my_global.h first storage/federated/ha_federated.cc: Use my_global.h first storage/federatedx/federatedx_io.cc: Use my_global.h first storage/federatedx/federatedx_io_mysql.cc: Use my_global.h first storage/federatedx/federatedx_io_null.cc: Use my_global.h first storage/federatedx/federatedx_txn.cc: Use my_global.h first storage/heap/ha_heap.cc: Use my_global.h first storage/innobase/handler/handler0alter.cc: Use my_global.h first storage/maria/ha_maria.cc: Use my_global.h first storage/maria/unittest/ma_maria_log_cleanup.c: Remove duplicated include files storage/maria/unittest/test_file.c: Added comment storage/myisam/ha_myisam.cc: Move sql_plugin.h first as this includes my_global.h storage/myisammrg/ha_myisammrg.cc: Use my_global.h first storage/oqgraph/oqgraph_thunk.cc: Use my_config.h and my_global.h first One could not include my_global.h before oqgraph_thunk.h (don't know why) storage/spider/ha_spider.cc: Use my_global.h first storage/spider/hs_client/config.cpp: Use my_global.h first storage/spider/hs_client/escape.cpp: Use my_global.h first storage/spider/hs_client/fatal.cpp: Use my_global.h first storage/spider/hs_client/hstcpcli.cpp: Use my_global.h first storage/spider/hs_client/socket.cpp: Use my_global.h first storage/spider/hs_client/string_util.cpp: Use my_global.h first storage/spider/spd_conn.cc: Use my_global.h first storage/spider/spd_copy_tables.cc: Use my_global.h first storage/spider/spd_db_conn.cc: Use my_global.h first storage/spider/spd_db_handlersocket.cc: Use my_global.h first storage/spider/spd_db_mysql.cc: Use my_global.h first storage/spider/spd_db_oracle.cc: Use my_global.h first storage/spider/spd_direct_sql.cc: Use my_global.h first storage/spider/spd_i_s.cc: Use my_global.h first storage/spider/spd_malloc.cc: Use my_global.h first storage/spider/spd_param.cc: Use my_global.h first storage/spider/spd_ping_table.cc: Use my_global.h first storage/spider/spd_sys_table.cc: Use my_global.h first storage/spider/spd_table.cc: Use my_global.h first storage/spider/spd_trx.cc: Use my_global.h first storage/xtradb/handler/handler0alter.cc: Use my_global.h first storage/xtradb/handler/i_s.cc: Use my_global.h first
* MDEV-6181 EITS could eat all tmpdir space and hangSergei Golubchik2014-08-041-5/+10
| | | | | | Don't ignore errors from Count_distinct_field::add(), pass them to the caller, so that it could abort the data collection loop.
* MDEV-6314 - Compile/run MariaDB with ASanSergey Vojtovich2014-06-101-0/+1
| | | | Fixed some compilation errors/warnings with ASan.
* Code cleanup:Sergey Petrunya2014-03-271-0/+117
| | | | | - Move [some] engine-agnostic tests from t/selectivity.test to t/selectivity_no_engine.test - Move Histogram::point_selectivity to sql_statistics.cc
* MDEV-5917: EITS: different order of predicates in IN (...) causes different ↵Sergey Petrunya2014-03-211-0/+2
| | | | | | estimates - Save range key before making field->pos_in_interval() call (like we do for non-equality ranges)
* MDEV-5901: EITS: killing the server leaves statistical tables in "marked as ↵Sergey Petrunya2014-03-191-1/+5
| | | | | | | | crashed" state - Part#2: call HA_EXTRA_FLUSH for the correct handler object, and call it after every change (ha_write_row, ha_update_row, ha_delete_row).
* MDEV-5901: EITS: killing the server leaves statistical tables in "marked as ↵Sergey Petrunya2014-03-191-0/+2
| | | | | | | crashed" state - Do like sp.cc does with mysql.proc table: call HA_EXTRA_FLUSH after we've modified a statistical table.
* 10.0-monty mergeSergei Golubchik2013-07-211-16/+19
|\ | | | | | | | | | | | | | | includes: * remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING" * introduce LOCK_share, now LOCK_ha_data is strictly for engines * rea_create_table() always creates .par file (even in "frm-only" mode) * fix a 5.6 bug, temp file leak on dummy ALTER TABLE
| * MDEV-4758 10.0-monty tree: ALTER TABLE CHANGE COLUMN doesn't drop EITS statsSergei Golubchik2013-07-091-1/+4
| | | | | | | | | | add missing rename_column_in_stat_tables(), delete_statistics_for_column(), delete_statistics_for_index(), rename_table_in_stat_tables() calls.
| * Applied all changes from Igor and SanjaMichael Widenius2013-06-151-2/+2
| |
| * Push a lot of small fixes to get larger parts to compileMichael Widenius2013-05-211-5/+5
| |
* | Fixed errors and compiler warnings found by buildbotMichael Widenius2013-05-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Solaris fixes: - Fixed that wait_timeout_func and wait_timeout tests works on solaris - We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO). - Fixed that compile-solaris-amd64-debug works (before that we got a wrong ELF class: ELFCLASS64 on linkage) - Added missing sync_with_master Other bug fixes: - Free memory for rpl_global_gtid_binlog_state before exit() to avoid 'accessing uninitalized mutex' error. BUILD/FINISH.sh: Fixed issues on Solaris with ksh BUILD/compile-solaris-amd64-debug: Added missing -m64 flag configure.cmake: We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO) mysql-test/suite/rpl/t/rpl_gtid_mdev4473.test: - Added missing sync_with_master (fix by knielsen) sql-common/client.c: Added () to get rid of compiler warning sql/item_strfunc.cc: Fixed compiler warning sql/log.cc: Free memory for static variable rpl_global_gtid_binlog_state before exit() - If we are compiling with safemalloc, we would try to call sf_free() for some members after sf_terminate() was called, which would result of trying to access the uninitalized mutex 'sf_mutex' sql/multi_range_read.cc: Fixed compiler warnings of converting double to ulong. sql/opt_range.cc: Fixed compiler warnings of converting double to ulong or uint - Better to have all variables that can be number of rows as 'ha_rows' sql/rpl_gtid.cc: Added rpl_binlog_state::free() to be able to free memory for static objects before exit() sql/rpl_gtid.h: Added rpl_binlog_state::free() to be able to free memory for static objects before exit() sql/set_var.cc: Fixed compiler warning sql/sql_join_cache.cc: Fixed compiler warnings of converting double to uint sql/sql_show.cc: Added cast to get rid of compiler warning sql/sql_statistics.cc: Remove code that didn't do anything. (store_record() with record[0] is a no-op) storage/xtradb/os/os0file.c: Added __attribute__ ((unused)) support-files/compiler_warnings.supp: Ignore warnings from atomic_add_64_nv (was not able to fix this with a cast as the macro is a bit different between systems) vio/viosocket.c: Added more DBUG_PRINT
* | Fixed bug mdev-4406.Igor Babaev2013-04-201-1/+1
| | | | | | | | | | This bug in the code of get_column_range_cardinality() could lead to wrong estimates of number of records in ranges for non-nullable columns.
* | Added comments.Igor Babaev2013-04-151-36/+164
| | | | | | | | | | Renamed the virtual method middle_point_pos for the class Field to pos_in_interval.
* | Fixed bug mdev-4389.Igor Babaev2013-04-121-0/+1
| | | | | | | | The selectivity of a range degenerated into a point never should be set to 0.
* | Fixed bug mdev-4363.Igor Babaev2013-04-061-2/+8
| | | | | | | | | | | | When calculating the selectivity of a range in the function get_column_range_cardinality a check whether NULL values are included into into the range must be done.
* | Fixed bug mdev-4369.Igor Babaev2013-04-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | The function was adjusted to be able to aggregate the counters of the merged elements. Before this change it was not possible to guarantee the correctness of the counters passed to the the call-back parameter walk_action. As a result, when some elements of a Unique object were flushed into disk the function passed to merge_walk() as the call-back parameter could return wrong counters of elements. This could lead to building wrong histograms.
* | Fixed bugs mdev-4357 and mdev-4359.Igor Babaev2013-04-051-2/+4
| | | | | | | | | | | | The values of the column HIST_TYPE from the statistical table mysql.column_stats were stored in the table and read from the table incorrectly.
* | Fixed bug mdev-4370.Igor Babaev2013-04-051-2/+2
| | | | | | | | | | | | Don't try to a histogram if it is not read into the cache for statistical data. It may happen so if optimizer_use_condition_selectivity is set to 3. This setting orders the optimizer not use histograms to calculate selectivity.
* | Fixed a valgrind complain on usage of an uninitialized value.Igor Babaev2013-04-011-1/+1
| | | | | | | | | | It popped up because the latest fix to handle properly null ranges was not complete.
* | Take into account the number of null values in any used column whenIgor Babaev2013-03-311-14/+30
| | | | | | | | calculating selectivity of conditions.
* | MergeIgor Babaev2013-03-301-6/+16
|\ \
| * | Added the type of histogram for mwl #253.Igor Babaev2013-03-301-6/+16
| | | | | | | | | | | | Introduced double precision height-balanced histograms.
* | | Fixed several bugs for mwl #253.Igor Babaev2013-03-301-9/+35
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | One of them is quite serious: the function table_cond_selectivity used the TABLE_REF structure for ref/eq_ref access methods as if they had been filled. In fact these structure are filled after the best execution plan has been chosen. The other bugs happened due to: - an erroneous attempt at get statistics on the result of materialization of a view - incorrect handling of ranges with no left/right limits when calculating selectivity of range conditions on non-indexed columns - lack of cleanup for some newly introduced fields
* | Added histogams for table columns.Igor Babaev2013-03-251-32/+294
| |
* | The pilot patch for mwl#253.Igor Babaev2013-03-111-0/+49
|/
* 5.5 mergeSergei Golubchik2013-01-291-1/+1
|
* MDEV-4011 Added per thread memory counting and usageMichael Widenius2013-01-231-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Base code and idea from a patch from by plinux at Taobao. The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC. Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc(). There are plenty of new asserts to ensure that for a debug server the counting is correct. Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS. - The later is there mainly to show how query is progressing. The following changes in interfaces was needed to get this to work: - init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root() - my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - Added flag for hash_init() so that one can mark hash table to be thread specific. - Added flags to init_tree() so that one can mark tree to be thread specific. - Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG. - Added flag to Warning_info::Warning_info() if the structure should be fully initialized. - String elements can now be marked as thread specific. - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. - Changed type of myf from int to ulong, as this is always a set of bit flags. Other things: - Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() - We now also show EXAMINED_ROWS in SHOW PROCESSLIST - Added new variable 'memory_used' - Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory. - Removed calls to the obsoleted function init_dynamic_array() - Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...) client/completion_hash.cc: Updated call to init_alloc_root() client/mysql.cc: Updated call to init_alloc_root() client/mysqlbinlog.cc: init_dynamic_array() -> my_init_dynamic_array() Updated call to init_alloc_root() client/mysqlcheck.c: Updated call to my_init_dynamic_array() client/mysqldump.c: Updated call to init_alloc_root() client/mysqltest.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Fixed compiler warnings extra/comp_err.c: Updated call to my_init_dynamic_array() extra/resolve_stack_dump.c: Updated call to my_init_dynamic_array() include/hash.h: Added HASH_THREAD_SPECIFIC include/heap.h: Added flag is internal temporary table. include/my_dir.h: Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags include/my_global.h: Changed type of myf from int to ulong, as this is always a set of bit flags. include/my_sys.h: Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE Added malloc_flags to DYNAMIC_ARRAY Added extra mysys flag argument to my_init_dynamic_array() Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci Updated paramaters for init_alloc_root() include/my_tree.h: Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables. Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG Updated parameters to init_tree() include/myisamchk.h: Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks. include/mysql.h: Added MYSQL_THREAD_SPECIFIC_MALLOC Used 'unused1' to mark memory as thread specific. include/mysql.h.pp: Updated file include/mysql_com.h: Used 'unused1' to mark memory as thread specific. Updated parameters for my_net_init() libmysql/libmysql.c: Updated call to init_alloc_root() to mark memory thread specific. libmysqld/emb_qcache.cc: Updated call to init_alloc_root() libmysqld/lib_sql.cc: Updated call to init_alloc_root() mysql-test/r/create.result: Updated results mysql-test/r/user_var.result: Updated results mysql-test/suite/funcs_1/datadict/processlist_priv.inc: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/datadict/processlist_val.inc: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/r/is_columns_is.result: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_no_prot.result: Updated results mysql-test/t/show_explain.test: Fixed usage of debug variable so that one can run test with --debug mysql-test/t/user_var.test: Added test of memory_usage variable. mysys/array.c: Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC All allocated memory is marked with the given my_flags. Removed obsolete function init_dynamic_array() mysys/default.c: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() mysys/hash.c: Updated call to my_init_dynamic_array_ci(). Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used. mysys/ma_dyncol.c: init_dynamic_array() -> my_init_dynamic_array() Added #if to get rid of compiler warnings mysys/mf_tempdir.c: Updated call to my_init_dynamic_array() mysys/my_alloc.c: Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC Extend MEM_ROOT with a flag if memory is thread specific. This is stored in block_size, to keep the size of the MEM_ROOT object identical as before. Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root() mysys/my_chmod.c: Updated DBUG_PRINT because of change of myf type mysys/my_chsize.c: Updated DBUG_PRINT because of change of myf type mysys/my_copy.c: Updated DBUG_PRINT because of change of myf type mysys/my_create.c: Updated DBUG_PRINT because of change of myf type mysys/my_delete.c: Updated DBUG_PRINT because of change of myf type mysys/my_error.c: Updated DBUG_PRINT because of change of myf type mysys/my_fopen.c: Updated DBUG_PRINT because of change of myf type mysys/my_fstream.c: Updated DBUG_PRINT because of change of myf type mysys/my_getwd.c: Updated DBUG_PRINT because of change of myf type mysys/my_lib.c: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Updated DBUG_PRINT because of change of myf type mysys/my_lock.c: Updated DBUG_PRINT because of change of myf type mysys/my_malloc.c: Store at start of each allocated memory block the size of the block and if the block is thread specific. Call malloc_size_cb_func, if set, with the memory allocated/freed. Updated DBUG_PRINT because of change of myf type mysys/my_open.c: Updated DBUG_PRINT because of change of myf type mysys/my_pread.c: Updated DBUG_PRINT because of change of myf type mysys/my_read.c: Updated DBUG_PRINT because of change of myf type mysys/my_redel.c: Updated DBUG_PRINT because of change of myf type mysys/my_rename.c: Updated DBUG_PRINT because of change of myf type mysys/my_seek.c: Updated DBUG_PRINT because of change of myf type mysys/my_sync.c: Updated DBUG_PRINT because of change of myf type mysys/my_thr_init.c: Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized. mysys/my_write.c: Updated DBUG_PRINT because of change of myf type mysys/mysys_priv.h: Updated parameters to sf_malloc and sf_realloc() mysys/safemalloc.c: Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory. Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory. Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC. Added sf_report_leaked_memory() to get list of memory not freed by a thread. mysys/tree.c: Added flags to init_tree() so that one can mark tree to be thread specific. Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG. Updated call to init_alloc_root() All allocated memory is marked with the given malloc flags mysys/waiting_threads.c: Updated call to my_init_dynamic_array() sql-common/client.c: Updated call to init_alloc_root() and my_net_init() to mark memory thread specific. Updated call to my_init_dynamic_array(). Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC. sql-common/client_plugin.c: Updated call to init_alloc_root() sql/debug_sync.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/event_scheduler.cc: Removed calls to net_end() as this is now done in ~THD() Call set_current_thd() to ensure that memory is assigned to right thread. sql/events.cc: my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/filesort.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/filesort_utils.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/ha_ndbcluster.cc: Updated call to init_alloc_root() Updated call to my_net_init() Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() sql/ha_ndbcluster_binlog.cc: Updated call to my_net_init() Updated call to init_sql_alloc() Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() sql/ha_partition.cc: Updated call to init_alloc_root() sql/handler.cc: Added MY_THREAD_SPECIFIC to allocated memory. Added missing call to my_dir_end() sql/item_func.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/item_subselect.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/item_sum.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/log.cc: More DBUG Updated call to init_alloc_root() sql/mdl.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/mysqld.cc: Added total_memory_used Updated call to init_alloc_root() Move mysql_cond_broadcast() before my_thread_end() Added mariadb_dbug_id() to count memory per THD instead of per thread. Added my_malloc_size_cb_func() callback function for my_malloc() to count memory. Move initialization of mysqld_server_started and mysqld_server_initialized earlier. Updated call to my_init_dynamic_array(). Updated call to my_net_init(). Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread. Added status variable 'memory_used'. Updated call to init_alloc_root() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/mysqld.h: Added set_current_thd() sql/net_serv.cc: Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC. Store in net->thread_specific_malloc if memory is thread specific. Mark memory to be thread specific if requested. sql/opt_range.cc: Updated call to my_init_dynamic_array() Updated call to init_sql_alloc() Added MY_THREAD_SPECIFIC to allocated memory. sql/opt_subselect.cc: Updated call to init_sql_alloc() to mark memory thread specific. sql/protocol.cc: Fixed compiler warning sql/records.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/rpl_filter.cc: Updated call to my_init_dynamic_array() sql/rpl_handler.cc: Updated call to my_init_dynamic_array2() sql/rpl_handler.h: Updated call to init_sql_alloc() sql/rpl_mi.cc: Updated call to my_init_dynamic_array() sql/rpl_tblmap.cc: Updated call to init_alloc_root() sql/rpl_utility.cc: Updated call to my_init_dynamic_array() sql/slave.cc: Initialize things properly before calling functions that allocate memory. Removed calls to net_end() as this is now done in ~THD() sql/sp_head.cc: Updated call to init_sql_alloc() Updated call to my_init_dynamic_array() Added parameter to warning_info() that it should be fully initialized. sql/sp_pcontext.cc: Updated call to my_init_dynamic_array() sql/sql_acl.cc: Updated call to init_sql_alloc() Updated call to my_init_dynamic_array() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_admin.cc: Added parameter to warning_info() that it should be fully initialized. sql/sql_analyse.h: Updated call to init_tree() to mark memory thread specific. sql/sql_array.h: Updated call to my_init_dynamic_array() to mark memory thread specific. sql/sql_audit.cc: Updated call to my_init_dynamic_array() sql/sql_base.cc: Updated call to init_sql_alloc() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_cache.cc: Updated comment sql/sql_class.cc: Added parameter to warning_info() that not initialize it until THD is fully created. Updated call to init_sql_alloc() Mark THD::user_vars has to be thread specific. Updated call to my_init_dynamic_array() Ensure that memory allocated by THD is assigned to the THD. More DBUG Always acll net_end() in ~THD() Assert that all memory signed to this THD is really deleted at ~THD. Fixed set_status_var_init() to not reset memory_used. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_class.h: Added MY_THREAD_SPECIFIC to allocated memory. Added malloc_size to THD to record allocated memory per THD. sql/sql_delete.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_error.cc: Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures. This is used by THD::THD() to not allocate memory until THD is ready. Added Warning_info::free_memory() sql/sql_error.h: Updated Warning_info() class. sql/sql_handler.cc: Updated call to init_alloc_root() to mark memory thread specific. sql/sql_insert.cc: More DBUG sql/sql_join_cache.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_lex.cc: Updated call to my_init_dynamic_array() sql/sql_lex.h: Updated call to my_init_dynamic_array() sql/sql_load.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_parse.cc: Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() Ensure that examined_row_count() is reset before query. Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() Don't restore thd->status_var.memory_used when restoring thd->status_var sql/sql_plugin.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_prepare.cc: Added parameter to warning_info() that it should be fully initialized. Updated call to init_sql_alloc() to mark memory thread specific. sql/sql_reload.cc: my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_select.cc: Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific. Added MY_THREAD_SPECIFIC to allocated memory. More DBUG sql/sql_servers.cc: Updated call to init_sql_alloc() to mark memory some memory thread specific. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_show.cc: Updated call to my_init_dynamic_array() Mark my_dir() memory thread specific. Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread. More DBUG. Added malloc_size and examined_row_count to SHOW PROCESSLIST. Added MY_THREAD_SPECIFIC to allocated memory. Updated call to init_sql_alloc() Added parameter to warning_info() that it should be fully initialized. sql/sql_statistics.cc: Fixed compiler warning sql/sql_string.cc: String elements can now be marked as thread specific. sql/sql_string.h: String elements can now be marked as thread specific. sql/sql_table.cc: Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() Fixed compiler warning sql/sql_test.cc: Updated call to my_init_dynamic_array() to mark memory thread specific. sql/sql_trigger.cc: Updated call to init_sql_alloc() sql/sql_udf.cc: Updated call to init_sql_alloc() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_update.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/table.cc: Updated call to init_sql_alloc(). Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC Updated call to init_sql_alloc() sql/thr_malloc.cc: Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC. sql/thr_malloc.h: Updated prototype for init_sql_alloc() sql/tztime.cc: Updated call to init_sql_alloc() Updated call to init_alloc_root() to mark memory thread specific. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/uniques.cc: Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific. sql/unireg.cc: Added MY_THREAD_SPECIFIC to allocated memory. storage/csv/ha_tina.cc: Updated call to init_alloc_root() storage/federated/ha_federated.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Ensure that memory allocated by fedarated is registered for the system, not for the thread. storage/federatedx/federatedx_io_mysql.cc: Updated call to my_init_dynamic_array() storage/federatedx/ha_federatedx.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() storage/heap/ha_heap.cc: Added MY_THREAD_SPECIFIC to allocated memory. storage/heap/heapdef.h: Added parameter to hp_get_new_block() to be able to do thread specific memory tagging. storage/heap/hp_block.c: Added parameter to hp_get_new_block() to be able to do thread specific memory tagging. storage/heap/hp_create.c: - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'. storage/heap/hp_open.c: Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. storage/heap/hp_write.c: Added new parameter to hp_get_new_block() storage/maria/ma_bitmap.c: Updated call to my_init_dynamic_array() storage/maria/ma_blockrec.c: Updated call to my_init_dynamic_array() storage/maria/ma_check.c: Updated call to init_alloc_root() storage/maria/ma_ft_boolean_search.c: Updated calls to init_tree() and init_alloc_root() storage/maria/ma_ft_nlq_search.c: Updated call to init_tree() storage/maria/ma_ft_parser.c: Updated call to init_tree() Updated call to init_alloc_root() storage/maria/ma_loghandler.c: Updated call to my_init_dynamic_array() storage/maria/ma_open.c: Updated call to my_init_dynamic_array() storage/maria/ma_sort.c: Updated call to my_init_dynamic_array() storage/maria/ma_write.c: Updated calls to my_init_dynamic_array() and init_tree() storage/maria/maria_pack.c: Updated call to init_tree() storage/maria/unittest/sequence_storage.c: Updated call to my_init_dynamic_array() storage/myisam/ft_boolean_search.c: Updated call to init_tree() Updated call to init_alloc_root() storage/myisam/ft_nlq_search.c: Updated call to init_tree() storage/myisam/ft_parser.c: Updated call to init_tree() Updated call to init_alloc_root() storage/myisam/ft_stopwords.c: Updated call to init_tree() storage/myisam/mi_check.c: Updated call to init_alloc_root() storage/myisam/mi_write.c: Updated call to my_init_dynamic_array() Updated call to init_tree() storage/myisam/myisamlog.c: Updated call to init_tree() storage/myisam/myisampack.c: Updated call to init_tree() storage/myisam/sort.c: Updated call to my_init_dynamic_array() storage/myisammrg/ha_myisammrg.cc: Updated call to init_sql_alloc() storage/perfschema/pfs_check.cc: Rest current_thd storage/perfschema/pfs_instr.cc: Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems. support-files/compiler_warnings.supp: Disable compiler warning from offsetof macro.
* Merged the fix for bug mdev-4019.Igor Babaev2013-01-131-2/+8
|\
| * Fixed bug mdev-4019.Igor Babaev2013-01-131-2/+8
| | | | | | | | | | | | | | | | The bug could cause a crash when several connections needed persistent statistics for the same table. Also added a missing call of set_statistics_for_table() in the code of the function mysql_update.
* | Merge 5.5-mwl248 -> 10.0-baseIgor Babaev2013-01-081-147/+193
|\ \ | |/
| * Fixed bug mdev-3979.Igor Babaev2013-01-081-147/+193
| | | | | | | | | | | | Made allocation of memory for statistical data in a table share to be thread safe. This memory is now allocated in a special MEM_ROOT that is created for each table share.
* | Merge maria-5.5-mwl248 -> 10.0-base.Igor Babaev2012-12-161-30/+19
|/
* Addressed all remaining issues from the review of the patchIgor Babaev2012-12-131-8/+54
| | | | | | | | | | | that introduced engine independent persistent statistics. In particular: - added an enumeration type for possible values of the system variable use_stat_tables - renamed KEY::real_rec_per_key to KEY::actual_rec_per_key - optimized the collection of statistical data for any primary key defined only on one column.
* Fixed bug mdev-3891.Igor Babaev2012-12-121-40/+62
| | | | | | | | | | | | If a query referenced some system statistical tables, but not all of them, then executing an ANALYZE command simultaneously with this query could lead to a deadlock. The fix prohibited reading statistics from system statistical tables for such queries. Removed the function unlock_tables_n_open_system_tables_for_write() as not used anymore. Performed some minor refactoring of the code in sql_statistics.cc.
* Addressed the following issue from the review of the patch forIgor Babaev2012-12-091-0/+2
| | | | | | | engine-independent statistics. If a table was created for InnoDB then the execution of the ANALYZE command over this table blocked any INSERT/DELETE/UPDATE of the table.
* Addressed the following issue from the review of the patch forIgor Babaev2012-12-081-7/+26
| | | | | | | | | | | | | engine-independent statistics. When the primary key was dropped or changed statistics on secondary indexes for the prefixes that included components of the primary key was not removed from the table mysql.index_stats. Also fixed: in the some cases when a column was changed statistics on the indexes that included this column was not removed from the table mysql.index_stats. Also disabled the test mdev-504 for --ps-protocol.
* Changed the names of the system tables for statistical data:Igor Babaev2012-12-051-77/+77
| | | | | | | | | table_stat -> table_stats column_stat -> column_stats index_stat -> index_stats to be in line with the names of innodb statistical tables from mysql-5.6: innodb_table_stats and innodb_index_stats.
View Lorry Controller Status