diff options
author | Monty <monty@mariadb.org> | 2021-04-14 13:40:36 +0300 |
---|---|---|
committer | Sergei Golubchik <serg@mariadb.org> | 2021-05-19 22:54:14 +0200 |
commit | cc125bebfe9d06870cf1b66bd42c730398c8aa30 (patch) | |
tree | 5c26ad6b047eee76a04cc2b37b992fa847327f22 | |
parent | b332ffc1af46691abc5a8eae1423ea053db62433 (diff) | |
download | mariadb-git-cc125bebfe9d06870cf1b66bd42c730398c8aa30.tar.gz |
Fix all warnings given by UBSAN
The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
memory access of integers. Fixed by using byte_order_generic.h when
compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
safe to have overflows (two cases, in item_func.cc).
Things fixed:
- Don't left shift signed values
(byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
constructors. This was needed as UBSAN checks that these types has
correct values when one copies an object.
(gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
deleted objects.
(events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
on Query_arena object.
- Fixed several cast of objects to an incompatible class!
(Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
This includes also ++ and -- of integers.
(Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
value_type is initialized to this instead of to -1, which is not a valid
enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
Other things:
- Changed struct st_position to an OBJECT and added an initialization
function to it to ensure that we do not copy or use uninitialized
members. The change to a class was also motived that we used "struct
st_position" and POSITION randomly trough the code which was
confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr. (This variable was before
only in 10.5 and up). It can now have one of two values:
ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
it virtual. This was an effort to get UBSAN to work with loaded storage
engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
in tabutil.cpp.
Changes that should not be needed but had to be done to suppress warnings
from UBSAN:
- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
some compile time warnings.
Fixes:
MDEV-25505 Assertion `old_flags == ((my_flags & 0x10000U) ? 1 : 0)
fixed (was caused by an old version if this commit).
Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia
Temporary commit to log changes for UBSAN
-rwxr-xr-x | BUILD/compile-pentium64-ubsan | 2 | ||||
-rw-r--r-- | mysql-test/suite/maria/alter.result | 9 | ||||
-rw-r--r-- | mysql-test/suite/maria/alter.test | 11 | ||||
-rw-r--r-- | mysql-test/suite/plugins/t/multiauth.test | 1 | ||||
-rw-r--r-- | sql/item_strfunc.cc | 6 | ||||
-rw-r--r-- | sql/sql_acl.cc | 55 | ||||
-rw-r--r-- | sql/sql_select.cc | 96 | ||||
-rw-r--r-- | sql/sql_select.h | 93 | ||||
-rw-r--r-- | sql/sql_type_int.h | 22 | ||||
-rw-r--r-- | storage/federated/ha_federated.cc | 3 | ||||
-rw-r--r-- | storage/maria/ma_check.c | 69 | ||||
-rw-r--r-- | storage/mroonga/vendor/groonga/lib/io.c | 4 | ||||
-rw-r--r-- | storage/mroonga/vendor/groonga/lib/ts/ts_expr_node.c | 4 |
13 files changed, 218 insertions, 157 deletions
diff --git a/BUILD/compile-pentium64-ubsan b/BUILD/compile-pentium64-ubsan index bf56d84283f..1a2b45413b0 100755 --- a/BUILD/compile-pentium64-ubsan +++ b/BUILD/compile-pentium64-ubsan @@ -24,6 +24,6 @@ path=`dirname $0` . "$path/SETUP.sh" extra_flags="$pentium64_cflags $debug_cflags -fsanitize=undefined -DWITH_UBSAN -Wno-conversion -Wno-uninitialized" -extra_configs="$pentium_configs $debug_configs -DWITH_UBSAN=ON -DMYSQL_MAINTAINER_MODE=NO --without-spider --without-tokudb" +extra_configs="$pentium_configs $debug_configs -DWITH_UBSAN=ON -DMYSQL_MAINTAINER_MODE=NO --without-spider" . "$path/FINISH.sh" diff --git a/mysql-test/suite/maria/alter.result b/mysql-test/suite/maria/alter.result index e1a57ed654f..0bb8c89c1db 100644 --- a/mysql-test/suite/maria/alter.result +++ b/mysql-test/suite/maria/alter.result @@ -184,3 +184,12 @@ SET GLOBAL aria_max_sort_file_size= @max_size.save; # # End of 10.4 test # +# +# MDEV-25505 Assertion `old_flags == ((my_flags & 0x10000U) ? 1 : 0)' +# failed in my_realloc +# +CREATE TABLE t1 (pk int, c text, primary key (pk), key(c(32))) ENGINE=Aria ROW_FORMAT=DYNAMIC; +ALTER TABLE t1 DISABLE KEYS; +INSERT INTO t1 VALUES (1, 'Nine chars or more'); +ALTER TABLE t1 ENABLE KEYS; +DROP TABLE t1; diff --git a/mysql-test/suite/maria/alter.test b/mysql-test/suite/maria/alter.test index f2950f0a6d3..525cd80f3d9 100644 --- a/mysql-test/suite/maria/alter.test +++ b/mysql-test/suite/maria/alter.test @@ -192,3 +192,14 @@ SET GLOBAL aria_max_sort_file_size= @max_size.save; --echo # --echo # End of 10.4 test --echo # + +--echo # +--echo # MDEV-25505 Assertion `old_flags == ((my_flags & 0x10000U) ? 1 : 0)' +--echo # failed in my_realloc +--echo # + +CREATE TABLE t1 (pk int, c text, primary key (pk), key(c(32))) ENGINE=Aria ROW_FORMAT=DYNAMIC; +ALTER TABLE t1 DISABLE KEYS; +INSERT INTO t1 VALUES (1, 'Nine chars or more'); +ALTER TABLE t1 ENABLE KEYS; +DROP TABLE t1; diff --git a/mysql-test/suite/plugins/t/multiauth.test b/mysql-test/suite/plugins/t/multiauth.test index 66f29bdbfec..655ab88b635 100644 --- a/mysql-test/suite/plugins/t/multiauth.test +++ b/mysql-test/suite/plugins/t/multiauth.test @@ -1,4 +1,5 @@ --source include/not_ubsan.inc + let $REGEX_VERSION_ID=/$mysql_get_server_version/VERSION_ID/; let $REGEX_PASSWORD_LAST_CHANGED=/password_last_changed": [0-9]*/password_last_changed": #/; let $REGEX_GLOBAL_PRIV=$REGEX_PASSWORD_LAST_CHANGED $REGEX_VERSION_ID; diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc index b09b90fe3d5..9f7ff0eca74 100644 --- a/sql/item_strfunc.cc +++ b/sql/item_strfunc.cc @@ -3867,6 +3867,7 @@ String *Item_load_file::val_str(String *str) File file; MY_STAT stat_info; char path[FN_REFLEN]; + size_t file_size; DBUG_ENTER("load_file"); if (!(file_name= args[0]->val_str(str)) @@ -3891,10 +3892,11 @@ String *Item_load_file::val_str(String *str) /* my_error(ER_TEXTFILE_NOT_READABLE, MYF(0), file_name->c_ptr()); */ goto err; } + file_size= stat_info.st_size; { THD *thd= current_thd; - if (stat_info.st_size > (long) thd->variables.max_allowed_packet) + if (file_size >= (size_t) thd->variables.max_allowed_packet) { push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN, ER_WARN_ALLOWED_PACKET_OVERFLOWED, @@ -3903,7 +3905,7 @@ String *Item_load_file::val_str(String *str) goto err; } } - if (tmp_value.alloc((size_t)stat_info.st_size)) + if (tmp_value.alloc(file_size)) goto err; if ((file= mysql_file_open(key_file_loadfile, file_name->ptr(), O_RDONLY, MYF(0))) < 0) diff --git a/sql/sql_acl.cc b/sql/sql_acl.cc index 7a048ea45a0..3a4c83e3139 100644 --- a/sql/sql_acl.cc +++ b/sql/sql_acl.cc @@ -6955,7 +6955,7 @@ int mysql_table_grant(THD *thd, TABLE_LIST *table_list, bool revoke_grant) { privilege_t column_priv(NO_ACL); - int result; + int result, res; List_iterator <LEX_USER> str_list (user_list); LEX_USER *Str, *tmp_Str; bool create_new_users=0; @@ -7156,10 +7156,10 @@ int mysql_table_grant(THD *thd, TABLE_LIST *table_list, revoke_grant)) result= TRUE; } - if (int res= replace_table_table(thd, grant_table, - tables.tables_priv_table().table(), - *Str, db_name, table_name, - rights, column_priv, revoke_grant)) + if ((res= replace_table_table(thd, grant_table, + tables.tables_priv_table().table(), + *Str, db_name, table_name, + rights, column_priv, revoke_grant))) { if (res > 0) { @@ -11277,7 +11277,7 @@ mysql_revoke_sp_privs(THD *thd, Grant_tables *tables, const Sp_handler *sph, bool mysql_revoke_all(THD *thd, List <LEX_USER> &list) { uint counter, revoked; - int result; + int result, res; ACL_DB *acl_db; DBUG_ENTER("mysql_revoke_all"); @@ -11361,30 +11361,33 @@ bool mysql_revoke_all(THD *thd, List <LEX_USER> &list) { for (counter= 0, revoked= 0 ; counter < column_priv_hash.records ; ) { - const char *user,*host; - GRANT_TABLE *grant_table= - (GRANT_TABLE*) my_hash_element(&column_priv_hash, counter); + const char *user,*host; + GRANT_TABLE *grant_table= ((GRANT_TABLE*) + my_hash_element(&column_priv_hash, counter)); + user= grant_table->user; host= safe_str(grant_table->host.hostname); if (!strcmp(lex_user->user.str,user) && !strcmp(lex_user->host.str, host)) - { - List<LEX_COLUMN> columns; - /* TODO(cvicentiu) refactor to use + { + List<LEX_COLUMN> columns; + /* TODO(cvicentiu) refactor replace_db_table to use + Db_table instead of TABLE directly. */ + if (replace_column_table(grant_table, + tables.columns_priv_table().table(), + *lex_user, columns, grant_table->db, + grant_table->tname, ALL_KNOWN_ACL, 1)) + result= -1; + + /* TODO(cvicentiu) refactor replace_db_table to use Db_table instead of TABLE directly. */ - if (replace_column_table(grant_table, - tables.columns_priv_table().table(), - *lex_user, columns, - grant_table->db, grant_table->tname, - ALL_KNOWN_ACL, 1)) - result= -1; - if (int res= replace_table_table(thd, grant_table, - tables.tables_priv_table().table(), - *lex_user, - grant_table->db, grant_table->tname, - ALL_KNOWN_ACL, NO_ACL, 1)) - { + if ((res= replace_table_table(thd, grant_table, + tables.tables_priv_table().table(), + *lex_user, grant_table->db, + grant_table->tname, ALL_KNOWN_ACL, + NO_ACL, 1))) + { if (res > 0) result= -1; else @@ -11397,8 +11400,8 @@ bool mysql_revoke_all(THD *thd, List <LEX_USER> &list) continue; } } - } - counter++; + } + counter++; } } while (revoked); diff --git a/sql/sql_select.cc b/sql/sql_select.cc index 41b6f8894ab..a7ba0c9ffff 100644 --- a/sql/sql_select.cc +++ b/sql/sql_select.cc @@ -366,7 +366,6 @@ bool dbug_user_var_equals_int(THD *thd, const char *name, int value) } #endif /* DBUG_OFF */ - /* Intialize POSITION structure. */ @@ -391,6 +390,100 @@ POSITION::POSITION() } +void JOIN::init(THD *thd_arg, List<Item> &fields_arg, + ulonglong select_options_arg, select_result *result_arg) +{ + join_tab= 0; + table= 0; + table_count= 0; + top_join_tab_count= 0; + const_tables= 0; + const_table_map= found_const_table_map= 0; + aggr_tables= 0; + eliminated_tables= 0; + join_list= 0; + implicit_grouping= FALSE; + sort_and_group= 0; + first_record= 0; + do_send_rows= 1; + duplicate_rows= send_records= 0; + found_records= accepted_rows= 0; + fetch_limit= HA_POS_ERROR; + thd= thd_arg; + sum_funcs= sum_funcs2= 0; + procedure= 0; + having= tmp_having= having_history= 0; + having_is_correlated= false; + group_list_for_estimates= 0; + select_options= select_options_arg; + result= result_arg; + lock= thd_arg->lock; + select_lex= 0; //for safety + select_distinct= MY_TEST(select_options & SELECT_DISTINCT); + no_order= 0; + simple_order= 0; + simple_group= 0; + rand_table_in_field_list= 0; + ordered_index_usage= ordered_index_void; + need_distinct= 0; + skip_sort_order= 0; + with_two_phase_optimization= 0; + save_qep= 0; + spl_opt_info= 0; + ext_keyuses_for_splitting= 0; + spl_opt_info= 0; + need_tmp= 0; + hidden_group_fields= 0; /*safety*/ + error= 0; + select= 0; + return_tab= 0; + ref_ptrs.reset(); + items0.reset(); + items1.reset(); + items2.reset(); + items3.reset(); + zero_result_cause= 0; + optimization_state= JOIN::NOT_OPTIMIZED; + have_query_plan= QEP_NOT_PRESENT_YET; + initialized= 0; + cleaned= 0; + cond_equal= 0; + having_equal= 0; + exec_const_cond= 0; + group_optimized_away= 0; + no_rows_in_result_called= 0; + positions= best_positions= 0; + pushdown_query= 0; + original_join_tab= 0; + explain= NULL; + tmp_table_keep_current_rowid= 0; + + all_fields= fields_arg; + if (&fields_list != &fields_arg) /* Avoid valgrind-warning */ + fields_list= fields_arg; + non_agg_fields.empty(); + bzero((char*) &keyuse,sizeof(keyuse)); + having_value= Item::COND_UNDEF; + tmp_table_param.init(); + tmp_table_param.end_write_records= HA_POS_ERROR; + rollup.state= ROLLUP::STATE_NONE; + + no_const_tables= FALSE; + first_select= sub_select; + set_group_rpa= false; + group_sent= 0; + + outer_ref_cond= pseudo_bits_cond= NULL; + in_to_exists_where= NULL; + in_to_exists_having= NULL; + emb_sjm_nest= NULL; + sjm_lookup_tables= 0; + sjm_scan_tables= 0; + is_orig_degenerated= false; + with_ties_order_count= 0; +}; + + static void trace_table_dependencies(THD *thd, JOIN_TAB *join_tabs, uint table_count) { @@ -29888,6 +29981,7 @@ bool JOIN::optimize_upper_rownum_func() return process_direct_rownum_comparison(thd, unit, outer_select->where); } + /** Test if the predicate compares rownum() with a constant diff --git a/sql/sql_select.h b/sql/sql_select.h index 3b736198541..d4e06f42249 100644 --- a/sql/sql_select.h +++ b/sql/sql_select.h @@ -1546,98 +1546,7 @@ public: } void init(THD *thd_arg, List<Item> &fields_arg, ulonglong select_options_arg, - select_result *result_arg) - { - join_tab= 0; - table= 0; - table_count= 0; - top_join_tab_count= 0; - const_tables= 0; - const_table_map= found_const_table_map= 0; - aggr_tables= 0; - eliminated_tables= 0; - join_list= 0; - implicit_grouping= FALSE; - sort_and_group= 0; - first_record= 0; - do_send_rows= 1; - duplicate_rows= send_records= 0; - found_records= accepted_rows= 0; - fetch_limit= HA_POS_ERROR; - thd= thd_arg; - sum_funcs= sum_funcs2= 0; - procedure= 0; - having= tmp_having= having_history= 0; - having_is_correlated= false; - group_list_for_estimates= 0; - select_options= select_options_arg; - result= result_arg; - lock= thd_arg->lock; - select_lex= 0; //for safety - select_distinct= MY_TEST(select_options & SELECT_DISTINCT); - no_order= 0; - simple_order= 0; - simple_group= 0; - rand_table_in_field_list= 0; - ordered_index_usage= ordered_index_void; - need_distinct= 0; - skip_sort_order= 0; - with_two_phase_optimization= 0; - save_qep= 0; - spl_opt_info= 0; - ext_keyuses_for_splitting= 0; - spl_opt_info= 0; - need_tmp= 0; - hidden_group_fields= 0; /*safety*/ - error= 0; - select= 0; - return_tab= 0; - ref_ptrs.reset(); - items0.reset(); - items1.reset(); - items2.reset(); - items3.reset(); - zero_result_cause= 0; - optimization_state= JOIN::NOT_OPTIMIZED; - have_query_plan= QEP_NOT_PRESENT_YET; - initialized= 0; - cleaned= 0; - cond_equal= 0; - having_equal= 0; - exec_const_cond= 0; - group_optimized_away= 0; - no_rows_in_result_called= 0; - positions= best_positions= 0; - pushdown_query= 0; - original_join_tab= 0; - explain= NULL; - tmp_table_keep_current_rowid= 0; - - all_fields= fields_arg; - if (&fields_list != &fields_arg) /* Avoid valgrind-warning */ - fields_list= fields_arg; - non_agg_fields.empty(); - bzero((char*) &keyuse,sizeof(keyuse)); - having_value= Item::COND_UNDEF; - tmp_table_param.init(); - tmp_table_param.end_write_records= HA_POS_ERROR; - rollup.state= ROLLUP::STATE_NONE; - - no_const_tables= FALSE; - first_select= sub_select; - set_group_rpa= false; - group_sent= 0; - - outer_ref_cond= pseudo_bits_cond= NULL; - in_to_exists_where= NULL; - in_to_exists_having= NULL; - emb_sjm_nest= NULL; - sjm_lookup_tables= 0; - sjm_scan_tables= 0; - is_orig_degenerated= false; - - with_ties_order_count= 0; - } + select_result *result_arg); /* True if the plan guarantees that it will be returned zero or one row */ bool only_const_tables() { return const_tables == table_count; } diff --git a/sql/sql_type_int.h b/sql/sql_type_int.h index fca00f25448..c22b1038890 100644 --- a/sql/sql_type_int.h +++ b/sql/sql_type_int.h @@ -77,23 +77,27 @@ public: } Longlong_null operator<<(const Longlong_null &llshift) const { + ulonglong res; + uint shift; if (is_null() || llshift.is_null()) return Longlong_null(); - uint shift= (uint) llshift.value(); - ulonglong res= (shift < sizeof(longlong) * 8) - ? ((ulonglong) value()) << shift - : 0; + shift= (uint) llshift.value(); + res= 0; + if (shift < sizeof(longlong) * 8) + res= ((ulonglong) value()) << shift; return Longlong_null((longlong) res); } Longlong_null operator>>(const Longlong_null &llshift) const { + ulonglong res; + uint shift; if (is_null() || llshift.is_null()) return Longlong_null(); - uint shift= (uint) llshift.value(); - ulonglong res= (shift < sizeof(longlong) * 8) - ? ((ulonglong) value()) >> shift - : 0; - return Longlong_null((longlong) res); + shift= (uint) llshift.value(); + res= 0; + if (shift < sizeof(longlong) * 8) + res= ((ulonglong) value()) >> shift; + return Longlong_null(res); } Longlong_null bit_count() const { diff --git a/storage/federated/ha_federated.cc b/storage/federated/ha_federated.cc index 758b335e13a..faaffa21f6b 100644 --- a/storage/federated/ha_federated.cc +++ b/storage/federated/ha_federated.cc @@ -3179,7 +3179,8 @@ int ha_federated::real_connect() sql_query.append('\''); mysql_close(mysql); mysql= NULL; - my_error(ER_FOREIGN_DATA_SOURCE_DOESNT_EXIST, MYF(0), sql_query.ptr()); + my_error(ER_FOREIGN_DATA_SOURCE_DOESNT_EXIST, MYF(0), + sql_query.c_ptr_safe()); remote_error_number= -1; DBUG_RETURN(-1); } diff --git a/storage/maria/ma_check.c b/storage/maria/ma_check.c index d996cd6a757..a312c959e3a 100644 --- a/storage/maria/ma_check.c +++ b/storage/maria/ma_check.c @@ -125,6 +125,7 @@ void maria_chk_init(HA_CHECK *param) param->max_stage= 1; param->stack_end_ptr= &my_thread_var->stack_ends_here; param->max_allowed_lsn= (LSN) ~0ULL; + param->malloc_flags= MY_THREAD_SPECIFIC; } @@ -2122,7 +2123,8 @@ int maria_chk_data_link(HA_CHECK *param, MARIA_HA *info, my_bool extend) } if (!(record= (uchar*) my_malloc(PSI_INSTRUMENT_ME, - share->base.default_rec_buff_size, MYF(0)))) + share->base.default_rec_buff_size, + MYF(param->malloc_flags)))) { _ma_check_print_error(param,"Not enough memory for record"); DBUG_RETURN(-1); @@ -2756,9 +2758,11 @@ int maria_repair(HA_CHECK *param, register MARIA_HA *info, if (!(sort_param.record= (uchar *) my_malloc(PSI_INSTRUMENT_ME, (uint) - share->base.default_rec_buff_size, MYF(0))) || + share->base.default_rec_buff_size, + MYF(param->malloc_flags))) || _ma_alloc_buffer(&sort_param.rec_buff, &sort_param.rec_buff_size, - share->base.default_rec_buff_size, MYF(0))) + share->base.default_rec_buff_size, + MYF(param->malloc_flags))) { _ma_check_print_error(param, "Not enough memory for extra record"); goto err; @@ -3703,7 +3707,8 @@ int maria_filecopy(HA_CHECK *param, File to,File from,my_off_t start, DBUG_ENTER("maria_filecopy"); buff_length=(ulong) MY_MIN(param->write_buffer_length,length); - if (!(buff=my_malloc(PSI_INSTRUMENT_ME, buff_length, MYF(0)))) + if (!(buff=my_malloc(PSI_INSTRUMENT_ME, buff_length, + MYF(param->malloc_flags)))) { buff=tmp_buff; buff_length=IO_SIZE; } @@ -3849,9 +3854,10 @@ int maria_repair_by_sort(HA_CHECK *param, register MARIA_HA *info, if (!(sort_param.record= (uchar*) my_malloc(PSI_INSTRUMENT_ME, (size_t) share->base.default_rec_buff_size, - MYF(0))) || + MYF(param->malloc_flags))) || _ma_alloc_buffer(&sort_param.rec_buff, &sort_param.rec_buff_size, - share->base.default_rec_buff_size, MYF(0))) + share->base.default_rec_buff_size, + MYF(param->malloc_flags))) { _ma_check_print_error(param, "Not enough memory for extra record"); goto err; @@ -3868,7 +3874,8 @@ int maria_repair_by_sort(HA_CHECK *param, register MARIA_HA *info, param->read_cache.end_of_file= sort_info.filelength; sort_param.wordlist=NULL; - init_alloc_root(PSI_INSTRUMENT_ME, &sort_param.wordroot, FTPARSER_MEMROOT_ALLOC_SIZE, 0, + init_alloc_root(PSI_INSTRUMENT_ME, &sort_param.wordroot, + FTPARSER_MEMROOT_ALLOC_SIZE, 0, MYF(param->malloc_flags)); sort_param.key_cmp=sort_key_cmp; @@ -4437,7 +4444,7 @@ int maria_repair_parallel(HA_CHECK *param, register MARIA_HA *info, if (!(sort_param=(MARIA_SORT_PARAM *) my_malloc(PSI_INSTRUMENT_ME, (uint) share->base.keys * (sizeof(MARIA_SORT_PARAM) + share->base.pack_reclength), - MYF(MY_ZEROFILL)))) + MYF(MY_ZEROFILL | param->malloc_flags)))) { _ma_check_print_error(param,"Not enough memory for key!"); goto err; @@ -4494,7 +4501,8 @@ int maria_repair_parallel(HA_CHECK *param, register MARIA_HA *info, sort_param[i].record= (((uchar *)(sort_param+share->base.keys))+ (share->base.pack_reclength * i)); if (_ma_alloc_buffer(&sort_param[i].rec_buff, &sort_param[i].rec_buff_size, - share->base.default_rec_buff_size, MYF(0))) + share->base.default_rec_buff_size, + MYF(param->malloc_flags))) { _ma_check_print_error(param,"Not enough memory!"); goto err; @@ -4519,7 +4527,8 @@ int maria_repair_parallel(HA_CHECK *param, register MARIA_HA *info, (FT_MAX_WORD_LEN_FOR_SORT * sort_param[i].keyinfo->seg->charset->mbmaxlen); sort_param[i].key_length+=ft_max_word_len_for_sort-HA_FT_MAXBYTELEN; - init_alloc_root(PSI_INSTRUMENT_ME, &sort_param[i].wordroot, FTPARSER_MEMROOT_ALLOC_SIZE, 0, + init_alloc_root(PSI_INSTRUMENT_ME, &sort_param[i].wordroot, + FTPARSER_MEMROOT_ALLOC_SIZE, 0, MYF(param->malloc_flags)); } } @@ -4554,6 +4563,13 @@ int maria_repair_parallel(HA_CHECK *param, register MARIA_HA *info, (void) pthread_attr_setdetachstate(&thr_attr,PTHREAD_CREATE_DETACHED); (void) my_setstacksize(&thr_attr, (size_t)my_thread_stack_size); + /* + We cannot mark future memory allocations as thread specific when + doing parallel repair as we don't have a THD for each thread. Sharing the + same THD this would requre mutex locks around mallocs/reallocs to ensure + that two threads does not use the same THD at once. + */ + param->malloc_flags= 0; for (i=0 ; i < sort_info.total_keys ; i++) { /* @@ -5226,7 +5242,8 @@ static int sort_get_next_record(MARIA_SORT_PARAM *sort_param) if (_ma_alloc_buffer(&sort_param->rec_buff, &sort_param->rec_buff_size, block_info.rec_len + - share->base.extra_rec_buff_size, MYF(0))) + share->base.extra_rec_buff_size, + MYF(param->malloc_flags))) { if (param->max_record_length >= block_info.rec_len) @@ -5503,9 +5520,11 @@ int _ma_sort_write_record(MARIA_SORT_PARAM *sort_param) MARIA_DYN_DELETE_BLOCK_HEADER; if (sort_info->buff_length < reclength) { - if (!(sort_info->buff=my_realloc(PSI_INSTRUMENT_ME, sort_info->buff, (uint) reclength, - MYF(MY_FREE_ON_ERROR | - MY_ALLOW_ZERO_PTR)))) + if (!(sort_info->buff= my_realloc(PSI_INSTRUMENT_ME, sort_info->buff, + (uint) reclength, + MYF(MY_FREE_ON_ERROR | + MY_ALLOW_ZERO_PTR | + param->malloc_flags)))) DBUG_RETURN(1); sort_info->buff_length=reclength; } @@ -5733,8 +5752,12 @@ static int sort_maria_ft_key_write(MARIA_SORT_PARAM *sort_param, share->rec_reflength) && (share->options & (HA_OPTION_PACK_RECORD | HA_OPTION_COMPRESS_RECORD))) - ft_buf= (MA_SORT_FT_BUF *)my_malloc(PSI_INSTRUMENT_ME, sort_param->keyinfo->block_length + - sizeof(MA_SORT_FT_BUF), MYF(MY_WME)); + ft_buf= (MA_SORT_FT_BUF *)my_malloc(PSI_INSTRUMENT_ME, + sort_param->keyinfo->block_length + + sizeof(MA_SORT_FT_BUF), + MYF(MY_WME | + sort_param->sort_info->param-> + malloc_flags)); if (!ft_buf) { @@ -6059,8 +6082,10 @@ static MA_SORT_KEY_BLOCKS *alloc_key_blocks(HA_CHECK *param, uint blocks, MA_SORT_KEY_BLOCKS *block; DBUG_ENTER("alloc_key_blocks"); - if (!(block= (MA_SORT_KEY_BLOCKS*) my_malloc(PSI_INSTRUMENT_ME, - (sizeof(MA_SORT_KEY_BLOCKS)+buffer_length+IO_SIZE)*blocks, MYF(0)))) + if (!(block= (MA_SORT_KEY_BLOCKS*) + my_malloc(PSI_INSTRUMENT_ME, + (sizeof(MA_SORT_KEY_BLOCKS)+buffer_length+IO_SIZE)*blocks, + MYF(param->malloc_flags)))) { _ma_check_print_error(param,"Not enough memory for sort-key-blocks"); return(0); @@ -6392,8 +6417,9 @@ void _ma_update_auto_increment_key(HA_CHECK *param, MARIA_HA *info, We have to use an allocated buffer instead of info->rec_buff as _ma_put_key_in_record() may use info->rec_buff */ - if (!(record= (uchar*) my_malloc(PSI_INSTRUMENT_ME, (size_t) share->base.default_rec_buff_size, - MYF(0)))) + if (!(record= (uchar*) my_malloc(PSI_INSTRUMENT_ME, + (size_t) share->base.default_rec_buff_size, + MYF(param->malloc_flags)))) { _ma_check_print_error(param,"Not enough memory for extra record"); DBUG_VOID_RETURN; @@ -7037,7 +7063,8 @@ static void print_bitmap_description(MARIA_SHARE *share, pgcache_page_no_t page, uchar *bitmap_data) { - char *tmp= my_malloc(PSI_INSTRUMENT_ME, MAX_BITMAP_INFO_LENGTH, MYF(MY_WME)); + char *tmp= my_malloc(PSI_INSTRUMENT_ME, MAX_BITMAP_INFO_LENGTH, + MYF(MY_WME | MY_THREADSAFE)); if (!tmp) return; _ma_get_bitmap_description(&share->bitmap, bitmap_data, page, tmp); diff --git a/storage/mroonga/vendor/groonga/lib/io.c b/storage/mroonga/vendor/groonga/lib/io.c index 6aecb6e53c4..2f59b87431d 100644 --- a/storage/mroonga/vendor/groonga/lib/io.c +++ b/storage/mroonga/vendor/groonga/lib/io.c @@ -337,7 +337,7 @@ grn_io_create(grn_ctx *ctx, const char *path, uint32_t header_size, if ((io = GRN_MALLOCN(grn_io, 1))) { grn_io_mapinfo *maps = NULL; if ((maps = GRN_CALLOC(sizeof(grn_io_mapinfo) * max_segment))) { - grn_strncpy(io->path, PATH_MAX, path, PATH_MAX); + grn_strncpy(io->path, PATH_MAX-1, path, PATH_MAX-1); io->header = header; io->user_header = (((byte *) header) + IO_HEADER_SIZE); io->maps = maps; @@ -656,7 +656,7 @@ grn_io_open(grn_ctx *ctx, const char *path, grn_io_mode mode) if ((io = GRN_MALLOC(sizeof(grn_io)))) { grn_io_mapinfo *maps = NULL; if ((maps = GRN_CALLOC(sizeof(grn_io_mapinfo) * max_segment))) { - grn_strncpy(io->path, PATH_MAX, path, PATH_MAX); + grn_strncpy(io->path, PATH_MAX-1, path, PATH_MAX-1); io->header = header; io->user_header = (((byte *) header) + IO_HEADER_SIZE); { diff --git a/storage/mroonga/vendor/groonga/lib/ts/ts_expr_node.c b/storage/mroonga/vendor/groonga/lib/ts/ts_expr_node.c index 4ae900034bb..c38d9709052 100644 --- a/storage/mroonga/vendor/groonga/lib/ts/ts_expr_node.c +++ b/storage/mroonga/vendor/groonga/lib/ts/ts_expr_node.c @@ -1787,7 +1787,7 @@ grn_ts_expr_key_node_filter(grn_ctx *ctx, grn_ts_expr_key_node *node, grn_ts_record *out, size_t *n_out) { size_t i, count; - grn_ts_bool key; + grn_ts_bool key= 0; switch (node->table->header.type) { case GRN_TABLE_HASH_KEY: { grn_hash *hash = (grn_hash *)node->table; @@ -1831,7 +1831,7 @@ grn_ts_expr_key_node_adjust(grn_ctx *ctx, grn_ts_expr_key_node *node, grn_ts_record *io, size_t n_io) { size_t i; - grn_ts_float key; + grn_ts_float key= 0; switch (node->table->header.type) { case GRN_TABLE_HASH_KEY: { grn_hash *hash = (grn_hash *)node->table; |