summaryrefslogtreecommitdiff
path: root/sql
Commit message (Collapse)AuthorAgeFilesLines
* Merge 10.1 into 10.2Marko Mäkelä2020-06-019-304/+288
|\
| * Thread safe histograms loadingSergey Vojtovich2020-05-294-103/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously multiple threads were allowed to load histograms concurrently. There were no known problems caused by this. But given amount of data races in this code, it'd happen sooner or later. To avoid scalability bottleneck, histograms loading is protected by per-TABLE_SHARE atomic variable. Whenever histograms were loaded by preceding statement (hot-path), a scalable load-acquire check is performed. Whenever histograms have to be loaded anew, mutual exclusion for loaders is established by atomic variable. If histograms are being loaded concurrently, statement waits until load is completed. - Table_statistics::total_hist_size moved to TABLE_STATISTICS_CB: only meaningful within TABLE_SHARE (not used for collected stats). - TABLE_STATISTICS_CB::histograms_can_be_read and TABLE_STATISTICS_CB::histograms_are_read are replaced with a tri state atomic variable. - Simplified away alloc_histograms_for_table_share(). Note: there's still likely a data race if a thread attempts accessing histograms data after it failed to load it (because of concurrent load). It was there previously and goes out of the scope of this effort. One way of fixing it could be reviving TABLE::histograms_are_read and adding appropriate checks whenever it is needed. Part of MDEV-19061 - table_share used for reading statistical tables is not protected
| * Thread safe statistics loadingSergey Vojtovich2020-05-293-60/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously multiple threads were allowed to load statistics concurrently. There were no known problems caused by this. But given amount of data races in this code, it'd happen sooner or later. To avoid scalability bottleneck, statistics loading is protected by per-TABLE_SHARE atomic variable. Whenever statistics were loaded by preceding statement (hot-path), a scalable load-acquire check is performed. Whenever statistics have to be loaded anew, mutual exclusion for loaders is established by atomic variable. If statistics are being loaded concurrently, statement waits until load is completed. TABLE_STATISTICS_CB::stats_can_be_read and TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic variable. Part of MDEV-19061 - table_share used for reading statistical tables is not protected
| * Simplified away statistics_for_tables_is_needed()Sergey Vojtovich2020-05-291-81/+31
| | | | | | | | | | | | | | | | | | | | Removed redundant loops, integrated logics into the caller instead. Unified condition in read_statistics_for_tables(), less "table_share != NULL" checks, no more potential "table_share == NULL" dereferencing. Part of MDEV-19061 - table_share used for reading statistical tables is not protected
| * MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of ↵Alexander Barkov2020-05-291-2/+1
| | | | | | | | | | | | | | | | | | address ... which does not point to an object of type 'Item_func' note: object is of type 'Item_bool' (on optimized builds) In Item_nodeset_func_predicate::val_nodeset, args[1] is not necessarily an Item_func descendant. It can be Item_bool. Removing a wrong cast. It was not really needed anyway.
| * fix pre-definition for embedded server for find_user_or_anon()Anel Husakovic2020-05-281-28/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pre-definitions are allowed for non-embedded. Failur catched with: ``` cmake ../../10.1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=g++-9 -DCMAKE_C_COMPILER=gcc-9 -DWITH_EMBEDDED_SERVER=ON -DCMAKE_BUILD_TYPE=Debug -DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,PERFSCHEMA,SPIDER,SPHINX}=N -DMYSQL_MAINTAINER_MODE=ON -DNOT_FOR_DISTRIBUTION=ON ``` Alternative fix would be ``` --- a/sql/sql_acl.cc +++ b/sql/sql_acl.cc @@ -201,8 +201,10 @@ LEX_STRING current_user= { C_STRING_WITH_LEN("*current_user") }; LEX_STRING current_role= { C_STRING_WITH_LEN("*current_role") }; LEX_STRING current_user_and_current_role= { C_STRING_WITH_LEN("*current_user_and_current_role") }; +#ifndef EMBEDDED_LIBRARY class ACL_USER; static ACL_USER *find_user_or_anon(const char *host, const char *user, const char *ip); +#endif ```
| * MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not ↵Anel Husakovic2020-05-284-42/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | granted the role - `SET DEFAULT ROLE xxx [FOR yyy]` should say: "User yyy has not been granted a role xxx" if: - The current user (not the user `yyy` in the FOR clause) can see the role xxx. It can see the role if: * role exists in `mysql.roles_mappings` (traverse the graph), * If the current user has read access on `mysql.user` table - in that case, it can see all roles, granted or not. - Otherwise it should be "Invalid role specification". In other words, it should not be possible to use `SET DEFAULT ROLE` to discover whether a specific role exist or not.
| * MDEV-15152 Optimistic parallel slave doesnt cope well with START SLAVE UNTILAndrei Elkin2020-05-264-19/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The immediate bug was caused by a failure to recognize a correct position to stop the slave applier run in optimistic parallel mode. There were the following set of issues that the analysis unveil. 1 incorrect estimate for the event binlog position passed to is_until_satisfied 2 wait for workers to complete by the driver thread did not account non-group events that could be left unprocessed and thus to mix up the last executed binlog group's file and position: the file remained old and the position related to the new rotated file 3 incorrect 'slave reached file:pos' by the parallel slave report in the error log 4 relay log UNTIL missed out the parallel slave branch in is_until_satisfied. The patch addresses all of them to simplify logics of log change notification in either the master and relay-log until case. P.1 is addressed with passing the event into is_until_satisfied() for proper analisis by the function. P.2 is fixed by changes in handle_queued_pos_update(). P.4 required removing relay-log change notification by workers. Instead the driver thread updates the notion of the current relay-log fully itself with aid of introduced bool Relay_log_info::until_relay_log_names_defer. An extra print out of the requested until file:pos is arranged with --log-warning=3.
* | Fix cmake warning - custom command succeeds without creating own OUTPUT.Vladislav Vaintroub2020-05-291-0/+1
| |
* | assert(a && b); -> assert(a); assert(b);Sergei Golubchik2020-05-271-7/+10
| |
* | Revert "MDEV-12445 : Rocksdb does not shutdown worker threads and aborts in ↵Sergei Golubchik2020-05-271-1/+0
| | | | | | | | | | | | | | | | | | | | | | memleak check on server shutdown" This reverts commit 6f1f9114971c3c158e9ac97313c92a24f6554415. because it doesn't do anything now (the server doesn't check my_disable_leak_check) and it never did anything before (because without `extern` it simply created a local instance of my_disable_leak_check, did not affect server's my_disable_leak_check).
* | MDEV-21958 Query having many NOT-IN clauses running forever and causing ↵Sergei Golubchik2020-05-271-0/+1
| | | | | | | | | | | | available free memory to use completely let thd->killed to abort range optimizer
* | bugfix: use THD::main_mem_root for kill error messageSergei Golubchik2020-05-271-1/+2
| | | | | | | | | | | | cannot use the current THD::mem_root, because it can be temporarily reassigned to something with a very different life time (e.g. to TABLE::mem_root or range optimizer mem_root).
* | MDEV-22558 wrong error for invalid utf8 table commentSergei Golubchik2020-05-271-0/+19
| |
* | Fixed crash in aria recovery when using bulk insertMonty2020-05-262-7/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MDEV-20578 Got error 126 when executing undo undo_key_delete upon Aria crash recovery The crash happens in this scenario: - Table with unique keys and non unique keys - Batch insert (LOAD DATA or INSERT ... SELECT) with REPLACE - Some insert succeeds followed by duplicate key error In the above scenario the table gets corrupted. The bug was that we don't generate any undo entry for the failed insert as the whole insert can be ignored by undo. The code did however not take into account that when bulk insert is used, we would write cached keys to the file on failure and undo would wrongly ignore these. Fixed by moving the writing of the cache keys after we write the aborted-insert event to the log.
* | MDEV-15152 Optimistic parallel slave doesnt cope well with START SLAVE UNTILAndrei Elkin2020-05-264-20/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The immediate bug was caused by a failure to recognize a correct position to stop the slave applier run in optimistic parallel mode. There were the following set of issues that the analysis unveil. 1 incorrect estimate for the event binlog position passed to is_until_satisfied 2 wait for workers to complete by the driver thread did not account non-group events that could be left unprocessed and thus to mix up the last executed binlog group's file and position: the file remained old and the position related to the new rotated file 3 incorrect 'slave reached file:pos' by the parallel slave report in the error log 4 relay log UNTIL missed out the parallel slave branch in is_until_satisfied. The patch addresses all of them to simplify logics of log change notification in either the master and relay-log until case. P.1 is addressed with passing the event into is_until_satisfied() for proper analisis by the function. P.2 is fixed by changes in handle_queued_pos_update(). P.4 required removing relay-log change notification by workers. Instead the driver thread updates the notion of the current relay-log fully itself with aid of introduced bool Relay_log_info::until_relay_log_names_defer. An extra print out of the requested until file:pos is arranged with --log-warning=3.
* | MDEV-20015 Assertion `!in_use->is_error()' failed in TABLE::update_virtual_fieldAleksey Midenkov2020-05-261-3/+5
| | | | | | | | | | | | | | | | | | | | | | update_virtual_field() is called as part of index rebuild in ha_myisam::repair() (MDEV-5800) which is done on bulk INSERT finish. Assertion in update_virtual_field() was put as part of MDEV-16222 because update_virtual_field() returns in_use->is_error(). The idea: wrongly mixed semantics of error status before update_virtual_field() and the status returned by update_virtual_field(). The former can falsely influence the latter.
* | Merge 10.1 into 10.2Marko Mäkelä2020-05-261-0/+1
|\ \ | |/
| * MDEV-21495: Conditional jump or move depends on uninitialised value in ↵Varun Gupta2020-05-261-0/+1
| | | | | | | | | | | | | | sel_arg_range_seq_next Initialize the parameter PARAM::max_key_part when we iterate over the ranges to get estimates from EITS.
* | MDEV-19751 Wrong partitioning by KEY() after primary key droppedAleksey Midenkov2020-05-261-0/+22
| | | | | | | | | | | | | | | | Default (empty) field list in partitioning by KEY() clause is assigned from primary key. If primary key is changed the partitioning field list is changed as well, so repartitioning required. Not applicable to any non-primary keys as default field list may be taken only from primary key.
* | MDEV-22461: JOIN::make_aggr_tables_info(): Assertion `select_options & (1ULL ↵Varun Gupta2020-05-252-5/+12
| | | | | | | | | | | | | | | | << 17)' failed. A temporary table is needed for window function computation but if only a NAMED WINDOW SPEC is used and there is no window function, then there is no need to create a temporary table as there is no stage to compute WINDOW FUNCTION
* | MDEV-22545: my_vsnprintf behaves not as in C standardOleksandr Byelkin2020-05-243-190/+202
| | | | | | | | Added parameter %T for string which should be visibly truncated.
* | Fixed compiler failure on windowsMonty2020-05-231-1/+1
| |
* | Fixed deadlock with LOCK TABLES and ALTER TABLEMonty2020-05-2312-34/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MDEV-21398 Deadlock (server hang) or assertion failure in Diagnostics_area::set_error_status upon ALTER under lock This failure could only happen if one locked the same table multiple times and then did an ALTER TABLE on the table. Major change is to change all instances of table->m_needs_reopen= true; to table->mark_table_for_reopen(); The main fix for the problem was to ensure that we mark all instances of the table in the locked_table_list and when we reopen the tables, we first close all tables before reopening and locking them. Other things: - Don't call thd->locked_tables_list.reopen_tables if there are no tables marked for reopen. (performance)
* | Merge remote-tracking branch 'origin/10.1' into 10.2Alexander Barkov2020-05-222-59/+131
|\ \ | |/
| * MDEV-22111 ERROR 1064 & 1033 and SIGSEGV on CREATE TABLE w/ various charsets ↵Alexander Barkov2020-05-222-60/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | on 10.4/5 optimized builds | Assertion `(uint) (table_check_constraints - share->check_constraints) == (uint) (share->table_check_constraints - share->field_check_constraints)' failed The code incorrectly assumed in multiple places that TYPELIB values cannot have 0x00 bytes inside. In fact they can: CREATE TABLE t1 (a ENUM(0x61, 0x0062) CHARACTER SET BINARY); Note, the TYPELIB value encoding used in FRM is ambiguous about 0x00. So this fix is partial. It fixes 0x00 bytes in many (but not all) places: - In the middle or in the end of a value: CREATE TABLE t1 (a ENUM(0x6100) ...); CREATE TABLE t1 (a ENUM(0x610062) ...); - In the beginning of the first value: CREATE TABLE t1 (a ENUM(0x0061)); CREATE TABLE t1 (a ENUM(0x0061), b ENUM('b')); - In the beginning of the second (and following) value of the *last* ENUM/SET in the table: CREATE TABLE t1 (a ENUM('a',0x0061)); CREATE TABLE t1 (a ENUM('a'), b ENUM('b',0x0061)); However, it does not fix 0x00 when: - 0x00 byte is in the beginning of a value of a non-last ENUM/SET causes an error: CREATE TABLE t1 (a ENUM('a',0x0061), b ENUM('b')); ERROR 1033 (HY000): Incorrect information in file: './test/t1.frm' This is an ambuguous case and will be fixed separately. We need a new TYPELIB encoding to fix this. Details: - unireg.cc The function pack_header() incorrectly used strlen() to detect a TYPELIB value length. Adding a new function typelib_values_packed_length() which uses TYPELIB::type_lengths[n] to detect the n-th value length, and reusing the new function in pack_header() and packed_fields_length() - table.cc fix_type_pointers() assumed in multiple places that values cannot have 0x00 inside and used strlen(TYPELIB::type_names[n]) to set the corresponding TYPELIB::type_lengths[n]. Also, fix_type_pointers() did not check the encoded data for consistency. Rewriting fix_type_pointers() code to populate TYPELIB::type_names[n] and TYPELIB::type_lengths[n] at the same time, so no additional loop with strlen() is needed any more. Adding many data consistency tests. Fixing the main loop in fix_type_pointers() to use memchr() instead of strchr() to handle 0x00 properly. Fixing create_key_infos() to return the result in a LEX_STRING rather that in a char*.
* | MDEV-22451: SIGSEGV in __memmove_avx_unaligned_erms/memcpy from _my_b_write ↵Sujatha2020-05-201-1/+3
|\ \ | |/ | | | | | | | | on CREATE after RESET MASTER Merge branch '10.1' into 10.2
| * MDEV-22451: SIGSEGV in __memmove_avx_unaligned_erms/memcpy from _my_b_write ↵Sujatha2020-05-201-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | on CREATE after RESET MASTER Analysis: ======== RESET MASTER TO # command deletes all binary log files listed in the index file, resets the binary log index file to be empty, and creates a new binary log with number #. When the user provided binary log number is greater than the max allowed value '2147483647' server fails to generate a new binary log. The RESET MASTER statement marks the binlog closure status as 'LOG_CLOSE_TO_BE_OPENED' and exits. Statements which follow RESET MASTER try to write to binary log they find the log_state != LOG_CLOSED and proceed to write to binary log cache and it results in crash. Fix: === During MYSQL_BIN_LOG open, if generation of new binary log name fails then the "log_state" needs to be marked as "LOG_CLOSED". With this further statements will find binary log as closed and they will skip writing to the binary log.
* | MDEV-22591 Debug build crashes on EXECUTE IMMEDIATE '... WHERE ?' USING IGNOREAlexander Barkov2020-05-191-1/+0
| | | | | | | | | | | | | | | | | | | | | | Removing a wrong DBUG_ASSERT: When Item_param gets "unfixed" in cleanup(), its "fixed" gets assigned to false, while item_item keeps the value. So the assert was wrong. Perhaps, instead of removing the assert, it was possible to reset item_type to NO_VALUE in cleanup. But this is not very important: it's implemented in 10.4 in a better way: Item_param::is_fixed() always returns true and it does not need to be "unfixed".
* | Merge 10.1 into 10.2Marko Mäkelä2020-05-191-1/+2
|\ \ | |/
| * MDEV-22520 Assertion `gathered_length == thd->lex->comment.length` failed in ↵Andrei Elkin2020-05-181-1/+2
| | | | | | | | | | | | | | | | | | | | | | binlog_defragment The assert was caused by early cleanup of a user variable participant in BINLOG @var,@var where it plays twice. That was unexpected by the base code to clear its value prematurely. Fixed with relocating the user var destruction after operations with its value is over.
* | Merge remote-tracking branch 'origin/10.1' into 10.2Alexander Barkov2020-05-162-14/+36
|\ \ | |/ | | | | Also, adding 10.2 related changes for MDEV-22579
| * MDEV-22579 No error when inserting DEFAULT(non_virtual_column) into a ↵Alexander Barkov2020-05-152-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | virtual column The code erroneously allowed both: INSERT INTO t1 (vcol) VALUES (DEFAULT); INSERT INTO t1 (vcol) VALUES (DEFAULT(non_virtual_column)); The former is OK, but the latter is not. Adding a new virtual method in Item: virtual bool vcol_assignment_allowed_value() const { return false; } Item_null, Item_param and Item_default_value override it. Item_default_value overrides it in the way to: - allow DEFAULT - disallow DEFAULT(col)
* | Merge 10.1 into 10.2Marko Mäkelä2020-05-151-3/+6
|\ \ | |/
| * MDEV-22498: SIGSEGV in Bitmap<64u>::merge on SELECTVarun Gupta2020-05-141-3/+6
| | | | | | | | | | | | | | | | For the case when the optimizer does the IN-EXISTS transformation, the equality condition is injected in the WHERE OR HAVING clause of the subquery. If the select list of the subquery has a reference to the parent select make sure to use the reference and not the original item.
* | Amend af784385b4a2b286000fa2c658e34283fe7bba60: Avoid vtable overheadMarko Mäkelä2020-05-152-4/+13
| | | | | | | | | | | | When neither MSAN nor Valgrind are enabled, declare Field::mark_unused_memory_as_defined() as an empty inline function, instead of declaring it as a virtual function.
* | Fix for using uninitialized memoryMonty2020-05-153-0/+16
| | | | | | | | | | | | | | | | | | MDEV-22073 MSAN use-of-uninitialized-value in collect_statistics_for_table() Other things: innodb.analyze_table was changed to mainly test statistic collection. Was discussed with Marko.
* | Fixed bugs found by valgrindMonty2020-05-152-1/+3
| | | | | | | | | | | | Other things: - Removed innodb_encryption_tables.test from valgrind as it takes a REALLY long time
* | Merge remote-tracking branch 'origin/10.1' into 10.2Alexander Barkov2020-05-141-0/+2
|\ \ | |/
| * MDEV-22502 MDB crashes in CREATE TABLE AS SELECT when the precision of ↵Alexander Barkov2020-05-141-0/+2
| | | | | | | | | | | | | | returning type = 0. TRUNCATE(decimal_5_5) erroneously tried to create a DECIMAL(0,0) column. Creating a DECIMAL(1,0) column instead.
* | Merge remote-tracking branch 'origin/10.1' into 10.2Alexander Barkov2020-05-141-0/+3
|\ \ | |/
| * MDEV-22503 MDB limits DECIMAL column precision to 16 doing CTAS with ↵Alexander Barkov2020-05-141-0/+3
| | | | | | | | | | | | | | | | floor/ceil over DECIMAL(X,Y) where X > 16 The DECIMAL data type branch in Item_func_int_val::fix_length_and_dec() incorrectly used DOUBLE-style length calculation, which resulted in a smaller data type than the actual result of FLOOR()/CEIL() needs.
* | Ensure that auto_increment fields are marked properly on updateMonty2020-05-131-0/+6
| | | | | | | | | | | | MDEV-19622 Assertion failures in ha_partition::set_auto_increment_if_higher upon UPDATE on Aria table
* | MDEV-22529 thd_query_safe() isn’t, causing InnoDB to hangMarko Mäkelä2020-05-121-4/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The function thd_query_safe() is used in the implementation of the following INFORMATION_SCHEMA views: information_schema.innodb_trx information_schema.innodb_locks information_schema.innodb_lock_waits information_schema.rocksdb_trx The implementation of the InnoDB views is in trx_i_s_common_fill_table(). This function invokes trx_i_s_possibly_fetch_data_into_cache(), which will acquire lock_sys->mutex and trx_sys->mutex in order to protect the set of active transactions and explicit locks. While holding those mutexes, it will traverse the collection of InnoDB transactions. For each transaction, thd_query_safe() will be invoked. When called via trx_i_s_common_fill_table(), thd_query_safe() is acquiring THD::LOCK_thd_data while holding the InnoDB locks. This will cause a deadlock with THD::awake() (such as executing KILL QUERY), because THD::awake() could invoke lock_trx_handle_wait(), which attempts to acquire lock_sys->mutex while already holding THD::lock_thd_data. thd_query_safe(): Invoke mysql_mutex_trylock() instead of mysql_mutex_lock(). Return the empty string if the mutex cannot be acquired without waiting.
* | Merge branch '10.2-release' into 10.2Oleksandr Byelkin2020-05-113-1/+14
|\ \
| * \ Merge branch '10.1' into 10.2mariadb-10.2.32Oleksandr Byelkin2020-05-082-0/+12
| |\ \ | | |/
| | * MDEV-22180 Planner opens unnecessary tables when updated table is referenced ↵Sergei Golubchik2020-05-081-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | by foreign keys under LOCK TABLES we still have to open everything, otherwise DML prelocking will try to take an MDL on a table that wasn't in the LOCK TABLES list.
| | * MDEV-22180 Planner opens unnecessary tables when updated table is referenced ↵Sergei Golubchik2020-05-061-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | by foreign keys only MDL-prelock but do not open FK child tables for read-only (RESTRICT) FK actions. Tables still needs to be opened for CASCADE actions, see 9180e8666b8
| * | MDEV-22504: Session tracking return incorrectly long traking dataOleksandr Byelkin2020-05-081-1/+2
| | | | | | | | | | | | When we move data we should fix buffer string length
* | | MDEV-13266: Race condition in ANALYZE TABLE / statistics collectionVarun Gupta2020-05-051-25/+21
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixing a race condition while collecting the engine independent statistics. Thread1> 1) start running "ANALYZE TABLE t PERISTENT FOR COLUMNS (..) INDEXES ($list)" 2) Walk through $list and save it in TABLE::keys_in_use_for_query 3) Close/re-open tables Thread2> 1) Make some use of table t. This involves taking table t from the table cache, and putting it back (with TABLE::keys_in_use_for_query reset to 0) Thread1> continue collecting EITS stats. Since TABLE::keys_in_use_for_query is set to 0 we will not collect statistics for indexes in $list.
View Lorry Controller Status