summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-23299: Bad data in mysql.func can cause crashbb-10.5-MDEV-23299-danielblack-pr-1643zbdba2020-07-3118-24/+76
| | | | | | | | | | | | | | | | | | | | | udf_init() can crash when the name field is empty. We correct this by applying CHECK column constraints on columns in the mysql.func tables. The constrains are as follows: * name cannot be an empty string; * ret, the return type can be; * STRING_RESULT(0), * REAL_RESULT(1), * INT_RESULT(2), * and DECIMAL_RESULT(4); * but not ROW_RESULT(3) or TIME_RESULT(5). * dl, cannot be empty string Upgrades ensure that any invalid functions are deleted. closes #1643
* Deb: Make RocksDB plugin depend on python3 as myrocks_hotbackup needs itOtto Kekäläinen2020-07-302-2/+1
| | | | | This fixes the Lintian error and thus the Lintian override can also be removed.
* Deb: Proper DH_ and DEB_ flag use in debian/rulesOtto Kekäläinen2020-07-301-5/+9
| | | | | | | - DEB_BUILD_HARDENING is only used with hardening-wrapper which is deprecated in Debian, so remove it - The word 'terse' should be checked in DEB_BUILD_OPTIONS and verbosity controlled by it
* Deb: Stop suggesting tinyca, upstream project does not exist anymoreOtto Kekäläinen2020-07-301-2/+1
| | | | | | | TinyCA has not been updated since 2016 and was removed from Debian in 2019 and the dev site https://tinyca.alioth.debian.org/ is offline. See https://tracker.debian.org/pkg/tinyca
* Deb: Simplify and unify autobake-deb.shOtto Kekäläinen2020-07-301-54/+24
| | | | | | | | | | | | | | | | | - Remove excess build flag from debian/rules that does nothing, introduced wrongly in commit 7cbde2d0a29e214afba4d5d30af890002fbf7db2. Instead implement the embedded server build skipping on Travis-CI correctly. - Simplify structure by doing all Travis-CI slimdown in one step. - Remove unnecessary -e from sed, as it does nothing. When regex is needed, use -r. Move -i to last so it is close to the file name it has an argument. - Remove backwards compat checks that are no longer relevant as neither Debian Jessie (was before Stretch) nor Ubuntu Trusty (before Xenial) are supported nor built for anymore. For example the GCC 4.8 check if not relevant anymore, since Debian Jessie already has 4.9 and Ubuntu Xenial has 5.3 and there is no GCC < 4.8 around anymore. - Skip building ColumnStore on both Travis-CI and Gitlab-CI as it is way too slow (time) and big (disk space) to pass.
* Deb: Cleanup and documentOtto Kekäläinen2020-07-3012-165/+112
| | | | | | | | | | | | | | - Remove unnecessary unused files - Remove duplicate encryption configuration sample from sources and re-use the identical file in RPM directory instead - Clean away harmful "default-character-set = utf8mb4" from client config as it is unnecassary (server enforces utf8mb4 anyway by default) and could cause issues with mysqlbinlog and other tools (MDEV-22981). - Update S3 plugin description to be long enough - Remove trailing whitespace from support-files and Debian packaging. - Clean away fixed Lintian issues - Clean away temporary Salsa-CI fixes now that 10.5.4 is out and is fixed - Apply wrap-and-sort -a -v
* Deb: On upgrades, stop both mysqld and mariadbd for backwards compatOtto Kekäläinen2020-07-302-0/+6
|
* Use mktemp instead of deprecated tempfileOtto Kekäläinen2020-07-301-1/+1
| | | | | This fixes the warning emitted during `/etc/init.d/mariadb restart`: WARNING: tempfile is deprecated; consider using mktemp instead.
* Fix spelling errorsOtto Kekäläinen2020-07-304-5/+5
|
* MDEV-22110 InnoDB unnecessarily writes unmodified pagesMarko Mäkelä2020-07-288-141/+114
| | | | | | | | | | | | | | | | | | | | At least since commit 6a7be48b1b0b4107bf6991240ae69fcec0b7189a InnoDB appears to be invoking buf_flush_note_modification() on pages that were exclusively latched but not modified in a mini-transaction. MTR_MEMO_MODIFY, mtr_t::modify(): Define not only in debug code, but also in release code. We will set the MTR_MEMO_MODIFY flag on the earliest mtr_t::m_memo entry that we find. MTR_LOG_NONE: Only use this mode in cases where the previous mode will be restored before anything is modified in the mini-transaction. MTR_MEMO_PAGE_X_MODIFY, MTR_MEMO_PAGE_SX_MODIFY: The allowed flag combinations that include MTR_MEMO_MODIFY. ReleaseBlocks: Only invoke buf_flush_note_modification() on those buffer pool blocks on which mtr_t::set_modified() and mtr_t::modify() were invoked.
* MDEV-23304 Insert into TEMPORARY TABLE fails to invoke mtr_t::set_modified()Marko Mäkelä2020-07-281-0/+6
| | | | | | | | | | | In MDEV-21724 or possibly already in MDEV-12353 a bug was introduced to the handling of temporary tables. Whether or not redo log will be written, mtr_t::set_modified() must be invoked to register any changes to a page. page_cur_insert_rec_low(): Invoke mtr_t::set_modified() also when skipping the redo log write.
* MCOL-4166 The solution to the minor upgrade issueRoman Nozdrin2020-07-284-4/+2
| | | | | | DEB package doesn't need expect and libreadline as dependencies RPM packages now depends on jemalloc again
* MDEV-23176: s3 curl library path fixDaniel Black2020-07-281-0/+1
| | | | | | | On FreeBSD (and possibly elsewhere) the FindPackage(curl) may have found the curl library in a non-standard directory. We use the exposed path as a library path to correct linking.
* MDEV-23175: my_timer_milliseconds clock_gettime for multiple platfomrsDaniel Black2020-07-282-14/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Small postfix to MDEV-23175 to ensure faster option on FreeBSD and compatibility to Solaris that isn't high resolution. ftime is left as a backup in case an implementation doesn't contain any of these clocks. FreeBSD $ ./unittest/mysys/my_rdtsc-t 1..11 # ----- Routine --------------- # myt.cycles.routine : 5 # myt.nanoseconds.routine : 11 # myt.microseconds.routine : 13 # myt.milliseconds.routine : 11 # myt.ticks.routine : 17 # ----- Frequency ------------- # myt.cycles.frequency : 3610295566 # myt.nanoseconds.frequency : 1000000000 # myt.microseconds.frequency : 1000000 # myt.milliseconds.frequency : 899 # myt.ticks.frequency : 136 # ----- Resolution ------------ # myt.cycles.resolution : 1 # myt.nanoseconds.resolution : 1 # myt.microseconds.resolution : 1 # myt.milliseconds.resolution : 7 # myt.ticks.resolution : 1 # ----- Overhead -------------- # myt.cycles.overhead : 26 # myt.nanoseconds.overhead : 19140 # myt.microseconds.overhead : 19036 # myt.milliseconds.overhead : 578 # myt.ticks.overhead : 21544 ok 1 - my_timer_init() did not crash ok 2 - The cycle timer is strictly increasing ok 3 - The cycle timer is implemented ok 4 - The nanosecond timer is increasing ok 5 - The nanosecond timer is implemented ok 6 - The microsecond timer is increasing ok 7 - The microsecond timer is implemented ok 8 - The millisecond timer is increasing ok 9 - The millisecond timer is implemented ok 10 - The tick timer is increasing ok 11 - The tick timer is implemented
* mtr: mysqld--help-aria test on OSX failDaniel Black2020-07-281-1/+1
| | | | | | | | | | | | | | | | | | Previous test failure: --- /Users/travis/build/grooverdan/mariadb-server/mysql-test/main/mysqld--help-aria.result 2020-04-29 03:44:40.000000000 +0000 +++ /Users/travis/build/grooverdan/mariadb-server/mysql-test/main/mysqld--help-aria.reject 2020-04-29 04:19:13.000000000 +0000 @@ -1,3 +1,4 @@ +[Warning] Setting lower_case_table_names=2 because file system for /Users/travis/build/grooverdan/mariadb-server/mysql-test/var/4/mariadbd.1/data/ is case insensitive [ERROR] mariadbd: Can't lock aria aria_log_control for exclusive use, error: #. Will retry for 0 seconds [ERROR] Plugin 'Aria' init function returned error. [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed. @@ -9,4 +10,5 @@ # # Check with existing directory # +[Warning] Setting lower_case_table_names=2 because file system for /Users/travis/build/grooverdan/mariadb-server/mysql-test/var/tmp/4/help/ is case insensitive [Warning] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some options may be missing from the help text
* MDEV-22621: perfschema add FreeBSD include header for pthread_getthreadid_npDaniel Black2020-07-281-0/+4
|
* MDEV-23229 Read of Uninitialized memory during buffer pool resizingThirunarayanan Balathandayuthapani2020-07-241-3/+2
| | | | | | | | | | | commit b1ab211dee599eabd9a5b886fafa3adea29ae041 (MDEV-15053) introduced the code to unfix the block earlier in buf_block_t::unfix(). After unfixing the block, InnoDB can withdraw the block from the buffer pool and deallocate it while doing buffer pool resizing. So subsequent assert could leads to uninitialized memory access of block. buf_block_t::unfix(): Unfix the block after checking the assert.
* MDEV-23216: LONGTEXT column with collation doesn't sortVarun Gupta2020-07-234-5/+40
| | | | | | | An overflow was happening with LONGTEXT columns, when the length was converted to the length in the strxfrm form (mem-comparable keys). Introduced a function to truncate the length to the max_sort_length before calculating the length of the strxfrm form.
* Fix test casesJan Lindström2020-07-232-0/+12
| | | | | | Changes to be committed: modified: mysql-test/suite/galera_3nodes/t/GCF-354.test modified: mysql-test/suite/galera_3nodes/t/inconsistency_shutdown.test
* thd->m_transaction_psi was not properly cleared for new connectionsMonty2020-07-234-3/+12
| | | | | | | | | | | | This happend when using XA transactions. I also added some extra asserts to ensure that m_transactions are properly cleared. Other things: - Removed set_time() from THD::init_for_queries() as dispatch_command() is already doing that. - Removed duplicate init_for_queries() from prepare_new_connection_state(). The init_for_queries() functions should only be called once per connection.
* MDEV-7947 strcmp() takes 0.37% in OLTP ROMonty2020-07-2330-245/+386
| | | | | | | | | | | | | | | | | | | This patch ensures that all identical character sets shares the same cs->csname. This allows us to replace strcmp() in my_charset_same() with comparisons of pointers. This fixes a long standing performance issue that could cause as strcmp() for every item sent trough the protocol class to the end user. One consequence of this patch is that we don't allow one to add a character definition in the Index.xml file that changes the csname of an existing character set. This is by design as changing character set names of existing ones is extremely dangerous, especially as some storage engines just records character set numbers. As we now have a hash over character set's csname, we can in the future use that for faster access to a specific character set. This could be done by changing the hash to non unique and use the hash to find the next character set with same csname.
* Fixed wrong free in comp_errMonty2020-07-231-1/+4
|
* Disable maria.max_length when using valgrind (too slow)Monty2020-07-231-0/+2
|
* Fixed removed warning from valgrind in Protocol::store_strMonty2020-07-231-2/+3
| | | | | The problem was that field_count is not initialized for the Protocol variable used when printing metadata.
* Remove String::lex_string() and String::lex_cstring()Monty2020-07-2318-72/+85
| | | | | | | | | - Better to use 'String *' directly. - Added String::get_value(LEX_STRING*) for the few cases where we want to convert a String to LEX_CSTRING. Other things: - Use StringBuffer for some functions to avoid mallocs
* Use larger buffer when reading binary and relay logsMonty2020-07-233-3/+11
| | | | - Should speed up replication
* Clean up Item_uint() & Item_int()Monty2020-07-233-19/+5
| | | | | - Removed val_str() and print() as these are handled by Item_int() - Use local StringBuffer for Item_int::print() to avoid mallocs
* Merge 10.4 into 10.5Marko Mäkelä2020-07-2313-58/+164
|\
| * MDEV-22778 Slow InnoDB shutdown on large instanceMarko Mäkelä2020-07-2310-58/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting with MDEV-17441 we would no longer have os_once, and we would always initialize zip_pad_info_t::mutex and dict_table_t::autoinc_mutex, even for tables are not in ROW_FORMAT=COMPRESSED nor include any AUTO_INCREMENT column. mutex_free() on those unnecessary objects would make shutdown very slow compared to older versions. Let us use std::mutex for those two mutexes, to reduce the overhead. The critical sections protected by these mutexes is very small, and therefore contention or the need for any instrumentation should be unlikely.
| * MDEV-21997 Server crashes in LEX::create_item_ident_sp upon use of unknown ↵Oleksandr Byelkin2020-07-223-0/+128
| | | | | | | | | | | | identifier If there is no current_select and variable is not found among SP variables it can be only an error.
| * MDEV-23254 Replace FSP_FLAGS_HAS_PAGE_COMPRESSION with ↵Thirunarayanan Balathandayuthapani2020-07-221-3/+3
| | | | | | | | | | | | | | | | fil_space_t::is_compressed InnoDB should replace FSP_FLAGS_HAS_PAGE_COMPRESSION check with fil_space_t::is_compressed(). fil_space_t::is_compressed() checks for both non full crc32 and crc32 format.
| * Fix regex on test.Jan Lindström2020-07-221-2/+2
| |
* | MDEV-23252 Assertion failure 'req_type.is_dblwr_recover() || err == ↵Thirunarayanan Balathandayuthapani2020-07-223-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | DB_SUCCESS' for page_compressed tables - This issue is caused by a5584b13d1e04f38b843602413669591aa65c359 (MDEV-15528). os_file_punch_hole() is added to fil_io() in MDEV-15528. But it fails to handle failure of os_file_punch_hole(). InnoDB should handle the DB_IO_NO_PUNCH_HOLE error and silently transform to DB_SUCCESS. InnoDB should set the punch hole flag correctly when tablespace is loaded fil_node_t::read_page0(): Set the punch hole flag when tablespace is loaded fil_io(): Handle the DB_IO_NO_PUNCH_HOLE error buf_flush_free_pages(): Checks the punch hole condition earlier using tablespace punch hole flag
* | MDEV-23254 Replace FSP_FLAGS_HAS_PAGE_COMPRESSION with ↵Thirunarayanan Balathandayuthapani2020-07-221-3/+3
| | | | | | | | | | | | | | | | fil_space_t::is_compressed InnoDB should replace FSP_FLAGS_HAS_PAGE_COMPRESSION check with fil_space_t::is_compressed(). fil_space_t::is_compressed() checks for both non full crc32 and crc32 format.
* | Fix regex on test.Jan Lindström2020-07-221-2/+2
| |
* | MDEV-21910 Deadlock between BF abort and manual KILL commandbb-10.5-MDEV-21910sjaakola2020-07-2216-216/+441
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When high priority replication slave applier encounters lock conflict in innodb, it will force the conflicting lock holder transaction (victim) to rollback. This is a must in multi-master sychronous replication model to avoid cluster lock-up. This high priority victim abort (aka "brute force" (BF) abort), is started from innodb lock manager while holding the victim's transaction's (trx) mutex. Depending on the execution state of the victim transaction, it may happen that the BF abort will call for THD::awake() to wake up the victim transaction for the rollback. Now, if BF abort requires THD::awake() to be called, then the applier thread executed locking protocol of: victim trx mutex -> victim THD::LOCK_thd_data If, at the same time another DBMS super user issues KILL command to abort the same victim, it will execute locking protocol of: victim THD::LOCK_thd_data -> victim trx mutex. These two locking protocol acquire mutexes in opposite order, hence unresolvable mutex locking deadlock may occur. The fix in this commit adds THD::wsrep_aborter flag to synchronize who can kill the victim This flag is set both when BF is called for from innodb and by KILL command. Either path of victim killing will bail out if victim's wsrep_killed is already set to avoid mutex conflicts with the other aborter execution. THD::wsrep_aborter records the aborter THD's ID. This is needed to preserve the right to kill the victim from different locations for the same aborter thread. It is also good error logging, to see who is reponsible for the abort. A new test case was added in galera.galera_bf_kill_debug.test for scenario where wsrep applier thread and manual KILL command try to kill same idle victim
* | Merge 10.4 into 10.5Marko Mäkelä2020-07-2124-182/+647
|\ \ | |/
| * Merge 10.3 into 10.4Marko Mäkelä2020-07-2113-93/+295
| |\
| | * MDEV-15880: ASAN heap-use-after-free with innodb_evict_tables_on_commit_debugMarko Mäkelä2020-07-211-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | trx_update_mod_tables_timestamp(): When implementing innodb_evict_tables_on_commit_debug, do not evict tables on which transactional locks exist. This debug variable was broken since its introduction in commit 947b0b5722117350c83656ee0b23502be59b7d2b.
| | * MDEV-16929 Assertion ... in close_thread_tables upon killing connectionMonty2020-07-213-0/+36
| | | | | | | | | | | | | | | Problem was that the code didn't handle a transaction created in innodb as part of a failed mysql_lock_tables()
| | * MDEV-20661 Virtual fields are not recalculated on system fields value assignmentAleksey Midenkov2020-07-209-57/+169
| | | | | | | | | | | | | | | | | | | | | Fix stale virtual field value in 4 cases: when virtual field depends on row_start/row_end in timestamp/trx_id versioned table. row_start dep is recalculated in vers_update_fields() (SQL and InnoDB layer). row_end dep is recalculated on history row insert.
| | * MDEV-22061 InnoDB: Assertion of missing row in sec index row_start upon ↵Aleksey Midenkov2020-07-205-34/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | REPLACE on a system-versioned table make_versioned_helper() appended new update field unconditionally while it should check if this field already exists in update vector. Misc renames to conform versioning prefix. vers_update_fields() name conforms with sql layer TABLE::vers_update_fields().
| * | MDEV-21953 deadlock between BACKUP STAGE BLOCK_COMMIT and parallel repl.Monty2020-07-2111-59/+210
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The issue was: T1, a parallel slave worker thread, is waiting for another worker thread to commit. While waiting, it has the MDL_BACKUP_COMMIT lock. T2, working for mariabackup, is doing BACKUP STAGE BLOCK_COMMIT and blocks all commits. This causes a deadlock as the thread T1 is waiting for can't commit. Fixed by moving locking of MDL_BACKUP_COMMIT from ha_commit_trans() to commit_one_phase_2() Other things: - Added a new argument to ha_comit_one_phase() to signal if the transaction was a write transaction. - Ensured that ha_maria::implicit_commit() is always called under MDL_BACKUP_COMMIT. This code is not needed in 10.5 - Ensure that MDL_Request values 'type' and 'ticket' are always initialized. This makes it easier to check the state of the MDL_Request. - Moved thd->store_globals() earlier in handle_rpl_parallel_thread() as thd->init_for_queries() could use a MDL that could crash if store_globals where not called. - Don't call ha_enable_transactions() in THD::init_for_queries() as this is both slow (uses MDL locks) and not needed.
| * | MDEV-22899 Assertion `field->col->is_binary() || field->prefix_len % ↵Eugene Kosov2020-07-203-26/+172
| | | | | | | | | | | | | | | | | | | | | | | | | | | field->col->mbmaxlen == 0' failed in dict_index_add_to_cache is_part_of_a_key(): detect is TEXT field is a part of some key ha_innobase::can_convert_blob(): now correctly detect whether our blob is a part of some key. Previously the check didn't work in some cases.
* | | MDEV-22970 Possible corruption of page_compressed tables, orThirunarayanan Balathandayuthapani2020-07-209-19/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when scrubbing is enabled buf_read_recv_pages(): Ignore the page to read if it is already present in the freed ranges. store_freed_or_init_rec(): Store the ranges only if scrubbing is enabled or page compressed tablespace. recv_init_crash_recovery_space(): Add the freed range only when scrubbing or page compressed tablespace. range_set::contains(): Search the value is present in ranges. range_set::remove_if_exists(): Remove the value if exist in ranges. mtr_t::init(): Handles the scenario that mini-transaction may allocate a page that had just been freed. recv_sys_t::parse(): Note down the FREE and INIT redo log irrespective of STORE value. Removed innodb_tablespaces_scrubbing from test case
* | | Merge 10.4 into 10.5Marko Mäkelä2020-07-2029-145/+265
|\ \ \ | |/ /
| * | Merge 10.3 into 10.4Marko Mäkelä2020-07-2025-115/+195
| |\ \ | | |/
| | * Merge 10.2 into 10.3Marko Mäkelä2020-07-2021-122/+166
| | |\
| | | * Merge 10.1 into 10.2Marko Mäkelä2020-07-2036-283/+358
| | | |\
| | | | * MDEV-23190 InnoDB data file extension is not crash-safeMarko Mäkelä2020-07-2024-197/+313
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When InnoDB is extending a data file, it is updating the FSP_SIZE field in the first page of the data file. In commit 8451e09073e8b1a300f177d74a9e3a530776640a (MDEV-11556) we removed a work-around for this bug and made recovery stricter, by making it track changes to FSP_SIZE via redo log records, and extend the data files before any changes are being applied to them. It turns out that the function fsp_fill_free_list() is not crash-safe with respect to this when it is initializing the change buffer bitmap page (page 1, or generally, N*innodb_page_size+1). It uses a separate mini-transaction that is committed (and will be written to the redo log file) before the mini-transaction that actually extended the data file. Hence, recovery can observe a reference to a page that is beyond the current end of the data file. fsp_fill_free_list(): Initialize the change buffer bitmap page in the same mini-transaction. The rest of the changes are fixing a bug that the use of the separate mini-transaction was attempting to work around. Namely, we must ensure that no other thread will access the change buffer bitmap page before our mini-transaction has been committed and all page latches have been released. That is, for read-ahead as well as neighbour flushing, we must avoid accessing pages that might not yet be durably part of the tablespace. fil_space_t::committed_size: The size of the tablespace as persisted by mtr_commit(). fil_space_t::max_page_number_for_io(): Limit the highest page number for I/O batches to committed_size. MTR_MEMO_SPACE_X_LOCK: Replaces MTR_MEMO_X_LOCK for fil_space_t::latch. mtr_x_space_lock(): Replaces mtr_x_lock() for fil_space_t::latch. mtr_memo_slot_release_func(): When releasing MTR_MEMO_SPACE_X_LOCK, copy space->size to space->committed_size. In this way, read-ahead or flushing will never be invoked on pages that do not yet exist according to FSP_SIZE.