summaryrefslogtreecommitdiff
path: root/storage
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-19794 Spider crash with XAbb-10.4-MDEV-19794Kentoku SHIBA2020-08-136-4/+191
|
* Merge 10.3 into 10.4Marko Mäkelä2020-08-106-194/+121
|\
| * Merge 10.2 into 10.3Marko Mäkelä2020-08-106-194/+121
| |\
| | * Merge 10.1 into 10.2Marko Mäkelä2020-08-108-390/+218
| | |\
| | | * MDEV-16115 Hang after reducing innodb_encryption_threadsMarko Mäkelä2020-08-106-374/+222
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The test encryption.create_or_replace would occasionally fail, because some fil_space_t::n_pending_ops would never be decremented. fil_crypt_find_space_to_rotate(): If rotate_thread_t::should_shutdown() holds due to innodb_encryption_threads having been reduced, do release the reference. fil_space_remove_from_keyrotation(), fil_space_next(): Declare the functions static, simplify a little, and define in the same compilation unit with the only caller, fil_crypt_find_space_to_rotate(). fil_crypt_key_mutex: Remove (unused).
| | * | Merge mariadb-10.2.33Marko Mäkelä2020-08-1042-101/+95
| | |\ \
| | * | | MDEV-23101 fixup: Remove redundant codeMarko Mäkelä2020-08-041-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lock_rec_has_to_wait_in_queue(): Remove an obviously redundant assertion that was added in commit a8ec45863b958757da61af3b2ce0a38b0a79d92c and also enclose a Galera-specific condition in #ifdef WITH_WSREP.
| | * | | MDEV-23101: SIGSEGV in lock_rec_unlock() when Galera is enabledJan Lindström2020-08-031-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lock_rec_has_to_wait wsrep_kill_victim lock_rec_create_low lock_rec_add_to_queue DeadlockChecker::select_victim() THD can't change from normal transaction to BF (brute force) transaction here, thus there is no need to syncronize access in wsrep_thd_is_BF function. lock_rec_has_to_wait_in_queue Add condition that lock is not NULL and add assertions if we are in strong state.
* | | | | Merge branch '10.3' into 10.4mariadb-10.4.14Sergei Golubchik2020-08-0641-99/+94
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch '10.2' into 10.3mariadb-10.3.24Sergei Golubchik2020-08-0641-99/+94
| |\ \ \ \ | | | |/ / | | |/| |
| | * | | Merge branch '10.1' into 10.2mariadb-10.2.33Sergei Golubchik2020-08-0642-101/+95
| | |\ \ \ | | | |/ / | | |/| / | | | |/
| | | * Merge branch 'merge/merge-tokudb-5.6' into 10.1Sergei Golubchik2020-08-0542-100/+95
| | | |\
| | | | * 5.6.49-89.0Sergei Golubchik2020-08-0447-148/+125
| | | | |
* | | | | MDEV-23345 Assertion not_redundant() == old.not_redundant()Marko Mäkelä2020-08-041-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In commit fd9ca2a742abe2e91b2b77e70915dec7bd3cd7e1 (MDEV-23295) we added a debug assertion, which caught a similar bug. prepare_inplace_alter_table_dict(): If we had promised that ALGORITHM=INPLACE or ALGORITHM=NOCOPY is supported, we must preserve the ROW_FORMAT.
* | | | | Merge branch '10.3' into 10.4Oleksandr Byelkin2020-08-0326-132/+524
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch '10.2' into 10.3Oleksandr Byelkin2020-08-0325-131/+523
| |\ \ \ \
| | * \ \ \ Merge branch '10.1' into 10.2Oleksandr Byelkin2020-08-031-2/+1
| | |\ \ \ \ | | | |/ / / | | |/| / / | | | |/ /
| | | * | compilation error on bintar-centos6-amd64-debugSergei Golubchik2020-08-031-2/+1
| | | | | | | | | | | | | | | | | | | | /home/buildbot/buildbot/build/storage/xtradb/mtr/mtr0mtr.cc:97:37: error: invalid access to non-static data member ‘fil_space_t::latch’ of NULL object [-Werror=invalid-offsetof]
| | * | | Merge remote-tracking branch 'connect/10.2' into 10.2Oleksandr Byelkin2020-08-0217-102/+505
| | |\ \ \
| | | * | | Fix compile error for KVM. Modified filamdbf.cppOlivier Bertrand2020-07-131-1/+1
| | | | | |
| | | * | | - Fix MDEV-22561 Unable to access DBF inside a ZIP archiveOlivier Bertrand2020-07-1314-100/+490
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | modified: storage/connect/filamdbf.cpp modified: storage/connect/filamdbf.h modified: storage/connect/filamzip.cpp modified: storage/connect/filamzip.h modified: storage/connect/ha_connect.cc modified: storage/connect/plgxml.cpp modified: storage/connect/tabdos.cpp modified: storage/connect/tabdos.h modified: storage/connect/tabfix.h - Add/Init Level class member modified: storage/connect/mongo.cpp modified: storage/connect/mongo.h modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h - Typo modified: storage/connect/connect.cc
| | | * | | - Fix MDEV-22571 and MDEV-22572. Allow multiple ZIP tableOlivier Bertrand2020-05-193-2/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and enable using special column in them. modified: storage/connect/tabzip.cpp modified: storage/connect/tabzip.h - Fix some compiler errors modified: storage/connect/tabcmg.cpp
| | | * | | - Fix ↵Olivier Bertrand2020-03-112-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://stackoverflow.com/questions/60625778/import-complex-xml-from-multiple-files-in-mariadb/60637429#60637429 Import complex XML from multiple files in MariaDB Some row results are missing and replaced by the last file one. Thats because Nx and Sx column members are not reset when changing file. modified: storage/connect/tabxml.cpp modified: storage/connect/tabxml.h
| | * | | | Merge branch '10.1' into 10.2Oleksandr Byelkin2020-08-0213-52/+34
| | |\ \ \ \ | | | | |/ / | | | |/| |
| | | * | | XtraDB 5.6.49-89.0Marko Mäkelä2020-07-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The only change between Percona XtraDB Server 5.6.48-88.0 and 5.6.49-89.0 (apart from the version number change) was percona/percona-server@25ec24092064c2ab95752705e592e0c038ec1111 which we had already addressed in commit 7c03edf2fe66855a8ce8f2575c3aaf66af975377 and commit c0fca2863bcbd7cd231f1aa747b4f8d999e3a00e.
| | | * | | MDEV-18496 Crash when Aria encryption is enabled but plugin not availableSergei Golubchik2020-07-291-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | fix uninitialized struct member
| | | * | | MDEV-18496 Crash when Aria encryption is enabled but plugin not availableSergei Golubchik2020-07-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | wait_while_table_is_used() should return an error if handler::extra() fails
| | | * | | MDEV-23272 Galera stack-use-after-scope error with ASAN buildTeemu Ollakka2020-07-242-18/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | THD proc info was assigned from stack allocated temporary buffer which went out of scope immediately after assignment. Fixed by removing the use of temp buffer and assign proc info from string literal.
| | | * | | MDEV-22903 heap-use-after-free while accessing fts cache deleted doc idsThirunarayanan Balathandayuthapani2020-07-232-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: ======= fts_cache_append_deleted_doc_ids() holds the deleted_lock and tries to access size of deleted_doc_ids. In the meantime, fts_cache_clear() clears the sync_heap before clearing deleted_doc_ids. It leads to invalid access of deleted_doc_ids. Fix: === fts_cache_clear() should free the sync_heap after clearing deleted_doc_ids.
| | | * | | MDEV-23268 SIGSEGV on srv_monitor_event if InnoDB is read-onlyMarko Mäkelä2020-07-236-18/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The srv_monitor_event and the srv_monitor_thread would not be created when InnoDB is in read-only mode. Yet, some code would unconditionally invoke os_event_set(srv_monitor_event).
| | | * | | MDEV-19232: Floating point precision / value comparison problemVarun Gupta2020-07-222-9/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The issue occurs when the subquery_cache is enabled. When there is a cache miss the division was leading to a value with scale 9. In the case of cache hit the value returned was of scale 9 and due to the different values for the scales the where condition evaluated to FALSE, hence the output was incomplete. To fix this problem we need to round up the decimal to the limit mentioned in Item::decimals. This would make sure the values are compared with the same scale.
| | * | | | MDEV-14711 Fix-upThirunarayanan Balathandayuthapani2020-07-311-1/+1
| | | | | |
| * | | | | MDEV-19632 Replication aborts with ER_SLAVE_CONVERSION_FAILED upon CREATE ↵Alexander Barkov2020-08-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... SELECT in ORACLE mode - Adding optional qualifiers to data types: CREATE TABLE t1 (a schema.DATE); Qualifiers now work only for three pre-defined schemas: mariadb_schema oracle_schema maxdb_schema These schemas are virtual (hard-coded) for now, but may turn into real databases on disk in the future. - mariadb_schema.TYPE now always resolves to a true MariaDB data type TYPE without sql_mode specific translations. - oracle_schema.DATE translates to MariaDB DATETIME. - maxdb_schema.TIMESTAMP translates to MariaDB DATETIME. - Fixing SHOW CREATE TABLE to use a qualifier for a data type TYPE if the current sql_mode translates TYPE to something else. The above changes fix the reported problem, so this script: SET sql_mode=ORACLE; CREATE TABLE t2 AS SELECT mariadb_date_column FROM t1; is now replicated as: SET sql_mode=ORACLE; CREATE TABLE t2 (mariadb_date_column mariadb_schema.DATE); and the slave can unambiguously treat DATE as the true MariaDB DATE without ORACLE specific translation to DATETIME. Similar, SET sql_mode=MAXDB; CREATE TABLE t2 AS SELECT mariadb_timestamp_column FROM t1; is now replicated as: SET sql_mode=MAXDB; CREATE TABLE t2 (mariadb_timestamp_column mariadb_schema.TIMESTAMP); so the slave treats TIMESTAMP as the true MariaDB TIMESTAMP without MAXDB specific translation to DATETIME.
* | | | | | Merge 10.3 into 10.4Marko Mäkelä2020-07-314-122/+164
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Merge 10.2 into 10.3Marko Mäkelä2020-07-314-113/+163
| |\ \ \ \ \ | | |/ / / /
| | * | | | MDEV-11799 Doublewrite recovery can corrupt data pagesMarko Mäkelä2020-07-314-114/+164
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The purpose of the InnoDB doublewrite buffer is to make InnoDB tolerant against cases where the server was killed in the middle of a page write. (In Linux, killing a process may interrupt a write system call, typically on a 4096-byte boundary.) There may exist multiple copies of a page number in the doublewrite buffer. Recovery should choose the latest valid copy of the page. By design, the FIL_PAGE_LSN must not precede the latest checkpoint LSN nor be later than the end of the recovered log. For page_compressed and encrypted pages, we were missing proper consistency checks. In the 10.4 data set generated for in MDEV-23231, the data file contained a valid page_compressed page, and an identical copy of that page was also present in the doublewrite buffer. But, recovery would incorrectly consider the page invalid and restore an uncompressed copy of the same page that had been written before the log checkpoint. (In fact, no redo log was to be applied to that page.) buf_dblwr_process(): Validate the FIL_PAGE_LSN in the doublewrite buffer pages, and always skip page 0, because those pages should have been recovered by Datafile::restore_from_doublewrite() if necessary. Datafile::restore_from_doublewrite(): Choose the latest applicable page from the doublewrite buffer. recv_dblwr_t::find_page(): Also validate encrypted or page_compressed pages. recv_dblwr_t::validate_page(): New function to validate a page, either a copy in a data file or in the doublewrite buffer. Also validate encrypted or page_compressed pages. This is joint work with Thirunarayanan Balathandayuthapani.
* | | | | | Merge 10.3 into 10.4Marko Mäkelä2020-07-3122-206/+137
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Merge 10.2 into 10.3Marko Mäkelä2020-07-3120-203/+124
| |\ \ \ \ \ | | |/ / / /
| | * | | | MDEV-23198 Crash in REPLACEMarko Mäkelä2020-07-311-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | row_vers_impl_x_locked_low(): clust_offsets may point to memory that is allocated by mem_heap_alloc() and may have been freed. For initializing clust_offsets, try to use the stack-allocated buffer instead of a pointer that may point to freed memory. This fixes a regression that was introduced in commit f0aa073f2bf3d8d85b3d028df89cdb4cdfc4002d (MDEV-20950).
| | * | | | MDEV-23332 Index online status assert failure in btr_search_drop_page_hash_indexThirunarayanan Balathandayuthapani2020-07-301-20/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: ======== In row_merge_drop_indexes(), InnoDB drops only the index from dictionary and frees the index pages but it maintains the index object if the table is being used by other DML threads. It sets the online status of the index to ONLINE_INDEX_ABORTED_DROPPED. Removing the index from dictionary doesn't remove the corressponding ahi entries of the index. When block is being reused, InnoDB tries to remove ahi entries for the block and it fails if index online status is ONLINE_INDEX_ABORTED_DROPPED. Fix: ==== MDEV-22456 allows the index ahi entries to be dropped lazily. so checking online status in btr_search_drop_page_hash_index() is meaningless and should be removed.
| | * | | | MDEV-23339 innodb_force_recovery=2 may still abort the rollback of recovered ↵Marko Mäkelä2020-07-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | transactions trx_rollback_active(), trx_rollback_resurrected(): Replace an incorrect condition that we failed to replace in commit b68f1d847f1fc00eed795e20162effc8fbc4119b (MDEV-21217).
| | * | | | MDEV-23308 CHECK TABLE attempts to access parent_right_page_no=FIL_NULLMarko Mäkelä2020-07-281-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mysql/mysql-server@e00ad49edc8b07317b52c9efd0810f2cbc57877a which introduced WL#6326 to MySQL 5.7.2 added a buffer page acquisition to CHECK TABLE code (solely for the purpose of obeying the changed latching order), but failed to check that a parent page actually exists. It would not necessarily exist in a corrupted index where a parent page is missing pointer records to child pages.
| | * | | | MDEV-23137: aarch64, postfix - cmake includeDaniel Black2020-07-281-0/+1
| | | | | |
| | * | | | rocksdb: FreeBSD disable jemalloc searchDaniel Black2020-07-281-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FreeBSD's inbuilt default jemalloc means its pointless to do a package search on it. The paths are already set by the system defaults.
| | * | | | MDEV-23051: riscv64 fails build (atomics)Daniel Black2020-07-282-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | riscv64 fails to build because the use of #include <atomic> needs to link with -latomic. per https://github.com/riscv/riscv-gnu-toolchain/issues/183#issuecomment-253721765
| | * | | | MDEV-23137: RocksDB: undefined reference to crc32c_arm64Krunal Bauskar2020-07-281-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RocksDB fails to build on arm64: undefined reference to `crc32c_arm64(unsigned int, unsigned char const*, unsigned int)' MariaDB uses storage/rocksdb/build_rocksdb.cmake to compile RocksDB. Said cmake missed adding crc32c_arm64 compilation target so if machine native architecture supported crc32 then complier would enable usage of function defined in crc32c_arm64 causing the listed error. Added crc32c_arm64 complition target. closes #1642
| | * | | | MDEV-12474: rocksdb: mtr - rocksdb.concurrent_alter use shDaniel Black2020-07-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FreeBSD doesn't have bash installed by default and sh has sufficient job control for this test. $ mysql-test/mtr --mem --max-test-fail=30 --force --parallel=1 rocksdb.concurrent_alter Logging: /home/dan/mariadb-server-10.5/mysql-test/mysql-test-run.pl --mem --max-test-fail=30 --force --parallel=1 rocksdb.concurrent_alter vardir: /usr/home/dan/build-mariadb-server-10.5/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/usr/home/dan/build-mariadb-server-10.5/mysql-test/var'... - symlinking 'var' to '/tmp/var_auto_P81m' Checking supported features... MariaDB Version 10.5.4-MariaDB - SSL connections supported - binaries built with wsrep patch Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 rocksdb.concurrent_alter 'write_committed' [ pass ] 16348 rocksdb.concurrent_alter 'write_prepared' [ pass ] 16771 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 33.119 of 41 seconds executing testcases Completed: All 2 tests were successful. $ uname -a FreeBSD freebsd 12.1-RELEASE-p6 FreeBSD 12.1-RELEASE-p6 GENERIC amd64
| | * | | | MDEV-23233: Race condition for btr_search_drop_page_hash_index() in ↵Thirunarayanan Balathandayuthapani2020-07-271-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | buf_page_create() commit ad6171b91cac33e70bb28fa6865488b2c65e858c (MDEV-22456) introduced code to buf_page_create() that would lazily drop adaptive hash index entries for an index that has been evicted from the data dictionary cache. Unfortunately, that call was missing adequate protection. While the btr_search_drop_page_hash_index(block) was executing, the block could be reused for something else. buf_page_create(): If btr_search_drop_page_hash_index() must be invoked, pin the block before releasing the buf_pool->page_hash lock, so that the block cannot be grabbed by other threads.
| | * | | | MDEV-14711 Assertion `mode == 16 || mode == 12 || ↵Thirunarayanan Balathandayuthapani2020-07-241-2/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | !fix_block->page.file_page_was_freed' failed in buf_page_get_gen (rollback requesting a freed undo page) Problem: ======= In buf_cur_optimistic_latch_leaves(), requesting a left block with BTR_GET after releasing current block. But there is no guarantee that left block could be still available. Fix: ==== (1) In btr_cur_optimistic_latch_leaves(), replace the BUF_GET with BUF_GET_POSSIBLY_FREED for fetching left block. (2) Once InnoDB acquires left block, it should check FIL_PAGE_NEXT with current block page number. If not, release cursor->left_block and return false.
| | * | | | MDEV-23134 SEGV in dict_load_table_one during restart after server crashThirunarayanan Balathandayuthapani2020-07-231-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: ======== dict_load_table_one() doesn't handle the scenario where clustered index page is FIL_NULL when DICT_ERR_IGNORE_INDEX_ROOT mode is set. Fix: ==== InnoDB should set the file_unreadable when it can't find the clustered index root page.
View Lorry Controller Status