summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-27058: Reduce the size of buf_block_t and buf_page_tbb-10.6-MDEV-27058Marko Mäkelä2021-11-1685-3303/+2829
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | buf_page_t::frame: Moved from buf_block_t::frame. All 'thin' buf_page_t describing compressed-only ROW_FORMAT=COMPRESSED pages will have frame=nullptr, while all 'fat' buf_block_t will have a non-null frame pointing to aligned innodb_page_size bytes. This eliminates the need for separate states for BUF_BLOCK_FILE_PAGE and BUF_BLOCK_ZIP_PAGE. buf_page_t::lock: Moved from buf_block_t::lock. Also 'thin' page descriptors will have a page latch. The IO_PIN state that was used for discarding or creating the uncompressed page frame of a ROW_FORMAT=COMPRESSED block is replaced by a combination of read-fix and page X-latch. page_zip_des_t::fix: Replaces state_, buf_fix_count_, io_fix_, status of buf_page_t with a single std::atomic<uint32_t>. All modifications will use store(), fetch_add(), fetch_sub(). This space was previously wasted to alignment on 64-bit systems. We will use the following encoding: buf_page_t::NOT_USED=0 (previously BUF_BLOCK_NOT_USED) buf_page_t::MEMORY=1 (previously BUF_BLOCK_MEMORY) buf_page_t::REMOVE_HASH=2 (previously BUF_BLOCK_REMOVE_HASH) buf_page_t::FREED=3 + fix_count: pages marked as freed in the file buf_page_t::UNFIXED=1U<<29 + fix_count: normal pages buf_page_t::IBUF_EXIST=2U<<29 + fix_count: normal pages; may need ibuf merge buf_page_t::REINIT=3U<<29 + fix_count: reinitialized pages (skip doublewrite) buf_page_t::READ_FIX=4U<<29 + fix_count: read-fixed pages (also X-latched) buf_page_t::WRITE_FIX=5U<<29 + fix_count: write-fixed pages (also U-latched) buf_page_t::WRITE_FIX_IBUF=6U<<29 + fix_count: write-fixed; may have ibuf buf_page_t::WRITE_FIX_REINIT=7U<<29 + fix_count: write-fixed (no doublewrite) buf_page_t::write_complete(): Change WRITE_FIX or WRITE_FIX_REINIT to UNFIXED, and WRITE_FIX_IBUF to IBUF_EXIST, before releasing the U-latch. buf_page_t::read_complete(): Renamed from buf_page_read_complete(). Change READ_FIX to UNFIXED or IBUF_EXIST, before releasing the X-latch. buf_page_t::can_relocate(): If the page latch is being held or waited for, or the block is buffer-fixed or io-fixed, return false. (The condition on the page latch is new.) Outside buf_page_get_gen(), buf_page_get_low() and buf_page_free(), we will acquire the page latch before fix(), and unfix() before unlocking. buf_page_t::flush(): Replaces buf_flush_page(). Optimize the handling of FREED pages. buf_pool_t::release_freed_page(): Assume that buf_pool.mutex is held by the caller. buf_page_t::is_read_fixed(), buf_page_t::is_write_fixed(): New predicates. buf_page_get_low(): Ignore guesses that are read-fixed because they may not yet be registered in buf_pool.page_hash and buf_pool.LRU. buf_page_optimistic_get(): Acquire latch before buffer-fixing. buf_page_make_young(): Leave read-fixed blocks alone, because they might not be registered in buf_pool.LRU yet. recv_sys_t::recover_deferred(), recv_sys_t::recover_low(): Possibly fix MDEV-26326, by holding a page X-latch instead of only buffer-fixing the page.
* MDEV-27058: Move buf_page_t::slot to IORequest::slotMarko Mäkelä2021-11-167-68/+52
| | | | | | | MDEV-23855 and MDEV-23399 already moved some transient data fields from buffer pool page descriptors to IORequest, but the write buffer of PAGE_COMPRESSED or ENCRYPTED tables was missed. Since is only needed during asynchronous page write requests, it belongs to IORequest.
* MDEV-26769 follow-up: Remove unnecessary page lockingMarko Mäkelä2021-11-162-28/+38
| | | | | | | | | | | btr_cur_optimistic_latch_leaves(): Use transactional_shared_lock_guard. btr_cur_latch_leaves(): Avoid acquiring some page latches, because the changes are already blocked by index->lock. btr_cur_search_to_nth_level_func(): Remove a redundant variable retrying_for_search_prev=!!prev_tree_blocks, and avoid acquiring some page latches.
* Merge 10.5 into 10.6Marko Mäkelä2021-11-161-4/+15
|\
| * MDEV-27059 page_zip_dir_insert() may corrupt ROW_FORMAT=COMPRESSED tablesMarko Mäkelä2021-11-161-4/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In commit 7ae21b18a6b73bbc3bf1ff448faf60c29ac1d386 (MDEV-12353) the recovery of ROW_FORMAT=COMPRESSED tables was changed. Changes would be logged in a physical format for the compressed page image, so that the page need not be decompressed or compressed during recovery. page_zip_write_rec(): Log any update of the delete-mark flag in the ROW_FORMAT=COMPRESSED page. page_zip_dir_insert(): Copy the delete-mark flag. A delete-marked record may be inserted by btr_cur_pessimistic_update() via btr_cur_insert_if_possible(), page_cur_tuple_insert(), page_cur_insert_rec_zip(). In the observed scenario, it was an ROLLBACK. Presumably, the test case involved repeated DELETE and INSERT of the same key, or updating a key back and forth. This change alone might make the adjustment in page_zip_write_rec() redundant, but we play it safe because we failed to create a minimal test case for this scenario.
* | MDEV-27028 [ERROR] [FATAL] InnoDB: SYS_VIRTUAL.TABLE_ID mismatchMarko Mäkelä2021-11-161-52/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | If the server is killed during any DDL operation that is about to delete an .ibd file, recovery could crash when attempting to load the table definition of the being-dropped table. By design of commit 1bd681c8b3c5213ce1f7976940a7dc38b48a0d39 (MDEV-25506 part 3), a table whose name starts with #sql-ib in the data dictionary may belong to an uncommitted transaction. So, we must ignore any missing SYS_COLUMNS, SYS_FIELDS, and SYS_VIRTUAL records for such tables. The "ID mismatch" error messages were misleading; they really mean "record not found".
* | Merge 10.5 into 10.6Marko Mäkelä2021-11-1638-91/+426
|\ \ | |/
| * Merge 10.4 into 10.5Marko Mäkelä2021-11-1631-81/+351
| |\
| | * MDEV-23805 Make Online DDL to Instant DDL when table is emptybb-10.4-MDEV-23805Thirunarayanan Balathandayuthapani2021-11-1223-13/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - In ha_innobase::prepare_inplace_alter_table(), InnoDB should check whether the table is empty. If the table is empty then server should avoid downgrading the MDL after prepare phase. It is more like instant alter, does change only in dicationary and metadata. - Changed few debug test case to make non-empty DDL table
| | * Merge branch '10.3' into 10.4Vladislav Vaintroub2021-11-123-5/+13
| | |\
| | | * Merge branch '10.2' into 10.3Vladislav Vaintroub2021-11-123-5/+13
| | | |\
| | | | * MDEV-27030 vcol.vcol_keys_myisam fails on Windows x64, with Visual Studio 2022Vladislav Vaintroub2021-11-111-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Upon investigation, decided this to be a compiler bug (happens with new compiler, on code that did not change for the last 15 years) Fixed by de-optimizing single function remove_key(), using MSVC pragma
| | | | * MDEV-26991: CURRENT_TEST: main.mysql_binary_zero_insert 'grep' is not ↵bb-10.2-MDEV-26991Brandon Nesterenko2021-11-112-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | recognized as an internal or external command, operable program or batch file. Removed grep from mysqldump command stream and instead, extend the search_file pattern to search for rows containing binary zeros instead of any occurance of '00' in the input
| | * | | MDEV-23766: fix by assert (Windows)Sergei Krivonos2021-11-102-14/+13
| | | | |
| | * | | MDEV-23766: Re-add Json_writer unit test.Sergei Petrunia2021-11-092-2/+4
| | | | |
| | * | | In case WITH_WSREP is enabled, build wsrep as pluginVladislav Vaintroub2021-11-093-13/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If it is not enabled, build wsrep as static "stub" library from wsrep_dummy.cc ´ Allow static plugins to export symbols (on Unix) wsrep_info relies on wsrep defined symbols (e.g LOCK_wsrep_config_state) exported from mysqld
| | * | | improve build, allow sql library to be built in parallel with builtinsVladislav Vaintroub2021-11-094-8/+49
| | | | |
| | * | | Add new option NOT_EMBEDDED, for pluginsVladislav Vaintroub2021-11-093-8/+20
| | | | | | | | | | | | | | | | | | | | Means, plugin will not be available in embedded, even if compiled-in
| | * | | Revert "improve build, allow sql library to be built in parallel with builtins"Sergei Krivonos2021-11-094-49/+8
| | | | | | | | | | | | | | | | | | | | This reverts commit 1a3570dec35733e725cc6000a06ec666facf4235.
| | * | | Revert "In case WITH_WSREP is enabled, build wsrep as plugin"Sergei Krivonos2021-11-093-8/+13
| | | | | | | | | | | | | | | | | | | | This reverts commit e45f7f485a4c8133962a4082636412745ed07093.
| | * | | MDEV-23766: buildfix: postpone new unittest until its dependency resolutionSergei Krivonos2021-11-091-4/+0
| | | | |
| | * | | MDEV-23766: fix by my_json_writer testSergei Krivonos2021-11-093-21/+19
| | | | |
| | * | | MDEV-23766: Make Json_writer assert when one tries to author invalid JSONSergei Petrunia2021-11-094-8/+163
| | | | | | | | | | | | | | | | | | | | - Add unit test.
| | * | | MDEV-23766: Make Json_writer assert when one tries to author invalid JSONSergei Petrunia2021-11-092-27/+17
| | | | | | | | | | | | | | | | | | | | | | | | | Code cleanup: Remove Json_writer::is_on_fmt_helper_call. We already maintain this state in fmt_helper.
| | * | | In case WITH_WSREP is enabled, build wsrep as pluginVladislav Vaintroub2021-11-093-13/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If it is not enabled, build wsrep as static "stub" library from wsrep_dummy.cc ´ Allow static plugins to export symbols (on Unix) wsrep_info relies on wsrep defined symbols (e.g LOCK_wsrep_config_state) exported from mysqld
| | * | | improve build, allow sql library to be built in parallel with builtinsVladislav Vaintroub2021-11-094-8/+49
| | | | |
| | * | | MDEV-23766: add Json_writer consistency asserts to check array/object sequenceSergei Krivonos2021-11-092-42/+116
| | | | |
| | * | | MDEV-23766: Fix get_best_disjunct_quick by assert:Sergei Krivonos2021-11-091-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in test main.range_vs_index_merge one path requires named JSON object: assert.c:0(.annobin_assert.c_end)[0x7fe9d2270a76] sql/my_json_writer.cc:43(Json_writer::on_start_object())[0x555e284f048a] sql/my_json_writer.cc:59(Json_writer::start_object())[0x555e284ee6e8] sql/my_json_writer.h:377(Json_writer_object::Json_writer_object(THD*))[0x555e281dce11] sql/opt_range.cc:5137(get_best_disjunct_quick(PARAM*, SEL_IMERGE*, double))[0x555e287c576b] sql/opt_range.cc:5492(merge_same_index_scans(PARAM*, SEL_IMERGE*, TRP_INDEX_MERGE*, double))[0x555e287c6cf6] sql/opt_range.cc:5287(get_best_disjunct_quick(PARAM*, SEL_IMERGE*, double))[0x555e287c607a] sql/opt_range.cc:3000(SQL_SELECT::test_quick_select another one requires unnamed JSON: mariadbd: /home/name/server/sql/my_json_writer.cc:379: bool Single_line_formatting_helper::on_add_member(const char*, size_t): Assertion `state== INACTIVE || state == assert.c:0(.annobin_assert.c_end)[0x7f33d8df8a76] sql/my_json_writer.cc:380(Single_line_formatting_helper::on_add_member(char const*, unsigned long))[0x558362f6a717] sql/my_json_writer.cc:150(Json_writer::add_member(char const*, unsigned long))[0x558362f69a91] sql/my_json_writer.cc:146(Json_writer::add_member(char const*))[0x558362f69a5f] sql/my_json_writer.h:383(Json_writer_object::Json_writer_object(THD*, char const*))[0x558362ceccaa] sql/opt_range.cc:5139(get_best_disjunct_quick(PARAM*, SEL_IMERGE*, double))[0x5583632407d0] sql/opt_range.cc:3000(SQL_SELECT::test_quick_select
| | * | | MDEV-23766: Fix fix_semijoin_strategies_for_picked_join_order by assertSergei Krivonos2021-11-091-3/+3
| | | | |
| * | | | MDEV-27047: Replication fails to remove affected queries from query cacheMarko Mäkelä2021-11-163-3/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rows_log_event::do_apply_event(): Correct the mistake that was made in the merge 5f8561a6bcdb66e05ca539365cce33a9fc1817a2. In Galera, the query cache will be invalidated near the end of the function.
| * | | | MDEV-27016: Assertion 'id.page_no() < space.size' failedMarko Mäkelä2021-11-161-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | buf_flush_check_neighbors(): Relax a debug assertion that could fail for the very last page(s) of a ROW_FORMAT=COMPRESSED tables using a 1024-byte or 2048-byte page size. This assertion started to fail after commit d09426f9e60fd93296464ec9eb5f9d85566437d3 (MDEV-26537) modified the .ibd file extension to occur in steps of 4096 bytes.
| * | | | MDEV-18439 Windows builds should have core_file=1 by defaultVladislav Vaintroub2021-11-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | reapply patch 96b472c0ae798da43ca9f4735dfafe35b2f38fda It was not merged correctly into 10.5+
* | | | | Merge branch '10.5' into 10.6Vladislav Vaintroub2021-11-121-0/+8
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-27030 vcol.vcol_keys_myisam fails on Windows x64, with Visual Studio 2022Vladislav Vaintroub2021-11-121-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Upon investigation, decided this to be a compiler bug (happens with new compiler, on code that did not change for the last 15 years) Fixed by de-optimizing single function remove_key(), using MSVC pragma
* | | | | MDEV-26121 [Note] InnoDB: Resetting invalid pagebb-10.6-MDEV-27006Thirunarayanan Balathandayuthapani2021-11-101-12/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In dict_index_t::clear(), InnoDB frees all the page except root page. root page leaf segment has reset and does reinitialize again. t in fseg_create(), we do have the assumption that only FIL_PAGE_TYPE_TRX_SYS or FIL_PAGE_TYPE_TRX_SYS page should be re-created for non-full-crc32 format. This assumption is wrong in case of rollback of bulk insert operation.
* | | | | MDEV-27006 Assertion `!lock_trx_has_sys_table_locks(trx)' failed in dberr_t ↵Thirunarayanan Balathandayuthapani2021-11-093-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | row_discard_tablespace_for_mysql(dict_table_t*, trx_t*) - InnoDB import operation fails when it tries to unlock data dictionary lock before releasing the lock on system tables.
* | | | | MDEV-26826 fixup for Valgrind and MemorySanitizerMarko Mäkelä2021-11-091-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The debug assertion that was added in commit 9b967c4c31555174a293922d4717d83b7d68bc76 tripped Valgrind and MemorySanitizer. buf_block_init(): Assert that block->page.hash was zero-initialized.
* | | | | Merge 10.5 into 10.6Marko Mäkelä2021-11-0959-99/+1770
|\ \ \ \ \ | |/ / / /
| * | | | Merge 10.4 into 10.5Marko Mäkelä2021-11-0946-55/+1536
| |\ \ \ \ | | |/ / /
| | * | | Merge 10.3 into 10.4Marko Mäkelä2021-11-0946-65/+1534
| | |\ \ \ | | | |/ /
| | | * | Merge 10.2 into 10.3Marko Mäkelä2021-11-0929-35/+1362
| | | |\ \ | | | | |/
| | | | * Remove a warning for clang 11 or earlierMarko Mäkelä2021-11-092-2/+2
| | | | | | | | | | | | | | | | | | | | This fixes up commit d22c8cae00f7a7517c9b8228efbb543037c23c97
| | | | * Remove restarts from encrypt_and_grep testst-10.2-mergeMarko Mäkelä2021-11-092-10/+12
| | | | |
| | | | * Merge mariadb-10.2.41 into 10.2Marko Mäkelä2021-11-0923-277/+477
| | | | |\
| | | | * | MDEV-26561 mariabackup release locksDaniel Black2021-11-091-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous threads locked need to be released too. This occurs if the initialization of any of the non-first mutex/conditition variables errors occurs.
| | | | * | MDEV-26561 Fix a bug due to unreleased lockryancaicse2021-11-091-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Fix a bug of unreleased lock ctrl_mutex in the method create_worker_threads
| | | | * | bump the VERSIONbb-10.2-bumpversionDaniel Bartholomew2021-11-081-1/+1
| | | | | |
| | | | * | MDEV-22522 RPM packages have meaningless summary/descriptionAlexey Bychko2021-11-081-6/+67
| | | | | | | | | | | | | | | | | | | | | | | | added summary/description per package.
| | | | * | MDEV-25610 Assertion `escape != -1' failed in Item_func_like::val_intbb-10.2-bar-MDEV-25610Alexander Barkov2021-11-086-2/+1123
| | | | | |
| | | | * | appveyor - do not use buggy cygwin bison.Vladislav Vaintroub2021-11-041-1/+7
| | | | | |