summaryrefslogtreecommitdiff
path: root/storage/rocksdb
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-22933 - remove ---source include/not_threadpool.inc from testsbb-10.5-wlad2Vladislav Vaintroub2020-06-181-1/+0
|
* Cleaned up compile failure if DEBUG_SYNC is disabled in DEBUG buildsMonty2020-06-141-0/+2
|
* MDEV-21831: Assertion `length == pack_length()' failed in ↵Varun Gupta2020-06-113-1/+18
| | | | | | | | Field_inet6::sort_string upon INSERT into RocksDB table For INET6 columns the values are stored as BINARY columns and returned to the client in TEXT format. For rocksdb the indexes store mem-comparable images for columns, so use the pack_length() to store the mem-comparable form for INET6 columns. This would also remain consistent with CHAR columns.
* MDEV-22858 Remove unused innodb_mem_validate_usec, innodb_master_purge_usecMarko Mäkelä2020-06-101-2/+0
| | | | | | | | MONITOR_SRV_MEM_VALIDATE_MICROSECOND, MEM_PERIODIC_CHECK, SRV_MASTER_MEM_VALIDATE_INTERVAL: Remove. These were unused ever since UNIV_MEM_DEBUG was removed. MONITOR_SRV_PURGE_MICROSECOND: Remove. This was always unused.
* MDEV-22434 UPDATE on RocksDB table with WITHOUT OVERLAPS failsNikita Malyavin2020-06-054-9/+28
| | | | | | | | | | | | | | | | | | | | Insert worked incorrect as well. RocksDB used table->record[0] internally to store some intermediate results for key conversion, during index searching among other operations. So table->record[0] is spoiled during ha_rnd_index_map in ha_check_overlaps, so in turn the broken record data was inserted. The fix is to store RocksDB intermediate result in its own buffer instead of table->record[0]. `rocksdb` MTR suite is is checked and runs fine. No need for additional tests. The existing overlaps.test covers the case completely. However, I am not going to add anything related to rocksdb to suite, to keep it away from additional dependencies. To run tests with RocksDB engine, one can add following to engines.combinations: [rocksdb] plugin-load=$HA_ROCKSDB_SO default-storage-engine=rocksdb rocksdb
* MDEV-22641: Provide SIMD optimized wrapper for zlib crc32() (#1558)mysqlonarm2020-06-012-8/+9
| | | | | | | | | | | | | | | | | | | | | | Existing implementation used my_checksum (from mysys) for calculating table checksum and binlog checksum. This implementation was optimized for powerpc only and lacked SIMD implementation for x86 (using clmul) and ARM (using ACLE) instead used zlib-crc32. mariabackup had its own copy of the crc32 implementation using hardware optimized implementation only for x86 and lagged hardware based implementation for powerpc and ARM. Patch helps unifies all such calls and help aggregate all of them using an unified interface my_checksum(). Said unification also enables hardware optimized calls for all architecture viz. x86, ARM, POWERPC. Default always fallback to zlib crc32. Thanks to Daniel Black for reviewing, fixing and testing PowerPC changes. Thanks to Marko and Daniel for early code feedback.
* Merge 10.4 into 10.5Marko Mäkelä2020-05-312-9/+1
|\
| * Merge 10.3 into 10.4Marko Mäkelä2020-05-302-9/+1
| |\
| | * Merge 10.2 into 10.3Marko Mäkelä2020-05-292-9/+1
| | |\
| | | * Revert "MDEV-12445 : Rocksdb does not shutdown worker threads and aborts in ↵Sergei Golubchik2020-05-271-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | memleak check on server shutdown" This reverts commit 6f1f9114971c3c158e9ac97313c92a24f6554415. because it doesn't do anything now (the server doesn't check my_disable_leak_check) and it never did anything before (because without `extern` it simply created a local instance of my_disable_leak_check, did not affect server's my_disable_leak_check).
| | | * fix rocksdb zstd detectionSergei Golubchik2020-05-271-1/+1
| | | | | | | | | | | | | | | | | | | | WITH_ROCKSDB_ZSTD must use the same capitalization as in the foreach loop
* | | | Merge branch '10.4' into 10.5Sergei Golubchik2020-05-091-4/+5
|\ \ \ \ | |/ / /
| * | | MDEV-21794: Optimizer flag rowid_filter leads to long querySergei Petrunia2020-05-071-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rowid Filter check is just like Index Condition Pushdown check: before we check the filter, we must check if we have walked out of the range we are scanning. (If we did, we should return, and not continue the scan). Consequences of this: - Rowid filtering doesn't work for keys that have partially-covered blob columns (just like Index Condition Pushdown) - The rowid filter function has three return values: CHECK_POS (passed) CHECK_NEG (filtered out), CHECK_OUT_OF_RANGE. All of the above is implemented in this patch
* | | | Merge 10.4 into 10.5Marko Mäkelä2020-04-254-7/+7
|\ \ \ \ | |/ / / | | | | | | | | | | | | The functional changes of commit 5836191c8f0658d5d75484766fdcc3d838b0a5c1 (MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
| * | | Merge 10.3 into 10.4Marko Mäkelä2020-04-164-7/+7
| |\ \ \ | | |/ / | | | | | | | | | | | | | | | | | | | | In main.index_merge_myisam we remove the test that was added in commit a2d24def8cc42d27c72d833abfb39ef24a2b96ba because it duplicates the test case that was added in commit 5af12e463549e4bbc2ce6ab720d78937d5e5db4e.
| | * | Merge 10.2 into 10.3Marko Mäkelä2020-04-154-7/+7
| | |\ \ | | | |/
| | | * fix rocksdb compression detectionSergei Golubchik2020-04-081-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | cmake module name is BZip2, not BZIP2. variable names are WITH_ROCKSDB_zstd and WITH_ROCKSDB_snappy. it's SNAPPY_FOUND in older cmake (and both in newer cmake). also, make it verbose.
| | | * MDEV-20604: Duplicate key value is silently truncated to 64 characters in ↵Oleksandr Byelkin2020-04-013-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | print_keydup_error Added indication of truncated string for "s" and "M" formats
* | | | MDEV-21889 Typo fix: ER_KEY_DOES_NOT_EXISTSAleksey Midenkov2020-04-041-1/+1
| | | | | | | | | | | | | | | | libmariadb revision updated.
* | | | Merge 10.4 into 10.5Marko Mäkelä2020-03-304-42/+61
|\ \ \ \ | |/ / /
| * | | Merge 10.3 into 10.4Marko Mäkelä2020-03-304-42/+61
| |\ \ \ | | |/ /
| | * | Merge 10.2 into 10.3Marko Mäkelä2020-03-304-42/+61
| | |\ \ | | | |/
| | | * MDEV-21930 RocksDB does not compile anymore, with Visual StudioVladislav Vaintroub2020-03-234-42/+61
| | | | | | | | | | | | | | | | | | | | | | | | Update submodule, change the source file list rename CACHE_LINE_SIZE in ut0counter, so it does not conflics with rocksdb sources, which also defines this constant now.
* | | | Updated optimizer costs in multi_range_read_info_const() and sql_select.ccMonty2020-03-276-13/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - multi_range_read_info_const now uses the new records_in_range interface - Added handler::avg_io_cost() - Don't calculate avg_io_cost() in get_sweep_read_cost if avg_io_cost is not 1.0. In this case we trust the avg_io_cost() from the handler. - Changed test_quick_select to use TIME_FOR_COMPARE instead of TIME_FOR_COMPARE_IDX to align this with the rest of the code. - Fixed bug when using test_if_cheaper_ordering where we didn't use keyread if index was changed - Fixed a bug where we didn't use index only read when using order-by-index - Added keyread_time() to HEAP. The default keyread_time() was optimized for blocks and not suitable for HEAP. The effect was the HEAP prefered table scans over ranges for btree indexes. - Fixed get_sweep_read_cost() for HEAP tables - Ensure that range and ref have same cost for simple ranges Added a small cost (MULTI_RANGE_READ_SETUP_COST) to ranges to ensure we favior ref for range for simple queries. - Fixed that matching_candidates_in_table() uses same number of records as the rest of the optimizer - Added avg_io_cost() to JT_EQ_REF cost. This helps calculate the cost for HEAP and temporary tables better. A few tests changed because of this. - heap::read_time() and heap::keyread_time() adjusted to not add +1. This was to ensure that handler::keyread_time() doesn't give higher cost for heap tables than for normal tables. One effect of this is that heap and derived tables stored in heap will prefer key access as this is now regarded as cheap. - Changed cost for index read in sql_select.cc to match multi_range_read_info_const(). All index cost calculation is now done trough one function. - 'ref' will now use quick_cost for keys if it exists. This is done so that for '=' ranges, 'ref' is prefered over 'range'. - scan_time() now takes avg_io_costs() into account - get_delayed_table_estimates() uses block_size and avg_io_cost() - Removed default argument to test_if_order_by_key(); simplifies code
* | | | Added page_range to records_in_range() to improve range statisticsMonty2020-03-272-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prototype change: - virtual ha_rows records_in_range(uint inx, key_range *min_key, - key_range *max_key) + virtual ha_rows records_in_range(uint inx, const key_range *min_key, + const key_range *max_key, + page_range *res) The handler can ignore the page_range parameter. In the case the handler updates the parameter, the optimizer can deduce the following: - If previous range's last key is on the same block as next range's first key - If the current key range is in one block - We can also assume that the first and last block read are cached! This can be used for a better calculation of IO seeks when we estimate the cost of a range index scan. The parameter is fully implemented for MyISAM, Aria and InnoDB. A separate patch will update handler::multi_range_read_info_const() to take the benefits of this change and also remove the double records_in_range() calls that are not anymore needed.
* | | | Replace handler::primary_key_is_clustered() with handler::pk_is_clustering_key()Monty2020-03-242-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was done to both simplify the code and also to be easier to handle storage engines that are clustered on some other index than the primary key. As pk_is_clustering_key() and is_clustering_key now are using only index_flags, these where removed from all storage engines.
* | | | MDEV-21303 Make executables MariaDB namedRasmus Johansson2020-03-211-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To change all executables to have a mariadb name I had to: - Do name changes in every CMakeLists.txt that produces executables - CREATE_MARIADB_SYMLINK was removed and GET_SYMLINK added by Wlad to reuse the function in other places also - The scripts/CMakeLists.txt could make use of GET_SYMLINK instead of introducing redundant code, but I thought I'll leave that for next release - A lot of changes to debian/.install and debian/.links files due to swapping of real executable and symlink. I did not however change the name of the manpages, so the real name is still mysql there and mariadb are symlinks. - The Windows part needed a change now when we made the executables mariadb -named. MSI (and ZIP) do not support symlinks and to not break backward compatibility we had to include mysql named binaries also. Done by Wlad
* | | | MDEV-742: Fix clang -Winconsistent-missing-overrideMarko Mäkelä2020-03-211-2/+2
| | | |
* | | | merge 10.4 to 10.5Monty2020-03-183-3/+3
|\ \ \ \ | |/ / /
| * | | Removed double records_in_range calls from multi_range_read_info_constMonty2020-03-173-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was to remove a performance regression between 10.3 and 10.4 In 10.5 we will have a better implementation of records_in_range that will enable us to get more statistics. This change was not done in 10.4 because the 10.5 will be part of a larger change that is not suitable for the GA 10.4 version Other things: - Changed default handler block_size to 8192 to fix things statistics for engines that doesn't set the block size. - Fixed a bug in spider when using multiple part const ranges (Patch from Kentoku)
* | | | update the test result for new perfschemaSergei Golubchik2020-03-161-1/+1
| | | |
* | | | MDEV-742 XA PREPAREd transaction survive disconnect/server restartAndrei Elkin2020-03-146-9/+245
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lifted long standing limitation to the XA of rolling it back at the transaction's connection close even if the XA is prepared. Prepared XA-transaction is made to sustain connection close or server restart. The patch consists of - binary logging extension to write prepared XA part of transaction signified with its XID in a new XA_prepare_log_event. The concusion part - with Commit or Rollback decision - is logged separately as Query_log_event. That is in the binlog the XA consists of two separate group of events. That makes the whole XA possibly interweaving in binlog with other XA:s or regular transaction but with no harm to replication and data consistency. Gtid_log_event receives two more flags to identify which of the two XA phases of the transaction it represents. With either flag set also XID info is added to the event. When binlog is ON on the server XID::formatID is constrained to 4 bytes. - engines are made aware of the server policy to keep up user prepared XA:s so they (Innodb, rocksdb) don't roll them back anymore at their disconnect methods. - slave applier is refined to cope with two phase logged XA:s including parallel modes of execution. This patch does not address crash-safe logging of the new events which is being addressed by MDEV-21469. CORNER CASES: read-only, pure myisam, binlog-*, @@skip_log_bin, etc Are addressed along the following policies. 1. The read-only at reconnect marks XID to fail for future completion with ER_XA_RBROLLBACK. 2. binlog-* filtered XA when it changes engine data is regarded as loggable even when nothing got cached for binlog. An empty XA-prepare group is recorded. Consequent Commit-or-Rollback succeeds in the Engine(s) as well as recorded into binlog. 3. The same applies to the non-transactional engine XA. 4. @@skip_log_bin=OFF does not record anything at XA-prepare (obviously), but the completion event is recorded into binlog to admit inconsistency with slave. The following actions are taken by the patch. At XA-prepare: when empty binlog cache - don't do anything to binlog if RO, otherwise write empty XA_prepare (assert(binlog-filter case)). At Disconnect: when Prepared && RO (=> no binlogging was done) set Xid_cache_element::error := ER_XA_RBROLLBACK *keep* XID in the cache, and rollback the transaction. At XA-"complete": Discover the error, if any don't binlog the "complete", return the error to the user. Kudos ----- Alexey Botchkov took to drive this work initially. Sergei Golubchik, Sergei Petrunja, Marko Mäkelä provided a number of good recommendations. Sergei Voitovich made a magnificent review and improvements to the code. They all deserve a bunch of thanks for making this work done!
* | | | Merge branch '10.4' into 10.5Oleksandr Byelkin2020-03-111-0/+1
|\ \ \ \ | |/ / /
| * | | Fix compile failure, compare_key_parts in handler shadowed by MyRocksSergei Petrunia2020-02-191-0/+1
| | | | | | | | | | | | | | | | | | | | The two functions have different signature. Use "using ..." to prevent shadowing
* | | | MDEV-21743 Split up SUPER privilege to smaller privilegesAlexander Barkov2020-03-101-1/+1
| | | |
* | | | cleanup: pass trxid by valueSergei Golubchik2020-03-101-2/+2
| | | |
* | | | perfschema transaction instrumentation related changesSergei Golubchik2020-03-101-2/+2
| | | |
* | | | perfschema table io instrumentation related changesSergei Golubchik2020-03-101-5/+3
| | | |
* | | | perfschema memory related instrumentation changesSergei Golubchik2020-03-103-18/+19
| | | |
* | | | Cleanup: Make MONITOR_LSN_CHECKPOINT_AGE a value.Marko Mäkelä2020-03-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Compute MONITOR_LSN_CHECKPOINT_AGE on demand in srv_mon_process_existing_counter(). This allows us to remove the overhead of MONITOR_SET calls for the counter.
* | | | MDEV-15058: Revert the changes to INFORMATION_SCHEMAMarko Mäkelä2020-02-121-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For compatibility with diagnostic software, let us return a dummy buffer pool identifier 0 and restore the columns that were initially deleted in commit 1a6f708ec594ac0ae2dd30db926ab07b100fa24b: information_schema.innodb_buffer_page.pool_id information_schema.innodb_buffer_page_lru.pool_id information_schema.innodb_buffer_pool_stats.pool_id information_schema.innodb_cmpmem.buffer_pool_instance information_schema.innodb_cmpmem_reset.buffer_pool_instance Thanks to Vladislav Vaintroub for pointing this out.
* | | | MDEV-15058: Deprecate and ignore innodb_buffer_pool_instancesMarko Mäkelä2020-02-121-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our benchmarking efforts indicate that the reasons for splitting the buf_pool in commit c18084f71b02ea707c6461353e6cfc15d7553bc6 have mostly gone away, possibly as a result of mysql/mysql-server@ce6109ebfdedfdf185e391a0c97dc6d33867ed78 or similar work. Only in one write-heavy benchmark where the working set size is ten times the buffer pool size, the buf_pool->mutex would be less contended with 4 buffer pool instances than with 1 instance, in buf_page_io_complete(). That contention could be alleviated further by making more use of std::atomic and by splitting buf_pool_t::mutex further (MDEV-15053). We will deprecate and ignore the following parameters: innodb_buffer_pool_instances innodb_page_cleaners There will be only one buffer pool and one page cleaner task. In a number of INFORMATION_SCHEMA views, columns that indicated the buffer pool instance will be removed: information_schema.innodb_buffer_page.pool_id information_schema.innodb_buffer_page_lru.pool_id information_schema.innodb_buffer_pool_stats.pool_id information_schema.innodb_cmpmem.buffer_pool_instance information_schema.innodb_cmpmem_reset.buffer_pool_instance
* | | | Merge 10.4 into 10.5Marko Mäkelä2020-02-071-1/+2
|\ \ \ \ | |/ / /
| * | | Fixed compiler warnings from gcc 7.4.1Monty2020-01-291-1/+2
| | | | | | | | | | | | | | | | - Fixed possible error in rocksdb/rdb_datadic.cc
* | | | MDEV-21581 Helper functions and methods for CHARSET_INFOAlexander Barkov2020-01-281-16/+17
| | | |
* | | | Merge 10.4 into 10.5Marko Mäkelä2019-12-272-1/+3
|\ \ \ \ | |/ / /
| * | | Merge 10.3 into 10.4Marko Mäkelä2019-12-272-1/+3
| |\ \ \ | | |/ /
| | * | Merge 10.2 into 10.3Marko Mäkelä2019-12-272-2/+2
| | |\ \ | | | |/
| | | * Disable -Werror for rocksdb submoduleSergei Golubchik2019-12-172-2/+2
| | | | | | | | | | | | | | | | | | | | We cannot have -Werror for third-party submodules because whenever they break the build we cannot fix it as needed.
* | | | Merge 10.4 into 10.5Marko Mäkelä2019-12-162-2/+2
|\ \ \ \ | |/ / /
View Lorry Controller Status