summaryrefslogtreecommitdiff
path: root/storage/xtradb/log
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-18214 remove some duplicated MONITOR countersEugene Kosov2019-05-061-21/+0
| | | | | | | MONITOR_PENDING_LOG_WRITE MONITOR_PENDING_CHECKPOINT_WRITE MONITOR_LOG_IO: read values from log_t members instead of updating own monitor variables
* MDEV-18659: Fix string truncation/overflow in InnoDB and XtraDBMarko Mäkelä2019-03-061-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Fix the warnings issued by GCC 8 -Wstringop-truncation and -Wstringop-overflow in InnoDB and XtraDB. This work is motivated by Jan Lindström. The patch mainly differs from his original one as follows: (1) We remove explicit initialization of stack-allocated string buffers. The minimum amount of initialization that is needed is a terminating NUL character. (2) GCC issues a warning for invoking strncpy(dest, src, sizeof dest) because if strlen(src) >= sizeof dest, there would be no terminating NUL byte in dest. We avoid this problem by invoking strncpy() with a limit that is 1 less than the buffer size, and by always writing NUL to the last byte of the buffer. (3) We replace strncpy() with memcpy() or strcpy() in those cases when the result is functionally equivalent. Note: fts_fetch_index_words() never deals with len==UNIV_SQL_NULL. This was enforced by an assertion that limits the maximum length to FTS_MAX_WORD_LEN. Also, the encoding that InnoDB uses for the compressed fulltext index is not byte-order agnostic, that is, InnoDB data files that use FULLTEXT INDEX are not portable between big-endian and little-endian systems.
* Merge 10.1 into 10.1Marko Mäkelä2019-02-021-1/+40
|\ | | | | | | This is joint work with Oleksandr Byelkin.
| * Merge branch 'merge-xtradb-5.6' into 10.0Oleksandr Byelkin2019-01-281-1/+40
| |\
| | * 5.6.42-84.2Oleksandr Byelkin2019-01-241-1/+40
| | |
| | * 5.6.41-84.1Sergei Golubchik2018-10-271-2/+2
| | |
* | | Merge branch '10.0' into 10.1Sergei Golubchik2018-10-301-2/+2
|\ \ \ | |/ /
| * | Merge branch '5.5' into 10.0Sergei Golubchik2018-10-271-2/+2
| |\ \
| | * \ Merge branch 'merge/merge-xtradb-5.5' into 5.5mariadb-5.5.62Sergei Golubchik2018-10-241-2/+2
| | |\ \
| | | * | 5.5.61-38.13Sergei Golubchik2018-10-241-2/+2
| | | | |
| * | | | compiler warningSergei Golubchik2018-04-201-3/+0
| | | | | | | | | | | | | | | | | | | | bad merge
* | | | | MDEV-17003 service_manager_extend_timeout() being called too oftenMarko Mäkelä2018-08-211-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | buf_dump(): Only generate the output when shutdown is in progress. log_write_up_to(): Only generate the output before actually writing to the redo log files. srv_purge_should_exit(): Rate-limit the output, and instead of displaying the work done, indicate the work that remains to be done until the completion of the slow shutdown.
* | | | | Follow-up fix to MDEV-14705: Flush log at shutdownMarko Mäkelä2018-04-261-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In commit e7f4e61f6e013e8879a64dae3ab0370ec137a121 the call fil_flush_file_spaces(FIL_LOG) is necessary. Tablespaces will be flushed as part of the redo log checkpoint, but the redo log will not necessarily be flushed, depending on innodb_flush_method.
* | | | | Fix warnings in InnoDB & XtraDB post MDEV-14705Vicențiu Ciorbaru2018-04-101-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | There was a missing argument to service_manager_extend_timeout call and the signness of arguments did not match.
* | | | | MDEV-14705: Extend timeout for waiting for transactionsMarko Mäkelä2018-04-101-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If innodb_fast_shutdown<2, all transactions of active connections will be rolled back on shutdown. This can take a long time, and the systemd shutdown timeout should be extended during the wait. logs_empty_and_mark_files_at_shutdown(): Extend the timeout when waiting for other threads to complete.
* | | | | Fix a compilation errorMarko Mäkelä2018-04-061-1/+2
| | | | |
* | | | | MDEV-14705: Follow-up fixesMarko Mäkelä2018-04-061-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | buf_flush_remove(): Disable the output for now, because we certainly do not want this after every page flush on shutdown. It must be rate-limited somehow. There already is a timeout extension for waiting the page cleaner to exit in logs_empty_and_mark_files_at_shutdown(). log_write_up_to(): Use correct format. srv_purge_should_exit(): Move the timeout extension to the appropriate place, from one of the callers.
* | | | | MDEV-14705: slow innodb startup/shutdown can exceed systemd timeoutDaniel Black2018-04-062-8/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use systemd EXTEND_TIMEOUT_USEC to advise systemd of progress Move towards progress measures rather than pure time based measures. Progress reporting at numberious shutdown/startup locations incuding: * For innodb_fast_shutdown=0 trx_roll_must_shutdown() for rolling back incomplete transactions. * For merging the change buffer (in srv_shutdown(bool ibuf_merge)) * For purging history, srv_do_purge Thanks Marko for feedback and suggestions.
* | | | | MDEV-14705: Speed up InnoDB shutdownDaniel Black2018-04-061-22/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Suggested by Marko on github pr #576 buf_all_freed only needs to be called once, not 3 times. buf_all_freed will always return TRUE if it returns. It will crash if any page was not flushed so its effectively an assert anyway. The following calls are likely redundant and could be removed: fil_flush_file_spaces(FIL_TYPE_TABLESPACE); fil_flush_file_spaces(FIL_TYPE_LOG);
* | | | | Merge branch 'github/10.0' into 10.1Sergei Golubchik2018-02-021-3/+3
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch 'merge-xtradb-5.6' into 10.0Vicențiu Ciorbaru2018-01-241-0/+3
| |\ \ \ \ | | | |_|/ | | |/| |
| | * | | 5.6.38-83.0Vicențiu Ciorbaru2018-01-231-1/+1
| | | | |
| * | | | Merge remote-tracking branch '5.5' into 10.0Vicențiu Ciorbaru2018-01-241-3/+3
| |\ \ \ \ | | | |/ / | | |/| |
| | * | | Merge branch 'merge/merge-xtradb-5.5' into 5.5mariadb-5.5.59Sergei Golubchik2018-01-181-3/+3
| | |\ \ \ | | | | |/ | | | |/|
| | | * | 5.5.58-38.10Sergei Golubchik2018-01-171-3/+3
| | | | |
| | * | | MDEV-13814 Extra logging when innodb_log_archive=ONMarko Mäkelä2018-01-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB with -DUNIV_LOG_ARCHIVE
| | * | | MDEV-14174 crash on start with innodb-track-changed-pagesMarko Mäkelä2018-01-101-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The XtraDB option innodb_track_changed_pages causes the function log_group_read_log_seg() to be invoked even when recv_sys==NULL, leading to the SIGSEGV. This regression was caused by MDEV-11027 InnoDB log recovery is too noisy
| | * | | Merge branch 'merge-xtradb-5.5' into 5.5mariadb-5.5.57Sergei Golubchik2017-07-181-39/+109
| | |\ \ \ | | | |/ /
| | | * | 5.5.55-38.8Sergei Golubchik2017-07-181-39/+109
| | | | |
| * | | | Follow-up to MDEV-11027: Use recv_sys_t::report()Marko Mäkelä2017-12-131-6/+5
| | | | |
* | | | | MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit ↵Marko Mäkelä2018-01-082-42/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | boundary This bug affects both writing and reading encrypted redo log in MariaDB 10.1, starting from version 10.1.3 which added support for innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup will sometimes fail when innodb_encrypt_log is used. MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected. log_block_get_start_lsn(): Remove. This function would cause trouble if a log segment that is being read is crossing a 32-bit boundary of the LSN, because this function does not allow the most significant 32 bits of the LSN to change. log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read(): Add the parameter "lsn" for the start LSN of the block. log_blocks_encrypt(): Remove (unused function).
* | | | | MDEV-12634: Uninitialised ROW_MERGE_RESERVE_SIZE bytes written to temporary fileJan Lindström2017-09-281-1/+1
| | | | | | | | | | | | | | | | | | | | After review cleanup.
* | | | | Merge 10.0 into 10.1Marko Mäkelä2017-09-221-1/+1
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-13814 Extra logging when innodb_log_archive=ONMarko Mäkelä2017-09-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | log_group_read_log_seg(): Only display the message during recovery, not during normal operation. When the XtraDB configuration parameter innodb_log_archive is set, this function will be called during normal operation.
* | | | | MDEV-13807 mariabackup --apply-log-only does generate redo log by performing ↵Marko Mäkelä2017-09-161-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | rollback and possibly other tasks Skip rollback and other redo-log-generating tasks if srv_apply_log_only is set. Instead of assigning the debug variable recv_no_log_write = FALSE, assign it to srv_apply_log_only, so that any unwanted writes are caught.
* | | | | MDEV-12634: Uninitialised ROW_MERGE_RESERVE_SIZE bytes written to tem…Jan Lindström2017-09-141-1/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | …porary file Fixed by removing writing key version to start of every block that was encrypted. Instead we will use single key version from log_sys crypt info. After this MDEV also blocks writen to row log are encrypted and blocks read from row log aren decrypted if encryption is configured for the table. innodb_status_variables[], struct srv_stats_t Added status variables for merge block and row log block encryption and decryption amounts. Removed ROW_MERGE_RESERVE_SIZE define. row_merge_fts_doc_tokenize Remove ROW_MERGE_RESERVE_SIZE row_log_t Add index, crypt_tail, crypt_head to be used in case of encryption. row_log_online_op, row_log_table_close_func Before writing a block encrypt it if encryption is enabled row_log_table_apply_ops, row_log_apply_ops After reading a block decrypt it if encryption is enabled row_log_allocate Allocate temporary buffers crypt_head and crypt_tail if needed. row_log_free Free temporary buffers crypt_head and crypt_tail if they exist. row_merge_encrypt_buf, row_merge_decrypt_buf Removed. row_merge_buf_create, row_merge_buf_write Remove ROW_MERGE_RESERVE_SIZE row_merge_build_indexes Allocate temporary buffer used in decryption and encryption if needed. log_tmp_blocks_crypt, log_tmp_block_encrypt, log_temp_block_decrypt New functions used in block encryption and decryption log_tmp_is_encrypted New function to check is encryption enabled. Added test case innodb-rowlog to force creating a row log and verify that operations are done using introduced status variables.
* | | | | Merge 10.0 into 10.1Marko Mäkelä2017-09-071-0/+1
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-13253 After rebuilding redo logs, InnoDB can leak data from redo log bufferMarko Mäkelä2017-09-071-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | recv_reset_logs(): Initialize the redo log buffer, so that no data from the old redo log can be written to the new redo log. This bug has very little impact before MariaDB 10.2. The innodb_log_encrypt option that was introduced in MariaDB 10.1 increases the impact. If the redo log used to be encrypted, and it is being resized and encryption disabled, then previously encrypted data could end up being written to the new redo log in clear text. This resulted in encryption.innodb_encrypt_log test failures in MariaDB 10.2.
* | | | | Merge branch '10.0' into 10.1Marko Mäkelä2017-08-301-4/+2
|\ \ \ \ \ | |/ / / /
| * | | | WL#8845: Clarify the message about redo log format incompatibilityMarko Mäkelä2017-08-291-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | recv_find_max_checkpoint(): Refer to MariaDB 10.2.2 instead of MySQL 5.7.9. Do not hint that a binary downgrade might be possible, because there are many changes in InnoDB 5.7 that could make downgrade impossible: a column appended to SYS_INDEXES, added SYS_* tables, undo log format changes, and so on.
* | | | | Merge branch '10.0' into 10.1Sergei Golubchik2017-08-081-0/+2
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch 'merge-xtradb-5.6' into 10.0Vicențiu Ciorbaru2017-08-021-0/+2
| |\ \ \ \ | | | |_|/ | | |/| |
| | * | | 5.6.36-82.1Vicențiu Ciorbaru2017-08-021-0/+2
| | | | |
| * | | | MDEV-12975 InnoDB redo log minimum size check uses detected file size ↵Marko Mäkelä2017-06-191-63/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | instead of requested innodb_log_file_size log_calc_max_ages(): Use the requested size in the check, instead of the detected redo log size. The redo log will be resized at startup if it differs from what has been requested.
* | | | | MDEV-12975 InnoDB redo log minimum size check uses detected file size ↵Marko Mäkelä2017-06-191-51/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | instead of requested innodb_log_file_size log_calc_max_ages(): Use the requested size in the check, instead of the detected redo log size. The redo log will be resized at startup if it differs from what has been requested.
* | | | | Cleanup of MDEV-12600: crash during install_db with innodb_page_size=32K and ↵Marko Mäkelä2017-06-081-23/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ibdata1=3M The doublewrite buffer pages must fit in the first InnoDB system tablespace data file. The checks that were added in the initial patch (commit 112b21da37dad0fbb28bc65f9ab5a3ba6c0c2186) were at too high level and did not cover all cases. innodb.log_data_file_size: Test all innodb_page_size combinations. fsp_header_init(): Never return an error. Move the change buffer creation to the only caller that needs to do it. btr_create(): Clean up the logic. Remove the error log messages. buf_dblwr_create(): Try to return an error on non-fatal failure. Check that the first data file is big enough for creating the doublewrite buffers. buf_dblwr_process(): Check if the doublewrite buffer is available. Display the message only if it is available. recv_recovery_from_checkpoint_start_func(): Remove a redundant message about FIL_PAGE_FILE_FLUSH_LSN mismatch when crash recovery has already been initiated. fil_report_invalid_page_access(): Simplify the message. fseg_create_general(): Do not emit messages to the error log. innobase_init(): Revert the changes. trx_rseg_create(): Refactor (no functional change).
* | | | | MDEV-12113: install_db shows corruption for rest encryption with ↵Jan Lindström2017-06-012-22/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.
* | | | | 10.1 additions for MDEV-12052 Shutdown crash presumably due to master thread ↵Marko Mäkelä2017-05-261-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | activity btr_defragment_thread(): Create the thread in the same place as other threads. Do not invoke btr_defragment_shutdown(), because row_drop_tables_for_mysql_in_background() in the master thread can still keep invoking btr_defragment_remove_table(). logs_empty_and_mark_files_at_shutdown(): Wait for btr_defragment_thread() to exit. innobase_start_or_create_for_mysql(), innobase_shutdown_for_mysql(): Skip encryption and scrubbing in innodb_read_only_mode. srv_export_innodb_status(): Do not export encryption or scrubbing statistics in innodb_read_only mode, because the threads will not be running.
* | | | | Merge 10.0 into 10.1Marko Mäkelä2017-05-231-0/+1
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-6262 analyze the coverity report on mariadbSergei Golubchik2017-05-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | uploaded 10.0, analyzed everything with the Impact=High (and a couple of Medium)
View Lorry Controller Status