From 5099033c26826fd2625b6424134999853e33a29d Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 29 May 2008 18:33:33 +0300 Subject: WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t" Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation Fixed wrong call to strmake Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert Allow storing year 2155 in year field When running with purify/valgrind avoid copying structures over themself Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called include/maria.h: Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation include/my_tree.h: Added macro 'reset_free_element()' to be able to ignore calls to the external free function. Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree mysql-test/install_test_db.sh: Upgrade to new mysql_install_db options mysql-test/r/maria-mvcc.result: New tests mysql-test/r/maria.result: New tests mysql-test/suite/ndb/r/ndb_auto_increment.result: Fixed error message now when bulk insert is not always called mysql-test/suite/ndb/t/ndb_auto_increment.test: Fixed error message now when bulk insert is not always called mysql-test/t/maria-mvcc.test: Added testing of versioning of count(*) mysql-test/t/maria-page-checksum.test: Added comment mysql-test/t/maria.test: More tests mysys/hash.c: Code style change sql/field.cc: Allow storing year 2155 in year field sql/ha_ndbcluster.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_ndbcluster.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.cc: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/ha_partition.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored sql/handler.cc: Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit sql/handler.h: Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted) sql/item.cc: Style fix Removed compiler warning sql/log_event.cc: Added new argument to ha_end_bulk_insert() sql/log_event_old.cc: Added new argument to ha_end_bulk_insert() sql/mysqld.cc: Removed compiler warning sql/protocol.cc: Added DBUG sql/sql_class.cc: Added DBUG Fixed wrong call to strmake sql/sql_insert.cc: Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's) Added new argument to ha_end_bulk_insert() sql/sql_load.cc: Added new argument to ha_end_bulk_insert() sql/sql_parse.cc: Style fixes Avoid goto in common senario sql/sql_select.cc: When running with purify/valgrind avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_select.h: Avoid copying structures over themself. This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings sql/sql_table.cc: Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped Added new argument to ha_end_bulk_insert() storage/archive/ha_archive.cc: Added new argument to end_bulk_insert() storage/archive/ha_archive.h: Added new argument to end_bulk_insert() storage/federated/ha_federated.cc: Added new argument to end_bulk_insert() storage/federated/ha_federated.h: Added new argument to end_bulk_insert() storage/maria/Makefile.am: Added ma_state.c and ma_state.h storage/maria/ha_maria.cc: Versioning of count(*) and checksum - share->state.state is now assumed to be correct, not handler->state - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of not versioned and not concurrent insertable table, file->s->state.state contains the correct state information Other things: - file->s -> share - Added DBUG_ASSERT() for unlikely case - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table) - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed storage/maria/ha_maria.h: New argument to end_bulk_insert() storage/maria/ma_blockrec.c: Update for versioning of count(*) and checksum Keep share->state.state.data_file_length up to date (not info->state->data_file_length) Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c storage/maria/ma_check.c: Update and use share->state.state instead of info->state info->s to share Update info->state at end of repair Call _ma_reset_state() to update share->state_history at end of repair storage/maria/ma_checkpoint.c: Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables storage/maria/ma_close.c: Remember state history for running transaction even if table is closed storage/maria/ma_commit.c: Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed storage/maria/ma_delete.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/ma_delete_all.c: Versioning of count(*) and checksum: - Ensure that share->state.state is updated, as here is where we store the primary information storage/maria/ma_dynrec.c: Use lock_key_trees instead of concurrent_insert to check if trees should be locked. This allows us to lock trees both for concurrent_insert and for index versioning. storage/maria/ma_extra.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - share->concurrent_insert -> share->non_transactional_concurrent_insert - Don't update share->state.state from info->state if transactional table Optimization: - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED storage/maria/ma_info.c: Get most state information from current state storage/maria/ma_init.c: Add hash table and free function to store states for closed tables Install hook for transaction commit/rollback to update history state storage/maria/ma_key_recover.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state storage/maria/ma_locking.c: Versioning of count(*) and checksum: - Call virtual functions (if exists) to restore/update status - Move _ma_xxx_status() functions to ma_state.c info->s -> share storage/maria/ma_open.c: Versioning of count(*) and checksum: - For not transactional tables, set info->state to point to new allocated state structure. - Initialize new info->state_start variable that points to state at start of transaction - Copy old history states from hash table (maria_stored_states) first time the table is opened - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree - For now, only enable versioning of tables without keys (to be fixed in soon!) - Added new virtual function to restore status in maria_lock_database) More DBUG storage/maria/ma_page.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Modify share->state.state.key_file_length under share->intern_lock storage/maria/ma_range.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees info->s -> share storage/maria/ma_recovery.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state - Update state information on close and when reenabling logging storage/maria/ma_rkey.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rnext_same.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees - Only skip rows based on file length if non_transactional_concurrent_insert is set storage/maria/ma_rprev.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_rsame.c: Versioning of count(*) and checksum: - Lock trees based on share->lock_key_trees storage/maria/ma_sort.c: Use share->state.state instead of info->state Fixed indentation storage/maria/ma_static.c: Added maria_stored_state storage/maria/ma_update.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records - Remove optimization for index file update as it doesn't work for transactional tables storage/maria/ma_write.c: Versioning of count(*) and checksum: - Always update info->state->checksum and info->state->records storage/maria/maria_def.h: Move MARIA_STATUS_INFO to ma_state.h Changes to MARIA_SHARE: - Added state_history to store count(*)/checksum states - Added in_trans as counter if table is used by running transactions - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert. - Added virtual function lock_restore_status Changes to MARIA_HA: - save_state -> state_save - Added state_start to store state at start of transaction storage/maria/maria_pack.c: Versioning of count(*) and checksum: - Use share->state.state instead of info->state Indentation fixes storage/maria/trnman.c: Added hook 'trnnam_end_trans_hook' that is called when transaction ends Added trn->used_tables that is used to an entry for all tables used by transaction More DBUG Changed return type of trnman_end_trn() to my_bool Added trnman_get_min_trid() to get minimum trid in use. Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id storage/maria/trnman.h: Added 'used_tables' Moved all pointers into same groups to get better memory alignment storage/maria/trnman_public.h: Added prototypes for new functions and variables Chagned return type of trnman_end_trn() to my_bool storage/myisam/ha_myisam.cc: Added argument to end_bulk_insert() if operation should be aborted storage/myisam/ha_myisam.h: Added argument to end_bulk_insert() if operation should be aborted storage/maria/ma_state.c: Functions to handle state of count(*) and checksum storage/maria/ma_state.h: Structures and declarations to handle state of count(*) and checksum --- storage/archive/ha_archive.cc | 2 +- storage/archive/ha_archive.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index 967e315d4a4..6c6a34e75ec 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -1511,7 +1511,7 @@ void ha_archive::start_bulk_insert(ha_rows rows) Other side of start_bulk_insert, is end_bulk_insert. Here we turn off the bulk insert flag, and set the share dirty so that the next select will call sync for us. */ -int ha_archive::end_bulk_insert() +int ha_archive::end_bulk_insert(bool table_will_be_deleted) { DBUG_ENTER("ha_archive::end_bulk_insert"); bulk_insert= FALSE; diff --git a/storage/archive/ha_archive.h b/storage/archive/ha_archive.h index ab630ed22fd..22f8302982d 100644 --- a/storage/archive/ha_archive.h +++ b/storage/archive/ha_archive.h @@ -134,7 +134,7 @@ public: int optimize(THD* thd, HA_CHECK_OPT* check_opt); int repair(THD* thd, HA_CHECK_OPT* check_opt); void start_bulk_insert(ha_rows rows); - int end_bulk_insert(); + int end_bulk_insert(bool table_will_be_deleted); enum row_type get_row_type() const { return ROW_TYPE_COMPRESSED; -- cgit v1.2.1 From c558b3e06885aecccb2cbcc250918745712f2f65 Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 8 Apr 2009 18:55:26 +0200 Subject: Fix Valgrind errors seen in buildbot. Fix mysql-test-run.pl to not terminate early when warnings in error logs are detected during server shutdown. Instead, give a nice summary report at the end of the failures. Fix code to make 100% sure no failures will go undetected. Revert earlier wrong change. Fix race with port allocation semaphore file permissions. Adjust testsuite to copy with new PBXT engine now in the tree. The PBXT engine causes an extra table to appear in the INFORMATION_SCHEMA. This causes different output for a few test cases. dbug/dbug.c: If DbugParse() is called multiple times, the stack->keywords for the top stack frame could be overwritten without being freed, causing a memory leak reported by Valgrind. include/my_global.h: Add useful macro for different values when Valgrind (HAVE_purify) and not. mysql-test/extra/rpl_tests/rpl_auto_increment.test: Omit pbxt variables from show variables output. mysql-test/include/have_pbxt.inc: Add facility to disable test if PBXT engine is not available. mysql-test/lib/mtr_report.pm: Give a nice summary report at the end of tests of any warnings seen in logs during server shutdowns. mysql-test/lib/mtr_unique.pm: Move chmod 777 to greatly reduce the risk of leaving the port semaphore file unaccessible bu other users. mysql-test/mysql-test-run.pl: Don't abort in case of warnings detected, instead give a nice summary report. Fix code to make 100% sure no failures will go undetected. Revert earlier wrong change when master disconnects early. mysql-test/r/information_schema.result: Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/r/information_schema_all_engines.result: New file for information_schema tests that depend on which engines are available. mysql-test/r/information_schema_db.result: Move part of test to information_schema_all_engines. mysql-test/r/innodb-autoinc.result: Omit pbxt variables from show variables output. mysql-test/r/mysqlshow.result: Move part of test to information_schema_all_engines. mysql-test/suite/rpl/r/rpl_auto_increment.result: Omit pbxt variables from show variables output. mysql-test/t/information_schema.test: Omit PBXT INFORMATION_SCHEMA table from output. Move part of test to information_schema_all_engines. mysql-test/t/information_schema_all_engines.test: New file for information_schema tests that depend on which engines are available. mysql-test/t/information_schema_db.test: Move part of test to information_schema_all_engines. mysql-test/t/innodb-autoinc.test: Omit pbxt variables from show variables output. mysql-test/t/mysqlshow.test: Move part of test to information_schema_all_engines. mysql-test/valgrind.supp: Add variant suppression (different system library versions). Add suppression for problem with inet_ntoa(). sql/mysqld.cc: Fix missing DBUG_RETURN. Fix uninitialised thd->connect_utime, likely introduced by pool_of_threads. sql/set_var.cc: Fix one-byte buffer overflow in several places. Fix unsafe use of String::c_ptr() of stack-allocated String buffer. sql/sql_select.cc: Silence valgrind warning due to GCC bug. sql/sql_string.h: Document potential problem with String::c_ptr() and String() constructor with caller-supplied buffer. storage/archive/azio.c: Silence Valgrind false warning for libz. --- storage/archive/azio.c | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/azio.c b/storage/archive/azio.c index 59fbe2182ee..6d088f279ac 100644 --- a/storage/archive/azio.c +++ b/storage/archive/azio.c @@ -37,6 +37,40 @@ void putLong(File file, uLong x); uLong getLong(azio_stream *s); void read_header(azio_stream *s, unsigned char *buffer); +/* + Valgrind normally gives false alarms for zlib operations, in the form of + "conditional jump depends on uninitialised values" etc. The reason is + explained in the zlib FAQ (http://www.zlib.net/zlib_faq.html#faq36): + + "That is intentional for performance reasons, and the output of deflate + is not affected." + + Also discussed on a blog + (http://www.sirena.org.uk/log/2006/02/19/zlib-generating-valgrind-warnings/): + + "...loop unrolling in the zlib library causes the mentioned + “Conditional jump or move depends on uninitialised value(s)” + warnings. These are safe since the results of the comparison are + subsequently ignored..." + + "the results of the calculations are discarded by bounds checking done + after the loop exits" + + Fix by initializing the memory allocated by zlib when running under Valgrind. + + This fix is safe, since such memory is only used internally by zlib, so we + will not hide any bugs in mysql this way. +*/ +static void *az_allocator(void *dummy, uInt items, uInt size) +{ + return my_malloc((size_t)items*(size_t)size, IF_PURIFY(MY_ZEROFILL, MYF(0))); +} + +static void az_free(void *dummy, void *address) +{ + my_free(address, MYF(MY_ALLOW_ZERO_PTR)); +} + /* =========================================================================== Opens a gzip (.gz) file for reading or writing. The mode parameter is as in fopen ("rb" or "wb"). The file is given either by file descriptor @@ -52,8 +86,8 @@ int az_open (azio_stream *s, const char *path, int Flags, File fd) int level = Z_DEFAULT_COMPRESSION; /* compression level */ int strategy = Z_DEFAULT_STRATEGY; /* compression strategy */ - s->stream.zalloc = (alloc_func)0; - s->stream.zfree = (free_func)0; + s->stream.zalloc = az_allocator; + s->stream.zfree = az_free; s->stream.opaque = (voidpf)0; memset(s->inbuf, 0, AZ_BUFSIZE_READ); memset(s->outbuf, 0, AZ_BUFSIZE_WRITE); -- cgit v1.2.1 From b125770aaadd09e839ad9211047e88095984308b Mon Sep 17 00:00:00 2001 From: unknown Date: Wed, 6 May 2009 14:03:24 +0200 Subject: We are now using Valgrind rather than purify, and have for quite some time. Consequently, rename HAVE_purify to HAVE_valgrind, and related changes. Leave some comments about purify when not clear that they apply also to Valgrind. Fix redundant IF_VALGRIND declaration. Misc. small fixes: - Fixes for pool-of-threads patch. - Fixes for push of PBXT storage engine. - mysql-test-run.pl fix. - Fix build problem in compile-pentium64-max. BUILD/SETUP.sh: Rename purify -> valgrind. BUILD/build_mccge.sh: Rename purify -> valgrind. BUILD/compile-dist: Fix that PBXT was missing in source tarball after `BUILD/compile-dist && make dist` BUILD/compile-pentium64-max: Fix a build problem with BUILD/compile-pentium64-max on CentOS/Fedora Core 10 amd64. On these systems, there is libz.so but no libz.a. Finding libz.so, ./configure decides to use system zlib. But since BUILD/compile-pentium64-max builds a fully static binary with -all-static, the link of mysqld fails due to missing libz.a. Fix by using bundled zlib in the build script. BUILD/compile-solaris-sparc-purify: Rename purify -> valgrind. include/m_string.h: Rename purify -> valgrind. include/my_global.h: Rename purify -> valgrind. mysql-test/Makefile.am: Fix that PBXT test suite was missing from `make dist` source tarball. mysql-test/lib/mtr_unique.pm: Better fix to avoid races when chmod'ing the semaphore file. (Though using chmod 666 shared files in /tmp/ is still not a very good solution). mysql-test/t/pool_of_threads.cnf: Fix that test case pool_of_threads fails if run on mysqld with no --with-libevent support. mysys/mf_qsort.c: Rename purify -> valgrind. mysys/my_alloc.c: Rename purify -> valgrind. mysys/my_init.c: Rename purify -> valgrind. mysys/my_rnd.c: Rename purify -> valgrind. mysys/safemalloc.c: Rename purify -> valgrind. scripts/mysql_config.pl.in: Rename purify -> valgrind. scripts/mysql_config.sh: Rename purify -> valgrind. sql/field_conv.cc: Rename purify -> valgrind. sql/filesort.cc: Rename purify -> valgrind. sql/ha_partition.cc: Rename purify -> valgrind. sql/hostname.cc: Rename purify -> valgrind. sql/item_timefunc.cc: Rename purify -> valgrind. sql/log_event.cc: Rename purify -> valgrind. sql/log_event_old.cc: Rename purify -> valgrind. sql/my_decimal.h: Rename purify -> valgrind. sql/mysqld.cc: Rename purify -> valgrind. Fix redundant IF_VALGRIND declaration. sql/opt_range.cc: Rename purify -> valgrind. sql/opt_range.h: Rename purify -> valgrind. sql/records.cc: Rename purify -> valgrind. sql/rpl_rli.cc: Rename purify -> valgrind. sql/rpl_rli.h: Rename purify -> valgrind. sql/set_var.cc: Fix missing static declaration on pool_of_threads. sql/slave.cc: Rename purify -> valgrind. sql/sql_base.cc: Rename purify -> valgrind. sql/sql_binlog.cc: Rename purify -> valgrind. sql/sql_class.cc: Rename purify -> valgrind. sql/sql_list.h: Rename purify -> valgrind. sql/sql_load.cc: Rename purify -> valgrind. sql/sql_select.cc: Rename purify -> valgrind. sql/table.cc: Rename purify -> valgrind. storage/archive/azio.c: Rename purify -> valgrind. storage/innobase/buf/buf0buf.c: Rename purify -> valgrind. storage/innobase/include/univ.i: Rename purify -> valgrind. storage/innobase/srv/srv0start.c: Rename purify -> valgrind. storage/maria/ha_maria.cc: Rename purify -> valgrind. storage/maria/ma_blockrec.c: Rename purify -> valgrind. storage/maria/ma_check.c: Rename purify -> valgrind. storage/maria/ma_loghandler.c: Rename purify -> valgrind. storage/maria/ma_packrec.c: Rename purify -> valgrind. storage/maria/ma_page.c: Rename purify -> valgrind. storage/maria/ma_pagecrc.c: Rename purify -> valgrind. storage/maria/ma_search.c: Rename purify -> valgrind. storage/myisam/mi_check.c: Rename purify -> valgrind. storage/myisam/mi_page.c: Rename purify -> valgrind. storage/myisam/mi_search.c: Rename purify -> valgrind. storage/myisammrg/ha_myisammrg.cc: Rename purify -> valgrind. strings/bcmp.c: Rename purify -> valgrind. strings/decimal.c: Rename purify -> valgrind. strings/strmake.c: Rename purify -> valgrind. --- storage/archive/azio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'storage/archive') diff --git a/storage/archive/azio.c b/storage/archive/azio.c index 6d088f279ac..7443c3f0e02 100644 --- a/storage/archive/azio.c +++ b/storage/archive/azio.c @@ -63,7 +63,7 @@ void read_header(azio_stream *s, unsigned char *buffer); */ static void *az_allocator(void *dummy, uInt items, uInt size) { - return my_malloc((size_t)items*(size_t)size, IF_PURIFY(MY_ZEROFILL, MYF(0))); + return my_malloc((size_t)items*(size_t)size, IF_VALGRIND(MY_ZEROFILL, MYF(0))); } static void az_free(void *dummy, void *address) -- cgit v1.2.1 From 48083d73d3dc6b1c678dbb6df3b49f420cce84ad Mon Sep 17 00:00:00 2001 From: unknown Date: Fri, 22 May 2009 14:38:50 +0200 Subject: After-merge fixes for problems seen in buildbot after merging MySQL-5.1.35. - Version number. - Valgrind false alarms in libz. - New variant of suppression for Valgrind warning in dlclose(). - Fix double free() in plugin init error case. configure.in: Fix version number. We should reset the maria variant back to `1' when the MySQL version number increases. include/my_sys.h: Fix false alarms in Valgrind for zlib. Apply same fix as for archive storage handler also to the cases of compression in the client protocol, and to the compression SQL function. mysql-test/valgrind.supp: A new variant of the dlclose() suppression is needed now. mysys/my_compress.c: Fix false alarms in Valgrind for zlib. Apply same fix as for archive storage handler also to the cases of compression in the client protocol, and to the compression SQL function. sql/handler.cc: Fix a double free() in error case for plugin initialisation. sql/item_strfunc.cc: Fix false alarms in Valgrind for zlib. Apply same fix as for archive storage handler also to the cases of compression in the client protocol, and to the compression SQL function. --- storage/archive/azio.c | 40 ++++------------------------------------ 1 file changed, 4 insertions(+), 36 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/azio.c b/storage/archive/azio.c index 6030029ca9a..b6d66fc0739 100644 --- a/storage/archive/azio.c +++ b/storage/archive/azio.c @@ -16,6 +16,8 @@ #include #include +#include "my_sys.h" + static int const gz_magic[2] = {0x1f, 0x8b}; /* gzip magic header */ static int const az_magic[3] = {0xfe, 0x03, 0x01}; /* az magic header */ @@ -37,40 +39,6 @@ void putLong(File file, uLong x); uLong getLong(azio_stream *s); void read_header(azio_stream *s, unsigned char *buffer); -/* - Valgrind normally gives false alarms for zlib operations, in the form of - "conditional jump depends on uninitialised values" etc. The reason is - explained in the zlib FAQ (http://www.zlib.net/zlib_faq.html#faq36): - - "That is intentional for performance reasons, and the output of deflate - is not affected." - - Also discussed on a blog - (http://www.sirena.org.uk/log/2006/02/19/zlib-generating-valgrind-warnings/): - - "...loop unrolling in the zlib library causes the mentioned - “Conditional jump or move depends on uninitialised value(s)” - warnings. These are safe since the results of the comparison are - subsequently ignored..." - - "the results of the calculations are discarded by bounds checking done - after the loop exits" - - Fix by initializing the memory allocated by zlib when running under Valgrind. - - This fix is safe, since such memory is only used internally by zlib, so we - will not hide any bugs in mysql this way. -*/ -static void *az_allocator(void *dummy, uInt items, uInt size) -{ - return my_malloc((size_t)items*(size_t)size, IF_VALGRIND(MY_ZEROFILL, MYF(0))); -} - -static void az_free(void *dummy, void *address) -{ - my_free(address, MYF(MY_ALLOW_ZERO_PTR)); -} - /* =========================================================================== Opens a gzip (.gz) file for reading or writing. The mode parameter is as in fopen ("rb" or "wb"). The file is given either by file descriptor @@ -86,8 +54,8 @@ int az_open (azio_stream *s, const char *path, int Flags, File fd) int level = Z_DEFAULT_COMPRESSION; /* compression level */ int strategy = Z_DEFAULT_STRATEGY; /* compression strategy */ - s->stream.zalloc = az_allocator; - s->stream.zfree = az_free; + s->stream.zalloc = my_az_allocator; + s->stream.zfree = my_az_free; s->stream.opaque = (voidpf)0; memset(s->inbuf, 0, AZ_BUFSIZE_READ); memset(s->outbuf, 0, AZ_BUFSIZE_WRITE); -- cgit v1.2.1 From b96f268acc3a693b673fceccc62311e169496731 Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Wed, 6 Jan 2010 23:27:53 +0200 Subject: Removed compiler warnings Fixed sporadic test failure for suit/pbxt/t/lock_multi.test Fixed sporadic test faulure for suit/rpl/t/do_grant.test OpenSolaris 5.11-x86 now compiles (tested with 32 bit) BUILD/compile-solaris-amd64-debug-forte: Added execute bit BUILD/compile-solaris-x86-32: Added execute bit BUILD/compile-solaris-x86-32-debug: Added execute bit BUILD/compile-solaris-x86-32-debug-forte: Added execute bit BUILD/compile-solaris-x86-forte-32: Added execute bit extra/libevent/devpoll.c: Removed compiler warning extra/libevent/evbuffer.c: Removed compiler warning extra/libevent/select.c: Removed compiler warning mysql-test/mysql-test-run.pl: Fixed sporadic test faulure for suit/rpl/t/do_grant.test (Seen on OpenSolaris) mysql-test/suite/pbxt/r/lock_multi.result: Fixed sporadic test failure for suit/pbxt/t/lock_multi.test (seen in buildbot) This was done by merging the test with main/lock_multi.test mysql-test/suite/pbxt/t/lock_multi.test: Fixed sporadic test failure for suit/pbxt/t/lock_multi.test (seen in buildbot) This was done by merging the test with main/lock_multi.test mysys/my_sync.c: Removed compiler warnings sql/ha_ndbcluster.cc: Fixed linking error on OpenSolaris when compiling without ndb Bug #34866 Can't compile on Solaris 9/Sparc with gcc storage/archive/azlib.h: Removed compiler warning about redefined symbols storage/maria/ma_blockrec.c: Removed compiler warning storage/maria/ma_loghandler.c: Removed compiler warning storage/maria/ma_test3.c: Removed compiler warning storage/myisam/mi_test3.c: Removed compiler warning storage/pbxt/src/ha_pbxt.cc: Removed compiler warning thr_main -> thr_main_pbxt storage/pbxt/src/restart_xt.cc: thr_main -> thr_main_pbxt storage/pbxt/src/thread_xt.cc: thr_main -> thr_main_pbxt This was needed as thr_main() is an internal thread function on OpenSolaris() storage/pbxt/src/thread_xt.h: thr_main -> thr_main_pbxt storage/xtradb/srv/srv0srv.c: Use compatiblity macro to get code to work on OpenSolaris support-files/compiler_warnings.supp: Ignore compiler warning from yassl --- storage/archive/azlib.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/azlib.h b/storage/archive/azlib.h index d7abb40b2ae..29a6329fb0a 100644 --- a/storage/archive/azlib.h +++ b/storage/archive/azlib.h @@ -33,10 +33,9 @@ (zlib format), rfc1951.txt (deflate format) and rfc1952.txt (gzip format). */ -#include - #include "../../mysys/mysys_priv.h" #include +#include #ifdef __cplusplus extern "C" { -- cgit v1.2.1 From 20739646d793d658d1ec6ded46e1d9b90f5d5fb4 Mon Sep 17 00:00:00 2001 From: unknown Date: Thu, 1 Apr 2010 17:34:51 +0300 Subject: Maria WL#61 Interface for maria extensions. Alternative plugin interface with additional info (maturity and string version). CMakeLists.txt: Maria plugin interface used. config/ac-macros/plugins.m4: Maria plugin interface used. configure.in: Maria plugin interface used. include/mysql/plugin.h: Maria plugin interface added. include/mysql/plugin_auth.h.pp: Maria plugin interface added. plugin/auth/auth_socket.c: Maria plugin interface added. plugin/auth/dialog.c: Maria plugin interface added. plugin/daemon_example/daemon_example.cc: Maria plugin interface added. plugin/fulltext/plugin_example.c: Maria plugin interface added. sql/ha_ndbcluster.cc: Maria plugin interface added. sql/ha_partition.cc: Maria plugin interface added. sql/log.cc: Maria plugin interface added. sql/sql_acl.cc: Maria plugin interface added. sql/sql_builtin.cc.in: Maria plugin interface used. sql/sql_plugin.cc: Maria plugin interface added. sql/sql_plugin.h: Maria plugin interface used. sql/sql_show.cc: Maria plugin interface added. storage/archive/ha_archive.cc: Maria plugin interface added. storage/blackhole/ha_blackhole.cc: Maria plugin interface added. storage/csv/ha_tina.cc: Maria plugin interface added. storage/example/ha_example.cc: Maria plugin interface added. storage/federated/ha_federated.cc: Maria plugin interface added. storage/federatedx/ha_federatedx.cc: Maria plugin interface added. storage/heap/ha_heap.cc: Maria plugin interface added. storage/ibmdb2i/ha_ibmdb2i.cc: Maria plugin interface added. storage/innobase/handler/ha_innodb.cc: Maria plugin interface added. storage/innodb_plugin/handler/i_s.cc: Maria plugin interface added. storage/maria/ha_maria.cc: Maria plugin interface added. storage/myisam/ha_myisam.cc: Maria plugin interface added. storage/myisammrg/ha_myisammrg.cc: Maria plugin interface added. storage/pbxt/src/ha_pbxt.cc: Maria plugin interface added. storage/xtradb/handler/ha_innodb.cc: Maria plugin interface added. storage/xtradb/handler/i_s.cc: Maria plugin interface added. storage/xtradb/handler/i_s.h: Maria plugin interface added. --- storage/archive/ha_archive.cc | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) (limited to 'storage/archive') diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index ad54782996c..b5b222d88d1 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -1655,4 +1655,21 @@ mysql_declare_plugin(archive) NULL /* config options */ } mysql_declare_plugin_end; +maria_declare_plugin(archive) +{ + MYSQL_STORAGE_ENGINE_PLUGIN, + &archive_storage_engine, + "ARCHIVE", + "Brian Aker, MySQL AB", + "Archive storage engine", + PLUGIN_LICENSE_GPL, + archive_db_init, /* Plugin Init */ + archive_db_done, /* Plugin Deinit */ + 0x0300 /* 3.0 */, + NULL, /* status variables */ + NULL, /* system variables */ + "1.0", /* string version */ + MariaDB_PLUGIN_MATURITY_STABLE /* maturity */ +} +maria_declare_plugin_end; -- cgit v1.2.1 From ba9d750bca12dfc82b0ea2b8f7f0f80ed104bb7d Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Wed, 21 Jul 2010 02:07:45 +0300 Subject: Fix for LP#588599, mysqlbug#45377: ARCHIVE tables aren't discoverable after OPTIMIZE Fix based on code from Stewart Smith storage/archive/ha_archive.cc: Ensure that the frm definition is stored in optimized tables. --- storage/archive/ha_archive.cc | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index e3151d35e23..97d9f071b1f 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -1290,10 +1290,11 @@ int ha_archive::repair(THD* thd, HA_CHECK_OPT* check_opt) */ int ha_archive::optimize(THD* thd, HA_CHECK_OPT* check_opt) { - DBUG_ENTER("ha_archive::optimize"); int rc= 0; azio_stream writer; char writer_filename[FN_REFLEN]; + char* frm_string; + DBUG_ENTER("ha_archive::optimize"); init_archive_reader(); @@ -1304,12 +1305,28 @@ int ha_archive::optimize(THD* thd, HA_CHECK_OPT* check_opt) share->archive_write_open= FALSE; } + if (!(frm_string= (char*) malloc(archive.frm_length))) + return ENOMEM; + + azread_frm(&archive, frm_string); + /* Lets create a file to contain the new data */ fn_format(writer_filename, share->table_name, "", ARN, MY_REPLACE_EXT | MY_UNPACK_FILENAME); if (!(azopen(&writer, writer_filename, O_CREAT|O_RDWR|O_BINARY))) - DBUG_RETURN(HA_ERR_CRASHED_ON_USAGE); + { + free(frm_string); + DBUG_RETURN(HA_ERR_CRASHED_ON_USAGE); + } + + rc= azwrite_frm(&writer, frm_string, archive.frm_length); + free(frm_string); + if (rc) + { + rc= HA_ERR_CRASHED_ON_USAGE; + goto error; + } /* An extended rebuild is a lot more effort. We open up each row and re-record it. @@ -1387,7 +1404,6 @@ int ha_archive::optimize(THD* thd, HA_CHECK_OPT* check_opt) // make the file we just wrote be our data file rc = my_rename(writer_filename,share->data_file_name,MYF(0)); - DBUG_RETURN(rc); error: DBUG_PRINT("ha_archive", ("Failed to recover, error was %d", rc)); -- cgit v1.2.1 From 069a068c90d8fa6ccb8028f3f2cd42c238b04699 Mon Sep 17 00:00:00 2001 From: Sergei Golubchik Date: Fri, 23 Jul 2010 22:37:21 +0200 Subject: restore the unintentinally broken ABI --- storage/archive/ha_archive.cc | 2 +- storage/archive/ha_archive.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index fd0993c9914..988337ec50e 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -1539,7 +1539,7 @@ void ha_archive::start_bulk_insert(ha_rows rows) Other side of start_bulk_insert, is end_bulk_insert. Here we turn off the bulk insert flag, and set the share dirty so that the next select will call sync for us. */ -int ha_archive::end_bulk_insert(bool table_will_be_deleted) +int ha_archive::end_bulk_insert() { DBUG_ENTER("ha_archive::end_bulk_insert"); bulk_insert= FALSE; diff --git a/storage/archive/ha_archive.h b/storage/archive/ha_archive.h index 22f8302982d..ab630ed22fd 100644 --- a/storage/archive/ha_archive.h +++ b/storage/archive/ha_archive.h @@ -134,7 +134,7 @@ public: int optimize(THD* thd, HA_CHECK_OPT* check_opt); int repair(THD* thd, HA_CHECK_OPT* check_opt); void start_bulk_insert(ha_rows rows); - int end_bulk_insert(bool table_will_be_deleted); + int end_bulk_insert(); enum row_type get_row_type() const { return ROW_TYPE_COMPRESSED; -- cgit v1.2.1 From 9f85560212241f07b7bdff7df639c0b9a519bed5 Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Fri, 3 Sep 2010 20:55:56 +0300 Subject: Enable archive tables to work with mysql_upgrade / repair Made long file names from previous patch shorter mysql-test/r/archive.result: Added testing of repair (for upgrade) of 5.0 tables. mysql-test/std_data/archive_5_0.ARM: Archive table created in MySQL 5.0 mysql-test/std_data/archive_5_0.ARZ: Archive table created in MySQL 5.0 mysql-test/std_data/archive_5_0.frm: Archive table created in MySQL 5.0 mysql-test/std_data/long_table_name.MYD: Made long file names shorter mysql-test/std_data/long_table_name.MYI: Made long file names shorter mysql-test/std_data/long_table_name.frm: Made long file names shorter mysql-test/t/archive.test: Added testing of repair (for upgrade) of 5.0 tables. sql/sql_table.cc: Allow recreate to open crashed tables. sql/table.cc: Fix error message if storage engine doesn't exists. storage/archive/azio.c: Reset status values in case archive is of old versions storage/archive/ha_archive.cc: Fix to allow one to open old versions of table during repair Reset status variables for old version tables If the the table is of old version, force upgrade with ALTER TABLE when doing repair storage/archive/ha_archive.h: Added variables to detect old versions --- storage/archive/azio.c | 11 +++++++++++ storage/archive/ha_archive.cc | 42 +++++++++++++++++++++++++++++++----------- storage/archive/ha_archive.h | 3 ++- 3 files changed, 44 insertions(+), 12 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/azio.c b/storage/archive/azio.c index a24350dd454..fc54d98ab15 100644 --- a/storage/archive/azio.c +++ b/storage/archive/azio.c @@ -150,6 +150,17 @@ int az_open (azio_stream *s, const char *path, int Flags, File fd) } else { + /* Reset values in case of old version of archive file */ + s->rows= 0; + s->forced_flushes= 0; + s->shortest_row= 0; + s->longest_row= 0; + s->auto_increment= 0; + s->check_point= 0; + s->comment_start_pos= 0; + s->comment_length= 0; + s->frm_start_pos= 0; + s->frm_length= 0; check_header(s); /* skip the .az header */ } diff --git a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc index 988337ec50e..d0cb34455b9 100644 --- a/storage/archive/ha_archive.cc +++ b/storage/archive/ha_archive.cc @@ -293,7 +293,7 @@ int ha_archive::read_data_header(azio_stream *file_to_read) DBUG_PRINT("ha_archive", ("Version %u", data_buffer[1])); if ((data_buffer[0] != (uchar)ARCHIVE_CHECK_HEADER) && - (data_buffer[1] != (uchar)ARCHIVE_VERSION)) + (data_buffer[1] == 1 || data_buffer[1] == 2)) DBUG_RETURN(HA_ERR_CRASHED_ON_USAGE); DBUG_RETURN(0); @@ -360,9 +360,19 @@ ARCHIVE_SHARE *ha_archive::get_share(const char *table_name, int *rc) my_free(share, MYF(0)); DBUG_RETURN(NULL); } - stats.auto_increment_value= archive_tmp.auto_increment + 1; - share->rows_recorded= (ha_rows)archive_tmp.rows; - share->crashed= archive_tmp.dirty; + share->version= archive_tmp.version; + if (archive_tmp.version == ARCHIVE_VERSION) + { + stats.auto_increment_value= archive_tmp.auto_increment + 1; + share->rows_recorded= (ha_rows)archive_tmp.rows; + share->crashed= archive_tmp.dirty; + } + else + { + /* Used by repair */ + share->rows_recorded= ~(ha_rows) 0; + stats.auto_increment_value= 0; + } /* If archive version is less than 3, It should be upgraded before use. @@ -512,10 +522,19 @@ int ha_archive::open(const char *name, int mode, uint open_options) case 0: break; case HA_ERR_CRASHED_ON_USAGE: + DBUG_PRINT("ha_archive", ("archive table was crashed")); if (open_options & HA_OPEN_FOR_REPAIR) + { + rc= 0; break; + } /* fall through */ case HA_ERR_TABLE_NEEDS_UPGRADE: + if (open_options & HA_OPEN_FOR_REPAIR) + { + rc= 0; + break; + } free_share(); /* fall through */ default: @@ -535,13 +554,6 @@ int ha_archive::open(const char *name, int mode, uint open_options) thr_lock_data_init(&share->lock, &lock, NULL); - DBUG_PRINT("ha_archive", ("archive table was crashed %s", - rc == HA_ERR_CRASHED_ON_USAGE ? "yes" : "no")); - if (rc == HA_ERR_CRASHED_ON_USAGE && open_options & HA_OPEN_FOR_REPAIR) - { - DBUG_RETURN(0); - } - DBUG_RETURN(rc); } @@ -1267,6 +1279,14 @@ int ha_archive::rnd_pos(uchar * buf, uchar *pos) DBUG_RETURN(get_row(&archive, buf)); } +int ha_archive::check_for_upgrade(HA_CHECK_OPT *check_opt) +{ + if (share->version < ARCHIVE_VERSION) + return HA_ADMIN_NEEDS_ALTER; + return 0; +} + + /* This method repairs the meta file. It does this by walking the datafile and rewriting the meta file. If EXTENDED repair is requested, we attempt to diff --git a/storage/archive/ha_archive.h b/storage/archive/ha_archive.h index ab630ed22fd..653a13b242d 100644 --- a/storage/archive/ha_archive.h +++ b/storage/archive/ha_archive.h @@ -35,7 +35,7 @@ typedef struct st_archive_record_buffer { typedef struct st_archive_share { char *table_name; char data_file_name[FN_REFLEN]; - uint table_name_length,use_count; + uint table_name_length,use_count, version; pthread_mutex_t mutex; THR_LOCK lock; azio_stream archive_write; /* Archive file we are working with */ @@ -133,6 +133,7 @@ public: int create(const char *name, TABLE *form, HA_CREATE_INFO *create_info); int optimize(THD* thd, HA_CHECK_OPT* check_opt); int repair(THD* thd, HA_CHECK_OPT* check_opt); + int check_for_upgrade(HA_CHECK_OPT *check_opt); void start_bulk_insert(ha_rows rows); int end_bulk_insert(); enum row_type get_row_type() const -- cgit v1.2.1 From 44fec70b7968778808673d9668c5fe08da03a7ca Mon Sep 17 00:00:00 2001 From: Sergei Golubchik Date: Thu, 9 Sep 2010 15:35:47 +0200 Subject: build dynamic plugins with the -shared libtool option to avoid double compilation --- storage/archive/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'storage/archive') diff --git a/storage/archive/Makefile.am b/storage/archive/Makefile.am index 94e98c468ad..254c95bf68b 100644 --- a/storage/archive/Makefile.am +++ b/storage/archive/Makefile.am @@ -36,8 +36,8 @@ noinst_PROGRAMS = archive_test archive_reader EXTRA_LTLIBRARIES = ha_archive.la pkgplugin_LTLIBRARIES = @plugin_archive_shared_target@ ha_archive_la_LDFLAGS = -module -rpath $(pkgplugindir) -ha_archive_la_CXXFLAGS= $(AM_CXXFLAGS) -DMYSQL_DYNAMIC_PLUGIN -ha_archive_la_CFLAGS = $(AM_CFLAGS) -DMYSQL_DYNAMIC_PLUGIN +ha_archive_la_CXXFLAGS= -shared $(AM_CXXFLAGS) -DMYSQL_DYNAMIC_PLUGIN +ha_archive_la_CFLAGS = -shared $(AM_CFLAGS) -DMYSQL_DYNAMIC_PLUGIN ha_archive_la_SOURCES = ha_archive.cc azio.c -- cgit v1.2.1