summaryrefslogtreecommitdiff
path: root/storage
diff options
context:
space:
mode:
authorunknown <istruewing@stella.local>2007-11-15 20:25:43 +0100
committerunknown <istruewing@stella.local>2007-11-15 20:25:43 +0100
commitc8450b278d7f049e0afa49624c52e0f61b541126 (patch)
tree69cf3030a364a25215179c5f289ef3804553bfc8 /storage
parent62d3c3d3a1c7ceed7170913189dd0e4437f2d37e (diff)
downloadmariadb-git-c8450b278d7f049e0afa49624c52e0f61b541126.tar.gz
Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE
corrupts a MERGE table Bug 26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging Bug 26377 - Deadlock with MERGE and FLUSH TABLE Bug 25038 - Waiting TRUNCATE Bug 25700 - merge base tables get corrupted by optimize/analyze/repair table Bug 30275 - Merge tables: flush tables or unlock tables causes server to crash Bug 19627 - temporary merge table locking Bug 27660 - Falcon: merge table possible Bug 30273 - merge tables: Can't lock file (errno: 155) The problems were: Bug 26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table 1. A thread trying to lock a MERGE table performs busy waiting while REPAIR TABLE or a similar table administration task is ongoing on one or more of its MyISAM tables. 2. A thread trying to lock a MERGE table performs busy waiting until all threads that did REPAIR TABLE or similar table administration tasks on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK TABLES. The difference against problem #1 is that the busy waiting takes place *after* the administration task. It is terminated by UNLOCK TABLES only. 3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the lock. This does *not* require a MERGE table. The first FLUSH TABLES can be replaced by any statement that requires other threads to reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke the problem. Bug 26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging Trying DML on a MERGE table, which has a child locked and repaired by another thread, made an infinite loop in the server. Bug 26377 - Deadlock with MERGE and FLUSH TABLE Locking a MERGE table and its children in parent-child order and flushing the child deadlocked the server. Bug 25038 - Waiting TRUNCATE Truncating a MERGE child, while the MERGE table was in use, let the truncate fail instead of waiting for the table to become free. Bug 25700 - merge base tables get corrupted by optimize/analyze/repair table Repairing a child of an open MERGE table corrupted the child. It was necessary to FLUSH the child first. Bug 30275 - Merge tables: flush tables or unlock tables causes server to crash Flushing and optimizing locked MERGE children crashed the server. Bug 19627 - temporary merge table locking Use of a temporary MERGE table with non-temporary children could corrupt the children. Temporary tables are never locked. So we do now prohibit non-temporary chidlren of a temporary MERGE table. Bug 27660 - Falcon: merge table possible It was possible to create a MERGE table with non-MyISAM children. Bug 30273 - merge tables: Can't lock file (errno: 155) This was a Windows-only bug. Table administration statements sometimes failed with "Can't lock file (errno: 155)". These bugs are fixed by a new implementation of MERGE table open. When opening a MERGE table in open_tables() we do now add the child tables to the list of tables to be opened by open_tables() (the "query_list"). The children are not opened in the handler at this stage. After opening the parent, open_tables() opens each child from the now extended query_list. When the last child is opened, we remove the children from the query_list again and attach the children to the parent. This behaves similar to the old open. However it does not open the MyISAM tables directly, but grabs them from the already open children. When closing a MERGE table in close_thread_table() we detach the children only. Closing of the children is done implicitly because they are in thd->open_tables. For more detail see the comment at the top of ha_myisammrg.cc. Changed from open_ltable() to open_and_lock_tables() in all places that can be relevant for MERGE tables. The latter can handle tables added to the list on the fly. When open_ltable() was used in a loop over a list of tables, the list must be temporarily terminated after every table for open_and_lock_tables(). table_list->required_type is set to FRMTYPE_TABLE to avoid open of special tables. Handling of derived tables is suppressed. These details are handled by the new function open_n_lock_single_table(), which has nearly the same signature as open_ltable() and can replace it in most cases. In reopen_tables() some of the tables open by a thread can be closed and reopened. When a MERGE child is affected, the parent must be closed and reopened too. Closing of the parent is forced before the first child is closed. Reopen happens in the order of thd->open_tables. MERGE parents do not attach their children automatically at open. This is done after all tables are reopened. So all children are open when attaching them. Special lock handling like mysql_lock_abort() or mysql_lock_remove() needs to be suppressed for MERGE children or forwarded to the parent. This depends on the situation. In loops over all open tables one suppresses child lock handling. When a single table is touched, forwarding is done. Behavioral changes: =================== This patch changes the behavior of temporary MERGE tables. Temporary MERGE must have temporary children. The old behavior was wrong. A temporary table is not locked. Hence even non-temporary children were not locked. See Bug 19627 - temporary merge table locking. You cannot change the union list of a non-temporary MERGE table when LOCK TABLES is in effect. The following does *not* work: CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...; LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE; ALTER TABLE m1 ... UNION=(t1,t2) ...; However, you can do this with a temporary MERGE table. You cannot create a MERGE table with CREATE ... SELECT, neither as a temporary MERGE table, nor as a non-temporary MERGE table. CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...; Gives error message: table is not BASE TABLE. include/my_base.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN. include/myisammrg.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added element 'children_attached' to MYRG_INFO. Added declarations for myrg_parent_open(), myrg_attach_children() and myrg_detach_children() for the new MERGE table open approach. mysql-test/extra/binlog_tests/blackhole.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Preliminarily added new error message with a comment. mysql-test/r/create.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test result. mysql-test/r/delayed.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test result from here to merge.result. mysql-test/r/merge.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed/added test result. mysql-test/r/myisam.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test result for bug 8306 from here to merge.result. mysql-test/suite/binlog/r/binlog_stm_blackhole.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test result. mysql-test/t/create.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed error number. mysql-test/t/delayed.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test from here to merge.test. mysql-test/t/merge.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test for new temporary MERGE table behavior. Exchanged error numbers by symbolic codes. Added tests. Included are tests for bugs 8306 (moved from myisam.test), 26379, 19627, 25038, 25700, 26377, 26867, 27660, 30275, and 30273. Fixed changes resulting from disabled CREATE...SELECT. Integrated tests moved from delayed.test and myisam.test to here. mysql-test/t/myisam.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test for bug 8306 from here to merge.test. mysys/thr_lock.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added code to let the owner of a high priority lock (TL_WRITE_ONLY) to bypass its own lock. sql/ha_ndbcluster_binlog.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/handler.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/mysql_priv.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Removed declaration of check_merge_table_access(). It is now static in sql_parse.cc. Added declaration for fix_merge_after_open(). Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. Added inline functions simple_open_n_lock_tables() and open_and_lock_tables(), which call open_and_lock_tables_derived() and add the argument for 'derived'. Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Added 'thd' argument to init_tmp_table_share(). sql/slave.cc: ug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added comment. sql/sql_base.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Defined new functions add_merge_table_list(), attach_merge_children(), detach_merge_children(), and fix_merge_after_open() for the new MERGE table open approach. Added calls of the new functions to close_handle_and_leave_table_as_lock(), close_thread_tables(), close_thread_table(), unlink_open_table(), reopen_name_locked_table(), reopen_table(), drop_locked_tables(), close_temporary_table(), and open_tables() respectively. Prevented special lock handling of merge children (like mysql_lock_remove, mysql_lock_merge or mysql_lock_abort) at many places. Some of these calls are forwarded to the parent table instead. Added code to set thd->some_tables_deleted for every thread that has a table open that we are flushing. Added code for MERGE tables to unlink_open_table(). Added MERGE children to the list of unusable tables in open_table(). Added MERGE table handling to reopen_table(). Added lock handling and closing of a parent before the children in close_data_files_and_morph_locks(). Added code for re-attaching children in reopen_tables(). Added MYSQL_LOCK_NOTIFY_IF_NEED_REOPEN to the locking flags and error reporting after mysql_lock_tables() in reopen_tables(). Added lock handling and closing of a parent before the children in close_old_data_files(). Added lock handling and detaching in drop_locked_tables(). Added code for removing the children list from the statement list to prepare for a repetition in open_tables(). Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Disabled use of open_ltable() for MERGE tables. Removed function simple_open_n_lock_tables(). It is now inline declared in mysql_priv.h. Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. open_and_lock_tables() is now inline declared in mysql_priv.h. Added a check for end-of-list in two loops in lock_tables(). Added 'thd' argument to init_tmp_table_share(). sql/sql_insert.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table() in handle_delayed_insert(). Reestablished LEX settings after lex initialization. Added 'thd' argument to init_tmp_table_share(). sql/sql_parse.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Made check_merge_table_access() a static function. Disabled use of CREATE...SELECT for MERGE tables. sql/sql_partition.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed comment typo. sql/sql_select.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/sql_table.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Optimized use of mysql_ha_flush() in mysql_rm_table_part2(). Disabled the use of MERGE tables with prepare_for_restore() and prepare_for_repair(). Changed from open_ltable() to open_n_lock_single_table() in mysql_alter_table() and mysql_checksum_table(). Disabled change of child list under LOCK TABLES. Initialized table_list->table in mysql_recreate_table(). sql/sql_trigger.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added code for allowing CREATE TRIGGER under LOCK TABLE, to be able to test it with MERGE tables. sql/table.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). Setting table_map_id from query_id in init_tmp_table_share(). Added member function TABLE::is_children_attached(). sql/table.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added access method get_table_def_version() to TABLE_SHARE. Added elements for MERGE tables to TABLE and TABLE_LIST. storage/myisam/ha_myisam.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added an unrelated comment to the function comment of table2myisam(). storage/myisam/ha_myisam.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added new member function MI_INFO::file_ptr(). storage/myisammrg/ha_myisammrg.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added callback functions to support parent open and children attach of MERGE tables. Changed ha_myisammrg::open() to initialize storage engine structures and create a list of child tables only. Child tables are not opened. Added ha_myisammrg::attach_children(), which does now the main part of MERGE open. Added ha_myisammrg::detach_children(). Added calls to ::attach_children() and ::detach_children() to ::extra() on HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN respectively. Added a check for matching TEMPORARY type for children against parent. Added a check for table def version. Added support for thd->open_options to attach_children(). Changed child path name generation for temporary tables so that it does nothing special for temporary tables. storage/myisammrg/ha_myisammrg.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added elements to class ha_myisammrg to support the new open approach. Changed empty destructor definition to a declaration. Implemented in ha_myisammrg.cc. Added declaration for methods attach_children() and detach_children(). Added definition for method table_ptr() for use with callback functions. storage/myisammrg/myrg_close.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added a check to avoid closing of MyISAM tables when the child tables are not attached. Added freeing of rec_per_key_part when the child tables are not attached. storage/myisammrg/myrg_extra.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Some ::extra() functions and ::reset() can be called when children are detached. storage/myisammrg/myrg_open.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Kept old myrg_open() for MERGE use independent from MySQL. Removed an always true condition in myrg_open(). Set children_attached for independent MERGE use in myrg_open(). Added myrg_parent_open(), myrg_attach_children(), and myrg_detach_children() for the new MERGE table open approach. mysql-test/r/merge-big.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table New test result mysql-test/t/merge-big.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table New test case
Diffstat (limited to 'storage')
-rw-r--r--storage/myisam/ha_myisam.cc4
-rw-r--r--storage/myisam/ha_myisam.h4
-rw-r--r--storage/myisammrg/ha_myisammrg.cc668
-rw-r--r--storage/myisammrg/ha_myisammrg.h9
-rw-r--r--storage/myisammrg/myrg_close.c34
-rw-r--r--storage/myisammrg/myrg_extra.c4
-rw-r--r--storage/myisammrg/myrg_open.c334
7 files changed, 956 insertions, 101 deletions
diff --git a/storage/myisam/ha_myisam.cc b/storage/myisam/ha_myisam.cc
index ca4c40547ee..dcedcf03f65 100644
--- a/storage/myisam/ha_myisam.cc
+++ b/storage/myisam/ha_myisam.cc
@@ -119,6 +119,10 @@ static void mi_check_print_msg(MI_CHECK *param, const char* msg_type,
definition for further use in mi_create or for a check for underlying
table conformance in merge engine.
+ The caller needs to free *recinfo_out after use. Since *recinfo_out
+ and *keydef_out are allocated with a my_multi_malloc, *keydef_out
+ is freed automatically when *recinfo_out is freed.
+
RETURN VALUE
0 OK
!0 error code
diff --git a/storage/myisam/ha_myisam.h b/storage/myisam/ha_myisam.h
index e8594fc9039..ca44ae9ad87 100644
--- a/storage/myisam/ha_myisam.h
+++ b/storage/myisam/ha_myisam.h
@@ -140,4 +140,8 @@ class ha_myisam: public handler
*engine_callback,
ulonglong *engine_data);
#endif
+ MI_INFO *file_ptr(void)
+ {
+ return file;
+ }
};
diff --git a/storage/myisammrg/ha_myisammrg.cc b/storage/myisammrg/ha_myisammrg.cc
index 8a914e8a2de..f91b0dc7a92 100644
--- a/storage/myisammrg/ha_myisammrg.cc
+++ b/storage/myisammrg/ha_myisammrg.cc
@@ -14,6 +14,82 @@
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+/*
+ MyISAM MERGE tables
+
+ A MyISAM MERGE table is kind of a union of zero or more MyISAM tables.
+
+ Besides the normal form file (.frm) a MERGE table has a meta file
+ (.MRG) with a list of tables. These are paths to the MyISAM table
+ files. The last two components of the path contain the database name
+ and the table name respectively.
+
+ When a MERGE table is open, there exists an TABLE object for the MERGE
+ table itself and a TABLE object for each of the MyISAM tables. For
+ abbreviated writing, I call the MERGE table object "parent" and the
+ MyISAM table objects "children".
+
+ A MERGE table is almost always opened through open_and_lock_tables()
+ and hence through open_tables(). When the parent appears in the list
+ of tables to open, the initial open of the handler does nothing but
+ read the meta file and collect a list of TABLE_LIST objects for the
+ children. This list is attached to the parent TABLE object as
+ TABLE::child_l. The end of the children list is saved in
+ TABLE::child_last_l.
+
+ Back in open_tables(), add_merge_table_list() is called. It updates
+ each list member with the lock type and a back pointer to the parent
+ TABLE_LIST object TABLE_LIST::parent_l. The list is then inserted in
+ the list of tables to open, right behind the parent. Consequently,
+ open_tables() opens the children, one after the other. The TABLE
+ references of the TABLE_LIST objects are implicitly set to the open
+ tables. The children are opened as independent MyISAM tables, right as
+ if they are used by the SQL statement.
+
+ TABLE_LIST::parent_l is required to find the parent 1. when the last
+ child has been opened and children are to be attached, and 2. when an
+ error happens during child open and the child list must be removed
+ from the queuery list. In these cases the current child does not have
+ TABLE::parent set or does not have a TABLE at all respectively.
+
+ When the last child is open, attach_merge_children() is called. It
+ removes the list of children from the open list. Then the children are
+ "attached" to the parent. All required references between parent and
+ children are set up.
+
+ The MERGE storage engine sets up an array with references to the
+ low-level MyISAM table objects (MI_INFO). It remembers the state of
+ the table in MYRG_INFO::children_attached.
+
+ Every child TABLE::parent references the parent TABLE object. That way
+ TABLE objects belonging to a MERGE table can be identified.
+ TABLE::parent is required because the parent and child TABLE objects
+ can live longer than the parent TABLE_LIST object. So the path
+ child->pos_in_table_list->parent_l->table can be broken.
+
+ If necessary, the compatibility of parent and children is checked.
+ This check is necessary when any of the objects are reopend. This is
+ detected by comparing the current table def version against the
+ remembered child def version. On parent open, the list members are
+ initialized to an "impossible"/"undefined" version value. So the check
+ is always executed on the first attach.
+
+ The version check is done in myisammrg_attach_children_callback(),
+ which is called for every child. ha_myisammrg::attach_children()
+ initializes 'need_compat_check' to FALSE and
+ myisammrg_attach_children_callback() sets it ot TRUE if a table
+ def version mismatches the remembered child def version.
+
+ Finally the parent TABLE::children_attached is set.
+
+ ---
+
+ On parent open the storage engine structures are allocated and initialized.
+ They stay with the open table until its final close.
+
+
+*/
+
#ifdef USE_PRAGMA_IMPLEMENTATION
#pragma implementation // gcc: Class implementation
#endif
@@ -22,14 +98,11 @@
#include "mysql_priv.h"
#include <mysql/plugin.h>
#include <m_ctype.h>
+#include "../myisam/ha_myisam.h"
#include "ha_myisammrg.h"
#include "myrg_def.h"
-/*****************************************************************************
-** MyISAM MERGE tables
-*****************************************************************************/
-
static handler *myisammrg_create_handler(handlerton *hton,
TABLE_SHARE *table,
MEM_ROOT *mem_root)
@@ -38,10 +111,23 @@ static handler *myisammrg_create_handler(handlerton *hton,
}
+/**
+ @brief Constructor
+*/
+
ha_myisammrg::ha_myisammrg(handlerton *hton, TABLE_SHARE *table_arg)
:handler(hton, table_arg), file(0)
{}
+
+/**
+ @brief Destructor
+*/
+
+ha_myisammrg::~ha_myisammrg(void)
+{}
+
+
static const char *ha_myisammrg_exts[] = {
".MRG",
NullS
@@ -89,25 +175,311 @@ const char *ha_myisammrg::index_type(uint key_number)
}
-int ha_myisammrg::open(const char *name, int mode, uint test_if_locked)
+/**
+ @brief Callback function for open of a MERGE parent table.
+
+ @detail This function adds a TABLE_LIST object for a MERGE child table
+ to a list of tables of the parent TABLE object. It is called for
+ each child table.
+
+ The list of child TABLE_LIST objects is kept in the TABLE object of
+ the parent for the whole life time of the MERGE table. It is
+ inserted in the statement list behind the MERGE parent TABLE_LIST
+ object when the MERGE table is opened. It is removed from the
+ statement list after the last child is opened.
+
+ All memeory used for the child TABLE_LIST objects and the strings
+ referred by it are taken from the parent TABLE::mem_root. Thus they
+ are all freed implicitly at the final close of the table.
+
+ TABLE::child_l -> TABLE_LIST::next_global -> TABLE_LIST::next_global
+ # # ^ # ^
+ # # | # |
+ # # +--------- TABLE_LIST::prev_global
+ # # |
+ # |<--- TABLE_LIST::prev_global |
+ # |
+ TABLE::child_last_l -----------------------------------------+
+
+ @param[in] callback_param data pointer as given to myrg_parent_open()
+ @param[in] filename file name of MyISAM table
+ without extension.
+
+ @return status
+ @retval 0 OK
+ @retval != 0 Error
+*/
+
+static int myisammrg_parent_open_callback(void *callback_param,
+ const char *filename)
+{
+ ha_myisammrg *ha_myrg;
+ TABLE *parent;
+ TABLE_LIST *child_l;
+ const char *db;
+ const char *table_name;
+ uint dirlen;
+ char dir_path[FN_REFLEN];
+ DBUG_ENTER("myisammrg_parent_open_callback");
+
+ /* Extract child table name and database name from filename. */
+ dirlen= dirname_length(filename);
+ if (dirlen >= FN_REFLEN)
+ {
+ /* purecov: begin inspected */
+ DBUG_PRINT("error", ("name too long: '%.64s'", filename));
+ my_errno= ENAMETOOLONG;
+ DBUG_RETURN(1);
+ /* purecov: end */
+ }
+ table_name= filename + dirlen;
+ dirlen--; /* Strip off trailing '/'. */
+ memcpy(dir_path, filename, dirlen);
+ dir_path[dirlen]= '\0';
+ db= base_name(dir_path);
+ dirlen-= db - dir_path; /* This is now the length of 'db'. */
+ DBUG_PRINT("myrg", ("open: '%s'.'%s'", db, table_name));
+
+ ha_myrg= (ha_myisammrg*) callback_param;
+ parent= ha_myrg->table_ptr();
+
+ /* Get a TABLE_LIST object. */
+ if (!(child_l= (TABLE_LIST*) alloc_root(&parent->mem_root,
+ sizeof(TABLE_LIST))))
+ {
+ /* purecov: begin inspected */
+ DBUG_PRINT("error", ("my_malloc error: %d", my_errno));
+ DBUG_RETURN(1);
+ /* purecov: end */
+ }
+ bzero((char*) child_l, sizeof(TABLE_LIST));
+
+ /* Set database (schema) name. */
+ child_l->db_length= dirlen;
+ child_l->db= strmake_root(&parent->mem_root, db, dirlen);
+ /* Set table name. */
+ child_l->table_name_length= strlen(table_name);
+ child_l->table_name= strmake_root(&parent->mem_root, table_name,
+ child_l->table_name_length);
+ /* Convert to lowercase if required. */
+ if (lower_case_table_names && child_l->table_name_length)
+ child_l->table_name_length= my_casedn_str(files_charset_info,
+ child_l->table_name);
+ /* Set alias. */
+ child_l->alias= child_l->table_name;
+
+ /* Initialize table map to 'undefined'. */
+ child_l->init_child_def_version();
+
+ /* Link TABLE_LIST object into the parent list. */
+ if (!parent->child_last_l)
+ {
+ /* Initialize parent->child_last_l when handling first child. */
+ parent->child_last_l= &parent->child_l;
+ }
+ *parent->child_last_l= child_l;
+ child_l->prev_global= parent->child_last_l;
+ parent->child_last_l= &child_l->next_global;
+
+ DBUG_RETURN(0);
+}
+
+
+/**
+ @brief Callback function for attaching a MERGE child table.
+
+ @detail This function retrieves the MyISAM table handle from the
+ next child table. It is called for each child table.
+
+ @param[in] callback_param data pointer as given to
+ myrg_attach_children()
+
+ @return pointer to open MyISAM table structure
+ @retval !=NULL OK, returning pointer
+ @retval NULL, my_errno == 0 Ok, no more child tables
+ @retval NULL, my_errno != 0 error
+*/
+
+static MI_INFO *myisammrg_attach_children_callback(void *callback_param)
+{
+ ha_myisammrg *ha_myrg;
+ TABLE *parent;
+ TABLE *child;
+ TABLE_LIST *child_l;
+ MI_INFO *myisam;
+ DBUG_ENTER("myisammrg_attach_children_callback");
+
+ my_errno= 0;
+ ha_myrg= (ha_myisammrg*) callback_param;
+ parent= ha_myrg->table_ptr();
+
+ /* Get child list item. */
+ child_l= ha_myrg->next_child_attach;
+ if (!child_l)
+ {
+ DBUG_PRINT("myrg", ("No more children to attach"));
+ DBUG_RETURN(NULL);
+ }
+ child= child_l->table;
+ DBUG_PRINT("myrg", ("child table: '%s'.'%s' 0x%lx", child->s->db.str,
+ child->s->table_name.str, (long) child));
+ /*
+ Prepare for next child. Used as child_l in next call to this function.
+ We cannot rely on a NULL-terminated chain.
+ */
+ if (&child_l->next_global == parent->child_last_l)
+ {
+ DBUG_PRINT("myrg", ("attaching last child"));
+ ha_myrg->next_child_attach= NULL;
+ }
+ else
+ ha_myrg->next_child_attach= child_l->next_global;
+
+ /* Set parent reference. */
+ child->parent= parent;
+
+ /*
+ Do a quick compatibility check. The table def version is set when
+ the table share is created. The child def version is copied
+ from the table def version after a sucessful compatibility check.
+ We need to repeat the compatibility check only if a child is opened
+ from a different share than last time it was used with this MERGE
+ table.
+ */
+ DBUG_PRINT("myrg", ("table_def_version last: %lu current: %lu",
+ (ulong) child_l->get_child_def_version(),
+ (ulong) child->s->get_table_def_version()));
+ if (child_l->get_child_def_version() != child->s->get_table_def_version())
+ ha_myrg->need_compat_check= TRUE;
+
+ /*
+ If parent is temporary, children must be temporary too and vice
+ versa. This check must be done for every child on every open because
+ the table def version can overlap between temporary and
+ non-temporary tables. We need to detect the case where a
+ non-temporary table has been replaced with a temporary table of the
+ same version. Or vice versa. A very unlikely case, but it could
+ happen.
+ */
+ if (child->s->tmp_table != parent->s->tmp_table)
+ {
+ DBUG_PRINT("error", ("temporary table mismatch parent: %d child: %d",
+ parent->s->tmp_table, child->s->tmp_table));
+ my_errno= HA_ERR_WRONG_MRG_TABLE_DEF;
+ goto err;
+ }
+
+ /* Extract the MyISAM table structure pointer from the handler object. */
+ if ((child->file->ht->db_type != DB_TYPE_MYISAM) ||
+ !(myisam= ((ha_myisam*) child->file)->file_ptr()))
+ {
+ DBUG_PRINT("error", ("no MyISAM handle for child table: '%s'.'%s' 0x%lx",
+ child->s->db.str, child->s->table_name.str,
+ (long) child));
+ my_errno= HA_ERR_WRONG_MRG_TABLE_DEF;
+ }
+ DBUG_PRINT("myrg", ("MyISAM handle: 0x%lx my_errno: %d",
+ (long) myisam, my_errno));
+
+ err:
+ DBUG_RETURN(my_errno ? NULL : myisam);
+}
+
+
+/**
+ @brief Open a MERGE parent table, not its children.
+
+ @detail This function initializes the MERGE storage engine structures
+ and adds a child list of TABLE_LIST to the parent TABLE.
+
+ @param[in] name MERGE table path name
+ @param[in] mode read/write mode, unused
+ @param[in] test_if_locked open flags
+
+ @return status
+ @retval 0 OK
+ @retval -1 Error, my_errno gives reason
+*/
+
+int ha_myisammrg::open(const char *name, int mode __attribute__((unused)),
+ uint test_if_locked)
+{
+ DBUG_ENTER("ha_myisammrg::open");
+ DBUG_PRINT("myrg", ("name: '%s' table: 0x%lx", name, (long) table));
+ DBUG_PRINT("myrg", ("test_if_locked: %u", test_if_locked));
+
+ /* Save for later use. */
+ this->test_if_locked= test_if_locked;
+
+ /* retrieve children table list. */
+ my_errno= 0;
+ if (!(file= myrg_parent_open(name, myisammrg_parent_open_callback, this)))
+ {
+ DBUG_PRINT("error", ("my_errno %d", my_errno));
+ DBUG_RETURN(my_errno ? my_errno : -1);
+ }
+ DBUG_PRINT("myrg", ("MYRG_INFO: 0x%lx", (long) file));
+ DBUG_RETURN(0);
+}
+
+
+/**
+ @brief Attach children to a MERGE table.
+
+ @detail Let the storage engine attach its children through a callback
+ function. Check table definitions for consistency.
+
+ @note Special thd->open_options may be in effect. We can make use of
+ them in attach. I.e. we use HA_OPEN_FOR_REPAIR to report the names
+ of mismatching child tables. We cannot transport these options in
+ ha_myisammrg::test_if_locked because they may change after the
+ parent is opened. The parent is kept open in the table cache over
+ multiple statements and can be used by other threads. Open options
+ can change over time.
+
+ @return status
+ @retval 0 OK
+ @retval != 0 Error, my_errno gives reason
+*/
+
+int ha_myisammrg::attach_children(void)
{
- MI_KEYDEF *keyinfo;
- MI_COLUMNDEF *recinfo;
- MYRG_TABLE *u_table;
- uint recs;
- uint keys= table->s->keys;
- int error;
- char name_buff[FN_REFLEN];
+ MYRG_TABLE *u_table;
+ MI_COLUMNDEF *recinfo;
+ MI_KEYDEF *keyinfo;
+ uint recs;
+ uint keys= table->s->keys;
+ int error;
+ DBUG_ENTER("ha_myisammrg::attach_children");
+ DBUG_PRINT("myrg", ("table: '%s'.'%s' 0x%lx", table->s->db.str,
+ table->s->table_name.str, (long) table));
+ DBUG_PRINT("myrg", ("test_if_locked: %u", this->test_if_locked));
+ DBUG_ASSERT(!this->file->children_attached);
+
+ /*
+ Initialize variables that are used, modified, and/or set by
+ myisammrg_attach_children_callback().
+ 'next_child_attach' traverses the chain of TABLE_LIST objects
+ that has been compiled during myrg_parent_open(). Every call
+ to myisammrg_attach_children_callback() moves the pointer to
+ the next object.
+ 'need_compat_check' is set by myisammrg_attach_children_callback()
+ if a child fails the table def version check.
+ 'my_errno' is set by myisammrg_attach_children_callback() in
+ case of an error.
+ */
+ next_child_attach= table->child_l;
+ need_compat_check= FALSE;
+ my_errno= 0;
- DBUG_PRINT("info", ("ha_myisammrg::open"));
- if (!(file=myrg_open(fn_format(name_buff,name,"","",
- MY_UNPACK_FILENAME|MY_APPEND_EXT),
- mode, test_if_locked)))
+ if (myrg_attach_children(this->file, this->test_if_locked |
+ current_thd->open_options,
+ myisammrg_attach_children_callback, this))
{
- DBUG_PRINT("info", ("ha_myisammrg::open exit %d", my_errno));
- return (my_errno ? my_errno : -1);
+ DBUG_PRINT("error", ("my_errno %d", my_errno));
+ DBUG_RETURN(my_errno ? my_errno : -1);
}
- DBUG_PRINT("info", ("ha_myisammrg::open myrg_extrafunc..."));
+ DBUG_PRINT("myrg", ("calling myrg_extrafunc"));
myrg_extrafunc(file, query_cache_invalidate_by_MyISAM_filename_ref);
if (!(test_if_locked == HA_OPEN_WAIT_IF_LOCKED ||
test_if_locked == HA_OPEN_ABORT_IF_LOCKED))
@@ -116,69 +488,147 @@ int ha_myisammrg::open(const char *name, int mode, uint test_if_locked)
if (!(test_if_locked & HA_OPEN_WAIT_IF_LOCKED))
myrg_extra(file,HA_EXTRA_WAIT_LOCK,0);
- if (table->s->reclength != stats.mean_rec_length && stats.mean_rec_length)
- {
- DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu",
- table->s->reclength, stats.mean_rec_length));
- if (test_if_locked & HA_OPEN_FOR_REPAIR)
- myrg_print_wrong_table(file->open_tables->table->filename);
- error= HA_ERR_WRONG_MRG_TABLE_DEF;
- goto err;
- }
- if ((error= table2myisam(table, &keyinfo, &recinfo, &recs)))
- {
- /* purecov: begin inspected */
- DBUG_PRINT("error", ("Failed to convert TABLE object to MyISAM "
- "key and column definition"));
- goto err;
- /* purecov: end */
- }
- for (u_table= file->open_tables; u_table < file->end_table; u_table++)
+ /*
+ The compatibility check is required only if one or more children do
+ not match their table def version from the last check. This will
+ always happen at the first attach because the reference child def
+ version is initialized to 'undefined' at open.
+ */
+ DBUG_PRINT("myrg", ("need_compat_check: %d", need_compat_check));
+ if (need_compat_check)
{
- if (check_definition(keyinfo, recinfo, keys, recs,
- u_table->table->s->keyinfo, u_table->table->s->rec,
- u_table->table->s->base.keys,
- u_table->table->s->base.fields, false))
+ TABLE_LIST *child_l;
+
+ if (table->s->reclength != stats.mean_rec_length && stats.mean_rec_length)
{
- error= HA_ERR_WRONG_MRG_TABLE_DEF;
+ DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu",
+ table->s->reclength, stats.mean_rec_length));
if (test_if_locked & HA_OPEN_FOR_REPAIR)
- myrg_print_wrong_table(u_table->table->filename);
- else
+ myrg_print_wrong_table(file->open_tables->table->filename);
+ error= HA_ERR_WRONG_MRG_TABLE_DEF;
+ goto err;
+ }
+ /*
+ Both recinfo and keyinfo are allocated by my_multi_malloc(), thus
+ only recinfo must be freed.
+ */
+ if ((error= table2myisam(table, &keyinfo, &recinfo, &recs)))
+ {
+ /* purecov: begin inspected */
+ DBUG_PRINT("error", ("failed to convert TABLE object to MyISAM "
+ "key and column definition"));
+ goto err;
+ /* purecov: end */
+ }
+ for (u_table= file->open_tables; u_table < file->end_table; u_table++)
+ {
+ if (check_definition(keyinfo, recinfo, keys, recs,
+ u_table->table->s->keyinfo, u_table->table->s->rec,
+ u_table->table->s->base.keys,
+ u_table->table->s->base.fields, false))
{
- my_free((uchar*) recinfo, MYF(0));
- goto err;
+ DBUG_PRINT("error", ("table definition mismatch: '%s'",
+ u_table->table->filename));
+ error= HA_ERR_WRONG_MRG_TABLE_DEF;
+ if (!(this->test_if_locked & HA_OPEN_FOR_REPAIR))
+ {
+ my_free((uchar*) recinfo, MYF(0));
+ goto err;
+ }
+ myrg_print_wrong_table(u_table->table->filename);
}
}
+ my_free((uchar*) recinfo, MYF(0));
+ if (error == HA_ERR_WRONG_MRG_TABLE_DEF)
+ goto err;
+
+ /* All checks passed so far. Now update child def version. */
+ for (child_l= table->child_l; ; child_l= child_l->next_global)
+ {
+ child_l->set_child_def_version(
+ child_l->table->s->get_table_def_version());
+
+ if (&child_l->next_global == table->child_last_l)
+ break;
+ }
}
- my_free((uchar*) recinfo, MYF(0));
- if (error == HA_ERR_WRONG_MRG_TABLE_DEF)
- goto err;
#if !defined(BIG_TABLES) || SIZEOF_OFF_T == 4
/* Merge table has more than 2G rows */
if (table->s->crashed)
{
+ DBUG_PRINT("error", ("MERGE table marked crashed"));
error= HA_ERR_WRONG_MRG_TABLE_DEF;
goto err;
}
#endif
- return (0);
+ DBUG_RETURN(0);
+
err:
- myrg_close(file);
- file=0;
- return (my_errno= error);
+ myrg_detach_children(file);
+ DBUG_RETURN(my_errno= error);
}
+
+/**
+ @brief Detach all children from a MERGE table.
+
+ @note Detach must not touch the children in any way.
+ They may have been closed at ths point already.
+ All references to the children should be removed.
+
+ @return status
+ @retval 0 OK
+ @retval != 0 Error, my_errno gives reason
+*/
+
+int ha_myisammrg::detach_children(void)
+{
+ DBUG_ENTER("ha_myisammrg::detach_children");
+ DBUG_ASSERT(this->file && this->file->children_attached);
+
+ if (myrg_detach_children(this->file))
+ {
+ /* purecov: begin inspected */
+ DBUG_PRINT("error", ("my_errno %d", my_errno));
+ DBUG_RETURN(my_errno ? my_errno : -1);
+ /* purecov: end */
+ }
+ DBUG_RETURN(0);
+}
+
+
+/**
+ @brief Close a MERGE parent table, not its children.
+
+ @note The children are expected to be closed separately by the caller.
+
+ @return status
+ @retval 0 OK
+ @retval != 0 Error, my_errno gives reason
+*/
+
int ha_myisammrg::close(void)
{
- return myrg_close(file);
+ int rc;
+ DBUG_ENTER("ha_myisammrg::close");
+ /*
+ Children must not be attached here. Unless the MERGE table has no
+ children. In this case children_attached is always true.
+ */
+ DBUG_ASSERT(!this->file->children_attached || !this->file->tables);
+ rc= myrg_close(file);
+ file= 0;
+ DBUG_RETURN(rc);
}
int ha_myisammrg::write_row(uchar * buf)
{
+ DBUG_ENTER("ha_myisammrg::write_row");
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_write_count);
if (file->merge_insert_method == MERGE_INSERT_DISABLED || !file->tables)
- return (HA_ERR_TABLE_READONLY);
+ DBUG_RETURN(HA_ERR_TABLE_READONLY);
if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_INSERT)
table->timestamp_field->set_time();
@@ -186,13 +636,14 @@ int ha_myisammrg::write_row(uchar * buf)
{
int error;
if ((error= update_auto_increment()))
- return error;
+ DBUG_RETURN(error); /* purecov: inspected */
}
- return myrg_write(file,buf);
+ DBUG_RETURN(myrg_write(file,buf));
}
int ha_myisammrg::update_row(const uchar * old_data, uchar * new_data)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_update_count);
if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_UPDATE)
table->timestamp_field->set_time();
@@ -201,6 +652,7 @@ int ha_myisammrg::update_row(const uchar * old_data, uchar * new_data)
int ha_myisammrg::delete_row(const uchar * buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_delete_count);
return myrg_delete(file,buf);
}
@@ -209,6 +661,7 @@ int ha_myisammrg::index_read_map(uchar * buf, const uchar * key,
key_part_map keypart_map,
enum ha_rkey_function find_flag)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_key_count);
int error=myrg_rkey(file,buf,active_index, key, keypart_map, find_flag);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -219,6 +672,7 @@ int ha_myisammrg::index_read_idx_map(uchar * buf, uint index, const uchar * key,
key_part_map keypart_map,
enum ha_rkey_function find_flag)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_key_count);
int error=myrg_rkey(file,buf,index, key, keypart_map, find_flag);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -228,6 +682,7 @@ int ha_myisammrg::index_read_idx_map(uchar * buf, uint index, const uchar * key,
int ha_myisammrg::index_read_last_map(uchar *buf, const uchar *key,
key_part_map keypart_map)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_key_count);
int error=myrg_rkey(file,buf,active_index, key, keypart_map,
HA_READ_PREFIX_LAST);
@@ -237,6 +692,7 @@ int ha_myisammrg::index_read_last_map(uchar *buf, const uchar *key,
int ha_myisammrg::index_next(uchar * buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_next_count);
int error=myrg_rnext(file,buf,active_index);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -245,6 +701,7 @@ int ha_myisammrg::index_next(uchar * buf)
int ha_myisammrg::index_prev(uchar * buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_prev_count);
int error=myrg_rprev(file,buf, active_index);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -253,6 +710,7 @@ int ha_myisammrg::index_prev(uchar * buf)
int ha_myisammrg::index_first(uchar * buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_first_count);
int error=myrg_rfirst(file, buf, active_index);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -261,6 +719,7 @@ int ha_myisammrg::index_first(uchar * buf)
int ha_myisammrg::index_last(uchar * buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_last_count);
int error=myrg_rlast(file, buf, active_index);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -271,6 +730,7 @@ int ha_myisammrg::index_next_same(uchar * buf,
const uchar *key __attribute__((unused)),
uint length __attribute__((unused)))
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_next_count);
int error=myrg_rnext_same(file,buf);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -280,12 +740,14 @@ int ha_myisammrg::index_next_same(uchar * buf,
int ha_myisammrg::rnd_init(bool scan)
{
+ DBUG_ASSERT(this->file->children_attached);
return myrg_reset(file);
}
int ha_myisammrg::rnd_next(uchar *buf)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_rnd_next_count);
int error=myrg_rrnd(file, buf, HA_OFFSET_ERROR);
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -295,6 +757,7 @@ int ha_myisammrg::rnd_next(uchar *buf)
int ha_myisammrg::rnd_pos(uchar * buf, uchar *pos)
{
+ DBUG_ASSERT(this->file->children_attached);
ha_statistic_increment(&SSV::ha_read_rnd_count);
int error=myrg_rrnd(file, buf, my_get_ptr(pos,ref_length));
table->status=error ? STATUS_NOT_FOUND: 0;
@@ -303,6 +766,7 @@ int ha_myisammrg::rnd_pos(uchar * buf, uchar *pos)
void ha_myisammrg::position(const uchar *record)
{
+ DBUG_ASSERT(this->file->children_attached);
ulonglong row_position= myrg_position(file);
my_store_ptr(ref, ref_length, (my_off_t) row_position);
}
@@ -311,6 +775,7 @@ void ha_myisammrg::position(const uchar *record)
ha_rows ha_myisammrg::records_in_range(uint inx, key_range *min_key,
key_range *max_key)
{
+ DBUG_ASSERT(this->file->children_attached);
return (ha_rows) myrg_records_in_range(file, (int) inx, min_key, max_key);
}
@@ -318,6 +783,7 @@ ha_rows ha_myisammrg::records_in_range(uint inx, key_range *min_key,
int ha_myisammrg::info(uint flag)
{
MYMERGE_INFO mrg_info;
+ DBUG_ASSERT(this->file->children_attached);
(void) myrg_status(file,&mrg_info,flag);
/*
The following fails if one has not compiled MySQL with -DBIG_TABLES
@@ -387,6 +853,23 @@ int ha_myisammrg::info(uint flag)
int ha_myisammrg::extra(enum ha_extra_function operation)
{
+ if (operation == HA_EXTRA_ATTACH_CHILDREN)
+ {
+ int rc= attach_children();
+ if (!rc)
+ (void) extra(HA_EXTRA_NO_READCHECK); // Not needed in SQL
+ return(rc);
+ }
+ else if (operation == HA_EXTRA_DETACH_CHILDREN)
+ {
+ /*
+ Note that detach must not touch the children in any way.
+ They may have been closed at ths point already.
+ */
+ int rc= detach_children();
+ return(rc);
+ }
+
/* As this is just a mapping, we don't have to force the underlying
tables to be closed */
if (operation == HA_EXTRA_FORCE_REOPEN ||
@@ -404,6 +887,7 @@ int ha_myisammrg::reset(void)
int ha_myisammrg::extra_opt(enum ha_extra_function operation, ulong cache_size)
{
+ DBUG_ASSERT(this->file->children_attached);
if ((specialflag & SPECIAL_SAFE_MODE) && operation == HA_EXTRA_WRITE_CACHE)
return 0;
return myrg_extra(file, operation, (void*) &cache_size);
@@ -411,11 +895,13 @@ int ha_myisammrg::extra_opt(enum ha_extra_function operation, ulong cache_size)
int ha_myisammrg::external_lock(THD *thd, int lock_type)
{
+ DBUG_ASSERT(this->file->children_attached);
return myrg_lock_database(file,lock_type);
}
uint ha_myisammrg::lock_count(void) const
{
+ DBUG_ASSERT(this->file->children_attached);
return file->tables;
}
@@ -425,6 +911,7 @@ THR_LOCK_DATA **ha_myisammrg::store_lock(THD *thd,
enum thr_lock_type lock_type)
{
MYRG_TABLE *open_table;
+ DBUG_ASSERT(this->file->children_attached);
for (open_table=file->open_tables ;
open_table != file->end_table ;
@@ -519,47 +1006,50 @@ int ha_myisammrg::create(const char *name, register TABLE *form,
uint dirlgt= dirname_length(name);
DBUG_ENTER("ha_myisammrg::create");
+ /* Allocate a table_names array in thread mem_root. */
if (!(table_names= (const char**)
thd->alloc((create_info->merge_list.elements+1) * sizeof(char*))))
DBUG_RETURN(HA_ERR_OUT_OF_MEM);
+
+ /* Create child path names. */
for (pos= table_names; tables; tables= tables->next_local)
{
const char *table_name;
- TABLE *tbl= 0;
- if (create_info->options & HA_LEX_CREATE_TMP_TABLE)
- tbl= find_temporary_table(thd, tables);
- if (!tbl)
- {
- /*
- Construct the path to the MyISAM table. Try to meet two conditions:
- 1.) Allow to include MyISAM tables from different databases, and
- 2.) allow for moving DATADIR around in the file system.
- The first means that we need paths in the .MRG file. The second
- means that we should not have absolute paths in the .MRG file.
- The best, we can do, is to use 'mysql_data_home', which is '.'
- in mysqld and may be an absolute path in an embedded server.
- This means that it might not be possible to move the DATADIR of
- an embedded server without changing the paths in the .MRG file.
- */
- uint length= build_table_filename(buff, sizeof(buff),
- tables->db, tables->table_name, "", 0);
- /*
- If a MyISAM table is in the same directory as the MERGE table,
- we use the table name without a path. This means that the
- DATADIR can easily be moved even for an embedded server as long
- as the MyISAM tables are from the same database as the MERGE table.
- */
- if ((dirname_length(buff) == dirlgt) && ! memcmp(buff, name, dirlgt))
- table_name= tables->table_name;
- else
- if (! (table_name= thd->strmake(buff, length)))
- DBUG_RETURN(HA_ERR_OUT_OF_MEM);
- }
+
+ /*
+ Construct the path to the MyISAM table. Try to meet two conditions:
+ 1.) Allow to include MyISAM tables from different databases, and
+ 2.) allow for moving DATADIR around in the file system.
+ The first means that we need paths in the .MRG file. The second
+ means that we should not have absolute paths in the .MRG file.
+ The best, we can do, is to use 'mysql_data_home', which is '.'
+ in mysqld and may be an absolute path in an embedded server.
+ This means that it might not be possible to move the DATADIR of
+ an embedded server without changing the paths in the .MRG file.
+
+ Do the same even for temporary tables. MERGE children are now
+ opened through the table cache. They are opened by db.table_name,
+ not by their path name.
+ */
+ uint length= build_table_filename(buff, sizeof(buff),
+ tables->db, tables->table_name, "", 0);
+ /*
+ If a MyISAM table is in the same directory as the MERGE table,
+ we use the table name without a path. This means that the
+ DATADIR can easily be moved even for an embedded server as long
+ as the MyISAM tables are from the same database as the MERGE table.
+ */
+ if ((dirname_length(buff) == dirlgt) && ! memcmp(buff, name, dirlgt))
+ table_name= tables->table_name;
else
- table_name= tbl->s->path.str;
+ if (! (table_name= thd->strmake(buff, length)))
+ DBUG_RETURN(HA_ERR_OUT_OF_MEM); /* purecov: inspected */
+
*pos++= table_name;
}
*pos=0;
+
+ /* Create a MERGE meta file from the table_names array. */
DBUG_RETURN(myrg_create(fn_format(buff,name,"","",
MY_RESOLVE_SYMLINKS|
MY_UNPACK_FILENAME|MY_APPEND_EXT),
@@ -642,7 +1132,7 @@ static int myisammrg_init(void *p)
myisammrg_hton->db_type= DB_TYPE_MRG_MYISAM;
myisammrg_hton->create= myisammrg_create_handler;
myisammrg_hton->panic= myisammrg_panic;
- myisammrg_hton->flags= HTON_CAN_RECREATE|HTON_NO_PARTITION;
+ myisammrg_hton->flags= HTON_NO_PARTITION;
return 0;
}
diff --git a/storage/myisammrg/ha_myisammrg.h b/storage/myisammrg/ha_myisammrg.h
index 91aabe277f7..977c45d1435 100644
--- a/storage/myisammrg/ha_myisammrg.h
+++ b/storage/myisammrg/ha_myisammrg.h
@@ -27,8 +27,12 @@ class ha_myisammrg: public handler
MYRG_INFO *file;
public:
+ TABLE_LIST *next_child_attach; /* next child to attach */
+ uint test_if_locked; /* flags from ::open() */
+ bool need_compat_check; /* if need compatibility check */
+
ha_myisammrg(handlerton *hton, TABLE_SHARE *table_arg);
- ~ha_myisammrg() {}
+ ~ha_myisammrg();
const char *table_type() const { return "MRG_MyISAM"; }
const char **bas_ext() const;
const char *index_type(uint key_number);
@@ -53,6 +57,8 @@ class ha_myisammrg: public handler
{ return ulonglong2double(stats.data_file_length) / IO_SIZE + file->tables; }
int open(const char *name, int mode, uint test_if_locked);
+ int attach_children(void);
+ int detach_children(void);
int close(void);
int write_row(uchar * buf);
int update_row(const uchar * old_data, uchar * new_data);
@@ -85,6 +91,7 @@ class ha_myisammrg: public handler
void update_create_info(HA_CREATE_INFO *create_info);
void append_create_info(String *packet);
MYRG_INFO *myrg_info() { return file; }
+ TABLE *table_ptr() { return table; }
bool check_if_incompatible_data(HA_CREATE_INFO *info, uint table_changes);
int check(THD* thd, HA_CHECK_OPT* check_opt);
};
diff --git a/storage/myisammrg/myrg_close.c b/storage/myisammrg/myrg_close.c
index baae24634b3..b402a35a253 100644
--- a/storage/myisammrg/myrg_close.c
+++ b/storage/myisammrg/myrg_close.c
@@ -23,9 +23,37 @@ int myrg_close(MYRG_INFO *info)
MYRG_TABLE *file;
DBUG_ENTER("myrg_close");
- for (file=info->open_tables ; file != info->end_table ; file++)
- if ((new_error=mi_close(file->table)))
- error=new_error;
+ /*
+ Assume that info->children_attached means that this is called from
+ direct use of MERGE, not from a MySQL server. In this case the
+ children must be closed and info->rec_per_key_part is part of the
+ 'info' multi_alloc.
+ If info->children_attached is false, this is called from a MySQL
+ server. Children are closed independently but info->rec_per_key_part
+ must be freed.
+ Just in case of a server panic (myrg_panic()) info->children_attached
+ might be true. We would close the children though they should be
+ closed independently and info->rec_per_key_part is not freed.
+ This should be acceptable for a panic.
+ In case of a MySQL server and no children, children_attached is
+ always true. In this case no rec_per_key_part has been allocated.
+ So it is correct to use the branch where an empty list of tables is
+ (not) closed.
+ */
+ if (info->children_attached)
+ {
+ for (file= info->open_tables; file != info->end_table; file++)
+ {
+ /* purecov: begin inspected */
+ if ((new_error= mi_close(file->table)))
+ error= new_error;
+ else
+ file->table= NULL;
+ /* purecov: end */
+ }
+ }
+ else
+ my_free((uchar*) info->rec_per_key_part, MYF(MY_ALLOW_ZERO_PTR));
delete_queue(&info->by_key);
pthread_mutex_lock(&THR_LOCK_open);
myrg_open_list=list_delete(myrg_open_list,&info->open_list);
diff --git a/storage/myisammrg/myrg_extra.c b/storage/myisammrg/myrg_extra.c
index a1e6e3f8d4d..3d14f6a56e6 100644
--- a/storage/myisammrg/myrg_extra.c
+++ b/storage/myisammrg/myrg_extra.c
@@ -29,6 +29,8 @@ int myrg_extra(MYRG_INFO *info,enum ha_extra_function function,
DBUG_ENTER("myrg_extra");
DBUG_PRINT("info",("function: %lu", (ulong) function));
+ if (!info->children_attached)
+ DBUG_RETURN(1);
if (function == HA_EXTRA_CACHE)
{
info->cache_in_use=1;
@@ -73,6 +75,8 @@ int myrg_reset(MYRG_INFO *info)
MYRG_TABLE *file;
DBUG_ENTER("myrg_reset");
+ if (!info->children_attached)
+ DBUG_RETURN(1);
info->cache_in_use=0;
info->current_table=0;
info->last_used_table= info->open_tables;
diff --git a/storage/myisammrg/myrg_open.c b/storage/myisammrg/myrg_open.c
index 500d3a29327..6fb1888f84d 100644
--- a/storage/myisammrg/myrg_open.c
+++ b/storage/myisammrg/myrg_open.c
@@ -26,8 +26,14 @@
open a MyISAM MERGE table
if handle_locking is 0 then exit with error if some table is locked
if handle_locking is 1 then wait if table is locked
-*/
+ NOTE: This function is not used in the MySQL server. It is for
+ MERGE use independent from MySQL. Currently there is some code
+ duplication between myrg_open() and myrg_parent_open() +
+ myrg_attach_children(). Please duplicate changes in these
+ functions or make common sub-functions.
+*/
+/* purecov: begin unused */
MYRG_INFO *myrg_open(const char *name, int mode, int handle_locking)
{
@@ -107,13 +113,11 @@ MYRG_INFO *myrg_open(const char *name, int mode, int handle_locking)
key_parts*sizeof(long),
MYF(MY_WME|MY_ZEROFILL))))
goto err;
- if (files)
- {
- m_info->open_tables=(MYRG_TABLE *) (m_info+1);
- m_info->rec_per_key_part=(ulong *) (m_info->open_tables+files);
- m_info->tables= files;
- files= 0;
- }
+ DBUG_ASSERT(files);
+ m_info->open_tables=(MYRG_TABLE *) (m_info+1);
+ m_info->rec_per_key_part=(ulong *) (m_info->open_tables+files);
+ m_info->tables= files;
+ files= 0;
m_info->reclength=isam->s->base.reclength;
min_keys= isam->s->base.keys;
errpos=3;
@@ -163,6 +167,7 @@ MYRG_INFO *myrg_open(const char *name, int mode, int handle_locking)
/* this works ok if the table list is empty */
m_info->end_table=m_info->open_tables+files;
m_info->last_used_table=m_info->open_tables;
+ m_info->children_attached= TRUE;
VOID(my_close(fd,MYF(0)));
end_io_cache(&file);
@@ -189,3 +194,316 @@ err:
my_errno=save_errno;
DBUG_RETURN (NULL);
}
+/* purecov: end */
+
+
+/**
+ @brief Open parent table of a MyISAM MERGE table.
+
+ @detail Open MERGE meta file to get the table name paths for the child
+ tables. Count the children. Allocate and initialize MYRG_INFO
+ structure. Call a callback function for each child table.
+
+ @param[in] parent_name merge table name path as "database/table"
+ @param[in] callback function to call for each child table
+ @param[in] callback_param data pointer to give to the callback
+
+ @return MYRG_INFO pointer
+ @retval != NULL OK
+ @retval NULL Error
+
+ @note: Currently there is some code duplication between myrg_open()
+ and myrg_parent_open() + myrg_attach_children(). Please duplicate
+ changes in these functions or make common sub-functions.
+*/
+
+MYRG_INFO *myrg_parent_open(const char *parent_name,
+ int (*callback)(void*, const char*),
+ void *callback_param)
+{
+ MYRG_INFO *m_info;
+ int rc;
+ int errpos;
+ int save_errno;
+ int insert_method;
+ uint length;
+ uint dir_length;
+ uint child_count;
+ size_t name_buff_length;
+ File fd;
+ IO_CACHE file_cache;
+ char parent_name_buff[FN_REFLEN * 2];
+ char child_name_buff[FN_REFLEN];
+ DBUG_ENTER("myrg_parent_open");
+
+ rc= 1;
+ errpos= 0;
+ bzero((char*) &file_cache, sizeof(file_cache));
+
+ /* Open MERGE meta file. */
+ if ((fd= my_open(fn_format(parent_name_buff, parent_name, "", MYRG_NAME_EXT,
+ MY_UNPACK_FILENAME|MY_APPEND_EXT),
+ O_RDONLY | O_SHARE, MYF(0))) < 0)
+ goto err; /* purecov: inspected */
+ errpos= 1;
+
+ if (init_io_cache(&file_cache, fd, 4 * IO_SIZE, READ_CACHE, 0, 0,
+ MYF(MY_WME | MY_NABP)))
+ goto err; /* purecov: inspected */
+ errpos= 2;
+
+ /* Count children. Determine insert method. */
+ child_count= 0;
+ insert_method= 0;
+ while ((length= my_b_gets(&file_cache, child_name_buff, FN_REFLEN - 1)))
+ {
+ /* Remove line terminator. */
+ if (child_name_buff[length - 1] == '\n')
+ child_name_buff[--length]= '\0';
+
+ /* Skip empty lines. */
+ if (!child_name_buff[0])
+ continue; /* purecov: inspected */
+
+ /* Skip comments, but evaluate insert method. */
+ if (child_name_buff[0] == '#')
+ {
+ if (!strncmp(child_name_buff + 1, "INSERT_METHOD=", 14))
+ {
+ /* Compare buffer with global methods list: merge_insert_method. */
+ insert_method= find_type(child_name_buff + 15,
+ &merge_insert_method, 2);
+ }
+ continue;
+ }
+
+ /* Count the child. */
+ child_count++;
+ }
+
+ /* Allocate MERGE parent table structure. */
+ if (!(m_info= (MYRG_INFO*) my_malloc(sizeof(MYRG_INFO) +
+ child_count * sizeof(MYRG_TABLE),
+ MYF(MY_WME | MY_ZEROFILL))))
+ goto err; /* purecov: inspected */
+ errpos= 3;
+ m_info->open_tables= (MYRG_TABLE*) (m_info + 1);
+ m_info->tables= child_count;
+ m_info->merge_insert_method= insert_method > 0 ? insert_method : 0;
+ /* This works even if the table list is empty. */
+ m_info->end_table= m_info->open_tables + child_count;
+ if (!child_count)
+ {
+ /* Do not attach/detach an empty child list. */
+ m_info->children_attached= TRUE;
+ }
+
+ /* Call callback for each child. */
+ dir_length= dirname_part(parent_name_buff, parent_name, &name_buff_length);
+ my_b_seek(&file_cache, 0);
+ while ((length= my_b_gets(&file_cache, child_name_buff, FN_REFLEN - 1)))
+ {
+ /* Remove line terminator. */
+ if (child_name_buff[length - 1] == '\n')
+ child_name_buff[--length]= '\0';
+
+ /* Skip empty lines and comments. */
+ if (!child_name_buff[0] || (child_name_buff[0] == '#'))
+ continue;
+
+ if (!has_path(child_name_buff))
+ {
+ VOID(strmake(parent_name_buff + dir_length, child_name_buff,
+ sizeof(parent_name_buff) - 1 - dir_length));
+ VOID(cleanup_dirname(child_name_buff, parent_name_buff));
+ }
+ else
+ fn_format(child_name_buff, child_name_buff, "", "", 0);
+ DBUG_PRINT("info", ("child: '%s'", child_name_buff));
+
+ /* Callback registers child with handler table. */
+ if ((rc= (*callback)(callback_param, child_name_buff)))
+ goto err; /* purecov: inspected */
+ }
+
+ end_io_cache(&file_cache);
+ VOID(my_close(fd, MYF(0)));
+
+ m_info->open_list.data= (void*) m_info;
+ pthread_mutex_lock(&THR_LOCK_open);
+ myrg_open_list= list_add(myrg_open_list, &m_info->open_list);
+ pthread_mutex_unlock(&THR_LOCK_open);
+
+ DBUG_RETURN(m_info);
+
+ /* purecov: begin inspected */
+ err:
+ save_errno= my_errno;
+ switch (errpos) {
+ case 3:
+ my_free((char*) m_info, MYF(0));
+ /* Fall through */
+ case 2:
+ end_io_cache(&file_cache);
+ /* Fall through */
+ case 1:
+ VOID(my_close(fd, MYF(0)));
+ }
+ my_errno= save_errno;
+ DBUG_RETURN (NULL);
+ /* purecov: end */
+}
+
+
+/**
+ @brief Attach children to a MyISAM MERGE parent table.
+
+ @detail Call a callback function for each child table.
+ The callback returns the MyISAM table handle of the child table.
+ Check table definition match.
+
+ @param[in] m_info MERGE parent table structure
+ @param[in] handle_locking if contains HA_OPEN_FOR_REPAIR, warn about
+ incompatible child tables, but continue
+ @param[in] callback function to call for each child table
+ @param[in] callback_param data pointer to give to the callback
+
+ @return status
+ @retval 0 OK
+ @retval != 0 Error
+
+ @note: Currently there is some code duplication between myrg_open()
+ and myrg_parent_open() + myrg_attach_children(). Please duplicate
+ changes in these functions or make common sub-functions.
+*/
+
+int myrg_attach_children(MYRG_INFO *m_info, int handle_locking,
+ MI_INFO *(*callback)(void*),
+ void *callback_param)
+{
+ ulonglong file_offset;
+ MI_INFO *myisam;
+ int rc;
+ int errpos;
+ int save_errno;
+ uint idx;
+ uint child_nr;
+ uint key_parts;
+ uint min_keys;
+ DBUG_ENTER("myrg_attach_children");
+ DBUG_PRINT("myrg", ("handle_locking: %d", handle_locking));
+
+ rc= 1;
+ errpos= 0;
+ file_offset= 0;
+ LINT_INIT(key_parts);
+ min_keys= 0;
+ child_nr= 0;
+ while ((myisam= (*callback)(callback_param)))
+ {
+ DBUG_PRINT("myrg", ("child_nr: %u table: '%s'",
+ child_nr, myisam->filename));
+ DBUG_ASSERT(child_nr < m_info->tables);
+
+ /* Special handling when the first child is attached. */
+ if (!child_nr)
+ {
+ m_info->reclength= myisam->s->base.reclength;
+ min_keys= myisam->s->base.keys;
+ key_parts= myisam->s->base.key_parts;
+ if (!m_info->rec_per_key_part)
+ {
+ if(!(m_info->rec_per_key_part= (ulong*)
+ my_malloc(key_parts * sizeof(long), MYF(MY_WME|MY_ZEROFILL))))
+ goto err; /* purecov: inspected */
+ errpos= 1;
+ }
+ }
+
+ /* Add MyISAM table info. */
+ m_info->open_tables[child_nr].table= myisam;
+ m_info->open_tables[child_nr].file_offset= (my_off_t) file_offset;
+ file_offset+= myisam->state->data_file_length;
+
+ /* Check table definition match. */
+ if (m_info->reclength != myisam->s->base.reclength)
+ {
+ DBUG_PRINT("error", ("definition mismatch table: '%s' repair: %d",
+ myisam->filename,
+ (handle_locking & HA_OPEN_FOR_REPAIR)));
+ my_errno= HA_ERR_WRONG_MRG_TABLE_DEF;
+ if (handle_locking & HA_OPEN_FOR_REPAIR)
+ {
+ myrg_print_wrong_table(myisam->filename);
+ continue;
+ }
+ goto err;
+ }
+
+ m_info->options|= myisam->s->options;
+ m_info->records+= myisam->state->records;
+ m_info->del+= myisam->state->del;
+ m_info->data_file_length+= myisam->state->data_file_length;
+ if (min_keys > myisam->s->base.keys)
+ min_keys= myisam->s->base.keys; /* purecov: inspected */
+ for (idx= 0; idx < key_parts; idx++)
+ m_info->rec_per_key_part[idx]+= (myisam->s->state.rec_per_key_part[idx] /
+ m_info->tables);
+ child_nr++;
+ }
+
+ if (my_errno == HA_ERR_WRONG_MRG_TABLE_DEF)
+ goto err;
+ if (sizeof(my_off_t) == 4 && file_offset > (ulonglong) (ulong) ~0L)
+ {
+ my_errno= HA_ERR_RECORD_FILE_FULL;
+ goto err;
+ }
+ /* Don't mark table readonly, for ALTER TABLE ... UNION=(...) to work */
+ m_info->options&= ~(HA_OPTION_COMPRESS_RECORD | HA_OPTION_READ_ONLY_DATA);
+ m_info->keys= min_keys;
+ m_info->last_used_table= m_info->open_tables;
+ m_info->children_attached= TRUE;
+ DBUG_RETURN(0);
+
+err:
+ save_errno= my_errno;
+ switch (errpos) {
+ case 1:
+ my_free((char*) m_info->rec_per_key_part, MYF(0));
+ m_info->rec_per_key_part= NULL;
+ }
+ my_errno= save_errno;
+ DBUG_RETURN(1);
+}
+
+
+/**
+ @brief Detach children from a MyISAM MERGE parent table.
+
+ @param[in] m_info MERGE parent table structure
+
+ @note Detach must not touch the children in any way.
+ They may have been closed at ths point already.
+ All references to the children should be removed.
+
+ @return status
+ @retval 0 OK
+*/
+
+int myrg_detach_children(MYRG_INFO *m_info)
+{
+ DBUG_ENTER("myrg_detach_children");
+ if (m_info->tables)
+ {
+ /* Do not attach/detach an empty child list. */
+ m_info->children_attached= FALSE;
+ bzero((char*) m_info->open_tables, m_info->tables * sizeof(MYRG_TABLE));
+ }
+ m_info->records= 0;
+ m_info->del= 0;
+ m_info->data_file_length= 0;
+ m_info->options= 0;
+ DBUG_RETURN(0);
+}
+