Table definition cache, part 2

The table opening process now works the following way:
- Create common TABLE_SHARE object
- Read the .frm file and unpack it into the TABLE_SHARE object
- Create a TABLE object based on the information in the TABLE_SHARE
  object and open a handler to the table object

Other noteworthy changes:
- In TABLE_SHARE the most common strings are now LEX_STRING's
- Better error message when table is not found
- Variable table_cache is now renamed 'table_open_cache'
- New variable 'table_definition_cache' that is the number of table defintions that will be cached
- strxnmov() calls are now fixed to avoid overflows
- strxnmov() will now always add one end \0 to result
- engine objects are now created with a TABLE_SHARE object instead of a TABLE object.
- After creating a field object one must call field->init(table) before using it

- For a busy system this change will give you:
 - Less memory usage for table object
 - Faster opening of tables (if it's has been in use or is in table definition cache)
 - Allow you to cache many table definitions objects
 - Faster drop of table


mysql-test/mysql-test-run.sh:
  Fixed some problems with --gdb option
  Test both with socket and tcp/ip port that all old servers are killed
mysql-test/r/flush_table.result:
  More tests with lock table with 2 threads + flush table
mysql-test/r/information_schema.result:
  Removed old (now wrong) result
mysql-test/r/innodb.result:
  Better error messages (thanks to TDC patch)
mysql-test/r/merge.result:
  Extra flush table test
mysql-test/r/ndb_bitfield.result:
  Better error messages (thanks to TDC patch)
mysql-test/r/ndb_partition_error.result:
  Better error messages (thanks to TDC patch)
mysql-test/r/query_cache.result:
  Remove tables left from old tests
mysql-test/r/temp_table.result:
  Test truncate with temporary tables
mysql-test/r/variables.result:
  Table_cache -> Table_open_cache
mysql-test/t/flush_table.test:
  More tests with lock table with 2 threads + flush table
mysql-test/t/merge.test:
  Extra flush table test
mysql-test/t/multi_update.test:
  Added 'sleep' to make test predictable
mysql-test/t/query_cache.test:
  Remove tables left from old tests
mysql-test/t/temp_table.test:
  Test truncate with temporary tables
mysql-test/t/variables.test:
  Table_cache -> Table_open_cache
mysql-test/valgrind.supp:
  Remove warning that may happens becasue threads dies in different order
mysys/hash.c:
  Fixed wrong DBUG_PRINT
mysys/mf_dirname.c:
  More DBUG
mysys/mf_pack.c:
  Better comment
mysys/mf_tempdir.c:
  More DBUG
  Ensure that we call cleanup_dirname() on all temporary directory paths.
  
  If we don't do this, we will get a failure when comparing temporary table
  names as in some cases the temporary table name is run through convert_dirname())
mysys/my_alloc.c:
  Indentation fix
sql/examples/ha_example.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/examples/ha_example.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/examples/ha_tina.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/examples/ha_tina.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/field.cc:
  Update for table definition cache:
  - Field creation now takes TABLE_SHARE instead of TABLE as argument
    (This is becasue field definitions are now cached in TABLE_SHARE)
    When a field is created, one now must call field->init(TABLE) before using it
  - Use s->db instead of s->table_cache_key
  - Added Field::clone() to create a field in TABLE from a field in TABLE_SHARE
  - make_field() takes TABLE_SHARE as argument instead of TABLE
  - move_field() -> move_field_offset()
sql/field.h:
  Update for table definition cache:
  - Field creation now takes TABLE_SHARE instead of TABLE as argument
    (This is becasue field definitions are now cached in TABLE_SHARE)
    When a field is created, one now must call field->init(TABLE) before using it
  - Added Field::clone() to create a field in TABLE from a field in TABLE_SHARE
  - make_field() takes TABLE_SHARE as argument instead of TABLE
  - move_field() -> move_field_offset()
sql/ha_archive.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_archive.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_berkeley.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Changed name of argument create() to not hide internal 'table' variable.
  table->s  -> table_share
sql/ha_berkeley.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_blackhole.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_blackhole.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_federated.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Fixed comments
  Remove index variable and replace with pointers (simple optimization)
  move_field() -> move_field_offset()
  Removed some strlen() calls
sql/ha_federated.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_heap.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Simplify delete_table() and create() as the given file names are now without extension
sql/ha_heap.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_innodb.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_innodb.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_myisam.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Remove not needed fn_format()
  Fixed for new table->s structure
sql/ha_myisam.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_myisammrg.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Don't set 'is_view' for MERGE tables
  Use new interface to find_temporary_table()
sql/ha_myisammrg.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Added flag HA_NO_COPY_ON_ALTER
sql/ha_ndbcluster.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Fixed wrong calls to strxnmov()
  Give error HA_ERR_TABLE_DEF_CHANGED if table definition has changed
  drop_table -> intern_drop_table()
  table->s -> table_share
  Move part_info to TABLE
  Fixed comments & DBUG print's
  New arguments to print_error()
sql/ha_ndbcluster.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
sql/ha_partition.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  We can't set up or use part_info when creating handler as there is not yet any table object
  New ha_intialise() to work with TDC (Done by Mikael)
sql/ha_partition.h:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  Got set_part_info() from Mikael
sql/handler.cc:
  We new use TABLE_SHARE instead of TABLE when creating engine handlers
  ha_delete_table() now also takes database as an argument
  handler::ha_open() now takes TABLE as argument
  ha_open() now calls ha_allocate_read_write_set()
  Simplify ha_allocate_read_write_set()
  Remove ha_deallocate_read_write_set()
  Use table_share (Cached by table definition cache)
sql/handler.h:
  New table flag: HA_NO_COPY_ON_ALTER (used by merge tables)
  Remove ha_deallocate_read_write_set()
  get_new_handler() now takes TABLE_SHARE as argument
  ha_delete_table() now gets database as argument
sql/item.cc:
  table_name and db are now LEX_STRING objects
  When creating fields, we have now have to call field->init(table)
  move_field -> move_field_offset()
sql/item.h:
  tmp_table_field_from_field_type() now takes an extra paramenter 'fixed_length' to allow one to force usage of CHAR
   instead of BLOB
sql/item_cmpfunc.cc:
  Fixed call to tmp_table_field_from_field_type()
sql/item_create.cc:
  Assert if new not handled cast type
sql/item_func.cc:
  When creating fields, we have now have to call field->init(table)
  dummy_table used by 'sp' now needs a TABLE_SHARE object
sql/item_subselect.cc:
  Trivial code cleanups
sql/item_sum.cc:
  When creating fields, we have now have to call field->init(table)
sql/item_timefunc.cc:
  Item_func_str_to_date::tmp_table_field() now replaced by call to
   tmp_table_field_from_field_type() (see item_timefunc.h)
sql/item_timefunc.h:
  Simply tmp_table_field()
sql/item_uniq.cc:
  When creating fields, we have now have to call field->init(table)
sql/key.cc:
  Added 'KEY' argument to 'find_ref_key' to simplify code
sql/lock.cc:
  More debugging
  Use create_table_def_key() to create key for table cache
  Allocate TABLE_SHARE properly when creating name lock
  Fix that locked_table_name doesn't test same table twice
sql/mysql_priv.h:
  New functions for table definition cache
  New interfaces to a lot of functions.
  New faster interface to find_temporary_table() and close_temporary_table()
sql/mysqld.cc:
  Added support for table definition cache of size 'table_def_size'
  Fixed som calls to strnmov()
  Changed name of 'table_cache' to 'table_open_cache'
sql/opt_range.cc:
  Use new interfaces
  Fixed warnings from valgrind
sql/parse_file.cc:
  Safer calls to strxnmov()
  Fixed typo
sql/set_var.cc:
  Added variable 'table_definition_cache'
  Variable table_cache renamed to 'table_open_cache'
sql/slave.cc:
  Use new interface
sql/sp.cc:
  Proper use of TABLE_SHARE
sql/sp_head.cc:
  Remove compiler warnings
  We have now to call field->init(table)
sql/sp_head.h:
  Pointers to parsed strings are now const
sql/sql_acl.cc:
  table_name is now a LEX_STRING
sql/sql_base.cc:
  Main implementation of table definition cache
  (The #ifdef's are there for the future when table definition cache will replace open table cache)
  Now table definitions are cached indepndent of open tables, which will speed up things when a table is in use at once from several places
  Views are not yet cached; For the moment we only cache if a table is a view or not.
  
  Faster implementation of find_temorary_table()
  Replace 'wait_for_refresh()' with the more general function 'wait_for_condition()'
  Drop table is slightly faster as we can use the table definition cache to know the type of the table
sql/sql_cache.cc:
  table_cache_key and table_name are now LEX_STRING
  'sDBUG print fixes
sql/sql_class.cc:
  table_cache_key is now a LEX_STRING
  safer strxnmov()
sql/sql_class.h:
  Added number of open table shares (table definitions)
sql/sql_db.cc:
  safer strxnmov()
sql/sql_delete.cc:
  Use new interface to find_temporary_table()
sql/sql_derived.cc:
  table_name is now a LEX_STRING
sql/sql_handler.cc:
  TABLE_SHARE->db and TABLE_SHARE->table_name are now LEX_STRING's
sql/sql_insert.cc:
  TABLE_SHARE->db and TABLE_SHARE->table_name are now LEX_STRING's
sql/sql_lex.cc:
  Make parsed string a const (to quickly find out if anything is trying to change the query string)
sql/sql_lex.h:
  Make parsed string a const (to quickly find out if anything is trying to change the query string)
sql/sql_load.cc:
  Safer strxnmov()
sql/sql_parse.cc:
  Better error if wrong DB name
sql/sql_partition.cc:
  part_info moved to TABLE from TABLE_SHARE
  Indentation changes
sql/sql_select.cc:
  Indentation fixes
  Call field->init(TABLE) for new created fields
  Update create_tmp_table() to use TABLE_SHARE properly
sql/sql_select.h:
  Call field->init(TABLE) for new created fields
sql/sql_show.cc:
  table_name is now a LEX_STRING
  part_info moved to TABLE
sql/sql_table.cc:
  Use table definition cache to speed up delete of tables
  Fixed calls to functions with new interfaces
  Don't use 'share_not_to_be_used'
  Instead of doing openfrm() when doing repair, we now have to call
  get_table_share() followed by open_table_from_share().
  Replace some fn_format() with faster unpack_filename().
  Safer strxnmov()
  part_info is now in TABLE
  Added Mikaels patch for partition and ALTER TABLE
  Instead of using 'TABLE_SHARE->is_view' use 'table_flags() & HA_NO_COPY_ON_ALTER
sql/sql_test.cc:
  table_name and table_cache_key are now LEX_STRING's
sql/sql_trigger.cc:
  TABLE_SHARE->db and TABLE_SHARE->table_name are now LEX_STRING's
  safer strxnmov()
  Removed compiler warnings
sql/sql_update.cc:
  Call field->init(TABLE) after field is created
sql/sql_view.cc:
  safer strxnmov()
  Create common TABLE_SHARE object for views to allow us to cache if table is a view
sql/structs.h:
  Added SHOW_TABLE_DEFINITIONS
sql/table.cc:
  Creation and destruct of TABLE_SHARE objects that are common for many TABLE objects
  
  The table opening process now works the following way:
  - Create common TABLE_SHARE object
  - Read the .frm file and unpack it into the TABLE_SHARE object
  - Create a TABLE object based on the information in the TABLE_SHARE
    object and open a handler to the table object
  
  open_table_def() is written in such a way that it should be trival to add parsing of the .frm files in new formats
sql/table.h:
  TABLE objects for the same database table now share a common TABLE_SHARE object
  In TABLE_SHARE the most common strings are now LEX_STRING's
sql/unireg.cc:
  Changed arguments to rea_create_table() to have same order as other functions
  Call field->init(table) for new created fields
sql/unireg.h:
  Added OPEN_VIEW
strings/strxnmov.c:
  Change strxnmov() to always add end \0
  This makes usage of strxnmov() safer as most of MySQL code assumes that strxnmov() will create a null terminated string

1 files changed, 143 insertions, 71 deletions

diff --git a/sql/sql_table.cc b/sql/sql_table.cc
index 93315e3b9f1..90f498f42dd 100644
--- a/sql/sql_table.cc
+++ b/sql/sql_table.cc
@@ -279,9 +279,22 @@ int mysql_rm_table_part2(THD *thd, TABLE_LIST *tables, bool if_exists,
   String wrong_tables;
   int error;
   bool some_tables_deleted=0, tmp_table_deleted=0, foreign_key_error=0;
-
   DBUG_ENTER("mysql_rm_table_part2");
 
+  /*
+    If we have the table in the definition cache, we don't have to check the
+    .frm file to find if the table is a normal table (not view) and what
+    engine to use.
+  */
+
+  for (table= tables; table; table= table->next_local)
+  {
+    TABLE_SHARE *share;
+    table->db_type= DB_TYPE_UNKNOWN;
+    if ((share= get_cached_table_share(table->db, table->table_name)))
+      table->db_type= share->db_type;
+  }
+
   if (lock_table_names(thd, tables))
     DBUG_RETURN(1);
 
@@ -291,16 +304,17 @@ int mysql_rm_table_part2(THD *thd, TABLE_LIST *tables, bool if_exists,
   for (table= tables; table; table= table->next_local)
   {
     char *db=table->db;
-    db_type table_type= DB_TYPE_UNKNOWN;
+    db_type table_type;
 
     mysql_ha_flush(thd, table, MYSQL_HA_CLOSE_FINAL, TRUE);
-    if (!close_temporary_table(thd, db, table->table_name))
+    if (!close_temporary_table(thd, table))
     {
       tmp_table_deleted=1;
       continue;					// removed temporary table
     }
 
     error=0;
+    table_type= table->db_type;
     if (!drop_temporary)
     {
       abort_locked_tables(thd, db, table->table_name);
@@ -314,14 +328,15 @@ int mysql_rm_table_part2(THD *thd, TABLE_LIST *tables, bool if_exists,
 	DBUG_RETURN(-1);
       }
       alias= (lower_case_table_names == 2) ? table->alias : table->table_name;
-      /* remove form file and isam files */
+      /* remove .frm file and engine files */
       build_table_path(path, sizeof(path), db, alias, reg_ext);
     }
-    if (drop_temporary ||
-       (access(path,F_OK) &&
-         ha_create_table_from_engine(thd,db,alias)) ||
-        (!drop_view &&
-	 mysql_frm_type(thd, path, &table_type) != FRMTYPE_TABLE))
+    if (table_type == DB_TYPE_UNKNOWN &&
+        (drop_temporary ||
+         (access(path, F_OK) &&
+          ha_create_table_from_engine(thd, db, alias)) ||
+         (!drop_view &&
+          mysql_frm_type(thd, path, &table_type) != FRMTYPE_TABLE)))
     {
       // Table was not found on disk and table can't be created from engine
       if (if_exists)
@@ -337,7 +352,7 @@ int mysql_rm_table_part2(THD *thd, TABLE_LIST *tables, bool if_exists,
       if (table_type == DB_TYPE_UNKNOWN)
 	mysql_frm_type(thd, path, &table_type);
       *(end=fn_ext(path))=0;			// Remove extension for delete
-      error= ha_delete_table(thd, table_type, path, table->table_name,
+      error= ha_delete_table(thd, table_type, path, db, table->table_name,
                              !dont_log_query);
       if ((error == ENOENT || error == HA_ERR_NO_SUCH_TABLE) && 
 	  (if_exists || table_type == DB_TYPE_UNKNOWN))
@@ -398,16 +413,19 @@ int mysql_rm_table_part2(THD *thd, TABLE_LIST *tables, bool if_exists,
 }
 
 
-int quick_rm_table(enum db_type base,const char *db,
+bool quick_rm_table(enum db_type base,const char *db,
 		   const char *table_name)
 {
   char path[FN_REFLEN];
-  int error=0;
+  bool error= 0;
+  DBUG_ENTER("quick_rm_table");
+
   build_table_path(path, sizeof(path), db, table_name, reg_ext);
   if (my_delete(path,MYF(0)))
-    error=1; /* purecov: inspected */
+    error= 1; /* purecov: inspected */
   *fn_ext(path)= 0;                             // Remove reg_ext
-  return ha_delete_table(current_thd, base, path, table_name, 0) || error;
+  DBUG_RETURN(ha_delete_table(current_thd, base, path, db, table_name, 0) ||
+              error);
 }
 
 /*
@@ -1612,7 +1630,8 @@ bool mysql_create_table(THD *thd,const char *db, const char *table_name,
   if (create_info->row_type == ROW_TYPE_DYNAMIC)
     db_options|=HA_OPTION_PACK_RECORD;
   alias= table_case_name(create_info, table_name);
-  if (!(file=get_new_handler((TABLE*) 0, thd->mem_root, create_info->db_type)))
+  if (!(file=get_new_handler((TABLE_SHARE*) 0, thd->mem_root,
+                             create_info->db_type)))
   {
     my_error(ER_OUTOFMEMORY, MYF(0), 128);//128 bytes invented
     DBUG_RETURN(TRUE);
@@ -1717,8 +1736,8 @@ bool mysql_create_table(THD *thd,const char *db, const char *table_name,
     build_table_path(path, sizeof(path), db, alias, reg_ext);
 
   /* Check if table already exists */
-  if ((create_info->options & HA_LEX_CREATE_TMP_TABLE)
-      && find_temporary_table(thd,db,table_name))
+  if ((create_info->options & HA_LEX_CREATE_TMP_TABLE) &&
+      find_temporary_table(thd, db, table_name))
   {
     if (create_info->options & HA_LEX_CREATE_IF_NOT_EXISTS)
     {
@@ -1744,6 +1763,7 @@ bool mysql_create_table(THD *thd,const char *db, const char *table_name,
       my_error(ER_TABLE_EXISTS_ERROR,MYF(0),table_name);
       goto unlock_and_end;
     }
+    DBUG_ASSERT(get_cached_table_share(db, alias) == 0);
   }
 
   /*
@@ -1777,13 +1797,14 @@ bool mysql_create_table(THD *thd,const char *db, const char *table_name,
     create_info->data_file_name= create_info->index_file_name= 0;
   create_info->table_options=db_options;
 
-  if (rea_create_table(thd, path, db, table_name,
-                       create_info, fields, key_count,
-		       key_info_buffer, file))
+  if (rea_create_table(thd, path, db, table_name, create_info, fields,
+                       key_count, key_info_buffer, file))
     goto unlock_and_end;
+
   if (create_info->options & HA_LEX_CREATE_TMP_TABLE)
   {
     /* Open table and put in temporary table list */
+    *fn_ext(path)= 0;
     if (!(open_temporary_table(thd, path, db, table_name, 1)))
     {
       (void) rm_temporary_table(create_info->db_type, path);
@@ -1868,6 +1889,7 @@ TABLE *create_table_from_items(THD *thd, HA_CREATE_INFO *create_info,
 			       MYSQL_LOCK **lock)
 {
   TABLE tmp_table;		// Used during 'create_field()'
+  TABLE_SHARE share;
   TABLE *table= 0;
   uint select_field_count= items->elements;
   /* Add selected items to field list */
@@ -1879,7 +1901,9 @@ TABLE *create_table_from_items(THD *thd, HA_CREATE_INFO *create_info,
 
   tmp_table.alias= 0;
   tmp_table.timestamp_field= 0;
-  tmp_table.s= &tmp_table.share_not_to_be_used;
+  tmp_table.s= &share;
+  init_tmp_table_share(&share, "", 0, "", "");
+
   tmp_table.s->db_create_options=0;
   tmp_table.s->blob_ptr_size= portable_sizeof_char_ptr;
   tmp_table.s->db_low_byte_first= test(create_info->db_type == DB_TYPE_MYISAM ||
@@ -1970,11 +1994,13 @@ mysql_rename_table(enum db_type base,
   char from[FN_REFLEN], to[FN_REFLEN], lc_from[FN_REFLEN], lc_to[FN_REFLEN];
   char *from_base= from, *to_base= to;
   char tmp_name[NAME_LEN+1];
-  handler *file= (base == DB_TYPE_UNKNOWN ? 0 :
-                  get_new_handler((TABLE*) 0, thd->mem_root, base));
+  handler *file;
   int error=0;
   DBUG_ENTER("mysql_rename_table");
 
+  file= (base == DB_TYPE_UNKNOWN ? 0 :
+         get_new_handler((TABLE_SHARE*) 0, thd->mem_root, base));
+
   build_table_path(from, sizeof(from), old_db, old_name, "");
   build_table_path(to, sizeof(to), new_db, new_name, "");
 
@@ -2035,17 +2061,19 @@ mysql_rename_table(enum db_type base,
 static void wait_while_table_is_used(THD *thd,TABLE *table,
 				     enum ha_extra_function function)
 {
-  DBUG_PRINT("enter",("table: %s", table->s->table_name));
   DBUG_ENTER("wait_while_table_is_used");
-  safe_mutex_assert_owner(&LOCK_open);
+  DBUG_PRINT("enter", ("table: '%s'  share: 0x%lx  db_stat: %u  version: %u",
+                       table->s->table_name.str, (ulong) table->s,
+                       table->db_stat, table->s->version));
 
   VOID(table->file->extra(function));
   /* Mark all tables that are in use as 'old' */
   mysql_lock_abort(thd, table);			// end threads waiting on lock
 
   /* Wait until all there are no other threads that has this table open */
-  remove_table_from_cache(thd, table->s->db,
-                          table->s->table_name, RTFC_WAIT_OTHER_THREAD_FLAG);
+  remove_table_from_cache(thd, table->s->db.str,
+                          table->s->table_name.str,
+                          RTFC_WAIT_OTHER_THREAD_FLAG);
   DBUG_VOID_RETURN;
 }
 
@@ -2167,11 +2195,15 @@ static int prepare_for_restore(THD* thd, TABLE_LIST* table,
 }
 
 
-static int prepare_for_repair(THD* thd, TABLE_LIST *table_list,
+static int prepare_for_repair(THD *thd, TABLE_LIST *table_list,
 			      HA_CHECK_OPT *check_opt)
 {
   int error= 0;
   TABLE tmp_table, *table;
+  TABLE_SHARE *share;
+  char from[FN_REFLEN],tmp[FN_REFLEN+32];
+  const char **ext;
+  MY_STAT stat_info;
   DBUG_ENTER("prepare_for_repair");
 
   if (!(check_opt->sql_flags & TT_USEFRM))
@@ -2179,12 +2211,26 @@ static int prepare_for_repair(THD* thd, TABLE_LIST *table_list,
 
   if (!(table= table_list->table))		/* if open_ltable failed */
   {
-    char name[FN_REFLEN];
-    build_table_path(name, sizeof(name), table_list->db,
-                     table_list->table_name, "");
-    if (openfrm(thd, name, "", 0, 0, 0, &tmp_table))
+    char key[MAX_DBKEY_LENGTH];
+    uint key_length;
+
+    key_length= create_table_def_key(thd, key, table_list, 0);
+    pthread_mutex_lock(&LOCK_open);
+    if (!(share= (get_table_share(thd, table_list, key, key_length, 0,
+                                  &error))))
+    {
+      pthread_mutex_unlock(&LOCK_open);
       DBUG_RETURN(0);				// Can't open frm file
+    }
+
+    if (open_table_from_share(thd, share, "", 0, 0, 0, &tmp_table))
+    {
+      release_table_share(share, RELEASE_NORMAL);
+      pthread_mutex_unlock(&LOCK_open);
+      DBUG_RETURN(0);                           // Out of memory
+    }
     table= &tmp_table;
+    pthread_mutex_unlock(&LOCK_open);
   }
 
   /*
@@ -2197,18 +2243,16 @@ static int prepare_for_repair(THD* thd, TABLE_LIST *table_list,
     - Run a normal repair using the new index file and the old data file
   */
 
-  char from[FN_REFLEN],tmp[FN_REFLEN+32];
-  const char **ext= table->file->bas_ext();
-  MY_STAT stat_info;
-
   /*
     Check if this is a table type that stores index and data separately,
     like ISAM or MyISAM
   */
+  ext= table->file->bas_ext();
   if (!ext[0] || !ext[1])
     goto end;					// No data file
 
-  strxmov(from, table->s->path, ext[1], NullS);	// Name of data file
+  // Name of data file
+  strxmov(from, table->s->normalized_path.str, ext[1], NullS);
   if (!my_stat(from, &stat_info, MYF(0)))
     goto end;				// Can't use USE_FRM flag
 
@@ -2272,7 +2316,11 @@ static int prepare_for_repair(THD* thd, TABLE_LIST *table_list,
 
 end:
   if (table == &tmp_table)
-    closefrm(table);				// Free allocated memory
+  {
+    pthread_mutex_lock(&LOCK_open);
+    closefrm(table, 1);				// Free allocated memory
+    pthread_mutex_unlock(&LOCK_open);
+  }
   DBUG_RETURN(error);
 }
 
@@ -2439,8 +2487,8 @@ static bool mysql_admin_table(THD* thd, TABLE_LIST* tables,
       const char *old_message=thd->enter_cond(&COND_refresh, &LOCK_open,
 					      "Waiting to get writelock");
       mysql_lock_abort(thd,table->table);
-      remove_table_from_cache(thd, table->table->s->db,
-                              table->table->s->table_name,
+      remove_table_from_cache(thd, table->table->s->db.str,
+                              table->table->s->table_name.str,
                               RTFC_WAIT_OTHER_THREAD_FLAG |
                               RTFC_CHECK_KILLED_FLAG);
       thd->exit_cond(old_message);
@@ -2603,8 +2651,8 @@ send_result_message:
       else if (open_for_modify)
       {
         pthread_mutex_lock(&LOCK_open);
-        remove_table_from_cache(thd, table->table->s->db,
-                                table->table->s->table_name, RTFC_NO_FLAG);
+        remove_table_from_cache(thd, table->table->s->db.str,
+                                table->table->s->table_name.str, RTFC_NO_FLAG);
         pthread_mutex_unlock(&LOCK_open);
         /* Something may be modified, that's why we have to invalidate cache */
         query_cache_invalidate3(thd, table->table, 0);
@@ -2782,7 +2830,7 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table,
                              HA_CREATE_INFO *create_info,
                              Table_ident *table_ident)
 {
-  TABLE **tmp_table;
+  TABLE *tmp_table;
   char src_path[FN_REFLEN], dst_path[FN_REFLEN];
   char *db= table->db;
   char *table_name= table->table_name;
@@ -2820,13 +2868,13 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table,
     goto err;
 
   if ((tmp_table= find_temporary_table(thd, src_db, src_table)))
-    strxmov(src_path, (*tmp_table)->s->path, reg_ext, NullS);
+    strxmov(src_path, tmp_table->s->path.str, reg_ext, NullS);
   else
   {
     strxmov(src_path, mysql_data_home, "/", src_db, "/", src_table,
 	    reg_ext, NullS);
     /* Resolve symlinks (for windows) */
-    fn_format(src_path, src_path, "", "", MYF(MY_UNPACK_FILENAME));
+    unpack_filename(src_path, src_path);
     if (lower_case_table_names)
       my_casedn_str(files_charset_info, src_path);
     if (access(src_path, F_OK))
@@ -2866,7 +2914,7 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table,
   {
     strxmov(dst_path, mysql_data_home, "/", db, "/", table_name,
 	    reg_ext, NullS);
-    fn_format(dst_path, dst_path, "", "", MYF(MY_UNPACK_FILENAME));
+    unpack_filename(dst_path, dst_path);
     if (!access(dst_path, F_OK))
       goto table_exists;
   }
@@ -2888,8 +2936,8 @@ bool mysql_create_like_table(THD* thd, TABLE_LIST* table,
     creation, instead create the table directly (for both normal
     and temporary tables).
   */
-  *fn_ext(dst_path)= 0;
-  err= ha_create_table(dst_path, create_info, 1);
+  *fn_ext(dst_path)= 0;                         // Remove .frm
+  err= ha_create_table(thd, dst_path, db, table_name, create_info, 1);
 
   if (create_info->options & HA_LEX_CREATE_TMP_TABLE)
   {
@@ -3466,7 +3514,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
     }
     else
     {
-      if (table->s->tmp_table)
+      if (table->s->tmp_table != NO_TMP_TABLE)
       {
 	if (find_temporary_table(thd,new_db,new_name_buff))
 	{
@@ -3477,7 +3525,8 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       else
       {
 	char dir_buff[FN_REFLEN];
-	strxnmov(dir_buff, FN_REFLEN, mysql_real_data_home, new_db, NullS);
+	strxnmov(dir_buff, sizeof(dir_buff)-1,
+                 mysql_real_data_home, new_db, NullS);
 	if (!access(fn_format(new_name_buff,new_name_buff,dir_buff,reg_ext,0),
 		    F_OK))
 	{
@@ -3510,7 +3559,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       ALTER_DROP_PARTITION + ALTER_COALESCE_PARTITION +
       ALTER_REORGANISE_PARTITION))
   {
-    partition_info *tab_part_info= table->s->part_info;
+    partition_info *tab_part_info= table->part_info;
     if (!tab_part_info)
     {
       my_error(ER_PARTITION_MGMT_ON_NONPARTITIONED, MYF(0));
@@ -3886,11 +3935,11 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
        There was no partitioning before and no partitioning defined.
        Obviously no work needed.
     */
-    if (table->s->part_info)
+    if (table->part_info)
     {
       if (!thd->lex->part_info &&
           create_info->db_type == old_db_type)
-        thd->lex->part_info= table->s->part_info;
+        thd->lex->part_info= table->part_info;
     }
     if (thd->lex->part_info)
     {
@@ -3898,7 +3947,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
         Need to cater for engine types that can handle partition without
         using the partition handler.
       */
-      if (thd->lex->part_info != table->s->part_info)
+      if (thd->lex->part_info != table->part_info)
         partition_changed= TRUE;
       if (create_info->db_type != DB_TYPE_PARTITION_DB)
         thd->lex->part_info->default_engine_type= create_info->db_type;
@@ -3940,6 +3989,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       else
       {
 	*fn_ext(new_name)=0;
+        table->s->version= 0;                   // Force removal of table def
 	close_cached_table(thd, table);
 	if (mysql_rename_table(old_db_type,db,table_name,new_db,new_alias))
 	  error= -1;
@@ -4314,7 +4364,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
 
       */
       uint old_lock_type;
-      partition_info *part_info= table->s->part_info;
+      partition_info *part_info= table->part_info;
       char path[FN_REFLEN+1];
       uint db_options= 0, key_count, syntax_len;
       KEY *key_info_buffer;
@@ -4383,9 +4433,18 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
         DBUG_RETURN(TRUE);
       }
       thd->proc_info="end";
-      write_bin_log(thd, FALSE);
-      send_ok(thd);
-      DBUG_RETURN(FALSE);
+      query_cache_invalidate3(thd, table_list, 0);
+      error= ha_commit_stmt(thd);
+      if (ha_commit(thd))
+        error= 1;
+      if (!error)
+      {
+        close_thread_tables(thd);
+        write_bin_log(thd, FALSE);
+        send_ok(thd);
+        DBUG_RETURN(FALSE);
+      }
+      DBUG_RETURN(error);
     }
   }
 #endif
@@ -4454,15 +4513,17 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       bzero((void*) &tbl, sizeof(tbl));
       tbl.db= new_db;
       tbl.table_name= tbl.alias= tmp_name;
+      /* Table is in thd->temporary_tables */
       new_table= open_table(thd, &tbl, thd->mem_root, (bool*) 0,
                             MYSQL_LOCK_IGNORE_FLUSH);
     }
     else
     {
       char path[FN_REFLEN];
-      my_snprintf(path, sizeof(path), "%s/%s/%s", mysql_data_home,
-                  new_db, tmp_name);
-      fn_format(path,path,"","",4);
+      /* table is a normal table: Create temporary table in same directory */
+      strxnmov(path, sizeof(path)-1, mysql_data_home, "/",new_db, "/",
+               tmp_name, NullS);
+      unpack_filename(path, path);
       new_table=open_temporary_table(thd, path, new_db, tmp_name,0);
     }
     if (!new_table)
@@ -4478,7 +4539,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
   thd->proc_info="copy to tmp table";
   next_insert_id=thd->next_insert_id;		// Remember for logging
   copied=deleted=0;
-  if (new_table && !new_table->s->is_view)
+  if (new_table && !(new_table->file->table_flags() & HA_NO_COPY_ON_ALTER))
   {
     new_table->timestamp_field_type= TIMESTAMP_NO_AUTO_SET;
     new_table->next_number_field=new_table->found_next_number_field;
@@ -4489,7 +4550,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
   thd->last_insert_id=next_insert_id;		// Needed for correct log
   thd->count_cuted_fields= CHECK_FIELD_IGNORE;
 
-  if (table->s->tmp_table)
+  if (table->s->tmp_table != NO_TMP_TABLE)
   {
     /* We changed a temporary table */
     if (error)
@@ -4498,7 +4559,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
 	The following function call will free the new_table pointer,
 	in close_temporary_table(), so we can safely directly jump to err
       */
-      close_temporary_table(thd,new_db,tmp_name);
+      close_temporary_table(thd, new_table, 1, 1);
       goto err;
     }
     /* Close lock if this is a transactional table */
@@ -4508,11 +4569,11 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       thd->lock=0;
     }
     /* Remove link to old table and rename the new one */
-    close_temporary_table(thd, table->s->db, table_name);
+    close_temporary_table(thd, table, 1, 1);
     /* Should pass the 'new_name' as we store table name in the cache */
     if (rename_temporary_table(thd, new_table, new_db, new_name))
     {						// Fatal error
-      close_temporary_table(thd,new_db,tmp_name);
+      close_temporary_table(thd, new_table, 1, 1);
       my_free((gptr) new_table,MYF(0));
       goto err;
     }
@@ -4522,7 +4583,8 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
 
   if (new_table)
   {
-    intern_close_table(new_table);              /* close temporary table */
+    /* close temporary table that will be the new table */
+    intern_close_table(new_table);
     my_free((gptr) new_table,MYF(0));
   }
   VOID(pthread_mutex_lock(&LOCK_open));
@@ -4565,6 +4627,7 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       close the original table at before doing the rename
     */
     table_name=thd->strdup(table_name);		// must be saved
+    table->s->version= 0;                	// Force removal of table def
     close_cached_table(thd, table);
     table=0;					// Marker that table is closed
   }
@@ -4597,18 +4660,24 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
       closing the locked table.
     */
     if (table)
+    {
+      table->s->version= 0;            	        // Force removal of table def
       close_cached_table(thd,table);
+    }
     VOID(pthread_mutex_unlock(&LOCK_open));
     goto err;
   }
   if (thd->lock || new_name != table_name)	// True if WIN32
   {
     /*
-      Not table locking or alter table with rename
-      free locks and remove old table
+      Not table locking or alter table with rename.
+      Free locks and remove old table
     */
     if (table)
+    {
+      table->s->version= 0;              	// Force removal of table def
       close_cached_table(thd,table);
+    }
     VOID(quick_rm_table(old_db_type,db,old_name));
   }
   else
@@ -4631,7 +4700,10 @@ bool mysql_alter_table(THD *thd,char *new_db, char *new_name,
 	reopen_tables(thd,1,0))
     {						// This shouldn't happen
       if (table)
+      {
+        table->s->version= 0;                   // Force removal of table def
 	close_cached_table(thd,table);		// Remove lock for table
+      }
       VOID(pthread_mutex_unlock(&LOCK_open));
       goto err;
     }
@@ -4775,8 +4847,8 @@ copy_data_between_tables(TABLE *from,TABLE *to,
 					      MYF(MY_FAE | MY_ZEROFILL));
     bzero((char*) &tables,sizeof(tables));
     tables.table= from;
-    tables.alias= tables.table_name= (char*) from->s->table_name;
-    tables.db=    (char*) from->s->db;
+    tables.alias= tables.table_name= from->s->table_name.str;
+    tables.db=    from->s->db.str;
     error=1;
 
     if (thd->lex->select_lex.setup_ref_array(thd, order_num) ||