diff options
author | Olivier Bertrand <bertrandop@gmail.com> | 2014-07-20 12:31:42 +0200 |
---|---|---|
committer | Olivier Bertrand <bertrandop@gmail.com> | 2014-07-20 12:31:42 +0200 |
commit | 4b2092ebeac009ba1cfbb2c8a6c3ace0eab76bcb (patch) | |
tree | f9a451304c1dabf1a6b320a54d252d36cdf66487 | |
parent | 6c8fbe61ea5724d6beaa03345ead8a5d6a9366f7 (diff) | |
download | mariadb-git-4b2092ebeac009ba1cfbb2c8a6c3ace0eab76bcb.tar.gz |
This is a new version of the CONNECT storage engine. It was developed in
a sub-branch of this one and merged by pushing all the changes from it.
This version adds the following to CONNECT:
- MRR support (similar to the MyISAM one)
- Block, Remote and dynamic indexing
- Partitioning support (using the PARTITION engine)
Here is a list of the commited changes made in the sub-branch:
========================================================================
------------------------------------------------------------
revno: 4009
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Thu 2014-07-17 18:13:51 +0200
message:
This commit brings many changes, in particular two important ones:
1) Support of partitioning by connect. A table can be partitioned
by files, this is an enhanced MULTIPLE table. It can be also
partitioned by sub-tables like TBL and this enables table sharding.
2) Handling a CONNECT bug that causes in some cases extraneous rows
to remain in the table after an UPDATE or DELETE when the command
uses indexing (for not fixed file tables). Until a real fix is
done, CONNECT tries to ignore indexing and if it cannot do it
abort the command with an error message.
- Add tests on partitioning
added:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
storage/connect/mysql-test/connect/t/part_table.test
- Temporary fix
modified:
sql/sql_partition.cc
- Add partition support
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
storage/connect/reldef.h
storage/connect/tabdos.cpp
- Add functions ha_connect::IsUnique and ha_connect::CheckColumnList
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Prevent updating a partition table column that is part of
the partition function (outward tables only)
modified:
storage/connect/ha_connect.cc
- Support INSERT/UPDATE/DELETE for PROXY tables
modified:
storage/connect/tabutil.cpp
- Handle the bug on updating rows via indexing. Waiting for a real fix,
Don't use indexing when possible else raise an error and abort.
modified:
storage/connect/ha_connect.cc
- dbuserp->UseTemp set to TMP_AUTO
modified:
storage/connect/connect.cc
- Add members nox, abort and only
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Add arguments nox and abort to CntCloseTable
modified:
storage/connect/connect.cc
storage/connect/connect.h
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/ha_connect.cc
- Add arguments abort to CloseTableFile and RenameTempFile
modified:
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabvct.cpp
storage/connect/xtable.h
- Fix info->records when file does not exists
modified:
storage/connect/connect.cc
- Close XML table when opened for info
modified:
storage/connect/connect.cc
- Add function VCTFAM::GetFileLength
modified:
storage/connect/filamvct.cpp
storage/connect/filamvct.h
- Column option DISTRIB -> ENUM
modified:
storage/connect/ha_connect.cc
- Options connect, query_string and partname allways available
modified:
storage/connect/ha_connect.cc
- Add function MYSQLC::GetTableSize
modified:
storage/connect/myconn.cpp
storage/connect/myconn.h
- Add new special columns (PARTNAME, FNAME, FPATH, FTYPE and FDISK)
modified:
storage/connect/colblk.cpp
storage/connect/colblk.h
storage/connect/plgdbsem.h
storage/connect/table.cpp
- Add function ExtractFromPath
modified:
storage/connect/colblk.cpp
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
- Enhance Cardinality for some table types
modified:
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabsys.cpp
storage/connect/tabsys.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
- Add test on special column
modified:
storage/connect/tabfmt.cpp
- Add new files (added for block indexing)
modified:
storage/connect/CMakeLists.txt
------------------------------------------------------------
revno: 4007 [merge]
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-31 12:31:26 +0200
message:
- Begin adding support of partition tables
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
- Add INSERT/UPDATE support to PROXY tables
modified:
storage/connect/tabutil.cpp
storage/connect/tabutil.h
- Take care of SPECIAL columns
modified:
storage/connect/filamdbf.cpp
storage/connect/reldef.h
storage/connect/tabfmt.cpp
-Typo and misc
modified:
storage/connect/odbconn.cpp
storage/connect/tabfix.cpp
storage/connect/xindex.cpp
------------------------------------------------------------
revno: 4006
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-10 12:21:08 +0200
message:
- FIX some MAP and XMAP errors (such as mapped indexes not closed)
Do not put version in XML files header
Remove HTON_NO_PARTITION for testing
Fix a wrong return (instead of DBUG_RETURN) in index_init
Plus a few typos
modified:
storage/connect/connect.cc
storage/connect/filter.cpp
storage/connect/ha_connect.cc
storage/connect/maputil.cpp
storage/connect/mysql-test/connect/r/alter_xml.result
storage/connect/mysql-test/connect/r/xml.result
storage/connect/table.cpp
storage/connect/tabxml.cpp
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 4005
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Fri 2014-05-02 15:55:45 +0200
message:
- Adding fetched columns to Dynamic index key (unique only)
Fix two bugs concerning added KXYCOL's:
1 - Not set during reading
2 - Val_K not set in FastFind
modified:
storage/connect/connect.cc
storage/connect/filamtxt.h
storage/connect/tabdos.cpp
storage/connect/tabfix.cpp
storage/connect/table.cpp
storage/connect/valblk.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 4003
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Wed 2014-04-30 10:48:29 +0200
message:
- Implementation of adding selected columns to dynamic indexes.
modified:
storage/connect/connect.cc
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabvct.cpp
storage/connect/tabvct.h
storage/connect/xindex.cpp
storage/connect/xindex.h
------------------------------------------------------------
revno: 4001
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-04-26 00:17:26 +0200
message:
- Implement dynamic indexing
modified:
storage/connect/connect.cc
storage/connect/filter.cpp
storage/connect/filter.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/table.cpp
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 3995
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sun 2014-03-23 18:49:19 +0100
message:
- Work in progress
modified:
storage/connect/filter.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/mysql-test/connect/r/alter.result
storage/connect/mysql-test/connect/r/xml.result
------------------------------------------------------------
revno: 3991
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Mon 2014-03-10 18:59:36 +0100
message:
- Adding files needed for block indexing
added:
storage/connect/array.cpp
storage/connect/array.h
storage/connect/blkfil.cpp
storage/connect/blkfil.h
storage/connect/filter.cpp
storage/connect/filter.h
========================================================================
This commit of the main branch adds:
- A change needed to have the engine function check_if_supported_inplace_alter
called for partition tables (was done manually in the sub-branch) by adding
the preparser define: PARTITION_SUPPORTS_INPLACE_ALTER
modified:
sql/CMakeLists.txt
- A fix concerning the FileExists function. It was needed to force the function
table_flags to return the same flags for all partitions. This is tested by
the partition engine and raises an error if flags are not equal.
The way file name, table name and connection string are retrieved has been
modified to cope with it.
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
- A few typos, such as the version string.
modified:
storage/connect/ha_connect.cc
- Updating some test result files because some warnings are no more raised.
modified:
storage/connect/mysql-test/connect/r/occur.result
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/pivot.result
-rw-r--r-- | sql/CMakeLists.txt | 2 | ||||
-rw-r--r-- | storage/connect/ha_connect.cc | 45 | ||||
-rw-r--r-- | storage/connect/ha_connect.h | 3 | ||||
-rw-r--r-- | storage/connect/mysql-test/connect/r/occur.result | 2 | ||||
-rw-r--r-- | storage/connect/mysql-test/connect/r/part_file.result | 4 | ||||
-rw-r--r-- | storage/connect/mysql-test/connect/r/pivot.result | 2 | ||||
-rw-r--r-- | storage/connect/reldef.cpp | 13 |
7 files changed, 41 insertions, 30 deletions
diff --git a/sql/CMakeLists.txt b/sql/CMakeLists.txt index ad4b12813d3..18f78cfd59b 100644 --- a/sql/CMakeLists.txt +++ b/sql/CMakeLists.txt @@ -30,7 +30,7 @@ ${CMAKE_CURRENT_BINARY_DIR}/lex_hash.h SET_SOURCE_FILES_PROPERTIES(${GEN_SOURCES} PROPERTIES GENERATED 1) -ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS) +ADD_DEFINITIONS(-DMYSQL_SERVER -DHAVE_EVENT_SCHEDULER -DHAVE_POOL_OF_THREADS -DPARTITION_SUPPORTS_INPLACE_ALTER) IF(SSL_DEFINES) ADD_DEFINITIONS(${SSL_DEFINES}) ENDIF() diff --git a/storage/connect/ha_connect.cc b/storage/connect/ha_connect.cc index f0261dda7ff..ff66b03649c 100644 --- a/storage/connect/ha_connect.cc +++ b/storage/connect/ha_connect.cc @@ -170,7 +170,7 @@ #define SZWMIN 4194304 // Minimum work area size 4M extern "C" { - char version[]= "Version 1.03.0002 May 03, 2014"; + char version[]= "Version 1.03.0002 July 17, 2014"; char compver[]= "Version 1.03.0002 " __DATE__ " " __TIME__; #if defined(WIN32) @@ -745,7 +745,7 @@ ulonglong ha_connect::table_flags() const flags|= (HA_HAS_RECORDS | HA_STATS_RECORDS_IS_EXACT); // No data change on ALTER for outward tables - if (!IsFileType(type) || hp->FileExists(pos->filename)) + if (!IsFileType(type) || hp->FileExists(pos->filename, true)) flags|= HA_NO_COPY_ON_ALTER; } // endif pos @@ -815,6 +815,22 @@ PTOS ha_connect::GetTableOptionStruct(TABLE_SHARE *s) } // end of GetTableOptionStruct /****************************************************************************/ +/* Return the string eventually formatted with partition name. */ +/****************************************************************************/ +char *ha_connect::GetRealString(const char *s) +{ + char *sv; + + if (IsPartitioned() && s) { + sv= (char*)PlugSubAlloc(xp->g, NULL, strlen(s) + strlen(partname)); + sprintf(sv, s, partname); + } else + sv= (char*)s; + + return sv; +} // end of GetRealString + +/****************************************************************************/ /* Return the value of a string option or NULL if not specified. */ /****************************************************************************/ char *ha_connect::GetStringOption(char *opname, char *sdef) @@ -826,7 +842,7 @@ char *ha_connect::GetStringOption(char *opname, char *sdef) LEX_STRING cnc= (tshp) ? tshp->connect_string : table->s->connect_string; if (cnc.length) - opval= cnc.str; + opval= GetRealString(cnc.str); } else if (!stricmp(opname, "Query_String")) opval= thd_query_string(table->in_use)->str; @@ -837,11 +853,11 @@ char *ha_connect::GetStringOption(char *opname, char *sdef) else if (!stricmp(opname, "Type")) opval= (char*)options->type; else if (!stricmp(opname, "Filename")) - opval= (char*)options->filename; + opval= GetRealString(options->filename); else if (!stricmp(opname, "Optname")) opval= (char*)options->optname; else if (!stricmp(opname, "Tabname")) - opval= (char*)options->tabname; + opval= GetRealString(options->tabname); else if (!stricmp(opname, "Tablist")) opval= (char*)options->tablist; else if (!stricmp(opname, "Database") || @@ -5665,7 +5681,7 @@ int ha_connect::create(const char *name, TABLE *table_arg, #endif // WITH_PARTITION_STORAGE_ENGINE if (g->Alchecked == 0 && - (!IsFileType(type) || FileExists(options->filename))) { + (!IsFileType(type) || FileExists(options->filename, false))) { if (part_info) { sprintf(g->Message, "Data repartition in %s is unchecked", partname); push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0, g->Message); @@ -5699,7 +5715,8 @@ int ha_connect::create(const char *name, TABLE *table_arg, #if defined(WITH_PARTITION_STORAGE_ENGINE) if (part_info) - strcpy(partname, strrchr(name, (inward ? slash : '#')) + 1); + strcpy(partname, + decode(g, strrchr(name, (inward ? slash : '#')) + 1)); #endif // WITH_PARTITION_STORAGE_ENGINE if ((rc= optimize(table->in_use, NULL))) { @@ -5736,10 +5753,12 @@ int ha_connect::create(const char *name, TABLE *table_arg, - file does not exist or is void - user has file privilege */ -bool ha_connect::FileExists(const char *fn) +bool ha_connect::FileExists(const char *fn, bool bf) { if (!fn || !*fn) return false; + else if (IsPartitioned() && bf) + return true; if (table) { char *s, tfn[_MAX_PATH], filename[_MAX_PATH], path[128]; @@ -5904,7 +5923,7 @@ bool ha_connect::NoFieldOptionChange(TABLE *tab) */ enum_alter_inplace_result ha_connect::check_if_supported_inplace_alter(TABLE *altered_table, - Alter_inplace_info *ha_alter_info) + Alter_inplace_info *ha_alter_info) { DBUG_ENTER("check_if_supported_alter"); @@ -6007,7 +6026,7 @@ ha_connect::check_if_supported_inplace_alter(TABLE *altered_table, char *fn= GetStringOption("filename"); tshp= NULL; - if (FileExists(fn)) { + if (FileExists(fn, false)) { strcpy(g->Message, "Operation denied. Table data would be lost."); my_message(ER_UNKNOWN_ERROR, g->Message, MYF(0)); DBUG_RETURN(HA_ALTER_ERROR); @@ -6067,8 +6086,10 @@ fin: "Alter operations not supported together by CONNECT", MYF(0)); DBUG_RETURN(HA_ALTER_ERROR); } else if (outward) { - push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0, - "This is an outward table, table data were not modified."); + if (IsFileType(type)) + push_warning(thd, Sql_condition::WARN_LEVEL_WARN, 0, + "This is an outward table, table data were not modified."); + DBUG_RETURN(HA_ALTER_INPLACE_EXCLUSIVE_LOCK); } else DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED); diff --git a/storage/connect/ha_connect.h b/storage/connect/ha_connect.h index e5b58a7a6b8..610e9019b60 100644 --- a/storage/connect/ha_connect.h +++ b/storage/connect/ha_connect.h @@ -182,6 +182,7 @@ public: static bool connect_init(void); static bool connect_end(void); TABTYPE GetRealType(PTOS pos= NULL); + char *GetRealString(const char *s); char *GetStringOption(char *opname, char *sdef= NULL); PTOS GetTableOptionStruct(TABLE_SHARE *s= NULL); bool GetBooleanOption(char *opname, bool bdef); @@ -193,7 +194,7 @@ public: bool SetIntegerOption(char *opname, int n); bool SameInt(TABLE *tab, char *opn); bool SameBool(TABLE *tab, char *opn); - bool FileExists(const char *fn); + bool FileExists(const char *fn, bool bf); bool NoFieldOptionChange(TABLE *tab); PFOS GetFieldOptionStruct(Field *fp); void *GetColumnOption(PGLOBAL g, void *field, PCOLINFO pcf); diff --git a/storage/connect/mysql-test/connect/r/occur.result b/storage/connect/mysql-test/connect/r/occur.result index fbcda4660be..a497dfc9942 100644 --- a/storage/connect/mysql-test/connect/r/occur.result +++ b/storage/connect/mysql-test/connect/r/occur.result @@ -193,8 +193,6 @@ Kevin 8 Lisbeth 2 Mary 2 ALTER TABLE xpet MODIFY number INT NOT NULL; -Warnings: -Warning 1105 This is an outward table, table data were not modified. SELECT * FROM xpet; name race number John dog 2 diff --git a/storage/connect/mysql-test/connect/r/part_file.result b/storage/connect/mysql-test/connect/r/part_file.result index b342059bff0..4a343500bf7 100644 --- a/storage/connect/mysql-test/connect/r/part_file.result +++ b/storage/connect/mysql-test/connect/r/part_file.result @@ -88,8 +88,6 @@ FNAME FTYPE # Testing partitioning on a void outward table # ALTER TABLE dr1 file_name='part*.*'; -Warnings: -Warning 1105 This is an outward table, table data were not modified. CREATE TABLE t1 ( rwid INT(6) DEFAULT 0 SPECIAL=ROWID, rnum INT(6) DEFAULT 0 SPECIAL=ROWNUM, @@ -104,8 +102,6 @@ PARTITION by range columns(id) ( PARTITION `1` VALUES LESS THAN(10), PARTITION `2` VALUES LESS THAN(50), PARTITION `3` VALUES LESS THAN(MAXVALUE)); -Warnings: -Warning 1105 Open(rb) error 2 on DATADIR/test/part%s.txt: No such file or directory SHOW INDEX FROM t1; Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment t1 0 PRIMARY 1 id A NULL NULL NULL XINDEX diff --git a/storage/connect/mysql-test/connect/r/pivot.result b/storage/connect/mysql-test/connect/r/pivot.result index 4b39a21d3d9..349db89fa35 100644 --- a/storage/connect/mysql-test/connect/r/pivot.result +++ b/storage/connect/mysql-test/connect/r/pivot.result @@ -59,8 +59,6 @@ Joe 5 14.00 0.00 12.00 # Restricting the columns in a Pivot Table # ALTER TABLE pivex DROP COLUMN week; -Warnings: -Warning 1105 This is an outward table, table data were not modified. SELECT * FROM pivex; Who Beer Car Food Beth 51.00 0.00 29.00 diff --git a/storage/connect/reldef.cpp b/storage/connect/reldef.cpp index f483171ea7a..58bcbd202f3 100644 --- a/storage/connect/reldef.cpp +++ b/storage/connect/reldef.cpp @@ -143,16 +143,13 @@ char *RELDEF::GetStringCatInfo(PGLOBAL g, PSZ what, PSZ sdef) char *name, *sval= NULL, *s= Hc->GetStringOption(what, sdef); if (s) { - if (Hc->IsPartitioned() && - (!stricmp(what, "filename") || !stricmp(what, "tabname") - || !stricmp(what, "connect"))) { - name= Hc->GetPartName(); - sval= (char*)PlugSubAlloc(g, NULL, strlen(s) + strlen(name)); - sprintf(sval, s, name); - } else { + if (!Hc->IsPartitioned() || + (stricmp(what, "filename") && stricmp(what, "tabname") + && stricmp(what, "connect"))) { sval= (char*)PlugSubAlloc(g, NULL, strlen(s) + 1); strcpy(sval, s); - } // endif partitioned + } else + sval= s; } else if (!stricmp(what, "filename")) { // Return default file name |