Fix for bug #27643 "query failed : 1114 (The table '' is full)

Problem:

HASH indexes on VARCHAR columns with binary collations did not ignore trailing spaces from strings before comparisons. This could result in duplicate records being successfully inserted into a MEMORY table with unique key constraints.

As a direct consequence of the above, internal MEMORY tables used for GROUP BY calculation in testcases for bug #27643 contained duplicate rows which resulted in duplicate key errors when converting those temporary tables to MyISAM. Additionally, that error was incorrectly converted to the 'table is full' error.

Solution:

- ignore trailing spaces in VARCHAR fields with binary collations when calculating hashes.
- return a proper error from create_myisam_from_heap() when conversion fails.


mysql-test/r/ctype_ucs2_def.result:
  Added a testcase for bug #27643.
mysql-test/r/heap_hash.result:
  Added a testcase for bug #27643.
mysql-test/t/ctype_ucs2_def.test:
  Added a testcase for bug #27643.
mysql-test/t/heap_hash.test:
  Added a testcase for bug #27643.
sql/sql_select.cc:
  Return an appropriate error instead of 'table is full' when conversion from MEMORY to MyISAM fails.
strings/ctype-bin.c:
  Added my_hash_sort_8bit_bin() which ignores trailing spaces when calculating hashes, and is now used for VARCHAR columns instead of my_hash_sort_bin().
strings/ctype-mb.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_mb_bin().
strings/ctype-ucs2.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_ucs2_bin().

1 files changed, 16 insertions, 0 deletions

diff --git a/mysql-test/r/heap_hash.result b/mysql-test/r/heap_hash.result
index 80abcebbfea..cb47573b502 100644
--- a/mysql-test/r/heap_hash.result
+++ b/mysql-test/r/heap_hash.result
@@ -366,3 +366,19 @@ explain select a from t1 where a in (1,3);
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
 1	SIMPLE	t1	range	a	a	5	NULL	4	Using where
 drop table t1;
+End of 4.1 tests
+CREATE TABLE t1(col1 VARCHAR(32) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, 
+col2 VARCHAR(32) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, 
+UNIQUE KEY key1 USING HASH (col1, col2)) ENGINE=MEMORY;
+INSERT INTO t1 VALUES('A', 'A');
+INSERT INTO t1 VALUES('A ', 'A ');
+ERROR 23000: Duplicate entry 'A -A ' for key 1
+DROP TABLE t1;
+CREATE TABLE t1(col1 VARCHAR(32) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, 
+col2 VARCHAR(32) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, 
+UNIQUE KEY key1 USING HASH (col1, col2)) ENGINE=MEMORY;
+INSERT INTO t1 VALUES('A', 'A');
+INSERT INTO t1 VALUES('A ', 'A ');
+ERROR 23000: Duplicate entry 'A -A ' for key 1
+DROP TABLE t1;
+End of 5.0 tests