Bug#20086: Can't get data from key partitioned tables with VARCHAR key

The problem appeared because the same values produced different hash
during INSERT and SELECT for VARCHAR data type.
Fix:
VARCHAR required special treatment to avoid hashing of length bytes
(leftmost one or two bytes) as well as trailing bytes beyond real length,
which could contain garbage. Fix is done by introducing hash() - new method
in the Field class.


mysql-test/r/partition_innodb.result:
  Adding test case
mysql-test/r/partition_pruning.result:
  Fixing test results (results differ due to changes in hash function)
mysql-test/t/partition_innodb.test:
  Adding test case
sql/field.cc:
  Adding generic hash() method, and a special
  method for VARCHAR.
sql/field.h:
  Adding prototypes for new methods
sql/key.cc:
  Mark columns for write before executinf of set_key_image().
  Thanks for Mikael for suggesting this fix.
sql/sql_partition.cc:
  Removing old hash code.
  Using new methid field->hash() instead.

1 files changed, 30 insertions, 0 deletions

diff --git a/sql/field.cc b/sql/field.cc
index b51e5b63779..0105c780d12 100644
--- a/sql/field.cc
+++ b/sql/field.cc
@@ -1243,6 +1243,21 @@ uint Field::offset()
 }
 
 
+void Field::hash(ulong *nr, ulong *nr2)
+{
+  if (is_null())
+  {
+    *nr^= (*nr << 1) | 1;
+  }
+  else
+  {
+    uint len= pack_length();
+    CHARSET_INFO *cs= charset();
+    cs->coll->hash_sort(cs, (uchar*) ptr, len, nr, nr2);
+  }
+}
+
+
 void Field::copy_from_tmp(int row_offset)
 {
   memcpy(ptr,ptr+row_offset,pack_length());
@@ -6923,6 +6938,21 @@ uint Field_varstring::is_equal(create_field *new_field)
 }
 
 
+void Field_varstring::hash(ulong *nr, ulong *nr2)
+{
+  if (is_null())
+  {
+    *nr^= (*nr << 1) | 1;
+  }
+  else
+  {
+    uint len=  length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
+    CHARSET_INFO *cs= charset();
+    cs->coll->hash_sort(cs, (uchar*) ptr + length_bytes, len, nr, nr2);
+  }
+}
+
+
 /****************************************************************************
 ** blob type
 ** A blob is saved as a length and a pointer. The length is stored in the