Bug #18359924: INNODB AND MYISAM CORRUPTION ON PREFIX INDEXES

The problem was in the validation of the input data for blob types.
When assigned binary data, the character blob types were only checking if 
the length of these data is a multiple of the minimum char length for the 
destination charset. 
And since e.g. UTF-8's minimum character length is 1 (becuase it's 
variable length) even byte sequences that are invalid utf-8 strings (e.g. 
wrong leading byte etc) were copied verbatim into utf-8 columns when
coming from binary strings or fields.
Storing invalid data into string columns was having all kinds of ill effects 
on code that assumed that the encoding data are valid to begin with.

Fixed by additionally checking the incoming binary string for validity when 
assigning it to a non-binary string column.
Made sure the conversions to charsets with no known "invalid" ranges 
are not covered by the extra check.
Removed trailing spaces.

Test case added.

1 files changed, 3 insertions, 5 deletions

diff --git a/sql/field.cc b/sql/field.cc
index c65346423a0..3069a1ef60b 100644
--- a/sql/field.cc
+++ b/sql/field.cc
@@ -7376,8 +7376,7 @@ int Field_blob::store(const char *from,uint length,CHARSET_INFO *cs)
       If content of the 'from'-address is cached in the 'value'-object
       it is possible that the content needs a character conversion.
     */
-    uint32 dummy_offset;
-    if (!String::needs_conversion(length, cs, field_charset, &dummy_offset))
+    if (!String::needs_conversion_on_storage(length, cs, field_charset))
     {
       Field_blob::store_length(length);
       bmove(ptr+packlength, &from, sizeof(char*));
@@ -7980,7 +7979,7 @@ int Field_enum::store(const char *from,uint length,CHARSET_INFO *cs)
   String tmpstr(buff,sizeof(buff), &my_charset_bin);
 
   /* Convert character set if necessary */
-  if (String::needs_conversion(length, cs, field_charset, &not_used))
+  if (String::needs_conversion_on_storage(length, cs, field_charset))
   { 
     uint dummy_errors;
     tmpstr.copy(from, length, cs, field_charset, &dummy_errors);
@@ -8196,12 +8195,11 @@ int Field_set::store(const char *from,uint length,CHARSET_INFO *cs)
   int err= 0;
   char *not_used;
   uint not_used2;
-  uint32 not_used_offset;
   char buff[STRING_BUFFER_USUAL_SIZE];
   String tmpstr(buff,sizeof(buff), &my_charset_bin);
 
   /* Convert character set if necessary */
-  if (String::needs_conversion(length, cs, field_charset, &not_used_offset))
+  if (String::needs_conversion_on_storage(length, cs, field_charset))
   { 
     uint dummy_errors;
     tmpstr.copy(from, length, cs, field_charset, &dummy_errors);