Bug#29261: Sort order of the collation wasn't used when comparing trailing

spaces.

When the my_strnncollsp_simple function compares two strings and one is a prefix
of another then this function compares characters in the rest of longer key
with the space character to find whether the longer key is greater or less.
But the sort order of the collation isn't used in this comparison. This may
lead to a wrong comparison result, wrongly created index or wrong order of the
result set of a query with the ORDER BY clause.

Now the my_strnncollsp_simple function uses collation sort order to compare
the characters in the rest of longer key with the space character.


mysql-test/t/ctype_collate.test:
  Added a test case for the bug#29261: Sort order of the collation wasn't used
  when comparing trailing spaces.
mysql-test/r/ctype_collate.result:
  Added a test case for the bug#29261: Sort order of the collation wasn't used
  when comparing trailing spaces.
strings/ctype-simple.c:
  Bug#29261: Sort order of the collation wasn't used when comparing trailing
  spaces.
  Now the my_strnncollsp_simple function uses collation sort order to compare
  the characters in the rest of longer key with the space character.

1 files changed, 1 insertions, 1 deletions

diff --git a/strings/ctype-simple.c b/strings/ctype-simple.c
index e57204f8d33..fca5607e152 100644
--- a/strings/ctype-simple.c
+++ b/strings/ctype-simple.c
@@ -180,7 +180,7 @@ int my_strnncollsp_simple(CHARSET_INFO * cs, const uchar *a, uint a_length,
     for (end= a + a_length-length; a < end ; a++)
     {
       if (*a != ' ')
-	return (*a < ' ') ? -swap : swap;
+	return (map[*a] < ' ') ? -swap : swap;
     }
   }
   return res;