Fixed bug #18360: Type aggregation for IN and CASE may lead to a wrong

result

The IN function aggregates result types of all expressions. It uses that 
type in comparison of left expression and expressions in right part. 
This approach works in most cases. But let's consider the case when the
right part contains both strings and integers. In that case this approach may
cause wrong results because all strings which do not start with a digit are
evaluated as 0.
CASE uses the same approach when a CASE expression is given thus it's also
affected.

The idea behind this fix is to make IN function to compare expressions with
different result types differently. For example a string in the left
part will be compared as string with strings specified in right part and
will be converted to real for comparison to int or real items in the right
part.

A new function called collect_cmp_types() is added. It collects different
result types for comparison of first item in the provided list with each 
other item in the list. 

The Item_func_in class now can refer up to 5 cmp_item objects: 1 for each
result type for comparison purposes. cmp_item objects are allocated according
to found result types. The comparison of the left expression with any
right part expression is now based only on result types of these expressions.

The Item_func_case class is modified in the similar way when a CASE
expression is specified. Now it can allocate up to 5 cmp_item objects
to compare CASE expression with WHEN expressions of different types.
The comparison of the CASE expression with any WHEN expression now based only 
on result types of these expressions.



sql/item.cc:
  Cleaned up an outdated comment.
sql/item_cmpfunc.cc:
      Fixed bug #18360: Type aggregation for IN and CASE may lead to a wrong result
      A new function called collect_cmp_types() is added. It collects different
      result types for comparison of first item in the provided list with each 
      other item in the list. 
      The Item_func_in class now can refer up to 5 cmp_item objects: 1 for each
      result type for comparison purposes. cmp_item objects are allocated according
      to found result types. The comparison of the left expression with any
      right part expression is now based only on result types of these expressions.
      The Item_func_case class is modified in the similar way when a CASE
      expression is specified. Now it can allocate up to 5 cmp_item objects
      to compare CASE expression with WHEN expressions of different types.
      The comparison of the CASE expression with any WHEN expression now based only 
      on result types of these expressions.
sql/item_cmpfunc.h:
      Fixed bug#18360: Type aggregation for IN and CASE may lead to a wrong result
      The Item_func_in class now can refer up to 5 cmp_item objects.
      The Item_func_case class is modified in the similar way.
sql/opt_range.cc:
      Fixed bug #18360: Type aggregation for IN and CASE may lead to a wrong
      resultSmall code changes due to changes in Item_func_in class.
mysql-test/t/view.test:
  Added a test comment
mysql-test/t/func_in.test:
  Added a test case for bug#18360: Type aggregation for IN and CASE may lead to a wrong
  result
mysql-test/r/func_in.result:
  Added a test case for bug#18360: Type aggregation for IN and CASE may lead to a wrong
  result

1 files changed, 68 insertions, 0 deletions

diff --git a/mysql-test/r/func_in.result b/mysql-test/r/func_in.result
index 0236cbfe26f..b88e5a66f96 100644
--- a/mysql-test/r/func_in.result
+++ b/mysql-test/r/func_in.result
@@ -343,3 +343,71 @@ some_id
 1
 2
 drop table t1;
+create table t1(f1 char(1));
+insert into t1 values ('a'),('b'),('1');
+select f1 from t1 where f1 in ('a',1);
+f1
+a
+1
+select f1, case f1 when 'a' then '+' when 1 then '-' end from t1;
+f1	case f1 when 'a' then '+' when 1 then '-' end
+a	+
+b	NULL
+1	-
+create index t1f1_idx on t1(f1);
+select f1 from t1 where f1 in ('a',1);
+f1
+1
+a
+explain select f1 from t1 where f1 in ('a',1);
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t1	index	NULL	t1f1_idx	2	NULL	3	Using where; Using index
+select f1 from t1 where f1 in ('a','b');
+f1
+a
+b
+explain select f1 from t1 where f1 in ('a','b');
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t1	index	t1f1_idx	t1f1_idx	2	NULL	3	Using where; Using index
+select f1 from t1 where f1 in (2,1);
+f1
+1
+explain select f1 from t1 where f1 in (2,1);
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t1	index	t1f1_idx	t1f1_idx	2	NULL	3	Using where; Using index
+create table t2(f2 int, index t2f2(f2));
+insert into t2 values(0),(1),(2);
+select f2 from t2 where f2 in ('a',2);
+f2
+0
+2
+Warnings:
+Warning	1292	Truncated incorrect DOUBLE value: 'a'
+Warning	1292	Truncated incorrect DOUBLE value: 'a'
+Warning	1292	Truncated incorrect DOUBLE value: 'a'
+explain select f2 from t2 where f2 in ('a',2);
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t2	index	NULL	t2f2	5	NULL	3	Using where; Using index
+select f2 from t2 where f2 in ('a','b');
+f2
+0
+Warnings:
+Warning	1292	Truncated incorrect DOUBLE value: 'a'
+Warning	1292	Truncated incorrect DOUBLE value: 'b'
+explain select f2 from t2 where f2 in ('a','b');
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t2	index	t2f2	t2f2	5	NULL	3	Using where; Using index
+Warnings:
+Warning	1292	Truncated incorrect DOUBLE value: 'a'
+Warning	1292	Truncated incorrect DOUBLE value: 'b'
+select f2 from t2 where f2 in (1,'b');
+f2
+0
+1
+Warnings:
+Warning	1292	Truncated incorrect DOUBLE value: 'b'
+Warning	1292	Truncated incorrect DOUBLE value: 'b'
+explain select f2 from t2 where f2 in (1,'b');
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t2	index	NULL	t2f2	5	NULL	3	Using where; Using index
+drop table t1, t2;