MDEV-14862: Server crashes in Bitmap<64u>::merge / add_key_fieldbb-10.0-MDEV-14862

It is backport of MySQL fix:

Bug #19434916: FATAL_SIGNAL IN ADD_KEY_EQUAL_FIELDS() WITH
               UPDATE VIEW USING OUTER SUBQUERY

Issue:
-----
While resolving a column which refers to a table/view in an
outer query, it's respecitve item object is marked with the
outer query's select_lex object. But when the column refers
to a view or if the column is part of a subquery in the
HAVING clause, an Item_ref object is created. While the
reference to the outer query is stored by the Item_ref
object, the same is not stored in it's real_item.

This creates a problem with the IN-TO-EXISTS optmization.
When there is an index over the column in the inner query,
it will be considered since the column's real_item object
will be mistaken for a local field. This will lead to a
crash.

SOLUTION:
---------
Under the current design, the only way to fix this issue is
to check the reginfo.join_tab for a NULL value. If yes, the
query should not be worrying about the key use.

The testcase and comments added as part of the fix for
Bug#17766653 have been backported.

4 files changed, 58 insertions, 1 deletions

diff --git a/mysql-test/r/update_innodb.result b/mysql-test/r/update_innodb.result
index 88c86c50625..9ab12c28c3e 100644
--- a/mysql-test/r/update_innodb.result
+++ b/mysql-test/r/update_innodb.result
@@ -29,3 +29,12 @@ CREATE ALGORITHM=UNDEFINED VIEW `v1` AS select `t4`.`c1` AS `c1`,`t4`.`c2` AS `c
 UPDATE t1 a JOIN t2 b ON a.c1 = b.c1 JOIN v1 vw ON b.c2 = vw.c1 JOIN t3 del ON vw.c2 = del.c2 SET a.c2 = ( SELECT max(t.c1) FROM t3 t, v1 i WHERE del.c2 = t.c2 AND vw.c3 = i.c3 AND t.c3 = 4 ) WHERE a.c2 IS NULL OR a.c2 < '2011-05-01';
 drop view v1;
 drop table t1,t2,t3,t4;
+#
+# MDEV-14862: Server crashes in Bitmap<64u>::merge / add_key_field
+#
+CREATE TABLE t1 (a INT) ENGINE=InnoDB;
+CREATE VIEW v1 AS SELECT * FROM t1;
+CREATE TABLE t2 (b INT) ENGINE=InnoDB;
+DELETE FROM v1 WHERE a IN ( SELECT a FROM t2 );
+drop view v1;
+drop table t1,t2;
diff --git a/mysql-test/t/update_innodb.test b/mysql-test/t/update_innodb.test
index 67c356c4e2e..4ea321a60d2 100644
--- a/mysql-test/t/update_innodb.test
+++ b/mysql-test/t/update_innodb.test
@@ -37,3 +37,15 @@ UPDATE t1 a JOIN t2 b ON a.c1 = b.c1 JOIN v1 vw ON b.c2 = vw.c1 JOIN t3 del ON v
 
 drop view v1;
 drop table t1,t2,t3,t4;
+
+--echo #
+--echo # MDEV-14862: Server crashes in Bitmap<64u>::merge / add_key_field
+--echo #
+
+CREATE TABLE t1 (a INT) ENGINE=InnoDB;
+CREATE VIEW v1 AS SELECT * FROM t1;
+CREATE TABLE t2 (b INT) ENGINE=InnoDB;
+DELETE FROM v1 WHERE a IN ( SELECT a FROM t2 );
+
+drop view v1;
+drop table t1,t2;
diff --git a/sql/item_subselect.cc b/sql/item_subselect.cc
index eb1e1a3d9b7..9f508ad3f83 100644
--- a/sql/item_subselect.cc
+++ b/sql/item_subselect.cc
@@ -1782,6 +1782,27 @@ Item_in_subselect::single_value_transformer(JOIN *join)
       runtime created Ref item which is deleted at the end
       of the statement. Thus one of 'substitution' arguments
       can be broken in case of PS.
+
+      @todo
+      Why do we use real_item()/substitutional_item() instead of the plain
+      left_expr?
+      Because left_expr might be a rollbackable item, and we fail to properly
+      rollback all copies of left_expr at end of execution, so we want to
+      avoid creating copies of left_expr as much as possible, so we use
+      real_item() instead.
+      Doing a proper rollback is difficult: the change was registered for the
+      original item which was the left argument of IN. Then this item was
+      copied to left_expr, which is copied below to substitution->args[0]. To
+      do a proper rollback, we would have to restore the content
+      of both copies as well as the original item. There might be more copies,
+      if AND items have been constructed.
+      The same applies to the right expression.
+      However, using real_item()/substitutional_item() brings its own
+      problems: for example, we lose information that the item is an outer
+      reference; the item can thus wrongly be considered for a Keyuse (causing
+      bug#17766653).
+      When WL#6570 removes the "rolling back" system, all
+      real_item()/substitutional_item() in this file should be removed.
     */ 
     substitution= func->create(left_expr, where_item);
     have_to_be_excluded= 1;
@@ -2070,6 +2091,9 @@ Item_in_subselect::create_single_in_to_exists_cond(JOIN *join,
   }
   else
   {
+    /*
+      Grep for "WL#6570" to see the relevant comment about real_item.
+    */
     Item *item= (Item*) select_lex->item_list.head()->real_item();
 
     if (select_lex->table_list.elements)
diff --git a/sql/sql_select.cc b/sql/sql_select.cc
index 3c2b62a6540..d0e5864ef6c 100644
--- a/sql/sql_select.cc
+++ b/sql/sql_select.cc
@@ -4362,7 +4362,19 @@ add_key_field(JOIN *join,
               Field *field, bool eq_func, Item **value, uint num_values,
               table_map usable_tables, SARGABLE_PARAM **sargables)
 {
-  uint optimize= 0;  
+  uint optimize= 0;
+
+  if (field->table->reginfo.join_tab == NULL)
+  {
+    /*
+       Due to a bug in IN-to-EXISTS (grep for real_item() in item_subselect.cc
+       for more info), an index over a field from an outer query might be
+       considered here, which is incorrect. Their query has been fully
+       optimized already so their reginfo.join_tab is NULL and we reject them.
+    */
+    return;
+  }
+
   if (eq_func &&
       ((join->is_allowed_hash_join_access() &&
         field->hash_join_is_possible() &&