MDEV-21866: Assertion `!result' failed in convert_const_to_int upon 2nd execution of PSbb-10.2-MDEV-21866

Consider the following use case:
MariaDB [test]> CREATE TABLE t1 (field1 BIGINT DEFAULT -1);
MariaDB [test]> CREATE VIEW v1 AS SELECT DISTINCT field1 FROM t1;

Repeated execution of the following query as a Prepared Statement

MariaDB [test]> PREPARE stmt FROM 'SELECT * FROM v1 WHERE field1 <=> NULL';
MariaDB [test]> EXECUTE stmt;

results in a crash for a server built with DEBUG.

MariaDB [test]> EXECUTE stmt;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Assertion failed: (!result), function convert_const_to_int, file item_cmpfunc.cc, line 476.
Abort trap: 6 (core dumped)

The crash inside the function convert_const_to_int() happens by the reason
that the value -1 is stored in an instance of the class Field_longlong
on restoring its original value in the statement
  result= field->store(orig_field_val, TRUE);
that leads to assigning the value 1 to the variable 'result' with subsequent
crash in the DBUG_ASSERT statement following it
  DBUG_ASSERT(!result);

The main matter here is why this assertion failure happens on the second
execution of the prepared statement and doens't on the first one.
On first handling of the statement
  'EXECUTE stmt;'
a temporary table is created for serving the query involving the view 'v1'.
The table is created by the function create_tmp_table() in the following
calls trace: (trace #1)
  JOIN::prepare (at sql_select.cc:725)
    st_select_lex::handle_derived
      LEX::handle_list_of_derived
        TABLE_LIST::handle_derived
          mysql_handle_single_derived
            mysql_derived_prepare
              select_union::create_result_table
                create_tmp_table

Note, that the data member TABLE::status of a TABLE instance returned by the
function create_tmp_table() has the value 0.

Later the function setup_table_map() is called on the TABLE instance just
created for the sake of the temporary table (calls trace #2 is below):
  JOIN::prepare (at sql_select.cc:737)
    setup_tables_and_check_access
      setup_tables
        setup_table_map
where the data member TABLE::status is set to the value STATUS_NO_RECORD.

After that when execution of the method JOIN::prepare reaches calling of
the function setup_without_group() the following calls trace is invoked
  JOIN::prepare
    setup_without_group
      setup_conds
        Item_func::fix_fields
          Item_func_equal::fix_length_and_dec
            Item_bool_rowready_func2::fix_length_and_dec
              Item_func::setup_args_and_comparator
                Item_func::convert_const_compared_to_int_field
                  convert_const_to_int

There is the following code snippet in the function convert_const_to_int()
at the line item_cmpfunc.cc:448
    bool save_field_value= (field_item->const_item() ||
                            !(field->table->status & STATUS_NO_RECORD));
Since field->table->status has bits STATUS_NO_RECORD set the variable
save_field_value is false and therefore neither the method
Field_longlong::val_int() nor the method Field_longlong::store is called
on the Field instance that has the numeric value -1.
That is the reason why first execution of the Prepared Statement for the query
  'SELECT * FROM v1 WHERE field1 <=> NULL'
is successful.

On second running of the statement 'EXECUTE stmt' a new temporary tables
is also created by running the calls trace #1 but the trace #2 is not executed
by the reason that data member SELECT_LEX::first_cond_optimization has been set
to false on first execution of the prepared statemet (in the method
JOIN::optimize_inner()). As a consequence, the data member TABLE::status for
a temporary table just created doesn't have the flags STATUS_NO_RECORD set and
therefore on re-execution of the prepared statement the methods
Field_longlong::val_int() and Field_longlong::store() are called for the field
having the value -1 and the DBUG_ASSERT(!result) is fired.

To fix the issue the data member TABLE::status has to be assigned the value
STATUS_NO_RECORD in every place where the macros empty_record() is called
to emptify a record for just instantiated TABLE object created on behalf
the new temporary table.

3 files changed, 31 insertions, 1 deletions

diff --git a/mysql-test/r/ps.result b/mysql-test/r/ps.result
index f1f779ef03f..1cd41e5b3a4 100644
--- a/mysql-test/r/ps.result
+++ b/mysql-test/r/ps.result
@@ -5474,5 +5474,18 @@ id	select_type	table	type	possible_keys	key	key_len	ref	rows	filtered	Extra
 2	SUBQUERY	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	Impossible WHERE noticed after reading const tables
 DEALLOCATE PREPARE stmt;
 DROP TABLE t1, t2, t3;
+#
+# MDEV-21866: Assertion `!result' failed in convert_const_to_int upon 2nd execution of PS
+#
+CREATE TABLE t1 (a BIGINT DEFAULT -1);
+CREATE VIEW v1 AS SELECT DISTINCT a FROM t1;
+PREPARE stmt FROM 'SELECT * FROM v1 WHERE a <=> NULL';
+EXECUTE stmt;
+a
+EXECUTE stmt;
+a
+DEALLOCATE PREPARE stmt;
+DROP VIEW v1;
+DROP TABLE t1;
 # End of 10.2 tests
 #
diff --git a/mysql-test/t/ps.test b/mysql-test/t/ps.test
index 2e7b43ad748..4097d28a949 100644
--- a/mysql-test/t/ps.test
+++ b/mysql-test/t/ps.test
@@ -4961,5 +4961,21 @@ EXECUTE stmt;
 DEALLOCATE PREPARE stmt;
 DROP TABLE t1, t2, t3;
 
+--echo #
+--echo # MDEV-21866: Assertion `!result' failed in convert_const_to_int upon 2nd execution of PS
+--echo #
+
+CREATE TABLE t1 (a BIGINT DEFAULT -1);
+CREATE VIEW v1 AS SELECT DISTINCT a FROM t1;
+PREPARE stmt FROM 'SELECT * FROM v1 WHERE a <=> NULL';
+EXECUTE stmt;
+EXECUTE stmt;
+
+# Cleanup
+DEALLOCATE PREPARE stmt;
+DROP VIEW v1;
+DROP TABLE t1;
+
+
 --echo # End of 10.2 tests
 --echo #
diff --git a/sql/sql_select.cc b/sql/sql_select.cc
index 3079145c1b9..a331f4f3dbc 100644
--- a/sql/sql_select.cc
+++ b/sql/sql_select.cc
@@ -17556,7 +17556,7 @@ create_tmp_table(THD *thd, TMP_TABLE_PARAM *param, List<Item> &fields,
 
   // Make empty record so random data is not written to disk
   empty_record(table);
-
+  table->status= STATUS_NO_RECORD;
   thd->mem_root= mem_root_save;
 
   DBUG_RETURN(table);
@@ -18557,6 +18557,7 @@ bool instantiate_tmp_table(TABLE *table, KEY *keyinfo,
       return TRUE;
     // Make empty record so random data is not written to disk
     empty_record(table);
+    table->status= STATUS_NO_RECORD;
   }
   if (open_tmp_table(table))
     return TRUE;