Bug #42037: Queries containing a subquery with DISTINCT and

            ORDER BY could cause a server crash

Dependent subqueries like

  SELECT COUNT(*) FROM t1, t2 WHERE t2.b
   IN (SELECT DISTINCT t2.b FROM t2 WHERE t2.b = t1.a)

caused a memory leak proportional to the
number of outer rows.


The make_simple_join() function has been modified to
JOIN class method to store join_tab_reexec and
table_reexec values in the parent join only
(make_simple_join of tmp_join may access these values
via 'this' pointer of the parent JOIN).

NOTE: this patch doesn't include standard test case (this is
"out of memory" bug). See bug #42037 page for test cases.


sql/sql_select.cc:
  Bug #42037: Queries containing a subquery with DISTINCT and
              ORDER BY could cause a server crash
  
  The make_simple_join() function has been modified to
  JOIN class method to store join_tab_reexec and
  table_reexec values in the parent join only.
sql/sql_select.h:
  Bug #42037: Queries containing a subquery with DISTINCT and
              ORDER BY could cause a server crash
  
  1. The make_simple_join() function has been modified to
     JOIN class method.
  
  2. Type of JOIN::table_reexec field has been changed from
     TABLE** to TABLE *table_reexec[1]: this field always was
     NULL or a pointer to one-element array of pointers, so
     a pointer to a pointer has been replaced with one pointer
     and unnecessary memory allocation has been eliminated.

1 files changed, 7 insertions, 2 deletions

diff --git a/sql/sql_select.h b/sql/sql_select.h
index 8ece01d3286..75a905043d2 100644
--- a/sql/sql_select.h
+++ b/sql/sql_select.h
@@ -352,9 +352,12 @@ public:
     cleared only at the end of the execution of the whole query and not caching
     allocations that occur in repetition at execution time will result in 
     excessive memory usage.
+    Note: make_simple_join always creates an execution plan that accesses
+    a single table, thus it is sufficient to have a one-element array for
+    table_reexec.
   */  
   SORT_FIELD *sortorder;                        // make_unireg_sortorder()
-  TABLE **table_reexec;                         // make_simple_join()
+  TABLE *table_reexec[1];                       // make_simple_join()
   JOIN_TAB *join_tab_reexec;                    // make_simple_join()
   /* end of allocation caching storage */
 
@@ -384,7 +387,7 @@ public:
     exec_tmp_table1= 0;
     exec_tmp_table2= 0;
     sortorder= 0;
-    table_reexec= 0;
+    table_reexec[0]= 0;
     join_tab_reexec= 0;
     thd= thd_arg;
     sum_funcs= sum_funcs2= 0;
@@ -476,6 +479,8 @@ public:
     return (unit == &thd->lex->unit && (unit->fake_select_lex == 0 ||
                                         select_lex == unit->fake_select_lex));
   }
+private:
+  bool make_simple_join(JOIN *join, TABLE *tmp_table);
 };