A fix and a test case for Bug#6513 "Test Suite: Values inserted by using

cursor is interpreted latin1 character and Bug#9819 "Cursors: Mysql Server
Crash while fetching from table with 5 million records."
A fix for a possible memory leak when fetching into an SP cursor
in a long loop.
The patch uses a common implementation of cursors in the binary protocol and 
in stored procedures and implements materialized cursors.
For implementation details, see comments in sql_cursor.cc


include/my_sys.h:
  - declaration for multi_alloc_root
libmysqld/Makefile.am:
  - drop protocol_cursor.cc, add sql_cursor.cc (replaces the old
  implementation of cursors with a new one)
mysql-test/r/ctype_ujis.result:
  - test results fixed (a test case for Bug#6513)
mysql-test/r/sp-big.result:
  - test results fixed (a test case for Bug#9819)
mysql-test/t/ctype_ujis.test:
  Add a test case for Bug#6513 "Test Suite: Values inserted by using cursor is
   interpreted latin1 character"
mysql-test/t/sp-big.test:
  Add a restricted test case for Bug#9819 "Cursors: Mysql Server Crash
  while fetching from table with 5 million records."
mysys/my_alloc.c:
  - an implementation of multi_alloc_root; this is largely a copy-paste
    from mulalloc.c, but the function is small and there is no easy way
    to reuse the existing C function.
sql/Makefile.am:
  - add sql_cursor.h, sql_cursor.cc (a new implementation of stored procedure
  cursors) and drop protocol_cursor.cc (the old one)
sql/handler.cc:
  - now TABLE object has its mem_root always initialized.
    Adjust the implementation handler::ha_open
sql/item_subselect.cc:
  - adjust to the changed declaration of st_select_lex_unit::prepare
sql/protocol.h:
  - drop Protocol_cursor
sql/sp_head.cc:
  - move juggling with Query_arena::free_list and Item::next to
    sp_eval_func_item, as this is needed in 3 places already.
sql/sp_head.h:
  - declare a no-op implementation for cleanup_stmt in sp_instr_cpush.
    This method is needed for non-materializing cursors, which are yet not 
    used in stored procedures.
  - declaration for sp_eval_func_item
sql/sp_rcontext.cc:
  - reimplement sp_cursor using the new implementation of server side cursors.
  - use sp_eval_func_item to assign values of SP variables from the
    row fetched from a cursor. This should fix a possible memory leak in 
    the old implementation of sp_cursor::fetch
sql/sp_rcontext.h:
  - reimplement sp_cursor using the new implementation of server side cursors.
sql/sql_class.cc:
  - disable the functionality that closes transient cursors at commit/rollback;
    transient cursors are not used in 5.0, instead we use materialized ones.
    To be enabled in a later version.
sql/sql_class.h:
  - adjust to the rename Cursor -> Server_side_cursor
  - additional declarations of select_union used in materialized cursors
sql/sql_derived.cc:
  - reuse bits of tmp table code in UNION, derived tables, and materialized
    cursors
  - cleanup comments
sql/sql_lex.h:
  - declarations of auxiliary methods used by materialized cursors
  - a cleanup in st_select_lex_unit interface
sql/sql_list.h:
  - add an array operator new[] to class Sql_alloc
sql/sql_prepare.cc:
  - split the tight coupling of cursors and prepared statements to reuse 
    the same implementation in stored procedures
  - cleanups of error processing in Prepared_statement::{prepare,execute}
sql/sql_select.cc:
  - move the implementation of sensitive (non-materializing) cursors to 
    sql_cursor.cc
  - make temporary tables self-contained: the table, its record and fields
    are allocated in TABLE::mem_root. This implementation is not clean
    and resets thd->mem_root several times because of the way create_tmp_table 
    works (many additional things are done inside it).
  - adjust to the changed declaration of st_select_lex_unit::prepare
sql/sql_select.h:
  - move the declaration of sensitive (non-materializing) cursors to 
    sql_cursor.cc
sql/sql_union.cc:
  - move pieces of st_select_unit::prepare to select_union and st_table
    methods to be able to reuse code in the implementation of materialized
    cursors
sql/sql_view.cc:
  - adjust to the changed signature of st_select_lex_unit::prepare
sql/table.cc:
  - implement auxiliary st_table methods for use with temporary tables
sql/table.h:
  - add declarations for auxiliary methods of st_table used to work with 
   temporary tables
tests/mysql_client_test.c:
  - if cursors are materialized, a parallel update of the table used
    in the cursor may go through: update the test.
sql/sql_cursor.cc:
  New BitKeeper file ``sql/sql_cursor.cc'' -- implementation of server side
  cursors
sql/sql_cursor.h:
  New BitKeeper file ``sql/sql_cursor.h'' - declarations for
  server side cursors.

1 files changed, 76 insertions, 133 deletions

diff --git a/sql/sp_rcontext.cc b/sql/sp_rcontext.cc
index 748c09f56c7..fcb7719aeb1 100644
--- a/sql/sp_rcontext.cc
+++ b/sql/sp_rcontext.cc
@@ -25,6 +25,7 @@
 
 #include "mysql.h"
 #include "sp_head.h"
+#include "sql_cursor.h"
 #include "sp_rcontext.h"
 #include "sp_pcontext.h"
 
@@ -45,31 +46,18 @@ int
 sp_rcontext::set_item_eval(THD *thd, uint idx, Item **item_addr,
 			   enum_field_types type)
 {
-  extern Item *sp_eval_func_item(THD *thd, Item **it, enum_field_types type,
-				 Item *reuse, bool use_callers_arena);
   Item *it;
   Item *reuse_it;
-  Item *old_item_next;
   /* sp_eval_func_item will use callers_arena */
-  Item *old_free_list= thd->spcont->callers_arena->free_list;
   int res;
-  LINT_INIT(old_item_next);
 
-  if ((reuse_it= get_item(idx)))
-    old_item_next= reuse_it->next;
+  reuse_it= get_item(idx);
   it= sp_eval_func_item(thd, item_addr, type, reuse_it, TRUE);
   if (! it)
     res= -1;
   else
   {
     res= 0;
-    if (reuse_it && it == reuse_it)
-    {
-      // A reused item slot, where the constructor put it in the free_list,
-      // so we have to restore the list.
-      thd->spcont->callers_arena->free_list= old_free_list;
-      it->next= old_item_next;
-    }
     set_item(idx, it);
   }
 
@@ -170,7 +158,8 @@ sp_rcontext::pop_cursors(uint count)
  */
 
 sp_cursor::sp_cursor(sp_lex_keeper *lex_keeper, sp_instr_cpush *i)
-  :m_lex_keeper(lex_keeper), m_prot(NULL), m_isopen(0), m_current_row(NULL),
+  :m_lex_keeper(lex_keeper),
+   server_side_cursor(NULL),
    m_i(i)
 {
   /*
@@ -182,59 +171,37 @@ sp_cursor::sp_cursor(sp_lex_keeper *lex_keeper, sp_instr_cpush *i)
 
 
 /*
-  pre_open cursor
+  Open an SP cursor
 
   SYNOPSIS
-    pre_open()
-    THD		Thread handler
+    open()
+    THD		         Thread handler
 
-  NOTES
-    We have to open cursor in two steps to make it easy for sp_instr_copen
-    to reuse the sp_instr::exec_stmt() code.
-    If this function returns 0, post_open should not be called
 
   RETURN
-   0  ERROR
+   0 in case of success, -1 otherwise
 */
 
-sp_lex_keeper*
-sp_cursor::pre_open(THD *thd)
+int
+sp_cursor::open(THD *thd)
 {
-  if (m_isopen)
+  if (server_side_cursor)
   {
     my_message(ER_SP_CURSOR_ALREADY_OPEN, ER(ER_SP_CURSOR_ALREADY_OPEN),
                MYF(0));
-    return NULL;
+    return -1;
   }
-  init_alloc_root(&m_mem_root, MEM_ROOT_BLOCK_SIZE, MEM_ROOT_PREALLOC);
-  if ((m_prot= new Protocol_cursor(thd, &m_mem_root)) == NULL)
-    return NULL;
-
-  /* Save for execution. Will be restored in post_open */
-  m_oprot= thd->protocol;
-  m_nseof= thd->net.no_send_eof;
-
-  /* Change protocol for execution */
-  thd->protocol= m_prot;
-  thd->net.no_send_eof= TRUE;
-  return m_lex_keeper;
-}
-
-
-void
-sp_cursor::post_open(THD *thd, my_bool was_opened)
-{
-  thd->net.no_send_eof= m_nseof; // Restore the originals
-  thd->protocol= m_oprot;
-  if ((m_isopen= was_opened))
-    m_current_row= m_prot->data;
+  if (mysql_open_cursor(thd, (uint) ALWAYS_MATERIALIZED_CURSOR, &result,
+                        &server_side_cursor))
+    return -1;
+  return 0;
 }
 
 
 int
 sp_cursor::close(THD *thd)
 {
-  if (! m_isopen)
+  if (! server_side_cursor)
   {
     my_message(ER_SP_CURSOR_NOT_OPEN, ER(ER_SP_CURSOR_NOT_OPEN), MYF(0));
     return -1;
@@ -247,106 +214,82 @@ sp_cursor::close(THD *thd)
 void
 sp_cursor::destroy()
 {
-  if (m_prot)
-  {
-    delete m_prot;
-    m_prot= NULL;
-    free_root(&m_mem_root, MYF(0));
-  }
-  m_isopen= FALSE;
+  delete server_side_cursor;
+  server_side_cursor= 0;
 }
 
+
 int
 sp_cursor::fetch(THD *thd, List<struct sp_pvar> *vars)
 {
-  List_iterator_fast<struct sp_pvar> li(*vars);
-  sp_pvar_t *pv;
-  MYSQL_ROW row;
-  uint fldcount;
-
-  if (! m_isopen)
+  if (! server_side_cursor)
   {
     my_message(ER_SP_CURSOR_NOT_OPEN, ER(ER_SP_CURSOR_NOT_OPEN), MYF(0));
     return -1;
   }
-  if (m_current_row == NULL)
+  if (vars->elements != result.get_field_count())
   {
-    my_message(ER_SP_FETCH_NO_DATA, ER(ER_SP_FETCH_NO_DATA), MYF(0));
+    my_message(ER_SP_WRONG_NO_OF_FETCH_ARGS,
+               ER(ER_SP_WRONG_NO_OF_FETCH_ARGS), MYF(0));
     return -1;
   }
 
-  row= m_current_row->data;
-  for (fldcount= 0 ; (pv= li++) ; fldcount++)
-  {
-    Item *it;
-    Item *reuse;
-    uint rsize;
-    Item *old_item_next;
-    Item *old_free_list= thd->free_list;
-    const char *s;
-    LINT_INIT(old_item_next);
-
-    if (fldcount >= m_prot->get_field_count())
-    {
-      my_message(ER_SP_WRONG_NO_OF_FETCH_ARGS,
-                 ER(ER_SP_WRONG_NO_OF_FETCH_ARGS), MYF(0));
-      return -1;
-    }
+  result.set_spvar_list(vars);
 
-    if ((reuse= thd->spcont->get_item(pv->offset)))
-      old_item_next= reuse->next;
+  /* Attempt to fetch one row */
+  if (server_side_cursor->is_open())
+    server_side_cursor->fetch(1);
 
-    s= row[fldcount];
-    if (!s)
-      it= new(reuse, &rsize) Item_null();
-    else
-    {
-      /*
-        Length of data can be calculated as:
-        pointer_to_next_not_null_object - s -1
-        where the last -1 is to remove the end \0
-      */
-      uint len;
-      MYSQL_ROW next= row+fldcount+1;
-      while (!*next)                             // Skip nulls
-        next++;
-      len= (*next -s)-1;
-      switch (sp_map_result_type(pv->type)) {
-      case INT_RESULT:
-	it= new(reuse, &rsize) Item_int(s);
-	break;
-      case REAL_RESULT:
-        it= new(reuse, &rsize) Item_float(s, len);
-	break;
-      case DECIMAL_RESULT:
-        it= new(reuse, &rsize) Item_decimal(s, len, thd->db_charset);
-        break;
-      case STRING_RESULT:
-        /* TODO: Document why we do an extra copy of the string 's' here */
-        it= new(reuse, &rsize) Item_string(thd->strmake(s, len), len,
-					   thd->db_charset);
-        break;
-      case ROW_RESULT:
-      default:
-        DBUG_ASSERT(0);
-      }
-    }
-    it->rsize= rsize;
-    if (reuse && it == reuse)
-    {
-      // A reused item slot, where the constructor put it in the free_list,
-      // so we have to restore the list.
-      thd->free_list= old_free_list;
-      it->next= old_item_next;
-    }
-    thd->spcont->set_item(pv->offset, it);
-  }
-  if (fldcount < m_prot->get_field_count())
+  /*
+    If the cursor was pointing after the last row, the fetch will
+    close it instead of sending any rows.
+  */
+  if (! server_side_cursor->is_open())
   {
-    my_message(ER_SP_WRONG_NO_OF_FETCH_ARGS,
-               ER(ER_SP_WRONG_NO_OF_FETCH_ARGS), MYF(0));
+    my_message(ER_SP_FETCH_NO_DATA, ER(ER_SP_FETCH_NO_DATA), MYF(0));
     return -1;
   }
-  m_current_row= m_current_row->next;
+
   return 0;
 }
+
+
+/***************************************************************************
+ Select_fetch_into_spvars
+****************************************************************************/
+
+int Select_fetch_into_spvars::prepare(List<Item> &fields, SELECT_LEX_UNIT *u)
+{
+  /*
+    Cache the number of columns in the result set in order to easily
+    return an error if column count does not match value count.
+  */
+  field_count= fields.elements;
+  return select_result_interceptor::prepare(fields, u);
+}
+
+
+bool Select_fetch_into_spvars::send_data(List<Item> &items)
+{
+  List_iterator_fast<struct sp_pvar> pv_iter(*spvar_list);
+  List_iterator_fast<Item> item_iter(items);
+  sp_pvar_t *pv;
+  Item *item;
+
+  /* Must be ensured by the caller */
+  DBUG_ASSERT(spvar_list->elements == items.elements);
+
+  /*
+    Assign the row fetched from a server side cursor to stored
+    procedure variables.
+  */
+  for (; pv= pv_iter++, item= item_iter++; )
+  {
+    Item *reuse= thd->spcont->get_item(pv->offset);
+    /* Evaluate a new item on the arena of the calling instruction */
+    Item *it= sp_eval_func_item(thd, &item, pv->type, reuse, TRUE);
+
+    thd->spcont->set_item(pv->offset, it);
+  }
+  return FALSE;
+}