Bug#8407 (Stored functions/triggers ignore exception handler)

Bug 18914 (Calling certain SPs from triggers fail)
Bug 20713 (Functions will not not continue for SQLSTATE VALUE '42S02')
Bug 21825 (Incorrect message error deleting records in a table with a
  trigger for inserting)
Bug 22580 (DROP TABLE in nested stored procedure causes strange dependency
  error)
Bug 25345 (Cursors from Functions)


This fix resolves a long standing issue originally reported with bug 8407,
which affect the behavior of Stored Procedures, Stored Functions and Trigger
in many different ways, causing symptoms reported by all the bugs listed.
In all cases, the root cause of the problem traces back to 8407 and how the
server locks tables involved with sub statements.

Prior to this fix, the implementation of stored routines would:
- compute the transitive closure of all the tables referenced by a top level
statement
- open and lock all the tables involved
- execute the top level statement
"transitive closure of tables" means collecting:
- all the tables,
- all the stored functions,
- all the views,
- all the table triggers
- all the stored procedures
involved, and recursively inspect these objects definition to find more
references to more objects, until the list of every object referenced does
not grow any more.
This mechanism is known as "pre-locking" tables before execution.
The motivation for locking all the tables (possibly) used at once is to
prevent dead locks.

One problem with this approach is that, if the execution path the code
really takes during runtime does not use a given table, and if the table is
missing, the server would not execute the statement.
This in particular has a major impact on triggers, since a missing table
referenced by an update/delete trigger would prevent an insert trigger to run.

Another problem is that stored routines might define SQL exception handlers
to deal with missing tables, but the server implementation would never give
user code a chance to execute this logic, since the routine is never
executed when a missing table cause the pre-locking code to fail.

With this fix, the internal implementation of the pre-locking code has been
relaxed of some constraints, so that failure to open a table does not
necessarily prevent execution of a stored routine.

In particular, the pre-locking mechanism is now behaving as follows:

1) the first step, to compute the transitive closure of all the tables
possibly referenced by a statement, is unchanged.

2) the next step, which is to open all the tables involved, only attempts
to open the tables added by the pre-locking code, but silently fails without
reporting any error or invoking any exception handler is the table is not
present. This is achieved by trapping internal errors with
Prelock_error_handler

3) the locking step only locks tables that were successfully opened.

4) when executing sub statements, the list of tables used by each statements
is evaluated as before. The tables needed by the sub statement are expected
to be already opened and locked. Statement referencing tables that were not
opened in step 2) will fail to find the table in the open list, and only at
this point will execution of the user code fail.

5) when a runtime exception is raised at 4), the instruction continuation
destination (the next instruction to execute in case of SQL continue
handlers) is evaluated.
This is achieved with sp_instr::exec_open_and_lock_tables()

6) if a user exception handler is present in the stored routine, that
handler is invoked as usual, so that ER_NO_SUCH_TABLE exceptions can be
trapped by stored routines. If no handler exists, then the runtime execution
will fail as expected.

With all these changes, a side effect is that view security is impacted, in
two different ways.

First, a view defined as "select stored_function()", where the stored
function references a table that may not exist, is considered valid.
The rationale is that, because the stored function might trap exceptions
during execution and still return a valid result, there is no way to decide
when the view is created if a missing table really cause the view to be invalid.

Secondly, testing for existence of tables is now done later during
execution. View security, which consist of trapping errors and return a
generic ER_VIEW_INVALID (to prevent disclosing information) was only
implemented at very specific phases covering *opening* tables, but not
covering the runtime execution. Because of this existing limitation,
errors that were previously trapped and converted into ER_VIEW_INVALID are
not trapped, causing table names to be reported to the user.
This change is exposing an existing problem, which is independent and will
be resolved separately.


mysql-test/r/information_schema_db.result:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/r/sp-error.result:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/r/sp.result:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/r/trigger.result:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/r/view.result:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/t/sp-error.test:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/t/sp.test:
  Revised the pre-locking code implementation, aligned the tests.
mysql-test/t/trigger.test:
  Revised the pre-locking code implementation, aligned the tests.
sql/lock.cc:
  table->placeholder now checks for schema_table
sql/mysqld.cc:
  my_message_sql(): invoke internal exception handlers
sql/sp_head.cc:
  exec_open_and_lock_tables(): open and lock tables, or return the
  continuation destination of this instruction
sql/sp_head.h:
  exec_open_and_lock_tables(): open and lock tables, or return the
  continuation destination of this instruction
sql/sql_base.cc:
  Prelock_error_handler: delay open table errors until execution
sql/sql_class.cc:
  THD: add internal error handler, as an exception mechanism.
sql/sql_class.h:
  THD: add internal error handler, as an exception mechanism.
sql/sql_update.cc:
  table->placeholder now checks for schema_table
sql/table.cc:
  st_table_list::hide_view_error(): masked more errors for view security
sql/table.h:
  table->placeholder now checks for schema_table, and unopened tables

10 files changed, 271 insertions, 21 deletions

diff --git a/sql/lock.cc b/sql/lock.cc
index 2afe1de59f5..bf1512b754c 100644
--- a/sql/lock.cc
+++ b/sql/lock.cc
@@ -566,7 +566,7 @@ TABLE_LIST *mysql_lock_have_duplicate(THD *thd, TABLE_LIST *needle,
 
   for (; haystack; haystack= haystack->next_global)
   {
-    if (haystack->placeholder() || haystack->schema_table)
+    if (haystack->placeholder())
       continue;
     table2= haystack->table;
     if (table2->s->tmp_table == TMP_TABLE)
diff --git a/sql/mysqld.cc b/sql/mysqld.cc
index 4ee8c2e7e31..b6054cd30f4 100644
--- a/sql/mysqld.cc
+++ b/sql/mysqld.cc
@@ -2411,6 +2411,14 @@ static int my_message_sql(uint error, const char *str, myf MyFlags)
   */
   if ((thd= current_thd))
   {
+    /*
+      TODO: There are two exceptions mechanism (THD and sp_rcontext),
+      this could be improved by having a common stack of handlers.
+    */
+    if (thd->handle_error(error,
+                          MYSQL_ERROR::WARN_LEVEL_ERROR))
+      DBUG_RETURN(0);
+
     if (thd->spcont &&
         thd->spcont->handle_error(error, MYSQL_ERROR::WARN_LEVEL_ERROR, thd))
     {
diff --git a/sql/sp_head.cc b/sql/sp_head.cc
index de0edabda3e..32f7d0b7f8c 100644
--- a/sql/sp_head.cc
+++ b/sql/sp_head.cc
@@ -2374,16 +2374,11 @@ sp_lex_keeper::reset_lex_and_exec_core(THD *thd, uint *nextp,
       m_lex->mark_as_requiring_prelocking(lex_query_tables_own_last);
     }
   }
-    
+
   reinit_stmt_before_use(thd, m_lex);
-  /*
-    If requested check whenever we have access to tables in LEX's table list
-    and open and lock them before executing instructtions core function.
-  */
-  if (open_tables &&
-      (check_table_access(thd, SELECT_ACL, m_lex->query_tables, 0) ||
-       open_and_lock_tables(thd, m_lex->query_tables)))
-      res= -1;
+
+  if (open_tables)
+    res= instr->exec_open_and_lock_tables(thd, m_lex->query_tables, nextp);
 
   if (!res)
     res= instr->exec_core(thd, nextp);
@@ -2432,6 +2427,33 @@ sp_lex_keeper::reset_lex_and_exec_core(THD *thd, uint *nextp,
   sp_instr class functions
 */
 
+int sp_instr::exec_open_and_lock_tables(THD *thd, TABLE_LIST *tables,
+                                        uint *nextp)
+{
+  int result;
+
+  /*
+    Check whenever we have access to tables for this statement
+    and open and lock them before executing instructions core function.
+  */
+  if (check_table_access(thd, SELECT_ACL, tables, 0)
+      || open_and_lock_tables(thd, tables))
+  {
+    get_cont_dest(nextp);
+    result= -1;
+  }
+  else
+    result= 0;
+
+  return result;
+}
+
+void sp_instr::get_cont_dest(uint *nextp)
+{
+  *nextp= m_ip+1;
+}
+
+
 int sp_instr::exec_core(THD *thd, uint *nextp)
 {
   DBUG_ASSERT(0);
@@ -2612,6 +2634,15 @@ sp_instr_set_trigger_field::print(String *str)
   value->print(str);
 }
 
+/*
+  sp_instr_opt_meta
+*/
+
+void sp_instr_opt_meta::get_cont_dest(uint *nextp)
+{
+  *nextp= m_cont_dest;
+}
+
 
 /*
  sp_instr_jump class functions
diff --git a/sql/sp_head.h b/sql/sp_head.h
index 0139f879ce4..837d737f228 100644
--- a/sql/sp_head.h
+++ b/sql/sp_head.h
@@ -458,6 +458,28 @@ public:
   
   virtual int execute(THD *thd, uint *nextp) = 0;
 
+  /**
+    Execute <code>open_and_lock_tables()</code> for this statement.
+    Open and lock the tables used by this statement, as a pre-requisite
+    to execute the core logic of this instruction with
+    <code>exec_core()</code>.
+    If this statement fails, the next instruction to execute is also returned.
+    This is useful when a user defined SQL continue handler needs to be
+    executed.
+    @param thd the current thread
+    @param tables the list of tables to open and lock
+    @param nextp the continuation instruction, returned to the caller if this
+    method fails.
+    @return zero on success, non zero on failure.
+  */
+  int exec_open_and_lock_tables(THD *thd, TABLE_LIST *tables, uint *nextp);
+
+  /**
+    Get the continuation destination of this instruction.
+    @param nextp the continuation destination (output)
+  */
+  virtual void get_cont_dest(uint *nextp);
+
   /*
     Execute core function of instruction after all preparations (e.g.
     setting of proper LEX, saving part of the thread context have been
@@ -722,6 +744,8 @@ public:
   virtual void set_destination(uint old_dest, uint new_dest)
     = 0;
 
+  virtual void get_cont_dest(uint *nextp);
+
 protected:
 
   sp_instr *m_optdest;		// Used during optimization
diff --git a/sql/sql_base.cc b/sql/sql_base.cc
index a4318f7b4bf..52f08331069 100644
--- a/sql/sql_base.cc
+++ b/sql/sql_base.cc
@@ -28,6 +28,59 @@
 #include <io.h>
 #endif
 
+/**
+  This internal handler is used to trap internally
+  errors that can occur when executing open table
+  during the prelocking phase.
+*/
+class Prelock_error_handler : public Internal_error_handler
+{
+public:
+  Prelock_error_handler()
+    : m_handled_errors(0), m_unhandled_errors(0)
+  {}
+
+  virtual ~Prelock_error_handler() {}
+
+  virtual bool handle_error(uint sql_errno,
+                            MYSQL_ERROR::enum_warning_level level,
+                            THD *thd);
+
+  bool safely_trapped_errors();
+
+private:
+  int m_handled_errors;
+  int m_unhandled_errors;
+};
+
+
+bool
+Prelock_error_handler::handle_error(uint sql_errno,
+                                    MYSQL_ERROR::enum_warning_level /* level */,
+                                    THD * /* thd */)
+{
+  if (sql_errno == ER_NO_SUCH_TABLE)
+  {
+    m_handled_errors++;
+    return TRUE;                                // 'TRUE', as per coding style
+  }
+
+  m_unhandled_errors++;
+  return FALSE;                                 // 'FALSE', as per coding style
+}
+
+
+bool Prelock_error_handler::safely_trapped_errors()
+{
+  /*
+    If m_unhandled_errors != 0, something else, unanticipated, happened,
+    so the error is not trapped but returned to the caller.
+    Multiple ER_NO_SUCH_TABLE can be raised in case of views.
+  */
+  return ((m_handled_errors > 0) && (m_unhandled_errors == 0));
+}
+
+
 TABLE *unused_tables;				/* Used by mysql_test */
 HASH open_cache;				/* Used by mysql_test */
 
@@ -1334,7 +1387,10 @@ TABLE *open_table(THD *thd, TABLE_LIST *table_list, MEM_ROOT *mem_root,
         VOID(pthread_mutex_unlock(&LOCK_open));
       }
     }
-    my_error(ER_TABLE_NOT_LOCKED, MYF(0), alias);
+    if ((thd->locked_tables) && (thd->locked_tables->lock_count > 0))
+      my_error(ER_TABLE_NOT_LOCKED, MYF(0), alias);
+    else
+      my_error(ER_NO_SUCH_TABLE, MYF(0), table_list->db, table_list->alias);
     DBUG_RETURN(0);
   }
 
@@ -2092,6 +2148,8 @@ int open_tables(THD *thd, TABLE_LIST **start, uint *counter, uint flags)
   MEM_ROOT new_frm_mem;
   /* Also used for indicating that prelocking is need */
   TABLE_LIST **query_tables_last_own;
+  bool safe_to_ignore_table;
+
   DBUG_ENTER("open_tables");
   /*
     temporary mem_root for new .frm parsing.
@@ -2147,6 +2205,7 @@ int open_tables(THD *thd, TABLE_LIST **start, uint *counter, uint flags)
 
   for (tables= *start; tables ;tables= tables->next_global)
   {
+    safe_to_ignore_table= FALSE;                // 'FALSE', as per coding style
     /*
       Ignore placeholders for derived tables. After derived tables
       processing, link to created temporary table will be put here.
@@ -2166,9 +2225,28 @@ int open_tables(THD *thd, TABLE_LIST **start, uint *counter, uint flags)
       DBUG_RETURN(-1);
     }
     (*counter)++;
-    
-    if (!tables->table &&
-	!(tables->table= open_table(thd, tables, &new_frm_mem, &refresh, flags)))
+
+    if (!tables->table)
+    {
+      if (tables->prelocking_placeholder)
+      {
+        /*
+          For the tables added by the pre-locking code, attempt to open
+          the table but fail silently if the table does not exist.
+          The real failure will occur when/if a statement attempts to use
+          that table.
+        */
+        Prelock_error_handler prelock_handler;
+        thd->push_internal_handler(& prelock_handler);
+        tables->table= open_table(thd, tables, &new_frm_mem, &refresh, flags);
+        thd->pop_internal_handler();
+        safe_to_ignore_table= prelock_handler.safely_trapped_errors();
+      }
+      else
+        tables->table= open_table(thd, tables, &new_frm_mem, &refresh, flags);
+    }
+
+    if (!tables->table)
     {
       free_root(&new_frm_mem, MYF(MY_KEEP_PREALLOC));
 
@@ -2219,6 +2297,14 @@ int open_tables(THD *thd, TABLE_LIST **start, uint *counter, uint flags)
         close_tables_for_reopen(thd, start);
 	goto restart;
       }
+
+      if (safe_to_ignore_table)
+      {
+        DBUG_PRINT("info", ("open_table: ignoring table '%s'.'%s'",
+                            tables->db, tables->alias));
+        continue;
+      }
+
       result= -1;				// Fatal error
       break;
     }
@@ -2522,7 +2608,7 @@ bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags)
 static void mark_real_tables_as_free_for_reuse(TABLE_LIST *table)
 {
   for (; table; table= table->next_global)
-    if (!table->placeholder() && !table->schema_table)
+    if (!table->placeholder())
       table->table->query_id= 0;
 }
 
@@ -2594,7 +2680,7 @@ int lock_tables(THD *thd, TABLE_LIST *tables, uint count, bool *need_reopen)
       DBUG_RETURN(-1);
     for (table= tables; table; table= table->next_global)
     {
-      if (!table->placeholder() && !table->schema_table)
+      if (!table->placeholder())
 	*(ptr++)= table->table;
     }
 
@@ -2636,7 +2722,7 @@ int lock_tables(THD *thd, TABLE_LIST *tables, uint count, bool *need_reopen)
 
       for (table= tables; table != first_not_own; table= table->next_global)
       {
-        if (!table->placeholder() && !table->schema_table)
+        if (!table->placeholder())
         {
           table->table->query_id= thd->query_id;
           if (check_lock_and_start_stmt(thd, table->table, table->lock_type))
@@ -2663,7 +2749,7 @@ int lock_tables(THD *thd, TABLE_LIST *tables, uint count, bool *need_reopen)
     TABLE_LIST *first_not_own= thd->lex->first_not_own_table();
     for (table= tables; table != first_not_own; table= table->next_global)
     {
-      if (!table->placeholder() && !table->schema_table &&
+      if (!table->placeholder() &&
 	  check_lock_and_start_stmt(thd, table->table, table->lock_type))
       {
 	ha_rollback_stmt(thd);
diff --git a/sql/sql_class.cc b/sql/sql_class.cc
index 3b612dadcd0..a662ae290fd 100644
--- a/sql/sql_class.cc
+++ b/sql/sql_class.cc
@@ -273,6 +273,38 @@ THD::THD()
   substitute_null_with_insert_id = FALSE;
   thr_lock_info_init(&lock_info); /* safety: will be reset after start */
   thr_lock_owner_init(&main_lock_id, &lock_info);
+
+  m_internal_handler= NULL;
+}
+
+
+void THD::push_internal_handler(Internal_error_handler *handler)
+{
+  /*
+    TODO: The current implementation is limited to 1 handler at a time only.
+    THD and sp_rcontext need to be modified to use a common handler stack.
+  */
+  DBUG_ASSERT(m_internal_handler == NULL);
+  m_internal_handler= handler;
+}
+
+
+bool THD::handle_error(uint sql_errno,
+                       MYSQL_ERROR::enum_warning_level level)
+{
+  if (m_internal_handler)
+  {
+    return m_internal_handler->handle_error(sql_errno, level, this);
+  }
+
+  return FALSE;                                 // 'FALSE', as per coding style
+}
+
+
+void THD::pop_internal_handler()
+{
+  DBUG_ASSERT(m_internal_handler != NULL);
+  m_internal_handler= NULL;
 }
 
 
diff --git a/sql/sql_class.h b/sql/sql_class.h
index 05034ebd573..d03845ae663 100644
--- a/sql/sql_class.h
+++ b/sql/sql_class.h
@@ -1071,6 +1071,48 @@ public:
   SAVEPOINT *savepoints;
 };
 
+/**
+  This class represents the interface for internal error handlers.
+  Internal error handlers are exception handlers used by the server
+  implementation.
+*/
+class Internal_error_handler
+{
+protected:
+  Internal_error_handler() {}
+  virtual ~Internal_error_handler() {}
+
+public:
+  /**
+    Handle an error condition.
+    This method can be implemented by a subclass to achieve any of the
+    following:
+    - mask an error internally, prevent exposing it to the user,
+    - mask an error and throw another one instead.
+    When this method returns true, the error condition is considered
+    'handled', and will not be propagated to upper layers.
+    It is the responsability of the code installing an internal handler
+    to then check for trapped conditions, and implement logic to recover
+    from the anticipated conditions trapped during runtime.
+
+    This mechanism is similar to C++ try/throw/catch:
+    - 'try' correspond to <code>THD::push_internal_handler()</code>,
+    - 'throw' correspond to <code>my_error()</code>,
+    which invokes <code>my_message_sql()</code>,
+    - 'catch' correspond to checking how/if an internal handler was invoked,
+    before removing it from the exception stack with
+    <code>THD::pop_internal_handler()</code>.
+
+    @param sql_errno the error number
+    @param level the error level
+    @param thd the calling thread
+    @return true if the error is handled
+  */
+  virtual bool handle_error(uint sql_errno,
+                            MYSQL_ERROR::enum_warning_level level,
+                            THD *thd) = 0;
+};
+
 
 /*
   For each client connection we create a separate thread with THD serving as
@@ -1659,6 +1701,31 @@ public:
       *p_db_length= db_length;
     return FALSE;
   }
+
+public:
+  /**
+    Add an internal error handler to the thread execution context.
+    @param handler the exception handler to add
+  */
+  void push_internal_handler(Internal_error_handler *handler);
+
+  /**
+    Handle an error condition.
+    @param sql_errno the error number
+    @param level the error level
+    @return true if the error is handled
+  */
+  virtual bool handle_error(uint sql_errno,
+                            MYSQL_ERROR::enum_warning_level level);
+
+  /**
+    Remove the error handler last pushed.
+  */
+  void pop_internal_handler();
+
+private:
+  /** The current internal error handler for this thread, or NULL. */
+  Internal_error_handler *m_internal_handler;
 };
 
 
diff --git a/sql/sql_update.cc b/sql/sql_update.cc
index 4043fe17a46..3b295b63c2b 100644
--- a/sql/sql_update.cc
+++ b/sql/sql_update.cc
@@ -783,7 +783,7 @@ reopen_tables:
       tl->lock_type= using_update_log ? TL_READ_NO_INSERT : TL_READ;
       tl->updating= 0;
       /* Update TABLE::lock_type accordingly. */
-      if (!tl->placeholder() && !tl->schema_table && !using_lock_tables)
+      if (!tl->placeholder() && !using_lock_tables)
         tl->table->reginfo.lock_type= tl->lock_type;
     }
   }
diff --git a/sql/table.cc b/sql/table.cc
index 5c72ac6ccbf..623560fbe83 100644
--- a/sql/table.cc
+++ b/sql/table.cc
@@ -2091,7 +2091,9 @@ void st_table_list::hide_view_error(THD *thd)
       thd->net.last_errno == ER_SP_DOES_NOT_EXIST ||
       thd->net.last_errno == ER_PROCACCESS_DENIED_ERROR ||
       thd->net.last_errno == ER_COLUMNACCESS_DENIED_ERROR ||
-      thd->net.last_errno == ER_TABLEACCESS_DENIED_ERROR)
+      thd->net.last_errno == ER_TABLEACCESS_DENIED_ERROR ||
+      thd->net.last_errno == ER_TABLE_NOT_LOCKED ||
+      thd->net.last_errno == ER_NO_SUCH_TABLE)
   {
     TABLE_LIST *top= top_table();
     thd->clear_error();
diff --git a/sql/table.h b/sql/table.h
index 70e64439af5..d1b0510cea3 100644
--- a/sql/table.h
+++ b/sql/table.h
@@ -643,7 +643,7 @@ typedef struct st_table_list
   int view_check_option(THD *thd, bool ignore_failure);
   bool setup_underlying(THD *thd);
   void cleanup_items();
-  bool placeholder() {return derived || view; }
+  bool placeholder() {return derived || view || schema_table || !table; }
   void print(THD *thd, String *str);
   bool check_single_table(st_table_list **table, table_map map,
                           st_table_list *view);