Fixes for Aria transaction handling with lock tables

MDEV-10130 Assertion `share->in_trans == 0' failed in storage/maria/ma_close.c
MDEV-10378 Assertion `trn' failed in virtual int ha_maria::start_stmt

The problem was that maria_handler->trn was not properly reset
at commit/rollback and ha_maria::exernal_lock() could get confused
because.

There was some old code in ha_maria::implicit_commit() that tried
to take care of this, but it was not bullet proof.

Fixed by adding list of all tables that is part of the maria transaction to
TRN.

A nice side effect was of the fix is that loops in
ha_maria::implict_commit() got to be much simpler.

Other things:
- Fixed a bug in mysql_admin_table() where argument open_for_modify
  was wrongly reset for the next table in the chain
- rollback admin command also in case of fatal error.
- Split _ma_set_trn_for_table() to three version to simplify code
  and debugging.
- Several new asserts to detect the original problem (that file was
  not properly removed from trn before calling ma_close())

1 files changed, 3 insertions, 1 deletions

diff --git a/sql/sql_handler.cc b/sql/sql_handler.cc
index 5fc7c20d409..52ce42e2e6a 100644
--- a/sql/sql_handler.cc
+++ b/sql/sql_handler.cc
@@ -155,10 +155,11 @@ static void mysql_ha_close_table(SQL_HANDLER *handler)
 {
   THD *thd= handler->thd;
   TABLE *table= handler->table;
+  DBUG_ENTER("mysql_ha_close_table");
 
   /* check if table was already closed */
   if (!table)
-    return;
+    DBUG_VOID_RETURN;
 
   if (!table->s->tmp_table)
   {
@@ -184,6 +185,7 @@ static void mysql_ha_close_table(SQL_HANDLER *handler)
   }
   my_free(handler->lock);
   handler->init();
+  DBUG_VOID_RETURN;
 }
 
 /*