<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/mariadb-git.git/sql/temporary_tables.cc, branch bb-mdev-274</title>
<subtitle>github.com: MariaDB/server.git
</subtitle>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/'/>
<entry>
<title>Merge 10.3 into 10.4</title>
<updated>2019-07-25T15:42:06+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-07-25T15:42:06+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=e9c1701e11e2441435223cc7c00c467f58aaff19'/>
<id>e9c1701e11e2441435223cc7c00c467f58aaff19</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.2 into 10.3</title>
<updated>2019-07-25T12:31:11+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-07-25T12:31:11+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=fdef9f9b8960479919285d6aec0bb30d27fa4723'/>
<id>fdef9f9b8960479919285d6aec0bb30d27fa4723</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.1 into 10.2</title>
<updated>2019-07-25T09:14:27+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-07-25T09:08:50+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=b6ac67389dfed3255004b82d29625577c3335a0a'/>
<id>b6ac67389dfed3255004b82d29625577c3335a0a</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.3 into 10.4</title>
<updated>2019-06-19T07:49:00+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-06-19T07:49:00+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=02979daab48d871e8c43a227b8f4462128439f17'/>
<id>02979daab48d871e8c43a227b8f4462128439f17</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.2 into 10.3</title>
<updated>2019-06-19T05:56:10+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-06-19T05:56:10+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=192aa295b4bc877d0fed24154c57304c3f789179'/>
<id>192aa295b4bc877d0fed24154c57304c3f789179</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>MDEV-19595 fixed</title>
<updated>2019-06-18T21:35:44+00:00</updated>
<author>
<name>Michael Widenius</name>
<email>monty@mariadb.org</email>
</author>
<published>2019-06-18T21:35:44+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=8acbf9c1f961aa1008ef509e059e1a09943f5ed3'/>
<id>8acbf9c1f961aa1008ef509e059e1a09943f5ed3</id>
<content type='text'>
The test cases for the MDEV found several independent bugs
in MariaDB server and Aria:
- If a temporary table was marked as crashed, it could never
  be deleted.
- Opening of a crashed temporary table gave an error message
  but the error was never forwarded to the caller which caused
  an assert() in my_ok()
- init_read_record() did mmap of all temporary tables, which is
  probably not a good idea as this area can potentially be
  very big. Changed code to only mmap internal temporary tables.
- mmap-ed tables where not unmapped in case of repair/optimize
  which caused bad data in table and crashes if the original
  table files where replaced with new ones (as the old mmap
  was still in place). Fixed by removing the mmap in case
  of repair.
- Cleaned up usage of code that disabled mmap in Aria
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The test cases for the MDEV found several independent bugs
in MariaDB server and Aria:
- If a temporary table was marked as crashed, it could never
  be deleted.
- Opening of a crashed temporary table gave an error message
  but the error was never forwarded to the caller which caused
  an assert() in my_ok()
- init_read_record() did mmap of all temporary tables, which is
  probably not a good idea as this area can potentially be
  very big. Changed code to only mmap internal temporary tables.
- mmap-ed tables where not unmapped in case of repair/optimize
  which caused bad data in table and crashes if the original
  table files where replaced with new ones (as the old mmap
  was still in place). Fixed by removing the mmap in case
  of repair.
- Cleaned up usage of code that disabled mmap in Aria
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.3 into 10.4</title>
<updated>2019-05-29T19:17:00+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-05-29T19:17:00+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=f98bb23168ee9bc0da8aa7111f35cf2539986387'/>
<id>f98bb23168ee9bc0da8aa7111f35cf2539986387</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge 10.2 into 10.3</title>
<updated>2019-05-29T08:32:46+00:00</updated>
<author>
<name>Marko Mäkelä</name>
<email>marko.makela@mariadb.com</email>
</author>
<published>2019-05-29T08:32:46+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=90a91936852368774559a3ef2660e63e1e6f50e3'/>
<id>90a91936852368774559a3ef2660e63e1e6f50e3</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>MDEV-19076: rpl_parallel_temptable result mismatch '-33 optimistic'</title>
<updated>2019-05-20T10:16:26+00:00</updated>
<author>
<name>Sujatha</name>
<email>sujatha.sivakumar@mariadb.com</email>
</author>
<published>2019-05-16T11:06:20+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=5a2110e7cf98a14da8fe12797521b425a69a9373'/>
<id>5a2110e7cf98a14da8fe12797521b425a69a9373</id>
<content type='text'>
Problem:
========
The test now fails with the following trace:

CURRENT_TEST: rpl.rpl_parallel_temptable
--- /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.result
+++ /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.reject
@@ -194,7 +194,6 @@
 30    conservative
 31    conservative
 32    optimistic
-33    optimistic

Analysis:
=========
The part of test which fails with result content mismatch is given below.

CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t4 VALUES (32);
INSERT INTO t4 VALUES (33);
INSERT INTO t1 SELECT a, "optimistic" FROM t4;

slave_parallel_mode=optimistic

The expectation of the above test script is, INSERT FROM SELECT should read both
32, 33 and populate table 't1'. But this expectation fails occasionally.

All three INSERT statements are handed over to three different slave parallel
workers. Temporary tables are not safe for parallel replication. They were
designed to be visible to one thread only, so have no table locking.  Thus there
is no protection against two conflicting transactions committing in parallel and
things like that.

So anything that uses temporary tables will be serialized with anything before
it, when using parallel replication by using a "wait_for_prior_commit" function
call. This will ensure that the each transaction is executed sequentially.

But there exists a code path in which the above wait doesn't happen.  Because of
this at times INSERT from SELECT doesn't wait for the INSERT (33) to complete
and it completes its executes and enters commit stage.  Hence only row 32 is
found in those cases resulting in test failure.

The wait needs to be added within "open_temporary_table" call. The code looks
like this within "open_temporary_table".

Each thread tries to open temporary table in 3 different ways:

case 1: Find a temporary table which is already in use by using
         find_temporary_table(tl) &amp;&amp; wait_for_prior_commit()
case 2: If above failed then try to look for temporary table which is marked for
        free for reuse. This internally calls "wait_for_prior_commit()" if table
        is found.
         find_and_use_tmp_table(tl, &amp;table)
case 3: If none of the above open a new table handle from table share.
         if (!table &amp;&amp; (share= find_tmp_table_share(tl)))
         { table= open_temporary_table(share, tl-&gt;get_table_name(), true); }

At present the "wait_for_prior_commit" happens only in case 1 &amp; 2.

Fix:
====
On slave add a call for "wait_for_prior_commit" for case 3.

The above wait on slave will solve the issue. A more detailed fix would be to
mark temporary tables as not safe for parallel execution on the master side.
In order to do that, on the master side, mark the Gtid_log_event specific flag
FL_TRANSACTIONAL to be false all the time. So that they are not scheduled
parallely.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Problem:
========
The test now fails with the following trace:

CURRENT_TEST: rpl.rpl_parallel_temptable
--- /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.result
+++ /mariadb/10.4/mysql-test/suite/rpl/r/rpl_parallel_temptable.reject
@@ -194,7 +194,6 @@
 30    conservative
 31    conservative
 32    optimistic
-33    optimistic

Analysis:
=========
The part of test which fails with result content mismatch is given below.

CREATE TEMPORARY TABLE t4 (a INT PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t4 VALUES (32);
INSERT INTO t4 VALUES (33);
INSERT INTO t1 SELECT a, "optimistic" FROM t4;

slave_parallel_mode=optimistic

The expectation of the above test script is, INSERT FROM SELECT should read both
32, 33 and populate table 't1'. But this expectation fails occasionally.

All three INSERT statements are handed over to three different slave parallel
workers. Temporary tables are not safe for parallel replication. They were
designed to be visible to one thread only, so have no table locking.  Thus there
is no protection against two conflicting transactions committing in parallel and
things like that.

So anything that uses temporary tables will be serialized with anything before
it, when using parallel replication by using a "wait_for_prior_commit" function
call. This will ensure that the each transaction is executed sequentially.

But there exists a code path in which the above wait doesn't happen.  Because of
this at times INSERT from SELECT doesn't wait for the INSERT (33) to complete
and it completes its executes and enters commit stage.  Hence only row 32 is
found in those cases resulting in test failure.

The wait needs to be added within "open_temporary_table" call. The code looks
like this within "open_temporary_table".

Each thread tries to open temporary table in 3 different ways:

case 1: Find a temporary table which is already in use by using
         find_temporary_table(tl) &amp;&amp; wait_for_prior_commit()
case 2: If above failed then try to look for temporary table which is marked for
        free for reuse. This internally calls "wait_for_prior_commit()" if table
        is found.
         find_and_use_tmp_table(tl, &amp;table)
case 3: If none of the above open a new table handle from table share.
         if (!table &amp;&amp; (share= find_tmp_table_share(tl)))
         { table= open_temporary_table(share, tl-&gt;get_table_name(), true); }

At present the "wait_for_prior_commit" happens only in case 1 &amp; 2.

Fix:
====
On slave add a call for "wait_for_prior_commit" for case 3.

The above wait on slave will solve the issue. A more detailed fix would be to
mark temporary tables as not safe for parallel execution on the master side.
In order to do that, on the master side, mark the Gtid_log_event specific flag
FL_TRANSACTIONAL to be false all the time. So that they are not scheduled
parallely.
</pre>
</div>
</content>
</entry>
<entry>
<title>MDEV-17167 - InnoDB: Failing assertion: table-&gt;get_ref_count() == 0 upon</title>
<updated>2019-05-14T12:23:09+00:00</updated>
<author>
<name>Sergey Vojtovich</name>
<email>svoj@mariadb.org</email>
</author>
<published>2018-09-12T12:36:45+00:00</published>
<link rel='alternate' type='text/html' href='http://trove.baserock.org/cgit/delta/mariadb-git.git/commit/?id=95fb88d5469e9d601aa9c2f319d1b561925e9795'/>
<id>95fb88d5469e9d601aa9c2f319d1b561925e9795</id>
<content type='text'>
             truncating a temporary table

TRUNCATE expects only one TABLE instance (which is used by TRUNCATE
itself) to be open. However this requirement wasn't enforced after
"MDEV-5535: Cannot reopen temporary table".

Fixed by closing unused table instances before performing TRUNCATE.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
             truncating a temporary table

TRUNCATE expects only one TABLE instance (which is used by TRUNCATE
itself) to be open. However this requirement wasn't enforced after
"MDEV-5535: Cannot reopen temporary table".

Fixed by closing unused table instances before performing TRUNCATE.
</pre>
</div>
</content>
</entry>
</feed>
