| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\ |
|
| | |\ |
|
| | | |
| | |
| | |
| | | |
* Update wrong zip-code
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
truncating a temporary table
TRUNCATE expects only one TABLE instance (which is used by TRUNCATE
itself) to be open. However this requirement wasn't enforced after
"MDEV-5535: Cannot reopen temporary table".
Fixed by closing unused table instances before performing TRUNCATE.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Remove unused InnoDB function parameters and functions.
i_s_sys_virtual_fill_table(): Do not allocate heap memory.
mtr_is_block_fix(): Replace with mtr_memo_contains().
mtr_is_page_fix(): Replace with mtr_memo_contains_page().
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- Added sql/mariadb.h file that should be included first by files in sql
directory, if sql_plugin.h is not used (sql_plugin.h adds SHOW variables
that must be done before my_global.h is included)
- Removed a lot of include my_global.h from include files
- Removed include's of some files that my_global.h automatically includes
- Removed duplicated include's of my_sys.h
- Replaced include my_config.h with my_global.h
|
| |\ \
| |/ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- thread_pool_size command line option upper limit increased to 100 000
(same as for max_connections)
- thread_pool_size system variable upper limit is maximum of 128 or
the value given at command line
- thread groups are now allocated dynamically
Different limit for command line option and system variable was done to
avoid additional mutex for all_groups and threadpool_max_size.
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
BY CACHING OR REDUCING CREATEEVENT CALLS".
5.5 versions of MySQL server performed worse than 5.1 versions
under single-connection workload in autocommit mode on Windows XP.
Part of this slowdown can be attributed to overhead associated
with constant creation/destruction of MDL_lock objects in the MDL
subsystem. The problem is that creation/destruction of these
objects causes creation and destruction of associated
synchronization primitives, which are expensive on Windows XP.
This patch tries to alleviate this problem by introducing a cache
of unused MDL_object_lock objects. Instead of destroying such
objects we put them into the cache and then reuse with a new
key when creation of a new object is requested.
To limit the size of this cache, a new --metadata-locks-cache-size
start-up parameter was introduced.
mysql-test/r/mysqld--help-notwin.result:
Updated test after adding --metadata-locks-cache-size
parameter.
mysql-test/r/mysqld--help-win.result:
Updated test after adding --metadata-locks-cache-size
parameter.
mysql-test/suite/sys_vars/r/metadata_locks_cache_size_basic.result:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic-master.opt:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic.test:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
sql/mdl.cc:
Introduced caching of unused MDL_object_lock objects, in order to
avoid costs associated with constant creation and destruction of
such objects in single-connection workloads run in autocommit mode.
Such costs can be pretty high on systems where creation and
destruction of synchronization primitives require a system call
(e.g. Windows XP).
To implement this cache,a list of unused MDL_object_lock instances
was added to MDL_map object. Instead of being destroyed
MDL_object_lock instances are put into this list and re-used later
when creation of a new instance is required. Also added
MDL_lock::m_version counter to allow threads having outstanding
references to an MDL_object_lock instance to notice that it has
been moved to the unused objects list.
Added a global variable for a start-up parameter that limits
the size of the unused objects list.
Note that we don't cache MDL_scoped_lock objects since they
are supposed to be created only during execution of DDL
statements and therefore should not affect performance much.
sql/mdl.h:
Added a global variable for start-up parameter that limits the
size of the unused MDL_object_lock objects list and constant
for its default value.
sql/sql_plist.h:
Added I_P_List<>::pop_front() function.
sql/sys_vars.cc:
Introduced --metadata-locks-cache-size start-up parameter
for specifying size of the cache of unused MDL_object_lock
objects.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
the list. Previously, the counter would only be incremented if the
insertion method push_front() was used, in which case the counter
wouldn't be incremented if a element was inserted using the push_back()
and/or insert_after() methods.
Currently this does not affect the code base because there isn't any
code that uses a counted list with the push_back() or insert_after()
methods.
|
| |
|
|
|
| |
for code other then in mdl.h. Since the adapter is generic, it is
better located in sql_plist.h.
|
| |
|
|
|
|
|
|
|
|
|
| |
for MyISAM in sysbench OLTP_RW test".
Fixes compilation warnings about local variable
shadowing I_P_List_fast_push_back::last member.
Renamed I_P_List_fast_push_back::last member
to m_last. Also to keep member naming consistent
renamed I_P_List::first member to m_first.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
locks for DML statements and changes the way MDL locks
are acquired/granted in contended case.
Instead of backing-off when a lock conflict is encountered
and waiting for it to go away before restarting open_tables()
process we now wait for lock to be released without releasing
any previously acquired locks. If conflicting lock goes away
we resume opening tables. If waiting leads to a deadlock we
try to resolve it by backing-off and restarting open_tables()
immediately.
As result both waiting for possibility to acquire and
acquiring of a metadata lock now always happen within the
same MDL API call. This has allowed to make release of a lock
and granting it to the most appropriate pending request an
atomic operation.
Thanks to this it became possible to wake up during release
of lock only those waiters which requests can be satisfied
at the moment as well as wake up only one waiter in case
when granting its request would prevent all other requests
from being satisfied. This solves thundering herd problem
which occured in cases when we were releasing some lock and
woke up many waiters for SNRW or X locks (this was the issue
in bug#52289 "performance regression for MyISAM in sysbench
OLTP_RW test".
This also allowed to implement more fair (FIFO) scheduling
among waiters with the same priority.
It also opens the door for introducing new types of requests
for metadata locks such as low-prio SNRW lock which is
necessary in order to support LOCK TABLES LOW_PRIORITY WRITE.
Notice that after this sometimes can report ER_LOCK_DEADLOCK
error in cases in which it has not happened before.
Particularly we will always report this error if waiting for
conflicting lock has happened in the middle of transaction
and resulted in a deadlock. Before this patch the error was
not reported if deadlock could have been resolved by backing
off all metadata locks acquired by the current statement.
mysql-test/r/mdl_sync.result:
Added test coverage for some aspects of deadlock handling in
metadata locking subsystem.
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/r/sp_sync.result:
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
Adjusted test results after renaming MDL_context::
m_waiting_for_lock rwlock to m_LOCK_waiting_for.
mysql-test/suite/rpl/r/rpl_sp.result:
Adjusted test case after implementing new approach to
acquiring metadata locks in open_tables(). We no longer
release all MDL locks acquired by statement before waiting
for conflicting lock to go away. As result DROP FUNCTION
statement has to wait for DML statement which managed to
acquire metadata lock on function being dropped and now
waits for other conflicting metadata lock to go away.
mysql-test/suite/rpl/t/rpl_sp.test:
Adjusted test case after implementing new approach to
acquiring metadata locks in open_tables(). We no longer
release all MDL locks acquired by statement before waiting
for conflicting lock to go away. As result DROP FUNCTION
statement has to wait for DML statement which managed to
acquire metadata lock on function being dropped and now
waits for other conflicting metadata lock to go away.
mysql-test/t/mdl_sync.test:
Added test coverage for some aspects of deadlock handling in
metadata locking subsystem.
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
mysql-test/t/sp_sync.test:
Adjusted test case after removing back-off in general case
when conflicting metadata lock is encountered during
open_tables() (now this happens only if waiting for
conflicting lock to go away leads to a deadlock).
sql/mdl.cc:
Changed MDL subsystem to support new approach to acquring
metadata locks in open tables and more fair and efficient
scheduling of metadata locks. To implement this:
- Made releasing of the lock and granting it to the most
appropriate pending request atomic operation. As result it
became possible to wake up only those waiters requests from
which can be satisfied at the moment as well as wake-up
only one waiter in case when granting its request would
prevent all other requests from being satisfied.
This solved thundering herd problem which occured in cases
when we were releasing some lock and woke up many waiters
for SNRW or X locks (this was the issue in Bug #52289
"performance regression for MyISAM in sysbench OLTP_RW
test".
To emphasize above changes wake_up_waiters() was renamed
to MDL_context::reschedule_waiters().
- Changed code to add tickets for new requests to the back of
waiters queue and to select tickets to be satisfied from
the head of the queue if possible (this makes scheduling of
requests with the same priority fair). To be able to do
this efficiently we now use for waiting and granted queues
version of I_P_List class which provides fast push_back()
method.
- Members and methods of MDL_context related to sending
and waiting for signal were moved to separate MDL_wait
class.
- Since in order to avoid race conditions we must grant the
lock only to the context which was not chosen as a victim
of deadlock, killed or aborted due to timeout
MDL_wait::set_status() (former awake()) was changed not to
send signal if signal slot is already occupied and to
indicate this fact through its return value. As another
consequence MDL_wait::timed_wait() method was changed to
handle timeout (optionally) and abort due to kill as
signals which make signal slot occupied.
- Renamed MDL_context::acquire_lock_impl() to acquire_lock().
Changed it to be able correctly process requests for shared
locks when there are open HANDLERs, made this method more
optimized for acquisition of shared locks. As part of this
change moved code common between try_acquire_lock() and
acquire_lock() to new try_acquire_lock_impl() method.
Also adjusted acquire_lock()'s code to take into account
the fact that in cases when lock is granted as result of
MDL_context::reschedule_waiters() call (i.e. when it is
granted after waiting for lock conflict to go away)
updating MDL_lock state is responsibility of the thread
calling reschedule_waiters().
- Changed MDL_context::find_deadlock() to send VICTIM
signal even if victim is the context which has initiated
deadlock detection. This is required in order to avoid
races in cases when the same context simultaneously is
chosen as a victim and its request for lock is satisfied.
As result return value of this method became unnecessary
and it was changed to return void.
Adjusted MDL_lock::find_deadlock() method to take into
account that now there can be a discrepancy between
MDL_context::m_waiting_for value being set and real state
of the ticket this member points to.
- Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
and MDL_context::stop_waiting() to done_waiting_for().
- Finally, removed MDL_context::wait_for_lock() method.
sql/mdl.h:
Changed MDL subsystem to support new approach to acquring
metadata locks in open tables and more fair and efficient
scheduling of metadata locks. To implement this:
- Members and methods of MDL_context related to sending
and waiting for signal were moved to separate MDL_wait
class.
- Since now in order to avoid race conditions we must grant
the lock only to the context which was not chosen as a
victim of deadlock, killed or aborted due to timeout
MDL_wait::set_status (former awake()) was changed not to
send signal if signal slot is already occupied and to
indicate this fact through its return value.
Also NORMAL_WAKE_UP signal became GRANTED, and timeouts
and aborts due to kill became full blown signals rather
than simple return values.
- MDL_wait::timed_wait() now takes extra parameter that
indicates whether signal should be set if timeout is
reached.
- Enabled fast push_back() operation in MDL_context::m_tickets
list to make move_ticket_after_trans_sentinel() method more
efficient.
- Removed MDL_context::wait_for_lock() method.
- Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for
and MDL_context::stop_waiting() to done_waiting_for().
- MDL_context::acquire_lock_impl() became acquire_lock().
- Introduced MDL_context::try_acquire_lock_impl() as a
place for code shared by try_acquire_lock and
acquire_lock().
- Due to fact that now VICTIM signal is sent even if victim
is the context which has initiated deadlock detection
find_deadlock() no longer needs a return value.
sql/sql_base.cc:
Implemented new approach to acquiring metadata locks in
open_tables(). We no longer perform back-off when conflicting
metadata lock is encountered. Instead we wait for this lock
to go away while holding all locks which were acquired so
far. Back-off is only used in situation when further waiting
will cause a deadlock which could be avoided by performing
back-off and restarting open_tables() process. Absence of
waiting between back-off and restart of acquiring metadata
locks can't lead to livelocks as MDL subsystem was changed
to make release of lock and granting it to waiting lock
an atomic action, so back-off will automatically give way
to other participants of deadlock loop.
Accordingly:
- open_table_get_mdl_lock() and open_and_process_routine()
were changed to wait for conflicting metadata lock to
go away without back-off. Only if such wait leads to a
deadlock back-off is requested. As part of this change
new error handler class was introduced which converts,
if possible, ER_LOCK_DEADLOCK error to a request for
back-off and re-start of open_tables() process.
- Open_table_context::recover_from_failed_open() was changed
not to wait in case of metadata lock conflict. Instead we
immediately proceed to re-acquiring locks.
- Open_table_context::request_backoff_action() now always
emits error if back-off is requested in the middle of
transaction as we can't be sure that releasing lock
which were acquired only by current statement will
resolve a deadlock. Before this patch such situations were
successfully detected thanks to the fact that we called
MDL_context::wait_for_lock() method in
recover_from_failed_open().
- In order to avoid deadlocks open_tables() code was adjusted
to flush open HANDLERs for which there are pending requests
for X locks before restarting the process of acquiring
metadata locks.
- Changed close_tables_for_reopen() not to reset MDL_request
for tables belonging to the tail of prelocking list. It is
no longer necessary as these MDL_request objects won't be
used for any waiting.
- Adjusted comment in tdc_wait_for_old_version() to avoid
mentioning removed MDL_context::wait_for_lock() method.
sql/sql_base.h:
As we no longer wait for conflicting metadata lock away in
Open_table_context::recover_from_failed_open() method,
Open_table_context::OT_WAIT_MDL_LOCK action was renamed to
OT_MDL_CONFLICT.
Also Open_table_context::m_failed_mdl_request became
unnecessary and was removed.
sql/sql_plist.h:
Extended I_P_List template to support efficient push_back()
operation if it is parameterized with an appropriate policy
class.
sql/sql_show.cc:
Adjusted code after removal of MDL_context::wait_for_lock()
method. Now if one needs to acquire metadata lock with waiting
one has to use a variant of MDL_context::acquire_lock() method.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
condition variable per context instead of one mutex and one conditional
variable for the whole subsystem.
This should increase concurrency in this subsystem.
It also opens the way for further changes which are necessary to solve
such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
and bug #37346 "innodb does not detect deadlock between update and alter
table".
Two other notable changes done by this patch:
- MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object metadata lock is acquired. Now this has
to be done by explicit calls outside of MDL subsystem.
- Instead of using separate MDL_context for opening system tables/tables
for purposes of I_S we now create MDL savepoint in the main context
before opening tables and rollback to this savepoint after closing
them. This means that it is now possible to get ER_LOCK_DEADLOCK error
even not inside a transaction. This might happen in unlikely case when
one runs DDL on one of system tables while also running DDL on some
other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
will be addressed by advanced deadlock detector for MDL subsystem which
we plan to implement.
mysql-test/include/handler.inc:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/handler_innodb.result:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/handler_myisam.result:
Adjusted handler_myisam.test and handler_innodb.test to the fact that
exclusive metadata locks on tables are now acquired according to
alphabetical order of fully qualified table names instead of order
in which tables are mentioned in statement.
mysql-test/r/mdl_sync.result:
Adjusted mdl_sync.test to the fact that exclusive metadata locks on
tables are now acquired according to alphabetical order of fully
qualified table names instead of order in which tables are mentioned
in statement.
mysql-test/t/mdl_sync.test:
Adjusted mdl_sync.test to the fact that exclusive metadata locks on
tables are now acquired according to alphabetical order of fully
qualified table names instead of order in which tables are mentioned
in statement.
sql/events.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/ha_ndbcluster.cc:
Since manipulations with open table state no longer install proxy
MDL_context it does not make sense to perform them in order to
satisfy assert in mysql_rm_tables_part2(). Removed them per agreement
with Cluster team. This has not broken test suite since scenario in
which deadlock can occur and assertion fails is not covered by tests.
sql/lock.cc:
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object exclusive metadata lock is acquired.
Now this has to be done by explicit calls outside of MDL subsystem.
sql/log.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/mdl.cc:
Changed metadata locking subsystem to use mutex per lock and condition
variable per context instead of one mutex and one conditional variable
for the whole subsystem.
Changed approach to handling of global metadata locks. Instead of
implicitly acquiring intention exclusive locks when user requests
per-object upgradeable or exclusive locks now we require them to be
acquired explicitly in the same way as ordinary metadata locks.
In fact global lock are now ordinary metadata locks in new GLOBAL
namespace.
To implement these changes:
- Removed LOCK_mdl mutex and COND_mdl condition variable.
- Introduced MDL_lock::m_mutex mutexes which protect individual lock
objects.
- Replaced mdl_locks hash with MDL_map class, which has hash for
MDL_lock objects as a member and separate mutex which protects this
hash. Methods of this class allow to find(), find_or_create() or
remove() MDL_lock objects in concurrency-friendly fashion (i.e.
for most common operation, find_or_create(), we don't acquire
MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR
for this idea and benchmarks!). Added three auxiliary members to
MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to
support this concurrency-friendly behavior.
- Introduced MDL_context::m_ctx_wakeup_cond condition variable to be
used for waiting until this context's pending request can be
satisfied or its thread has to perform actions to resolve potential
deadlock. Context which want to wait add ticket corresponding to the
request to an appropriate queue of waiters in MDL_lock object so
they can be noticed when other contexts change state of lock and be
awaken by them by signalling on MDL_context::m_ctx_wakeup_cond.
As consequence MDL_ticket objects has to be used for any waiting
in metadata locking subsystem including one which happens in
MDL_context::wait_for_locks() method.
Another consequence is that MDL_context is no longer copyable and
can't be saved/restored when working with system tables.
- Made MDL_lock an abstract class, which delegates specifying exact
compatibility matrix to its descendants. Added MDL_global_lock child
class for global lock (The old is_lock_type_compatible() method
became can_grant_lock() method of this class). Added MDL_object_lock
class to represent per-object lock (The old MDL_lock::can_grant_lock()
became its method). Choice between two classes happens based on MDL
namespace in MDL_lock::create() method.
- Got rid of MDL_lock::type member as its meaning became ambigous for
global locks.
- To simplify waking up of contexts waiting for lock split waiting queue
in MDL_lock class in two queues. One for pending requests for exclusive
(including intention exclusive) locks and another for requests for
shared locks.
- Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and
MDL_object_lock classes which allows to wake up waiting contexts after
state of lock changes. Replaced old duplicated code with calls to this
method.
- Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/
global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock()
and MDL_context::release_ticket() methods to use MDL_map and
MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up
waiters according to the approach described above. The latter method
also was renamed to MDL_context::release_lock().
- Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and
release_lock() not to handle global locks. They are now supposed to
be taken explicitly like ordinary metadata locks.
- Added helper MDL_context::try_acquire_global_intention_exclusive_lock()
and acquire_global_intention_exclusive_lock() methods.
- Moved common code from MDL_context::acquire_global_shared_lock() and
acquire_global_intention_exclusive_lock() to new method -
MDL_context::acquire_lock_impl().
- Moved common code from MDL_context::try_acquire_shared_lock(),
try_acquire_global_intention_exclusive_lock()/exclusive_lock()
to MDL_context::try_acquire_lock_impl().
- Since acquiring of several exclusive locks can no longer happen under
single LOCK_mdl mutex the approach to it had to be changed. Now we do
it in one by one fashion. This is done in alphabetical order to avoid
deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly
(as part of this change moved code responsible for acquiring single
exclusive lock to new MDL_context::acquire_exclusive_lock_impl()
method).
- Since we no longer have single LOCK_mdl mutex which protects all
MDL_context::m_is_waiting_in_mdl members using these members to
determine if we have really awaken context holding conflicting
shared lock became inconvinient. Got rid of this member and changed
notify_shared_lock() helper function and process of acquiring
of/upgrading to exclusive lock not to rely on such information.
Now in MDL_context::acquire_exclusive_lock_impl() and
MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply
re-try to wake up threads holding conflicting shared locks after
small time out.
- Adjusted MDL_context::can_wait_lead_to_deadlock() and
MDL_ticket::has_pending_conflicting_lock() to use per-lock
mutexes instead of LOCK_mdl. To do this introduced
MDL_lock::has_pending_exclusive_lock() method.
sql/mdl.h:
Changed metadata locking subsystem to use mutex per lock and condition
variable per context instead of one mutex and one conditional variable
for the whole subsystem. In order to implement this change:
- Added MDL_key::cmp() method to be able to sort MDL_key objects
alphabetically. Changed length fields in MDL_key class to uint16
as 16-bit is enough for length of any key.
- Changed MDL_ticket::get_ctx() to return pointer to non-const
object in order to be able to use MDL_context::awake() method
for such contexts.
- Got rid of unlocked versions of can_wait_lead_to_deadlock()/
has_pending_conflicting_lock() methods in MDL_context and
MDL_ticket. We no longer has single mutex which protects all
locks. Thus one always has to use versions of these methods
which acquire per-lock mutexes.
- MDL_request_list type of list now counts its elements.
- Added MDL_context::m_ctx_wakeup_cond condition variable to be used
for waiting until this context's pending request can be satisfied
or its thread has to perform actions to resolve potential deadlock.
Added awake() method to wake up context from such wait.
Addition of condition variable made MDL_context uncopyable.
As result we no longer can save/restore MDL_context when working
with system tables. Instead we create MDL savepoint before opening
those tables and rollback to it once they are closed.
- MDL_context::release_ticket() became release_lock() method.
- Added auxiliary MDL_context::acquire_exclusive_lock_impl() method
which does all necessary work to acquire exclusive lock on one object
but should not be used directly as it does not enforce any asserts
ensuring that no deadlocks are possible.
- Since we no longer need to know if thread trying to acquire exclusive
lock managed to wake up any threads having conflicting shared locks
(as, anyway, we will try to wake up such threads again shortly)
- MDL_context::m_is_waiting_in_mdl member became unnecessary and
notify_shared_lock() no longer needs to be friend of MDL_context.
Changed approach to handling of global metadata locks. Instead of
implicitly acquiring intention exclusive locks when user requests
per-object upgradeable or exclusive locks now we require them to be
acquired explicitly in the same way as ordinary metadata locks.
- Added new GLOBAL namespace for such locks.
- Added new type of lock to be requested MDL_INTENTION_EXCLISIVE.
- Added MDL_context::try_acquire_global_intention_exclusive_lock()
and acquire_global_intention_exclusive_lock() methods.
- Moved common code from MDL_context::acquire_global_shared_lock()
and acquire_global_intention_exclusive_lock() to new method -
MDL_context::acquire_lock_impl().
- Moved common code from MDL_context::try_acquire_shared_lock(),
try_acquire_global_intention_exclusive_lock()/exclusive_lock()
to MDL_context::try_acquire_lock_impl().
- Added helper MDL_context::is_global_lock_owner() method to be
able easily to find what kind of global lock this context holds.
- MDL_context::m_has_global_shared_lock became unnecessary as
global read lock is now represented by ordinary ticket.
- Removed assert in MDL_context::set_lt_or_ha_sentinel() which became
false for cases when we execute LOCK TABLES under global read lock
mode.
sql/mysql_priv.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result calls
opening/closing system tables were changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sp.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sp.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sql_base.cc:
close_thread_tables():
Since we no longer use separate MDL_context for opening system
tables we need to avoid releasing all transaction locks when
closing system table. Releasing metadata lock on system table
is now responsibility of THD::restore_backup_open_tables_state().
open_table_get_mdl_lock(),
Open_table_context::recover_from_failed_open():
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object upgradable or exclusive metadata lock is
acquired. So this have to be done explicitly from these calls.
Changed Open_table_context class to store MDL_request object for
global intention exclusive lock acquired when opening tables.
open_table():
Do not release metadata lock if we have failed to open table as
this lock might have been acquired by one of previous statements
in transaction, and therefore should not be released.
open_system_tables_for_read()/close_system_tables()/
open_performance_schema_table():
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
close_performance_schema_table():
Got rid of duplicated code.
sql/sql_class.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. Also releasing
metadata lock on system table is now responsibility of
THD::restore_backup_open_tables_state().
Adjusted assert in THD::cleanup() to take into account fact that now
we also use MDL sentinel for global read lock.
sql/sql_class.h:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. As result:
- 'mdl_context' member was moved out of Open_tables_state to THD class.
enter_locked_tables_mode()/leave_locked_tables_mode() had to follow.
- Methods of THD responsible for saving/restoring open table state were
changed to use Open_tables_backup class which in addition to
Open_table_state has a member for this savepoint.
Changed Open_table_context class to store MDL_request object for
global intention exclusive lock acquired when opening tables.
sql/sql_delete.cc:
MDL subsystem no longer implicitly acquires global intention exclusive
metadata lock when per-object exclusive metadata lock is acquired.
Now this has to be done by explicit calls outside of MDL subsystem.
sql/sql_help.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
sql/sql_parse.cc:
Adjusted assert reload_acl_and_cache() to the fact that global read
lock now takes full-blown metadata lock.
sql/sql_plist.h:
Added support for element counting to I_P_List list template.
One can use policy classes to specify if such counting is needed
or not needed for particular list.
sql/sql_show.cc:
Instead of using separate MDL_context for opening tables for I_S
purposes we now create MDL savepoint in the main context before
opening tables and rollback to this savepoint after closing them.
To support this and similar change for system tables methods of
THD responsible for saving/restoring open table state were changed
to use Open_tables_backup class which in addition to Open_table_state
has a member for this savepoint. As result code opening/closing tables
for I_S purposes was changed to use Open_tables_backup instead of
Open_table_state class as well.
sql/sql_table.cc:
mysql_rm_tables_part2():
Since now global intention exclusive metadata lock is ordinary
metadata lock we no longer can rely that by releasing MDL locks
on all tables we will release all locks acquired by this routine.
So in non-LOCK-TABLES mode we have to release all locks acquired
explicitly.
prepare_for_repair(), mysql_alter_table():
MDL subsystem no longer implicitly acquires global intention
exclusive metadata lock when per-object exclusive metadata lock
is acquired. Now this has to be done by explicit calls outside of
MDL subsystem.
sql/tztime.cc:
Instead of using separate MDL_context for opening system tables we now
create MDL savepoint in the main context before opening such tables
and rollback to this savepoint after closing them. To support this
change methods of THD responsible for saving/restoring open table
state were changed to use Open_tables_backup class which in addition
to Open_table_state has a member for this savepoint. As result code
opening/closing system tables was changed to use Open_tables_backup
instead of Open_table_state class as well.
Also changed code not to use special mechanism for open system tables
when it is not really necessary.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change is supposed to reduce number of ER_LOCK_DEADLOCK
errors which occur when multi-statement transaction encounters
conflicting metadata lock in cases when waiting is possible.
The idea is not to fail ER_LOCK_DEADLOCK error immediately when
we encounter conflicting metadata lock. Instead we release all
metadata locks acquired by current statement and start to wait
until conflicting lock go away. To avoid deadlocks we use simple
empiric which aborts waiting with ER_LOCK_DEADLOCK error if it
turns out that somebody is waiting for metadata locks owned by
this transaction.
This patch also fixes bug #46273 "MySQL 5.4.4 new MDL: Bug#989
is not fully fixed in case of ALTER".
The bug was that concurrent execution of UPDATE or MULTI-UPDATE
statement as a part of multi-statement transaction that already
has used table being updated and ALTER TABLE statement might have
resulted of loss of isolation between this transaction and ALTER
TABLE statement, which manifested itself as changes performed by
ALTER TABLE becoming visible in transaction and wrong binary log
order as a consequence.
This problem occurred when UPDATE or MULTI-UPDATE's wait in
mysql_lock_tables() call was aborted due to metadata lock
upgrade performed by concurrent ALTER TABLE. After such abort all
metadata locks held by transaction were released but transaction
silently continued to be executed as if nothing has happened.
We solve this problem by changing our code not to release all
locks in such case. Instead we release only locks which were
acquired by current statement and then try to reacquire them
by restarting open/lock tables process. We piggyback on simple
deadlock detector implementation since this change has to be
done anyway for it.
mysql-test/include/handler.inc:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/handler_innodb.result:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/handler_myisam.result:
After introduction of basic deadlock detector for metadata locks
it became necessary to change parts of test for HANDLER statements
which covered some of scenarios in which ER_LOCK_DEADLOCK error
was detected in absence of real deadlock (with new deadlock detector
this no longer happens).
Also adjusted test to the fact that HANDLER READ for the table no
longer will be blocked by ALTER TABLE for the same table which awaits
for metadata lock upgrade (this is due to removal of mysql_lock_abort()
from wait_while_table_is_used()).
mysql-test/r/mdl_sync.result:
Added test coverage for basic deadlock detection in metadata
locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL:
Bug#989 is not fully fixed in case of ALTER".
mysql-test/r/sp-lock.result:
Adjusted test coverage for metadata locking for stored routines
since after introduction of basic deadlock detector for metadata
locks number of scenarios in which ER_LOCK_DEADLOCK error in
absence of deadlock has decreased.
mysql-test/t/mdl_sync.test:
Added test coverage for basic deadlock detection in metadata
locking subsystem and for bug #46273 "MySQL 5.4.4 new MDL:
Bug#989 is not fully fixed in case of ALTER".
mysql-test/t/sp-lock.test:
Adjusted test coverage for metadata locking for stored routines
since after introduction of basic deadlock detector for metadata
locks number of scenarios in which ER_LOCK_DEADLOCK error in
absence of deadlock has decreased.
sql/log_event_old.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/mdl.cc:
Changed metadata locking subsystem to support basic deadlock detection
with a help of the following simple empiric -- we assume that there is
a deadlock if there is a connection which has to wait for a metadata
lock which is currently acquired by some connection which is itself
waiting to be able to acquire some shared metadata lock.
To implement this change:
- Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods
which allow to find out if there is someone waiting for metadata
lock which is held by the connection and therefore deadlocks are
possible if this connection is going to wait for some metadata lock.
To do this added version of MDL_ticket::has_pending_conflicting_lock()
method which assumes that its caller already owns LOCK_mdl mutex.
- Changed MDL_context::wait_for_locks() to use one of the above methods
to check if somebody is waiting for metadata lock owned by this
context (and therefore deadlock is possible) and emit ER_LOCK_DEADLOCK
error in this case. Also now we mark context of connections waiting
inside of this method by setting MDL_context::m_is_waiting_in_mdl
member. Thanks to this such connection could be waken up if some
other connection starts waiting for one of its metadata locks and
so a deadlock can occur.
- Adjusted notify_shared_lock() to wake up connections which wait inside
MDL_context::wait_for_locks() while holding shared metadata lock.
- Changed MDL_ticket::upgrade_shared_lock_to_exclusive() to add
temporary ticket for exclusive lock to MDL_lock::waiting queue, so
request for metadata lock upgrade can be properly detected by our
empiric.
Also now this method invokes a callback which forces transactions
holding shared metadata lock on the table to call MDL_context::
can_wait_lead_to_deadlock() method even if they don't need any new
metadata locks. Thanks to this such transactions can detect deadlocks/
livelocks between MDL and table-level locks.
Also reduced timeouts between calls to notify_shared_lock()
in MDL_ticket::upgrade_shared_lock_to_exclusive() and
MDL_context::acquire_exclusive_locks(). This was necessary
to get rid of call to mysql_lock_abort() in wait_while_table_is_used().
(Now we instead rely on notify_shared_lock() timely calling
mysql_lock_abort_for_thread() for the table on which lock
is being upgraded/acquired).
sql/mdl.h:
- Added a version of MDL_ticket::has_pending_conflicting_lock() method
to be used in situations when caller already has acquired LOCK_mdl
mutex.
- Added MDL_context::can_wait_lead_to_deadlock()/_impl() methods
which allow to find out if there is someone waiting for metadata lock
which is held by this connection and thus deadlocks are possible if
this connections will start waiting for some metadata lock.
- Added MDL_context::m_is_waiting_in_mdl member to mark connections
waiting in MDL_context::wait_for_locks() method of metadata locking
subsystem. Added getter method for this private member to make it
accessible in notify_shared_lock() auxiliary so we can wake-up such
connections if they hold shared metadata locks.
- Finally, added mysql_abort_transactions_with_shared_lock() callback
to be able force transactions which don't need any new metadata
locks still call MDL_context::can_wait_lead_to_deadlock() and detect
some of deadlocks between metadata locks and table-level locks.
sql/mysql_priv.h:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_base.cc:
Changed approach to metadata locking for multi-statement transactions.
We no longer fail ER_LOCK_DEADLOCK error immediately when we encounter
conflicting metadata lock. Instead we release all metadata locks
acquired by current statement and start to wait until conflicting
locks to go away by calling MDL_context::wait_for_locks() method.
To avoid deadlocks the latter implements simple empiric which aborts
waiting with ER_LOCK_DEADLOCK error if it turns out that somebody
is waiting for metadata locks owned by this transaction.
To implement the change described above:
- Introduced Open_table_context::m_start_of_statement_svp member to
store state of metadata locks at the start of the statement.
- Changed Open_table_context::request_backoff_action() not to
fail with ER_LOCK_DEADLOCK immediately if back-off is requested
due to conflicting metadata lock.
- Added new argument for close_tables_for_reopen() procedure which
allows to specify subset of metadata locks to be released.
- Changed open_tables() not to release all metadata locks acquired
by current transaction when metadata lock conflict is discovered.
Instead we release only locks acquired by current statement.
- Changed open_ltable() and open_and_lock_tables_derived() not to emit
ER_LOCK_DEADLOCK error when mysql_lock_tables() is aborted in
multi-statement transaction when somebody tries to acquire exclusive
metadata lock on the table. Instead we release metadata locks acquired
by current statement and try to wait until they can be re-acquired.
- Adjusted tdc_wait_for_old_versions() to check if there is someone
waiting for one of metadata locks held by this connection and run
deadlock detection in order to avoid deadlocks in some
situations.
- Added mysql_abort_transactions_with_shared_lock() callback which
allows to force transactions holding shared metadata lock on the
table to call MDL_context::can_wait_lead_to_deadlock() even if they
don't need any new metadata locks so they can detect potential
deadlocks between metadata locking subsystem and table-level locks.
- Adjusted wait_while_table_is_used() not to set TABLE::version to
0 as it is now done only when necessary by the above-mentioned
callback. Also removed unnecessary call to mysql_lock_abort().
Instead we rely on code performing metadata lock upgrade aborting
waits on the table-level lock for this table by calling
mysql_lock_abort_for_thread() (invoked by
mysql_notify_thread_having_shared_lock()). In future this should
allow to reduce number of scenarios in which we produce
ER_LOCK_DEADLOCK error even though no real deadlock exists.
sql/sql_class.h:
Introduced Open_table_context::m_start_of_statement_svp member to
store state of metadata locks at the start of the statement.
Replaced Open_table_context::m_can_deadlock member with m_has_locks
member to reflect the fact that we no longer unconditionally emit
ER_LOCK_DEADLOCK error for transaction having some metadata locks
when conflicting metadata lock is discovered.
sql/sql_insert.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_plist.h:
Made I_P_List_iterator<T, B> usable with const lists.
sql/sql_show.cc:
close_tables_for_reopen() now takes one more argument which
specifies at which point it should stop releasing metadata
locks acquired by this connection.
sql/sql_update.cc:
Changed UPDATE and MULTI-UPDATE code not to release all metadata
locks when calls to mysql_lock_tables() are aborted. Instead we
release only locks which are acquired by this statement and then
try to reacquire them by calling open_tables(). This solves
bug #46273 "MySQL 5.4.4 new MDL: Bug#989 is not fully fixed in
case of ALTER".
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"HANDLER statements within a transaction might lead to deadlocks".
Introduce a notion of a sentinel to MDL_context. A sentinel
is a ticket that separates all tickets in the context into two
groups: before and after it. Currently we can have (and need) only
one designated sentinel -- it separates all locks taken by LOCK
TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK
and all other locks, which must be released at COMMIT or ROLLBACK.
The tricky part is maintaining the sentinel up to date when
someone release its corresponding ticket. This can happen, e.g.
if someone issues DROP TABLE under LOCK TABLES (generally,
see all calls to release_all_locks_for_name()).
MDL_context::release_ticket() is modified to take care of it.
******
A fix and a test case for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
An attempt to mix HANDLER SQL statements, which are transaction-
agnostic, an open multi-statement transaction,
and DDL against the involved tables (in a concurrent connection)
could lead to a deadlock. The deadlock would occur when
HANDLER OPEN or HANDLER READ would have to wait on a conflicting
metadata lock. If the connection that issued HANDLER statement
also had other metadata locks (say, acquired in scope of a
transaction), a classical deadlock situation of mutual wait
could occur.
Incompatible change: entering LOCK TABLES mode automatically
closes all open HANDLERs in the current connection.
Incompatible change: previously an attempt to wait on a lock
in a connection that has an open HANDLER statement could wait
indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK
is produced.
The idea of the fix is to merge thd->handler_mdl_context
with the main mdl_context of the connection, used for transactional
locks. This makes deadlock detection possible, since all waits
with locks are "visible" and available to analysis in a single
MDL context of the connection.
Since HANDLER locks and transactional locks have a different life
cycle -- HANDLERs are explicitly open and closed, and so
are HANDLER locks, explicitly acquired and released, whereas
transactional locks "accumulate" till the end of a transaction
and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT,
a concept of "sentinel" was introduced to MDL_context.
All locks, HANDLER and others, reside in the same linked list.
However, a selected element of the list separates locks with
different life cycle. HANDLER locks always reside at the
end of the list, after the sentinel. Transactional locks are
prepended to the beginning of the list, before the sentinel.
Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only
release those locks that reside before the sentinel. HANDLER locks
must be released explicitly as part of HANDLER CLOSE statement,
or an implicit close.
The same approach with sentinel
is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES
statement has never worked together, the implementation is
made simple and only maintains one sentinel, which is used either
for HANDLER locks, or for LOCK TABLES locks.
mysql-test/include/handler.inc:
Add test coverage for Bug#46224 "HANDLER statements within a
transaction might lead to deadlocks".
Extended HANDLER coverage to cover a mix of HANDLER, transactions
and DDL statements.
mysql-test/r/handler_innodb.result:
Update results (Bug#46224).
mysql-test/r/handler_myisam.result:
Update results (Bug#46224).
sql/lock.cc:
Remove thd->some_tables_deleted, it's never used.
sql/log_event.cc:
No need to check for thd->locked_tables_mode,
it's done inside release_transactional_locks().
sql/mdl.cc:
Implement the concept of HANDLER and LOCK TABLES "sentinel".
Implement a method to clone an acquired ticket.
Do not return tickets beyond the sentinel when acquiring
locks, create a copy.
Remove methods to merge and backup MDL_context, they are now
not used (Hurra!). This opens a path to a proper constructor
and destructor of class MDL_context (to be done in a separate
patch).
Modify find_ticket() to provide information about where
the ticket position is with regard to the sentinel.
sql/mdl.h:
Add declarations necessary for the implementation of the concept
of "sentinel", a dedicated ticket separating transactional and
non-transactional locks.
sql/mysql_priv.h:
Add mark_tmp_table_for_reuse() declaration,
a function to "close" a single session (temporary) table.
sql/sql_base.cc:
Remove thd->some_tables_deleted.
Modify deadlock-prevention asserts and deadlock detection
heuristics to take into account that from now on HANDLER locks
reside in the same locking context.
Add broadcast_refresh() to mysql_notify_thread_having_shared_lock():
this is necessary for the case when a thread having a shared lock
is asleep in tdc_wait_for_old_versions(). This situation is only
possible with HANDLER t1 OPEN; FLUSH TABLE (since all over code paths
that lead to tdc_wait_for_old_versions() always have an
empty MDL_context). Previously the server would simply deadlock
in this situation.
sql/sql_class.cc:
Remove now unused member "THD::some_tables_deleted".
Move mysql_ha_cleanup() a few lines above in THD::cleanup()
to make sure that all handlers are closed when it's time to
destroy the MDL_context of this connection.
Remove handler_mdl_context and handler_tables.
sql/sql_class.h:
Remove THD::handler_tables, THD::handler_mdl_context,
THD::some_tables_deleted.
sql/sql_handler.cc:
Remove thd->handler_tables.
Remove thd->handler_mdl_context.
Rewrite mysql_ha_open() to have no special provision for MERGE
tables, now that we don't have to manipulate with thd->handler_tables
it's easy to do.
Remove dead code.
Fix a bug in mysql_ha_flush() when we would always flush
a temporary HANDLER when mysql_ha_flush() is called (actually
mysql_ha_flush() never needs to flush temporary tables).
sql/sql_insert.cc:
Update a comment, no more thd->some_tables_deleted.
sql/sql_parse.cc:
Implement an incompatible change: entering LOCK TABLES closes
active HANDLERs, if any.
Now that we have a sentinel, we don't need to check
for thd->locked_tables_mode when releasing metadata locks in
COMMIT/ROLLBACK.
sql/sql_plist.h:
Add new (now necessary) methods to the list class.
sql/sql_prepare.cc:
Make sure we don't release HANDLER locks when rollback to a
savepoint, set to not keep locks taken at PREPARE.
sql/sql_servers.cc:
Update to a new signature of MDL_context::release_all_locks().
sql/sql_table.cc:
Remove thd->some_tables_deleted.
sql/transaction.cc:
Add comments.
Make sure rollback to (MDL) savepoint works under LOCK TABLES and
with HANDLER tables.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
----------------------------------------------------------
revno: 2617.23.20
committer: Konstantin Osipov <kostja@sun.com>
branch nick: mysql-6.0-runtime
timestamp: Wed 2009-03-04 16:31:31 +0300
message:
WL#4284 "Transactional DDL locking"
Review comments: "Objectify" the MDL API.
MDL_request and MDL_context still need manual construction and
destruction, since they are used in environment that is averse
to constructors/destructors.
sql/mdl.cc:
Improve comments.
Add asserts to backup()/restore_from_backup()/merge() methods.
Fix an order bug in the error path of mdl_acquire_exclusive_locks():
we used to first free a ticket object, and only then exclude
it from the list of tickets.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
------------------------------------------------------------
revno: 2617.23.18
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 4284-6.0
timestamp: Mon 2009-03-02 18:18:26 -0300
message:
Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order
WL#4284: Transactional DDL locking
This is a prerequisite patch:
These changes are intended to split lock requests from granted
locks and to allow the memory and lifetime of granted locks to
be managed within the MDL subsystem. Furthermore, tickets can
now be shared and therefore are used to satisfy multiple lock
requests, but only shared locks can be recursive.
The problem is that the MDL subsystem morphs lock requests into
granted locks locks but does not manage the memory and lifetime
of lock requests, and hence, does not manage the memory of
granted locks either. This can be problematic because it puts the
burden of tracking references on the users of the subsystem and
it can't be easily done in transactional contexts where the locks
have to be kept around for the duration of a transaction.
Another issue is that recursive locks (when the context trying to
acquire a lock already holds a lock on the same object) requires
that each time the lock is granted, a unique lock request/granted
lock structure structure must be kept around until the lock is
released. This can lead to memory leaks in transactional contexts
as locks taken during the transaction should only be released at
the end of the transaction. This also leads to unnecessary wake
ups (broadcasts) in the MDL subsystem if the context still holds
a equivalent of the lock being released.
These issues are exacerbated due to the fact that WL#4284 low-level
design says that the implementation should "2) Store metadata locks
in transaction memory root, rather than statement memory root" but
this is not possible because a memory root, as implemented in mysys,
requires all objects allocated from it to be freed all at once.
This patch combines review input and significant code contributions
from Konstantin Osipov (kostja) and Dmitri Lenev (dlenev).
mysql-test/r/mdl_sync.result:
Add test case result.
mysql-test/t/mdl_sync.test:
Add test case for shared lock upgrade case.
sql/event_db_repository.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/ha_ndbcluster_binlog.cc:
Use new function names to initialize MDL lock requests.
sql/lock.cc:
Rename MDL functions.
sql/log_event.cc:
The MDL request now holds the table and database name data (MDL_KEY).
sql/mdl.cc:
Move the MDL key to the MDL_LOCK structure in order to make the
object suitable for allocation from a fixed-size allocator. This
allows the simplification of the lists in the MDL_LOCK object,
which now are just two, one for granted tickets and other for
waiting (upgraders) tickets.
Recursive requests for a shared lock on the same object can now
be granted using the same lock ticket. This schema is only used
for shared locks because that the only case that matters. This
is used to avoid waste of resources in case a context (connection)
already holds a shared lock on a object.
sql/mdl.h:
Introduce a metadata lock object key which is used to uniquely
identify lock objects.
Separate the structure used to represent pending lock requests
from the structure used to represent granted metadata locks.
Rename functions used to manipulate locks requests in order to
have a more consistent function naming schema.
sql/sp_head.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/sql_acl.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_base.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_class.h:
Last acquired lock before the savepoint was set.
sql/sql_delete.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_handler.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_insert.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_parse.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_plist.h:
Typedef for iterator type.
sql/sql_plugin.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_servers.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/sql_show.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_table.cc:
Various changes to accommodate that lock requests are separated
from lock tickets (granted locks).
sql/sql_trigger.cc:
Save reference to the lock ticket so it can be downgraded later.
sql/sql_udf.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
sql/table.cc:
Rename mdl_alloc_lock to mdl_request_alloc.
sql/table.h:
Separate MDL lock requests from lock tickets (granted locks).
storage/myisammrg/ha_myisammrg.cc:
Rename alloc_mdl_locks to alloc_mdl_requests.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
------------------------------------------------------------
revno: 2630.6.1
committer: Konstantin Osipov <konstantin@mysql.com>
branch nick: mysql-6.0-3726
timestamp: Tue 2008-05-27 13:45:34 +0400
message:
Remove an unused argument from release_table_share().
Remove unused members from TABLE_SHARE struct.
Review comments in scope of WL#3726 "DDL locking for all metadata
objects"
sql/mysql_priv.h:
Update declaration.
sql/sql_base.cc:
Upate declaration and the comment (release_share()).
sql/sql_plist.h:
A cosmetic change, is_empty() is a const method.
sql/sql_table.cc:
Update to use the new declaration of release_table_share().
sql/sql_view.cc:
Update to use the new declaration of release_table_share().
sql/table.cc:
Update to use the new declaration of release_table_share().
Remove dead code.
sql/table.h:
Remove unused members of TABLE.
|
|
|
Backport of:
------------------------------------------------------------
revno: 2630.4.1
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Fri 2008-05-23 17:54:03 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
------------------------------------------------------------
This is the first patch in series. It transforms the metadata
locking subsystem to use a dedicated module (mdl.h,cc). No
significant changes in the locking protocol.
The import passes the test suite with the exception of
deprecated/removed 6.0 features, and MERGE tables. The latter
are subject to a fix by WL#4144.
Unfortunately, the original changeset comments got lost in a merge,
thus this import has its own (largely insufficient) comments.
This patch fixes Bug#25144 "replication / binlog with view breaks".
Warning: this patch introduces an incompatible change:
Under LOCK TABLES, it's no longer possible to FLUSH a table that
was not locked for WRITE.
Under LOCK TABLES, it's no longer possible to DROP a table or
VIEW that was not locked for WRITE.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.2
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:03:45 +0400
message:
WL#3726 "DDL locking for all metadata objects".
After review fixes in progress.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.3
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 14:08:51 +0400
message:
WL#3726 "DDL locking for all metadata objects"
Fixed failing Windows builds by adding mdl.cc to the lists
of files needed to build server/libmysqld on Windows.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.4
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sat 2008-05-24 21:57:58 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Fix for assert failures in kill.test which occured when one
tried to kill ALTER TABLE statement on merge table while it
was waiting in wait_while_table_is_used() for other connections
to close this table.
These assert failures stemmed from the fact that cleanup code
in this case assumed that temporary table representing new
version of table was open with adding to THD::temporary_tables
list while code which were opening this temporary table wasn't
always fulfilling this.
This patch changes code that opens new version of table to
always do this linking in. It also streamlines cleanup process
for cases when error occurs while we have new version of table
open.
******
WL#3726 "DDL locking for all metadata objects"
Add libmysqld/mdl.cc to .bzrignore.
******
Backport of:
------------------------------------------------------------
revno: 2630.4.6
committer: Dmitry Lenev <dlenev@mysql.com>
branch nick: mysql-6.0-3726-w
timestamp: Sun 2008-05-25 00:33:22 +0400
message:
WL#3726 "DDL locking for all metadata objects".
Addition to the fix of assert failures in kill.test caused by
changes for this worklog.
Make sure we close the new table only once.
.bzrignore:
Add libmysqld/mdl.cc
libmysqld/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of libmysqld.
libmysqld/Makefile.am:
Added files implementing new meta-data locking subsystem to the server.
mysql-test/include/handler.inc:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/create.result:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/r/flush.result:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/r/flush_table.result:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/r/handler_innodb.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/handler_myisam.result:
Use separate connection for waiting while threads performing DDL
operations conflicting with open HANDLER tables reach blocked
state. This is required because now we check and close tables open
by HANDLER statements in this connection conflicting with DDL in
another each time open_tables() is called and thus select from I_S
which is used for waiting will unblock DDL operations if issued
from connection with open HANDLERs.
mysql-test/r/information_schema.result:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange tests to match 6.0 better (fewer merge conflicts).
mysql-test/r/kill.result:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/r/lock.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/r/partition_column_prune.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/partition_pruning.result:
Update results (same results in 6.0), WL#3726
mysql-test/r/ps_ddl.result:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/r/sp.result:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/r/view.result:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock
mysql-test/r/view_grant.result:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/r/view_multi.result:
Added test case for bug#25144 "replication / binlog with view
breaks".
mysql-test/suite/rpl/t/disabled.def:
Disable test for deprecated features (they don't work with new MDL).
mysql-test/t/create.test:
Adjusted test case after change in implementation of CREATE TABLE
... SELECT. We no longer have special check in open_table() which
catches the case when we select from the table created. Instead we
rely on unique_table() call which happens after opening and
locking all tables.
mysql-test/t/disabled.def:
Disable merge.test, subject of WL#4144
mysql-test/t/flush.test:
FLUSH TABLES WITH READ LOCK can no longer happen under LOCK
TABLES. Updated test accordingly.
mysql-test/t/flush_table.test:
Under LOCK TABLES we no longer allow to do FLUSH TABLES for tables
locked for read. Updated test accordingly.
mysql-test/t/information_schema.test:
Additional test for WL#3726 "DDL locking for all metadata
objects". Check that we use high-priority metadata lock requests
when filling I_S tables.
Rearrange the results for easier merges with 6.0.
mysql-test/t/kill.test:
Added tests checking that DDL and DML statements waiting for
metadata locks can be interrupted by KILL command.
mysql-test/t/lock.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation write locks on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/lock_multi.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/ps_ddl.test:
We no longer invalidate prepared CREATE TABLE ... SELECT statement
if target table changes. This is OK since it is not strictly
necessary.
The first change is wrong, is caused by FLUSH TABLE
now flushing all unused tables. This is a regression that
Dmitri fixed in 6.0 in a follow up patch.
mysql-test/t/sp.test:
Under LOCK TABLES we no longer allow accessing views which were
not explicitly locked. To access view we need to obtain metadata
lock on it and doing this under LOCK TABLES may lead to deadlocks.
mysql-test/t/trigger_notembedded.test:
Adjusted test case to the changes of status in various places
caused by change in implementation FLUSH TABLES WITH READ LOCK,
which is now takes global metadata lock before flushing tables and
therefore waits on at these places.
mysql-test/t/view.test:
One no longer is allowed to do DROP VIEW under LOCK TABLES even if
this view is locked by LOCK TABLES. The problem is that in such
situation even "write locks" on view are not mutually exclusive so
upgrading metadata lock which is required for dropping of view
will lead to deadlock.
mysql-test/t/view_grant.test:
ALTER VIEW implementation was changed to open a view only after
checking that user which does alter has appropriate privileges on
it. This means that in case when user's privileges are
insufficient for this we won't check that new view definer is the
same as original one or user performing alter has SUPER privilege.
Adjusted test case accordingly.
mysql-test/t/view_multi.test:
Added test case for bug#25144 "replication / binlog with view
breaks".
sql/CMakeLists.txt:
Added mdl.cc to the list of files needed for building of server.
sql/Makefile.am:
Added files implementing new meta-data locking subsystem to the
server.
sql/event_db_repository.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when TABLE_LIST objects is also allocated
there or on stack.
sql/ha_ndbcluster.cc:
Adjusted code to work nicely with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/ha_ndbcluster_binlog.cc:
Adjusted code to work with new metadata locking subsystem.
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/handler.cc:
update_frm_version():
Directly update TABLE_SHARE::mysql_version member instead of
going through all TABLE instances for this table (old code was a
legacy from pre-table-definition-cache days).
sql/lock.cc:
Use new metadata locking subsystem. Threw away most of functions
related to name locking as now one is supposed to use metadata
locking API instead. In lock_global_read_lock() and
unlock_global_read_lock() in order to avoid problems with global
read lock sneaking in at the moment when we perform FLUSH TABLES
or ALTER TABLE under LOCK TABLES and when tables being reopened
are protected only by metadata locks we also have to take global
shared meta data lock.
sql/log_event.cc:
Adjusted code to work with new metadata locking subsystem. For
tables open by slave thread for applying RBR events allocate
memory for lock request object in the same chunk of memory as
TABLE_LIST objects for them. In order to ensure that we keep these
objects around until tables are open always close tables before
calling Relay_log_info::clear_tables_to_lock(). Use new auxiliary
Relay_log_info::slave_close_thread_tables() method to enforce
this.
sql/log_event_old.cc:
Adjusted code to work with new metadata locking subsystem. Since
for tables open by slave thread for applying RBR events memory for
lock request object is allocated in the same chunk of memory as
TABLE_LIST objects for them we have to ensure that we keep these
objects around until tables are open. To ensure this we always
close tables before calling
Relay_log_info::clear_tables_to_lock(). To enfore this we use
new auxiliary Relay_log_info::slave_close_thread_tables()
method.
sql/mdl.cc:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mdl.h:
Implemented new metadata locking subsystem and API described in
WL3726 "DDL locking for all metadata objects".
sql/mysql_priv.h:
- close_thread_tables()/close_tables_for_reopen() now has one more
argument which indicates that metadata locks should be released
but not removed from the context in order to be used later in
mdl_wait_for_locks() and tdc_wait_for_old_version().
- close_cached_table() routine is no longer public.
- Thread waiting in wait_while_table_is_used() can be now killed
so this function returns boolean to make caller aware of such
situation.
- We no longer have table cache as separate entity instead used
and unused TABLE instances are linked to TABLE_SHARE objects in
table definition cache.
- Now third argument of open_table() is also used for requesting
table repair or auto-discovery of table's new definition. So its
type was changed from bool to enum.
- Added tdc_open_view() function for opening view by getting its
definition from disk (and table cache in future).
- reopen_name_locked_table() no longer needs "link_in" argument as
now we have exclusive metadata locks instead of dummy TABLE
instances when this function is called.
- find_locked_table() now takes head of list of TABLE instances
instead of always scanning through THD::open_tables list. Also
added find_write_locked_table() auxiliary.
- reopen_tables(), close_cached_tables() no longer have
mark_share_as_old and wait_for_placeholder arguments. Instead of
relying on this parameters and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock.
- We no longer need drop_locked_tables() and
abort_locked_tables().
- mysql_ha_rm_tables() now always assume that LOCK_open is not
acquired by caller.
- Added notify_thread_having_shared_lock() callback invoked by
metadata locking subsystem when acquiring an exclusive lock, for
each thread that has a conflicting shared metadata lock.
- Introduced expel_table_from_cache() as replacement for
remove_table_from_cache() (the main difference is that this new
function assumes that caller follows metadata locking protocol
and never waits).
- Threw away most of functions related to name locking. One should
use new metadata locking subsystem and API instead.
sql/mysqld.cc:
Got rid of call initializing/deinitializing table cache since now
it is embedded into table definition cache. Added calls for
initializing/ deinitializing metadata locking subsystem.
sql/rpl_rli.cc:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/rpl_rli.h:
Introduced auxiliary Relay_log_info::slave_close_thread_tables()
method which is used for enforcing that we always close tables
open for RBR before deallocating TABLE_LIST elements and MDL_LOCK
objects for them.
sql/set_var.cc:
close_cached_tables() no longer has wait_for_placeholder argument.
Instead of relying on this parameter and related behavior FLUSH
TABLES WITH READ LOCK now takes global shared metadata lock.
sql/sp_head.cc:
For tables added to the statement's table list by prelocking
algorithm we allocate these objects either on the same memory as
corresponding table list elements or on THD::locked_tables_root
(if we are building table list for LOCK TABLES).
sql/sql_acl.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_base.cc:
Changed code to use new MDL subsystem. Got rid of separate table
cache. Now used and unused TABLE instances are linked to the
TABLE_SHAREs in table definition cache.
check_unused():
Adjusted code to the fact that we no longer have separate table
cache. Removed dead code.
table_def_free():
Free TABLE instances referenced from TABLE_SHARE objects before
destroying table definition cache.
get_table_share():
Added assert which ensures that noone will be able to access
table (and its share) without acquiring some kind of metadata
lock first.
close_handle_and_leave_table_as_lock():
Adjusted code to the fact that TABLE instances now are linked to
list in TABLE_SHARE.
list_open_tables():
Changed this function to use table definition cache instead of
table cache.
free_cache_entry():
Unlink freed TABLE elements from the list of all TABLE instances
for the table in TABLE_SHARE.
kill_delayed_thread_for_table():
Added auxiliary for killing delayed insert threads for
particular table.
close_cached_tables():
Got rid of wait_for_refresh argument as we now rely on global
shared metadata lock to prevent FLUSH WITH READ LOCK sneaking in
when we are reopening tables. Heavily reworked this function to
use new MDL code and not to rely on separate table cache entity.
close_open_tables():
We no longer have separate table cache.
close_thread_tables():
Release metadata locks after closing all tables. Added skip_mdl
argument which allows us not to remove metadata lock requests
from the context in case when we are going to use this requests
later in mdl_wait_for_locks() and tdc_wait_for_old_versions().
close_thread_table()/close_table_for_reopen():
Since we no longer have separate table cache and all TABLE
instances are linked to TABLE_SHARE objects in table definition
cache we have to link/unlink TABLE object to/from appropriate
lists in the share.
name_lock_locked_table():
Moved redundant code to find_write_locked_table() function and
adjusted code to the fact that wait_while_table_is_used() can
now return with an error if our thread is killed.
reopen_table_entry():
We no longer need "link_in" argument as with MDL we no longer
call this function with dummy TABLE object pre-allocated and
added to the THD::open_tables. Also now we add newly-open TABLE
instance to the list of share's used TABLE instances.
table_cache_insert_placeholder():
Got rid of name-locking legacy.
lock_table_name_if_not_cached():
Moved to sql_table.cc the only place where it is used. It was
also reimplemented using new MDL API.
open_table():
- Reworked this function to use new MDL subsystem.
- Changed code to deal with table definition cache directly
instead of going through separate table cache.
- Now third argument is also used for requesting table repair
or auto-discovery of table's new definition. So its type was
changed from bool to enum.
find_locked_table()/find_write_locked_table():
Accept head of list of TABLE objects as first argument and use
this list instead of always searching in THD::open_tables list.
Also added auxiliary for finding write-locked locked tables.
reopen_table():
Adjusted function to work with new MDL subsystem and to properly
manuipulate with lists of used/unused TABLE instaces in
TABLE_SHARE.
reopen_tables():
Removed mark_share_as_old parameter. Instead of relying on it
and related behavior FLUSH TABLES WITH READ LOCK now takes
global shared metadata lock. Changed code after removing
separate table cache.
drop_locked_tables()/abort_locked_tables():
Got rid of functions which are no longer needed.
unlock_locked_tables():
Moved this function from sql_parse.cc and changed it to release
memory which was used for allocating metadata lock requests for
tables open and locked by LOCK TABLES.
tdc_open_view():
Intoduced function for opening a view by getting its definition
from disk (and table cache in future).
reopen_table_entry():
Introduced function for opening table definitions while holding
exclusive metatadata lock on it.
open_unireg_entry():
Got rid of this function. Most of its functionality is relocated
to open_table() and open_table_fini() functions, and some of it
to reopen_table_entry() and tdc_open_view(). Also code
resposible for auto-repair and auto-discovery of tables was
moved to separate function.
open_table_entry_fini():
Introduced function which contains common actions which finalize
process of TABLE object creation.
auto_repair_table():
Moved code responsible for auto-repair of table being opened
here.
handle_failed_open_table_attempt()
Moved code responsible for handling failing attempt to open
table to one place (retry due to lock conflict/old version,
auto-discovery and repair).
open_tables():
- Flush open HANDLER tables if they have old version of if there
is conflicting metadata lock against them (before this moment
we had this code in open_table()).
- When we open view which should be processed via derived table
on the second execution of prepared statement or stored
routine we still should call open_table() for it in order to
obtain metadata lock on it and prepare its security context.
- In cases when we discover that some special handling of
failure to open table is needed call
handle_failed_open_table_attempt() which handles all such
scenarios.
open_ltable():
Handling of various special scenarios of failure to open a table
was moved to separate handle_failed_open_table_attempt()
function.
remove_db_from_cache():
Removed this function as it is no longer used.
notify_thread_having_shared_lock():
Added callback which is invoked by MDL subsystem when acquiring
an exclusive lock, for each thread that has a conflicting shared
metadata lock.
expel_table_from_cache():
Introduced function for removing unused TABLE instances. Unlike
remove_table_from_cache() it relies on caller following MDL
protocol and having appropriate locks when calling it and thus
does not do any waiting if table is still in use.
tdc_wait_for_old_version():
Added function which allows open_tables() to wait in cases when
we discover that we should back-off due to presence of old
version of table.
abort_and_upgrade_lock():
Use new MDL calls.
mysql_wait_completed_table():
Got rid of unused function.
open_system_tables_for_read/for_update()/performance_schema_table():
Allocate MDL_LOCK objects on execution memory root in cases when
TABLE_LIST objects for corresponding tables is allocated on
stack.
close_performance_schema_table():
Release metadata locks after closing tables.
******
Use I_P_List for free/used tables list in the table share.
sql/sql_binlog.cc:
Use Relay_log_info::slave_close_thread_tables() method to enforce
that we always close tables open for RBR before deallocating
TABLE_LIST elements and MDL_LOCK objects for them.
sql/sql_class.cc:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
sql/sql_class.h:
Added meta-data locking contexts as part of Open_tables_state
context. Also introduced THD::locked_tables_root memory root
which is to be used for allocating MDL_LOCK objects for tables in
LOCK TABLES statement (end of lifetime for such objects is UNLOCK
TABLES so we can't use statement or execution root for them).
Note: handler_mdl_context and locked_tables_root and
mdl_el_root will be removed by subsequent patches.
sql/sql_db.cc:
mysql_rm_db() does not really need to call remove_db_from_cache()
as it drops each table in the database using
mysql_rm_table_part2(), which performs all necessary operations on
table (definition) cache.
sql/sql_delete.cc:
Use the new metadata locking API for TRUNCATE.
sql/sql_handler.cc:
Changed HANDLER implementation to use new metadata locking
subsystem. Note that MDL_LOCK objects for HANDLER tables are
allocated in the same chunk of heap memory as TABLE_LIST object
for those tables.
sql/sql_insert.cc:
mysql_insert():
find_locked_table() now takes head of list of TABLE object as
its argument instead of always scanning through THD::open_tables
list.
handle_delayed_insert():
Allocate metadata lock request object for table open by delayed
insert thread on execution memroot. create_table_from_items():
We no longer allocate dummy TABLE objects for tables being
created if they don't exist. As consequence
reopen_name_locked_table() no longer has link_in argument.
open_table() now has one more argument which is not relevant for
temporary tables.
sql/sql_parse.cc:
- Moved unlock_locked_tables() routine to sql_base.cc and made
available it in other files. Got rid of some redundant code by
using this function.
- Replaced boolean TABLE_LIST::create member with enum
open_table_type member.
- Use special memory root for allocating MDL_LOCK objects for
tables open and locked by LOCK TABLES (these object should live
till UNLOCK TABLES so we can't allocate them on statement nor
execution memory root). Also properly set metadata lock
upgradability attribure for those tables.
- Under LOCK TABLES it is no longer allowed to flush tables which
are not write-locked as this breaks metadata locking protocol
and thus potentially might lead to deadlock.
- Added auxiliary adjust_mdl_locks_upgradability() function.
sql/sql_partition.cc:
Adjusted code to the fact that reopen_tables() no longer has
"mark_share_as_old" argument. Got rid of comments which are no
longer true.
sql/sql_plist.h:
Added I_P_List template class for parametrized intrusive doubly
linked lists and I_P_List_iterator for corresponding iterator.
Unlike for I_List<> list elements of such list can participate in
several lists. Unlike List<> such lists are doubly-linked and
intrusive.
sql/sql_plugin.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_prepare.cc:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take exclusive
metadata lock on it.
sql/sql_rename.cc:
Use new metadata locking subsystem in implementation of RENAME
TABLE.
sql/sql_servers.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables. Got rid of redundant code by using unlock_locked_tables()
function.
sql/sql_show.cc:
Acquire shared metadata lock when we are getting information for
I_S table directly from TABLE_SHARE without doing full-blown table
open. We use high priority lock request in this situation in
order to avoid deadlocks.
Also allocate metadata lock requests objects (MDL_LOCK) on
execution memory root in cases when TABLE_LIST objects are also
allocated there
sql/sql_table.cc:
mysql_rm_table():
Removed comment which is no longer relevant.
mysql_rm_table_part2():
Now caller of mysql_ha_rm_tables() should not own LOCK_open.
Adjusted code to use new metadata locking subsystem instead of
name-locks.
lock_table_name_if_not_cached():
Moved this function from sql_base.cc to this file and
reimplemented it using metadata locking API.
mysql_create_table():
Adjusted code to use new MDL API.
wait_while_table_is_used():
Changed function to use new MDL subsystem. Made thread waiting
in it killable (this also led to introduction of return value so
caller can distinguish successful executions from situations
when waiting was aborted).
close_cached_tables():
Thread waiting in this function is killable now. As result it
has return value for distinguishing between succes and failure.
Got rid of redundant boradcast_refresh() call.
prepare_for_repair():
Use MDL subsystem instead of name-locks.
mysql_admin_table():
mysql_ha_rm_tables() now always assumes that caller doesn't own
LOCK_open.
mysql_repair_table():
We should mark all elements of table list as requiring
upgradable metadata locks.
mysql_create_table_like():
Use new MDL subsystem instead of name-locks.
create_temporary_tables():
We don't need to obtain metadata locks when creating temporary
table.
mysql_fast_or_online_alter_table():
Thread waiting in wait_while_table_is_used() is now killable.
mysql_alter_table():
Adjusted code to work with new MDL subsystem and to the fact
that threads waiting in what_while_table_is_used() and
close_cached_table() are now killable.
sql/sql_test.cc:
We no longer have separate table cache. TABLE instances are now
associated with/linked to TABLE_SHARE objects in table definition
cache.
sql/sql_trigger.cc:
Adjusted code to work with new metadata locking subsystem. Also
reopen_tables() no longer has mark_share_as_old argument (Instead
of relying on this parameter and related behavior FLUSH TABLES
WITH READ LOCK now takes global shared metadata lock).
sql/sql_udf.cc:
Allocate metadata lock requests objects (MDL_LOCK) on execution
memory root in cases when we use stack TABLE_LIST objects to open
tables.
sql/sql_update.cc:
Adjusted code to work with new meta-data locking subsystem.
sql/sql_view.cc:
Added proper meta-data locking to implementations of
CREATE/ALTER/DROP VIEW statements. Now we obtain exclusive
meta-data lock on a view before creating/ changing/dropping it.
This ensures that all concurrent statements that use this view
will finish before our statement will proceed and therefore we
will get correct order of statements in the binary log.
Also ensure that TABLE_LIST::mdl_upgradable attribute is properly
propagated for underlying tables of view.
sql/table.cc:
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock request objects for all elements of table list.
sql/table.h:
TABLE_SHARE:
Got rid of unused members. Introduced members for storing lists
of used and unused TABLE objects for this share.
TABLE:
Added members for linking TABLE objects into per-share lists of
used and unused TABLE instances. Added member for holding
pointer to metadata lock for this table.
TABLE_LIST:
Replaced boolean TABLE_LIST::create member with enum
open_table_type member. This allows easily handle situation in
which instead of opening the table we want only to take
exclusive meta-data lock on it (we need this in order to handle
ALTER VIEW and CREATE VIEW statements).
Introduced new mdl_upgradable member for marking elements of
table list for which we need to take upgradable shared metadata
lock instead of plain shared metadata lock. Added pointer for
holding pointer to MDL_LOCK for the table.
Added auxiliary alloc_mdl_locks() function for allocating metadata
lock requests objects for all elements of table list. Added
auxiliary set_all_mdl_upgradable() function for marking all
elements in table list as requiring upgradable metadata locks.
storage/myisammrg/ha_myisammrg.cc:
Allocate MDL_LOCK objects for underlying tables of MERGE table.
To be reworked once Ingo pushes his patch for WL4144.
|
