| Commit message (Collapse) | Author | Age | Files | Lines |
|\ |
|
| |
| |
| |
| | |
Added schema info to wsrep messages above debug level.
|
| |
| |
| |
| | |
sessions to cover all exclusive MDL locks
|
| |\
| | |
| | |
| | |
| | | |
bzr merge -r4588 maria/10.0
|
| |\ \ |
|
| |\ \ \
| | | | |
| | | | |
| | | | |
| | | | | |
* bzr merge -rtag:mariadb-10.0.10 maria/10.0.
|
| |\ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
* Fix for post-merge build failures.
|
| |\ \ \ \ \ |
|
| |\ \ \ \ \ \
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
and a number of fixes to make this buildable.
Run also few short multi-master high conflict rate tests, with no issues
|
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
This is now otherwise on level wsrep-25.9, but storage/innobase has not been fully merged
wsrep-5.5 is not good source for that, so we probably have to cherry pick innodb changes from wsrep-5.6
|
| | | | | | | | |
|
| |\ \ \ \ \ \ \
| | |/ / / / / / |
|
| | | | | | | | |
|
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
can compile with -Wshadow and get much fewer warnings)
- Changed ER(ER_...) to ER_THD(thd, ER_...) when thd was known or if there was many calls to current_thd in the same function.
- Changed ER(ER_..) to ER_THD_OR_DEFAULT(current_thd, ER...) in some places where current_thd is not necessary defined.
- Removing calls to current_thd when we have access to thd
Part of this is optimization (not calling current_thd when not needed),
but part is bug fixing for error condition when current_thd is not defined
(For example on startup and end of mysqld)
Notable renames done as otherwise a lot of functions would have to be changed:
- In JOIN structure renamed:
examined_rows -> join_examined_rows
record_count -> join_record_count
- In Field, renamed new_field() to make_new_field()
Other things:
- Added DBUG_ASSERT(thd == tmp_thd) in Item_singlerow_subselect() just to be safe.
- Removed old 'tab' prefix in JOIN_TAB::save_explain_data() and use members directly
- Added 'thd' as argument to a few functions to avoid calling current_thd.
|
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Replaced hash with lock-free hash.
|
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Removed MDL map partitions. Won't be needed when this MDEV is implemented.
|
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Removed MDL objects cache. Won't be needed when this MDEV is implemented.
|
|\ \ \ \ \ \ \ \
| | |_|_|_|_|_|/
| |/| | | | | | |
|
| | |_|_|_|_|/
| |/| | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
main.trigger_notembedded fail in buildbot
user locks are not affected by lowed_case_table_names
|
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
as CHARSET_INFO is already const, using const on it
is redundant and results in compiler warnings (on Windows)
|
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Updated MDL_context's lock methods to accept double precision
lock_wait_timeout.
|
| | | | | | | |
|
|/ / / / / /
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Merged lp:maria/maria-10.0-galera up to revision 3879.
Added a new functions to handler API to forcefully abort_transaction,
producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These
were added for future possiblity to add more storage engines that
could use galera replication.
|
| |_|_|_|/
|/| | | |
| | | | |
| | | | | |
various places in the code.
|
| |_|_|/
|/| | |
| | | |
| | | |
| | | |
| | | | |
Let TABLE_SHARE::tdc.free_tables, TABLE_SHARE::tdc.all_tables,
TABLE_SHARE::tdc.flushed and corresponding invariants be protected by
per-share TABLE_SHARE::tdc.LOCK_table_share instead of global LOCK_open.
|
| |_|/
|/| |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Reduced number of my_hash_sort_bin() calls from 4 to 1 per query.
Reduced number of memory accesses done by my_hash_sort_bin().
Details:
- let MDL subsystem use pre-calculated hash value for hash
inserts and deletes
- let table cache use pre-calculated MDL hash value
- MDL namespace is excluded from hash value calculation, so that
hash value can be used by table cache as is
- hash value for MDL is calculated as resulting hash value + MDL
namespace
- extended hash implementation to accept user defined hash function
|
| | |
| | |
| | |
| | |
| | | |
- restore find_ticket() private status, declare friend function instead
- added metadata_lock_info to DEFAULT_SUITES
|
| |/
|/| |
|
|\ \
| |/
|/|
| |
| |
| |
| |
| | |
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- mdl.cc and mdl.h merged completely
- mysql_system_tables*.sql merged completely
- Fixed wrong merge of lock_tables
- Added some missing functions:
- bool THD::notify_shared_lock()
- Dynamic_array::pop, Dynamic_array::del
- Added MDL_context_owner to THD
- Added metadata_locks_hash_instances
|
| | |
|
| | |
|
|\ \
| |/
|/| |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The patch contributed by Konstantin Osipov applied.
Native comments:
Implement multiple user-level locks per connection.
GET_LOCK() function in MySQL allows a connection to hold at most
one user level lock. Taking a new lock automatically releases the
old lock, if any.
The limit of one lock per session existed since early versions
of MySQL didn't have a deadlock detector for SQL locks.
MDL patches in MySQL 5.5 added a deadlock detector,
so starting from 5.5 it became possible to take multiple locks
in any order -- a deadlock, should it occur, would be detected
and an error returned to the client which closed the wait chain.
This is exactly what is done in this patch: ULLs are moved
to use MDL subsystem.
|
| |\ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
ROBUST AGAINST BUGS IN CALLERS".
Both MDL subsystems and Table Definition Cache code assume
that callers ensure that names of objects passed to them are
not longer than NAME_LEN bytes. Unfortunately due to bugs in
callers this assumption might be broken in some cases. As
result we get nasty bugs causing buffer overruns when we
construct MDL key or TDC key from object names.
This patch makes MDL and TDC code more robust against such
bugs by ensuring that we always checking size of result
buffer when constructing MDL and TDC keys. This doesn't
free its callers from ensuring that both db and table names
are shorter than NAME_LEN bytes. But at least these steps
prevents buffer overruns in case of bug in caller, replacing
them with less harmful behavior.
This is 5.5-only version of patch.
Changed code of MDL_key::mdl_key_init() to take into account
size of buffer for the key.
Introduced new version of create_table_def_key() helper function
which constructs TDC key without risk of result buffer overrun.
Places in code that construct TDC keys were changed to use this
function.
Also changed rm_temporary_table() and open_new_frm() functions
to avoid use of "unsafe" strmov() and strxmov() functions and
use safer strnxmov() instead.
|
| | |
| | |
| | |
| | |
| | | |
Post-push cleanup: removal of unneeded assertions.
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Code in MDL subsystem assumes that identifiers of objects can't
be longer than NAME_LEN characters. This assumption was broken
when one tried to construct MDL_key based on table alias, which
can have arbitrary length. Since MDL_key's (and MDL locks) are
not really used for table aliases this patch changes code to
not initialize MDL_key object for table list element representing
aliases.
|
|\ \ \
| |/ / |
|
| |\ \
| | |/ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
FOREVER MDL LOCK
Analysis:
----------
While granting MDL lock for the lock requests in wait queue,
first the lock is granted to the high priority lock types
and then to the low priority lock types.
MDL Priority Matrix,
+-------------+----+---+---+---+----+-----+
| Locks | | | | | | |
| has Priority| | | | | | |
| over ---> | S | SR| SW| SU| SNW| SNRW|
+-------------+----+---+---+---+----+-----+
| X | + | + | + | + | + | + |
+-------------|----|---|---|---|----|-----|
| SNRW | - | + | + | - | - | - |
+-------------|----|---|---|---|----|-----|
| SNW | - | - | + | - | - | - |
+-------------+----+---+---+---+----+-----+
Here '+' means, Lock priority is higher.
'-' means, Has same priority
In the scenario where,
*. Lock wait queue has requests of type S/SR/SW/SU.
*. And locks of high priority X/SNRW/SNW are requested
continuously.
In this case, while granting lock, always first high priority
lock requests(X/SNRW/SNW) are considered. Low priority
locks(S/SR/SW/SU) will not get chance and they will
wait forever.
In the scenario for which this bug is reported, application
executed many LOCK TABLES ... WRITE statements concurrently.
These statements request SNRW lock. Also there were some
connections trying to execute DML statements requesting SR
lock. Since SNRW lock request has higher priority (and as
they were too many waiting SNRW requests) lock is always
granted to it. So, lock request SR will wait forever, resulting
in DML starvation.
How is this handled in 5.1?
---------------------------
Even in 5.1 we have low priority lock starvation issue.
But, in 5.1 thread locking, system variable
"max_write_lock_count" can be configured to grant
some pending read lock requests. After
"max_write_lock_count" of write lock grants all the low
priority locks are granted.
Why this issue is seen in 5.5/trunk?
---------------------------------
In 5.5/trunk MDL locking, "max_write_lock_count" system
variable exists but not used in MDL, only thread lock uses
it. So no effect of "max_write_lock_count" in MDL locking.
This means that starvation of metadata locks is possible
even if max_write_lock_count is used.
Looks like, customer was using "max_write_lock_count" in
5.1 and when upgraded to 5.5, starvation is seen because
of not having effect of "max_write_lock_count" in MDL.
Fix:
----------
As a fix, support for max_write_lock_count is added to MDL.
To maintain write lock counter per MDL_lock object, new
member "m_hog_lock_count" is added in MDL_lock.
And following logic is added to increment the counter in
function reschedule_waiters,
(reschedule_waiters function is called while thread is
releasing the lock)
- After granting lock request from the wait queue.
- Check if there are any S/SR/SU/SW exists in the wait queue
- If yes then increment the "m_hog_lock_count"
And following logic is added in the same function to
handle pending S/SU/SR/SW locks
- Before granting locks
- Check if max_write_lock_count <= m_hog_lock_count
- If Yes, then try to grant S/SR/SW/SU locks.
(Since all of these has same priority, all locks are
granted together. But some lock grant may fail because
of grant incompatibility)
- Reset m_hog_lock_count if there no low priority lock
requests in wait queue.
- return
Note:
--------------------------
In the lock priority matrix explained above,
though X has priority over the SNW and SNRW. X locks is
taken mostly for RENAME, TRUNCATE, CREATE ... operations.
So lock type X may not be requested in loop continuously
in real world applications, as compared to other lock
request types. So, lock request of type SNW and SNRW are
not starved. So, we can grant all S/SR/SU/SW in one shot,
without considering SNW & SNRW lock request starvation.
ALTER table operations take SU lock first and then
upgrade to SNW if required. All S, SR, SW, SU have same
lock priority. So while granting SU, request of types
SR, SW, S are also granted in one shot. So, lock request
of type SU->SNW in loop will not make other low priority
lock request to starve.
But, when there is request for lock of type SNRW, lock
requests of lower priority types are not granted. And if
SNRW is requested in loop continuously then all
S, SR, SW, SU are starved.
This patch addresses the latter scenario.
When we have S/SR/SW/SU in wait queue and if
there are
- Continuous SNRW lock requests
- OR one or more X and Continuous SNRW lock requests.
- OR one SNW and Continuous SNRW lock requests.
- OR one SNW, one or more X and continuous SNRW lock
requests.
in wait queue then, S/SR/SW/SU lock request are starved.
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
and small collateral changes
mysql-test/lib/My/Test.pm:
somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
this differs, because we don't rewrite general log queries, and multi-statement
packets are logged as a one entry. this result file is identical to what mysql-5.6.5
produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
|
|\ \
| |/ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
BY CACHING OR REDUCING CREATEEVENT CALLS".
5.5 versions of MySQL server performed worse than 5.1 versions
under single-connection workload in autocommit mode on Windows XP.
Part of this slowdown can be attributed to overhead associated
with constant creation/destruction of MDL_lock objects in the MDL
subsystem. The problem is that creation/destruction of these
objects causes creation and destruction of associated
synchronization primitives, which are expensive on Windows XP.
This patch tries to alleviate this problem by introducing a cache
of unused MDL_object_lock objects. Instead of destroying such
objects we put them into the cache and then reuse with a new
key when creation of a new object is requested.
To limit the size of this cache, a new --metadata-locks-cache-size
start-up parameter was introduced.
mysql-test/r/mysqld--help-notwin.result:
Updated test after adding --metadata-locks-cache-size
parameter.
mysql-test/r/mysqld--help-win.result:
Updated test after adding --metadata-locks-cache-size
parameter.
mysql-test/suite/sys_vars/r/metadata_locks_cache_size_basic.result:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic-master.opt:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic.test:
Added test coverage for newly introduced --metadata_locks_cache_size
start-up parameter and corresponding global read-only variable.
sql/mdl.cc:
Introduced caching of unused MDL_object_lock objects, in order to
avoid costs associated with constant creation and destruction of
such objects in single-connection workloads run in autocommit mode.
Such costs can be pretty high on systems where creation and
destruction of synchronization primitives require a system call
(e.g. Windows XP).
To implement this cache,a list of unused MDL_object_lock instances
was added to MDL_map object. Instead of being destroyed
MDL_object_lock instances are put into this list and re-used later
when creation of a new instance is required. Also added
MDL_lock::m_version counter to allow threads having outstanding
references to an MDL_object_lock instance to notice that it has
been moved to the unused objects list.
Added a global variable for a start-up parameter that limits
the size of the unused objects list.
Note that we don't cache MDL_scoped_lock objects since they
are supposed to be created only during execution of DDL
statements and therefore should not affect performance much.
sql/mdl.h:
Added a global variable for start-up parameter that limits the
size of the unused MDL_object_lock objects list and constant
for its default value.
sql/sql_plist.h:
Added I_P_List<>::pop_front() function.
sql/sys_vars.cc:
Introduced --metadata-locks-cache-size start-up parameter
for specifying size of the cache of unused MDL_object_lock
objects.
|
|\ \
| |/ |
|
| |\ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
GRADUALLY IF A TRIGGER EXISTS".
This bug manifested itself in two ways:
- Firstly execution of any data-changing statement which
required prelocking (i.e. involved stored function or
trigger) as part of transaction slowed down a bit all
subsequent statements in this transaction. So performance
in transaction which periodically involved such statements
gradually degraded over time.
- Secondly execution of any data-changing statement which
required prelocking as part of transaction prevented
concurrent FLUSH TABLES WITH READ LOCK from proceeding
until the end of transaction instead of end of particular
statement.
The problem was caused by incorrect handling of metadata lock
used in FTWRL implementation for statements requiring prelocked
mode.
Each statement which changes data acquires global IX lock
with STATEMENT duration. This lock is supposed to block
concurrent FTWRL from proceeding until the statement ends.
When entering prelocked mode, durations of all metadata locks
acquired so far were changed to EXPLICIT, to prevent
substatements from releasing these locks. When prelocked mode
was left, durations of metadata locks were changed to
TRANSACTIONAL (with a few exceptions) so they can be properly
released at the end of transaction.
Unfortunately, this meant that the global IX lock blocking
FTWRL with STATEMENT duration was moved to TRANSACTIONAL
duration after execution of statement requiring prelocking.
Since each subsequent statement that required prelocking and
tried to acquire global IX lock with STATEMENT duration got
a new instance of MDL_ticket, which was later moved to
TRANSACTIONAL duration, this led to unwarranted growth of
number of tickets with TRANSACITONAL duration in this
connection's MDL_context. As result searching for other
tickets in it became slow and acquisition of other metadata
locks by this transaction started to hog CPU.
Moreover, this also meant that after execution of statement
requiring prelocking concurrent FTWRL was blocked
until the end of transaction instead of end of statement.
This patch solves this problem by not moving locks to EXPLICIT
duration when thread enters prelocked mode (unless it is a real
LOCK TABLES mode). This step turned out to be not really
necessary as substatements don't try to release metadata locks.
Consequently, the global IX lock blocking FTWRL keeps its
STATEMENT duration and is properly released at the end of
statement and the above issue goes away.
mysql-test/r/flush.result:
Added test for bug #12641342 - "61401: UPDATE PERFORMANCE
DEGRADES GRADUALLY IF A TRIGGER EXISTS".
mysql-test/t/flush.test:
Added test for bug #12641342 - "61401: UPDATE PERFORMANCE
DEGRADES GRADUALLY IF A TRIGGER EXISTS".
sql/mdl.h:
Added comment describing various types of metadata lock
duration.
sql/sql_class.cc:
Since we no longer change duration of metadata locks to EXPLICIT
when entering prelocked mode (unless it is a real LOCK TABLES)
there is no need to restore proper duration of the locks when
leaving prelocked mode.
sql/sql_class.h:
Do not change duration of metadata locks to EXPLICIT when
entering prelocking mode (unless it is a real LOCK TABLES).
This allows to avoid problems with restoring correct duration
when leaving this mode. It is possible to do this as
substatements won't release metadata locks in any case.
sql/sql_parse.cc:
Added assert checking that we won't release metadata locks
when in substatement.
|
| |/ |
|
|\ \
| |/ |
|
| |
| |
| |
| |
| | |
for code other then in mdl.h. Since the adapter is generic, it is
better located in sql_plist.h.
|