diff options
| author | Varun Gupta <varun.gupta@mariadb.com> | 2020-10-29 21:38:06 +0530 |
|---|---|---|
| committer | Varun Gupta <varun.gupta@mariadb.com> | 2020-10-29 22:19:51 +0530 |
| commit | 7ed231d13e86db11982d3d78c9c9681a88107a5b (patch) | |
| tree | d854f82fde5c34bec95f2fe848bba79e5c253671 /mysql-test/t/order_by.test | |
| parent | 2e5450af052040848042c6eae4f03efa23c5f8fc (diff) | |
| download | mariadb-git-10.2-mdev24033.tar.gz | |
MDEV-24033: SIGSEGV in __memcmp_avx2_movbe from queue_insert | SIGSEGV in __memcmp_avx2_movbe from native_compare10.2-mdev24033
The issue here was the system variable max_sort_length was being applied
to decimals and it was truncating the value for decimals to the number
of bytes set by max_sort_length.
This was leading to a buffer overflow as the values were written
to the buffer without truncation and then we moved the offset to
the number of bytes(set by max_sort_length), that are needed for comparison.
The fix is to not apply max_sort_length for fixed size types like INT,
DECIMALS and only apply max_sort_length for CHAR, VARCHARS, TEXT and
BLOBS.
Diffstat (limited to 'mysql-test/t/order_by.test')
| -rw-r--r-- | mysql-test/t/order_by.test | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/mysql-test/t/order_by.test b/mysql-test/t/order_by.test index f6886d6d45c..e9802f95721 100644 --- a/mysql-test/t/order_by.test +++ b/mysql-test/t/order_by.test @@ -2272,4 +2272,25 @@ ORDER BY id+1 DESC; DROP TABLE t1; +--echo # +--echo # MDEV-24033: SIGSEGV in __memcmp_avx2_movbe from queue_insert | SIGSEGV in __memcmp_avx2_movbe from native_compare +--echo # + +SET @save_max_length_for_sort_data=@@max_length_for_sort_data; +SET @save_max_sort_length= @@max_sort_length; +SET @save_sql_select_limit= @@sql_select_limit; + +CREATE TABLE t1 (a DECIMAL(64,0), b INT); +INSERT INTO t1 VALUES (1,1), (2,2), (3,3), (4,4); + +SET max_length_for_sort_data= 30; +SET sql_select_limit = 3; +SET max_sort_length=8; +SELECT * FROM t1 ORDER BY a+1; + +SET max_length_for_sort_data=@save_max_length_for_sort_data; +SET max_sort_length= @save_max_sort_length; +SET sql_select_limit= @save_sql_select_limit; +DROP TABLE t1; + --echo # End of 10.2 tests |