diff options
| author | Varun Gupta <varun.gupta@mariadb.com> | 2020-07-22 14:44:25 +0530 |
|---|---|---|
| committer | Varun Gupta <varun.gupta@mariadb.com> | 2020-07-22 14:44:25 +0530 |
| commit | 62d73df6b270cc94ba577e96d3bf325170f306fe (patch) | |
| tree | d2071cbebbd674ac42b00816a139eb2c1a97c661 /mysql-test/suite/sys_vars | |
| parent | 57ec42bc321dee796ce8e711a4499cd665513009 (diff) | |
| download | mariadb-git-62d73df6b270cc94ba577e96d3bf325170f306fe.tar.gz | |
MDEV-19232: Floating point precision / value comparison problem
The issue occurs when the subquery_cache is enabled.
When there is a cache miss the division was leading to a value with scale 9.
In the case of cache hit the value returned was of scale 9 and due to the different
values for the scales the where condition evaluated to FALSE, hence the output
was incomplete.
To fix this problem we need to round up the decimal to the limit mentioned in
Item::decimals. This would make sure the values are compared with the same
scale.
Diffstat (limited to 'mysql-test/suite/sys_vars')
| -rw-r--r-- | mysql-test/suite/sys_vars/r/div_precision_increment_func.result | 16 |
1 files changed, 8 insertions, 8 deletions
diff --git a/mysql-test/suite/sys_vars/r/div_precision_increment_func.result b/mysql-test/suite/sys_vars/r/div_precision_increment_func.result index ee8b7c5691d..ffe23eb3cef 100644 --- a/mysql-test/suite/sys_vars/r/div_precision_increment_func.result +++ b/mysql-test/suite/sys_vars/r/div_precision_increment_func.result @@ -50,9 +50,9 @@ INSERT into t1(name, salary, income_tax) values('Record_2', 501, 501*2.5/1000); INSERT into t1(name, salary, income_tax) values('Record_3', 210, 210*2.5/1000); SELECT * from t1; id name salary income_tax -1 Record_1 100011 250.027 -2 Record_2 501 1.2525 -3 Record_3 210 0.525 +1 Record_1 100011 250.03 +2 Record_2 501 1.25 +3 Record_3 210 0.53 ## Creating new connection ## ## Verifying session & global value of variable ## SELECT @@global.div_precision_increment = 2; @@ -67,11 +67,11 @@ INSERT into t1(name, salary, income_tax) values('Record_5', 501, 501*2.5/1000); INSERT into t1(name, salary, income_tax) values('Record_6', 210, 210*2.5/1000); SELECT * from t1; id name salary income_tax -1 Record_1 100011 250.027 -2 Record_2 501 1.2525 -3 Record_3 210 0.525 -4 Record_4 100011 250.027 -5 Record_5 501 1.2525 +1 Record_1 100011 250.03 +2 Record_2 501 1.25 +3 Record_3 210 0.53 +4 Record_4 100011 250.028 +5 Record_5 501 1.253 6 Record_6 210 0.525 ## Dropping table t1 ## drop table t1; |