summaryrefslogtreecommitdiff
path: root/Docs
diff options
context:
space:
mode:
authorunknown <jani@hynda.mysql.fi>2001-12-10 15:07:40 +0200
committerunknown <jani@hynda.mysql.fi>2001-12-10 15:07:40 +0200
commit718825a9b0b9517201b9109c4f88cfaa6b2f20a6 (patch)
treea107de5feb5a32ed6dba1c9beb8b4330f5921dfd /Docs
parentd71c2b60eeb9afa855c8dc2411dbc58e9a9b1083 (diff)
downloadmariadb-git-718825a9b0b9517201b9109c4f88cfaa6b2f20a6.tar.gz
Added information about floating point numbers.
Diffstat (limited to 'Docs')
-rw-r--r--Docs/manual.texi139
1 files changed, 128 insertions, 11 deletions
diff --git a/Docs/manual.texi b/Docs/manual.texi
index ae2f1248458..ffa5bdfaafb 100644
--- a/Docs/manual.texi
+++ b/Docs/manual.texi
@@ -44369,6 +44369,7 @@ the server runs, for example, in @code{safe_mysqld} or @code{mysql.server}.
* Problems with alias:: Problems with @code{alias}
* Deleting from related tables:: Deleting Rows from Related Tables
* No matching rows:: Solving Problems with No Matching Rows
+* Problems with float:: Problems with floating point comparison
@end menu
@node Case sensitivity, Using DATE, Query Issues, Query Issues
@@ -44611,7 +44612,7 @@ id's per query if the @code{related_column} is an index. If the
number of arguments in the @code{IN} clause.
-@node No matching rows, , Deleting from related tables, Query Issues
+@node No matching rows, Problems with float, Deleting from related tables, Query Issues
@appendixsubsec Solving Problems with No Matching Rows
@cindex no matching rows
@@ -44639,16 +44640,10 @@ the table that was last removed from the query.
@item
If you are comparing @code{FLOAT} or @code{DOUBLE} columns with numbers that
-have decimals, you can't use @code{=}! This problem is common in most
-computer languages because floating-point values are not exact values:
-
-@example
-mysql> SELECT * FROM table_name WHERE float_column=3.5;
- ->
-mysql> SELECT * FROM table_name WHERE float_column between 3.45 and 3.55;
-@end example
-
-In most cases, changing the @code{FLOAT} to a @code{DOUBLE} will fix this!
+have decimals, you can't use @code{'='}. This problem is common in most
+computer languages because floating-point values are not exact values.
+In most cases, changing the @code{FLOAT} to a @code{DOUBLE} will fix this.
+@xref{Problems with float}.
@item
If you still can't figure out what's wrong, create a minimal test that can
@@ -44666,6 +44661,128 @@ shell> mysql test2 < query.sql
Post the test file using @code{mysqlbug} to @email{mysql@@lists.mysql.com}.
@end enumerate
+@node Problems with float, , No matching rows, Query Issues
+@appendixsubsec Problems with floating point comparison
+
+Floating point numbers cause confusion sometimes, because these numbers
+are not stored as exact values inside computer architecture. What one
+can see on the screen usually is not the exact value of the number.
+
+Field types @code{FLOAT} and @code{DECIMAL} are such.
+
+@example
+CREATE TABLE t1 (i int, d1 decimal(9,2), d2 decimal(9,2));
+INSERT INTO t1 values (1, 101.40, 21.40), (1, -80.00, 0.00), (2, 0.00, 0.00),
+(2, -13.20, 0.00), (2, 59.60, 46.40), (2, 30.40, 30.40), (3, 37.00, 7.40),
+(3, -29.60, 0.00), (4, 60.00, 15.40), (4, -10.60, 0.00), (4, -34.00, 0.00),
+(5, 33.00, 0.00), (5, -25.80, 0.00), (5, 0.00, 7.20), (6, 0.00, 0.00),
+(6, -51.40, 0.00);
+
+mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b FROM t1 GROUP BY i HAVING a <> b;
++------+--------+-------+
+| i | a | b |
++------+--------+-------+
+| 1 | 21.40 | 21.40 |
+| 2 | 76.80 | 76.80 |
+| 3 | 7.40 | 7.40 |
+| 4 | 15.40 | 15.40 |
+| 5 | 7.20 | 7.20 |
+| 6 | -51.40 | 0.00 |
++------+--------+-------+
+@end example
+
+The result is correct. Although the first five records look like they
+shouldn't pass the comparison test, they may do so because the
+difference between the numbers show up around tenth decimal, or so
+depending on computer architecture.
+
+The problem cannot be solved by using ROUND() (or similar function),
+because the result is still a floating point number. Example:
+
+@example
+mysql> SELECT i, ROUND(SUM(d1), 2) AS a, ROUND(SUM(d2), 2) AS b FROM t1 GROUP BY i HAVING a <> b;
++------+--------+-------+
+| i | a | b |
++------+--------+-------+
+| 1 | 21.40 | 21.40 |
+| 2 | 76.80 | 76.80 |
+| 3 | 7.40 | 7.40 |
+| 4 | 15.40 | 15.40 |
+| 5 | 7.20 | 7.20 |
+| 6 | -51.40 | 0.00 |
++------+--------+-------+
+@end example
+
+This is what the numbers in row 'a' look like:
+@example
+mysql> SELECT i, ROUND(SUM(d1), 2)*1.0000000000000000 AS a, ROUND(SUM(d2), 2) AS b FROM t1 GROUP BY i HAVING a <> b;
++------+----------------------+-------+
+| i | a | b |
++------+----------------------+-------+
+| 1 | 21.3999999999999986 | 21.40 |
+| 2 | 76.7999999999999972 | 76.80 |
+| 3 | 7.4000000000000004 | 7.40 |
+| 4 | 15.4000000000000004 | 15.40 |
+| 5 | 7.2000000000000002 | 7.20 |
+| 6 | -51.3999999999999986 | 0.00 |
++------+----------------------+-------+
+@end example
+
+Depending on the computer architecture you may or may not see similar results.
+Each CPU may evaluate floating point numbers differently. For example in
+some machines you may get 'right' results by multiplaying both arguments
+with 1, an example follows.
+
+@strong{WARNING: NEVER TRUST THIS METHOD IN YOUR APPLICATION, THIS IS
+AN EXAMPLE OF A WRONG METHOD!!!}
+
+@example
+mysql> SELECT i, ROUND(SUM(d1), 2)*1 AS a, ROUND(SUM(d2), 2)*1 AS b FROM t1 GROUP
+BY i HAVING a <> b;
++------+--------+------+
+| i | a | b |
++------+--------+------+
+| 6 | -51.40 | 0.00 |
++------+--------+------+
+@end example
+
+The reason why the above example seems to be working is that on the
+particular machine where the test was done, the CPU floating point
+arithmetics happens to round the numbers to same, but there is no
+rule that any CPU should do so, so it cannot be trusted.
+
+The correct way to do floating point number comparison is to first
+decide on what is the wanted tolerance between the numbers and then do
+the comparsion against the tolerance number. For example, if we agree on
+that floating point numbers should be regarded the same, if they are
+same with precision of one of ten thousand (0.0001), the comparsion
+should be done like this:
+
+@example
+mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b FROM t1 GROUP BY i HAVING ABS(a - b) > 0.0001;
++------+--------+------+
+| i | a | b |
++------+--------+------+
+| 6 | -51.40 | 0.00 |
++------+--------+------+
+1 row in set (0.00 sec)
+@end example
+
+And vice versa, if we wanted to get rows where the numbers are the same,
+the test would be:
+
+@example
+mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b FROM t1 GROUP BY i HAVING ABS(a - b) < 0.0001;
++------+-------+-------+
+| i | a | b |
++------+-------+-------+
+| 1 | 21.40 | 21.40 |
+| 2 | 76.80 | 76.80 |
+| 3 | 7.40 | 7.40 |
+| 4 | 15.40 | 15.40 |
+| 5 | 7.20 | 7.20 |
++------+-------+-------+
+@end example
@node Table Definition Issues, , Query Issues, Problems
@appendixsec Table Definition Related Issues