Bug#45261: Crash, stored procedure + decimal

The problem was that creating a DECIMAL column from a decimal
value could lead to a failed assertion as decimal values can
have a higher precision than those attached to a table. The
assert could be triggered by creating a table from a decimal
with a large (> 30) scale. Also, there was a problem in
calculating the number of digits in the integral and fractional
parts if both exceeded the maximum number of digits permitted
by the new decimal type.

The solution is to ensure that truncation procedure is executed
when deducing a DECIMAL column from a decimal value of higher
precision. If the integer part is equal to or bigger than the
maximum precision for the DECIMAL type (65), the integer part
is truncated to fit and the fractional becomes zero. Otherwise,
the fractional part is truncated to fit into the space left
after the integer part is copied.

This patch borrows code and ideas from Martin Hansson's patch.

mysql-test/r/type_newdecimal.result:
  Add test case result for Bug#45261. Also, update test case to
  reflect that an additive operation increases the precision of
  the resulting type by 1.
mysql-test/t/type_newdecimal.test:
  Add test case for Bug#45261
sql/field.cc:
  Added DBUG_ASSERT to ensure object's invariant is maintained.
  Implement method to create a field to hold a decimal value
  from an item.
sql/field.h:
  Explain member variable. Add method to create a new decimal field.
sql/item.cc:
  The precision should only be capped when storing the value
  on a table. Also, this makes it impossible to calculate the
  integer part if Item::decimals (the scale) is larger than the
  precision.
sql/item.h:
  Simplify calculation of integer part.
sql/item_cmpfunc.cc:
  Do not limit the precision. It will be capped later.
sql/item_func.cc:
  Use new method for allocating a new decimal field.
  Add a specialized method for retrieving the precision
  of a user variable item.
sql/item_func.h:
  Add method to return the precision of a user variable.
sql/item_sum.cc:
  Use new method for allocating a new decimal field.
sql/my_decimal.h:
  The integer part could be improperly calculated for a decimal
  with 31 digits in the fractional part.
sql/sql_select.cc:
  Use new method which truncates the integer or decimal parts
  as needed.

1 files changed, 5 insertions, 9 deletions

diff --git a/sql/my_decimal.h b/sql/my_decimal.h
index 21669e82c44..b1df1395dcd 100644
--- a/sql/my_decimal.h
+++ b/sql/my_decimal.h
@@ -48,10 +48,12 @@ C_MODE_END
   digits * number of decimal digits in one our big digit - number of decimal
   digits in one our big digit decreased by 1 (because we always put decimal
   point on the border of our big digits))
+
+  This value is 65 due to historical reasons partly due to it being used
+  as the maximum allowed precision and not the actual maximum precision.
 */
 #define DECIMAL_MAX_PRECISION (DECIMAL_MAX_POSSIBLE_PRECISION - 8*2)
 #define DECIMAL_MAX_SCALE 30
-#define DECIMAL_NOT_SPECIFIED 31
 
 /**
   maximum length of string representation (number of maximum decimal
@@ -75,12 +77,6 @@ inline uint my_decimal_size(uint precision, uint scale)
 }
 
 
-inline int my_decimal_int_part(uint precision, uint decimals)
-{
-  return precision - ((decimals == DECIMAL_NOT_SPECIFIED) ? 0 : decimals);
-}
-
-
 /**
   my_decimal class limits 'decimal_t' type to what we need in MySQL.
 
@@ -184,7 +180,7 @@ inline uint my_decimal_length_to_precision(uint length, uint scale,
 }
 
 inline uint32 my_decimal_precision_to_length_no_truncation(uint precision,
-                                                           uint8 scale,
+                                                           uint scale,
                                                            bool unsigned_flag)
 {
   /*
@@ -196,7 +192,7 @@ inline uint32 my_decimal_precision_to_length_no_truncation(uint precision,
                   (unsigned_flag || !precision ? 0 : 1));
 }
 
-inline uint32 my_decimal_precision_to_length(uint precision, uint8 scale,
+inline uint32 my_decimal_precision_to_length(uint precision, uint scale,
                                              bool unsigned_flag)
 {
   /*