Bug#45261: Crash, stored procedure + decimal

The problem was that creating a DECIMAL column from a decimal
value could lead to a failed assertion as decimal values can
have a higher precision than those attached to a table. The
assert could be triggered by creating a table from a decimal
with a large (> 30) scale. Also, there was a problem in
calculating the number of digits in the integral and fractional
parts if both exceeded the maximum number of digits permitted
by the new decimal type.

The solution is to ensure that truncation procedure is executed
when deducing a DECIMAL column from a decimal value of higher
precision. If the integer part is equal to or bigger than the
maximum precision for the DECIMAL type (65), the integer part
is truncated to fit and the fractional becomes zero. Otherwise,
the fractional part is truncated to fit into the space left
after the integer part is copied.

This patch borrows code and ideas from Martin Hansson's patch.

mysql-test/r/type_newdecimal.result:
  Add test case result for Bug#45261. Also, update test case to
  reflect that an additive operation increases the precision of
  the resulting type by 1.
mysql-test/t/type_newdecimal.test:
  Add test case for Bug#45261
sql/field.cc:
  Added DBUG_ASSERT to ensure object's invariant is maintained.
  Implement method to create a field to hold a decimal value
  from an item.
sql/field.h:
  Explain member variable. Add method to create a new decimal field.
sql/item.cc:
  The precision should only be capped when storing the value
  on a table. Also, this makes it impossible to calculate the
  integer part if Item::decimals (the scale) is larger than the
  precision.
sql/item.h:
  Simplify calculation of integer part.
sql/item_cmpfunc.cc:
  Do not limit the precision. It will be capped later.
sql/item_func.cc:
  Use new method for allocating a new decimal field.
  Add a specialized method for retrieving the precision
  of a user variable item.
sql/item_func.h:
  Add method to return the precision of a user variable.
sql/item_sum.cc:
  Use new method for allocating a new decimal field.
sql/my_decimal.h:
  The integer part could be improperly calculated for a decimal
  with 31 digits in the fractional part.
sql/sql_select.cc:
  Use new method which truncates the integer or decimal parts
  as needed.

1 files changed, 85 insertions, 0 deletions

diff --git a/sql/field.cc b/sql/field.cc
index 3bfb54fbd15..426effa57cd 100644
--- a/sql/field.cc
+++ b/sql/field.cc
@@ -2485,12 +2485,97 @@ Field_new_decimal::Field_new_decimal(uint32 len_arg,
 {
   precision= my_decimal_length_to_precision(len_arg, dec_arg, unsigned_arg);
   set_if_smaller(precision, DECIMAL_MAX_PRECISION);
+  DBUG_ASSERT(precision >= dec);
   DBUG_ASSERT((precision <= DECIMAL_MAX_PRECISION) &&
               (dec <= DECIMAL_MAX_SCALE));
   bin_size= my_decimal_get_binary_size(precision, dec);
 }
 
 
+/**
+  Create a field to hold a decimal value from an item.
+
+  @remark The MySQL DECIMAL data type has a characteristic that needs to be
+          taken into account when deducing the type from a Item_decimal.
+
+  But first, let's briefly recap what is the new MySQL DECIMAL type:
+
+  The declaration syntax for a decimal is DECIMAL(M,D), where:
+
+  * M is the maximum number of digits (the precision).
+    It has a range of 1 to 65.
+  * D is the number of digits to the right of the decimal separator (the scale).
+    It has a range of 0 to 30 and must be no larger than M.
+
+  D and M are used to determine the storage requirements for the integer
+  and fractional parts of each value. The integer part is to the left of
+  the decimal separator and to the right is the fractional part. Hence:
+
+  M is the number of digits for the integer and fractional part.
+  D is the number of digits for the fractional part.
+
+  Consequently, M - D is the number of digits for the integer part. For
+  example, a DECIMAL(20,10) column has ten digits on either side of
+  the decimal separator.
+
+  The characteristic that needs to be taken into account is that the
+  backing type for Item_decimal is a my_decimal that has a higher
+  precision (DECIMAL_MAX_POSSIBLE_PRECISION, see my_decimal.h) than
+  DECIMAL.
+
+  Drawing a comparison between my_decimal and DECIMAL:
+
+  * M has a range of 1 to 81.
+  * D has a range of 0 to 81.
+
+  There can be a difference in range if the decimal contains a integer
+  part. This is because the fractional part must always be on a group
+  boundary, leaving at least one group for the integer part. Since each
+  group is 9 (DIG_PER_DEC1) digits and there are 9 (DECIMAL_BUFF_LENGTH)
+  groups, the fractional part is limited to 72 digits if there is at
+  least one digit in the integral part.
+
+  Although the backing type for a DECIMAL is also my_decimal, every
+  time a my_decimal is stored in a DECIMAL field, the precision and
+  scale are explicitly capped at 65 (DECIMAL_MAX_PRECISION) and 30
+  (DECIMAL_MAX_SCALE) digits, following my_decimal truncation procedure
+  (FIX_INTG_FRAC_ERROR).
+*/
+
+Field_new_decimal *
+Field_new_decimal::new_decimal_field(const Item *item)
+{
+  uint32 len;
+  uint intg= item->decimal_int_part(), scale= item->decimals;
+
+  DBUG_ASSERT(item->decimal_precision() >= item->decimals);
+
+  /*
+    Employ a procedure along the lines of the my_decimal truncation process:
+    - If the integer part is equal to or bigger than the maximum precision:
+      Truncate integer part to fit and the fractional becomes zero.
+    - Otherwise:
+      Truncate fractional part to fit.
+  */
+  if (intg >= DECIMAL_MAX_PRECISION)
+  {
+    intg= DECIMAL_MAX_PRECISION;
+    scale= 0;
+  }
+  else
+  {
+    uint room= min(DECIMAL_MAX_PRECISION - intg, DECIMAL_MAX_SCALE);
+    if (scale > room)
+      scale= room;
+  }
+
+  len= my_decimal_precision_to_length(intg + scale, scale, item->unsigned_flag);
+
+  return new Field_new_decimal(len, item->maybe_null, item->name, scale,
+                               item->unsigned_flag);
+}
+
+
 int Field_new_decimal::reset(void)
 {
   store_value(&decimal_zero);