MDEV-23118 FORMAT(d1,dec) where dec=0/38 and d1 is DECIMAL(38,38) gives incorrect results

FORMAT() can print more integer digits (than the argument has)
if rounding happens:

  FORMAT(9.9,0) -> '10'

The old code did not take this into account.

Fix:

1. One extra digit is needed in case of rounding

- If args[1] is a not-NULL constant, then reserve space for one extra integer
  digit if the requested number of decimals is less than args[0]->decimals.

- Otherwise, reserve space for one extra integer digit if
  args[0]->decimals is not 0, because rounding can potentially happen
  (depending on the exact data in arguments).

2. One extra digit is also needed if the argument has no integer digits,
   e.g. in a data type like DECIMAL(38,38).

The conditions 1 and 2 are ORed.

3. Fixing FORMAT_MAX_DECIMALS from 30 to 38. This was forgotten in 10.2.1
   (when the limit for the number of fractional digits in DECIMAL was extended).

1 files changed, 1 insertions, 1 deletions

diff --git a/mysql-test/main/func_str.result b/mysql-test/main/func_str.result
index 4a01935785c..7a6af68b1d8 100644
--- a/mysql-test/main/func_str.result
+++ b/mysql-test/main/func_str.result
@@ -2708,7 +2708,7 @@ create table t1(a float);
 insert into t1 values (1.33);
 select format(a, 2) from t1;
 Catalog	Database	Table	Table_alias	Column	Column_alias	Type	Length	Max length	Is_null	Flags	Decimals	Charsetnr
-def					format(a, 2)	253	56	4	Y	0	39	8
+def					format(a, 2)	253	57	4	Y	0	39	8
 format(a, 2)
 1.33
 drop table t1;