testsuite: Widen acceptance window of T12545 (#19414)

In a sequel of #19414, I wrote a script that measures min and max allocation
bounds of T12545 based on randomly modifying -dunique-increment. I got a spread
of as much as 4.8%. But instead of widening the acceptance window further (to
5%), I committed the script as part of this commit, so that false positive
increases can easily be diagnosed by comparing min and max bounds to HEAD.

Indeed, for !5814 we have seen T12545 go from -0.3% to 3.3% after a rebase.
I made sure that the min and max bounds actually stayed the same.
In the future, this kind of check can very easily be done in a matter of a
minute. Maybe we should increase the acceptance threshold if we need to check
often (leave a comment on #19414 if you had to check), but I've not been bitten
by it for half a year, which seems OK.

Metric Increase:
    T12545

2 files changed, 44 insertions, 4 deletions

diff --git a/testsuite/tests/perf/compiler/T12545.measure.sh b/testsuite/tests/perf/compiler/T12545.measure.sh
new file mode 100755
index 0000000000..e578c7197a
--- /dev/null
+++ b/testsuite/tests/perf/compiler/T12545.measure.sh
@@ -0,0 +1,39 @@
+#!/usr/bin/env sh
+
+cat <<EOF
+This script is for painlessly finding min and max allocations of T12545 based on
+playing with -dunique-increment (see #19414) to diagnose if a metric increase is
+a false positive.
+A ratio of about 4.8% (48 per mille) is expected.
+EOF
+
+# https://stackoverflow.com/a/4774063/388010
+TOP="$( cd -- "$(dirname "$0")/../../../../" >/dev/null 2>&1 ; pwd -P )"
+GHC=${GHC:-$TOP/_validate/stage1/bin/ghc}
+
+echo "Using GHC=$GHC. Feel free to override via env var"
+
+function measure() {
+  $GHC -fforce-recomp -v0 -dunique-increment=$1 T12545.hs +RTS -t 2>&1 | cut -f1 -d',' | grep -o -P '\d+'
+}
+
+min=999999999999
+max=-999999999999
+while true; do
+  inc=$((1 + $RANDOM % 1000000))
+  n=$(measure $inc)
+  any_change=false
+  if [ $n -lt $min ]; then
+    min=$n
+    any_change=true
+    echo "New min: $min (on $inc)"
+  fi
+  if [ $n -gt $max ]; then
+    max=$n
+    any_change=true
+    echo "New max: $max (on $inc)"
+  fi
+  if [ "$any_change" = true ]; then
+    echo "New ratio: $(($max*1000/$min - 1000)) per mille"
+  fi
+done
diff --git a/testsuite/tests/perf/compiler/all.T b/testsuite/tests/perf/compiler/all.T
index eadf2ff920..ee764f15cb 100644
--- a/testsuite/tests/perf/compiler/all.T
+++ b/testsuite/tests/perf/compiler/all.T
@@ -211,12 +211,13 @@ test('T12234',
      compile,
      [''])
 
-# T12545 is sensitive to -dunique-increments changes, see #19414.
-# I've seen variations of 3% by playing with that parameter, so that's the
-# current acceptance window.
+# T12545 is sensitive to -dunique-increments changes, see #19414. I've seen
+# variations of as much as 4.8% by playing with that parameter, but I think
+# it's better to check with T12545.measure.sh that lower and upper bounds of
+# allocations indeed haven't changed and then simply accept the metric increase.
 test('T12545',
      [ only_ways(['normal']),
-       collect_compiler_stats('bytes allocated', 3),
+       collect_compiler_stats('bytes allocated', 3), # Don't increase it, run T12545.measure.sh and leave a comment on #19414!
        extra_clean(['T12545a.hi', 'T12545a.o'])
      ],
      multimod_compile,