retainedlines.t - deterministic results and fixup tests under failure

The existing logic is only "correct" when all the tests pass. When they
fail the tests are revealed to be non-determinstic. The other problem is
that once a single test case fails and leaks an entry in the stash the
following tests are all contaminated and trigger "false failures". The
combination makes things look much more broken than they are.

This patch fixes the tests so that they are deterministic and the
"leaking" between test cases is stopped.

See GH Issue #20174

1 files changed, 13 insertions, 2 deletions

diff --git a/t/comp/retainedlines.t b/t/comp/retainedlines.t
index 4dc69a2fd5..35f1de0d15 100644
--- a/t/comp/retainedlines.t
+++ b/t/comp/retainedlines.t
@@ -55,7 +55,15 @@ sub check_retained_lines {
     # Is there a more efficient way to write this?
     my @expect_lines = (undef, map ({"$_\n"} split "\n", $prog), "\n", ';');
 
-    my @keys = grep {!$seen{$_}} grep { /eval/ } keys %::;
+    # sort in decreasing number so that $keys[0] is the from the most
+    # recent eval. In theory we should only have one, but if something
+    # breaks we might have more than one, and keys will return them in a
+    # random order, so if we dont do this then failing tests will have
+    # inconsistent results from run to run.
+    my @keys = map { $_->[0] }
+               sort { $b->[1] <=> $a->[1] }
+               map { (!$seen{$_} and /eval (\d+)/) ? [ $_, $1 ] : ()  }
+               keys %::;
 
     is ((scalar @keys), 1, "1 new eval");
 
@@ -67,7 +75,10 @@ sub check_retained_lines {
     for (0..$#expect_lines) {
 	is ($got_lines[$_], $expect_lines[$_], "Line $_ is correct");
     }
-    $seen{$keys[0]}++;
+    # if we are "leaking" evals we only want to fail the current test,
+    # so we need to mark them all seen (older code only marked $keys[0]
+    # seen and this caused tests to fail that actually worked properly.)
+    $seen{$_}++ for @keys;
 }
 
 my $name = 'foo';