[perl #74022] Parser hangs on some Unicode characters

This changes the definition of isIDFIRST_utf8 to avoid any characters
that would put the parser in a loop.

isIDFIRST_utf8 is used all over the place in toke.c. Almost every
instance is followed by a call to S_scan_word. S_scan_word is only
called when it is known that there is a word to scan.

What was happening was that isIDFIRST_utf8 would accept a character,
but S_scan_word in toke.t would then reject it, as it was using
is_utf8_alnum, resulting in an infinite number of zero-length
identifiers.

Another possible solution was to change S_scan_word to use
isIDFIRST_utf8 or similar, but that has back-compatibility problems,
as it stops q·foo· from being a strings and makes it an identi-
fier instead.

1 files changed, 6 insertions, 1 deletions

diff --git a/t/comp/parser.t b/t/comp/parser.t
index 5c64d11fb2..16b4a826d1 100644
--- a/t/comp/parser.t
+++ b/t/comp/parser.t
@@ -3,7 +3,7 @@
 # Checks if the parser behaves correctly in edge cases
 # (including weird syntax errors)
 
-print "1..122\n";
+print "1..123\n";
 
 sub failed {
     my ($got, $expected, $name) = @_;
@@ -355,6 +355,11 @@ is($@, "", "multiline whitespace inside substitute expression");
 
 # Add new tests HERE:
 
+# bug #74022: Loop on characters in \p{OtherIDContinue}
+# This test hangs if it fails.
+eval chr 0x387;
+is(1,1, '[perl #74022] Parser looping on OtherIDContinue chars');
+
 # More awkward tests for #line. Keep these at the end, as they will screw
 # with sane line reporting for any other test failures