Allow => to quote built-in keywords across lines

This is the second try.  5969c5766a5d3 had a bug in it under non-
MAD builds.

If I have a sub I can use its name as a bareword as long as I suffix
it with =>, even if the => is on the next line:

$ ./perl -Ilib -e 'sub tim; warn tim' -e '=>'
tim at -e line 1.

If I want to use a built-in keyword’s name as a bareword, I can put =>
after it:

$ ./perl -Ilib -e 'warn time =>'
time at -e line 1.

But if I combine the two (keyword + newline), it does not work:

$ ./perl -Ilib -e 'warn time' -e ' =>'
1373611283 at -e line 1.

unless I override the keyword:

$ ./perl -Ilib -Msubs=time -e 'warn time' -e ' =>'
time at -e line 1.

=> after a bareword is checked for in two places in toke.c.  The first
comes before a comment saying ‘NO SKIPSPACE BEFORE HERE!’; it only
skips spaces and finds a => on the same line.  The second comes later;
it skips vertical space and comments, too.

But the second check is in a code path that is not reached by keywords
that are not overridden (as is the ‘NO SKIPSPACE’ comment).

This commit adds an extra check for built-in keywords after we have
determined that the keyword is not overridden.  In that case, there is
no reason we cannot use skipspace, as we no longer have to worry about
what PL_oldbufptr etc. point to.

This commit leaves __DATA__ and __END__ alone, since they
are special, problematic and controversial.  (See, e.g.,
<https://rt.perl.org/rt3/Ticket/Display.html?id=78348#txn-1234355>.)

Allowing whitespace to be scanned across line boundaries without
increasing the line number (something this commit has to do to make
this work) can cause the way PL_linestr is handled to change.

PL_linestr usually holds just the current line when reading from a
handle.  Now it can hold the current line plus the next line or seve-
ral lines, depending on how much whitespace is to be found there.

When '\n' or '#' was encountered, the lexer would modify the buffer in
place and add a null, setting PL_bufend to point to that null.  That
would make it look as though the end of the line had been reached, and
avoided having to scan to find the end of a comment.

In string eval and quote-like operators, the end of the comment does
have to be scanned for.  We can’t just fake EOL and read the next
line of input.

Under MAD builds, the end of the comment was being scanned for any-
way, even when reading from a handle.  So everything worked under MAD,
which was what I tested 5969c5766a5d3 under.

This commit changes the '\n' and '#' handling to match the MAD code
(scan for the end of the comment instead of faking a buffer trunca-
tion), which 5969c5766a5d3 failed to do.

1 files changed, 5 insertions, 1 deletions

diff --git a/t/base/lex.t b/t/base/lex.t
index 7ef753807b..7821e763a3 100644
--- a/t/base/lex.t
+++ b/t/base/lex.t
@@ -1,6 +1,6 @@
 #!./perl
 
-print "1..93\n";
+print "1..94\n";
 
 $x = 'x';
 
@@ -443,3 +443,7 @@ y # comment
  ;
 print "not " unless $_ eq 'b';
 print "ok 93 - y <comment> <newline> ...\n";
+
+print "not " unless (time
+                     =>) eq time=>;
+print "ok 94 - => quotes keywords across lines\n";