[perl #123802] Assertion failure with /$0{}/

If the parser (perly.c) pops tokens when encountering a syntax error,
it can pop inner lexing scopes (which handle the contents of quote-
like operators).

If there is a pending token on the pending token stack, then the cur-
rent lexing state is LEX_KNOWNEXT.  It usually gets set to a pending
value stored in PL_lex_defer when the last pending token is emitted.

If scopes are exited when there is a pending token, then the state is
reset, since it is localised, even thought we still have a token pend-
ing.  We have code to account for that and still emit the pending
token.  (See 7aa8cb0dec1.)  But the pending lexing state is still
used after the pending token is emitted.  So we can end up with
LEX_INTERPEND when there is no inner lexing scope.

LEX_INTERPEND will cause sublex_done (in toke.c) to be called, which
does a LEAVE.  If the current scope does not belong to it, we end up
exiting a scope set up by the parser, which frees the parser stack
(via SAVEDESTRUCTOR_X and clear_yystack in perly.c).  The parser is
still using the stack, which holds reference counts on the CV in
PL_compcv, so reference counts get screwed up.

We need to check whether we have a proper lexing scope set up if the
lexing state is LEX_INTERPEND.

This is a follow-up to f4460c6f7a, which was a similar bug, but
occurred with LEX_INTERPCONCAT, rather than LEX_INTERPEND.

1 files changed, 8 insertions, 0 deletions

diff --git a/toke.c b/toke.c
index db511c5337..33ae20fa66 100644
--- a/toke.c
+++ b/toke.c
@@ -4493,6 +4493,14 @@ Perl_yylex(pTHX)
 	/* FALLTHROUGH */
 
     case LEX_INTERPEND:
+	/* Treat state as LEX_NORMAL if we have no inner lexing scope.
+	   XXX This hack can be removed if we stop setting PL_lex_state to
+	   LEX_KNOWNEXT, as can the hack under LEX_INTREPCONCAT below.  */
+	if (UNLIKELY(!PL_lex_inwhat)) {
+	    PL_lex_state = LEX_NORMAL;
+	    break;
+	}
+
 	if (PL_lex_dojoin) {
 	    const U8 dojoin_was = PL_lex_dojoin;
 	    PL_lex_dojoin = FALSE;