Stop eval "END OF TERMS" from leaking

I found this memory leak by evaluating lines of the Copying file as
Perl code. :-)

The parser requires yylex to return exactly one token with each call.
Sometimes yylex needs to record a few tokens ahead of time, so its
puts them in its forced token stack.  The next call to yylex then pops
the pending token off that stack.

Ops belong to their subroutines.  If the subroutine is freed before
its root is attached, all the ops created when PL_compcv pointed
to that sub are freed as well.  To avoid crashes, the ops on the
savestack and the forced token stack are specially marked so they are
not freed when the sub is freed.

When it comes to evaluating "END OF TERMS AND CONDITIONS", the END
token causes a subroutine to be created and placed in PL_compcv.  The
OF token is treated by the lexer as a method call on the TERMS pack-
age.  The TERMS token is placed in the forced token stack as an sv in
an op for a WORD token, and a METHOD token for OF is returned.  As
soon as the parser sees the OF, it generates an error, which results
in LEAVE_SCOPE being called, which frees the subroutine for END while
TERMS is still on the forced token stack.  So the subroutine’s op
cleanup skips that op.  Then the parser calls back into the lexer,
which returns the TERMS token from the forced token stack.  Since
there has been an error, the parser discards that token, so the op
is never freed.  The forced token stack cleanup that happens in
parser_free does not catch this, as the token is no longer on
that stack.

Earlier, to solve the problem of yylex returning freed ops to the
parser, resulting in crashes, I set the op_savefree flag on ops on the
forced token stack.  But that resulted in a leak.

So now I am using a different approach: When the sub is freed and
frees all its ops, have it also look in the parser’s forced token
stack, freeing any ops that belong to it, and setting the point-
ers to null.

1 files changed, 6 insertions, 0 deletions

diff --git a/proto.h b/proto.h
index 14d512b745..5bb335217c 100644
--- a/proto.h
+++ b/proto.h
@@ -5261,6 +5261,12 @@ PERL_CALLCONV void	Perl_opslab_free_nopad(pTHX_ OPSLAB *slab)
 #define PERL_ARGS_ASSERT_OPSLAB_FREE_NOPAD	\
 	assert(slab)
 
+PERL_CALLCONV void	Perl_parser_free_nexttoke_ops(pTHX_ yy_parser *parser, OPSLAB *slab)
+			__attribute__nonnull__(pTHX_1)
+			__attribute__nonnull__(pTHX_2);
+#define PERL_ARGS_ASSERT_PARSER_FREE_NEXTTOKE_OPS	\
+	assert(parser); assert(slab)
+
 #  if defined(PERL_DEBUG_READONLY_OPS)
 PERL_CALLCONV void	Perl_Slab_to_ro(pTHX_ OPSLAB *slab)
 			__attribute__nonnull__(pTHX_1);