Fix lexing of nested heredoc strings in token_get_all()

This fixes bug #60097.

Before two global variables CG(heredoc) and CG(heredoc_len) were used to
track the current heredoc label. In order to support nested heredoc
strings the *previous* heredoc label was assigned as the token value of
T_START_HEREDOC and the language_parser.y assigned that to CG(heredoc).

This created a dependency of the lexer on the parser. Thus the
token_get_all() function, which accesses the lexer directly without
also running the parser, was not able to tokenize nested heredoc strings
(and leaked memory). Same applies for the source-code highlighting
functions.

The new approach is to maintain a heredoc_label_stack in the lexer, which
contains all active heredoc labels.

As it is no longer required, T_START_HEREDOC and T_END_HEREDOC now don't
carry a token value anymore.

In order to make the work with zend_ptr_stack in this context more
convenient I added a new function zend_ptr_stack_top(), which retrieves the
top element of the stack (similar to zend_stack_top()).

1 files changed, 5 insertions, 0 deletions

diff --git a/Zend/zend_language_scanner.h b/Zend/zend_language_scanner.h
index a1a84b3bc9..c1a2313b2d 100644
--- a/Zend/zend_language_scanner.h
+++ b/Zend/zend_language_scanner.h
@@ -31,6 +31,7 @@ typedef struct _zend_lex_state {
 	unsigned char *yy_limit;
 	int yy_state;
 	zend_stack state_stack;
+	zend_ptr_stack heredoc_label_stack;
 
 	zend_file_handle *in;
 	uint lineno;
@@ -50,6 +51,10 @@ typedef struct _zend_lex_state {
 	const zend_encoding *script_encoding;
 } zend_lex_state;
 
+typedef struct _zend_heredoc_label {
+	char *label;
+	int length;
+} zend_heredoc_label;
 
 BEGIN_EXTERN_C()
 int zend_compare_file_handles(zend_file_handle *fh1, zend_file_handle *fh2);