Fix our sub with proto

yylex must emit exactly one token each time it is called.  Some-
times yylex needs to parse several tokens at once.  That’s what
the various force functions are for.  But that is also what
PL_pending_ident is for.

The various force_next, force_word, force_ident, etc., functions keep
a stack of tokens (PL_nextval/PL_nexttype) that yylex will check imme-
diately when called.

PL_pending_ident is used to track a single identifier that yylex will
hand off to S_pending_ident to handle.

S_pending_ident is the only piece of code for resolving an identi-
fier that could be lexical but could also be a package variable.
force_ident assumes it is looking for a package variable.

force_* takes precedence over PL_pending_ident.

All this means that, if an identifier needs to be looked up in the pad
on the next yylex invocation, it has to use PL_pending_ident, and the
force_* functions cannot be used at the same time.

Not realising that, when I made ‘our sub foo’ store the sub in the
pad I also made ‘our sub foo ($)’ into a syntax error, because it
was being parsed as ‘our sub ($) foo’ (the prototype being ‘forced’);
i.e., the pending tokens were being pulled out of the ‘queue’ in the
wrong order.  (I put queue in quotes, because one queue and one unre-
lated buffer together don’t exactly count as ‘a queue’.)

Changing PL_pending_ident to have precedence over the force stack
breaks ext/XS-APItest/t/swaptwostmts.t, because the statement-parsing
interface does not localise PL_pending_ident.  It could be changed to
do that, but I don’t think it is the right solution.

Having two separate pending token mechanisms makes things need-
lessly fragile.

This commit eliminates the PL_pending_ident mechanism and
modifies S_pending_ident (renaming it in the process to
S_force_ident_maybe_lex) to work with the force mechanism.  I was
going to merge it with force_ident, but the two make incompatible
assumptions that just complicate the code if merged.  S_pending_ident
needs the sigil in the same string buffer, to pass to the pad inter-
face.  force_ident needs to be able to work without a sigil present.

So now we only have one queue for pending tokens and the order is more
predictable.

1 files changed, 0 insertions, 1 deletions

diff --git a/sv.c b/sv.c
index 2bf9e8f68b..63523ddf43 100644
--- a/sv.c
+++ b/sv.c
@@ -11231,7 +11231,6 @@ Perl_parser_dup(pTHX_ const yy_parser *const proto, CLONE_PARAMS *const param)
     parser->multi_open	= proto->multi_open;
     parser->multi_start	= proto->multi_start;
     parser->multi_end	= proto->multi_end;
-    parser->pending_ident = proto->pending_ident;
     parser->preambled	= proto->preambled;
     parser->sublex_info	= proto->sublex_info; /* XXX not quite right */
     parser->linestr	= sv_dup_inc(proto->linestr, param);