| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
NOAMP is only emitted by toke.c when there are no parentheses. If
there is a parenthesis following a word, the lexer conjures up an '&'
token from nowhere.
|
| |
|
|
|
|
|
|
|
|
|
| |
Under mad builds, commit 5db1eb8 caused this warning:
$ PERL_XMLDUMP=/dev/null ./perl -Ilib -e 'foo:'
Invalid TOKEN object ignored at -e line 1.
Since I don’t understand the mad code so well, the easiest fix is to
revert back to using a PV, as we did before 5db1eb8. To record the
utf8ness, we sneak it behind the trailing null.
|
| |
|
|
|
| |
They have leaked since v5.15.9-35-g5db1eb8 (which probably broke mad
dumping of labels; to be addressed in the next commit).
|
| | |
|
| |
|
|
| |
This token is not used any more.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pad slot for a my sub now holds a stub with a prototype CV
attached to it by proto magic.
The prototype is cloned on scope entry. The stub in the pad is used
when cloning, so any code that references the sub before scope entry
will be able to see that stub become defined, making these behave
similarly:
our $x;
BEGIN { $x = \&foo }
sub foo { }
our $x;
my sub foo { }
BEGIN { $x = \&foo }
Constants are currently not cloned, but that may cause bugs in
pad_push. I’ll have to look into that.
On scope exit, lexical CVs go through leave_scope’s SAVEt_CLEARSV sec-
tion, like lexical variables. If the sub is referenced elsewhere, it
is abandoned, and its proto magic is stolen and attached to a new stub
stored in the pad. If the sub is not referenced elsewhere, it is
undefined via cv_undef.
To clone my subs on scope entry, we create a sequence of introcv and
clonecv ops. See the huge comment in block_end that explains why we
need two separate ops for each CV.
To allow my subs to be defined in inner subs (my sub foo; sub { sub
foo {} }), pad_add_name_pvn and S_pad_findlex now upgrade the entry
for a my sub to a CV to begin with, so that fake entries added to pads
(fake entries are those that reference outer pads) can share the same
CV. Otherwise newMYSUB would have to add the CV to every pad that
closes over the ‘my sub’ declaration. newMYSUB no longer throws away
the initial value replacing it with a new one.
Prototypes are not currently visible to sub calls at compile time,
because the lexer sees the empty stub. A future commit will
solve that.
When I added name heks to CV’s I made mistakes in a few places, by not
turning on the CVf_NAMED flag, or by not clearing the field when free-
ing the hek. Those code paths were not exercised enough by state
subs, so the problems did not show up till now. So this commit fixes
those, too.
One of the tests in lexsub.t, involving foreach loops, was incorrect,
and has been fixed. Another test has been added to the end for a par-
ticular case of state subs closing over my subs that I broke when ini-
tially trying to get sibling my subs to close over each other, before
I had separate introcv and clonecv ops.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since state variables are not shared between closures, but only
between invocations of the same closure, state subs should behave
the same way.
This was a little tricky. When we clone a sub, we now clone inner
state subs at the same time. When walking through the pad, cloning
items, we cannot simply clone the inner sub when we see it, because it
may close over things we haven’t cloned yet:
sub {
state sub foo;
my $x
sub foo { $x }
}
We can’t just delay cloning it and do it afterwards, because they may
be multiple subs closing over each other:
sub {
state sub foo;
state sub bar;
sub foo { \&bar }
sub bar { \&foo }
}
So *all* the entries in the new pad must be filled before any inner
subs can be cloned.
So what we do is put a stub in place of the cloned sub. And then
in a second pass clone the inner subs, reusing the stubs from the
first pass.
|
| |
|
|
|
|
|
|
|
|
| |
This commit does just enough to get things compiling. The padcv op
is still unimplemented (in fact, converting the padany to a padcv is
still not done), so you can’t actually run the code yet.
Bareword lookup in yylex now produces PRIVATEREF tokens for state
subs, so the grammar has been adjusted to accept a ‘subname’ in sub
calls (PRIVATEREF or WORD) where previously only a WORD was permitted.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In making ‘sub foo’ respect previous ‘our sub’ declarations in a
recent commit, I actually made ‘state sub foo’ into a syntax error.
(At the time, I patched up MYSUB in perly.y to keep the tests for ‘"my
sub" not yet implemented’ still working.) Basically, it was creat-
ing an empty pad entry, but returning something that perly.y was not
expecting.
This commit adjusts the grammar to allow the SUB branch of barestmt to
accept a PRIVATEREF for its subname, in addition to a WORD. It reuses
the subname rule that SUB used to use (before our subs were added),
gutting it to remove the special block handling, which SUB now tokes
care of. That means the MYSUB rule will no longer turn on CvSPECIAL
on the PL_compcv that is going to be thrown away anyway.
The code for special blocks (BEGIN, END, etc.) that turns on CvSPECIAL
now checks for state subs and skips those. It only applies to our
subs and package subs.
newMYSUB has now actually been written. It basically duplicates
newATTRSUB, except for GV-specific things. It does currently vivify a
GV and set CvGV, but I am hoping to change that later. I also hope to
merge some of the code later, too.
I changed the prototype of newMYSUB to make it easier to use. It is
not used anywhere on CPAN and has always simply died, so that should
be all right.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit switches all sub definitions, whether with ‘our’ or not,
to using S_force_ident_maybe_lex (formerly known as S_pending_ident).
This means that an unqualified (no our/my/state or package prefix)
‘sub foo’ declaration does a pad lookup, just like $foo.
It turns out that the vivification that I added to the then
S_pending_ident for CVs was unnecessary and actually buggy. We
*don’t* want to autovivify GVs for CVs, because they might be con-
stants or forward declarations, which are stored in a simpler form.
I also had to change the subname rule used by MYSUB in perly.y, since
it can now be fed a PRIVATEREF, which it does not expect. This may
prove to be temporary, but it keeps current tests passing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently the name is only allocated there. Nothing fetches it yet.
Notes on the implementation:
S_pending_ident contains the logic for determining whether $foo or
@foo refers to a lexical or package variable.
yylex defers to S_pending_ident if PL_pending_ident is set.
The KEY_sub case in yylex is changed to set PL_pending_ident instead
of using force_word. For package variables (including our),
S_pending_ident returns a WORD token, which is the same thing that
force_word produces. So *that* aspect of this change does not affect
the grammar. However....
The barestmt rule’s SUB branch begins with ‘SUB startsub subname’.
startsub is a null rule that creates a new sub in PL_compcv via
start_subparse(). subname is defined in terms of WORD and also checks
whether this is a special block, turning on CvSPECIAL(PL_compcv) if
it is. That flag has to be visible during compilation of the sub.
But for a lexical name, such as ‘our foo’, to be allocated in the
right pad, it has to come *before* startsub, i.e., ‘SUB subname
startsub’.
But subname needs to modify the sub that startsub created, set-
ting the flag.
So I copied (not moved, because MYSUB still uses it) the name-checking
code from the subname rule into the SUB branch of barestmt. Now that
uses WORD directly instead of invoking subname. That allows the code
there to set everything up in the right order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When parsing formats, the lexer invents tokens to feed to the parser.
So when the lexer dissects this:
format =
@<<<< @>>>>
$foo, $bar, $baz
.
The parser actually sees this (the parser knows that = . is like { }):
format =
; formline "@<<<< @>>>>\n", $foo, $bar, $baz;
.
The lexer makes no effort to make sure that the argument line is con-
tained within formline’s arguments. To make
{ do_stuff; $foo, bar }
work, the lexer supplies a ‘do’ before the block, if it is
inside a format.
This means that
$a, $b; $c, $d
feeds ($a, $b) to formline, wheras
{ $a, $b; $c, $d }
feeds ($c, $d) to formline. It also has various other
strange effects:
This script prints "# 0" as I would expect:
print "# ";
format =
@
(0 and die)
.
write
This one, locking parentheses, dies because ‘and’ has low precedence:
print "# ";
format =
@
0 and die
.
write
This does not work:
my $day = "Wed";
format =
@<<<<<<<<<<
({qw[ Sun 0 Mon 1 Tue 2 Wed 3 Thu 4 Fri 5 Sat 6 ]}->{$day})
.
write
You have to do this:
my $day = "Wed";
format =
@<<<<<<<<<<
({my %d = qw[ Sun 0 Mon 1 Tue 2 Wed 3 Thu 4 Fri 5 Sat 6 ]; \%d}->{$day})
.
write
which is very strange and shouldn’t even be valid syntax.
This does not work, because ‘no’ is not allowed in an expression:
use strict;
$::foo = "bar"
format =
@<<<<<<<<<<<
no strict; $foo
.
write;
Putting a block around it makes it work. Putting a semicolon before
‘no’ stop it from being a syntax error, but it silently does the
wrong thing.
I thought I could fix all these by putting an implicit do { ... }
around the argument line and removing the special-casing for an open-
ing brace, allowing anonymous hashrefs to work in formats, such
that this:
format =
@<<<< @>>>>
$foo, $bar, $baz
.
would turn into this:
format =
; formline "@<<<< @>>>>\n", do { $foo, $bar, $baz; };
.
But that will lead to madness like this ‘working’:
format =
@
}+do{
.
It would also stop lexicals declared in one format line from being
visible in another.
So instead this commit starts being honest with the parser. We still
have some ‘invented’ tokens, to indicate the start and end of a format
line, but now it is the parser itself that understands a sequence of
format lines, instead of being fed generated code.
So the example above is now presented to the parser like this:
format = ; FORMRBRACK
"@<<<< @>>>>\n" FORMLBRACK $foo, $bar, $baz ; FORMRBRACK
; .
Note about the semicolons: The parser expects to see a semicolon at
the end of each statement. So the lexer has to supply one before
FORMRBRACK. The final dot goes through the same code that handles
closing braces, which generates a semicolon for the same reason. It’s
easier to make the parser expect a semicolon before the final dot than
to change the } code in the lexer. We use the } code for . because it
handles the internal variables that keep track of how many nested lev-
els there, what kind, etc.
The extra ;FORMRBRACK after the = is there also to keep the lexer sim-
ple (ahem). When a newline is encountered during ‘normal’ (as opposed
to format picture) parsing inside a format, that’s when the semicolon
and FORMRBRACK are emitted. (There was already a semicolon there
before this commit. I have just added FORMRBRACK in the same spot.)
|
| |
|
|
|
| |
Otherwise you can end up with a format that has nonsensical op tree,
but it gets attached and you can call it anyway.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As long the argument line is not missing, braces can be used
for a format:
# valid
format foo {
@<<<
$a
}
but if the argument line is missing, you get a syntax error:
# invalid
format foo {
@<<<
}
and also if the name is missing:
# invalid
format {
@<<<
$a
}
If you use = then you can use a closing brace to terminate the format,
but only if the argument line is missing:
# valid, but useless
format =
@<<<
}
# invalid
format =
@<<<
$a
}
In commit 79072805 (perl 5.0 alpha 20), the lexer was changed to lie
to the parser about formats’ = and . delimiters, pretending they are
actually { and }, because the bracket-handling code takes care of all
the scoping issues (well, most of them).
Unfortunately, this implementation detail can leak through, but it is
far from consistent, as shown above.
Fixing this makes it easier to fix other bugs.
We do this by recording the fact that we are dealing with a format
token before jumping to the bracket code. And we feed format-specific
tokens to the parser in that case, so it can’t get fake brackets and
real brackets mixed up.
|
| |
|
|
|
|
|
|
| |
This commit makes given() alias $_ to the argument, using a slot in
the lexical pad if a lexical $_ is in scope, or $'_ otherwise.
This makes it work very similarly to foreach, and eliminates the
problem of List::Util functions not working inside given().
|
| |
|
|
|
|
|
|
| |
It is not necessary for the op containing the sv containing the name
of the package to last until the end of the block. Perl_package can
free the op immediately, as the only information it needs from it it
copies to the sv in PL_curstname. The version number can be treated
the same way.
|
| |
|
|
|
|
|
|
| |
Commit 421f30ed1e9 didn’t go far enough. If a special block happens
to replace a stub, then a format trying to close over variables in the
special block will be pointing to the wrong outer sub.
Such stubs shouldn’t usually happen, but perl shouldn’t crash.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this commit, qr// with a literal (compile-time) code block
will Do the Right Thing as regards closures and the scope of lexical vars;
in particular, the following now creates three regexes that match 1, 2
and 3:
for my $i (0..2) {
push @r, qr/^(??{$i})$/;
}
"1" =~ $r[1]; # matches
Previously, $i would be evaluated as undef in all 3 patterns.
This is achieved by wrapping the compilation of the pattern within a
new anonymous CV, which is then attached to the pattern. At run-time
pp_qr() clones the CV as well as copying the REGEXP; and when the code
block is executed, it does so using the pad of the cloned CV.
Which makes everything come out all right in the wash.
The CV is stored in a new field of the REGEXP, called qr_anoncv.
Note that run-time qr//s are still not fixed, e.g. qr/$foo(?{...})/;
nor is it yet fixed where the qr// is embedded within another pattern:
continuing with the code example from above,
my $i = 99;
"1" =~ $r[1]; # bare qr: matches: correct!
"X99" =~ /X$r[1]/; # embedded qr: matches: whoops, it's still
seeing the wrong $i
|
| | |
|
| |
|
|
|
| |
This meant changing LABEL's definition in perly.y, so most of this
commit is actually from the regened files.
|
| |
|
|
|
|
|
| |
Sync copyright dates with actual changes according to git history.
[Plus run regen_perly.h to update the SHA-256 checksums, and
regen/regcharclass.pl to update regcharclass.h]
|
| |
|
|
|
|
|
|
| |
This commit makes the __FILE__, __LINE__ and __PACKAGE__ token parse
the same way as nullary functions.
It adds two extra rules to perly.y to allow the op to be created in
toke.c, instead of directly inside the parser.
|
| |
|
|
|
|
|
| |
Move several pad functions into the core API. Document the pad
functions more consistently for perlapi. Fix the interface issues
around delimitation of lexical variable names, providing _pvn, _pvs,
_pv, and _sv forms of pad_add_name and pad_findmy.
|
| |
|
|
| |
There’s no arrow there.
|
| |
|
|
|
| |
Liberalise label syntax a little more, by allowing multiple adjacent
labels with no intervening statements, as in "foo: bar: baz:".
|
| |
|
|
|
|
|
| |
Some of the names for types of expression were misleading.
Rename "listexpr"->"optlistexpr", "listexprcom"->"optexpr", and
"argexpr"->"listexpr" to make them more accurately descriptive and
more consistent.
|
| |
|
|
|
|
| |
New API functions parse_fullexpr(), parse_listexpr(), parse_termexpr(),
and parse_arithexpr(), to parse an expression at various precedence
levels.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
> $ perl -d:Trace <<END
> warn "1";
> {
> warn "3";
> }
> warn "5";
> END
> >> -:1: warn "1";
> 1 at - line 1.
> >> -:5: warn "5"; <== wrong
> >> -:3: warn "3";
> 3 at - line 3.
> >> -:5: warn "5";
> 5 at - line 5.
I believe this is fixed by a one line change (plus `make regen_perly`):
|
| |
|
|
|
| |
Shift the structure of the GRAMPROG production (whole-file grammar)
to more closely match that of the other top-level productions.
|
| |
|
|
|
|
| |
The function scope() goes into the API as op_scope(), and mod() goes
into the API as op_lvalue(). Both marked experimental, because their
behaviour is a little quirky and not trivially dequirkable.
|
| |
|
|
|
| |
New API function parse_barestmt() parses a pure statement, with no label,
and returns just the statement's core ops, not attaching a state op.
|
| |
|
|
|
|
|
|
|
| |
Remove the line number parameter from newWHILEOP() and newFOROP()
functions. Instead, the line number for the impending COP is set by
parser code after constructing the ops. (In fact the parser was doing
this anyway in most cases.) This brings newWHILEOP() and newFOROP()
in line with the other op constructors, in that they do not concern
themselves with COPs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring of the grammar around statements. New production <barestmt>
encompasses a statement without label. It includes all statement types,
including declarations, with no unnecessary intermediate non-terminals.
It generates an op tree for the statement's content, with no leading
state op. The <fullstmt> production has just one rule, consisting of
optional label followed by <barestmt>. It puts a state op on the front
of the statement's content ops.
To support the regular statement op structure, the op sequence for for(;;)
loops no longer has a second state op between the initialisation and
the loop. Instead, the unstack op type is slightly adapted to achieve
the stack clearing without a state op.
The newFOROP() constructor function no longer generates a state op,
that now being the job of the <fullstmt> production. Consequently it
no longer takes a parameter stating what label is to go in the state op.
This brings it in line with the other op constructors.
|
| |
|
|
|
|
|
|
| |
Include <label> in productions before <decl> and <package_block>. This
means that labels can now appear at the beginning of all statement-like
things. There was no technical reason for the restriction of labels
to substantive statements, and that restriction in any case couldn't be
applied to PLUGSTMT-based plugged-in declarations.
|
| |
|
|
|
| |
New API function parse_block() parses a code block, including surrounding
braces. The block is a lexical scope, but not inherently a dynamic scope.
|
| |
|
|
|
| |
Set PL_expect at the start of parse_fullstmt() as well as at the start
of parse_stmtseq(). Test both.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
When recursing into the parser for recursive-descent parsing, put
a special entry on the bracket stack that generates a fake EOF if a
closing bracket belonging to an outer parser frame is seen. This keeps
the bracket stack balanced across a parse_stmtseq() frame, fixing
[perl #78222].
If a recursive-descent parser frame ends by yyunlex()ing an opening
bracket, pop its entry off the bracket stack and stash it in the
forced-token queue, to be revived when the token is re-lexed. This keeps
the bracket stack balanced across a parse_fullstmt() frame.
|
| |
|
|
|
|
| |
Put into the API op_append_elem, op_prepend_elem, and op_append_list. All
renamed from op_-less internal names. Parameter types for op_append_list
changed to match the rest of the op API and avoid some casting.
|
| |
|
|
|
| |
New API function parse_stmtseq() parses a sequence of statements, up to
closing brace or EOF.
|
| |
|
|
|
|
| |
VMS seems to have a 31 character limitation for external symbols. To be able to
fit into that, rename 'coerce_qwlist_to_paren_list' to
'munge_qwlist_to_paren_list'.
|
| |
|
|
|
|
|
|
|
|
| |
This makes a qw(...) list literal a distinct token type for the
parser, where previously it was munged into a "(",THING,")" sequence.
The change means that qw(...) can't accidentally supply parens to parts
of the grammar that want real parens. Due to many bits of code taking
advantage of that by "foreach my $x qw(...) {}", this patch also includes
a hack to coerce qw(...) to the old-style parenthesised THING, emitting
a deprecation warning along the way.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
yyparse() becomes reentrant. The yacc stack and related resources
are allocated in yyparse(), rather than in lex_start(), and they are
localised to yyparse(), preserving their values from any outer parser.
yyparse() now takes a parameter which determines which production it
will parse at the top level. New API function parse_fullstmt() uses this
facility to parse just a single statement. The top-level single-statement
production that is used for this then messes with the parser's head so
that the parsing stops without seeing EOF, and any lookahead token seen
after the statement is pushed back to the lexer.
|
| |
|
|
|
| |
The third production of <listexprcom>, "expr ','", could never be invoked,
because <expr> can already end with arbitrarily many commas.
|
| |
|
|
|
|
| |
There was a broken TOKEN_GETMAD attempting to handle the label preceding a
package-block statement, where no label is actually possible. The correct
behaviour for no label is a no-op, so just remove the TOKEN_GETMAD.
|
| |
|
|
|
|
| |
OPs relating to the package name and version were subject to double
freeing during error recovery from an incomplete package block. Fixed by
using the op_latefree mechanism to shift the op free time.
|
| |
|
|
|
|
| |
Package block syntax limits the scope of the package declaration to the
attached block. It's cleaner than requiring the declaration to come
inside the block.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
On Tue, Dec 01, 2009 at 08:22:38AM +0100, H.Merijn Brand wrote:
> perly.y: In function 'int Perl_madparse(PerlInterpreter*)':
> perly.y:335: error: cast from 'TOKEN*' to 'line_t' loses precision
> perly.y:348: error: cast from 'TOKEN*' to 'line_t' loses precision
> perly.y:430: error: cast from 'TOKEN*' to 'line_t' loses precision
Uses the same mechanism used in if, unless to retrieve an ival set in
toke.c
Signed-off-by: H.Merijn Brand <h.m.brand@xs4all.nl>
|
| |
|
|
|
|
| |
Currently no flags bits are used, and the length is cross-checked against
strlen() on the pointer, but the intent is to re-work the entire pad API to
be UTF-8 aware, from the current situation of char * pointers only.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Date: Tue, 27 Oct 2009 01:29:40 +0000
From: Zefram <zefram@fysh.org>
To: perl5-porters@perl.org
Subject: bareword sub lookups
Attached is a patch that changes how the tokeniser looks up subroutines,
when they're referenced by a bareword, for prototype and const-sub
purposes. Formerly, it has looked up bareword subs directly in the
package, which is contrary to the way the generated op tree looks up
the sub, via an rv2cv op. The patch makes the tokeniser generate the
rv2cv op earlier, and dig around in that.
The motivation for this is to allow modules to hook the rv2cv op
creation, to affect the name->subroutine lookup process. Currently,
such hooking affects op execution as intended, but everything goes wrong
with a bareword ref where the tokeniser looks at some unrelated CV,
or a blank space, in the package. With the patch in place, an rv2cv
hook correctly affects the tokeniser and therefore the prototype-based
aspects of parsing.
The patch also changes ck_subr (which applies the argument context and
checking parts of prototype behaviour) to handle subs referenced by an
RV const op inside the rv2cv, where formerly it would only handle a gv
op inside the rv2cv. This is to support the most likely kind of modified
rv2cv op.
[This commit includes the Makefile.PL for XS-APITest-KeywordRPN missing
from the original patch, as well as updates to perldiag.pod and a
MANIFEST sort]
|
