| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Setting PL_expect after force_next has no effect, as force_next
(called by force_version and force_word) picks up the current value of
PL_expect and arranges for it to be reset thereto after the forced
token is force-fed to the parser.
The KEY_require case should be setting PL_expect to XTERM (as it
already does) when there is no forced token (version or bareword),
because we expect a term after ‘require’, but to XOPERATOR when
there is a forced token, because we expect an operator after that
forced token.
Since the PL_expect assignment has no effect after force_next, we can
set it to XOPERATOR before calling potentially calling force_next, and
then to XTERM afterwards.
Loop exits had the same bug, so this fixes them all.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With some list operators, this happens:
$ ./miniperl -e 'warn({$_ => 1} + 1) if 0'
syntax error at -e line 1, near "} +"
Execution of -e aborted due to compilation errors.
Putting + before the { or changing warn to print makes the prob-
lem go away.
The lexer is losing track of what token it expects next, so it ends
up interpreting the + as a unary plus, instead of an infix plus. The
parser doesn’t like that.
It happens because of this logic under case '{' (aka leftbracket:) in
toke.c:yylex:
switch (PL_expect) {
case XTERM:
if (PL_oldoldbufptr == PL_last_lop)
PL_lex_brackstack[PL_lex_brackets++] = XTERM;
else
PL_lex_brackstack[PL_lex_brackets++] = XOPERATOR;
PL_lex_allbrackets++;
OPERATOR(HASHBRACK);
The value we put on the brackstack is what we expect to find after the
closing brace (case '}' pops it off).
This particular if/else goes all the back to ef6361f9c226 (perl
5.000), or at least that was when it moved inside the XTERM case.
Before that, we had this:
if (oldoldbufptr == last_lop)
lex_brackstack[lex_brackets++] = XTERM;
else
lex_brackstack[lex_brackets++] = XOPERATOR;
if (expect == XTERM)
OPERATOR(HASHBRACK);
So it appears that the XTERM/XOPERATOR distinction, based on last_lop
was the ‘old’ (and wrong) way of doing it, but it had to be changed in
perl 5.000 for cases other than XTERM. That it remained for XTERM was
probably an oversight, which is easy to understand, since I seem to be
the first one to stumble across this after 18 years (what’s the rele-
vant Klortho number?).
Removing this last_lop check causes no tests to fail. And it makes
sense, since anything coming right after an anonymous hash that could
be either an infix or prefix operator must be infix.
|
|
|
|
|
|
|
|
|
|
|
| |
See also the previous commit.
This one was caused by 9c74ccc.
Again, we can’t just check whether PL_lex_repl has the SvEVALED
flag set (which means we are in s///e), but must also check whether
PL_lex_repl == PL_linestr (which means we are in the replacement part
of s///e).
|
|
|
|
|
|
| |
It was meant to check whether it was inside the replacement part of
s///e, but it only checked that it was inside s///e. PL_lex_repl is
set on both sides, but is only equal to PL_linestr on the rhs.
|
|
|
|
|
|
|
|
| |
This should never have worked:
%_=(_,"Just another ");
$_="Perl hacker,\n";
s//_}->{_/e;print
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It may be an odd place to allow comments, but s//"" # hello/e has\
always worked, *unless* there happens to be a null before the first #.
scan_subst in toke.c wraps the replacement text in do { ... } when the
/e flag is present.
It was adding a line break before the final } if the replacement text
contained #, because otherwise the } would be commented out.
But to find the # it was using strchr, which stops at the first null.
So eval "s//'\0'#/e" would fail.
It makes little sense to me to check whether the replacement contains
# before adding the line break. It would be faster just to add the
line break without checking.
But then I discovered this bug:
s//"#" . <<END/e;
foo
END
__END__
Can't find string terminator "END" anywhere before EOF at - line 1.
So now I have two bugs to fix.
The easiest solution seems to be to omit the line break and make the
comment parser skip the } at the end of a s///e replacement.
|
|
|
|
|
|
| |
It is harder to hack on perl with someone looking over one’s shoulder
when there are comments like this, even when it is euphemistic in its
use of voiced dental stops instead of the voiceless kind.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Outside of string eval, this:
s//<<END/e; print "a
END
b\n";
prints this:
a
b
But when we have string eval involved,
eval 's//<<END/e; print "a
END
b\n"';
we get this:
a
b
Amazing!
The buggy code in question goes back to commit 0244c3a403.
Since PL_linestr only contains the contents of the replacement
("<<END"), it peeks into the outer lexing scope’s linestr buffer, mod-
ifying it in place to remove the here-doc body, by copying everything
after the here-doc back to the spot where the body begins.
It was off by one, however, and left an extra line break.
When the code in question is reached, the variables are set as follows:
bufptr = "; print \"a"... (just after the s///)
s = "\nb\\n\"" (newline after the heredoc terminator)
The herewas variable already contains everything after the quote-
like operator containing the <<heredoc marker to the end of the line
including the \n ("; print \"a\n").
But then we concatenate everything from s onwards. So we end up with
the \n before the here-doc body and the \n from after the here-doc
terminator juxtaposed.
So after using s to extract the re-eval string, we increment s so it
points afer the final newline.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit fixes here-docs in single-line re-evals in files (as
opposed to evals) and here-docs in single-line quote-like operators
inside re-evals.
In both cases, the here-doc parser has to look into an outer
lexing scope to find the here-doc body. And in both cases it
was stomping on PL_linestr (the current line buffer) while
PL_sublex_info.re_eval_start was pointing to an offset in that buffer.
(re_eval_start is used to construct the string to include in the
regexp’s stringification once the lexer reaches the end of the
re-eval.)
Fixing this entails moving re_eval_start and re_eval_str to
PL_parser->lex_shared, making the pre-localised values visible.
This is so that the code that peeks into an outer linestr buffer to
steal the here-doc body can set up re_eval_str in the right scope.
(re_eval_str is used to store the re-eval text when the here-
oc parser has no choice but to modify linestr; see also commit
db4442662555874019.)
It also entails making the stream-based parser (i.e., that reads from
an input stream) leave PL_linestr alone, instead of clobbering it and
then reconstructing part of it afterwards.
|
|
|
|
|
|
|
| |
The parser expects a semicolon at the end of every statement, so the
lexer provides one. It was not doing so for evals ending with a
semicolon, even if that semicolon was not a statement-terminating
semicolon.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit 5097bf9b8 only partially fixed this, or, rather, did the
groundwork for fixing it.
If we have a pattern like this:
/(?{<<foo . baz
bar
foo
})/
Then PL_linestr contains this while we are parsing the block:
"(?{<<foo . baz\nbar\nfoo\n})"
The code for parsing a here-doc in a multiline PL_linestr buffer
(which applies to here-docs in string evals or in quote-like operat-
ors) likes to modify PL_linestr to contain everything after the
<<heredoc marker except the here-doc body, which has been stolen (but
it oddly includes the last character of the marker, which does not
matter, as PL_bufptr is set to PL_linestart+1):
"o . baz\n})"
The regexp block parsing code expects to be able to extract the entire
block (as a string) from PL_linestr after parsing it. So it is not
helpful for S_scan_heredoc to go and modify it like that.
Before modifying PL_linestr, we can set aside a copy of the source
code (in PL_sublex_info.re_eval_str) from the beginning of the regexp
block to the end of PL_linestr, so that the regexp block code can
retrieve the original source from there.
We also adjust PL_sublex_info.re_eval_start so that at the end of the
regexp block PL_bufptr - PL_sublex_info.re_eval_start is the length of
the block.
Instead of clobbering PL_linestr, we can copy everything after the
here-doc to when the body begins. And this for two reasons: it
requires less allocation (I would have made that change in the end
anyway, for efficiency), and it makes it easier to calculate how much
to subtract from re_eval_start.
This fix does not apply to here-docs in quotes in multiline string
evals, which crashes and always has.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Guess what this prints:
s/${s|||, \""}Just another Perl hacker,
/anything/;
print
And look at this:
$ perl5.6.2 -e 's/${s|||;\""}/foo\n/; print;'
$ perl5.16.0 -e 's/${s|||;\""}/foo\n/; print;'
$ perl5.17.2 -e 's/${s|||;\""}/foo\n/; print;'
Bus error
$ ./miniperl -e 's/${s|||;\""}/foo\n/; print;'
Bus error
The first two gave no output, though they should have shown "foo".
And bleadperl now crashes.
When the lexer parses a quote-like operator, it begins by extracting
what is between the quotes. It puts it in an SV stored in the varia-
ble PL_lex_stuff. Then, if it is y/// or s///, it scans the replace-
ment part and puts it in an SV in PL_lex_repl. When it finishes with
it, it sets PL_lex_repl to NULL.
Now, if you put s/// in the pattern part of s/// (or y in s), the
inner s/// will clobber PL_lex_repl with its own replacement string.
So, when the outer s/// finish parsing its pattern and wants its
replacement string. If it is not there, it assumes it has already
parsed it (whether PL_lex_repl is set is how it remembers which half
of s/// it is parsing), and proceeds to feed bad code to the parser,
resulting in a bad op tree.
PL_lex_repl needs to be localised when a quote-like operator is
parsed. Since localisation for quote-like operators happens in a sep-
arate yylex call (yylex calls sublex_push, which does it) after the
string delimiters are found, at which point PL_lex_repl has already
been set (clobbering the previous value), we change the delim-
iter-scanning code (scan_{str,trans,subst}) to use the new
PL_sublex_info.repl, which sublex_push now copies into PL_lex_repl
after localising the latter.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the lexer encounters a quote-like operator, it extracts the con-
tents of the quotes and starts an inner lexing scope.
To handle eval "s//<<FOO/e\n...", the here-doc parser peeks into the
outer lexing scope’s PL_linestr (current line buffer, which inside an
eval contains the entire string of code being parsed; for quote-like
operators, that is where the contents of the quote are stored). It
only does this inside a string eval. When parsing a file, the input
comes in one line at a time. So the here-doc parser steals lines from
the input stream for s//<<FOO/e outside an eval.
This approach fails in this case, as the peekee is the linestr for
s///, not for the eval:
eval ' s//"${\<<END}"/e; print
Just another Perl hacker,
END
'or die $@
__END__
Can't find string terminator "END" anywhere before EOF at (eval 1) line 1.
We also need to do this peeking stuff outside of a string eval, to
solve this:
s//"${\<<END}"
Just another Perl hacker,
END
/e; print
__END__
Can't find string terminator "END" anywhere before EOF at - line 1.
In the first example above, we need to look not in the parent lexing
scope’s linestr, but in that of the grandparent.
To solve the second example, we need to check whether the outer lexing
scope is a quote-like operator when we are not in an eval.
For parsing here-docs in quotes in eval, we currently store two
things, the former buffer pointer and the former linestr, in
PL_sublex_info.super_{bufp,lines}tr. The values for upper scopes are
stashed away on the savestack somewhere.
We need to be able to iterate through the outer lexer scopes till we
find one with multiple lines. Retrieving the information from the
savestack would be too complex and error-prone.
Since PL_linestr is an SV, we can abuse a couple of fields in it.
Upgrading it to PVNV gives it both IVX and NVX fields, which are big
enough to store pointers.
IVX is already used to hold an op number. So for the innermost quoted
scope we still need to use PL_sublex_info.super_bufptr. When entering
a new lexing scope (in sublex_push), we can localise the IVX field of
the outer PL_linestr SV and set it to what PL_sublex_info.super_bufptr
was in that scope. SvIVX(linestr) is only used for an op number when
that linestr’s lexing scope is the innermost one.
PL_sublex_info.super_linestr can be eliminated and replaced with
SvNVX(PL_linestr).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The code that uses this is specifically for parsing <<foo inside a
quote-like operator inside a string eval.
This prints bar:
eval "s//<<foo/e
bar
foo
";
print $_ || $@;
This prints Can't find string terminator blah blah blah:
eval "s//<<foo/e #\0
bar
foo
";
print $_ || $@;
Nulls in comments are allowed elsewhere. This prints bar:
eval "\$_ = <<foo #\0
bar
foo
";
print $_ || $@;
The problem with strchr is that it is specifically for scanning null-
terminated strings. If embedded nulls are permitted (and should be in
this case), memchr should be used.
This code was added by 0244c3a403.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When <<END occurs on the last line of a quote-like operator inside a
string eval ("${\<<END}"), it peeks into the linestr buffer of the
parent lexing scope (quote-like operators start a new lexing scope
with the linestr buffer containing what is between the quotes) to find
the body of the here-doc. It modifies that buffer, stealing however
much it needs.
It was not leaving things in the consistent state that s///e checks
for when it finishes parsing the replacement (to make sure s//}+{/
doesn’t ‘work’). Specifically, it was not shrinking the parent buf-
fer, so when PL_bufend was reset in sublex_done to the end of the par-
ent buffer, it was pointing to the wrong spot.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This works fine:
eval ' s//<<END.""/e; print
Just another Perl hacker,
END
'or die $@
__END__
Just another Perl hacker,
But this doesn’t:
eval ' s//"$1".<<END/e; print
Just another Perl hacker,
END
'or die $@
__END__
Can't find string terminator "END" anywhere before EOF at (eval 1) line 1.
It fails because PL_sublex_info.super_buf*, added by commit
0244c3a403, are not localised, so, after the "", s/// sees its own
buffer pointers in those variables, instead of its parent string eval.
This used to happen only with s///e inside s///e, but that was because
here-docs would peek inside the parent linestr buffer only inside
s///e, and not other quote-like operators. That was fixed in
recent commits.
Simply moving the assignment of super_buf* into sublex_push does solve
the bug for a simple "", as "" does sublex_start, but not sublex_push.
We do need to localise those variables for "${\''}", however.
|
| |
|
|
|
|
|
|
|
| |
Add t/base/while.t testing the basic of a while loop with minimal
dependencies. Change t/cmd/while.t into a non-base test using "test.pl".
(Includes bugfixes by Zefram over Gerard's original patch.)
|
|
|
|
|
|
|
| |
# New Ticket Created by (Peter J. Acklam)
# Please include the string: [perl #81916]
# in the subject line of all future correspondence about this issue.
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=81916 >
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
t/base/rs.t needs to @INClude the lib directory before requiring
PerlIO::scalar, even if the require is in an eval, because it will
otherwise pick up a previously-installed copy.
Since 87b9e1600 (Eliminate PL_* accessor functions under ithreads), a
binary-incompatible change, I’ve been getting these errors:
t/base/rs......................................................dyld: lazy symbol binding failed: Symbol not found: _Perl_Istack_sp_ptr
Referenced from: /usr/local/lib/perl5/5.13.5/darwin-thread-multi-2level/auto/PerlIO/scalar/scalar.bundle
Expected in: dynamic lookup
dyld: Symbol not found: _Perl_Istack_sp_ptr
Referenced from: /usr/local/lib/perl5/5.13.5/darwin-thread-multi-2level/auto/PerlIO/scalar/scalar.bundle
Expected in: dynamic lookup
FAILED--no leader found
Failed a basic test (base/rs.t) -- cannot continue.
make: *** [test] Error 255
|
|
|
|
| |
Signed-off-by: David Golden <dagolden@cpan.org>
|
|
|
|
| |
Tests in base can't utilise pragmata, specifically no warnings 'deprecated';
|
| |
|
| |
|
|
|
|
|
|
| |
Changes the error message to "Unrecognized character %s; marked by <--
HERE after %s<-- HERE near column %d". This should make it a little
simpler to spot and correct the suspicious character.
|
| |
|
|
|
|
| |
misinterpreting "[~" as an OP_SMARTMATCH.
|
|
|
|
|
| |
Message-ID: <20080614182005.yxuyk1mntwkog0o0@horde.wizbit.be>
p4raw-id: //depot/perl@34054
|
|
|
|
|
| |
Message-ID: <20080526091258.vbcecy2dc00gwg08@horde.wizbit.be>
p4raw-id: //depot/perl@34024
|
|
|
|
|
| |
Message-ID: <20080430115530.a09bjj6tic480c80@horde.wizbit.be>
p4raw-id: //depot/perl@33769
|
|
|
|
|
| |
Message-Id: <57CEC660-0020-48DF-A72A-931BCADC2AEE@surfar.nu>
p4raw-id: //depot/perl@31699
|
|
|
|
|
|
|
|
| |
"no warnings" is probably a bit inappropriate for a
base test. Putting -X on the command-line is probably
not a good idea, since it may hide other interesting
warnings in the future.
p4raw-id: //depot/perl@31601
|
|
|
|
|
| |
Message-ID: <20070423224826.GC544@abigail.nl>
p4raw-id: //depot/perl@31048
|
|
|
|
|
|
|
|
|
|
| |
$/=\0?)
Message-Id: <20060408152533.C4D5F6D08C@ugli.hut.fi>
(although I should add that this version of Coverity is actually
raising a false positive here, albeit something still interesting)
p4raw-id: //depot/perl@27744
|
|
|
|
|
| |
Message-ID: <20051119061639.GA25086@petdance.com>
p4raw-id: //depot/perl@26178
|
|
|
|
|
|
| |
From: "Craig A. Berry" <craigberry@mac.com>
Message-ID: <3FF8F32A.5000108@mac.com>
p4raw-id: //depot/perl@22058
|
|
|
|
|
| |
Message-ID: <6727B1DACFCDD311A757009027CA8D69044B673A@Ex02.inhouse.start.de>
p4raw-id: //depot/perl@21938
|
|
|
| |
p4raw-id: //depot/perl@20931
|
|
|
|
|
| |
Message-ID: <20030822084934.GA1257@ratsnest.hole>
p4raw-id: //depot/perl@20840
|
|
|
|
|
|
| |
From: "Allen Smith" <easmith@beatrice.rutgers.edu>
Message-Id: <10209070525.ZM1584639@puck2.rutgers.edu>
p4raw-id: //depot/perl@17874
|
|
|
|
|
| |
Message-id: <20020816233107.E9388@fdgroup.com>
p4raw-id: //depot/perl@17736
|
|
|
|
|
| |
(After this the whole suite passes 98.30%)
p4raw-id: //depot/perl@16864
|
|
|
|
|
| |
Message-ID: <KpAi8gzkganW092yn@efn.org>
p4raw-id: //depot/perl@15106
|
|
|
| |
p4raw-id: //depot/perl@15083
|
|
|
|
|
| |
Message-ID: <esuh8gzkg2/b092yn@efn.org>
p4raw-id: //depot/perl@15078
|
|
|
|
|
| |
Message-ID: <hRWh8gzkgONX092yn@efn.org>
p4raw-id: //depot/perl@15052
|
|
|
|
|
| |
1.1999999999999996, or thereabouts (yes, even sprintf).
p4raw-id: //depot/perl@14884
|
|
|
| |
p4raw-id: //depot/perl@14591
|
|
|
|
|
|
| |
scalars stay in sync (unearthed this via the OS/390 gcvt()
brokenness)
p4raw-id: //depot/perl@14581
|