summaryrefslogtreecommitdiff
path: root/toke.c
Commit message (Collapse)AuthorAgeFilesLines
* [perl #88420] BOM support on Windows broken in 5.13.11Jan Dubois2011-04-131-1/+7
| | | | | | | When Perl reads the script in text mode, then the tell() position on the script handle may include stripped carriage return characters. Therefore the file position after reading the first line of the script may be one larger than the length of the input buffer.
* PATCH: partial [perl #86972]: Allow /aiaKarl Williamson2011-04-101-9/+16
| | | | | | This allows a second /a modifier to not have to be contiguous with the first. This patch changes only the part in toke.c where the modifiers are in suffix form.
* [perl #87064] eval no longer shares filtersFather Chrysostomos2011-04-031-3/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this commit: commit f07ec6dd59215a56bc1159449a9631be7a02a94d Author: Zefram <zefram@fysh.org> Date: Wed Oct 13 19:05:19 2010 +0100 remove filter inheritance option from lex_start The only uses of lex_start that had the new_filter parameter false, to make the new lexer context share source filters with the previous lexer context, were uses with rsfp null, which therefore never invoked source filters. Inheriting source filters from a logically unrelated file seems like a silly idea anyway. string evals could inherit the same source filter space as the cur- rently compiling code. Despite what the quoted commit message says, sharing source filters allows filters to be inherited in both direc- tions: A source filter created when the eval is being compiled also applies to the file with which it is sharing its space. There are at least 20 CPAN distributions relying on this behaviour (or, rather, what could be considered a Test::More bug). So this com- mit restores the source-filter-sharing capability. It does not change the current API or make public the API for sharing source filters, as this is supposed to be a temporary stop-gap measure for 5.14.
* fix compiler warning in toke.cDavid Mitchell2011-03-261-1/+1
| | | | | | The third arg to newSVOP must be non-null, and the macro expansion for SvREFCNT_inc can give a null value sometimes. So replace it with SvREFCNT_inc_NN and everyone's happy..
* reg_namedseq: Restructure so doesn't duplicate codeKarl Williamson2011-03-201-6/+20
| | | | | | | | | | | | | | | This routine now calls reg() recursively after converting the parse to something the rest of the code understands. This eliminates duplicated code, and allows for uniform treatment of code points, as things were getting out of sync. It also eliminates the restrction on how many characters a named sequence can expand to. toke now converts its input (which is in Unicode terms) to native on EBCDIC platforms, so the rest of the code can can continue to ignore that. The restriction on the length of the number of characters a named sequence is hereby removed, because reg() handles that.
* toke.c: Raise error for multiple regexp modsKarl Williamson2011-03-011-4/+40
| | | | | | | When the new regular expression modifiers being allowed in suffix-form were added on a very tight schedule, it was with the understanding that the error checking that only one can occur per regular experssion would be added later. This accomplishes that.
* [perl #79442] A #line "F" in a string eval doesn't update *{"_<F"}Father Chrysostomos2011-02-271-5/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | There are two problems here. String evals do not populate @{"_<..."} arrays the way parsed streams do. The lines are all put into the array before the parse. Commit e66cf94 added the code to copy (actually alias, but whatever) the lines into the new array when a #line directive is encountered. Inte- restingly, the following commit (8a5ee59) wrapped that code in an #ifndef USE_ITHREADS block, because ‘[c]hange 25409 [e66cf94] wasn’t necessary for threaded perls’. It seems it *was* necessary for threaded perls after all, because the lines are simply not copied. In non-threaded perls it wasn’t working properly either. The array in the new glob was the same array as the old (aliased), so the line numbers would be off if the #line directive contained a line number that differed. This commit does three things: • It removes the #ifndef, • It checks whether the line number has changed and aliases the indi- vidual elements of the array. • The breakpoints hash is not copied if the line number differs, as setting a breakpoint on (eval 1):1 (warn 1) in eval qq{warn 1;\n#line 1 "foo"\nwarn 2;} should not also set a breakpoint on foo:1 (warn 2).
* toke.c: 'Specialized /le message is only for substitutesKarl Williamson2011-02-211-0/+8
| | | | | m//le has to be the lexical comparison, so use the generic deprecation for that case
* move brace to same line as conditionalKarl Williamson2011-02-211-2/+1
|
* toke.c: fix commentKarl Williamson2011-02-211-2/+3
|
* Allow suffix form for /a /d /l /uKarl Williamson2011-02-191-2/+58
| | | | | | This patch contains the code changes for doing this, but not most of the pod changes, nor the new .t tests required. There were already tests in place to make sure that this didn't break backcompat.
* toke.c: Don't take the address of a registerKarl Williamson2011-02-191-1/+1
| | | | | I discovered after I pushed 858a358bdd94da8251cdb2210d9bec7c1bbe7464 that I had forgotten to 'git add' changes before committing.
* toke.c: Move suffix re mods message to one placeKarl Williamson2011-02-191-27/+30
| | | | | This involves a slight refactoring of the routine that handles parsing for the mods
* toke.c: silence win32 compiler warningKarl Williamson2011-02-151-1/+1
|
* perldiag: retitle Ambiguous use of %c{%s%s}Father Chrysostomos2011-02-131-0/+1
| | | | | | | | | | | | | | | | This is not very helpful: =item Ambiguous use of %c{%s%s} resolved to %c%s%s especially since it is functionally identical to the previous entry: =item Ambiguous use of %c{%s} resolved to %c%s Not only can diagnostics.pm never find it, but it is hard for human beings to understand what the different is at first glance, too. So filling in the second and fourth %s’s with the two possible values slays a twain of avians with one piece of petrified matter.
* Silence win32 smoke compiler warningKarl Williamson2011-02-131-2/+2
|
* Fix up \cX for 5.14Karl Williamson2011-02-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Throughout 5.13 there was temporary code to deprecate and forbid certain values of X following a \c in qq strings. This patch fixes this to the final 5.14 semantics. These are: 1) a utf8 non-ASCII character will croak. This is the same behavior as pre-5.13, but it gives a correct error message, rather than the malformed utf8 message previously. 2) \c{ and \cX where X is above ASCII will generate a deprecated message. The intent is to remove these capabilities in 5.16. The original agreement was to croak on above ASCII, but that does violate our stability policy, so I'm deprecating it instead. 3) A non-deprecated warning is generated for all other \cX; this is the same as throughout the 5.13 series. I did not have the tuits to use \c{} as I had planned in 5.14, but \N{} can be used instead.
* toke.c: Add commentKarl Williamson2011-01-271-1/+3
|
* Break out the generated function Perl_keywords() into keywords.c, a new file.Nicholas Clark2011-01-241-3403/+4
| | | | | | | | As it and Perl_yylex() both need FEATURE_IS_ENABLED, feature_is_enabled() is no longer static, and the two macro definitions move from toke.c to perl.h Previously, one had to cut and paste the output of perl_keywords.pl into the middle of toke.c, and it was not clear that it was generated code.
* CH] Change usage of regex/op common to common namesKarl Williamson2011-01-161-2/+2
| | | | | | | | | This patch changes the core functions to use the common names for the fields that are shared between op.c and regcomp.c, just for consistency of using one name throughout the core for the same thing. A grep of cpan shows that both names are used in various modules; so both names must be retained.
* Fix typos (spelling errors) in Perl sources.Peter J. Acklam) (via RT2011-01-071-4/+4
| | | | | | | | | # New Ticket Created by (Peter J. Acklam) # Please include the string: [perl #81904] # in the subject line of all future correspondence about this issue. # <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=81904 > Signed-off-by: Abigail <abigail@abigail.be>
* Silence some data truncation compiler warningsJan Dubois2010-12-161-1/+1
|
* permit labels to be stackedZefram2010-12-131-1/+1
| | | | | Liberalise label syntax a little more, by allowing multiple adjacent labels with no intervening statements, as in "foo: bar: baz:".
* recursive-descent expression parsingZefram2010-12-111-50/+451
| | | | | | New API functions parse_fullexpr(), parse_listexpr(), parse_termexpr(), and parse_arithexpr(), to parse an expression at various precedence levels.
* toke.c: Fix EBCDIC problemKarl Williamson2010-12-071-2/+2
| | | | | | Commit 356979f4a7d780fd67a92a9ca6c8659bd12e7168 failed to include two instances in toke.c that needed the same treatment, i.e., converting properly from I8 to native.
* toke.c: highlight problematic-mentioning commentKarl Williamson2010-12-071-3/+3
|
* [perl #77762] Constant assignment warningFather Chrysostomos2010-11-301-0/+1
| | | | | | | | | | | | With this patch: $ ./perl -we 'sub A () {1}; if (0) {my $foo = A or die}' $ ./perl -we 'sub A () {1}; if (0) {my $foo = 1 or die}' Found = in conditional, should be == at -e line 1. Since the value of a constant may not be known at the time the program is written, it should be perfectly acceptable to do a constant assign- ment in a conditional.
* In S_scan_const(), use my_snprintf() instead of my_sprintf().Nicholas Clark2010-11-301-2/+4
| | | | Paranoid linkers warn about using sprintf(), and rightly so.
* Make my $pi := 4; a syntax error.Nicholas Clark2010-11-281-1/+2
| | | | | Previously it interpreted := as an empty attribute list, and issued a deprecation warning. This change permits := to be used as a binding operator.
* Deprecate ?PATTERN? without explicit m operatorZefram2010-11-241-0/+2
| | | | | | | | | | Deprecate ?PATTERN?, recommending the equivalent m?PATTERN? syntax, in order to eventually allow the question mark to be used in new operators that would currently be ambiguous. (With minor reconciliation edits by David Golden) Signed-off-by: David Golden <dagolden@cpan.org>
* y///rFather Chrysostomos2010-11-021-2/+9
|
* In S_scan_const(), use the return value of my_sprintf() to avoid a strlen().Nicholas Clark2010-10-281-4/+4
| | | | Brought to you by the campaign for the elimination of strlen().
* function to parse isolated labelZefram2010-10-251-4/+91
| | | | | | | New API function parse_label() parses a label, separate from statements. If a label has not already been lexed and queued up, it does not use yylex(), but parses the label itself at the character level, to avoid unwanted lexing past an absent optional label.
* function to parse unlabelled statementZefram2010-10-251-2/+40
| | | | | New API function parse_barestmt() parses a pure statement, with no label, and returns just the statement's core ops, not attaching a state op.
* S_tokeq()'s fast scan loop should terminate on \\ not \Nicholas Clark2010-10-221-1/+2
| | | | | | | As-was, it would drop out of the scanner into the backslashed-backslash processing loop earlier than need be, and hence would be copying the octets of strings (in place) as soon as any backslash had been seen. Now it defers copying until copying is actually unavoidable.
* Add single-term prototypeDavid Golden2010-10-211-2/+2
| | | | | | | | | | | | | | | | The C<+> prototype is a special alternative to C<$> that will act like C<\[@%]> when given a literal array or hash variable, but will otherwise force scalar context on the argument. This is useful for functions which should accept either a literal array or an array reference as the argument: sub smartpush (+@) { my $aref = shift; die "Not an array or arrayref" unless ref $aref eq 'ARRAY'; push @$aref, @_; } When using the C<+> prototype, your function must check that the argument is of an acceptable type.
* function to parse Perl code blockZefram2010-10-211-16/+57
| | | | | New API function parse_block() parses a code block, including surrounding braces. The block is a lexical scope, but not inherently a dynamic scope.
* handle bracket stack better in recdescent parsingZefram2010-10-211-14/+54
| | | | | | | | | | | | | When recursing into the parser for recursive-descent parsing, put a special entry on the bracket stack that generates a fake EOF if a closing bracket belonging to an outer parser frame is seen. This keeps the bracket stack balanced across a parse_stmtseq() frame, fixing [perl #78222]. If a recursive-descent parser frame ends by yyunlex()ing an opening bracket, pop its entry off the bracket stack and stash it in the forced-token queue, to be revived when the token is re-lexed. This keeps the bracket stack balanced across a parse_fullstmt() frame.
* add lex_start to the APIZefram2010-10-211-11/+24
| | | | | lex_start() is added to the API, marked experimental, and documented. It also gains a flags parameter for foreseeable future use.
* avoid side-effecting source held in scalarZefram2010-10-211-6/+1
| | | | | | | | Syntax plugins can modify the source being parsed. It's fine for them to modify the lexer buffer, but this must not be the same scalar that was supplied to lex_start() and may be in use outside. Therefore always copy the scalar in lex_start() rather than just referencing it. Fixes [perl #78358].
* remove redundant lex_endZefram2010-10-211-11/+0
| | | | | | | The lex_end() function is redundant, because the lexer context object is actually finalised by parser_free(), triggered by the save stack. The lex_end() function has historically been empty, except when the PL_doextract global was being misused to store lexer state.
* replace PL_doextract with better kinds of variableZefram2010-10-211-7/+7
| | | | | | | | PL_doextract had two unrelated jobs, neither best served by an interpreter global variable. The first was to track the -x command-line switch. That is replaced with a local variable in S_parse_body(). The second was to track whether the lexer is in the middle of a =pod section. That is replaced with an element in PL_parser.
* remove filter inheritance option from lex_startZefram2010-10-211-6/+2
| | | | | | | | The only uses of lex_start that had the new_filter parameter false, to make the new lexer context share source filters with the previous lexer context, were uses with rsfp null, which therefore never invoked source filters. Inheriting source filters from a logically unrelated file seems like a silly idea anyway.
* APIify op list constructorsZefram2010-10-121-7/+7
| | | | | | Put into the API op_append_elem, op_prepend_elem, and op_append_list. All renamed from op_-less internal names. Parameter types for op_append_list changed to match the rest of the op API and avoid some casting.
* plugin mechanism to rewrite calls to a subroutineZefram2010-10-101-18/+1
| | | | | | | | | | | | | | | | | | | | | | | | New magic type PERL_MAGIC_checkcall attaches a function to a CV, which will be called as the second half of the op checker for an entersub op calling that CV. Default state, in the absence of this magic, is to process the CV's prototype if it has one, or apply list context to all the arguments if not. New API functions cv_get_call_checker() and cv_set_call_checker() provide a clean interface to this facility, hiding the internal use of magic. Expose in the API the new functions rv2cv_op_cv(), ck_entersub_args_list(), ck_entersub_args_proto(), and ck_entersub_args_proto_or_list(), which are meaningful segments of standard entersub op checking and are likely to be useful in plugged-in call checker functions. Expose new API function op_contextualize(), which is a public interface to the internal scalar()/list()/scalarvoid() functions. This API is likely to be required in most plugged-in call checker functions. Incidentally add new function mg_free_type(), in the API, which will remove magic of one type from an SV. (mg_free() removes all magic, and there isn't anything else more selective.)
* Create populate_isa() to de-duplicate logic to populate @ISA.Nicholas Clark2010-10-091-17/+7
| | | | | | | Previously yylex() was conditionally populating @AnyDBM_File::ISA (if it was not set, and the token dbmopen was seen), and init_predump_symbols() was populating @IO::File::ISA (unconditionally, but this is so early that nothing previously could have set it). This refactoring eliminates code duplication.
* Move the automatic instantiation of @AnyDBM_File::ISA to Perl_yylex()Nicholas Clark2010-10-091-1/+17
| | | | | | | | | | | case KEY_dbmopen in Perl_yylex() has always had special-case code to create @AnyDBM_File::ISA, using GV_ADDMULTI. S_gv_magicalize_isa() [part of Perl_gv_fetchpvn_flags()] has special case code to spot "AnyDBM_File::ISA" being created with GV_ADDMULTI, and populate the variable if it is empty. Grouping the special case code in one place makes more sense. Removing the special case code from gv.c means that there is no longer a check clause in the code path for *every* package's @ISA initialisation.
* [PATCH] function to parse Perl statement sequenceZefram2010-10-041-0/+46
| | | | | New API function parse_stmtseq() parses a sequence of statements, up to closing brace or EOF.
* toke.c: call regcurly instead of duplicating codeKarl Williamson2010-10-031-11/+2
| | | | Remove code that duplicates regcurly()
* systematically provide pv/pvn/pvs/sv quartetsZefram2010-09-281-2/+31
| | | | | Anywhere an API function takes a string in pvn form, ensure that there are corresponding pv, pvs, and sv APIs.
View Lorry Controller Status