summaryrefslogtreecommitdiff
path: root/toke.c
Commit message (Collapse)AuthorAgeFilesLines
* assert min identifier length in S_pending_ident()Zefram2017-12-081-0/+1
|
* fix oct/bin fp fractions in non-HEXFP_UQUAD buildsZefram2017-12-061-1/+1
| | | | | | | | | | The code for binaryish floating point literals, on builds where we're not confident of being able to fit a significand into an integer type, had built-in knowledge that the radix is 16, after the radix point. This gave erroneous values for octal and binary literals on those builds. This was shown up by the tests added in commit 58be57636a42d6c6fd404c48c4e1cb87870182df. Correct it to use the actual radix.
* limit digits based on radix for oct/bin fpTony Cook2017-12-061-2/+4
| | | | | | | All hexadecimal digits were being permitted in octal and binary floating point literals. (That octal and binary literals are permitted at all might be an accidental result of permitting hexadecimal?) Restrict which digits are permitted, in accordance with the radix.
* avoid negative shift in scan_num()Zefram2017-12-061-1/+1
| | | | | | Lengthy binaryish floating point literals used to perform illegal bit shifts. Ignore digits that are past the end of the significand at an earlier stage to avoid this. Code fix by Tony C. Fixes [perl #131894].
* assert legality of bitshifts in scan_num()Zefram2017-12-061-0/+4
| | | | | [perl #131894] found some negative-exponent shifting going on here. Make the illegality more accessible by asserting.
* Make Bad name error less unhelpfulFather Chrysostomos2017-12-041-8/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was brought up in perl #132485. Because ‘Bad name after...’ is a croak, it suppresses the more helpful hints like ‘Might be a runaway multi-line string’, in such cases as: use Moose; has erdef => ( isa => 'Int', is => 'ro, default => sub { 1 } ); has cxxc => ( isa => 'Int', is => 'ro', default => sub { 1 } ); We can allay this infelicity by emitting the ‘Missing operator before bareword’ before the Bad name croak, so in the example above we end up with: Bareword found where operator expected at - line 10, near "isa => 'Int" (Might be a runaway multi-line '' string starting on line 5) (Do you need to predeclare isa?) Bad name after Int' at - line 10. rather than just: Bad name after Int' at - line 10.
* Initialize the variable.Jarkko Hietaniemi2017-11-281-2/+2
|
* toke.c: Don’t leak memoryFather Chrysostomos2017-11-261-0/+1
|
* [perl #132485] Warn about "$foo'bar"Father Chrysostomos2017-11-261-5/+31
|
* toke.c: Comment typoFather Chrysostomos2017-11-261-1/+1
|
* toke.c: Convert to use is_utf8_non_invariant_stringKarl Williamson2017-11-261-4/+3
|
* toke.c lex_stuff_pvn(): Use fcn, not handrolled codeKarl Williamson2017-11-231-7/+1
| | | | | | Use the inline function that accomplishes the same thing as this hand-rolled code. The inline function should generate the same thing on ASCII platforms, but be faster on EBCDIC ones.
* localise $@ around source filtersZefram2017-11-131-1/+6
| | | | | | $@ could be clobbered by source filters, screwing up the reporting of errors in the filtered source. Prevent this by localising $@ around each call to a source filter. Fixes [perl #38920].
* avoid redundant initialisation around Newxz()Zefram2017-11-131-2/+4
| | | | | | Reduce Newxz() to Newx() where all relevant parts of the memory are being explicitly initialised, and don't explicitly zero memory that was already zeroed. [perl #36078]
* prevent invalid memory access in S_check_uni (RT #132433)Lukas Mai2017-11-121-1/+1
|
* add wrap_keyword_plugin function (RT #132413)Lukas Mai2017-11-111-0/+73
|
* parse yada-yada only as a statementZefram2017-11-101-1/+1
| | | | | | | | | | | | | | Commit f5727a1c71878a34f6255eb1a506c0b21af7d36f tried to make yada-yada be parsed consistently as a term expression, but actually things are more complicated than that. The tokeniser didn't accept yada-yada in the right contexts to make it usable as an expression, and changing that would require decisions on resolving ambiguities between yada-yada and flip-flop. It's also documented as being a statement rather than an expression, though with some incorrect information about ambiguities. Overall it looks more like the intent was for yada-yada to be a statement. This commit makes it grammatically treated as such, and also fixes up the dubious parts of the documentation. [perl #132150]
* toke.c: Add commentKarl Williamson2017-11-081-0/+2
|
* Dest buffer needs to be bigger for utf16_to_utf8()Karl Williamson2017-11-081-1/+4
| | | | | | | | | | These undocumented functions require the destination buffer to have the worst case size. However that size (previously listed as 3/2 * input) is wrong for EBCDIC. Correct the comments, and the single use of these in core. These functions do not have a way to avoid overflowing, which strikes me as wrong.
* restore error message for unterminated stringsLukas Mai2017-11-081-7/+13
| | | | | | | | | | | The previous strchr/memchr changes inadvertently broke the error message for perl -e '"'. Instead of Can't find string terminator '"' anywhere before EOF it became Can't find string terminator """ anywhere before EOF
* toke.c: Fix wrong use of memrchrKarl Williamson2017-11-071-1/+1
| | | | | | | | This was a replacement of strchr(), so should not have used the find-right-most memrchr. This was spotted by Christian Hansen. I don't know what the implications are, but thought I should get a fix in immediately.
* toke.c: use my_memrchr helper for portability [round 2]Nicolas R2017-11-061-2/+2
| | | | compilation broken on darwin using clang
* toke.c: Convert some strchr to memchrKarl Williamson2017-11-061-14/+20
| | | | | This allows things to work properly in the face of embedded NULs. See the branch merge message for more information.
* toke.c use memBEGINs with prev commitKarl Williamson2017-11-061-2/+4
|
* toke.c: Add limit parameter to 3 static functionsKarl Williamson2017-11-061-26/+31
| | | | | This will make it possible to fix to handle embedded NULs in the next commits.
* dquote.c: Use memchr() instead of strchr()Karl Williamson2017-11-061-2/+4
| | | | | | | This allows \x and \o to work properly in the face of embedded NULs. A limit parameter is added to each function, and that is passed to memchr (which replaces strchr). See the branch merge message for more information.
* Add memBEGINPs() to core and use itKarl Williamson2017-11-061-9/+16
| | | | | | This macro is like memBEGINs(), but the 'P' signifies we want a proper substring, meaning that the 2nd string parameter must not be the entire first parameter.
* Change some strBEGINs() to memBEGINs()Karl Williamson2017-11-061-11/+21
| | | | | | The latter is generally faster when the length is already known. This commit also changes a few hard-coded numbers to use sizeof().
* Change some strncmp(), etc. to strBEGINs()Karl Williamson2017-11-061-1/+1
| | | | | The latter is much clearer as to what's going on, and the programmer and program reader don't have to count characters.
* Use memEQs, memNEs in core filesKarl Williamson2017-11-061-10/+8
| | | | | | | | | | Where the length is known, we can use these functions which relieve the programmer and the program reader from having to count characters. The memFOO functions should also be slightly faster than the strFOO equivalents. In some instances in this commit, hard coded numbers are used. These come from the 'case' statement values that apply to them.
* Rename strEQs to strBEGINs; remove strNEsKarl Williamson2017-11-061-14/+14
| | | | | | | | | | The original names are confusing. See thread beginning with http://nntp.perl.org/group/perl.perl5.porters/244335 The two macros are mapped into just that one, complementing the result for the few cases where strNEs was used.
* add assertions about format lexingZefram2017-11-051-2/+4
| | | | | | [perl #132158] describes a situation in which a weird kind of unterminated format caused the parser to crash. Add some assertions that catch the messed up parser state before it actually crashes.
* yada-yada is a term, not an operatorLukas Mai2017-11-021-1/+1
|
* toke.c: Rmv useless codeKarl Williamson2017-11-021-7/+0
| | | | | | This code is unreachable because the code above makes sure that the variable can't be larger than PL_bufend, and the code is only executed if it is larger.
* toke.c: White space onlyKarl Williamson2017-11-021-33/+33
| | | | | Fix up some white space, including outdenting a block that was doubly indented
* (perl #132245) don't leak on \N{}Tony Cook2017-10-301-0/+1
| | | | | get_and_check_backslash_N_name() failed to free its working SV if the name was empty.
* (perl #132245) don't try to process a char range with no preceding charTony Cook2017-10-301-3/+3
| | | | | | A range like \N{}-0 eventually results in compilation failing, but before that, get_and_check_backslash_N_name() attempts to treat the memory before the empty output of \N{} as a character.
* toke.c: Add commentaryKarl Williamson2017-10-271-5/+6
|
* Use SvLEN_set/SvCUR_set in a few extra locationsNicolas R2017-10-191-2/+2
| | | | | | SvLEN was set without using the generic macro SvLEN_set. Use it in three extra locations, and also use SvCUR_set instead of SvCUR.
* Use preprocessor check for some DEBUG_X_TESTNicolas R2017-10-161-0/+10
| | | | | | | | | Most of the DEBUG_?_TEST calls are already protected by one '#idef DEBUGGING' check, but noticed a few of them which were not protected in sv.c and toke.c We should avoid these extra 'if' statements if perl is not compiled with debug option: -DDEBUGGING.
* get rid of "implicit fallthrough" warnings with gcc 7Lukas Mai2017-10-131-0/+1
|
* (perl #131777) prevent non-'=' assign ops tokens in sub signaturesTony Cook2017-09-191-3/+34
| | | | | | | | | | | | | | | | The yacc grammar introduced in d3d9da4a7 uses ASSIGNOP to represent the '=' used to introduce default values in subroutine signatures, unfortunately the parser returns ASSIGNOP for non-simple assignments, which allowed: sub foo ($x += 1) { ... } to default $x to 1. Modify yylex to accept only the simple assignment operator after a subroutine parameter. I'm not especially happy with the error recovery here.
* simplify s///eee lexing logicLukas Mai2017-09-081-8/+4
|
* (perl #131949) adjust s in case peekspace() moves the line stringTony Cook2017-08-311-1/+4
|
* [perl #131883] Include pkg in :prototype warningsFather Chrysostomos2017-08-281-0/+7
| | | | | | | | | | | | The subref-in-stash optimisation was causing the package name to be dropped in prototype warnings triggered by the :prototype() attribute syntax, since the GV containing the stash name and the sub name did not exist because of the optimisation. Commit 2eaf799e, which introduced said optimisation, simply did not include the package name in validate_proto’s ‘name’ parameter, but just the sub name. This commit makes it tell validate_proto to use the current stash name.
* Add another param to validate_protoFather Chrysostomos2017-08-281-2/+3
| | | | | I need this in order to fix bug #131883. Since it has a bit of churn, I’m putting it in a separate commit.
* (perl #131836) avoid a use-after-free after parsing a "sub" keywordTony Cook2017-08-281-0/+2
| | | | | | | | | | | | | The: d = skipspace(d); can reallocate linestr in the test case, invalidating s. This would end up in PL_bufptr from the embedded (PL_bufptr = s) in the TOKEN() macro. Assigning s to PL_bufptr and restoring s from PL_bufptr allows lex_next_chunk() to adjust the pointer to the reallocated buffer.
* Sprinkle diag_listed_as; perldiag tweaksFather Chrysostomos2017-08-211-0/+13
| | | | Trying to get tests passing after making diag.t smarter....
* (perl #131793) sanely handle PL_linestart > PL_bufptrTony Cook2017-08-171-4/+15
| | | | | | | In the test case, scan_ident() ends up fetching another line (updating PL_linestart), and since in this case we don't successfully parse ${identifier} s (and PL_bufptr) end up being before PL_linestart.
* (perl #131725) ignore the exponent on a decimal float if no digitsTony Cook2017-08-141-2/+19
| | | | | | | | | | | Previously the "1e-" in "1e--5" would be treated as "1", but consumed the "e-". This wasn't an issue for hex floats. I considered (and implemented) croaking instead, but this was inconsistent with the behaviour for hex floats, which only reach this code if a full hex float has been parsed.
View Lorry Controller Status