summaryrefslogtreecommitdiff
path: root/regen
Commit message (Collapse)AuthorAgeFilesLines
* Revert "Refactor opcode.pl - use c99 array initialization (using op enum)"Leon Timmermans2023-05-151-45/+11
| | | | This reverts commit ab28d7212b78f91ff5a0270d25b91a945c92c0e0.
* Revert "Refactor opcode.pl - with c99 array init index comments are not ↵Leon Timmermans2023-05-151-4/+8
| | | | | | necessary" This reverts commit f6405b442f40de7d3697fcd885bfbfa9ba55eab0.
* Remove duplicate "the" in commentsElvin Aslanov2023-05-032-2/+2
| | | | Fix spelling on various files pertaining to core Perl.
* regen/embed.pl: don't hide mathomsTomasz Konojacki2023-04-191-1/+1
| | | | | makedef.pl exports them unconditionally and embed.pl should use the same logic.
* regen/HeaderParser.pm - improved expression formatting and wrappingYves Orton2023-04-051-64/+285
| | | | | | | | | | | | | | | Karl complained about some of the wrapping logic we use for expressions. This tweaks the rules in a number of different ways in an attempt to produce more legible expressions. For instance if we have a complex expression with different parenthesized sub expressions, then try to put each sub expression on its own line. A previous patch ensures that we put shorter sub expressions first, and this patch builds on that to put each sub expression on its own line. We also use different logic to wrap the expressions, with the end result that each line should have the same number of defined() operations on it (with the exception of the last). We also try harder to line up logical operators and defined() functions.
* regen/HeaderParser.pm - remove comments from elif/else/endif when the ↵Yves Orton2023-04-051-32/+68
| | | | | | | | | contents is short it is a bit "noisy" to have comments that duplication the conditions when the original line with the condition is visible on the screen at the same time. this patch changes the rules so we only add these comments when the clause is 10 lines or more from its prior
* regen/HeaderParser.pm - with multi-term expressions put least firstYves Orton2023-04-051-1/+10
| | | | | if we have defined(X) && (defined(Y) || defined(Z)) put the defined(X) first because it has less operations in it.
* don't set a special filetype for generated .gitignoreLukas Mai2023-03-241-8/+18
| | | | | | | Previously it would default to Perl, which happens to produce the right comment character ("#"), but results in nonsensical syntax highlighting. Now we set $lang to the special value 'None', which still produces read-only declarations, but doesn't force a mode/filetype on editors.
* fix incorrect vi filetype declarations in generated filesLukas Mai2023-03-243-3/+3
| | | | | Vim's filetype declarations are case sensitive. The correct types for Perl, C, and Pod are perl, c, and pod, respectively.
* regen/embed.pl - change _aDEPTH and _pDEPTH to not have a leading underbarYves Orton2023-03-191-2/+2
| | | | | | | | | | | | | | The leading underbar is reserved by C. These defines are debugging only "recursion" depth related counters injected into the function macro wrappers when a function is marked as 'W', much the same way that aTHX_ and pTHX_ are when building under threaded builds. The functions are expected to incremented the depth parameter themselves. Note that "recursion" is quoted above because in practice currently they are only used by the regex engine when recursing virtually, and they do not relate to true C stack related recursion. (But they could be used for tracking C level recursion under debugging if someone needed it.)
* t/porting/deprecation.t - add tests for deprecation documentation and categoriesYves Orton2023-03-181-112/+114
|
* embed.fnc - document deprecate_xxx() macros and add them to handy.hYves Orton2023-03-181-1/+1
| | | | Also do not generate PERL_ARGS style macros for macros.
* regen/warnings.pl - line up categories, put them in date orderYves Orton2023-03-181-5/+6
| | | | This makes it easier to see the order the categories were added in.
* warnings.pm - add deprecated::version_downgrade categoryYves Orton2023-03-181-0/+1
| | | | | This also fixes the version_downgrade to show the correct version that version downgrades will be removed in.
* warnings.pm - add deprecated::goto_construct categoryYves Orton2023-03-181-0/+1
| | | | | This category applies to attempts to goto the internals of a block construct.
* warnings.pm - add deprecated::delimiter_will_be_paired categoryYves Orton2023-03-181-0/+1
| | | | | | Some delimiters are considered deprecated because in the future they will be used as part of a paired delimiter. This adds a new category for these cases.
* warnings.pm - add deprecated::apostrophe_as_package_separator as new ↵Yves Orton2023-03-181-1/+2
| | | | | | | deprecation category This category is about use of apostrophe as a package separator, eg for things like "Test::More::isn't()".
* warnings.pm - support deprecated::unicode_property_name categoryYves Orton2023-03-181-0/+1
| | | | | | This category is only used in the regex engine, we should be able to disable it specifically, as it seems like we will never actually remove demove support for the things it warns about.
* warnings.pm - add deprecated::dot_in_inc warings categoryYves Orton2023-03-181-0/+1
| | | | | Instead of using a generic warnings category switch to fine grained control.
* warnings.pm - support deprecated::smartmatch categoryYves Orton2023-03-181-12/+16
| | | | | | | | | | | | | | | | Currently we seem to lack a way to have a subcategory under deprecated. It seems reasonable to me that people might want to disable a specific subcategory warning while leaving the rest in place. This patch allows that. Note that both no warnings "deprecated"; and no warnings "deprecated::smartmatch"; work to disable the warning. Deprecated warnings shouldn't be "all or nothing", they should be specific and targetted.
* pp_ctl.c - add support for hooking require.Yves Orton2023-03-181-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This defines a new magic hash C<%{^HOOK}> which is intended to be used for hooking keywords. It is similar to %SIG in that the values it contains are validated on set, and it is not allowed to store something in C<%{^HOOK}> that isn't supposed to be there. Hooks are expected to be coderefs (people can use currying if they really want to put an object in there, the API is deliberately simple.) The C<%{^HOOK}> hash is documented to have keys of the form "${keyword}__${phase}" where $phase is either "before" or "after" and in this initial release two hooks are supported, "require__before" and "require__after": The C<require__before> hook is called before require is executed, including any @INC hooks that might be fired. It is called with the path of the file being required, just as would be stored in %INC. The hook may alter the filename by writing to $_[0] and it may return a coderef to be executed *after* the require has completed, otherwise the return is ignored. This coderef is also called with the path of the file which was required, and it will be called regardless as to whether the require (or its dependencies) die during execution. This mechanism makes it trivial and safe to share state between the initial hook and the coderef it returns. The C<require__after> hook is similar to the C<require__before> hook however except that it is called after the require completes (successfully or not), and its return is ignored always.
* scope.c - add mortal_destructor_sv() and mortal_svfunc_x()Yves Orton2023-03-181-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The function SAVEDESTRUCTOR_X() (save_destructor_x) can be used to execute a C function at the end of the current psuedo-block. Prior to this patch there was no "mortal" equivalent that would execute at the end of the current statement. We offer a collection of functions which are intended to free SV's at either point in time, but only support callbacks at the end of the current pseudo-block. This patch adds two such functions, "mortal_destructor_sv" which can be used to trigger a perl code reference to execute at the end of the current statement, and "mortal_svfunc_x" which can be used to trigger an SVFUNC_t C function at the end of the current statement. Both functions differ from save_destructor_x() in that instead of supporting a void pointer argument they both require their argument to be some sort of SV pointer. The Perl callback function triggered by "mortal_destructor_sv" may be provided no arguments, a single argument or a list of arguments, depending on the type of argument provided to mortal_destructor_sv(): when the argument is a raw AV (with no SV ref wrapping it), then the contents of the AV are passed in as a list of arguments. When the argument is anything else but NULL, the argument is provided as a single argument, and when it is NULL the perl function is called with no arguments. Both functions are implemented on top of a mortal SV (unseen by the user) which has PERL_MAGIC_destruct magic associated with it, which triggers the destructor behavior when the SV is freed. Both functions are provided with macros to match the normal SAVExx() API, with MORTALDESTRUCTOR_SV() wrapping mortal_destructor_sv() and MORTALSVFUNC_X() wrapping mortal_svfunc_x(). The heart of this logic cribbed from Leon Timmermans' Variable-OnDestruct. See the code at: https://metacpan.org/dist/Variable-OnDestruct/source/lib/Variable/OnDestruct.xs#L6-17 I am very grateful to him for his help on this. Any errors or omissions in this code are my fault, not his.
* regen/mg_vtable.pl - rename confusing var %sig to %vtable_confYves Orton2023-03-181-7/+6
|
* scope.c - improved RCPV support, SAVERCPV SAVEFREERCPVYves Orton2023-03-061-1/+2
| | | | | | | | | | | | | | | | | | | | | | | I misnamed some of the RCPV stuff when it was introduced. The macro which I called SAVERCPVFREE should have been called SAVERCPV, and there should also have been a SAVEFREERCPV macro. Note the naming system for these functions is a bit confusing. SAVEFREERCPV /just/ frees. SAVERCPV saves and restores (eg, its like local). SAVEFREESV is an exception. :-( and it behaves like SAVERCPV. See for instance SAVEPV or similar macros. There also should have been RCPV_REFCNT_dec() and RCPV_REFCNT_inc() macros. There was also missing documentation. This patch should fix all of these issues. Since this functionality has never been released in a production perl it should be fine to do these renames, nothing out there should use these macros yet. I noticed these oversights while fixing Scope::Upper, see also: https://github.com/Perl/perl5/issues/20861 https://rt.cpan.org/Ticket/Display.html?id=146897
* eval_sv(): call pp_entereval() via runopsDavid Mitchell2023-02-281-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Like the previous commit which did it for amagic_call() and call_sv(), this commit makes executing the faked-up OP_ENTEREVAL be executed as part of the runops loop rather than as a separate call. This is to allow shortly fixing up for a reference-counted stack. (CALLRUNOPS() will reify the stack if necessary, while the raw call to pp_entereval() won't fix up the stack unless its part of the runops loop too.) However, this is a bit more complex than call_sv() etc in that there is a good reason for calling pp_entereval() separately. The faked up OP_ENTEREVAL has its op_next set to NULL - this is the op which would normally be returned on failure of the eval compilation. By seeing whether the retuned value from pp_entereval() is NULL or not, eval_sv() can tell whether compilation failed. On the other hand, if pp_entereval() was made to be called as part of the runops loop, then the runops loop *always* finishes with PL_op set to NULL. So we can no lo longer distinguish between compile-failed and compile-succeeded-and-eval-ran-to-completion. This commit moves the entereval into the runops loop, but restores the ability to distinguish in a slightly hacky way. It adds a new private flag for OP_ENTEREVAL - OPpEVAL_EVALSV - which indicates to pp_entereval() that it was called from eval_sv(). And of course eval_sv() sets this flag on the OPpEVAL_EVALSV op it fakes up. If pp_entereval() fails to compile, then if that flag is set, it pushes a null pointer onto the argument stack before returning. Thus by checking whether *PL_stack_sp is NULL or not on return from CALLRUNOPS(), eval_sv() regains the ability to distinguish the two cases.
* Bump $VERSION in lib/feature.pm to 1.81.James E Keenan2023-02-241-1/+1
| | | | | Change is actually made in regen/feature.pl, then we run 'perl regen.pl'.
* Document that smartmatch is deprecated and will be removed in 5.42Philippe Bruhat (BooK)2023-02-251-0/+3
|
* Deprecate smartmatchPhilippe Bruhat (BooK)2023-02-251-1/+2
| | | | | Make the 'experimental::smartmatch' warning obsolete, and use 'deprecated' instead.
* Prefer scalar assignment to get caller's first return valueRichard Leach2023-02-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Multiple forms of syntax can be used to obtain a package name from `caller`, which emits this as its first return value, and assign that name to a lexical scalar. The following each achieve the same result, but with varying efficiency: * `sub callme { my $package = caller(2); ...}` * `sub callme { my ($package) = caller(2); ...}` * `sub callme { my $package = (caller(2))[0]; ...}` In the first example, `pp_caller` determines only the package name and pushes it to the stack. In the other two examples, the other 10 of `caller`'s return values are calculated and pushed onto the stack, before being discarded. This commit changes non-CPAN-first instances of the latter two forms in core to the first form. Note: There is a special exception to the equivalence described above, when caller is use in list context within the DB package. Such a usage instance in regen/warnings.pl therefore remains unchanged.
* pp: use PP macro consistentlyBranislav Zahradník2023-02-201-2/+3
|
* embed.pl - sort and dedupe flags in embef.fnc as part of tidyYves Orton2023-02-192-8/+17
| | | | | | | | | | | | | | | | | | | This ensures we use a canonical string for each possible flag variant, which makes it easier to search for flags with a given flag signature. It also exposed a mutex bug in flag handling which caused PerlEnv_putenv to be improperly marked as static, when it is in fact static inline. To validate there aren't any issues like this remaining in the script I set it up so the flags were shuffled during processing and ran embed.pl in a loop for a while and none of the output files changed, so I assume there are no further such issues. This patch also includes some basic validation of the flags so that if someone misses a line continuation the following lines are not treated as a new definition without any flags. I also ran perltidy on it according to the rules contained within the file.
* embed.fnc - sort entries alphabetically by function name.Yves Orton2023-02-191-9/+27
| | | | | | | | | | | | | | This is actually a library sort (lc with underbars removed), followed by a lexicographical sort. Comment lines are sticky to the line that follows them. Somehow the original version of this patch was missed in my earlier work on tidy_embed.pl, I think I messed up a rebase somehow. I noticed it was missing when I realized that new entries werent being sorted into place correctly. While this patch creates a fair bit of churn in the file right now, long term it will make it easier to use. Also note that the *output* files have not changed, which validates that the patch did not break anything.
* generated files - update mode lines to specify file typeElvin Aslanov2023-02-191-2/+3
| | | | | | | | | | This updates the mode-line for most of our generated files so that they include file type information so they will be properly syntax highlighted on github. This does not make any other functional changes to the files. [Note: Commit message rewritten by Yves]
* regen/regen_lib.pl - don't leave -new files in the tree when tests failYves Orton2023-02-191-1/+9
| | | | Unless the user requests we do so by setting REGEN_T_KEEP_CHANGES
* Field :param attributes, //= and ||= default assignmentsPaul "LeoNerd" Evans2023-02-101-0/+4
|
* Accept field VAR = EXPR on field varsPaul "LeoNerd" Evans2023-02-102-0/+6
| | | | | Allows non-constant expressions with side effects. Evaluated during the constructor of each instance.
* Initial attack at basic 'class' featurePaul "LeoNerd" Evans2023-02-104-2/+26
| | | | | | | | | | | | | Adds a new experimental warning, feature, keywords and enough parsing to implement basic classes with an empty `new` constructor method. Inject a $self lexical into method bodies; populate it with the object instance, suitably shifted Creates a new OP_METHSTART opcode to perform method setup Define an aux flag to remark which stashes are classes Basic implementation of fields. Basic anonymous methods.
* embed.pl - the 's', 'S', 'i' and 'I' flags are mutually exclusiveYves Orton2023-02-011-3/+16
| | | | | | | | | | | | | We had a bug where we processed the first one in the flags definition. Sorting the flags or rearranging them changes the output, which shouldn't happen. This also fixes the handling and specification of PerlEnv_putenv(), which was marked "si" when it should have been marked "i". This required changing its implementation from a Perl_ prefix to a S_ prefix and regenerating. I have run embed.pl in a loop with a local patch to shuffle the flags to see if there were any other order dependencies. No output files changed so I assume with this patch we are free of such bugs.
* Remove full stop in the 'try' feature headingDagfinn Ilmari Mannsåker2023-01-161-1/+1
| | | | None of the other headings in feature.pm have full stops.
* embed.pl - add a way to declare a parameter should be non-zeroYves Orton2023-01-141-1/+2
| | | | | This autogenerates the required asserts to validate a parameter is non-zero. It uses it to replace the check in regrepeat() as a first example.
* Correct one character typo appearing in lib/feature.pmJames E Keenan2023-01-111-2/+2
| | | | | | Since lib/feature.pm is a generated file, the actual changes are made in regen/feature.pl, followed by 'make regen' to regenerate lib/feature.pm (and then followed by 'make test_porting') to confirm.
* regcomp.pl - fixup intflags debug data to handle gaps properlyYves Orton2023-01-091-4/+19
| | | | | | | | | | | | | | We were not handling gaps in the sequence properly, and effectively showing the wrong flag names or missing the last flag. Now we die if there are any collisions or if any of the PREGf defines set more than one bit. This also adds some crude tests to validate that intflags serialization is working properly. Note, extflags handles more complex scenarios and seems to handle this gracefully already, hence the reason I haven't touched it as well. This also tweaks a comment in lexical_debug.t which part of this was cribbed from.
* regen/embed_lib.pl - don't add integer to reference and fixup source dataYves Orton2023-01-042-2/+9
| | | | | | | Dave noticed there was a bug where I was adding a reference to an integer. That did not make any sense. This patch fixes up the data so the source of the line is more clear. I will do a further follow up to improve this more, this is just to get the bug out of the way.
* regen/regcomp.pl - use regen/HeaderParser to read regnodes.h and etc.Yves Orton2022-12-241-16/+26
| | | | | | | | regen/regcomp.pl reads several header files, using HeaderParser will harden against line continuations or multiline comments. The current code is actually somewhat fragile, and minor changes to the header files it reads will break it. With this patch such changes would be dealt with by HeaderParser, which should already do the right thing.
* regen/feature.pl - use regen/HeaderParser to parse perl.hYves Orton2022-12-241-9/+11
| | | | | | Using HeaderParser hardens the code against possible future changes and ensures if there are any parse bugs they get fixed for all our regen code.
* regen/embed.pl - switch to using HeaderParser for code generation and etc.Yves Orton2022-12-244-359/+394
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | HeaderParser was designed to replace the old grouping code in embed.pl and regen/embed_lib.pl, which is used to generate embed.h and proto.h and embedvar.h It can handle "elif" in embed.fnc, and produces more consistently structured and formatted and readable output than the old code. It also has much better logic to dedupe expressions. Adding or changing a constraint in embed.fnc should no longer have any "action at a distance" effects on the output for other functions which were not changed or obviously affected by the change. The old code assumed that sorting the constraints applying to a given function definition was acceptable, with the result that "elif" was impossible to support properly, and creating the problem that adding a new constraint which sorted into a different position could change a large amount of the output, making it hard to verify that the change was correct. The new logic should, once the initial normalization is applied, ensure that any further changes are minimal. This patch also includes a new tool regen/normalize_embed.pl which will be run by make regen, which consistently formats embed.fnc itself, which should make maintaining the file easier, especially during code splits and reorgs. Function definitions can be lifted out, moved to the end, with new constraints, and the `make regen` will put it all back into the correct place and order. A number of tools and tests which use embed_lib.pl to load embed.fnc data have necessarily been changed to use the new HeaderParser based logic.
* regen/HeaderParser.pm - A module to parse our header filesYves Orton2022-12-242-0/+1614
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Consistent proper header file parsing with an OO interface. There are various traps and zaps parsing header files, such as line continuations, and multiline comments acting as a line continuation. There are also issues that the naive may overlook such as indented preprocessor directives, and such things. There are also some specialized tasks which we perform to construct header files from other header files, such as grouping content under similar guard clauses together, normalizing guard clauses, and the like. HeaderParser provides an API to handle these issues. The code which needs to read header files, or write header files, can use the HeaderParser to group and normalize the content they are reading or writing. This also frees the code generators from needing to worry about indentation, or such artifacts, HeaderParser normalizes it all away. This patch includes migrating everything to use the new infra, but it does not include some of the changes that would come with that new infra, so it would not pass test on THIS commit. Running make regen should "fix" this, although it is deliberate to make rebasing the branch easier. One of the notable changes in this commit is that embed.fnc is now under control of `make regen` (even though it is an input to `make regen`) and any changes will be automatically tied by running it, even if those changes also trigger other changes. HeaderParser sits underneath it all, so there is no chicken and egg problem that would require running `make regen` twice, the output of processing the modified embed.fnc would be identical to the output of processing the original. fixup
* Define OP_HELEMEXISTSOR, a handy LOGOP shortcut for HELEM existence testsPaul "LeoNerd" Evans2022-12-192-0/+7
| | | | | | | | | | | | | | | | | | This op is constructed using an OP_HELEM as the op_first and any scalar expression as the op_other. It is roughly equivalent to the following perl code: exists $hv{$key} ? $hv{$key} : OTHER except that the HV and the KEY expression are evaluated only once, and only one hv_* function is invoked to both test and obtain the value. It is therefore smaller and more efficient. Likewise, adding the OPpHELEMEXISTSOR_DELETE flag turns it into the equivalent of exists $hv{$key} ? delete $hv{$key} : OTHER
* regcomp.c - decompose into smaller filesYves Orton2022-12-091-2/+2
| | | | | | | | | | | | | | | | | This splits a bunch of the subcomponents of the regex engine into smaller files. regcomp_debug.c regcomp_internal.h regcomp_invlist.c regcomp_study.c regcomp_trie.c The only real change besides to the build machine to achieve the split is to also adds some new defines which can be used in embed.fnc to control exports without having to enumerate /every/ regex engine file. For instance all of regcomp*.c defines PERL_IN_REGCOMP_ANY, and this is used in embed.fnc to manage exports.
* Document postderef_qq support for ->$#* interpolationZakariyya Mughal2022-11-301-3/+4
| | | | This was added in commit <https://github.com/Perl/perl5/commit/ff25e5dbbad6ccf83f2e2a874a3c90294ea8cb47>.