summaryrefslogtreecommitdiff
path: root/makedef.pl
Commit message (Collapse)AuthorAgeFilesLines
* regen/embed.pl - switch to using HeaderParser for code generation and etc.Yves Orton2022-12-241-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | HeaderParser was designed to replace the old grouping code in embed.pl and regen/embed_lib.pl, which is used to generate embed.h and proto.h and embedvar.h It can handle "elif" in embed.fnc, and produces more consistently structured and formatted and readable output than the old code. It also has much better logic to dedupe expressions. Adding or changing a constraint in embed.fnc should no longer have any "action at a distance" effects on the output for other functions which were not changed or obviously affected by the change. The old code assumed that sorting the constraints applying to a given function definition was acceptable, with the result that "elif" was impossible to support properly, and creating the problem that adding a new constraint which sorted into a different position could change a large amount of the output, making it hard to verify that the change was correct. The new logic should, once the initial normalization is applied, ensure that any further changes are minimal. This patch also includes a new tool regen/normalize_embed.pl which will be run by make regen, which consistently formats embed.fnc itself, which should make maintaining the file easier, especially during code splits and reorgs. Function definitions can be lifted out, moved to the end, with new constraints, and the `make regen` will put it all back into the correct place and order. A number of tools and tests which use embed_lib.pl to load embed.fnc data have necessarily been changed to use the new HeaderParser based logic.
* makedef.pl: Properly define a hash elementKarl Williamson2022-12-101-1/+1
| | | | | | | I had been so used to working lately with C preprocessor directives that I used the same paradigm in Perl without noticing. If 'warnings' had been on, it would have caught this (the previous commit now turns them on).
* makedef.pl: Add 'use warnings'Karl Williamson2022-12-101-0/+1
| | | | This would have saved me a bunch of time.
* makedef.pl: Create $define{WIN32} etcKarl Williamson2022-12-101-5/+11
| | | | | | | | | A portion of makedef.pl must manually be kept in sync with changes to portions of perl.h. Prior to this commit, you had to remember that #ifdef WIN32 from perl.h had to be translated to a completely different paradigm for use in this file. This commit fixes that, which can help prepare the way for future programmatic syncing of the two files.
* makedef.pl: Consistently use constantKarl Williamson2022-12-101-27/+27
| | | | | This script defines a constant 'PLATFORM', but often uses the rhs of that constant. Change to always use the constant.
* Switch libc per-interpreter data when tTHX changesKarl Williamson2022-10-181-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As noted in the previous commit, some library functions now keep per-thread state. So far the only ones we care about are libc locale-changing ones. When perl changes threads by swapping out tTHX, those library functions need to be informed about the new value so that they remain in sync with what perl thinks the locale should be. This commit creates a function to do this, and changes the thread-changing macros to also call this as part of the change. For POSIX 2008, the function just calls uselocale() using the per-interpreter object introduced previously. For Windows, this commit adds a per-interpreter string of the current LC_ALL, and the function calls setlocale on that. We keep the same string for POSIX 2008 implementations that lack querylocale(), so this commit just enables that variable on Windows as well. The code is already in place to free the memory the string occupies when done. The commit also creates a mechanism to skip this during thread destruction. A thread in its death throes doesn't need to have accurate locale information, and the information needed to map from thread to what libc needs to know gets destroyed as part of those throes, while relics of the thread remain. I couldn't find a way to accurately know if we are dealing with a relic or not, so the solution I adopted was to just not switch during destruction. This commit completes fixing #20155.
* Some locale operations need to be done in proper threadKarl Williamson2022-10-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a step in solving #20155 The POSIX 2008 locale API introduces per-thread locales. But the previous global locale system is retained, probably for backward compatibility. The POSIX 2008 interface causes memory to be malloc'd that needs to be freed. In order to do this, the caller must first stop using that memory, by switching to another locale. perl accomplishes this during termination by switching to the global locale, which is always available and doesn't need to be freed. Perl has long assumed that all that was needed to switch threads was to change out tTHX. That's because that structure was intended to hold all the information for a given thread. But it turns out that this doesn't work when some library independently holds information about the thread's state. And there are now some libraries that do that. What was happening in this case was that perl thought that it was sufficient to switch tTHX to change to a different thread in order to do the freeing of memory, and then used the POSIX 2008 function to change to the global locale so that the memory could be safely freed. But the POSIX 2008 function doesn't care about tTHX, and actually was typically operating on a different thread, and so changed that thread to the global locale instead of the intended thread. Often that was the top-level thread, thread 0. That caused whatever thread it was to no longer be in the expected locale, and to no longer be thread-safe with regards to localess, This commit causes locale_term(), which has always been called from the actual terminating thread that POSIX 2008 knows about, to change to the global thread and free the memory. It also creates a new per-interpreter variable that effectively maps the tTHX thread to the associated POSIX 2008 memory. During perl_destruct(), it frees the memory this variable points to, instead of blindly assuming the memory to free is the current tTHX thread's. This fixes the symptoms associtated with #20155, but doesn't solve the whole problem. In general, a library that has independent thread status needs to be updated to the new thread when Perl changes threads using tTHX. Future commits will do this.
* locale: Create special variable to hold current LC_ALLKarl Williamson2022-10-181-0/+8
| | | | | | | | | | | | Some configurations require us to store the current locale for each category. Prior to this commit, this was done in the array PL_curlocales, with the entry for LC_ALL being in the highest element. Future commits will need just the value for LC_ALL in some other configurations, without needing the rest of the array. This commit splits off the LC_ALL element into its own per-interpreter variable to accommodate those. It always had to have special handling anyway beyond the rest of the array elements,
* Clean up perl.h/makedef.pl common logicKarl Williamson2022-10-181-14/+27
| | | | | This has gotten two twisty little mazy over time. Clean it up, add comments, and make sure the logic is the same on both.
* handy.h: Set macro to false if can't ever be trueKarl Williamson2022-10-101-0/+2
| | | | | | | | | | | | It's unlikely that perl will be compiled with out the LC_CTYPE locale category being enabled. But if it isn't, there is no sense in having per-interpreter variables for various conditions in it, and no sense having code that tests those variables. This commit changes a macro to always yield 'false' when this is disabled, adds a new similar macro, and changes some occurrences that test for a variable to use the macros instead of the variables. That way the compiler knows these to conditions can never be true.
* makedef.pl Skip PL_in_utf8_COLLATE_locale if no LC_COLLATEKarl Williamson2022-10-101-0/+1
| | | | This fixes #20371
* Don't export win32-only symbols on non-WindowsCraig A. Berry2022-09-231-0/+2
| | | | | This was causing link-time errors in the VMS build because it was trying to export symbols that don't exist.
* makedef.pl: Rmv obsolete symbolKarl Williamson2022-09-221-1/+0
| | | | This routine is now a static function, so needn't be in makedef.pl
* Use general locale mutex for numeric operationsKarl Williamson2022-09-091-7/+0
| | | | | | | | | | | | | | | | This commit removes the separate mutex for locking locale-related numeric operations on threaded perls; instead using the general locale one. The previous commit made that a general semaphore, so now suitable for use for this purpose as well. This means that the locale can be locked for the duration of some sprintf operations, longer than before this commit. But on most modern platforms, thread-safe locales cause this lock to expand just to a no-op; so there is no effect on these. And on the impacted platforms, one is not supposed to be using locales and threads in combination, as races can occur. This lock is used on those perls to keep Perl's manipulation of LC_NUMERIC thread-safe. And for those there is also no effect, as they already lock around those sprintf's.
* Make the locale mutex a general semaphoreKarl Williamson2022-09-091-0/+1
| | | | | Future commits will use this new capability, and in Configurations where no locale locking is currently necessary.
* locale.c: Move #define to perl.h; use it elsewhereKarl Williamson2022-09-091-1/+6
| | | | | Rather than recalculate this combined conditional, do it once in perl.h.
* Change some HAS_QUERYLOCALE to USE_QUERYLOCALEKarl Williamson2022-09-091-1/+1
| | | | | | The HAS_QUERYLOCALE comes from config.h Just because we have this doesn't mean we're going to use it. That is computed in perl.h, and other places should use its determination.
* Move utf8ness calc for $! into locale.c from mg.cKarl Williamson2022-09-021-1/+1
| | | | | | | | | | | locale.c has the infrastructure to handle this, so remove repeated logic. The removed code tried to discern better based on using script runs, but this actually doesn't help, so is removed. Since we're now using C99, we can remove the block that was previously needed, and now the code is properly indented, whereas before it wasn't
* locale.c: Cache the current LC_CTYPE locale nameKarl Williamson2022-08-311-0/+6
| | | | | | | | This is now used as a cache of length 1 to avoid having to lookup up the UTF-8ness as often. This commit also skips doing S_newctype() if the new boss is the same as the old
* makedef.pl: Rmv obsolete symbolKarl Williamson2022-08-241-1/+0
| | | | Spotted by Bram
* makedef: Export certain symbolsKarl Williamson2022-08-191-5/+0
| | | | | | | | This fixes #20090 These symbols were changed by fcbef0e to be defined in more Configurations, but as @bram-perl pointed out, makedef.pl didn't also change to export them in those expanded situations.
* locale.c: Add fcn to hide edge case undefined behaviorKarl Williamson2022-08-091-1/+7
| | | | | | | | | | | | The POSIX 2008 API has an edge case in that the result of most of the functions when called with a global (as opposed to a per-thread) locale is undefined. The duplocale() function is the exception which will create a per-thread locale containing the values copied from the global one. This commit just calls duplocale, if needed, and the caller need not concern itself with this possibility
* Regularize HAS_POSIX_2008_LOCALE, USE_POSIX_2008_LOCALEKarl Williamson2022-06-161-10/+0
| | | | | | | | | A platform shouldn't be required to use the Posix 2008 locale handling functions if they are present. Perhaps they are buggy. So, a separate define for using them was introduced, USE_POSIX_2008_LOCALE. But until this commit there were cases that were looking at the underlying availability of the functions, not if the Configuration called for their use.
* Add USE_LOCALE_THREADS #defineKarl Williamson2022-06-161-3/+5
| | | | | | | | This is in preparation for supporting configurations where there threads are available, but the locale handling code should ignore that fact. This stems from the unusual locale handling of z/OS, where any attempt is ignored to change locales after the first thread is created.
* allow building with -DPERL_MEM_LOG on Win32Tony Cook2022-06-161-1/+8
| | | | | | | | This appears to have been broken for a while, and became more broken with 75acd14e, which made newSV_type() inline. This will also prevent warnings about calls to undeclared functions on systems that don't need symbols to be exported.
* regen/embed.pl: Add static Perl_ not_inlined capabilityKarl Williamson2022-06-141-2/+2
| | | | This will be used in the next commit
* make PERL_USE_SAFE_PUTENV the default and the only optionTomasz Konojacki2022-05-291-4/+0
| | | | | | | Now environ isn't owned by Perl and calling setenv/putenv in XS code will no longer result in memory corruption. Fixes #19399
* make autodoc.pl and makedef.pl understand the I embed.fnc flagTomasz Konojacki2022-02-041-1/+1
| | | | | It's needed in case a publicly visible PERL_STATIC_FORCE_INLINE function is added.
* Remove NetWare supportDagfinn Ilmari Mannsåker2021-10-081-156/+3
| | | | The build has been broken since 2009.
* Move `use strict;` *out* of the BEGIN block in makedef.plNicholas Clark2021-09-291-3/+3
| | | | | | | | | `use strict;` was moved into the BEGIN block in 2015 as part of commit 196f7c461ba85005: export data symbols as data on Win32 This mistake wasn't spotted at the time, and meant that only the code within the BEGIN block was subject to strictures.
* The Makefile must run makedef.pl with -Ilib on the command lineNicholas Clark2021-09-291-1/+0
| | | | | | | | | | | | | | | | | It can't rely on adding 'lib' to @INC in a BEGIN block because during the build not all modules *are* in lib yet. For miniperl the two are not equivalent - adding -Ilib means that lib/buildcustomize.pl is run, and this sets up @INC properly to include all the toolchain modules in dist and cpan. In particular, constant is dual life and hence not available as lib/constant.pm until it has been copied from dist/constant/lib as part of the build process. Hence with a parallel build on AIX, before this commit there was a race condition between the job that builds dist/constant and the job that builds perl.exp. Sometimes the race was lost, resulting in a mysterious failed build. All other users of makedef.pl already invoke it with the appropriate -Ilib or -I..\lib, so this change won't affect them.
* MACH-O can't support C11 thread local in shared objects, so disable it.Nicholas Clark2021-09-071-1/+2
| | | | | | | | | | | | | | Apple's clang happily passes the probe and compiles _Thread_local no problem, but building extensions fails with ld: illegal thread local variable reference to regular symbol _PL_current_context for architecture arm64 The Internet suggests that the MACH-O format fundamentally can't support what is needed to implement C11 thread local storage in shared objects. It's frustrating that the error message is "Less Than Awesome" at explaining that this is the real problem here. Hence disable the use of C11 thread local storage in the hints file, and hence keep using pthreads.
* If we have thread local storage, use it instead of posix_getspecific().Nicholas Clark2021-09-071-0/+3
| | | | | | | | | | | | Declare and use a variable PL_current_context to store a pointer to the thread's interpreter struct. Use this to implement implicit context undef ITHREADS, instead of a call to posix_getspecific(). For normal threaded code, this will eliminate function calls. For threaded code in shared objects (the typical configuration for Linux distribution builds) there still needs to be a function call to get the pointer to thread local variables, but there will only be one per function, not one per read of the variable.
* replace all instances of PERL_IMPLICIT_CONTEXT with MULTIPLICITYTomasz Konojacki2021-06-091-3/+3
| | | | | | | | | | | | Since the removal of PERL_OBJECT (acfe0abcedaf592fb4b9cb69ce3468308ae99d91) PERL_IMPLICIT_CONTEXT and MULTIPLICITY have been synonymous and they're being used interchangeably. To simplify the code, this commit replaces all instances of PERL_IMPLICIT_CONTEXT with MULTIPLICITY. PERL_IMPLICIT_CONTEXT will stay defined for compatibility with XS modules.
* export the win32 symlink related symbolsTony Cook2021-01-051-0/+3
| | | | | | | I couldn't make the build fail due to these missing exports while I was developing the win32 symlink() support, but I did have it fail while testing an unrelated change, and reproduced it, so export them.
* Fix broken PERL_MEM_LOG under threadsKarl Williamson2020-12-191-0/+4
| | | | | | | | | | | | | This fixes GH #18341 There are problems with getenv() on threaded perls wchich can lead to incorrect results when compiled with PERL_MEM_LOG. Commit 0b83dfe6dd9b0bda197566adec923f16b9a693cd fixed this for some platforms, but as Tony Cook, pointed out there may be standards-compliant platforms that that didn't fix. The detailed comments outline the issues and (complicated) full solution.
* Refactor locale mutex setupKarl Williamson2020-12-081-3/+0
| | | | | | | | | | | | | | | | | | | | | | | This was prompted by my realization that even on a locale thread-safe platform, there are functions we call that may not be thread-safe in that they return their results in an internal static buffer, which may be process-wide instead of per-thread. Tomasz Konojacki++ briefly looked at Windows source code for localeconv() and this indeed did appear to be the case. If we thought a platform was thread-safe, no locale mutexes were set up, and instead the calls in the code to lock were no-oops. This would lead to potential races, the most likely candidate being localeconv(). None have been reported, at least as far as we know. Likely that function isn't called frequently. This would be true on both Posix 2008 and Windows platforms, except possibly for FreeBSD, which may be the only platform that we support that has a localeconv_l() function, which is supposed to be immune from this issue.. The solution adopted here is to test for all the possible functions that the Perl core uses that may be susceptible to this, and to set up the mutex if any are found. Thus there won't be no-ops where there should be a lock.
* Remove obsolete FCRYPT ifdefs and associated PL_cryptseen (#17624)Richard Leach2020-07-301-1/+0
| | | Co-authored-by: Karl Williamson <khw@cpan.org>
* Remove PL_appctx and PL_timesbase interpreter variablesDagfinn Ilmari Mannsåker2020-07-201-8/+0
| | | | These were only ever needed by Symbian.
* Remove PERL_GLOBAL_STRUCTDagfinn Ilmari Mannsåker2020-07-201-39/+3
| | | | | | | | This was originally added for MinGW, which no longer needs it, and only still used by Symbian, which is now removed. This also leaves perlapi.[ch] empty, but we keep the header for CPAN backwards compatibility.
* Remove the final remnants of 5005threads supportDagfinn Ilmari Mannsåker2020-07-201-30/+0
| | | | Only THREAD_RET_TYPE is still used.
* Fix a bunch of repeated-word typosDagfinn Ilmari Mannsåker2020-05-221-1/+1
| | | | | Mostly in comments and docs, but some in diagnostic messages and one case of 'or die die'.
* don't export PL_mbrlen_ps, PL_mbrtowc, PL_wcrtomb if not definedTony Cook2020-03-271-0/+12
| | | | | | | | | | | | | | These variables are only defined when the appropriate symbols are defined in config.h, which are controled by their respective names in config.sh/%Config. Note that the earlier code in makedef.pl only defines the intrpvar.h symbols for non-multiplicity builds, so there's no need to special case this check for threaded/multiplicity builds where these symbols are never defined. A sufficiently modern MSVC CRT does turn out to define these symbols, but fixing this error first.
* Add mutex for accessing ENVKarl Williamson2020-03-111-0/+1
|
* makedef.pl: Add commentKarl Williamson2020-03-021-0/+1
|
* The VC6 Chainsaw MassacreSteve Hay2019-10-171-1/+1
| | | | | Remove MS Visual C++ 6.0 support as agreed in the thread starting here: https://www.nntp.perl.org/group/perl.perl5.porters/2019/07/msg255625.html
* makedef.pl: export symbols flagged as CCraig A. Berry2019-09-201-1/+1
| | | | | | | | | | 0923255565af0741 broke the build on any platform that does not promiscuously export all symbols from a shared library because it moved a bunch of symbols from being API (A flag) to core-only (C flag), but the new C flag was unknown to the program that generates the export list. Some of these functions are already used outside the core (such as by the Encode extension) so we can't simply pull the plug on them.
* The Windows CE Chainsaw MassacreSteve Hay2019-06-181-183/+168
| | | | | Remove WinCE support as agreed in the thread starting here: https://www.nntp.perl.org/group/perl.perl5.porters/2018/07/msg251683.html
* embed.fnc: Change x flag to eKarl Williamson2019-05-301-1/+1
| | | | | | | | | | | | autodoc.pl has the x flag in the '=for apidoc' lines mean something completely different, but Devel::PPPort combines both these lines and the contents of embed.fnc into one file, expecting the flags to be consistent. This commit changes the 'x' in embed.fnc to be 'e', which means to not export this function. Changing embed.fnc and makedef.pl keeps the changes more contained, and lessens the chance that some module will be adversely affected (I didn't see anything likely in grepping cpan).
* Use POSIX locale functions if setlocale not availableKarl Williamson2019-03-061-21/+32
| | | | | | POSIX 2008 added an independent set of locale handling functions beyond the venerable setlocale. This commit changes so they can be used rather than have no locale handling if there is, say, a problem with setlocale.
View Lorry Controller Status