| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
Static analysis tools such as Coverity and clang report that we can
otherwise end up reading uninitialized data, and inspection agrees.
|
|
|
|
| |
Clearer to use the more direct operation
|
| |
|
|
|
|
| |
It's better to use a %z modifier than to cast the operand.
|
| |
|
|
|
|
|
|
| |
Later gcc compilers warn on these intentional fall throughs in a
switch(). Adding FALLTHROUGH would make things a lot less clear, so
turn off the warnings around the whole switch.
|
|
|
|
|
|
| |
This code was left in for a potential guide for the future. Now that
I'm getting round to implementing it, I'm going in a different
direction.
|
|
|
|
|
|
|
|
|
|
|
| |
This just detabifies to get rid of the mixed tab/space indentation.
Applying consistent indentation and dealing with other tabs are another issue.
Done with `expand -i`.
* vutil.* left alone, it's part of version.
* Left regen managed files alone for now.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Without per-thread locales, a multi-thread application is inherently
unsafe. IBM solves that by allowing you to set up the locale any way
you want, but after you've created a thread, all future locale changes
are ignored, and return failure.
But Perl itself changes the locale in a couple of cases. Recent changes
have surfaced this issue in one case, leading to a panic. And this
commit works around it, so that messages will be displayed in the locale
in effect before the threads were created.
The remaining case requires further investigation. Nothing in our suite
is failing.
|
|
|
|
|
|
|
|
|
| |
These aren't necessary as the called function has its own lock until
done copying into the local structure.
And these were breaking blead on Windows, as they are no longer defined.
The smoke I ran included more commits beyond the breaking one, so I
didn't catch it.
|
|
|
|
|
| |
These cases aren't supposed to happen, but unlock the mutex first;
we could get into deadlock in trying to output the death message.
|
|
|
|
|
|
|
|
|
| |
These locks for different functions all use the same underlying mutex;
but that may not always be the case. By creating separate names
used only when we think they will be necessary, the compiler will
complain if the conditions in the code that actually use them are the
same. Doing this showed a misspelling in an #ifdef, fixed in
9289d4dc7a3d24b20c6e25045e687321ee3e8faf
|
|
|
|
|
| |
This macro is for localeconv(); the new name is clearer as to the
meaning, and this preps for further changes.
|
| |
|
|
|
|
|
| |
Thus if we know we have the Posix versions, we have duplocale(), and
hence don't need to test separately for it.
|
|
|
|
|
| |
This misspelling led to the code assuming that the platform didn't have
a feature that, if used, would result in faster execution.
|
| |
|
|
|
|
|
|
|
|
| |
Many of the files in perl are for one thing only, and hence their
embedded documentation will be for that one thing. By creating a hash
here of them, those files don't have to worry about what section that
documentation goes under, and so it can be completely changed without
affecting them.
|
|
|
|
|
|
|
|
|
| |
For: https://github.com/Perl/perl5/pull/18201
Committer: Samanta Navarro is now a Perl author.
To keep 'make test_porting' happy: Increment $VERSION in several files.
Regenerate uconfig.h via './perl -Ilib regen/uconfig_h.pl'.
|
|
|
|
|
| |
This uses a new organization of sections that I came up with. I asked
for comments on p5p, but there were none.
|
|
|
|
|
| |
It only does anything under PERL_GLOBAL_STRUCT, which is gone.
Keep the dNOOP defintion for CPAN back-compat
|
|
|
|
|
|
|
|
| |
This was originally added for MinGW, which no longer needs it, and
only still used by Symbian, which is now removed.
This also leaves perlapi.[ch] empty, but we keep the header for CPAN
backwards compatibility.
|
|
|
|
|
|
|
| |
z/OS has two locale categories, LC_SYNTAX and LC_TOD, not found outside
IBM products. This makes Perl know about them, so that a program can
refer to them, but like other similar categories found on other OS's,
nothing more is done with them.
|
|
|
|
| |
Coverity CID 288709
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The previous commit added a mutex specifically for protecting against
simultaneous accesses of the environment. This commit changes the
normal getenv, putenv, and clearenv functions to use it, to avoid races.
This makes the code simpler in places where we've gotten burned and
added stuff to avoid races. Other places where we haven't known we were
getting burned could have existed until now. Now that comes
automatically, and we can remove the special cases we earlier stumbled
over.
getenv() returns a pointer to static memory, which can be overwritten at
any moment from another thread, or even another getenv from the same
thread. This commit changes the accesses to be under control of a
mutex, and in the case of getenv, a mortalized copy is created so that
there is no possible race.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit enhances these functions so that on threaded perls, they use
mbrtowc and wcrtomb when available, making them thread safe. The
substitution isn't completely transparent, as no effort is made to hide
any differences in errno setting upon error. And there may be slight
differences in edge case behavior on some platforms.
This commit also changes the behaviors so that they take a scalar
parameter instead of a char *, and this might be 'undef' or not be
forceable into a valid PV. If not a PV, the functions initialize the
shift state. Previously the shift state was always reinitialized with
every call, which meant these could not work on locales with shift
states.
In addition, there were several issues in mbtowc and wctomb that this
commit fixes.
mbtowc and wctomb, when used, are now run with a semaphore. This avoids
races if called at the same time in another thread.
The returned wide character from mbtowc() could well have been garbage.
The final parameter to mbtowc is now optional, as passing an SV allows
us to determine the length without the need for an extra parameter. It
is now used only to restrict the parsing of the string to shorter than
the actual length.
wctomb would segfault if the string parameter was shared or hadn't
been pre-allocated with a string of sufficient length to hold the
result.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit changes the behavior so that it takes a scalar parameter
instead of a char *, and thus might not be forceable into a valid PV.
When not a PV, the shift state is reinitialized, like calling mblen with
a NULL first parameter. Previously the shift state was always
reinitialized with every call, which meant this could not work on
locales with shift states.
This commit also changes to use mbrlen() on threaded perls transparently
(mostly), when available, to achieve thread-safe operation. It is not
completely transparent because mbrlen (under the very rare stateful
locales) returns a different value when it's resetting the shift state.
It also may set errno differently upon errors, and no effort is made to
hide that difference. Also mbrlen on some platforms can handle partial
characters.
[perl #133928] showed that someone was having trouble with shift states.
|
|
|
|
|
|
|
| |
This changes to use USE_POSIX_2008_LOCALE instead of
HAS_POSIX_2008_LOCALE. Rarely do they differ, but someone may choose to
configure their installation to not use these more modern functions,
even if available, perhaps because they're buggy on that system.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please see the ticket for a full explanation. This bug has been
submitted to glibc, without any real action forthcoming so far.
This invalidates the message cache each time the locale of LC_MESSAGES
is changed, as glibc should be doing this when uselocale changes that,
but glibc fails to do so.
This patch is an extension to the one submitted by Niko Tyni++.
I don't know how to test it, since a test would rely on several
different locales in different languages being available, and that
depends on what's installed on the platform. I suppose that one could
go through the available locales, and try to find three with different
wording for the same message. Doing so however would trigger the bug,
and at the end, if we didn't get three that differed, we wouldn't know
we wouldn't know if it is because of the bug, or that they just didn't
exist on the system.
However, below is a perl program that demonstrated the patch worked.
You could adjust it to the available locales. The buggy code shows the
same text for all locales. The fixed shows three different languages.
use strict;
use Locale::gettext;
use POSIX;
$ENV{LANG} = 'C.UTF-8';
for my $lang (qw(fi_FI fr_FR en_US)) {
$ENV{LANGUAGE} = $lang;
setlocale(LC_MESSAGES, '');
my $d = Locale::gettext->domain("bash");
print $d->get('syntax error'), "\n";
}
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This appears to abort because the supplied locale string isn't validly
encoded in the current code page, so we see the following steps:
1) an internal sizing call to mbstowcs_s() fails, but
2) the calling (CRT) code doesn't handle that, allocating a zero length
buffer
3) mbstowcs_s() is called with a buffer and a zero size, causing the
exception.
Since it's the conversion that fails, perform our own conversion.
Rather than using the current code page always use CP_UTF8, since
this is perl's typical non-Latin1 encoding.
Unfortunately we don't have the SVf_UTF8 flag at this point, so
all we can do is assume UTF-8.
This introduces a change in behaviour - previously locale names
were interpreted in the current code page, but most locale names
are ASCII, so it shouldn't matter.
One issue is that the return value is freed on the next LEAVE, but
all callers immediately use or copy the string.
|
|
|
|
|
| |
Coverity is right, so re-order these clauses. This code is executed
only if some very strange error occurs.
|
|
|
|
| |
And regen affected files
|
|
|
|
| |
The wrong #define was being tested for
|
|
|
|
|
| |
This was just an oversight. THe code doesn't get executed unless it's
trying to panic
|
| |
|
|
|
|
|
|
| |
This was failing in gcc 2.95. The original commit added a cast, but we
figured out that removing this other one that really served no purpose
causes this compiler to work.
|
|
|
|
|
|
| |
Karl pointed that a couple of my recent commits used (lower case)
safefree() rather than Safefree(), the latter having extra debugging
facilities.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following leaked:
LANG= perl -e1
because in S_emulate_setlocale(), it was
1) making a copy of $ENV{"LANG"};
2) throwing that copy away and replacing it with "C" when it discovered
that the string was empty.
A little judicious reordering of that chunk of code makes the issue go
away.
Showed up as failures of lib/locale_threads.t under valgrind / ASan.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For example the following leaked:
require POSIX; import POSIX ':locale_h';
setlocale(&POSIX::LC_ALL, 'aa_DJ.iso88591') or die;
use locale;
my $ok = 'A' lt chr 0x100;
Some code in Perl__mem_collxfrm() does a couple of
for (j = 1; j < 256; j++) { ... }
loops where for each chr(j) character it recursively calls itself, and
records the index of the 'smallest' / 'largest' result. However, when
updating cur_min_x / cur_max_x, it wasn't freeing the previous value.
The symptoms were that valgrind / Address Sanitizer found fault with
lib/locale.t
|
| |
|
|
|
|
|
|
| |
It is possible to have a single-threaded build use the thread-safe
locale setting operations. Add a word to indicate it's not 100% the
other way.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit 70bd6bc82ba64c1d197d3ec823f43c4a454b2920 fixed a leak (likely due
to a bug in glibc) by not duplicating the C locale object. However,
that meant that there's only one copy running around. And freeing that
will cause havoc, as its supposed to be there until destruction. What
appears to be happening is that the current locale object is freed upon
thread destruction, and that could be this global one. But I don't
understand why it's only happening on Free BSD and only on this version.
But this commit fixes the problem there, and makes sense. Simply don't
free this global object upon thread destruction.
This commit also changes it so it doesn't get destroyed at destruction
time, leaving it to the final PERL_SYS_TERM to free. I'm not sure, but
I think this fixes any issues with embedded perls.
|
|
|
|
|
| |
Indent a block newly formed in the previous commit.
Wrap some too-long lines
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On threaded perls, we create a locale object for LC_ALL "C" early in the
startup phase. When the user asks for that locale, we can just switch
to it instead of trying to create a new one.
Doing the creation worked, but ended up with a memory leak. My guess,
and its only a guess, is that it's a bug in glibc newlocale.c, in which
it does an early return, not doing proper cleanup, when it discovers it
can re-use an existing locale without needing to create a new one.
The reason I think its a glibc bug is that the sample one-liner sent
to me
PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=full ./perl -DLv -Ilib -e'require POSIX;POSIX::setlocale(&POSIX::LC_ALL, "C");' 2>&1 | more
produced a stack output of where the leaked memory had been allocated.
I put a print immediately after that line, and prints at the points
where things get freed. Every allocation was matched by an attempt to
free it. But clearly at least one failed. freelocale() returns void,
so can't be checked for failing.
Anyway, it's better to try not to create a new locale when we already
have an existing one, and doing so, as this commit does, causes the leak
to go away.
No tests are added, as there are plenty of similar tests already in the
suite, and they all should have been leaking.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some systems fake their locales, so that they pretend to accept a locale
change, but they either do nothing, making everything the C locale, or
on some systems there is a a second locale "C-UTF-8" that can be
switched to. Configure probes have been added to find such systems, and
this commit changes to use the results of these probes, so that we don't
try looking for other locales (any names we came up with would be
accepted as valid, but don't work, and tests were failing as a result).
Anything running the musl library fits, as does OpenBSD and its kin, as
they view locales as security risks. This commit allows us to take out
some code that was looking for particular OS's.
|
|
|
|
|
| |
C99 has wide character case changing. If those are available, use them
to be surer we have a Turkic locale.
|
| |
|