integrate (by hand) #18353 and #18359 from maint-5.8:

Introduce a cache for UTF-8 data: length and byte<->char offset
mapping are stored in a new type of magic.  Speeds up length(),
substr(), index(), rindex(), pos(), and some parts of s///.

The speedup varies a lot (on the usual suspects: what is the
access pattern of the data, compiler, CPU), but should be at
least one order of magnitude, and getting to the same magnitude
as byte string speeds, and in some cases  (length on unchanged data)
even reaching the byte string speed.  On the other hand, in some
cases (index) the byte speed is still faster by a factor of five
or so, but the bottleneck there does not seem to be any more
the byte<->char offset mapping (instead, the fbm_instr() speed).

There is one cache slot for the length, and only two for the 
byte<->char offset mapping (the first one for the start->offset,
and the second for the offset->offset+length, when talking
in substr() terms).

Code this hairy is bound to have hairy trolls hiding under it.
[...]
A small tweak on top of #18353: don't display mg_len bytes of
mg_ptr for PERL_MAGIC_utf8 because that's not what's there.

p4raw-id: //depot/perl@18530

1 files changed, 3 insertions, 0 deletions

diff --git a/proto.h b/proto.h
index 182867d62f..77ab0dffbd 100644
--- a/proto.h
+++ b/proto.h
@@ -449,6 +449,7 @@ PERL_CALLCONV int	Perl_magic_setsubstr(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV int	Perl_magic_settaint(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV int	Perl_magic_setuvar(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV int	Perl_magic_setvec(pTHX_ SV* sv, MAGIC* mg);
+PERL_CALLCONV int	Perl_magic_setutf8(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV int	Perl_magic_set_all_env(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV U32	Perl_magic_sizepack(pTHX_ SV* sv, MAGIC* mg);
 PERL_CALLCONV int	Perl_magic_wipepack(pTHX_ SV* sv, MAGIC* mg);
@@ -1249,6 +1250,8 @@ STATIC I32	S_expect_number(pTHX_ char** pattern);
 #  if defined(USE_ITHREADS)
 STATIC SV*	S_gv_share(pTHX_ SV *sv, CLONE_PARAMS *param);
 #  endif
+STATIC bool	S_utf8_mg_pos(pTHX_ SV *sv, MAGIC **mgp, STRLEN **cachep, I32 i, I32 *offsetp, I32 uoff, U8 **sp, U8 *start, U8 *send);
+STATIC bool	S_utf8_mg_pos_init(pTHX_ SV *sv, MAGIC **mgp, STRLEN **cachep, I32 i, I32 *offsetp, U8 *s, U8 *start);
 #if defined(PERL_COPY_ON_WRITE)
 STATIC void	S_sv_release_COW(pTHX_ SV *sv, char *pvx, STRLEN cur, STRLEN len, U32 hash, SV *after);
 #endif