integrate (by hand) #18353 and #18359 from maint-5.8:

Introduce a cache for UTF-8 data: length and byte<->char offset
mapping are stored in a new type of magic.  Speeds up length(),
substr(), index(), rindex(), pos(), and some parts of s///.

The speedup varies a lot (on the usual suspects: what is the
access pattern of the data, compiler, CPU), but should be at
least one order of magnitude, and getting to the same magnitude
as byte string speeds, and in some cases  (length on unchanged data)
even reaching the byte string speed.  On the other hand, in some
cases (index) the byte speed is still faster by a factor of five
or so, but the bottleneck there does not seem to be any more
the byte<->char offset mapping (instead, the fbm_instr() speed).

There is one cache slot for the length, and only two for the 
byte<->char offset mapping (the first one for the start->offset,
and the second for the offset->offset+length, when talking
in substr() terms).

Code this hairy is bound to have hairy trolls hiding under it.
[...]
A small tweak on top of #18353: don't display mg_len bytes of
mg_ptr for PERL_MAGIC_utf8 because that's not what's there.

p4raw-id: //depot/perl@18530

1 files changed, 10 insertions, 1 deletions

diff --git a/perl.h b/perl.h
index a3a6d10977..5c38aab117 100644
--- a/perl.h
+++ b/perl.h
@@ -2617,6 +2617,7 @@ Gid_t getegid (void);
 #define PERL_MAGIC_uvar_elem	  'u' /* Reserved for use by extensions */
 #define PERL_MAGIC_vec		  'v' /* vec() lvalue */
 #define PERL_MAGIC_vstring	  'V' /* SV was vstring literal */
+#define PERL_MAGIC_utf8		  'w' /* Cached UTF-8 information */
 #define PERL_MAGIC_substr	  'x' /* substr() lvalue */
 #define PERL_MAGIC_defelem	  'y' /* Shadow "foreach" iterator variable /
 					smart parameter vivification */
@@ -3205,7 +3206,8 @@ enum {		/* pass one of these to get_vtbl */
     want_vtbl_amagicelem,
     want_vtbl_regdata,
     want_vtbl_regdatum,
-    want_vtbl_backref
+    want_vtbl_backref,
+    want_vtbl_utf8
 };
 
 				/* Note: the lowest 8 bits are reserved for
@@ -3503,6 +3505,10 @@ EXT MGVTBL PL_vtbl_backref = 	  {0,	0,
 EXT MGVTBL PL_vtbl_ovrld   = 	  {0,	0,
 					0,	0,	MEMBER_TO_FPTR(Perl_magic_freeovrld)};
 
+EXT MGVTBL PL_vtbl_utf8 = {0,
+				MEMBER_TO_FPTR(Perl_magic_setutf8),
+					0,	0,	0};
+
 #else /* !DOINIT */
 
 EXT MGVTBL PL_vtbl_sv;
@@ -3541,6 +3547,7 @@ EXT MGVTBL PL_vtbl_amagic;
 EXT MGVTBL PL_vtbl_amagicelem;
 
 EXT MGVTBL PL_vtbl_backref;
+EXT MGVTBL PL_vtbl_utf8;
 
 #endif /* !DOINIT */
 
@@ -4160,6 +4167,8 @@ extern void moncontrol(int);
 #  define PIPESOCK_MODE
 #endif
 
+#define PERL_MAGIC_UTF8_CACHESIZE	2
+
 /* and finally... */
 #define PERL_PATCHLEVEL_H_IMPLICIT
 #include "patchlevel.h"