tr///; simplify $utf8 =~ tr/nonutf8/nonutf8/

The run-time code to handle a non-utf8 tr/// against a utf8 string
is complex, with many variants of similar code repeated depending on the
presence of the /s and /c flags.

Simplify them all into a single code block by changing how the translation
table is stored. Formerly, the tr struct contained possibly two tables:
the basic 0-255 slot one, plus in the presence of /c, a second one
to map the implicit search range (\x{100}...) against any residual
replacement chars not consumed by the first table.

This commit merges the two tables into a single unified whole. For example

    tr/\x00-\xfe/abcd/c

is equivalent to

    tr/xff-\x{7fffffff}/abcd/

which generates a 259-entry translation table consisting of:

    0x00  => -1
    0x01  => -1
    ...
    0xfe  => -1
    0xff  =>  a
    0x100 =>  b
    0x101 =>  c
    0x102 =>  d

In addition we store:
    1) the size of the translation table (0x103 in the example above);
    2) an extra 'wildcard' entry stored 1 slot beyond the main table,
       which specifies the action for any codepoints outside the range of
       the table (i.e. chars 0x103..0x7fffffff). This can be either:
        a) a character, when the last replacement char is repeated;
        b) -1 when /c isn't in effect;
        c) -2 when /d is in effect;
        c) -3 identity: when the replacement list is empty but not /d.

       In the example above, this would be
            0x103 =>  d

The addition of -3 as a valid slot value is new.

This makes the main runtime code for the utf8 string with non-utf8 tr//
case look like, at its core:

    size = tbl->size;
    mapped_ch = tbl->map[ch >= size ? size : ch];

which then processes mapped_ch based on whether its >=0, or -1/-2/-3.

This is a lot simpler than the old scheme, and should generally be faster
too.

1 files changed, 3 insertions, 11 deletions

diff --git a/op.h b/op.h
index 5ba716751c..ed4ff9d1a7 100644
--- a/op.h
+++ b/op.h
@@ -628,21 +628,13 @@ typedef enum {
 #endif
 
 
-/* basic and extended translation tables attached to OP_TRANS/OP_TRANSR ops */
+/* translation table attached to OP_TRANS/OP_TRANSR ops */
 
 typedef struct {
-    short map[256];
+    Size_t size; /* number of entries in map[], not including final slot */
+    short map[1]; /* Unwarranted chumminess */
 } OPtrans_map;
 
-/* used in the presence of tr///c to record any replacement chars that
- * are paired with the implicit 0x100..0x7fffffff search chars */
-typedef struct {
-    short map[256];
-    SSize_t excess_len; /* number of entries in map_ex[] */
-    short repeat_char;
-    short map_ex[1];  /* Unwarranted chumminess */
-} OPtrans_map_ex;
-
 
 /*
 =head1 Optree Manipulation Functions