toke.c: Don't assume is UTF-8 when it might not be

This code only works by coincidence on ASCII platforms, due to the
chance ways the underlying UTF-8 is represented.  But it definitely
doesn't on EBCDIC.  Test before assuming is UTF-8.

1 files changed, 3 insertions, 1 deletions

diff --git a/toke.c b/toke.c
index 95de997d51..e60f94b52f 100644
--- a/toke.c
+++ b/toke.c
@@ -11449,7 +11449,9 @@ Perl_scan_str(pTHX_ char *start, int keep_bracketed_quoted, int keep_delims, int
                              UTF8fARG(UTF, delim_byte_len, open_delim_str));
         }
 
-        close_delim_code = valid_utf8_to_uvchr((U8 *) close_delim_str, NULL);
+        close_delim_code = (UTF)
+                           ? valid_utf8_to_uvchr((U8 *) close_delim_str, NULL)
+                           : * (U8 *) close_delim_str;
     }
     else {  /* Here, the delimiter isn't paired, hence the close is the same as
                the open; and has aready been set up.  But make sure it isn't