summaryrefslogtreecommitdiff
path: root/utfebcdic.h
diff options
context:
space:
mode:
authorKarl Williamson <khw@cpan.org>2014-12-31 16:49:35 -0700
committerKarl Williamson <khw@cpan.org>2015-03-05 22:13:14 -0700
commitff982d0036bfe69b869b03451f0561fa790d4378 (patch)
tree7ca3eda23d8d523c9f31d94fc425235cfa1d67b2 /utfebcdic.h
parentb2a691af8242035dab97bd4c05b77ec3ad88955b (diff)
downloadperl-ff982d0036bfe69b869b03451f0561fa790d4378.tar.gz
utfebcdic.h: Add comment
Diffstat (limited to 'utfebcdic.h')
-rw-r--r--utfebcdic.h14
1 files changed, 14 insertions, 0 deletions
diff --git a/utfebcdic.h b/utfebcdic.h
index 23b4ae2014..a4f5fb5cb2 100644
--- a/utfebcdic.h
+++ b/utfebcdic.h
@@ -86,6 +86,20 @@
*
* tr16 also says that NEL and LF be swapped. We don't do that.
*
+ * Because of the way UTF-EBCDIC is constructed, the lowest 32 code points that
+ * aren't equivalent to ASCII characters nor C1 controls form the set of
+ * continuation bytes; the remaining 64 non-ASCII, non-control code points form
+ * the potential start bytes, in order. (However, the first 5 of these lead to
+ * malformed overlongs, so there really are only 59 start bytes.) Hence the
+ * UTF-EBCDIC for the smallest variant code point, 0x160, will have likely 0x41
+ * as its continuation byte, provided 0x41 isn't an ASCII or C1 equivalent.
+ * And its start byte will be the code point that is 37 (32+5) non-ASCII,
+ * non-control code points past it. (0 - 3F are controls, and 40 is SPACE,
+ * leaving 41 as the first potentially available one.) In contrast, on ASCII
+ * platforms, the first 64 (not 32) non-ASCII code points are the continuation
+ * bytes. And the first 2 (not 5) potential start bytes form overlong
+ * malformed sequences.
+ *
* EBCDIC characters above 0xFF are the same as Unicode in Perl's
* implementation of all 3 encodings, so for those Step 1 is trivial.
*