UTF-X encoding invariance for Encode:

 - move Encode::utf8_encode to utf8::encode (likewise decode,upgrade,downgrade,valid)
 - move the XS code for those to universal.c (so in miniperl)
 - add utf8::unicode_to_native and its inverse to allow EBCDIC to work in true unicode.
 - change ext/Encode/compile to use above.
 - Fix t/lib/encode.t for above
 - Teach t/lib/b.t to expect -uutf8
 - In utf8.c look for SWASHNEW rather than just utf8:: package to see if
   utf8.pm is needed.

p4raw-id: //depot/perlio@9198

1 files changed, 27 insertions, 1 deletions

diff --git a/lib/utf8.pm b/lib/utf8.pm
index f9055b5dd9..7c9a7dfb4e 100644
--- a/lib/utf8.pm
+++ b/lib/utf8.pm
@@ -52,7 +52,7 @@ source text.  Until UTF-8 becomes the default format for source
 text, this pragma should be used to recognize UTF-8 in the source.
 When UTF-8 becomes the standard source format, this pragma will
 effectively become a no-op.  This pragma already is a no-op on
-EBCDIC platforms (where it is alright to code perl in EBCDIC 
+EBCDIC platforms (where it is alright to code perl in EBCDIC
 rather than UTF-8).
 
 Enabling the C<utf8> pragma has the following effects:
@@ -81,6 +81,32 @@ of byte semantics.
 
 =back
 
+=head2 Utility functions
+
+The following functions are defined in the C<utf8::> package by the perl core.
+
+=over 4
+
+=item * $num_octets = utf8::upgrade($string);
+
+Converts internal representation of string to the perls internal UTF-X form.
+Returns the number of octets necessary to represent the string as UTF-X.
+
+=item * utf8::downgrade($string[, CHECK])
+
+Converts internal representation of string to be un-encoded bytes.
+
+=item * utf8::encode($string)
+
+Converts (in-place) I<$string> from logical characters to octet sequence
+representing it in perl's UTF-X encoding.
+
+=item * $flag = utf8::decode($string)
+
+Attempts to converts I<$string> in-place from perl's UTF-X encoding into logical characters.
+
+=back
+
 =head1 SEE ALSO
 
 L<perlunicode>, L<bytes>