From 6b659339f976d014a1a53731d86cedd01f5921ec Mon Sep 17 00:00:00 2001 From: Karl Williamson Date: Mon, 18 Jan 2016 14:25:02 -0700 Subject: Add qr/\b{lb}/ This adds the final Unicode boundary type previously missing from core Perl: the LineBreak one. This feature is already available in the Unicode::LineBreak module, but I've been told that there are portability and some other issues with that module. What's added here is a light-weight version that is lacking the customizable features of the module. This implements the default Line Breaking algorithm, but with the customizations that Unicode is expecting everybody to add, as their test file tests for them. In other words, this passes Unicode's fairly extensive furnished tests, but wouldn't if it didn't include certain customizations specified by Unicode beyond the basic algorithm. The implementation uses a look-up table of the characters surrounding a boundary to see if it is a suitable place to break a line. In a few cases, context needs to be taken into account, so there is code in addition to the lookup table to handle those. This should meet the needs for line breaking of many applications, without having to load the module. The algorithm is somewhat independent of the Unicode version, just like the other boundary types. Only if new rules are added, or existing ones modified is there need to go in and change this code. Otherwise, running regen/mk_invlists.pl should be sufficient when a new Unicode release is done to keep it up-to-date, again like the other Unicode boundary types. --- embed.fnc | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'embed.fnc') diff --git a/embed.fnc b/embed.fnc index a35bb51485..5acc2fc217 100644 --- a/embed.fnc +++ b/embed.fnc @@ -2305,6 +2305,18 @@ Es |bool |to_byte_substr |NN regexp * prog ERsn |I32 |reg_check_named_buff_matched |NN const regexp *rex \ |NN const regnode *scan EsnR |bool |isGCB |const GCB_enum before|const GCB_enum after +EsR |bool |isLB |LB_enum before \ + |LB_enum after \ + |NN const U8 * const strbeg \ + |NN const U8 * const curpos \ + |NN const U8 * const strend \ + |const bool utf8_target +EsR |LB_enum|advance_one_LB |NN U8 ** curpos \ + |NN const U8 * const strend \ + |const bool utf8_target +EsR |LB_enum|backup_one_LB |NN const U8 * const strbeg \ + |NN U8 ** curpos \ + |const bool utf8_target EsR |bool |isSB |SB_enum before \ |SB_enum after \ |NN const U8 * const strbeg \ -- cgit v1.2.1