unilbrk: Improve support for strings that contain CR-LF sequences.

* lib/gen-uni-tables.c (LBP_CR, LBP_LF): New enum values.
(get_lbp): For '\r' and '\n', return LBP_CR and LBP_LF, respectively,
instead of LBP_BK.
(debug_output_lbp, fill_org_lbp, debug_output_org_lbp,
lbp_value_to_string): Handle LBP_CR, LBP_LF.
* lib/unilbrk/lbrkprop2.h: Regenerated.
* lib/unilbrk/lbrktables.h (LBP_CR, LBP_LF): New enum values.
* lib/unilbrk.in.h (UC_BREAK_CR_BEFORE_LF): New enum value.
(u8_possible_linebreaks, u16_possible_linebreaks,
u32_possible_linebreaks, ulc_possible_linebreaks, u8_width_linebreaks,
u16_width_linebreaks, u32_width_linebreaks, ulc_width_linebreaks):
Declare also a _v2 variant, and use a C macro to activate it by default.
* lib/unilbrk/internal.h: New file.
* lib/unilbrk/u8-possible-linebreaks.c: Include unilbrk/internal.h.
(u8_possible_linebreaks_loop): Renamed from u8_possible_linebreaks. Add
'cr' parameter.
(u8_possible_linebreaks, u8_possible_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u16-possible-linebreaks.c: Include unilbrk/internal.h.
(u16_possible_linebreaks_loop): Renamed from u16_possible_linebreaks.
Add 'cr' parameter.
(u16_possible_linebreaks, u16_possible_linebreaks_v2): New functions.
* lib/unilbrk/u32-possible-linebreaks.c: Include unilbrk/internal.h.
(u32_possible_linebreaks_loop): Renamed from u32_possible_linebreaks.
Add 'cr' parameter.
(u32_possible_linebreaks, u32_possible_linebreaks_v2): New functions.
* lib/unilbrk/ulc-possible-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(ulc_possible_linebreaks_internal): Renamed from
ulc_possible_linebreaks. Add 'cr' parameter.
(ulc_possible_linebreaks, ulc_possible_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u8-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u8_width_linebreaks_internal): Renamed from u8_width_linebreaks. Add
'cr' parameter.
(u8_width_linebreaks, u8_width_linebreaks_v2): New functions.
(main): Update.
* lib/unilbrk/u16-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u16_width_linebreaks_internal): Renamed from u16_width_linebreaks. Add
'cr' parameter.
(u16_width_linebreaks, u16_width_linebreaks_v2): New functions.
* lib/unilbrk/u32-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(u32_width_linebreaks_internal): Renamed from u32_width_linebreaks. Add
'cr' parameter.
(u32_width_linebreaks, u32_width_linebreaks_v2): New functions.
* lib/unilbrk/ulc-width-linebreaks.c: Include unilbrk/internal.h,
unilbrk/lbrktables.h.
(ulc_width_linebreaks_internal): Renamed from ulc_width_linebreaks. Add
'cr' parameter.
(ulc_width_linebreaks, ulc_width_linebreaks_v2): New functions.
(main): Update.
* modules/unilbrk/u8-possible-linebreaks (Files): Add
lib/unilbrk/internal.h.
* modules/unilbrk/u16-possible-linebreaks (Files): Likewise.
* modules/unilbrk/u32-possible-linebreaks (Files): Likewise.
* modules/unilbrk/u8-width-linebreaks (Files): Likewise.
* tests/unilbrk/test-u8-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-u16-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-u32-possible-linebreaks.c (test_function): New
function, extracted from main. Add a test of CR-LF handling.
(main): Invoke it twice.
* tests/unilbrk/test-ulc-possible-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u8-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u16-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-u32-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.
* tests/unilbrk/test-ulc-width-linebreaks.c (test_function): New
function, extracted from main.
(main): Invoke it twice.

1 files changed, 87 insertions, 0 deletions

diff --git a/ChangeLog b/ChangeLog
index 06ace37069..7af2943f84 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,92 @@
 2021-12-29  Bruno Haible  <bruno@clisp.org>
 
+	unilbrk: Improve support for strings that contain CR-LF sequences.
+	* lib/gen-uni-tables.c (LBP_CR, LBP_LF): New enum values.
+	(get_lbp): For '\r' and '\n', return LBP_CR and LBP_LF, respectively,
+	instead of LBP_BK.
+	(debug_output_lbp, fill_org_lbp, debug_output_org_lbp,
+	lbp_value_to_string): Handle LBP_CR, LBP_LF.
+	* lib/unilbrk/lbrkprop2.h: Regenerated.
+	* lib/unilbrk/lbrktables.h (LBP_CR, LBP_LF): New enum values.
+	* lib/unilbrk.in.h (UC_BREAK_CR_BEFORE_LF): New enum value.
+	(u8_possible_linebreaks, u16_possible_linebreaks,
+	u32_possible_linebreaks, ulc_possible_linebreaks, u8_width_linebreaks,
+	u16_width_linebreaks, u32_width_linebreaks, ulc_width_linebreaks):
+	Declare also a _v2 variant, and use a C macro to activate it by default.
+	* lib/unilbrk/internal.h: New file.
+	* lib/unilbrk/u8-possible-linebreaks.c: Include unilbrk/internal.h.
+	(u8_possible_linebreaks_loop): Renamed from u8_possible_linebreaks. Add
+	'cr' parameter.
+	(u8_possible_linebreaks, u8_possible_linebreaks_v2): New functions.
+	(main): Update.
+	* lib/unilbrk/u16-possible-linebreaks.c: Include unilbrk/internal.h.
+	(u16_possible_linebreaks_loop): Renamed from u16_possible_linebreaks.
+	Add 'cr' parameter.
+	(u16_possible_linebreaks, u16_possible_linebreaks_v2): New functions.
+	* lib/unilbrk/u32-possible-linebreaks.c: Include unilbrk/internal.h.
+	(u32_possible_linebreaks_loop): Renamed from u32_possible_linebreaks.
+	Add 'cr' parameter.
+	(u32_possible_linebreaks, u32_possible_linebreaks_v2): New functions.
+	* lib/unilbrk/ulc-possible-linebreaks.c: Include unilbrk/internal.h,
+	unilbrk/lbrktables.h.
+	(ulc_possible_linebreaks_internal): Renamed from
+	ulc_possible_linebreaks. Add 'cr' parameter.
+	(ulc_possible_linebreaks, ulc_possible_linebreaks_v2): New functions.
+	(main): Update.
+	* lib/unilbrk/u8-width-linebreaks.c: Include unilbrk/internal.h,
+	unilbrk/lbrktables.h.
+	(u8_width_linebreaks_internal): Renamed from u8_width_linebreaks. Add
+	'cr' parameter.
+	(u8_width_linebreaks, u8_width_linebreaks_v2): New functions.
+	(main): Update.
+	* lib/unilbrk/u16-width-linebreaks.c: Include unilbrk/internal.h,
+	unilbrk/lbrktables.h.
+	(u16_width_linebreaks_internal): Renamed from u16_width_linebreaks. Add
+	'cr' parameter.
+	(u16_width_linebreaks, u16_width_linebreaks_v2): New functions.
+	* lib/unilbrk/u32-width-linebreaks.c: Include unilbrk/internal.h,
+	unilbrk/lbrktables.h.
+	(u32_width_linebreaks_internal): Renamed from u32_width_linebreaks. Add
+	'cr' parameter.
+	(u32_width_linebreaks, u32_width_linebreaks_v2): New functions.
+	* lib/unilbrk/ulc-width-linebreaks.c: Include unilbrk/internal.h,
+	unilbrk/lbrktables.h.
+	(ulc_width_linebreaks_internal): Renamed from ulc_width_linebreaks. Add
+	'cr' parameter.
+	(ulc_width_linebreaks, ulc_width_linebreaks_v2): New functions.
+	(main): Update.
+	* modules/unilbrk/u8-possible-linebreaks (Files): Add
+	lib/unilbrk/internal.h.
+	* modules/unilbrk/u16-possible-linebreaks (Files): Likewise.
+	* modules/unilbrk/u32-possible-linebreaks (Files): Likewise.
+	* modules/unilbrk/u8-width-linebreaks (Files): Likewise.
+	* tests/unilbrk/test-u8-possible-linebreaks.c (test_function): New
+	function, extracted from main. Add a test of CR-LF handling.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-u16-possible-linebreaks.c (test_function): New
+	function, extracted from main. Add a test of CR-LF handling.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-u32-possible-linebreaks.c (test_function): New
+	function, extracted from main. Add a test of CR-LF handling.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-ulc-possible-linebreaks.c (test_function): New
+	function, extracted from main.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-u8-width-linebreaks.c (test_function): New
+	function, extracted from main.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-u16-width-linebreaks.c (test_function): New
+	function, extracted from main.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-u32-width-linebreaks.c (test_function): New
+	function, extracted from main.
+	(main): Invoke it twice.
+	* tests/unilbrk/test-ulc-width-linebreaks.c (test_function): New
+	function, extracted from main.
+	(main): Invoke it twice.
+
+2021-12-29  Bruno Haible  <bruno@clisp.org>
+
 	uniwbrk: Add unit tests of CR-LF sequences.
 	* tests/uniwbrk/test-u8-wordbreaks.c (main): Add test of a string with
 	CR-LF.