diff options
| author | Seiya Kawashima <skawashima@uchicago.edu> | 2015-07-03 11:42:43 +0900 |
|---|---|---|
| committer | Daiki Ueno <ueno@gnu.org> | 2015-07-03 11:48:29 +0900 |
| commit | 2c5543396b7c208b34fa714430801945aefb1fe8 (patch) | |
| tree | 29e07e39beaa65e470c567cc9a38aaa93e176ad9 /lib | |
| parent | 841d0dcfa8103235836a1aeb0c1ad657f43ade85 (diff) | |
| download | gnulib-2c5543396b7c208b34fa714430801945aefb1fe8.tar.gz | |
unistr/uN-strtok: handle multibyte delimiters
Previously, uN_strtok moved PTR to the next unit to the token end.
When DELIM contained a multibyte character, the new position could
be a middle of a multibyte character.
* lib/unistr/u-strtok.h (FUNC): Place PTR at the next character
after the token.
* lib/unistr/u8-strtok.c (U_STRMBLEN): New macro.
* lib/unistr/u16-strtok.c (U_STRMBLEN): New macro.
* lib/unistr/u32-strtok.c (U_STRMBLEN): New macro.
* modules/unistr/u8-strtok (Depends-on): Depend on
unistr/u8-strmblen.
* modules/unistr/u16-strtok (Depends-on): Depend on
unistr/u16-strmblen.
* modules/unistr/u32-strtok (Depends-on): Depend on
unistr/u32-strmblen.
* tests/unistr/test-u-strtok.h: New file.
* tests/unistr/test-u8-strtok.c: New file.
* tests/unistr/test-u16-strtok.c: New file.
* tests/unistr/test-u32-strtok.c: New file.
* modules/unistr/u8-strtok-tests: New file.
* modules/unistr/u32-strtok-tests: New file.
* modules/unistr/u16-strtok-tests: New file.
Copyright-paperwork-exempt: yes
Co-authored-by: Daiki Ueno <ueno@gnu.org>
Diffstat (limited to 'lib')
| -rw-r--r-- | lib/unistr/u-strtok.h | 2 | ||||
| -rw-r--r-- | lib/unistr/u16-strtok.c | 1 | ||||
| -rw-r--r-- | lib/unistr/u32-strtok.c | 1 | ||||
| -rw-r--r-- | lib/unistr/u8-strtok.c | 1 |
4 files changed, 4 insertions, 1 deletions
diff --git a/lib/unistr/u-strtok.h b/lib/unistr/u-strtok.h index 4cd68304dd..edafa1ba46 100644 --- a/lib/unistr/u-strtok.h +++ b/lib/unistr/u-strtok.h @@ -40,9 +40,9 @@ FUNC (UNIT *str, const UNIT *delim, UNIT **ptr) UNIT *token_end = U_STRPBRK (str, delim); if (token_end) { + *ptr = token_end + U_STRMBLEN (token_end); /* NUL-terminate the token. */ *token_end = 0; - *ptr = token_end + 1; } else *ptr = NULL; diff --git a/lib/unistr/u16-strtok.c b/lib/unistr/u16-strtok.c index 62e2f49439..df36cf76df 100644 --- a/lib/unistr/u16-strtok.c +++ b/lib/unistr/u16-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint16_t #define U_STRSPN u16_strspn #define U_STRPBRK u16_strpbrk +#define U_STRMBLEN u16_strmblen #include "u-strtok.h" diff --git a/lib/unistr/u32-strtok.c b/lib/unistr/u32-strtok.c index 0bb16ff459..f8ef999f9d 100644 --- a/lib/unistr/u32-strtok.c +++ b/lib/unistr/u32-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint32_t #define U_STRSPN u32_strspn #define U_STRPBRK u32_strpbrk +#define U_STRMBLEN u32_strmblen #include "u-strtok.h" diff --git a/lib/unistr/u8-strtok.c b/lib/unistr/u8-strtok.c index efb6a9a824..1e4e6efbaf 100644 --- a/lib/unistr/u8-strtok.c +++ b/lib/unistr/u8-strtok.c @@ -24,4 +24,5 @@ #define UNIT uint8_t #define U_STRSPN u8_strspn #define U_STRPBRK u8_strpbrk +#define U_STRMBLEN u8_strmblen #include "u-strtok.h" |