diff options
| author | ph10 <ph10@2f5784b3-3f2a-0410-8824-cb99058d5e15> | 2010-11-24 17:39:25 +0000 |
|---|---|---|
| committer | ph10 <ph10@2f5784b3-3f2a-0410-8824-cb99058d5e15> | 2010-11-24 17:39:25 +0000 |
| commit | 78d4828aaa1f15dfc4b40c0f718112f6508254f8 (patch) | |
| tree | 6778081b7ffe34ee20aa62245e0fa8d2b4b00ee3 | |
| parent | 1c4a198f0a69223930a4b118a35a618342d20898 (diff) | |
| download | pcre-78d4828aaa1f15dfc4b40c0f718112f6508254f8.tar.gz | |
Tidies of documenation and code while preparing for release.
git-svn-id: svn://vcs.exim.org/pcre/code/trunk@579 2f5784b3-3f2a-0410-8824-cb99058d5e15
39 files changed, 1633 insertions, 1536 deletions
@@ -1,27 +1,27 @@ ChangeLog for PCRE ------------------ -Version 8.11 10-Oct-2010 +Version 8.11 24-Nov-2010 ------------------------ 1. (*THEN) was not working properly if there were untried alternatives prior - to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it - backtracked to try for "b" instead of moving to the next alternative branch - at the same level (in this case, to look for "c"). The Perl documentation - is clear that when (*THEN) is backtracked onto, it goes to the "next + to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it + backtracked to try for "b" instead of moving to the next alternative branch + at the same level (in this case, to look for "c"). The Perl documentation + is clear that when (*THEN) is backtracked onto, it goes to the "next alternative in the innermost enclosing group". - -2. (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern + +2. (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern such as (A(*COMMIT)B(*THEN)C|D) any failure after matching A should result in overall failure. Similarly, (*COMMIT) now overrides (*PRUNE) and - (*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides - (*THEN). - + (*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides + (*THEN). + 3. If \s appeared in a character class, it removed the VT character from the class, even if it had been included by some previous item, for example - in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part - of \s, but is part of the POSIX "space" class.) - + in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part + of \s, but is part of the POSIX "space" class.) + 4. A partial match never returns an empty string (because you can always match an empty string at the end of the subject); however the checking for an empty string was starting at the "start of match" point. This has been @@ -31,101 +31,101 @@ Version 8.11 10-Oct-2010 (previously it gave "no match"). 5. Changes have been made to the way PCRE_PARTIAL_HARD affects the matching - of $, \z, \Z, \b, and \B. If the match point is at the end of the string, + of $, \z, \Z, \b, and \B. If the match point is at the end of the string, previously a full match would be given. However, setting PCRE_PARTIAL_HARD - has an implication that the given string is incomplete (because a partial - match is preferred over a full match). For this reason, these items now + has an implication that the given string is incomplete (because a partial + match is preferred over a full match). For this reason, these items now give a partial match in this situation. [Aside: previously, the one case /t\b/ matched against "cat" with PCRE_PARTIAL_HARD set did return a partial - match rather than a full match, which was wrong by the old rules, but is - now correct.] - + match rather than a full match, which was wrong by the old rules, but is + now correct.] + 6. There was a bug in the handling of #-introduced comments, recognized when PCRE_EXTENDED is set, when PCRE_NEWLINE_ANY and PCRE_UTF8 were also set. If a UTF-8 multi-byte character included the byte 0x85 (e.g. +U0445, whose UTF-8 encoding is 0xd1,0x85), this was misinterpreted as a newline when scanning for the end of the comment. (*Character* 0x85 is an "any" newline, - but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several + but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several places in pcre_compile(). - -7. Related to (6) above, when pcre_compile() was skipping #-introduced - comments when looking ahead for named forward references to subpatterns, - the only newline sequence it recognized was NL. It now handles newlines + +7. Related to (6) above, when pcre_compile() was skipping #-introduced + comments when looking ahead for named forward references to subpatterns, + the only newline sequence it recognized was NL. It now handles newlines according to the set newline convention. - -8. SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the - former, but used strtoul(), whereas pcretest avoided strtoul() but did not + +8. SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the + former, but used strtoul(), whereas pcretest avoided strtoul() but did not cater for a lack of strerror(). These oversights have been fixed. - -9. Added --match-limit and --recursion-limit to pcregrep. + +9. Added --match-limit and --recursion-limit to pcregrep. 10. Added two casts needed to build with Visual Studio when NO_RECURSE is set. 11. When the -o option was used, pcregrep was setting a return code of 1, even when matches were found, and --line-buffered was not being honoured. - + 12. Added an optional parentheses number to the -o and --only-matching options - of pcregrep. - + of pcregrep. + 13. Imitating Perl's /g action for multiple matches is tricky when the pattern - can match an empty string. The code to do it in pcretest and pcredemo + can match an empty string. The code to do it in pcretest and pcredemo needed fixing: - + (a) When the newline convention was "crlf", pcretest got it wrong, skipping only one byte after an empty string match just before CRLF (this case just got forgotten; "any" and "anycrlf" were OK). - + (b) The pcretest code also had a bug, causing it to loop forever in UTF-8 - mode when an empty string match preceded an ASCII character followed by + mode when an empty string match preceded an ASCII character followed by a non-ASCII character. (The code for advancing by one character rather - than one byte was nonsense.) - + than one byte was nonsense.) + (c) The pcredemo.c sample program did not have any code at all to handle - the cases when CRLF is a valid newline sequence. - + the cases when CRLF is a valid newline sequence. + 14. Neither pcre_exec() nor pcre_dfa_exec() was checking that the value given as a starting offset was within the subject string. There is now a new - error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is - negative or greater than the length of the string. In order to test this, + error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is + negative or greater than the length of the string. In order to test this, pcretest is extended to allow the setting of negative starting offsets. - -15. In both pcre_exec() and pcre_dfa_exec() the code for checking that the - starting offset points to the beginning of a UTF-8 character was - unnecessarily clumsy. I tidied it up. - + +15. In both pcre_exec() and pcre_dfa_exec() the code for checking that the + starting offset points to the beginning of a UTF-8 character was + unnecessarily clumsy. I tidied it up. + 16. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a - bad UTF-8 sequence and one that is incomplete. - -17. Nobody had reported that the --include_dir option, which was added in - release 7.7 should have been called --include-dir (hyphen, not underscore) - for compatibility with GNU grep. I have changed it to --include-dir, but - left --include_dir as an undocumented synonym, and the same for - --exclude-dir, though that is not available in GNU grep, at least as of - release 2.5.4. - -18. At a user's suggestion, the macros GETCHAR and friends (which pick up UTF-8 + bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. + +17. Nobody had reported that the --include_dir option, which was added in + release 7.7 should have been called --include-dir (hyphen, not underscore) + for compatibility with GNU grep. I have changed it to --include-dir, but + left --include_dir as an undocumented synonym, and the same for + --exclude-dir, though that is not available in GNU grep, at least as of + release 2.5.4. + +18. At a user's suggestion, the macros GETCHAR and friends (which pick up UTF-8 characters from a string of bytes) have been redefined so as not to use - loops, in order to improve performance in some environments. At the same - time, I abstracted some of the common code into auxiliary macros to save + loops, in order to improve performance in some environments. At the same + time, I abstracted some of the common code into auxiliary macros to save repetition (this should not affect the compiled code). - -19. If \c was followed by a multibyte UTF-8 character, bad things happened. A - compile-time error is now given if \c is not followed by an ASCII - character, that is, a byte less than 128. (In EBCDIC mode, the code is + +19. If \c was followed by a multibyte UTF-8 character, bad things happened. A + compile-time error is now given if \c is not followed by an ASCII + character, that is, a byte less than 128. (In EBCDIC mode, the code is different, and any byte value is allowed.) - + 20. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ START_OPTIMIZE option, which is now allowed at compile time - but just passed through to pcre_exec() or pcre_dfa_exec(). This makes it available - to pcregrep and other applications that have no direct access to PCRE - options. The new /Y option in pcretest sets this option when calling - pcre_compile(). - + to pcregrep and other applications that have no direct access to PCRE + options. The new /Y option in pcretest sets this option when calling + pcre_compile(). + 21. Change 18 of release 8.01 broke the use of named subpatterns for recursive - back references. Groups containing recursive back references were forced to - be atomic by that change, but in the case of named groups, the amount of - memory required was incorrectly computed, leading to "Failed: internal - error: code overflow". This has been fixed. + back references. Groups containing recursive back references were forced to + be atomic by that change, but in the case of named groups, the amount of + memory required was incorrectly computed, leading to "Failed: internal + error: code overflow". This has been fixed. Version 8.10 25-Jun-2010 @@ -1,6 +1,27 @@ News about PCRE releases ------------------------ +Release 8.11 24-Nov-2010 +------------------------ + +A number of bugs in the library and in pcregrep have been fixed. As always, see +ChangeLog for details. The following are the non-bug-fix changes: + +. Added --match-limit and --recursion-limit to pcregrep. + +. Added an optional parentheses number to the -o and --only-matching options + of pcregrep. + +. Changed the way PCRE_PARTIAL_HARD affects the matching of $, \z, \Z, \b, and + \B. + +. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a + bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. + +. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ + START_OPTIMIZE option, which is now allowed at compile time + + Release 8.10 25-Jun-2010 ------------------------ diff --git a/configure.ac b/configure.ac index 242e1b0..28daa62 100644 --- a/configure.ac +++ b/configure.ac @@ -11,7 +11,7 @@ dnl be defined as -RC2, for example. For real releases, it should be empty. m4_define(pcre_major, [8]) m4_define(pcre_minor, [11]) m4_define(pcre_prerelease, [-RC1]) -m4_define(pcre_date, [2010-10-09]) +m4_define(pcre_date, [2010-11-24]) # Libtool shared library interface versions (current:revision:age) m4_define(libpcre_version, [0:1:0]) diff --git a/doc/html/index.html b/doc/html/index.html index 58dfe45..d9af7e1 100644 --- a/doc/html/index.html +++ b/doc/html/index.html @@ -1,10 +1,10 @@ <html> -<!-- This is a manually maintained file that is the root of the HTML version of - the PCRE documentation. When the HTML documents are built from the man - page versions, the entire doc/html directory is emptied, this file is then - copied into doc/html/index.html, and the remaining files therein are +<!-- This is a manually maintained file that is the root of the HTML version of + the PCRE documentation. When the HTML documents are built from the man + page versions, the entire doc/html directory is emptied, this file is then + copied into doc/html/index.html, and the remaining files therein are created by the 132html script. ---> +--> <head> <title>PCRE specification</title> </head> @@ -74,11 +74,11 @@ The HTML documentation for PCRE comprises the following pages: </table> <p> -There are also individual pages that summarize the interface for each function +There are also individual pages that summarize the interface for each function in the library: </p> -<table> +<table> <tr><td><a href="pcre_compile.html">pcre_compile</a></td> <td> Compile a regular expression</td></tr> @@ -129,7 +129,7 @@ in the library: <tr><td><a href="pcre_maketables.html">pcre_maketables</a></td> <td> Build character tables in current locale</td></tr> - + <tr><td><a href="pcre_refcount.html">pcre_refcount</a></td> <td> Maintain reference count in compiled pattern</td></tr> diff --git a/doc/html/pcreapi.html b/doc/html/pcreapi.html index 707eef1..8c86fb5 100644 --- a/doc/html/pcreapi.html +++ b/doc/html/pcreapi.html @@ -435,8 +435,9 @@ within the pattern (see the detailed description in the documentation). For those options that can be different in different parts of the pattern, the contents of the <i>options</i> argument specifies their settings at the start of compilation and execution. The PCRE_ANCHORED, -PCRE_BSR_<i>xxx</i>, and PCRE_NEWLINE_<i>xxx</i> options can be set at the time -of matching as well as at compile time. +PCRE_BSR_<i>xxx</i>, PCRE_NEWLINE_<i>xxx</i>, PCRE_NO_UTF8_CHECK, and +PCRE_NO_START_OPT options can be set at the time of matching as well as at +compile time. </P> <P> If <i>errptr</i> is NULL, <b>pcre_compile()</b> returns NULL immediately. @@ -671,6 +672,14 @@ were followed by ?: but named parentheses can still be used for capturing (and they acquire numbers in the usual way). There is no equivalent of this option in Perl. <pre> + NO_START_OPTIMIZE +</pre> +This is an option that acts at matching time; that is, it is really an option +for <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. If it is set at compile time, +it is remembered with the compiled pattern and assumed at matching time. For +details see the discussion of PCRE_NO_START_OPTIMIZE +<a href="#execoptions">below.</a> +<pre> PCRE_UCP </pre> This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, @@ -1480,7 +1489,11 @@ a pre-scan of the subject that takes place before the pattern is run. The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly causing performance to suffer, but ensuring that in cases where the result is "no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK) -are considered at every possible starting position in the subject string. +are considered at every possible starting position in the subject string. If +PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching +time. +</P> +<P> Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation. Consider the pattern <pre> @@ -2232,7 +2245,7 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC22" href="#TOC1">REVISION</a><br> <P> -Last updated: 13 November 2010 +Last updated: 21 November 2010 <br> Copyright © 1997-2010 University of Cambridge. <br> diff --git a/doc/html/pcrecallout.html b/doc/html/pcrecallout.html index ab2e2d5..4ae09a5 100644 --- a/doc/html/pcrecallout.html +++ b/doc/html/pcrecallout.html @@ -81,9 +81,9 @@ been scanned far enough. </P> <P> You can disable these optimizations by passing the PCRE_NO_START_OPTIMIZE -option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. This slows down the -matching process, but does ensure that callouts such as the example above are -obeyed. +option to <b>pcre_compile()</b>, <b>pcre_exec()</b>, or <b>pcre_dfa_exec()</b>, +or by starting the pattern with (*NO_START_OPT). This slows down the matching +process, but does ensure that callouts such as the example above are obeyed. </P> <br><a name="SEC3" href="#TOC1">THE CALLOUT INTERFACE</a><br> <P> @@ -206,9 +206,9 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC6" href="#TOC1">REVISION</a><br> <P> -Last updated: 29 September 2009 +Last updated: 21 November 2010 <br> -Copyright © 1997-2009 University of Cambridge. +Copyright © 1997-2010 University of Cambridge. <br> <p> Return to the <a href="index.html">PCRE index page</a>. diff --git a/doc/html/pcrecompat.html b/doc/html/pcrecompat.html index 1b25db9..2ee59e9 100644 --- a/doc/html/pcrecompat.html +++ b/doc/html/pcrecompat.html @@ -113,7 +113,7 @@ names map to capturing subpattern number 1. To avoid this confusing situation, an error is given at compile time. </P> <P> -12. Perl recognizes comments in some places that PCRE doesn't, for example, +12. Perl recognizes comments in some places that PCRE doesn't, for example, between the ( and ? at the start of a subpattern. </P> <P> diff --git a/doc/html/pcredemo.html b/doc/html/pcredemo.html index 7ce6db0..cbe03e1 100644 --- a/doc/html/pcredemo.html +++ b/doc/html/pcredemo.html @@ -277,7 +277,7 @@ if (!find_all) /* Check for -g */ } /* Before running the loop, check for UTF-8 and whether CRLF is a valid newline -sequence. First, find the options with which the regex was compiled; extract +sequence. First, find the options with which the regex was compiled; extract the UTF-8 state, and mask off all but the newline options. */ (void)pcre_fullinfo(re, NULL, PCRE_INFO_OPTIONS, &option_bits); @@ -303,7 +303,7 @@ if (option_bits == 0) /* See if CRLF is a valid newline sequence. */ -crlf_is_newline = +crlf_is_newline = option_bits == PCRE_NEWLINE_ANY || option_bits == PCRE_NEWLINE_CRLF || option_bits == PCRE_NEWLINE_ANYCRLF; @@ -362,11 +362,11 @@ for (;;) else if (utf8) /* Otherwise, ensure we */ { /* advance a whole UTF-8 */ while (ovector[1] < subject_length) /* character. */ - { + { if ((subject[ovector[1]] & 0xc0) != 0x80) break; ovector[1] += 1; } - } + } continue; /* Go round the loop again */ } diff --git a/doc/html/pcregrep.html b/doc/html/pcregrep.html index 1adbee9..f568e52 100644 --- a/doc/html/pcregrep.html +++ b/doc/html/pcregrep.html @@ -366,12 +366,12 @@ locale is specified, the PCRE library's default (usually the "C" locale) is used. There is no short form for this option. </P> <P> -<b>--match-limit</b>=<i>number</i> +<b>--match-limit</b>=<i>number</i> Processing some regular expression patterns can require a very large amount of memory, leading in some cases to a program crash if not enough is available. -Other patterns may take a very long time to search for all possible matching +Other patterns may take a very long time to search for all possible matching strings. The <b>pcre_exec()</b> function that is called by <b>pcregrep</b> to do -the matching has two parameters that can limit the resources that it uses. +the matching has two parameters that can limit the resources that it uses. <br> <br> The <b>--match-limit</b> option provides a means of limiting resource usage @@ -392,7 +392,7 @@ of calls, because not all calls to <b>match()</b> are recursive. This limit is of use only if it is set smaller than <b>--match-limit</b>. <br> <br> -There are no short forms for these options. The default settings are specified +There are no short forms for these options. The default settings are specified when the PCRE library is compiled, with the default default being 10 million. </P> <P> @@ -451,13 +451,13 @@ exclusive with <b>--file-offsets</b> and <b>--line-offsets</b>. </P> <P> <b>-o</b><i>number</i>, <b>--only-matching</b>=<i>number</i> -Show only the part of the line that matched the capturing parentheses of the +Show only the part of the line that matched the capturing parentheses of the given number. Up to 32 capturing parentheses are supported. Because these options can be given without an argument (see above), if an argument is present, it must be given in the same shell item, for example, -o3 or ---only-matching=2. The comments given for the non-argument case above also -apply to this case. If the specified capturing parentheses do not exist in the -pattern, or were not set in the match, nothing is output unless the file name +--only-matching=2. The comments given for the non-argument case above also +apply to this case. If the specified capturing parentheses do not exist in the +pattern, or were not set in the match, nothing is output unless the file name or line number are being printed. </P> <P> @@ -554,8 +554,8 @@ exception) in the next command line item. For example: -f/some/file -f /some/file </pre> -The exception is the <b>-o</b> option, which may appear with or without data. -Because of this, if data is present, it must follow immediately in the same +The exception is the <b>-o</b> option, which may appear with or without data. +Because of this, if data is present, it must follow immediately in the same item, for example -o3. </P> <P> @@ -575,7 +575,7 @@ specially unless it is at the start of an item. The exceptions to the above are the <b>--colour</b> (or <b>--color</b>) and <b>--only-matching</b> options, for which the data is optional. If one of these options does have data, it must be given in the first form, using an equals -character. Otherwise \fBpcregrep\P will assume that it has no data. +character. Otherwise <b>pcregrep</b> will assume that it has no data. </P> <br><a name="SEC9" href="#TOC1">MATCHING ERRORS</a><br> <P> @@ -590,7 +590,7 @@ there are more than 20 such errors, <b>pcregrep</b> gives up. <P> The <b>--match-limit</b> option of <b>pcregrep</b> can be used to set the overall resource limit; there is a second option called <b>--recursion-limit</b> that -sets a limit on the amount of memory (usually stack) that is used (see the +sets a limit on the amount of memory (usually stack) that is used (see the discussion of these options above). </P> <br><a name="SEC10" href="#TOC1">DIAGNOSTICS</a><br> diff --git a/doc/html/pcrematching.html b/doc/html/pcrematching.html index 80945ca..3d1acf6 100644 --- a/doc/html/pcrematching.html +++ b/doc/html/pcrematching.html @@ -106,7 +106,7 @@ The scan continues until either the end of the subject is reached, or there are no more unterminated paths. At this point, terminated paths represent the different matching possibilities (if there are none, the match has failed). Thus, if there is more than one possible match, this algorithm finds all of -them, and in particular, it finds the longest. The matches are returned in +them, and in particular, it finds the longest. The matches are returned in decreasing order of length. There is an option to stop the algorithm after the first match (which is necessarily the shortest) is found. </P> @@ -190,7 +190,7 @@ time. Although it is possible to do multi-segment matching using the standard algorithm (<b>pcre_exec()</b>), by retaining partially matched substrings, it is more complicated. The <a href="pcrepartial.html"><b>pcrepartial</b></a> -documentation gives details of partial matching and discusses multi-segment +documentation gives details of partial matching and discusses multi-segment matching. </P> <br><a name="SEC6" href="#TOC1">DISADVANTAGES OF THE ALTERNATIVE ALGORITHM</a><br> diff --git a/doc/html/pcrepartial.html b/doc/html/pcrepartial.html index d9229c0..1e33b58 100644 --- a/doc/html/pcrepartial.html +++ b/doc/html/pcrepartial.html @@ -73,9 +73,9 @@ subject string is reached successfully, but matching cannot continue because more characters are needed. However, at least one character in the subject must have been inspected. This character need not form part of the final matched string; lookbehind assertions and the \K escape sequence provide ways of -inspecting characters before the start of a matched substring. The requirement -for inspecting at least one character exists because an empty string can always -be matched; without such a restriction there would always be a partial match of +inspecting characters before the start of a matched substring. The requirement +for inspecting at least one character exists because an empty string can always +be matched; without such a restriction there would always be a partial match of an empty string at the end of the subject. </P> <P> @@ -83,7 +83,7 @@ If there are at least two slots in the offsets vector when <b>pcre_exec()</b> returns with a partial match, the first slot is set to the offset of the earliest character that was inspected when the partial match was found. For convenience, the second offset points to the end of the subject so that a -substring can easily be identified. +substring can easily be identified. </P> <P> For the majority of patterns, the first offset identifies the start of the @@ -100,7 +100,7 @@ with extra characters added to the subject. </P> <P> What happens when a partial match is identified depends on which of the two -partial matching options are set. +partial matching options are set. </P> <br><b> PCRE_PARTIAL_SOFT with pcre_exec() @@ -112,10 +112,10 @@ alternatives in the pattern are tried. If no complete match can be found, <b>pcre_exec()</b> returns PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. </P> <P> -This option is "soft" because it prefers a complete match over a partial match. -All the various matching items in a pattern behave as if the subject string is +This option is "soft" because it prefers a complete match over a partial match. +All the various matching items in a pattern behave as if the subject string is potentially complete. For example, \z, \Z, and $ match at the end of the -subject, as normal, and for \b and \B the end of the subject is treated as a +subject, as normal, and for \b and \B the end of the subject is treated as a non-alphanumeric. </P> <P> @@ -137,17 +137,17 @@ PCRE_PARTIAL_HARD with pcre_exec() <P> If PCRE_PARTIAL_HARD is set for <b>pcre_exec()</b>, it returns PCRE_ERROR_PARTIAL as soon as a partial match is found, without continuing to -search for possible complete matches. This option is "hard" because it prefers +search for possible complete matches. This option is "hard" because it prefers an earlier partial match over a later complete match. For this reason, the assumption is made that the end of the supplied subject string may not be the true end of the available data, and so, if \z, \Z, \b, \B, or $ are encountered at the end of the subject, the result is PCRE_ERROR_PARTIAL. </P> <P> -Setting PCRE_PARTIAL_HARD also affects the way <b>pcre_exec()</b> checks UTF-8 +Setting PCRE_PARTIAL_HARD also affects the way <b>pcre_exec()</b> checks UTF-8 subject strings for validity. Normally, an invalid UTF-8 sequence causes the -error PCRE_ERROR_BADUTF8. However, in the special case of a truncated UTF-8 -character at the end of the subject, PCRE_ERROR_SHORTUTF8 is returned when +error PCRE_ERROR_BADUTF8. However, in the special case of a truncated UTF-8 +character at the end of the subject, PCRE_ERROR_SHORTUTF8 is returned when PCRE_PARTIAL_HARD is set. </P> <br><b> @@ -304,8 +304,8 @@ From release 8.00, <b>pcre_exec()</b> can also be used to do multi-segment matching. Unlike <b>pcre_dfa_exec()</b>, it is not possible to restart the previous match with a new segment of data. Instead, new data must be added to the previous subject string, and the entire match re-run, starting from the -point where the partial match occurred. Earlier data can be discarded. It is -best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the +point where the partial match occurred. Earlier data can be discarded. It is +best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the end of a segment as the end of the subject when matching \z, \Z, \b, \B, and $. Consider an unanchored pattern that matches dates: <pre> @@ -333,8 +333,8 @@ whichever matching function is used. <P> 1. If the pattern contains a test for the beginning of a line, you need to pass the PCRE_NOTBOL option when the subject string for any call does start at the -beginning of a line. There is also a PCRE_NOTEOL option, but in practice when -doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which +beginning of a line. There is also a PCRE_NOTEOL option, but in practice when +doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which includes the effect of PCRE_NOTEOL. </P> <P> diff --git a/doc/html/pcrepattern.html b/doc/html/pcrepattern.html index 7ab17be..b1fa6e0 100644 --- a/doc/html/pcrepattern.html +++ b/doc/html/pcrepattern.html @@ -89,6 +89,12 @@ instead of recognizing only characters with codes less than 128 via a lookup table. </P> <P> +If a pattern starts with (*NO_START_OPT), it has the same effect as setting the +PCRE_NO_START_OPTIMIZE option either at compile or matching time. There are +also some more of these special sequences that are concerned with the handling +of newlines; they are described below. +</P> +<P> The remainder of this document discusses the patterns that are supported by PCRE when its main matching function, <b>pcre_exec()</b>, is used. From release 6.0, PCRE offers a second matching function, @@ -204,9 +210,9 @@ The following sections describe the use of each of the metacharacters. <br><a name="SEC4" href="#TOC1">BACKSLASH</a><br> <P> The backslash character has several uses. Firstly, if it is followed by a -non-alphanumeric character, it takes away any special meaning that character -may have. This use of backslash as an escape character applies both inside and -outside character classes. +character that is not a number or a letter, it takes away any special meaning +that character may have. This use of backslash as an escape character applies +both inside and outside character classes. </P> <P> For example, if you want to match a * character, you write \* in the pattern. @@ -216,6 +222,11 @@ non-alphanumeric with backslash to specify that it stands for itself. In particular, if you want to match a backslash, you write \\. </P> <P> +In UTF-8 mode, only ASCII numbers and letters have any special meaning after a +backslash. All other characters (in particular, those whose codepoints are +greater than 127) are treated as literals. +</P> +<P> If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the pattern (other than in a character class) and characters between a # outside a character class and the next newline are ignored. An escaping backslash can @@ -247,7 +258,7 @@ but when a pattern is being prepared by text editing, it is often easier to use one of the following escape sequences than the binary character it represents: <pre> \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any character + \cx "control-x", where x is any ASCII character \e escape (hex 1B) \f formfeed (hex 0C) \n linefeed (hex 0A) @@ -259,8 +270,12 @@ one of the following escape sequences than the binary character it represents: </pre> The precise effect of \cx is as follows: if x is a lower case letter, it is converted to upper case. Then bit 6 of the character (hex 40) is inverted. -Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; becomes hex -7B. +Thus \cz becomes hex 1A (z is 7A), but \c{ becomes hex 3B ({ is 7B), while +\c; becomes hex 7B (; is 3B). If the byte following \c has a value greater +than 127, a compile-time error occurs. This locks out non-ASCII characters in +both byte mode and UTF-8 mode. (When PCRE is compiled in EBCDIC mode, all byte +values are valid. A lower case letter is converted to upper case, and then the +0xc0 bits are flipped.) </P> <P> After \x, from zero to two hexadecimal digits are read (letters can be in @@ -421,7 +436,7 @@ any Unicode letter, and underscore. Note also that PCRE_UCP affects \b, and is noticeably slower when PCRE_UCP is set. </P> <P> -The sequences \h, \H, \v, and \V are features that were added to Perl at +The sequences \h, \H, \v, and \V are features that were added to Perl at release 5.10. In contrast to the other sequences, which match only ASCII characters by default, these always match certain high-valued codepoints in UTF-8 mode, whether or not PCRE_UCP is set. The horizontal space characters @@ -940,7 +955,7 @@ dollar, the only relationship being that they both involve newlines. Dot has no special meaning in a character class. </P> <P> -The escape sequence \N behaves like a dot, except that it is not affected by +The escape sequence \N behaves like a dot, except that it is not affected by the PCRE_DOTALL option. In other words, it matches any character except one that signifies the end of a line. </P> @@ -1040,12 +1055,26 @@ characters with values greater than 128 only when it is compiled with Unicode property support. </P> <P> -The character types \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, \w, and -\W may also appear in a character class, and add the characters that they -match to the class. For example, [\dABCDEF] matches any hexadecimal digit. A -circumflex can conveniently be used with the upper case character types to +The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, +\V, \w, and \W may appear in a character class, and add the characters that +they match to the class. For example, [\dABCDEF] matches any hexadecimal +digit. In UTF-8 mode, the PCRE_UCP option affects the meanings of \d, \s, \w +and their upper case partners, just as it does when they appear outside a +character class, as described in the section entitled +<a href="#genericchartypes">"Generic character types"</a> +above. The escape sequence \b has a different meaning inside a character +class; it matches the backspace character. The sequences \B, \N, \R, and \X +are not special inside a character class. Like any other unrecognized escape +sequences, they are treated as the literal characters "B", "N", "R", and "X" by +default, but cause an error if the PCRE_EXTRA option is set. +</P> +<P> +A circumflex can conveniently be used with the upper case character types to specify a more restricted set of characters than the matching lower case type. -For example, the class [^\W_] matches any letter or digit, but not underscore. +For example, the class [^\W_] matches any letter or digit, but not underscore, +whereas [\w] includes underscore. A positive character class should be read as +"something OR something OR ..." and a negative class as "NOT something AND NOT +something AND NOT ...". </P> <P> The only metacharacters that are recognized in character classes are backslash, @@ -1669,10 +1698,10 @@ example: (abc(def)ghi)\g{-1} </pre> The sequence \g{-1} is a reference to the most recently started capturing -subpattern before \g, that is, is it equivalent to \2. Similarly, \g{-2} -would be equivalent to \1. The use of relative references can be helpful in -long patterns, and also in patterns that are created by joining together -fragments that contain references within themselves. +subpattern before \g, that is, is it equivalent to \2 in this example. +Similarly, \g{-2} would be equivalent to \1. The use of relative references +can be helpful in long patterns, and also in patterns that are created by +joining together fragments that contain references within themselves. </P> <P> A back reference matches whatever actually matched the capturing subpattern in @@ -1802,8 +1831,7 @@ lookbehind assertion is needed to achieve the other effect. If you want to force a matching failure at some point in a pattern, the most convenient way to do it is with (?!) because an empty string always matches, so an assertion that requires there not to be an empty string must always fail. -The backtracking control verb (*FAIL) or (*F) is essentially a synonym for -(?!). +The backtracking control verb (*FAIL) or (*F) is a synonym for (?!). <a name="lookbehind"></a></P> <br><b> Lookbehind assertions @@ -1936,9 +1964,9 @@ already been matched. The two possible forms of conditional subpattern are: If the condition is satisfied, the yes-pattern is used; otherwise the no-pattern (if present) is used. If there are more than two alternatives in the subpattern, a compile-time error occurs. Each of the two alternatives may -itself contain nested subpatterns of any form, including conditional +itself contain nested subpatterns of any form, including conditional subpatterns; the restriction to two alternatives applies only at the level of -the condition. This pattern fragment is an example where the alternatives are +the condition. This pattern fragment is an example where the alternatives are complex: <pre> (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) @@ -1958,12 +1986,13 @@ condition is true if a capturing subpattern of that number has previously matched. If there is more than one capturing subpattern with the same number (see the earlier <a href="#recursion">section about duplicate subpattern numbers),</a> -the condition is true if any of them have been set. An alternative notation is +the condition is true if any of them have matched. An alternative notation is to precede the digits with a plus or minus sign. In this case, the subpattern number is relative rather than absolute. The most recently opened parentheses -can be referenced by (?(-1), the next most recent by (?(-2), and so on. In -looping constructs it can also make sense to refer to subsequent groups with -constructs such as (?(+2). +can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside +loops it can also make sense to refer to subsequent groups. The next +parentheses to be opened can be referenced as (?(+1), and so on. (The value +zero in any of these forms is not used; it provokes a compile-time error.) </P> <P> Consider the following pattern, which contains non-significant white space to @@ -1975,8 +2004,8 @@ three parts for ease of discussion: The first part matches an optional opening parenthesis, and if that character is present, sets it as the first captured substring. The second part matches one or more characters that are not parentheses. The third part is a -conditional subpattern that tests whether the first set of parentheses matched -or not. If they did, that is, if subject started with an opening parenthesis, +conditional subpattern that tests whether or not the first set of parentheses +matched. If they did, that is, if subject started with an opening parenthesis, the condition is true, and so the yes-pattern is executed and a closing parenthesis is required. Otherwise, since no-pattern is not present, the subpattern matches nothing. In other words, this pattern matches a sequence of @@ -2044,8 +2073,9 @@ alternative in the subpattern. It is always skipped if control reaches this point in the pattern; the idea of DEFINE is that it can be used to define "subroutines" that can be referenced from elsewhere. (The use of <a href="#subpatternsassubroutines">"subroutines"</a> -is described below.) For example, a pattern to match an IPv4 address could be -written like this (ignore whitespace and line breaks): +is described below.) For example, a pattern to match an IPv4 address such as +"192.168.23.245" could be written like this (ignore whitespace and line +breaks): <pre> (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) \b (?&byte) (\.(?&byte)){3} \b @@ -2078,7 +2108,7 @@ dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. <a name="comments"></a></P> <br><a name="SEC20" href="#TOC1">COMMENTS</a><br> <P> -There are two ways of including comments in patterns that are processed by +There are two ways of including comments in patterns that are processed by PCRE. In both cases, the start of the comment must not be in a character class, nor in the middle of any other sequence of related characters such as (?: or a subpattern name or number. The characters that make up a comment play no part @@ -2100,7 +2130,7 @@ default newline convention is in force: <pre> abc #comment \n still comment </pre> -On encountering the # character, <b>pcre_compile()</b> skips along, looking for +On encountering the # character, <b>pcre_compile()</b> skips along, looking for a newline in the pattern. The sequence \n is still literal at this stage, so it does not terminate the comment. Only an actual character with the code value 0x0a (the default newline) does so. @@ -2270,8 +2300,9 @@ difference: in the previous case the remaining alternative is at a deeper recursion level, which PCRE cannot use. </P> <P> -To change the pattern so that matches all palindromic strings, not just those -with an odd number of characters, it is tempting to change the pattern to this: +To change the pattern so that it matches all palindromic strings, not just +those with an odd number of characters, it is tempting to change the pattern to +this: <pre> ^((.)(?1)\2|.?)$ </pre> @@ -2433,7 +2464,8 @@ minimum length of matching subject, or that a particular character must be present. When one of these optimizations suppresses the running of a match, any included backtracking verbs will not, of course, be processed. You can suppress the start-of-match optimizations by setting the PCRE_NO_START_OPTIMIZE option -when calling <b>pcre_exec()</b>. +when calling <b>pcre_compile()</b> or <b>pcre_exec()</b>, or by starting the +pattern with (*NO_START_OPT). </P> <br><b> Verbs that act immediately @@ -2624,7 +2656,7 @@ matching name is found, normal "bumpalong" of one character happens (the <pre> (*THEN) or (*THEN:NAME) </pre> -This verb causes a skip to the next alternation in the innermost enclosing +This verb causes a skip to the next alternation in the innermost enclosing group if the rest of the pattern does not match. That is, it cancels pending backtracking, but only within the current alternation. Its name comes from the observation that it can be used for a pattern-based if-then-else block: @@ -2639,7 +2671,7 @@ overall match fails. If (*THEN) is not directly inside an alternation, it acts like (*PRUNE). </P> <P> -The above verbs provide four different "strengths" of control when subsequent +The above verbs provide four different "strengths" of control when subsequent matching fails. (*THEN) is the weakest, carrying on the match at the next alternation. (*PRUNE) comes next, failing the match at the current starting position, but allowing an advance to the next character (for an unanchored @@ -2673,7 +2705,7 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC28" href="#TOC1">REVISION</a><br> <P> -Last updated: 17 November 2010 +Last updated: 21 November 2010 <br> Copyright © 1997-2010 University of Cambridge. <br> diff --git a/doc/html/pcreprecompile.html b/doc/html/pcreprecompile.html index 83da226..91a8df0 100644 --- a/doc/html/pcreprecompile.html +++ b/doc/html/pcreprecompile.html @@ -125,8 +125,7 @@ usual way. <br><a name="SEC4" href="#TOC1">COMPATIBILITY WITH DIFFERENT PCRE RELEASES</a><br> <P> In general, it is safest to recompile all saved patterns when you update to a -new PCRE release, though not all updates actually require this. Recompiling is -definitely needed for release 7.2. +new PCRE release, though not all updates actually require this. </P> <br><a name="SEC5" href="#TOC1">AUTHOR</a><br> <P> @@ -139,9 +138,9 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC6" href="#TOC1">REVISION</a><br> <P> -Last updated: 13 June 2007 +Last updated: 17 November 2010 <br> -Copyright © 1997-2007 University of Cambridge. +Copyright © 1997-2010 University of Cambridge. <br> <p> Return to the <a href="index.html">PCRE index page</a>. diff --git a/doc/html/pcresample.html b/doc/html/pcresample.html index a06408e..dcd69bf 100644 --- a/doc/html/pcresample.html +++ b/doc/html/pcresample.html @@ -71,7 +71,7 @@ PCRE library. The program is provided as a simple coding example. </P> <P> -When you try to run +If you try to run <a href="pcredemo.html"><b>pcredemo</b></a> when PCRE is not installed in the standard library directory, you may get an error like this on some operating systems (e.g. Solaris): @@ -100,7 +100,7 @@ Cambridge CB2 3QH, England. REVISION </b><br> <P> -Last updated: 26 May 2010 +Last updated: 17 November 2010 <br> Copyright © 1997-2010 University of Cambridge. <br> diff --git a/doc/html/pcresyntax.html b/doc/html/pcresyntax.html index cb1e855..9fa3ebd 100644 --- a/doc/html/pcresyntax.html +++ b/doc/html/pcresyntax.html @@ -60,7 +60,7 @@ syntax. <P> <pre> \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any character + \cx "control-x", where x is any ASCII character \e escape (hex 1B) \f formfeed (hex 0C) \n newline (hex 0A) @@ -366,6 +366,7 @@ but some of them use Unicode properties if PCRE_UCP is set. You can use The following are recognized only at the start of a pattern or after one of the newline-setting options with similar syntax: <pre> + (*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE) (*UTF8) set UTF-8 mode (PCRE_UTF8) (*UCP) set PCRE_UCP (use Unicode properties for \d etc) </PRE> @@ -494,7 +495,7 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC27" href="#TOC1">REVISION</a><br> <P> -Last updated: 12 May 2010 +Last updated: 21 November 2010 <br> Copyright © 1997-2010 University of Cambridge. <br> diff --git a/doc/html/pcretest.html b/doc/html/pcretest.html index a48a79f..62aa468 100644 --- a/doc/html/pcretest.html +++ b/doc/html/pcretest.html @@ -215,6 +215,7 @@ options that do not correspond to anything in Perl: <b>/U</b> PCRE_UNGREEDY <b>/W</b> PCRE_UCP <b>/X</b> PCRE_EXTRA + <b>/Y</b> PCRE_NO_START_OPTIMIZE <b>/<JS></b> PCRE_JAVASCRIPT_COMPAT <b>/<cr></b> PCRE_NEWLINE_CR <b>/<lf></b> PCRE_NEWLINE_LF @@ -256,9 +257,9 @@ empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set in order to search for another, non-empty, match at the same point. If this second match fails, the start offset is advanced, and the normal match is retried. This imitates the way Perl handles such cases when -using the <b>/g</b> modifier or the <b>split()</b> function. Normally, the start -offset is advanced by one character, but if the newline convention recognizes -CRLF as a newline, and the current character is CR followed by LF, an advance +using the <b>/g</b> modifier or the <b>split()</b> function. Normally, the start +offset is advanced by one character, but if the newline convention recognizes +CRLF as a newline, and the current character is CR followed by LF, an advance of two is used. </P> <br><b> @@ -385,7 +386,7 @@ recognized: \t tab (\x09) \v vertical tab (\x0b) \nnn octal character (up to 3 octal digits) - always a byte unless > 255 in UTF-8 mode + always a byte unless > 255 in UTF-8 mode \xhh hexadecimal byte (up to 2 hex digits) \x{hh...} hexadecimal character, any number of digits in UTF-8 mode \A pass the PCRE_ANCHORED option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> @@ -416,7 +417,7 @@ recognized: \Y pass the PCRE_NO_START_OPTIMIZE option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> \Z pass the PCRE_NOTEOL option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> \? pass the PCRE_NO_UTF8_CHECK option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> - \>dd start the match at offset dd (optional "-"; then any number of digits); this sets the <i>startoffset</i> + \>dd start the match at offset dd (optional "-"; then any number of digits); this sets the <i>startoffset</i> argument for <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> \<cr> pass the PCRE_NEWLINE_CR option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> \<lf> pass the PCRE_NEWLINE_LF option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> @@ -424,11 +425,11 @@ recognized: \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> \<any> pass the PCRE_NEWLINE_ANY option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> </pre> -Note that \xhh always specifies one byte, even in UTF-8 mode; this makes it -possible to construct invalid UTF-8 sequences for testing purposes. On the +Note that \xhh always specifies one byte, even in UTF-8 mode; this makes it +possible to construct invalid UTF-8 sequences for testing purposes. On the other hand, \x{hh} is interpreted as a UTF-8 character in UTF-8 mode, -generating more than one byte if the value is greater than 127. When not in -UTF-8 mode, it generates one byte for values less than 256, and causes an error +generating more than one byte if the value is greater than 127. When not in +UTF-8 mode, it generates one byte for values less than 256, and causes an error for greater values. </P> <P> @@ -756,7 +757,7 @@ Cambridge CB2 3QH, England. </P> <br><a name="SEC15" href="#TOC1">REVISION</a><br> <P> -Last updated: 07 November 2010 +Last updated: 21 November 2010 <br> Copyright © 1997-2010 University of Cambridge. <br> diff --git a/doc/pcre.txt b/doc/pcre.txt index be58293..5d769df 100644 --- a/doc/pcre.txt +++ b/doc/pcre.txt @@ -2,7 +2,7 @@ This file contains a concatenation of the PCRE man pages, converted to plain text format for ease of searching with a text editor, or for use on systems that do not have a man page processor. The small individual files that give -synopses of each function in the library have not been included. Neither has +synopses of each function in the library have not been included. Neither has the pcredemo program. There are separate text files for the pcregrep and pcretest commands. ----------------------------------------------------------------------------- @@ -269,8 +269,8 @@ REVISION Last updated: 13 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREBUILD(3) PCREBUILD(3) @@ -600,8 +600,8 @@ REVISION Last updated: 29 September 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREMATCHING(3) PCREMATCHING(3) @@ -804,8 +804,8 @@ REVISION Last updated: 17 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREAPI(3) PCREAPI(3) @@ -1165,38 +1165,39 @@ COMPILING A PATTERN pcrepattern documentation). For those options that can be different in different parts of the pattern, the contents of the options argument specifies their settings at the start of compilation and execution. The - PCRE_ANCHORED, PCRE_BSR_xxx, and PCRE_NEWLINE_xxx options can be set at - the time of matching as well as at compile time. + PCRE_ANCHORED, PCRE_BSR_xxx, PCRE_NEWLINE_xxx, PCRE_NO_UTF8_CHECK, and + PCRE_NO_START_OPT options can be set at the time of matching as well as + at compile time. If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, - if compilation of a pattern fails, pcre_compile() returns NULL, and + if compilation of a pattern fails, pcre_compile() returns NULL, and sets the variable pointed to by errptr to point to a textual error mes- sage. This is a static string that is part of the library. You must not - try to free it. The offset from the start of the pattern to the byte + try to free it. The offset from the start of the pattern to the byte that was being processed when the error was discovered is placed in the - variable pointed to by erroffset, which must not be NULL. If it is, an + variable pointed to by erroffset, which must not be NULL. If it is, an immediate error is given. Some errors are not detected until checks are - carried out when the whole pattern has been scanned; in this case the + carried out when the whole pattern has been scanned; in this case the offset is set to the end of the pattern. - Note that the offset is in bytes, not characters, even in UTF-8 mode. - It may point into the middle of a UTF-8 character (for example, when + Note that the offset is in bytes, not characters, even in UTF-8 mode. + It may point into the middle of a UTF-8 character (for example, when PCRE_ERROR_BADUTF8 is returned for an invalid UTF-8 string). - If pcre_compile2() is used instead of pcre_compile(), and the error- - codeptr argument is not NULL, a non-zero error code number is returned - via this argument in the event of an error. This is in addition to the + If pcre_compile2() is used instead of pcre_compile(), and the error- + codeptr argument is not NULL, a non-zero error code number is returned + via this argument in the event of an error. This is in addition to the textual error message. Error codes and messages are listed below. - If the final argument, tableptr, is NULL, PCRE uses a default set of - character tables that are built when PCRE is compiled, using the - default C locale. Otherwise, tableptr must be an address that is the - result of a call to pcre_maketables(). This value is stored with the - compiled pattern, and used again by pcre_exec(), unless another table + If the final argument, tableptr, is NULL, PCRE uses a default set of + character tables that are built when PCRE is compiled, using the + default C locale. Otherwise, tableptr must be an address that is the + result of a call to pcre_maketables(). This value is stored with the + compiled pattern, and used again by pcre_exec(), unless another table pointer is passed to it. For more discussion, see the section on locale support below. - This code fragment shows a typical straightforward call to pcre_com- + This code fragment shows a typical straightforward call to pcre_com- pile(): pcre *re; @@ -1209,147 +1210,147 @@ COMPILING A PATTERN &erroffset, /* for error offset */ NULL); /* use default character tables */ - The following names for option bits are defined in the pcre.h header + The following names for option bits are defined in the pcre.h header file: PCRE_ANCHORED If this bit is set, the pattern is forced to be "anchored", that is, it - is constrained to match only at the first matching point in the string - that is being searched (the "subject string"). This effect can also be - achieved by appropriate constructs in the pattern itself, which is the + is constrained to match only at the first matching point in the string + that is being searched (the "subject string"). This effect can also be + achieved by appropriate constructs in the pattern itself, which is the only way to do it in Perl. PCRE_AUTO_CALLOUT If this bit is set, pcre_compile() automatically inserts callout items, - all with number 255, before each pattern item. For discussion of the + all with number 255, before each pattern item. For discussion of the callout facility, see the pcrecallout documentation. PCRE_BSR_ANYCRLF PCRE_BSR_UNICODE These options (which are mutually exclusive) control what the \R escape - sequence matches. The choice is either to match only CR, LF, or CRLF, + sequence matches. The choice is either to match only CR, LF, or CRLF, or to match any Unicode newline sequence. The default is specified when PCRE is built. It can be overridden from within the pattern, or by set- ting an option when a compiled pattern is matched. PCRE_CASELESS - If this bit is set, letters in the pattern match both upper and lower - case letters. It is equivalent to Perl's /i option, and it can be - changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE - always understands the concept of case for characters whose values are - less than 128, so caseless matching is always possible. For characters - with higher values, the concept of case is supported if PCRE is com- - piled with Unicode property support, but not otherwise. If you want to - use caseless matching for characters 128 and above, you must ensure - that PCRE is compiled with Unicode property support as well as with + If this bit is set, letters in the pattern match both upper and lower + case letters. It is equivalent to Perl's /i option, and it can be + changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE + always understands the concept of case for characters whose values are + less than 128, so caseless matching is always possible. For characters + with higher values, the concept of case is supported if PCRE is com- + piled with Unicode property support, but not otherwise. If you want to + use caseless matching for characters 128 and above, you must ensure + that PCRE is compiled with Unicode property support as well as with UTF-8 support. PCRE_DOLLAR_ENDONLY - If this bit is set, a dollar metacharacter in the pattern matches only - at the end of the subject string. Without this option, a dollar also - matches immediately before a newline at the end of the string (but not - before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored - if PCRE_MULTILINE is set. There is no equivalent to this option in + If this bit is set, a dollar metacharacter in the pattern matches only + at the end of the subject string. Without this option, a dollar also + matches immediately before a newline at the end of the string (but not + before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored + if PCRE_MULTILINE is set. There is no equivalent to this option in Perl, and no way to set it within a pattern. PCRE_DOTALL - If this bit is set, a dot metacharacter in the pattern matches a char- + If this bit is set, a dot metacharacter in the pattern matches a char- acter of any value, including one that indicates a newline. However, it - only ever matches one character, even if newlines are coded as CRLF. - Without this option, a dot does not match when the current position is + only ever matches one character, even if newlines are coded as CRLF. + Without this option, a dot does not match when the current position is at a newline. This option is equivalent to Perl's /s option, and it can - be changed within a pattern by a (?s) option setting. A negative class + be changed within a pattern by a (?s) option setting. A negative class such as [^a] always matches newline characters, independent of the set- ting of this option. PCRE_DUPNAMES - If this bit is set, names used to identify capturing subpatterns need + If this bit is set, names used to identify capturing subpatterns need not be unique. This can be helpful for certain types of pattern when it - is known that only one instance of the named subpattern can ever be - matched. There are more details of named subpatterns below; see also + is known that only one instance of the named subpattern can ever be + matched. There are more details of named subpatterns below; see also the pcrepattern documentation. PCRE_EXTENDED - If this bit is set, whitespace data characters in the pattern are + If this bit is set, whitespace data characters in the pattern are totally ignored except when escaped or inside a character class. White- space does not include the VT character (code 11). In addition, charac- ters between an unescaped # outside a character class and the next new- - line, inclusive, are also ignored. This is equivalent to Perl's /x - option, and it can be changed within a pattern by a (?x) option set- + line, inclusive, are also ignored. This is equivalent to Perl's /x + option, and it can be changed within a pattern by a (?x) option set- ting. - Which characters are interpreted as newlines is controlled by the - options passed to pcre_compile() or by a special sequence at the start - of the pattern, as described in the section entitled "Newline conven- + Which characters are interpreted as newlines is controlled by the + options passed to pcre_compile() or by a special sequence at the start + of the pattern, as described in the section entitled "Newline conven- tions" in the pcrepattern documentation. Note that the end of this type - of comment is a literal newline sequence in the pattern; escape + of comment is a literal newline sequence in the pattern; escape sequences that happen to represent a newline do not count. - This option makes it possible to include comments inside complicated - patterns. Note, however, that this applies only to data characters. - Whitespace characters may never appear within special character + This option makes it possible to include comments inside complicated + patterns. Note, however, that this applies only to data characters. + Whitespace characters may never appear within special character sequences in a pattern, for example within the sequence (?( that intro- duces a conditional subpattern. PCRE_EXTRA - This option was invented in order to turn on additional functionality - of PCRE that is incompatible with Perl, but it is currently of very - little use. When set, any backslash in a pattern that is followed by a - letter that has no special meaning causes an error, thus reserving - these combinations for future expansion. By default, as in Perl, a - backslash followed by a letter with no special meaning is treated as a + This option was invented in order to turn on additional functionality + of PCRE that is incompatible with Perl, but it is currently of very + little use. When set, any backslash in a pattern that is followed by a + letter that has no special meaning causes an error, thus reserving + these combinations for future expansion. By default, as in Perl, a + backslash followed by a letter with no special meaning is treated as a literal. (Perl can, however, be persuaded to give an error for this, by - running it with the -w option.) There are at present no other features - controlled by this option. It can also be set by a (?X) option setting + running it with the -w option.) There are at present no other features + controlled by this option. It can also be set by a (?X) option setting within a pattern. PCRE_FIRSTLINE - If this option is set, an unanchored pattern is required to match - before or at the first newline in the subject string, though the + If this option is set, an unanchored pattern is required to match + before or at the first newline in the subject string, though the matched text may continue over the newline. PCRE_JAVASCRIPT_COMPAT If this option is set, PCRE's behaviour is changed in some ways so that - it is compatible with JavaScript rather than Perl. The changes are as + it is compatible with JavaScript rather than Perl. The changes are as follows: - (1) A lone closing square bracket in a pattern causes a compile-time - error, because this is illegal in JavaScript (by default it is treated + (1) A lone closing square bracket in a pattern causes a compile-time + error, because this is illegal in JavaScript (by default it is treated as a data character). Thus, the pattern AB]CD becomes illegal when this option is set. - (2) At run time, a back reference to an unset subpattern group matches - an empty string (by default this causes the current matching alterna- - tive to fail). A pattern such as (\1)(a) succeeds when this option is - set (assuming it can find an "a" in the subject), whereas it fails by + (2) At run time, a back reference to an unset subpattern group matches + an empty string (by default this causes the current matching alterna- + tive to fail). A pattern such as (\1)(a) succeeds when this option is + set (assuming it can find an "a" in the subject), whereas it fails by default, for Perl compatibility. PCRE_MULTILINE - By default, PCRE treats the subject string as consisting of a single - line of characters (even if it actually contains newlines). The "start - of line" metacharacter (^) matches only at the start of the string, - while the "end of line" metacharacter ($) matches only at the end of + By default, PCRE treats the subject string as consisting of a single + line of characters (even if it actually contains newlines). The "start + of line" metacharacter (^) matches only at the start of the string, + while the "end of line" metacharacter ($) matches only at the end of the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY is set). This is the same as Perl. - When PCRE_MULTILINE it is set, the "start of line" and "end of line" - constructs match immediately following or immediately before internal - newlines in the subject string, respectively, as well as at the very - start and end. This is equivalent to Perl's /m option, and it can be + When PCRE_MULTILINE it is set, the "start of line" and "end of line" + constructs match immediately following or immediately before internal + newlines in the subject string, respectively, as well as at the very + start and end. This is equivalent to Perl's /m option, and it can be changed within a pattern by a (?m) option setting. If there are no new- - lines in a subject string, or no occurrences of ^ or $ in a pattern, + lines in a subject string, or no occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no effect. PCRE_NEWLINE_CR @@ -1358,32 +1359,32 @@ COMPILING A PATTERN PCRE_NEWLINE_ANYCRLF PCRE_NEWLINE_ANY - These options override the default newline definition that was chosen - when PCRE was built. Setting the first or the second specifies that a - newline is indicated by a single character (CR or LF, respectively). - Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the - two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies + These options override the default newline definition that was chosen + when PCRE was built. Setting the first or the second specifies that a + newline is indicated by a single character (CR or LF, respectively). + Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the + two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies that any of the three preceding sequences should be recognized. Setting - PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be + PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be recognized. The Unicode newline sequences are the three just mentioned, - plus the single characters VT (vertical tab, U+000B), FF (formfeed, - U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS - (paragraph separator, U+2029). The last two are recognized only in + plus the single characters VT (vertical tab, U+000B), FF (formfeed, + U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS + (paragraph separator, U+2029). The last two are recognized only in UTF-8 mode. - The newline setting in the options word uses three bits that are + The newline setting in the options word uses three bits that are treated as a number, giving eight possibilities. Currently only six are - used (default plus the five values above). This means that if you set - more than one newline option, the combination may or may not be sensi- + used (default plus the five values above). This means that if you set + more than one newline option, the combination may or may not be sensi- ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to - PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and + PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and cause an error. - The only time that a line break in a pattern is specially recognized - when compiling is when PCRE_EXTENDED is set. CR and LF are whitespace - characters, and so are ignored in this mode. Also, an unescaped # out- - side a character class indicates a comment that lasts until after the - next line break sequence. In other circumstances, line break sequences + The only time that a line break in a pattern is specially recognized + when compiling is when PCRE_EXTENDED is set. CR and LF are whitespace + characters, and so are ignored in this mode. Also, an unescaped # out- + side a character class indicates a comment that lasts until after the + next line break sequence. In other circumstances, line break sequences in patterns are treated as literal data. The newline option that is set at compile time becomes the default that @@ -1392,57 +1393,65 @@ COMPILING A PATTERN PCRE_NO_AUTO_CAPTURE If this option is set, it disables the use of numbered capturing paren- - theses in the pattern. Any opening parenthesis that is not followed by - ? behaves as if it were followed by ?: but named parentheses can still - be used for capturing (and they acquire numbers in the usual way). + theses in the pattern. Any opening parenthesis that is not followed by + ? behaves as if it were followed by ?: but named parentheses can still + be used for capturing (and they acquire numbers in the usual way). There is no equivalent of this option in Perl. + NO_START_OPTIMIZE + + This is an option that acts at matching time; that is, it is really an + option for pcre_exec() or pcre_dfa_exec(). If it is set at compile + time, it is remembered with the compiled pattern and assumed at match- + ing time. For details see the discussion of PCRE_NO_START_OPTIMIZE + below. + PCRE_UCP - This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, - \w, and some of the POSIX character classes. By default, only ASCII - characters are recognized, but if PCRE_UCP is set, Unicode properties - are used instead to classify characters. More details are given in the - section on generic character types in the pcrepattern page. If you set - PCRE_UCP, matching one of the items it affects takes much longer. The - option is available only if PCRE has been compiled with Unicode prop- + This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, + \w, and some of the POSIX character classes. By default, only ASCII + characters are recognized, but if PCRE_UCP is set, Unicode properties + are used instead to classify characters. More details are given in the + section on generic character types in the pcrepattern page. If you set + PCRE_UCP, matching one of the items it affects takes much longer. The + option is available only if PCRE has been compiled with Unicode prop- erty support. PCRE_UNGREEDY - This option inverts the "greediness" of the quantifiers so that they - are not greedy by default, but become greedy if followed by "?". It is - not compatible with Perl. It can also be set by a (?U) option setting + This option inverts the "greediness" of the quantifiers so that they + are not greedy by default, but become greedy if followed by "?". It is + not compatible with Perl. It can also be set by a (?U) option setting within the pattern. PCRE_UTF8 - This option causes PCRE to regard both the pattern and the subject as - strings of UTF-8 characters instead of single-byte character strings. - However, it is available only when PCRE is built to include UTF-8 sup- - port. If not, the use of this option provokes an error. Details of how - this option changes the behaviour of PCRE are given in the section on + This option causes PCRE to regard both the pattern and the subject as + strings of UTF-8 characters instead of single-byte character strings. + However, it is available only when PCRE is built to include UTF-8 sup- + port. If not, the use of this option provokes an error. Details of how + this option changes the behaviour of PCRE are given in the section on UTF-8 support in the main pcre page. PCRE_NO_UTF8_CHECK When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is - automatically checked. There is a discussion about the validity of - UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of - bytes is found, pcre_compile() returns an error. If you already know + automatically checked. There is a discussion about the validity of + UTF-8 strings in the main pcre page. If an invalid UTF-8 sequence of + bytes is found, pcre_compile() returns an error. If you already know that your pattern is valid, and you want to skip this check for perfor- - mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is - set, the effect of passing an invalid UTF-8 string as a pattern is - undefined. It may cause your program to crash. Note that this option - can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the + mance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is + set, the effect of passing an invalid UTF-8 string as a pattern is + undefined. It may cause your program to crash. Note that this option + can also be passed to pcre_exec() and pcre_dfa_exec(), to suppress the UTF-8 validity checking of subject strings. COMPILATION ERROR CODES - The following table lists the error codes than may be returned by - pcre_compile2(), along with the error messages that may be returned by - both compiling functions. As PCRE has developed, some error codes have + The following table lists the error codes than may be returned by + pcre_compile2(), along with the error messages that may be returned by + both compiling functions. As PCRE has developed, some error codes have fallen out of use. To avoid confusion, they have not been re-used. 0 no error @@ -1517,7 +1526,7 @@ COMPILATION ERROR CODES 66 (*MARK) must have an argument 67 this version of PCRE is not compiled with PCRE_UCP support - The numbers 32 and 10000 in errors 48 and 49 are defaults; different + The numbers 32 and 10000 in errors 48 and 49 are defaults; different values may be used if the limits were changed when PCRE was built. @@ -1526,32 +1535,32 @@ STUDYING A PATTERN pcre_extra *pcre_study(const pcre *code, int options const char **errptr); - If a compiled pattern is going to be used several times, it is worth + If a compiled pattern is going to be used several times, it is worth spending more time analyzing it in order to speed up the time taken for - matching. The function pcre_study() takes a pointer to a compiled pat- + matching. The function pcre_study() takes a pointer to a compiled pat- tern as its first argument. If studying the pattern produces additional - information that will help speed up matching, pcre_study() returns a - pointer to a pcre_extra block, in which the study_data field points to + information that will help speed up matching, pcre_study() returns a + pointer to a pcre_extra block, in which the study_data field points to the results of the study. The returned value from pcre_study() can be passed directly to - pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- - tains other fields that can be set by the caller before the block is + pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- + tains other fields that can be set by the caller before the block is passed; these are described below in the section on matching a pattern. - If studying the pattern does not produce any useful information, + If studying the pattern does not produce any useful information, pcre_study() returns NULL. In that circumstance, if the calling program - wants to pass any of the other fields to pcre_exec() or + wants to pass any of the other fields to pcre_exec() or pcre_dfa_exec(), it must set up its own pcre_extra block. - The second argument of pcre_study() contains option bits. At present, + The second argument of pcre_study() contains option bits. At present, no options are defined, and this argument should always be zero. - The third argument for pcre_study() is a pointer for an error message. - If studying succeeds (even if no data is returned), the variable it - points to is set to NULL. Otherwise it is set to point to a textual + The third argument for pcre_study() is a pointer for an error message. + If studying succeeds (even if no data is returned), the variable it + points to is set to NULL. Otherwise it is set to point to a textual error message. This is a static string that is part of the library. You - must not try to free it. You should test the error pointer for NULL + must not try to free it. You should test the error pointer for NULL after calling pcre_study(), to be sure that it has run successfully. This is a typical call to pcre_study(): @@ -1565,78 +1574,78 @@ STUDYING A PATTERN Studying a pattern does two things: first, a lower bound for the length of subject string that is needed to match the pattern is computed. This does not mean that there are any strings of that length that match, but - it does guarantee that no shorter strings match. The value is used by - pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to - match strings that are shorter than the lower bound. You can find out + it does guarantee that no shorter strings match. The value is used by + pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to + match strings that are shorter than the lower bound. You can find out the value in a calling program via the pcre_fullinfo() function. Studying a pattern is also useful for non-anchored patterns that do not - have a single fixed starting character. A bitmap of possible starting - bytes is created. This speeds up finding a position in the subject at + have a single fixed starting character. A bitmap of possible starting + bytes is created. This speeds up finding a position in the subject at which to start matching. - The two optimizations just described can be disabled by setting the - PCRE_NO_START_OPTIMIZE option when calling pcre_exec() or - pcre_dfa_exec(). You might want to do this if your pattern contains - callouts or (*MARK), and you want to make use of these facilities in - cases where matching fails. See the discussion of PCRE_NO_START_OPTI- + The two optimizations just described can be disabled by setting the + PCRE_NO_START_OPTIMIZE option when calling pcre_exec() or + pcre_dfa_exec(). You might want to do this if your pattern contains + callouts or (*MARK), and you want to make use of these facilities in + cases where matching fails. See the discussion of PCRE_NO_START_OPTI- MIZE below. LOCALE SUPPORT - PCRE handles caseless matching, and determines whether characters are - letters, digits, or whatever, by reference to a set of tables, indexed - by character value. When running in UTF-8 mode, this applies only to - characters with codes less than 128. By default, higher-valued codes + PCRE handles caseless matching, and determines whether characters are + letters, digits, or whatever, by reference to a set of tables, indexed + by character value. When running in UTF-8 mode, this applies only to + characters with codes less than 128. By default, higher-valued codes never match escapes such as \w or \d, but they can be tested with \p if - PCRE is built with Unicode character property support. Alternatively, - the PCRE_UCP option can be set at compile time; this causes \w and + PCRE is built with Unicode character property support. Alternatively, + the PCRE_UCP option can be set at compile time; this causes \w and friends to use Unicode property support instead of built-in tables. The use of locales with Unicode is discouraged. If you are handling charac- - ters with codes greater than 128, you should either use UTF-8 and Uni- + ters with codes greater than 128, you should either use UTF-8 and Uni- code, or use locales, but not try to mix the two. - PCRE contains an internal set of tables that are used when the final - argument of pcre_compile() is NULL. These are sufficient for many + PCRE contains an internal set of tables that are used when the final + argument of pcre_compile() is NULL. These are sufficient for many applications. Normally, the internal tables recognize only ASCII char- acters. However, when PCRE is built, it is possible to cause the inter- nal tables to be rebuilt in the default "C" locale of the local system, which may cause them to be different. - The internal tables can always be overridden by tables supplied by the + The internal tables can always be overridden by tables supplied by the application that calls PCRE. These may be created in a different locale - from the default. As more and more applications change to using Uni- + from the default. As more and more applications change to using Uni- code, the need for this locale support is expected to die away. - External tables are built by calling the pcre_maketables() function, - which has no arguments, in the relevant locale. The result can then be - passed to pcre_compile() or pcre_exec() as often as necessary. For - example, to build and use tables that are appropriate for the French - locale (where accented characters with values greater than 128 are + External tables are built by calling the pcre_maketables() function, + which has no arguments, in the relevant locale. The result can then be + passed to pcre_compile() or pcre_exec() as often as necessary. For + example, to build and use tables that are appropriate for the French + locale (where accented characters with values greater than 128 are treated as letters), the following code could be used: setlocale(LC_CTYPE, "fr_FR"); tables = pcre_maketables(); re = pcre_compile(..., tables); - The locale name "fr_FR" is used on Linux and other Unix-like systems; + The locale name "fr_FR" is used on Linux and other Unix-like systems; if you are using Windows, the name for the French locale is "french". - When pcre_maketables() runs, the tables are built in memory that is - obtained via pcre_malloc. It is the caller's responsibility to ensure - that the memory containing the tables remains available for as long as + When pcre_maketables() runs, the tables are built in memory that is + obtained via pcre_malloc. It is the caller's responsibility to ensure + that the memory containing the tables remains available for as long as it is needed. The pointer that is passed to pcre_compile() is saved with the compiled - pattern, and the same tables are used via this pointer by pcre_study() + pattern, and the same tables are used via this pointer by pcre_study() and normally also by pcre_exec(). Thus, by default, for any single pat- tern, compilation, studying and matching all happen in the same locale, but different patterns can be compiled in different locales. - It is possible to pass a table pointer or NULL (indicating the use of - the internal tables) to pcre_exec(). Although not intended for this - purpose, this facility could be used to match a pattern in a different + It is possible to pass a table pointer or NULL (indicating the use of + the internal tables) to pcre_exec(). Although not intended for this + purpose, this facility could be used to match a pattern in a different locale from the one in which it was compiled. Passing table pointers at run time is discussed below in the section on matching a pattern. @@ -1646,15 +1655,15 @@ INFORMATION ABOUT A PATTERN int pcre_fullinfo(const pcre *code, const pcre_extra *extra, int what, void *where); - The pcre_fullinfo() function returns information about a compiled pat- + The pcre_fullinfo() function returns information about a compiled pat- tern. It replaces the obsolete pcre_info() function, which is neverthe- less retained for backwards compability (and is documented below). - The first argument for pcre_fullinfo() is a pointer to the compiled - pattern. The second argument is the result of pcre_study(), or NULL if - the pattern was not studied. The third argument specifies which piece - of information is required, and the fourth argument is a pointer to a - variable to receive the data. The yield of the function is zero for + The first argument for pcre_fullinfo() is a pointer to the compiled + pattern. The second argument is the result of pcre_study(), or NULL if + the pattern was not studied. The third argument specifies which piece + of information is required, and the fourth argument is a pointer to a + variable to receive the data. The yield of the function is zero for success, or one of the following negative numbers: PCRE_ERROR_NULL the argument code was NULL @@ -1662,9 +1671,9 @@ INFORMATION ABOUT A PATTERN PCRE_ERROR_BADMAGIC the "magic number" was not found PCRE_ERROR_BADOPTION the value of what was invalid - The "magic number" is placed at the start of each compiled pattern as - an simple check against passing an arbitrary memory pointer. Here is a - typical call of pcre_fullinfo(), to obtain the length of the compiled + The "magic number" is placed at the start of each compiled pattern as + an simple check against passing an arbitrary memory pointer. Here is a + typical call of pcre_fullinfo(), to obtain the length of the compiled pattern: int rc; @@ -1675,131 +1684,131 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE, /* what is required */ &length); /* where to put the data */ - The possible values for the third argument are defined in pcre.h, and + The possible values for the third argument are defined in pcre.h, and are as follows: PCRE_INFO_BACKREFMAX - Return the number of the highest back reference in the pattern. The - fourth argument should point to an int variable. Zero is returned if + Return the number of the highest back reference in the pattern. The + fourth argument should point to an int variable. Zero is returned if there are no back references. PCRE_INFO_CAPTURECOUNT - Return the number of capturing subpatterns in the pattern. The fourth + Return the number of capturing subpatterns in the pattern. The fourth argument should point to an int variable. PCRE_INFO_DEFAULT_TABLES - Return a pointer to the internal default character tables within PCRE. - The fourth argument should point to an unsigned char * variable. This + Return a pointer to the internal default character tables within PCRE. + The fourth argument should point to an unsigned char * variable. This information call is provided for internal use by the pcre_study() func- - tion. External callers can cause PCRE to use its internal tables by + tion. External callers can cause PCRE to use its internal tables by passing a NULL table pointer. PCRE_INFO_FIRSTBYTE - Return information about the first byte of any matched string, for a - non-anchored pattern. The fourth argument should point to an int vari- - able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name + Return information about the first byte of any matched string, for a + non-anchored pattern. The fourth argument should point to an int vari- + able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name is still recognized for backwards compatibility.) - If there is a fixed first byte, for example, from a pattern such as + If there is a fixed first byte, for example, from a pattern such as (cat|cow|coyote), its value is returned. Otherwise, if either - (a) the pattern was compiled with the PCRE_MULTILINE option, and every + (a) the pattern was compiled with the PCRE_MULTILINE option, and every branch starts with "^", or (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set (if it were set, the pattern would be anchored), - -1 is returned, indicating that the pattern matches only at the start - of a subject string or after any newline within the string. Otherwise + -1 is returned, indicating that the pattern matches only at the start + of a subject string or after any newline within the string. Otherwise -2 is returned. For anchored patterns, -2 is returned. PCRE_INFO_FIRSTTABLE - If the pattern was studied, and this resulted in the construction of a + If the pattern was studied, and this resulted in the construction of a 256-bit table indicating a fixed set of bytes for the first byte in any - matching string, a pointer to the table is returned. Otherwise NULL is - returned. The fourth argument should point to an unsigned char * vari- + matching string, a pointer to the table is returned. Otherwise NULL is + returned. The fourth argument should point to an unsigned char * vari- able. PCRE_INFO_HASCRORLF - Return 1 if the pattern contains any explicit matches for CR or LF - characters, otherwise 0. The fourth argument should point to an int - variable. An explicit match is either a literal CR or LF character, or + Return 1 if the pattern contains any explicit matches for CR or LF + characters, otherwise 0. The fourth argument should point to an int + variable. An explicit match is either a literal CR or LF character, or \r or \n. PCRE_INFO_JCHANGED - Return 1 if the (?J) or (?-J) option setting is used in the pattern, - otherwise 0. The fourth argument should point to an int variable. (?J) + Return 1 if the (?J) or (?-J) option setting is used in the pattern, + otherwise 0. The fourth argument should point to an int variable. (?J) and (?-J) set and unset the local PCRE_DUPNAMES option, respectively. PCRE_INFO_LASTLITERAL - Return the value of the rightmost literal byte that must exist in any - matched string, other than at its start, if such a byte has been + Return the value of the rightmost literal byte that must exist in any + matched string, other than at its start, if such a byte has been recorded. The fourth argument should point to an int variable. If there - is no such byte, -1 is returned. For anchored patterns, a last literal - byte is recorded only if it follows something of variable length. For + is no such byte, -1 is returned. For anchored patterns, a last literal + byte is recorded only if it follows something of variable length. For example, for the pattern /^a\d+z\d+/ the returned value is "z", but for /^a\dz\d/ the returned value is -1. PCRE_INFO_MINLENGTH - If the pattern was studied and a minimum length for matching subject - strings was computed, its value is returned. Otherwise the returned - value is -1. The value is a number of characters, not bytes (this may - be relevant in UTF-8 mode). The fourth argument should point to an int - variable. A non-negative value is a lower bound to the length of any - matching string. There may not be any strings of that length that do + If the pattern was studied and a minimum length for matching subject + strings was computed, its value is returned. Otherwise the returned + value is -1. The value is a number of characters, not bytes (this may + be relevant in UTF-8 mode). The fourth argument should point to an int + variable. A non-negative value is a lower bound to the length of any + matching string. There may not be any strings of that length that do actually match, but every string that does match is at least that long. PCRE_INFO_NAMECOUNT PCRE_INFO_NAMEENTRYSIZE PCRE_INFO_NAMETABLE - PCRE supports the use of named as well as numbered capturing parenthe- - ses. The names are just an additional way of identifying the parenthe- + PCRE supports the use of named as well as numbered capturing parenthe- + ses. The names are just an additional way of identifying the parenthe- ses, which still acquire numbers. Several convenience functions such as - pcre_get_named_substring() are provided for extracting captured sub- - strings by name. It is also possible to extract the data directly, by - first converting the name to a number in order to access the correct + pcre_get_named_substring() are provided for extracting captured sub- + strings by name. It is also possible to extract the data directly, by + first converting the name to a number in order to access the correct pointers in the output vector (described with pcre_exec() below). To do - the conversion, you need to use the name-to-number map, which is + the conversion, you need to use the name-to-number map, which is described by these three values. The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size - of each entry; both of these return an int value. The entry size - depends on the length of the longest name. PCRE_INFO_NAMETABLE returns - a pointer to the first entry of the table (a pointer to char). The + of each entry; both of these return an int value. The entry size + depends on the length of the longest name. PCRE_INFO_NAMETABLE returns + a pointer to the first entry of the table (a pointer to char). The first two bytes of each entry are the number of the capturing parenthe- - sis, most significant byte first. The rest of the entry is the corre- + sis, most significant byte first. The rest of the entry is the corre- sponding name, zero terminated. - The names are in alphabetical order. Duplicate names may appear if (?| + The names are in alphabetical order. Duplicate names may appear if (?| is used to create multiple groups with the same number, as described in - the section on duplicate subpattern numbers in the pcrepattern page. - Duplicate names for subpatterns with different numbers are permitted - only if PCRE_DUPNAMES is set. In all cases of duplicate names, they - appear in the table in the order in which they were found in the pat- - tern. In the absence of (?| this is the order of increasing number; + the section on duplicate subpattern numbers in the pcrepattern page. + Duplicate names for subpatterns with different numbers are permitted + only if PCRE_DUPNAMES is set. In all cases of duplicate names, they + appear in the table in the order in which they were found in the pat- + tern. In the absence of (?| this is the order of increasing number; when (?| is used this is not necessarily the case because later subpat- terns may have lower numbers. - As a simple example of the name/number table, consider the following - pattern (assume PCRE_EXTENDED is set, so white space - including new- + As a simple example of the name/number table, consider the following + pattern (assume PCRE_EXTENDED is set, so white space - including new- lines - is ignored): (?<date> (?<year>(\d\d)?\d\d) - (?<month>\d\d) - (?<day>\d\d) ) - There are four named subpatterns, so the table has four entries, and - each entry in the table is eight bytes long. The table is as follows, + There are four named subpatterns, so the table has four entries, and + each entry in the table is eight bytes long. The table is as follows, with non-printing bytes shows in hexadecimal, and undefined bytes shown as ??: @@ -1808,31 +1817,31 @@ INFORMATION ABOUT A PATTERN 00 04 m o n t h 00 00 02 y e a r 00 ?? - When writing code to extract data from named subpatterns using the - name-to-number map, remember that the length of the entries is likely + When writing code to extract data from named subpatterns using the + name-to-number map, remember that the length of the entries is likely to be different for each compiled pattern. PCRE_INFO_OKPARTIAL - Return 1 if the pattern can be used for partial matching with - pcre_exec(), otherwise 0. The fourth argument should point to an int - variable. From release 8.00, this always returns 1, because the - restrictions that previously applied to partial matching have been - lifted. The pcrepartial documentation gives details of partial match- + Return 1 if the pattern can be used for partial matching with + pcre_exec(), otherwise 0. The fourth argument should point to an int + variable. From release 8.00, this always returns 1, because the + restrictions that previously applied to partial matching have been + lifted. The pcrepartial documentation gives details of partial match- ing. PCRE_INFO_OPTIONS - Return a copy of the options with which the pattern was compiled. The - fourth argument should point to an unsigned long int variable. These + Return a copy of the options with which the pattern was compiled. The + fourth argument should point to an unsigned long int variable. These option bits are those specified in the call to pcre_compile(), modified by any top-level option settings at the start of the pattern itself. In - other words, they are the options that will be in force when matching - starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with - the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, + other words, they are the options that will be in force when matching + starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with + the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, and PCRE_EXTENDED. - A pattern is automatically anchored by PCRE if all of its top-level + A pattern is automatically anchored by PCRE if all of its top-level alternatives begin with one of the following: ^ unless PCRE_MULTILINE is set @@ -1846,7 +1855,7 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE - Return the size of the compiled pattern, that is, the value that was + Return the size of the compiled pattern, that is, the value that was passed as the argument to pcre_malloc() when PCRE was getting memory in which to place the compiled data. The fourth argument should point to a size_t variable. @@ -1854,10 +1863,10 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_STUDYSIZE Return the size of the data block pointed to by the study_data field in - a pcre_extra block. That is, it is the value that was passed to + a pcre_extra block. That is, it is the value that was passed to pcre_malloc() when PCRE was getting memory into which to place the data - created by pcre_study(). If pcre_extra is NULL, or there is no study - data, zero is returned. The fourth argument should point to a size_t + created by pcre_study(). If pcre_extra is NULL, or there is no study + data, zero is returned. The fourth argument should point to a size_t variable. @@ -1865,21 +1874,21 @@ OBSOLETE INFO FUNCTION int pcre_info(const pcre *code, int *optptr, int *firstcharptr); - The pcre_info() function is now obsolete because its interface is too - restrictive to return all the available data about a compiled pattern. - New programs should use pcre_fullinfo() instead. The yield of - pcre_info() is the number of capturing subpatterns, or one of the fol- + The pcre_info() function is now obsolete because its interface is too + restrictive to return all the available data about a compiled pattern. + New programs should use pcre_fullinfo() instead. The yield of + pcre_info() is the number of capturing subpatterns, or one of the fol- lowing negative numbers: PCRE_ERROR_NULL the argument code was NULL PCRE_ERROR_BADMAGIC the "magic number" was not found - If the optptr argument is not NULL, a copy of the options with which - the pattern was compiled is placed in the integer it points to (see + If the optptr argument is not NULL, a copy of the options with which + the pattern was compiled is placed in the integer it points to (see PCRE_INFO_OPTIONS above). - If the pattern is not anchored and the firstcharptr argument is not - NULL, it is used to pass back information about the first character of + If the pattern is not anchored and the firstcharptr argument is not + NULL, it is used to pass back information about the first character of any matched string (see PCRE_INFO_FIRSTBYTE above). @@ -1887,21 +1896,21 @@ REFERENCE COUNTS int pcre_refcount(pcre *code, int adjust); - The pcre_refcount() function is used to maintain a reference count in + The pcre_refcount() function is used to maintain a reference count in the data block that contains a compiled pattern. It is provided for the - benefit of applications that operate in an object-oriented manner, + benefit of applications that operate in an object-oriented manner, where different parts of the application may be using the same compiled pattern, but you want to free the block when they are all done. When a pattern is compiled, the reference count field is initialized to - zero. It is changed only by calling this function, whose action is to - add the adjust value (which may be positive or negative) to it. The + zero. It is changed only by calling this function, whose action is to + add the adjust value (which may be positive or negative) to it. The yield of the function is the new value. However, the value of the count - is constrained to lie between 0 and 65535, inclusive. If the new value + is constrained to lie between 0 and 65535, inclusive. If the new value is outside these limits, it is forced to the appropriate limit value. - Except when it is zero, the reference count is not correctly preserved - if a pattern is compiled on one host and then transferred to a host + Except when it is zero, the reference count is not correctly preserved + if a pattern is compiled on one host and then transferred to a host whose byte-order is different. (This seems a highly unlikely scenario.) @@ -1911,18 +1920,18 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION const char *subject, int length, int startoffset, int options, int *ovector, int ovecsize); - The function pcre_exec() is called to match a subject string against a - compiled pattern, which is passed in the code argument. If the pattern - was studied, the result of the study should be passed in the extra - argument. This function is the main matching facility of the library, + The function pcre_exec() is called to match a subject string against a + compiled pattern, which is passed in the code argument. If the pattern + was studied, the result of the study should be passed in the extra + argument. This function is the main matching facility of the library, and it operates in a Perl-like manner. For specialist use there is also - an alternative matching function, which is described below in the sec- + an alternative matching function, which is described below in the sec- tion about the pcre_dfa_exec() function. - In most applications, the pattern will have been compiled (and option- - ally studied) in the same process that calls pcre_exec(). However, it + In most applications, the pattern will have been compiled (and option- + ally studied) in the same process that calls pcre_exec(). However, it is possible to save compiled patterns and study data, and then use them - later in different processes, possibly even on different hosts. For a + later in different processes, possibly even on different hosts. For a discussion about this, see the pcreprecompile documentation. Here is an example of a simple call to pcre_exec(): @@ -1941,10 +1950,10 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION Extra data for pcre_exec() - If the extra argument is not NULL, it must point to a pcre_extra data - block. The pcre_study() function returns such a block (when it doesn't - return NULL), but you can also create one for yourself, and pass addi- - tional information in it. The pcre_extra block contains the following + If the extra argument is not NULL, it must point to a pcre_extra data + block. The pcre_study() function returns such a block (when it doesn't + return NULL), but you can also create one for yourself, and pass addi- + tional information in it. The pcre_extra block contains the following fields (not necessarily in this order): unsigned long int flags; @@ -1955,7 +1964,7 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION const unsigned char *tables; unsigned char **mark; - The flags field is a bitmap that specifies which of the other fields + The flags field is a bitmap that specifies which of the other fields are set. The flag bits are: PCRE_EXTRA_STUDY_DATA @@ -1965,96 +1974,96 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_EXTRA_TABLES PCRE_EXTRA_MARK - Other flag bits should be set to zero. The study_data field is set in - the pcre_extra block that is returned by pcre_study(), together with + Other flag bits should be set to zero. The study_data field is set in + the pcre_extra block that is returned by pcre_study(), together with the appropriate flag bit. You should not set this yourself, but you may - add to the block by setting the other fields and their corresponding + add to the block by setting the other fields and their corresponding flag bits. The match_limit field provides a means of preventing PCRE from using up - a vast amount of resources when running patterns that are not going to - match, but which have a very large number of possibilities in their - search trees. The classic example is a pattern that uses nested unlim- + a vast amount of resources when running patterns that are not going to + match, but which have a very large number of possibilities in their + search trees. The classic example is a pattern that uses nested unlim- ited repeats. - Internally, PCRE uses a function called match() which it calls repeat- - edly (sometimes recursively). The limit set by match_limit is imposed - on the number of times this function is called during a match, which - has the effect of limiting the amount of backtracking that can take + Internally, PCRE uses a function called match() which it calls repeat- + edly (sometimes recursively). The limit set by match_limit is imposed + on the number of times this function is called during a match, which + has the effect of limiting the amount of backtracking that can take place. For patterns that are not anchored, the count restarts from zero for each position in the subject string. - The default value for the limit can be set when PCRE is built; the - default default is 10 million, which handles all but the most extreme - cases. You can override the default by suppling pcre_exec() with a - pcre_extra block in which match_limit is set, and - PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is + The default value for the limit can be set when PCRE is built; the + default default is 10 million, which handles all but the most extreme + cases. You can override the default by suppling pcre_exec() with a + pcre_extra block in which match_limit is set, and + PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT. - The match_limit_recursion field is similar to match_limit, but instead + The match_limit_recursion field is similar to match_limit, but instead of limiting the total number of times that match() is called, it limits - the depth of recursion. The recursion depth is a smaller number than - the total number of calls, because not all calls to match() are recur- + the depth of recursion. The recursion depth is a smaller number than + the total number of calls, because not all calls to match() are recur- sive. This limit is of use only if it is set smaller than match_limit. - Limiting the recursion depth limits the amount of stack that can be + Limiting the recursion depth limits the amount of stack that can be used, or, when PCRE has been compiled to use memory on the heap instead of the stack, the amount of heap memory that can be used. - The default value for match_limit_recursion can be set when PCRE is - built; the default default is the same value as the default for - match_limit. You can override the default by suppling pcre_exec() with - a pcre_extra block in which match_limit_recursion is set, and - PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the + The default value for match_limit_recursion can be set when PCRE is + built; the default default is the same value as the default for + match_limit. You can override the default by suppling pcre_exec() with + a pcre_extra block in which match_limit_recursion is set, and + PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. - The callout_data field is used in conjunction with the "callout" fea- + The callout_data field is used in conjunction with the "callout" fea- ture, and is described in the pcrecallout documentation. - The tables field is used to pass a character tables pointer to - pcre_exec(); this overrides the value that is stored with the compiled - pattern. A non-NULL value is stored with the compiled pattern only if - custom tables were supplied to pcre_compile() via its tableptr argu- + The tables field is used to pass a character tables pointer to + pcre_exec(); this overrides the value that is stored with the compiled + pattern. A non-NULL value is stored with the compiled pattern only if + custom tables were supplied to pcre_compile() via its tableptr argu- ment. If NULL is passed to pcre_exec() using this mechanism, it forces - PCRE's internal tables to be used. This facility is helpful when re- - using patterns that have been saved after compiling with an external - set of tables, because the external tables might be at a different - address when pcre_exec() is called. See the pcreprecompile documenta- + PCRE's internal tables to be used. This facility is helpful when re- + using patterns that have been saved after compiling with an external + set of tables, because the external tables might be at a different + address when pcre_exec() is called. See the pcreprecompile documenta- tion for a discussion of saving compiled patterns for later use. - If PCRE_EXTRA_MARK is set in the flags field, the mark field must be - set to point to a char * variable. If the pattern contains any back- - tracking control verbs such as (*MARK:NAME), and the execution ends up - with a name to pass back, a pointer to the name string (zero termi- - nated) is placed in the variable pointed to by the mark field. The - names are within the compiled pattern; if you wish to retain such a - name you must copy it before freeing the memory of a compiled pattern. - If there is no name to pass back, the variable pointed to by the mark - field set to NULL. For details of the backtracking control verbs, see + If PCRE_EXTRA_MARK is set in the flags field, the mark field must be + set to point to a char * variable. If the pattern contains any back- + tracking control verbs such as (*MARK:NAME), and the execution ends up + with a name to pass back, a pointer to the name string (zero termi- + nated) is placed in the variable pointed to by the mark field. The + names are within the compiled pattern; if you wish to retain such a + name you must copy it before freeing the memory of a compiled pattern. + If there is no name to pass back, the variable pointed to by the mark + field set to NULL. For details of the backtracking control verbs, see the section entitled "Backtracking control" in the pcrepattern documen- tation. Option bits for pcre_exec() - The unused bits of the options argument for pcre_exec() must be zero. - The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, - PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, - PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and + The unused bits of the options argument for pcre_exec() must be zero. + The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, + PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, + PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and PCRE_PARTIAL_HARD. PCRE_ANCHORED - The PCRE_ANCHORED option limits pcre_exec() to matching at the first - matching position. If a pattern was compiled with PCRE_ANCHORED, or - turned out to be anchored by virtue of its contents, it cannot be made + The PCRE_ANCHORED option limits pcre_exec() to matching at the first + matching position. If a pattern was compiled with PCRE_ANCHORED, or + turned out to be anchored by virtue of its contents, it cannot be made unachored at matching time. PCRE_BSR_ANYCRLF PCRE_BSR_UNICODE These options (which are mutually exclusive) control what the \R escape - sequence matches. The choice is either to match only CR, LF, or CRLF, - or to match any Unicode newline sequence. These options override the + sequence matches. The choice is either to match only CR, LF, or CRLF, + or to match any Unicode newline sequence. These options override the choice that was made or defaulted when the pattern was compiled. PCRE_NEWLINE_CR @@ -2063,107 +2072,110 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_NEWLINE_ANYCRLF PCRE_NEWLINE_ANY - These options override the newline definition that was chosen or - defaulted when the pattern was compiled. For details, see the descrip- - tion of pcre_compile() above. During matching, the newline choice - affects the behaviour of the dot, circumflex, and dollar metacharac- - ters. It may also alter the way the match position is advanced after a + These options override the newline definition that was chosen or + defaulted when the pattern was compiled. For details, see the descrip- + tion of pcre_compile() above. During matching, the newline choice + affects the behaviour of the dot, circumflex, and dollar metacharac- + ters. It may also alter the way the match position is advanced after a match failure for an unanchored pattern. - When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is - set, and a match attempt for an unanchored pattern fails when the cur- - rent position is at a CRLF sequence, and the pattern contains no - explicit matches for CR or LF characters, the match position is + When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is + set, and a match attempt for an unanchored pattern fails when the cur- + rent position is at a CRLF sequence, and the pattern contains no + explicit matches for CR or LF characters, the match position is advanced by two characters instead of one, in other words, to after the CRLF. The above rule is a compromise that makes the most common cases work as - expected. For example, if the pattern is .+A (and the PCRE_DOTALL + expected. For example, if the pattern is .+A (and the PCRE_DOTALL option is not set), it does not match the string "\r\nA" because, after - failing at the start, it skips both the CR and the LF before retrying. - However, the pattern [\r\n]A does match that string, because it con- + failing at the start, it skips both the CR and the LF before retrying. + However, the pattern [\r\n]A does match that string, because it con- tains an explicit CR or LF reference, and so advances only by one char- acter after the first failure. An explicit match for CR of LF is either a literal appearance of one of - those characters, or one of the \r or \n escape sequences. Implicit - matches such as [^X] do not count, nor does \s (which includes CR and + those characters, or one of the \r or \n escape sequences. Implicit + matches such as [^X] do not count, nor does \s (which includes CR and LF in the characters that it matches). - Notwithstanding the above, anomalous effects may still occur when CRLF + Notwithstanding the above, anomalous effects may still occur when CRLF is a valid newline sequence and explicit \r or \n escapes appear in the pattern. PCRE_NOTBOL This option specifies that first character of the subject string is not - the beginning of a line, so the circumflex metacharacter should not - match before it. Setting this without PCRE_MULTILINE (at compile time) - causes circumflex never to match. This option affects only the behav- + the beginning of a line, so the circumflex metacharacter should not + match before it. Setting this without PCRE_MULTILINE (at compile time) + causes circumflex never to match. This option affects only the behav- iour of the circumflex metacharacter. It does not affect \A. PCRE_NOTEOL This option specifies that the end of the subject string is not the end - of a line, so the dollar metacharacter should not match it nor (except - in multiline mode) a newline immediately before it. Setting this with- + of a line, so the dollar metacharacter should not match it nor (except + in multiline mode) a newline immediately before it. Setting this with- out PCRE_MULTILINE (at compile time) causes dollar never to match. This - option affects only the behaviour of the dollar metacharacter. It does + option affects only the behaviour of the dollar metacharacter. It does not affect \Z or \z. PCRE_NOTEMPTY An empty string is not considered to be a valid match if this option is - set. If there are alternatives in the pattern, they are tried. If all - the alternatives match the empty string, the entire match fails. For + set. If there are alternatives in the pattern, they are tried. If all + the alternatives match the empty string, the entire match fails. For example, if the pattern a?b? - is applied to a string not beginning with "a" or "b", it matches an - empty string at the start of the subject. With PCRE_NOTEMPTY set, this + is applied to a string not beginning with "a" or "b", it matches an + empty string at the start of the subject. With PCRE_NOTEMPTY set, this match is not valid, so PCRE searches further into the string for occur- rences of "a" or "b". PCRE_NOTEMPTY_ATSTART - This is like PCRE_NOTEMPTY, except that an empty string match that is - not at the start of the subject is permitted. If the pattern is + This is like PCRE_NOTEMPTY, except that an empty string match that is + not at the start of the subject is permitted. If the pattern is anchored, such a match can occur only if the pattern contains \K. - Perl has no direct equivalent of PCRE_NOTEMPTY or - PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern - match of the empty string within its split() function, and when using - the /g modifier. It is possible to emulate Perl's behaviour after + Perl has no direct equivalent of PCRE_NOTEMPTY or + PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern + match of the empty string within its split() function, and when using + the /g modifier. It is possible to emulate Perl's behaviour after matching a null string by first trying the match again at the same off- - set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that + set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that fails, by advancing the starting offset (see below) and trying an ordi- - nary match again. There is some code that demonstrates how to do this - in the pcredemo sample program. In the most general case, you have to - check to see if the newline convention recognizes CRLF as a newline, - and if so, and the current character is CR followed by LF, advance the + nary match again. There is some code that demonstrates how to do this + in the pcredemo sample program. In the most general case, you have to + check to see if the newline convention recognizes CRLF as a newline, + and if so, and the current character is CR followed by LF, advance the starting offset by two characters instead of one. PCRE_NO_START_OPTIMIZE - There are a number of optimizations that pcre_exec() uses at the start - of a match, in order to speed up the process. For example, if it is + There are a number of optimizations that pcre_exec() uses at the start + of a match, in order to speed up the process. For example, if it is known that an unanchored match must start with a specific character, it - searches the subject for that character, and fails immediately if it - cannot find it, without actually running the main matching function. + searches the subject for that character, and fails immediately if it + cannot find it, without actually running the main matching function. This means that a special item such as (*COMMIT) at the start of a pat- - tern is not considered until after a suitable starting point for the - match has been found. When callouts or (*MARK) items are in use, these + tern is not considered until after a suitable starting point for the + match has been found. When callouts or (*MARK) items are in use, these "start-up" optimizations can cause them to be skipped if the pattern is - never actually used. The start-up optimizations are in effect a pre- + never actually used. The start-up optimizations are in effect a pre- scan of the subject that takes place before the pattern is run. - The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, - possibly causing performance to suffer, but ensuring that in cases - where the result is "no match", the callouts do occur, and that items + The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, + possibly causing performance to suffer, but ensuring that in cases + where the result is "no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK) are considered at every possible starting - position in the subject string. Setting PCRE_NO_START_OPTIMIZE can - change the outcome of a matching operation. Consider the pattern + position in the subject string. If PCRE_NO_START_OPTIMIZE is set at + compile time, it cannot be unset at matching time. + + Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching + operation. Consider the pattern (*COMMIT)ABC @@ -2862,11 +2874,11 @@ AUTHOR REVISION - Last updated: 13 November 2010 + Last updated: 21 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRECALLOUT(3) PCRECALLOUT(3) @@ -2930,17 +2942,18 @@ MISSING CALLOUTS patterns, if it has been scanned far enough. You can disable these optimizations by passing the PCRE_NO_START_OPTI- - MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the - matching process, but does ensure that callouts such as the example - above are obeyed. + MIZE option to pcre_compile(), pcre_exec(), or pcre_dfa_exec(), or by + starting the pattern with (*NO_START_OPT). This slows down the matching + process, but does ensure that callouts such as the example above are + obeyed. THE CALLOUT INTERFACE - During matching, when PCRE reaches a callout point, the external func- - tion defined by pcre_callout is called (if it is set). This applies to - both the pcre_exec() and the pcre_dfa_exec() matching functions. The - only argument to the callout function is a pointer to a pcre_callout + During matching, when PCRE reaches a callout point, the external func- + tion defined by pcre_callout is called (if it is set). This applies to + both the pcre_exec() and the pcre_dfa_exec() matching functions. The + only argument to the callout function is a pointer to a pcre_callout block. This structure contains the following fields: int version; @@ -2956,81 +2969,81 @@ THE CALLOUT INTERFACE int pattern_position; int next_item_length; - The version field is an integer containing the version number of the - block format. The initial version was 0; the current version is 1. The - version number will change again in future if additional fields are + The version field is an integer containing the version number of the + block format. The initial version was 0; the current version is 1. The + version number will change again in future if additional fields are added, but the intention is never to remove any of the existing fields. - The callout_number field contains the number of the callout, as com- - piled into the pattern (that is, the number after ?C for manual call- + The callout_number field contains the number of the callout, as com- + piled into the pattern (that is, the number after ?C for manual call- outs, and 255 for automatically generated callouts). - The offset_vector field is a pointer to the vector of offsets that was - passed by the caller to pcre_exec() or pcre_dfa_exec(). When - pcre_exec() is used, the contents can be inspected in order to extract - substrings that have been matched so far, in the same way as for - extracting substrings after a match has completed. For pcre_dfa_exec() + The offset_vector field is a pointer to the vector of offsets that was + passed by the caller to pcre_exec() or pcre_dfa_exec(). When + pcre_exec() is used, the contents can be inspected in order to extract + substrings that have been matched so far, in the same way as for + extracting substrings after a match has completed. For pcre_dfa_exec() this field is not useful. The subject and subject_length fields contain copies of the values that were passed to pcre_exec(). - The start_match field normally contains the offset within the subject - at which the current match attempt started. However, if the escape - sequence \K has been encountered, this value is changed to reflect the - modified starting point. If the pattern is not anchored, the callout + The start_match field normally contains the offset within the subject + at which the current match attempt started. However, if the escape + sequence \K has been encountered, this value is changed to reflect the + modified starting point. If the pattern is not anchored, the callout function may be called several times from the same point in the pattern for different starting points in the subject. - The current_position field contains the offset within the subject of + The current_position field contains the offset within the subject of the current match pointer. - When the pcre_exec() function is used, the capture_top field contains - one more than the number of the highest numbered captured substring so - far. If no substrings have been captured, the value of capture_top is - one. This is always the case when pcre_dfa_exec() is used, because it + When the pcre_exec() function is used, the capture_top field contains + one more than the number of the highest numbered captured substring so + far. If no substrings have been captured, the value of capture_top is + one. This is always the case when pcre_dfa_exec() is used, because it does not support captured substrings. - The capture_last field contains the number of the most recently cap- - tured substring. If no substrings have been captured, its value is -1. + The capture_last field contains the number of the most recently cap- + tured substring. If no substrings have been captured, its value is -1. This is always the case when pcre_dfa_exec() is used. - The callout_data field contains a value that is passed to pcre_exec() - or pcre_dfa_exec() specifically so that it can be passed back in call- - outs. It is passed in the pcre_callout field of the pcre_extra data - structure. If no such data was passed, the value of callout_data in a - pcre_callout block is NULL. There is a description of the pcre_extra + The callout_data field contains a value that is passed to pcre_exec() + or pcre_dfa_exec() specifically so that it can be passed back in call- + outs. It is passed in the pcre_callout field of the pcre_extra data + structure. If no such data was passed, the value of callout_data in a + pcre_callout block is NULL. There is a description of the pcre_extra structure in the pcreapi documentation. - The pattern_position field is present from version 1 of the pcre_call- + The pattern_position field is present from version 1 of the pcre_call- out structure. It contains the offset to the next item to be matched in the pattern string. - The next_item_length field is present from version 1 of the pcre_call- + The next_item_length field is present from version 1 of the pcre_call- out structure. It contains the length of the next item to be matched in - the pattern string. When the callout immediately precedes an alterna- - tion bar, a closing parenthesis, or the end of the pattern, the length - is zero. When the callout precedes an opening parenthesis, the length + the pattern string. When the callout immediately precedes an alterna- + tion bar, a closing parenthesis, or the end of the pattern, the length + is zero. When the callout precedes an opening parenthesis, the length is that of the entire subpattern. - The pattern_position and next_item_length fields are intended to help - in distinguishing between different automatic callouts, which all have + The pattern_position and next_item_length fields are intended to help + in distinguishing between different automatic callouts, which all have the same callout number. However, they are set for all callouts. RETURN VALUES - The external callout function returns an integer to PCRE. If the value - is zero, matching proceeds as normal. If the value is greater than - zero, matching fails at the current point, but the testing of other + The external callout function returns an integer to PCRE. If the value + is zero, matching proceeds as normal. If the value is greater than + zero, matching fails at the current point, but the testing of other matching possibilities goes ahead, just as if a lookahead assertion had - failed. If the value is less than zero, the match is abandoned, and + failed. If the value is less than zero, the match is abandoned, and pcre_exec() or pcre_dfa_exec() returns the negative value. - Negative values should normally be chosen from the set of + Negative values should normally be chosen from the set of PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- - dard "no match" failure. The error number PCRE_ERROR_CALLOUT is - reserved for use by callout functions; it will never be used by PCRE + dard "no match" failure. The error number PCRE_ERROR_CALLOUT is + reserved for use by callout functions; it will never be used by PCRE itself. @@ -3043,11 +3056,11 @@ AUTHOR REVISION - Last updated: 29 September 2009 - Copyright (c) 1997-2009 University of Cambridge. + Last updated: 21 November 2010 + Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRECOMPAT(3) PCRECOMPAT(3) @@ -3202,8 +3215,8 @@ REVISION Last updated: 31 October 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREPATTERN(3) PCREPATTERN(3) @@ -3252,26 +3265,31 @@ PCRE REGULAR EXPRESSION DETAILS character types, instead of recognizing only characters with codes less than 128 via a lookup table. - The remainder of this document discusses the patterns that are sup- - ported by PCRE when its main matching function, pcre_exec(), is used. - From release 6.0, PCRE offers a second matching function, - pcre_dfa_exec(), which matches using a different algorithm that is not + If a pattern starts with (*NO_START_OPT), it has the same effect as + setting the PCRE_NO_START_OPTIMIZE option either at compile or matching + time. There are also some more of these special sequences that are con- + cerned with the handling of newlines; they are described below. + + The remainder of this document discusses the patterns that are sup- + ported by PCRE when its main matching function, pcre_exec(), is used. + From release 6.0, PCRE offers a second matching function, + pcre_dfa_exec(), which matches using a different algorithm that is not Perl-compatible. Some of the features discussed below are not available - when pcre_dfa_exec() is used. The advantages and disadvantages of the - alternative function, and how it differs from the normal function, are + when pcre_dfa_exec() is used. The advantages and disadvantages of the + alternative function, and how it differs from the normal function, are discussed in the pcrematching page. NEWLINE CONVENTIONS - PCRE supports five different conventions for indicating line breaks in - strings: a single CR (carriage return) character, a single LF (line- + PCRE supports five different conventions for indicating line breaks in + strings: a single CR (carriage return) character, a single LF (line- feed) character, the two-character sequence CRLF, any of the three pre- - ceding, or any Unicode newline sequence. The pcreapi page has further - discussion about newlines, and shows how to set the newline convention + ceding, or any Unicode newline sequence. The pcreapi page has further + discussion about newlines, and shows how to set the newline convention in the options arguments for the compiling and matching functions. - It is also possible to specify a newline convention by starting a pat- + It is also possible to specify a newline convention by starting a pat- tern string with one of the following five sequences: (*CR) carriage return @@ -3280,54 +3298,54 @@ NEWLINE CONVENTIONS (*ANYCRLF) any of the three above (*ANY) all Unicode newline sequences - These override the default and the options given to pcre_compile() or - pcre_compile2(). For example, on a Unix system where LF is the default + These override the default and the options given to pcre_compile() or + pcre_compile2(). For example, on a Unix system where LF is the default newline sequence, the pattern (*CR)a.b changes the convention to CR. That pattern matches "a\nb" because LF is - no longer a newline. Note that these special settings, which are not - Perl-compatible, are recognized only at the very start of a pattern, - and that they must be in upper case. If more than one of them is + no longer a newline. Note that these special settings, which are not + Perl-compatible, are recognized only at the very start of a pattern, + and that they must be in upper case. If more than one of them is present, the last one is used. - The newline convention affects the interpretation of the dot metachar- - acter when PCRE_DOTALL is not set, and also the behaviour of \N. How- - ever, it does not affect what the \R escape sequence matches. By - default, this is any Unicode newline sequence, for Perl compatibility. - However, this can be changed; see the description of \R in the section - entitled "Newline sequences" below. A change of \R setting can be com- + The newline convention affects the interpretation of the dot metachar- + acter when PCRE_DOTALL is not set, and also the behaviour of \N. How- + ever, it does not affect what the \R escape sequence matches. By + default, this is any Unicode newline sequence, for Perl compatibility. + However, this can be changed; see the description of \R in the section + entitled "Newline sequences" below. A change of \R setting can be com- bined with a change of newline convention. CHARACTERS AND METACHARACTERS - A regular expression is a pattern that is matched against a subject - string from left to right. Most characters stand for themselves in a - pattern, and match the corresponding characters in the subject. As a + A regular expression is a pattern that is matched against a subject + string from left to right. Most characters stand for themselves in a + pattern, and match the corresponding characters in the subject. As a trivial example, the pattern The quick brown fox matches a portion of a subject string that is identical to itself. When - caseless matching is specified (the PCRE_CASELESS option), letters are - matched independently of case. In UTF-8 mode, PCRE always understands - the concept of case for characters whose values are less than 128, so - caseless matching is always possible. For characters with higher val- - ues, the concept of case is supported if PCRE is compiled with Unicode - property support, but not otherwise. If you want to use caseless - matching for characters 128 and above, you must ensure that PCRE is + caseless matching is specified (the PCRE_CASELESS option), letters are + matched independently of case. In UTF-8 mode, PCRE always understands + the concept of case for characters whose values are less than 128, so + caseless matching is always possible. For characters with higher val- + ues, the concept of case is supported if PCRE is compiled with Unicode + property support, but not otherwise. If you want to use caseless + matching for characters 128 and above, you must ensure that PCRE is compiled with Unicode property support as well as with UTF-8 support. - The power of regular expressions comes from the ability to include - alternatives and repetitions in the pattern. These are encoded in the + The power of regular expressions comes from the ability to include + alternatives and repetitions in the pattern. These are encoded in the pattern by the use of metacharacters, which do not stand for themselves but instead are interpreted in some special way. - There are two different sets of metacharacters: those that are recog- - nized anywhere in the pattern except within square brackets, and those - that are recognized within square brackets. Outside square brackets, + There are two different sets of metacharacters: those that are recog- + nized anywhere in the pattern except within square brackets, and those + that are recognized within square brackets. Outside square brackets, the metacharacters are as follows: \ general escape character with several uses @@ -3346,7 +3364,7 @@ CHARACTERS AND METACHARACTERS also "possessive quantifier" { start min/max quantifier - Part of a pattern that is in square brackets is called a "character + Part of a pattern that is in square brackets is called a "character class". In a character class the only metacharacters are: \ general escape character @@ -3362,27 +3380,31 @@ CHARACTERS AND METACHARACTERS BACKSLASH The backslash character has several uses. Firstly, if it is followed by - a non-alphanumeric character, it takes away any special meaning that - character may have. This use of backslash as an escape character - applies both inside and outside character classes. - - For example, if you want to match a * character, you write \* in the - pattern. This escaping action applies whether or not the following - character would otherwise be interpreted as a metacharacter, so it is - always safe to precede a non-alphanumeric with backslash to specify - that it stands for itself. In particular, if you want to match a back- + a character that is not a number or a letter, it takes away any special + meaning that character may have. This use of backslash as an escape + character applies both inside and outside character classes. + + For example, if you want to match a * character, you write \* in the + pattern. This escaping action applies whether or not the following + character would otherwise be interpreted as a metacharacter, so it is + always safe to precede a non-alphanumeric with backslash to specify + that it stands for itself. In particular, if you want to match a back- slash, you write \\. - If a pattern is compiled with the PCRE_EXTENDED option, whitespace in - the pattern (other than in a character class) and characters between a + In UTF-8 mode, only ASCII numbers and letters have any special meaning + after a backslash. All other characters (in particular, those whose + codepoints are greater than 127) are treated as literals. + + If a pattern is compiled with the PCRE_EXTENDED option, whitespace in + the pattern (other than in a character class) and characters between a # outside a character class and the next newline are ignored. An escap- - ing backslash can be used to include a whitespace or # character as + ing backslash can be used to include a whitespace or # character as part of the pattern. - If you want to remove the special meaning from a sequence of charac- - ters, you can do so by putting them between \Q and \E. This is differ- - ent from Perl in that $ and @ are handled as literals in \Q...\E - sequences in PCRE, whereas in Perl, $ and @ cause variable interpola- + If you want to remove the special meaning from a sequence of charac- + ters, you can do so by putting them between \Q and \E. This is differ- + ent from Perl in that $ and @ are handled as literals in \Q...\E + sequences in PCRE, whereas in Perl, $ and @ cause variable interpola- tion. Note the following examples: Pattern PCRE matches Perl matches @@ -3392,20 +3414,20 @@ BACKSLASH \Qabc\$xyz\E abc\$xyz abc\$xyz \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - The \Q...\E sequence is recognized both inside and outside character + The \Q...\E sequence is recognized both inside and outside character classes. An isolated \E that is not preceded by \Q is ignored. Non-printing characters A second use of backslash provides a way of encoding non-printing char- - acters in patterns in a visible manner. There is no restriction on the - appearance of non-printing characters, apart from the binary zero that - terminates a pattern, but when a pattern is being prepared by text - editing, it is often easier to use one of the following escape + acters in patterns in a visible manner. There is no restriction on the + appearance of non-printing characters, apart from the binary zero that + terminates a pattern, but when a pattern is being prepared by text + editing, it is often easier to use one of the following escape sequences than the binary character it represents: \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any character + \cx "control-x", where x is any ASCII character \e escape (hex 1B) \f formfeed (hex 0C) \n linefeed (hex 0A) @@ -3415,48 +3437,52 @@ BACKSLASH \xhh character with hex code hh \x{hhh..} character with hex code hhh.. - The precise effect of \cx is as follows: if x is a lower case letter, - it is converted to upper case. Then bit 6 of the character (hex 40) is - inverted. Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; - becomes hex 7B. - - After \x, from zero to two hexadecimal digits are read (letters can be - in upper or lower case). Any number of hexadecimal digits may appear - between \x{ and }, but the value of the character code must be less + The precise effect of \cx is as follows: if x is a lower case letter, + it is converted to upper case. Then bit 6 of the character (hex 40) is + inverted. Thus \cz becomes hex 1A (z is 7A), but \c{ becomes hex 3B ({ + is 7B), while \c; becomes hex 7B (; is 3B). If the byte following \c + has a value greater than 127, a compile-time error occurs. This locks + out non-ASCII characters in both byte mode and UTF-8 mode. (When PCRE + is compiled in EBCDIC mode, all byte values are valid. A lower case + letter is converted to upper case, and then the 0xc0 bits are flipped.) + + After \x, from zero to two hexadecimal digits are read (letters can be + in upper or lower case). Any number of hexadecimal digits may appear + between \x{ and }, but the value of the character code must be less than 256 in non-UTF-8 mode, and less than 2**31 in UTF-8 mode. That is, - the maximum value in hexadecimal is 7FFFFFFF. Note that this is bigger + the maximum value in hexadecimal is 7FFFFFFF. Note that this is bigger than the largest Unicode code point, which is 10FFFF. - If characters other than hexadecimal digits appear between \x{ and }, + If characters other than hexadecimal digits appear between \x{ and }, or if there is no terminating }, this form of escape is not recognized. - Instead, the initial \x will be interpreted as a basic hexadecimal - escape, with no following digits, giving a character whose value is + Instead, the initial \x will be interpreted as a basic hexadecimal + escape, with no following digits, giving a character whose value is zero. Characters whose value is less than 256 can be defined by either of the - two syntaxes for \x. There is no difference in the way they are han- + two syntaxes for \x. There is no difference in the way they are han- dled. For example, \xdc is exactly the same as \x{dc}. - After \0 up to two further octal digits are read. If there are fewer - than two digits, just those that are present are used. Thus the + After \0 up to two further octal digits are read. If there are fewer + than two digits, just those that are present are used. Thus the sequence \0\x\07 specifies two binary zeros followed by a BEL character - (code value 7). Make sure you supply two digits after the initial zero + (code value 7). Make sure you supply two digits after the initial zero if the pattern character that follows is itself an octal digit. The handling of a backslash followed by a digit other than 0 is compli- cated. Outside a character class, PCRE reads it and any following dig- - its as a decimal number. If the number is less than 10, or if there + its as a decimal number. If the number is less than 10, or if there have been at least that many previous capturing left parentheses in the - expression, the entire sequence is taken as a back reference. A - description of how this works is given later, following the discussion + expression, the entire sequence is taken as a back reference. A + description of how this works is given later, following the discussion of parenthesized subpatterns. - Inside a character class, or if the decimal number is greater than 9 - and there have not been that many capturing subpatterns, PCRE re-reads + Inside a character class, or if the decimal number is greater than 9 + and there have not been that many capturing subpatterns, PCRE re-reads up to three octal digits following the backslash, and uses them to gen- - erate a data character. Any subsequent digits stand for themselves. In - non-UTF-8 mode, the value of a character specified in octal must be - less than \400. In UTF-8 mode, values up to \777 are permitted. For + erate a data character. Any subsequent digits stand for themselves. In + non-UTF-8 mode, the value of a character specified in octal must be + less than \400. In UTF-8 mode, values up to \777 are permitted. For example: \040 is another way of writing a space @@ -3474,32 +3500,32 @@ BACKSLASH \81 is either a back reference, or a binary zero followed by the two characters "8" and "1" - Note that octal values of 100 or greater must not be introduced by a + Note that octal values of 100 or greater must not be introduced by a leading zero, because no more than three octal digits are ever read. All the sequences that define a single character value can be used both - inside and outside character classes. In addition, inside a character - class, the sequence \b is interpreted as the backspace character (hex - 08). The sequences \B, \N, \R, and \X are not special inside a charac- - ter class. Like any other unrecognized escape sequences, they are - treated as the literal characters "B", "N", "R", and "X" by default, + inside and outside character classes. In addition, inside a character + class, the sequence \b is interpreted as the backspace character (hex + 08). The sequences \B, \N, \R, and \X are not special inside a charac- + ter class. Like any other unrecognized escape sequences, they are + treated as the literal characters "B", "N", "R", and "X" by default, but cause an error if the PCRE_EXTRA option is set. Outside a character class, these sequences have different meanings. Absolute and relative back references - The sequence \g followed by an unsigned or a negative number, option- - ally enclosed in braces, is an absolute or relative back reference. A + The sequence \g followed by an unsigned or a negative number, option- + ally enclosed in braces, is an absolute or relative back reference. A named back reference can be coded as \g{name}. Back references are dis- cussed later, following the discussion of parenthesized subpatterns. Absolute and relative subroutine calls - For compatibility with Oniguruma, the non-Perl syntax \g followed by a + For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or a number enclosed either in angle brackets or single quotes, is - an alternative syntax for referencing a subpattern as a "subroutine". - Details are discussed later. Note that \g{...} (Perl syntax) and - \g<...> (Oniguruma syntax) are not synonymous. The former is a back + an alternative syntax for referencing a subpattern as a "subroutine". + Details are discussed later. Note that \g{...} (Perl syntax) and + \g<...> (Oniguruma syntax) are not synonymous. The former is a back reference; the latter is a subroutine call. Generic character types @@ -3518,54 +3544,54 @@ BACKSLASH \W any "non-word" character There is also the single sequence \N, which matches a non-newline char- - acter. This is the same as the "." metacharacter when PCRE_DOTALL is + acter. This is the same as the "." metacharacter when PCRE_DOTALL is not set. - Each pair of lower and upper case escape sequences partitions the com- - plete set of characters into two disjoint sets. Any given character - matches one, and only one, of each pair. The sequences can appear both - inside and outside character classes. They each match one character of - the appropriate type. If the current matching point is at the end of - the subject string, all of them fail, because there is no character to + Each pair of lower and upper case escape sequences partitions the com- + plete set of characters into two disjoint sets. Any given character + matches one, and only one, of each pair. The sequences can appear both + inside and outside character classes. They each match one character of + the appropriate type. If the current matching point is at the end of + the subject string, all of them fail, because there is no character to match. - For compatibility with Perl, \s does not match the VT character (code - 11). This makes it different from the the POSIX "space" class. The \s - characters are HT (9), LF (10), FF (12), CR (13), and space (32). If + For compatibility with Perl, \s does not match the VT character (code + 11). This makes it different from the the POSIX "space" class. The \s + characters are HT (9), LF (10), FF (12), CR (13), and space (32). If "use locale;" is included in a Perl script, \s may match the VT charac- ter. In PCRE, it never does. - A "word" character is an underscore or any character that is a letter - or digit. By default, the definition of letters and digits is con- - trolled by PCRE's low-valued character tables, and may vary if locale- - specific matching is taking place (see "Locale support" in the pcreapi - page). For example, in a French locale such as "fr_FR" in Unix-like - systems, or "french" in Windows, some character codes greater than 128 - are used for accented letters, and these are then matched by \w. The + A "word" character is an underscore or any character that is a letter + or digit. By default, the definition of letters and digits is con- + trolled by PCRE's low-valued character tables, and may vary if locale- + specific matching is taking place (see "Locale support" in the pcreapi + page). For example, in a French locale such as "fr_FR" in Unix-like + systems, or "french" in Windows, some character codes greater than 128 + are used for accented letters, and these are then matched by \w. The use of locales with Unicode is discouraged. - By default, in UTF-8 mode, characters with values greater than 128 - never match \d, \s, or \w, and always match \D, \S, and \W. These - sequences retain their original meanings from before UTF-8 support was - available, mainly for efficiency reasons. However, if PCRE is compiled - with Unicode property support, and the PCRE_UCP option is set, the be- - haviour is changed so that Unicode properties are used to determine + By default, in UTF-8 mode, characters with values greater than 128 + never match \d, \s, or \w, and always match \D, \S, and \W. These + sequences retain their original meanings from before UTF-8 support was + available, mainly for efficiency reasons. However, if PCRE is compiled + with Unicode property support, and the PCRE_UCP option is set, the be- + haviour is changed so that Unicode properties are used to determine character types, as follows: \d any character that \p{Nd} matches (decimal digit) \s any character that \p{Z} matches, plus HT, LF, FF, CR \w any character that \p{L} or \p{N} matches, plus underscore - The upper case escapes match the inverse sets of characters. Note that - \d matches only decimal digits, whereas \w matches any Unicode digit, - as well as any Unicode letter, and underscore. Note also that PCRE_UCP - affects \b, and \B because they are defined in terms of \w and \W. + The upper case escapes match the inverse sets of characters. Note that + \d matches only decimal digits, whereas \w matches any Unicode digit, + as well as any Unicode letter, and underscore. Note also that PCRE_UCP + affects \b, and \B because they are defined in terms of \w and \W. Matching these sequences is noticeably slower when PCRE_UCP is set. - The sequences \h, \H, \v, and \V are features that were added to Perl - at release 5.10. In contrast to the other sequences, which match only - ASCII characters by default, these always match certain high-valued - codepoints in UTF-8 mode, whether or not PCRE_UCP is set. The horizon- + The sequences \h, \H, \v, and \V are features that were added to Perl + at release 5.10. In contrast to the other sequences, which match only + ASCII characters by default, these always match certain high-valued + codepoints in UTF-8 mode, whether or not PCRE_UCP is set. The horizon- tal space characters are: U+0009 Horizontal tab @@ -3600,104 +3626,104 @@ BACKSLASH Newline sequences - Outside a character class, by default, the escape sequence \R matches + Outside a character class, by default, the escape sequence \R matches any Unicode newline sequence. In non-UTF-8 mode \R is equivalent to the following: (?>\r\n|\n|\x0b|\f|\r|\x85) - This is an example of an "atomic group", details of which are given + This is an example of an "atomic group", details of which are given below. This particular group matches either the two-character sequence - CR followed by LF, or one of the single characters LF (linefeed, + CR followed by LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, U+000B), FF (formfeed, U+000C), CR (carriage return, U+000D), or NEL (next line, U+0085). The two-character sequence is treated as a single unit that cannot be split. - In UTF-8 mode, two additional characters whose codepoints are greater + In UTF-8 mode, two additional characters whose codepoints are greater than 255 are added: LS (line separator, U+2028) and PS (paragraph sepa- - rator, U+2029). Unicode character property support is not needed for + rator, U+2029). Unicode character property support is not needed for these characters to be recognized. It is possible to restrict \R to match only CR, LF, or CRLF (instead of - the complete set of Unicode line endings) by setting the option + the complete set of Unicode line endings) by setting the option PCRE_BSR_ANYCRLF either at compile time or when the pattern is matched. (BSR is an abbrevation for "backslash R".) This can be made the default - when PCRE is built; if this is the case, the other behaviour can be - requested via the PCRE_BSR_UNICODE option. It is also possible to - specify these settings by starting a pattern string with one of the + when PCRE is built; if this is the case, the other behaviour can be + requested via the PCRE_BSR_UNICODE option. It is also possible to + specify these settings by starting a pattern string with one of the following sequences: (*BSR_ANYCRLF) CR, LF, or CRLF only (*BSR_UNICODE) any Unicode newline sequence - These override the default and the options given to pcre_compile() or - pcre_compile2(), but they can be overridden by options given to + These override the default and the options given to pcre_compile() or + pcre_compile2(), but they can be overridden by options given to pcre_exec() or pcre_dfa_exec(). Note that these special settings, which - are not Perl-compatible, are recognized only at the very start of a - pattern, and that they must be in upper case. If more than one of them + are not Perl-compatible, are recognized only at the very start of a + pattern, and that they must be in upper case. If more than one of them is present, the last one is used. They can be combined with a change of newline convention; for example, a pattern can start with: (*ANY)(*BSR_ANYCRLF) They can also be combined with the (*UTF8) or (*UCP) special sequences. - Inside a character class, \R is treated as an unrecognized escape + Inside a character class, \R is treated as an unrecognized escape sequence, and so matches the letter "R" by default, but causes an error if PCRE_EXTRA is set. Unicode character properties When PCRE is built with Unicode character property support, three addi- - tional escape sequences that match characters with specific properties - are available. When not in UTF-8 mode, these sequences are of course - limited to testing characters whose codepoints are less than 256, but + tional escape sequences that match characters with specific properties + are available. When not in UTF-8 mode, these sequences are of course + limited to testing characters whose codepoints are less than 256, but they do work in this mode. The extra escape sequences are: \p{xx} a character with the xx property \P{xx} a character without the xx property \X an extended Unicode sequence - The property names represented by xx above are limited to the Unicode + The property names represented by xx above are limited to the Unicode script names, the general category properties, "Any", which matches any - character (including newline), and some special PCRE properties - (described in the next section). Other Perl properties such as "InMu- - sicalSymbols" are not currently supported by PCRE. Note that \P{Any} + character (including newline), and some special PCRE properties + (described in the next section). Other Perl properties such as "InMu- + sicalSymbols" are not currently supported by PCRE. Note that \P{Any} does not match any characters, so always causes a match failure. Sets of Unicode characters are defined as belonging to certain scripts. - A character from one of these sets can be matched using a script name. + A character from one of these sets can be matched using a script name. For example: \p{Greek} \P{Han} - Those that are not part of an identified script are lumped together as + Those that are not part of an identified script are lumped together as "Common". The current list of scripts is: Arabic, Armenian, Avestan, Balinese, Bamum, Bengali, Bopomofo, Braille, - Buginese, Buhid, Canadian_Aboriginal, Carian, Cham, Cherokee, Common, - Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, Devanagari, Egyp- - tian_Hieroglyphs, Ethiopic, Georgian, Glagolitic, Gothic, Greek, - Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, Impe- + Buginese, Buhid, Canadian_Aboriginal, Carian, Cham, Cherokee, Common, + Coptic, Cuneiform, Cypriot, Cyrillic, Deseret, Devanagari, Egyp- + tian_Hieroglyphs, Ethiopic, Georgian, Glagolitic, Gothic, Greek, + Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, Impe- rial_Aramaic, Inherited, Inscriptional_Pahlavi, Inscriptional_Parthian, - Javanese, Kaithi, Kannada, Katakana, Kayah_Li, Kharoshthi, Khmer, Lao, + Javanese, Kaithi, Kannada, Katakana, Kayah_Li, Kharoshthi, Khmer, Lao, Latin, Lepcha, Limbu, Linear_B, Lisu, Lycian, Lydian, Malayalam, - Meetei_Mayek, Mongolian, Myanmar, New_Tai_Lue, Nko, Ogham, Old_Italic, - Old_Persian, Old_South_Arabian, Old_Turkic, Ol_Chiki, Oriya, Osmanya, - Phags_Pa, Phoenician, Rejang, Runic, Samaritan, Saurashtra, Shavian, - Sinhala, Sundanese, Syloti_Nagri, Syriac, Tagalog, Tagbanwa, Tai_Le, - Tai_Tham, Tai_Viet, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, + Meetei_Mayek, Mongolian, Myanmar, New_Tai_Lue, Nko, Ogham, Old_Italic, + Old_Persian, Old_South_Arabian, Old_Turkic, Ol_Chiki, Oriya, Osmanya, + Phags_Pa, Phoenician, Rejang, Runic, Samaritan, Saurashtra, Shavian, + Sinhala, Sundanese, Syloti_Nagri, Syriac, Tagalog, Tagbanwa, Tai_Le, + Tai_Tham, Tai_Viet, Tamil, Telugu, Thaana, Thai, Tibetan, Tifinagh, Ugaritic, Vai, Yi. Each character has exactly one Unicode general category property, spec- - ified by a two-letter abbreviation. For compatibility with Perl, nega- - tion can be specified by including a circumflex between the opening - brace and the property name. For example, \p{^Lu} is the same as + ified by a two-letter abbreviation. For compatibility with Perl, nega- + tion can be specified by including a circumflex between the opening + brace and the property name. For example, \p{^Lu} is the same as \P{Lu}. If only one letter is specified with \p or \P, it includes all the gen- - eral category properties that start with that letter. In this case, in - the absence of negation, the curly brackets in the escape sequence are + eral category properties that start with that letter. In this case, in + the absence of negation, the curly brackets in the escape sequence are optional; these two examples have the same effect: \p{L} @@ -3749,50 +3775,50 @@ BACKSLASH Zp Paragraph separator Zs Space separator - The special property L& is also supported: it matches a character that - has the Lu, Ll, or Lt property, in other words, a letter that is not + The special property L& is also supported: it matches a character that + has the Lu, Ll, or Lt property, in other words, a letter that is not classified as a modifier or "other". - The Cs (Surrogate) property applies only to characters in the range - U+D800 to U+DFFF. Such characters are not valid in UTF-8 strings (see + The Cs (Surrogate) property applies only to characters in the range + U+D800 to U+DFFF. Such characters are not valid in UTF-8 strings (see RFC 3629) and so cannot be tested by PCRE, unless UTF-8 validity check- - ing has been turned off (see the discussion of PCRE_NO_UTF8_CHECK in + ing has been turned off (see the discussion of PCRE_NO_UTF8_CHECK in the pcreapi page). Perl does not support the Cs property. - The long synonyms for property names that Perl supports (such as - \p{Letter}) are not supported by PCRE, nor is it permitted to prefix + The long synonyms for property names that Perl supports (such as + \p{Letter}) are not supported by PCRE, nor is it permitted to prefix any of these properties with "Is". No character that is in the Unicode table has the Cn (unassigned) prop- erty. Instead, this property is assumed for any code point that is not in the Unicode table. - Specifying caseless matching does not affect these escape sequences. + Specifying caseless matching does not affect these escape sequences. For example, \p{Lu} always matches only upper case letters. - The \X escape matches any number of Unicode characters that form an + The \X escape matches any number of Unicode characters that form an extended Unicode sequence. \X is equivalent to (?>\PM\pM*) - That is, it matches a character without the "mark" property, followed - by zero or more characters with the "mark" property, and treats the - sequence as an atomic group (see below). Characters with the "mark" - property are typically accents that affect the preceding character. - None of them have codepoints less than 256, so in non-UTF-8 mode \X + That is, it matches a character without the "mark" property, followed + by zero or more characters with the "mark" property, and treats the + sequence as an atomic group (see below). Characters with the "mark" + property are typically accents that affect the preceding character. + None of them have codepoints less than 256, so in non-UTF-8 mode \X matches any one character. - Matching characters by Unicode property is not fast, because PCRE has - to search a structure that contains data for over fifteen thousand + Matching characters by Unicode property is not fast, because PCRE has + to search a structure that contains data for over fifteen thousand characters. That is why the traditional escape sequences such as \d and - \w do not use Unicode properties in PCRE by default, though you can + \w do not use Unicode properties in PCRE by default, though you can make them do so by setting the PCRE_UCP option for pcre_compile() or by starting the pattern with (*UCP). PCRE's additional properties - As well as the standard Unicode properties described in the previous - section, PCRE supports four more that make it possible to convert tra- + As well as the standard Unicode properties described in the previous + section, PCRE supports four more that make it possible to convert tra- ditional escape sequences such as \w and \s and POSIX character classes to use Unicode properties. PCRE uses these non-standard, non-Perl prop- erties internally when PCRE_UCP is set. They are: @@ -3802,40 +3828,40 @@ BACKSLASH Xsp Any Perl space character Xwd Any Perl "word" character - Xan matches characters that have either the L (letter) or the N (num- - ber) property. Xps matches the characters tab, linefeed, vertical tab, - formfeed, or carriage return, and any other character that has the Z + Xan matches characters that have either the L (letter) or the N (num- + ber) property. Xps matches the characters tab, linefeed, vertical tab, + formfeed, or carriage return, and any other character that has the Z (separator) property. Xsp is the same as Xps, except that vertical tab is excluded. Xwd matches the same characters as Xan, plus underscore. Resetting the match start - The escape sequence \K causes any previously matched characters not to + The escape sequence \K causes any previously matched characters not to be included in the final matched sequence. For example, the pattern: foo\Kbar - matches "foobar", but reports that it has matched "bar". This feature - is similar to a lookbehind assertion (described below). However, in - this case, the part of the subject before the real match does not have - to be of fixed length, as lookbehind assertions do. The use of \K does - not interfere with the setting of captured substrings. For example, + matches "foobar", but reports that it has matched "bar". This feature + is similar to a lookbehind assertion (described below). However, in + this case, the part of the subject before the real match does not have + to be of fixed length, as lookbehind assertions do. The use of \K does + not interfere with the setting of captured substrings. For example, when the pattern (foo)\Kbar matches "foobar", the first substring is still set to "foo". - Perl documents that the use of \K within assertions is "not well - defined". In PCRE, \K is acted upon when it occurs inside positive + Perl documents that the use of \K within assertions is "not well + defined". In PCRE, \K is acted upon when it occurs inside positive assertions, but is ignored in negative assertions. Simple assertions - The final use of backslash is for certain simple assertions. An asser- - tion specifies a condition that has to be met at a particular point in - a match, without consuming any characters from the subject string. The - use of subpatterns for more complicated assertions is described below. + The final use of backslash is for certain simple assertions. An asser- + tion specifies a condition that has to be met at a particular point in + a match, without consuming any characters from the subject string. The + use of subpatterns for more complicated assertions is described below. The backslashed assertions are: \b matches at a word boundary @@ -3846,49 +3872,49 @@ BACKSLASH \z matches only at the end of the subject \G matches at the first matching position in the subject - Inside a character class, \b has a different meaning; it matches the - backspace character. If any other of these assertions appears in a - character class, by default it matches the corresponding literal char- + Inside a character class, \b has a different meaning; it matches the + backspace character. If any other of these assertions appears in a + character class, by default it matches the corresponding literal char- acter (for example, \B matches the letter B). However, if the - PCRE_EXTRA option is set, an "invalid escape sequence" error is gener- + PCRE_EXTRA option is set, an "invalid escape sequence" error is gener- ated instead. - A word boundary is a position in the subject string where the current - character and the previous character do not both match \w or \W (i.e. - one matches \w and the other matches \W), or the start or end of the - string if the first or last character matches \w, respectively. In - UTF-8 mode, the meanings of \w and \W can be changed by setting the - PCRE_UCP option. When this is done, it also affects \b and \B. Neither - PCRE nor Perl has a separate "start of word" or "end of word" metase- - quence. However, whatever follows \b normally determines which it is. + A word boundary is a position in the subject string where the current + character and the previous character do not both match \w or \W (i.e. + one matches \w and the other matches \W), or the start or end of the + string if the first or last character matches \w, respectively. In + UTF-8 mode, the meanings of \w and \W can be changed by setting the + PCRE_UCP option. When this is done, it also affects \b and \B. Neither + PCRE nor Perl has a separate "start of word" or "end of word" metase- + quence. However, whatever follows \b normally determines which it is. For example, the fragment \ba matches "a" at the start of a word. - The \A, \Z, and \z assertions differ from the traditional circumflex + The \A, \Z, and \z assertions differ from the traditional circumflex and dollar (described in the next section) in that they only ever match - at the very start and end of the subject string, whatever options are - set. Thus, they are independent of multiline mode. These three asser- + at the very start and end of the subject string, whatever options are + set. Thus, they are independent of multiline mode. These three asser- tions are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options, which - affect only the behaviour of the circumflex and dollar metacharacters. - However, if the startoffset argument of pcre_exec() is non-zero, indi- + affect only the behaviour of the circumflex and dollar metacharacters. + However, if the startoffset argument of pcre_exec() is non-zero, indi- cating that matching is to start at a point other than the beginning of - the subject, \A can never match. The difference between \Z and \z is + the subject, \A can never match. The difference between \Z and \z is that \Z matches before a newline at the end of the string as well as at the very end, whereas \z matches only at the end. - The \G assertion is true only when the current matching position is at - the start point of the match, as specified by the startoffset argument - of pcre_exec(). It differs from \A when the value of startoffset is - non-zero. By calling pcre_exec() multiple times with appropriate argu- + The \G assertion is true only when the current matching position is at + the start point of the match, as specified by the startoffset argument + of pcre_exec(). It differs from \A when the value of startoffset is + non-zero. By calling pcre_exec() multiple times with appropriate argu- ments, you can mimic Perl's /g option, and it is in this kind of imple- mentation where \G can be useful. - Note, however, that PCRE's interpretation of \G, as the start of the + Note, however, that PCRE's interpretation of \G, as the start of the current match, is subtly different from Perl's, which defines it as the - end of the previous match. In Perl, these can be different when the - previously matched string was empty. Because PCRE does just one match + end of the previous match. In Perl, these can be different when the + previously matched string was empty. Because PCRE does just one match at a time, it cannot reproduce this behaviour. - If all the alternatives of a pattern begin with \G, the expression is + If all the alternatives of a pattern begin with \G, the expression is anchored to the starting match position, and the "anchored" flag is set in the compiled regular expression. @@ -3896,94 +3922,94 @@ BACKSLASH CIRCUMFLEX AND DOLLAR Outside a character class, in the default matching mode, the circumflex - character is an assertion that is true only if the current matching - point is at the start of the subject string. If the startoffset argu- - ment of pcre_exec() is non-zero, circumflex can never match if the - PCRE_MULTILINE option is unset. Inside a character class, circumflex + character is an assertion that is true only if the current matching + point is at the start of the subject string. If the startoffset argu- + ment of pcre_exec() is non-zero, circumflex can never match if the + PCRE_MULTILINE option is unset. Inside a character class, circumflex has an entirely different meaning (see below). - Circumflex need not be the first character of the pattern if a number - of alternatives are involved, but it should be the first thing in each - alternative in which it appears if the pattern is ever to match that - branch. If all possible alternatives start with a circumflex, that is, - if the pattern is constrained to match only at the start of the sub- - ject, it is said to be an "anchored" pattern. (There are also other + Circumflex need not be the first character of the pattern if a number + of alternatives are involved, but it should be the first thing in each + alternative in which it appears if the pattern is ever to match that + branch. If all possible alternatives start with a circumflex, that is, + if the pattern is constrained to match only at the start of the sub- + ject, it is said to be an "anchored" pattern. (There are also other constructs that can cause a pattern to be anchored.) - A dollar character is an assertion that is true only if the current - matching point is at the end of the subject string, or immediately + A dollar character is an assertion that is true only if the current + matching point is at the end of the subject string, or immediately before a newline at the end of the string (by default). Dollar need not - be the last character of the pattern if a number of alternatives are - involved, but it should be the last item in any branch in which it + be the last character of the pattern if a number of alternatives are + involved, but it should be the last item in any branch in which it appears. Dollar has no special meaning in a character class. - The meaning of dollar can be changed so that it matches only at the - very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at + The meaning of dollar can be changed so that it matches only at the + very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This does not affect the \Z assertion. The meanings of the circumflex and dollar characters are changed if the - PCRE_MULTILINE option is set. When this is the case, a circumflex - matches immediately after internal newlines as well as at the start of - the subject string. It does not match after a newline that ends the - string. A dollar matches before any newlines in the string, as well as - at the very end, when PCRE_MULTILINE is set. When newline is specified - as the two-character sequence CRLF, isolated CR and LF characters do + PCRE_MULTILINE option is set. When this is the case, a circumflex + matches immediately after internal newlines as well as at the start of + the subject string. It does not match after a newline that ends the + string. A dollar matches before any newlines in the string, as well as + at the very end, when PCRE_MULTILINE is set. When newline is specified + as the two-character sequence CRLF, isolated CR and LF characters do not indicate newlines. - For example, the pattern /^abc$/ matches the subject string "def\nabc" - (where \n represents a newline) in multiline mode, but not otherwise. - Consequently, patterns that are anchored in single line mode because - all branches start with ^ are not anchored in multiline mode, and a - match for circumflex is possible when the startoffset argument of - pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if + For example, the pattern /^abc$/ matches the subject string "def\nabc" + (where \n represents a newline) in multiline mode, but not otherwise. + Consequently, patterns that are anchored in single line mode because + all branches start with ^ are not anchored in multiline mode, and a + match for circumflex is possible when the startoffset argument of + pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. - Note that the sequences \A, \Z, and \z can be used to match the start - and end of the subject in both modes, and if all branches of a pattern - start with \A it is always anchored, whether or not PCRE_MULTILINE is + Note that the sequences \A, \Z, and \z can be used to match the start + and end of the subject in both modes, and if all branches of a pattern + start with \A it is always anchored, whether or not PCRE_MULTILINE is set. FULL STOP (PERIOD, DOT) AND \N Outside a character class, a dot in the pattern matches any one charac- - ter in the subject string except (by default) a character that signi- - fies the end of a line. In UTF-8 mode, the matched character may be + ter in the subject string except (by default) a character that signi- + fies the end of a line. In UTF-8 mode, the matched character may be more than one byte long. - When a line ending is defined as a single character, dot never matches - that character; when the two-character sequence CRLF is used, dot does - not match CR if it is immediately followed by LF, but otherwise it - matches all characters (including isolated CRs and LFs). When any Uni- - code line endings are being recognized, dot does not match CR or LF or + When a line ending is defined as a single character, dot never matches + that character; when the two-character sequence CRLF is used, dot does + not match CR if it is immediately followed by LF, but otherwise it + matches all characters (including isolated CRs and LFs). When any Uni- + code line endings are being recognized, dot does not match CR or LF or any of the other line ending characters. - The behaviour of dot with regard to newlines can be changed. If the - PCRE_DOTALL option is set, a dot matches any one character, without + The behaviour of dot with regard to newlines can be changed. If the + PCRE_DOTALL option is set, a dot matches any one character, without exception. If the two-character sequence CRLF is present in the subject string, it takes two dots to match it. - The handling of dot is entirely independent of the handling of circum- - flex and dollar, the only relationship being that they both involve + The handling of dot is entirely independent of the handling of circum- + flex and dollar, the only relationship being that they both involve newlines. Dot has no special meaning in a character class. - The escape sequence \N behaves like a dot, except that it is not - affected by the PCRE_DOTALL option. In other words, it matches any + The escape sequence \N behaves like a dot, except that it is not + affected by the PCRE_DOTALL option. In other words, it matches any character except one that signifies the end of a line. MATCHING A SINGLE BYTE Outside a character class, the escape sequence \C matches any one byte, - both in and out of UTF-8 mode. Unlike a dot, it always matches any - line-ending characters. The feature is provided in Perl in order to - match individual bytes in UTF-8 mode. Because it breaks up UTF-8 char- - acters into individual bytes, the rest of the string may start with a - malformed UTF-8 character. For this reason, the \C escape sequence is + both in and out of UTF-8 mode. Unlike a dot, it always matches any + line-ending characters. The feature is provided in Perl in order to + match individual bytes in UTF-8 mode. Because it breaks up UTF-8 char- + acters into individual bytes, the rest of the string may start with a + malformed UTF-8 character. For this reason, the \C escape sequence is best avoided. - PCRE does not allow \C to appear in lookbehind assertions (described - below), because in UTF-8 mode this would make it impossible to calcu- + PCRE does not allow \C to appear in lookbehind assertions (described + below), because in UTF-8 mode this would make it impossible to calcu- late the length of the lookbehind. @@ -3993,97 +4019,109 @@ SQUARE BRACKETS AND CHARACTER CLASSES closing square bracket. A closing square bracket on its own is not spe- cial by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, a lone closing square bracket causes a compile-time error. If a closing - square bracket is required as a member of the class, it should be the - first data character in the class (after an initial circumflex, if + square bracket is required as a member of the class, it should be the + first data character in the class (after an initial circumflex, if present) or escaped with a backslash. - A character class matches a single character in the subject. In UTF-8 + A character class matches a single character in the subject. In UTF-8 mode, the character may be more than one byte long. A matched character must be in the set of characters defined by the class, unless the first - character in the class definition is a circumflex, in which case the - subject character must not be in the set defined by the class. If a - circumflex is actually required as a member of the class, ensure it is + character in the class definition is a circumflex, in which case the + subject character must not be in the set defined by the class. If a + circumflex is actually required as a member of the class, ensure it is not the first character, or escape it with a backslash. - For example, the character class [aeiou] matches any lower case vowel, - while [^aeiou] matches any character that is not a lower case vowel. + For example, the character class [aeiou] matches any lower case vowel, + while [^aeiou] matches any character that is not a lower case vowel. Note that a circumflex is just a convenient notation for specifying the - characters that are in the class by enumerating those that are not. A - class that starts with a circumflex is not an assertion; it still con- - sumes a character from the subject string, and therefore it fails if + characters that are in the class by enumerating those that are not. A + class that starts with a circumflex is not an assertion; it still con- + sumes a character from the subject string, and therefore it fails if the current pointer is at the end of the string. - In UTF-8 mode, characters with values greater than 255 can be included - in a class as a literal string of bytes, or by using the \x{ escaping + In UTF-8 mode, characters with values greater than 255 can be included + in a class as a literal string of bytes, or by using the \x{ escaping mechanism. - When caseless matching is set, any letters in a class represent both - their upper case and lower case versions, so for example, a caseless - [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not - match "A", whereas a caseful version would. In UTF-8 mode, PCRE always - understands the concept of case for characters whose values are less - than 128, so caseless matching is always possible. For characters with - higher values, the concept of case is supported if PCRE is compiled - with Unicode property support, but not otherwise. If you want to use - caseless matching in UTF8-mode for characters 128 and above, you must - ensure that PCRE is compiled with Unicode property support as well as + When caseless matching is set, any letters in a class represent both + their upper case and lower case versions, so for example, a caseless + [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not + match "A", whereas a caseful version would. In UTF-8 mode, PCRE always + understands the concept of case for characters whose values are less + than 128, so caseless matching is always possible. For characters with + higher values, the concept of case is supported if PCRE is compiled + with Unicode property support, but not otherwise. If you want to use + caseless matching in UTF8-mode for characters 128 and above, you must + ensure that PCRE is compiled with Unicode property support as well as with UTF-8 support. - Characters that might indicate line breaks are never treated in any - special way when matching character classes, whatever line-ending - sequence is in use, and whatever setting of the PCRE_DOTALL and + Characters that might indicate line breaks are never treated in any + special way when matching character classes, whatever line-ending + sequence is in use, and whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class such as [^a] always matches one of these characters. - The minus (hyphen) character can be used to specify a range of charac- - ters in a character class. For example, [d-m] matches any letter - between d and m, inclusive. If a minus character is required in a - class, it must be escaped with a backslash or appear in a position - where it cannot be interpreted as indicating a range, typically as the + The minus (hyphen) character can be used to specify a range of charac- + ters in a character class. For example, [d-m] matches any letter + between d and m, inclusive. If a minus character is required in a + class, it must be escaped with a backslash or appear in a position + where it cannot be interpreted as indicating a range, typically as the first or last character in the class. It is not possible to have the literal character "]" as the end charac- - ter of a range. A pattern such as [W-]46] is interpreted as a class of - two characters ("W" and "-") followed by a literal string "46]", so it - would match "W46]" or "-46]". However, if the "]" is escaped with a - backslash it is interpreted as the end of range, so [W-\]46] is inter- - preted as a class containing a range followed by two other characters. - The octal or hexadecimal representation of "]" can also be used to end + ter of a range. A pattern such as [W-]46] is interpreted as a class of + two characters ("W" and "-") followed by a literal string "46]", so it + would match "W46]" or "-46]". However, if the "]" is escaped with a + backslash it is interpreted as the end of range, so [W-\]46] is inter- + preted as a class containing a range followed by two other characters. + The octal or hexadecimal representation of "]" can also be used to end a range. - Ranges operate in the collating sequence of character values. They can - also be used for characters specified numerically, for example - [\000-\037]. In UTF-8 mode, ranges can include characters whose values + Ranges operate in the collating sequence of character values. They can + also be used for characters specified numerically, for example + [\000-\037]. In UTF-8 mode, ranges can include characters whose values are greater than 255, for example [\x{100}-\x{2ff}]. If a range that includes letters is used when caseless matching is set, it matches the letters in either case. For example, [W-c] is equivalent - to [][\\^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if - character tables for a French locale are in use, [\xc8-\xcb] matches - accented E characters in both cases. In UTF-8 mode, PCRE supports the - concept of case for characters with values greater than 128 only when + to [][\\^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if + character tables for a French locale are in use, [\xc8-\xcb] matches + accented E characters in both cases. In UTF-8 mode, PCRE supports the + concept of case for characters with values greater than 128 only when it is compiled with Unicode property support. - The character types \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, \w, and \W - may also appear in a character class, and add the characters that they - match to the class. For example, [\dABCDEF] matches any hexadecimal - digit. A circumflex can conveniently be used with the upper case char- - acter types to specify a more restricted set of characters than the - matching lower case type. For example, the class [^\W_] matches any - letter or digit, but not underscore. - - The only metacharacters that are recognized in character classes are - backslash, hyphen (only where it can be interpreted as specifying a - range), circumflex (only at the start), opening square bracket (only - when it can be interpreted as introducing a POSIX class name - see the - next section), and the terminating closing square bracket. However, + The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, + \w, and \W may appear in a character class, and add the characters that + they match to the class. For example, [\dABCDEF] matches any hexadeci- + mal digit. In UTF-8 mode, the PCRE_UCP option affects the meanings of + \d, \s, \w and their upper case partners, just as it does when they + appear outside a character class, as described in the section entitled + "Generic character types" above. The escape sequence \b has a different + meaning inside a character class; it matches the backspace character. + The sequences \B, \N, \R, and \X are not special inside a character + class. Like any other unrecognized escape sequences, they are treated + as the literal characters "B", "N", "R", and "X" by default, but cause + an error if the PCRE_EXTRA option is set. + + A circumflex can conveniently be used with the upper case character + types to specify a more restricted set of characters than the matching + lower case type. For example, the class [^\W_] matches any letter or + digit, but not underscore, whereas [\w] includes underscore. A positive + character class should be read as "something OR something OR ..." and a + negative class as "NOT something AND NOT something AND NOT ...". + + The only metacharacters that are recognized in character classes are + backslash, hyphen (only where it can be interpreted as specifying a + range), circumflex (only at the start), opening square bracket (only + when it can be interpreted as introducing a POSIX class name - see the + next section), and the terminating closing square bracket. However, escaping other non-alphanumeric characters does no harm. POSIX CHARACTER CLASSES Perl supports the POSIX notation for character classes. This uses names - enclosed by [: and :] within the enclosing square brackets. PCRE also + enclosed by [: and :] within the enclosing square brackets. PCRE also supports this notation. For example, [01[:alpha:]%] @@ -4106,24 +4144,24 @@ POSIX CHARACTER CLASSES word "word" characters (same as \w) xdigit hexadecimal digits - The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), - and space (32). Notice that this list includes the VT character (code + The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), + and space (32). Notice that this list includes the VT character (code 11). This makes "space" different to \s, which does not include VT (for Perl compatibility). - The name "word" is a Perl extension, and "blank" is a GNU extension - from Perl 5.8. Another Perl extension is negation, which is indicated + The name "word" is a Perl extension, and "blank" is a GNU extension + from Perl 5.8. Another Perl extension is negation, which is indicated by a ^ character after the colon. For example, [12[:^digit:]] - matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the + matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not supported, and an error is given if they are encountered. - By default, in UTF-8 mode, characters with values greater than 128 do - not match any of the POSIX character classes. However, if the PCRE_UCP - option is passed to pcre_compile(), some of the classes are changed so + By default, in UTF-8 mode, characters with values greater than 128 do + not match any of the POSIX character classes. However, if the PCRE_UCP + option is passed to pcre_compile(), some of the classes are changed so that Unicode character properties are used. This is achieved by replac- ing the POSIX classes by other sequences, as follows: @@ -4136,31 +4174,31 @@ POSIX CHARACTER CLASSES [:upper:] becomes \p{Lu} [:word:] becomes \p{Xwd} - Negated versions, such as [:^alpha:] use \P instead of \p. The other + Negated versions, such as [:^alpha:] use \P instead of \p. The other POSIX classes are unchanged, and match only characters with code points less than 128. VERTICAL BAR - Vertical bar characters are used to separate alternative patterns. For + Vertical bar characters are used to separate alternative patterns. For example, the pattern gilbert|sullivan - matches either "gilbert" or "sullivan". Any number of alternatives may - appear, and an empty alternative is permitted (matching the empty + matches either "gilbert" or "sullivan". Any number of alternatives may + appear, and an empty alternative is permitted (matching the empty string). The matching process tries each alternative in turn, from left - to right, and the first one that succeeds is used. If the alternatives - are within a subpattern (defined below), "succeeds" means matching the + to right, and the first one that succeeds is used. If the alternatives + are within a subpattern (defined below), "succeeds" means matching the rest of the main pattern as well as the alternative in the subpattern. INTERNAL OPTION SETTING - The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and - PCRE_EXTENDED options (which are Perl-compatible) can be changed from - within the pattern by a sequence of Perl option letters enclosed + The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and + PCRE_EXTENDED options (which are Perl-compatible) can be changed from + within the pattern by a sequence of Perl option letters enclosed between "(?" and ")". The option letters are i for PCRE_CASELESS @@ -4170,47 +4208,47 @@ INTERNAL OPTION SETTING For example, (?im) sets caseless, multiline matching. It is also possi- ble to unset these options by preceding the letter with a hyphen, and a - combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- - LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, - is also permitted. If a letter appears both before and after the + combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- + LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, + is also permitted. If a letter appears both before and after the hyphen, the option is unset. - The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA - can be changed in the same way as the Perl-compatible options by using + The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA + can be changed in the same way as the Perl-compatible options by using the characters J, U and X respectively. - When one of these option changes occurs at top level (that is, not - inside subpattern parentheses), the change applies to the remainder of + When one of these option changes occurs at top level (that is, not + inside subpattern parentheses), the change applies to the remainder of the pattern that follows. If the change is placed right at the start of a pattern, PCRE extracts it into the global options (and it will there- fore show up in data extracted by the pcre_fullinfo() function). - An option change within a subpattern (see below for a description of - subpatterns) affects only that part of the subpattern that follows it, + An option change within a subpattern (see below for a description of + subpatterns) affects only that part of the subpattern that follows it, so (a(?i)b)c matches abc and aBc and no other strings (assuming PCRE_CASELESS is not - used). By this means, options can be made to have different settings - in different parts of the pattern. Any changes made in one alternative - do carry on into subsequent branches within the same subpattern. For + used). By this means, options can be made to have different settings + in different parts of the pattern. Any changes made in one alternative + do carry on into subsequent branches within the same subpattern. For example, (a(?i)b|c) - matches "ab", "aB", "c", and "C", even though when matching "C" the - first branch is abandoned before the option setting. This is because - the effects of option settings happen at compile time. There would be + matches "ab", "aB", "c", and "C", even though when matching "C" the + first branch is abandoned before the option setting. This is because + the effects of option settings happen at compile time. There would be some very weird behaviour otherwise. - Note: There are other PCRE-specific options that can be set by the - application when the compile or match functions are called. In some + Note: There are other PCRE-specific options that can be set by the + application when the compile or match functions are called. In some cases the pattern can contain special leading sequences such as (*CRLF) - to override what the application has set or what has been defaulted. - Details are given in the section entitled "Newline sequences" above. - There are also the (*UTF8) and (*UCP) leading sequences that can be - used to set UTF-8 and Unicode property modes; they are equivalent to + to override what the application has set or what has been defaulted. + Details are given in the section entitled "Newline sequences" above. + There are also the (*UTF8) and (*UCP) leading sequences that can be + used to set UTF-8 and Unicode property modes; they are equivalent to setting the PCRE_UTF8 and the PCRE_UCP options, respectively. @@ -4223,15 +4261,15 @@ SUBPATTERNS cat(aract|erpillar|) - matches "cataract", "caterpillar", or "cat". Without the parentheses, + matches "cataract", "caterpillar", or "cat". Without the parentheses, it would match "cataract", "erpillar" or an empty string. - 2. It sets up the subpattern as a capturing subpattern. This means - that, when the whole pattern matches, that portion of the subject + 2. It sets up the subpattern as a capturing subpattern. This means + that, when the whole pattern matches, that portion of the subject string that matched the subpattern is passed back to the caller via the - ovector argument of pcre_exec(). Opening parentheses are counted from - left to right (starting from 1) to obtain numbers for the capturing - subpatterns. For example, if the string "the red king" is matched + ovector argument of pcre_exec(). Opening parentheses are counted from + left to right (starting from 1) to obtain numbers for the capturing + subpatterns. For example, if the string "the red king" is matched against the pattern the ((red|white) (king|queen)) @@ -4239,12 +4277,12 @@ SUBPATTERNS the captured substrings are "red king", "red", and "king", and are num- bered 1, 2, and 3, respectively. - The fact that plain parentheses fulfil two functions is not always - helpful. There are often times when a grouping subpattern is required - without a capturing requirement. If an opening parenthesis is followed - by a question mark and a colon, the subpattern does not do any captur- - ing, and is not counted when computing the number of any subsequent - capturing subpatterns. For example, if the string "the white queen" is + The fact that plain parentheses fulfil two functions is not always + helpful. There are often times when a grouping subpattern is required + without a capturing requirement. If an opening parenthesis is followed + by a question mark and a colon, the subpattern does not do any captur- + ing, and is not counted when computing the number of any subsequent + capturing subpatterns. For example, if the string "the white queen" is matched against the pattern the ((?:red|white) (king|queen)) @@ -4252,37 +4290,37 @@ SUBPATTERNS the captured substrings are "white queen" and "queen", and are numbered 1 and 2. The maximum number of capturing subpatterns is 65535. - As a convenient shorthand, if any option settings are required at the - start of a non-capturing subpattern, the option letters may appear + As a convenient shorthand, if any option settings are required at the + start of a non-capturing subpattern, the option letters may appear between the "?" and the ":". Thus the two patterns (?i:saturday|sunday) (?:(?i)saturday|sunday) match exactly the same set of strings. Because alternative branches are - tried from left to right, and options are not reset until the end of - the subpattern is reached, an option setting in one branch does affect - subsequent branches, so the above patterns match "SUNDAY" as well as + tried from left to right, and options are not reset until the end of + the subpattern is reached, an option setting in one branch does affect + subsequent branches, so the above patterns match "SUNDAY" as well as "Saturday". DUPLICATE SUBPATTERN NUMBERS Perl 5.10 introduced a feature whereby each alternative in a subpattern - uses the same numbers for its capturing parentheses. Such a subpattern - starts with (?| and is itself a non-capturing subpattern. For example, + uses the same numbers for its capturing parentheses. Such a subpattern + starts with (?| and is itself a non-capturing subpattern. For example, consider this pattern: (?|(Sat)ur|(Sun))day - Because the two alternatives are inside a (?| group, both sets of cap- - turing parentheses are numbered one. Thus, when the pattern matches, - you can look at captured substring number one, whichever alternative - matched. This construct is useful when you want to capture part, but + Because the two alternatives are inside a (?| group, both sets of cap- + turing parentheses are numbered one. Thus, when the pattern matches, + you can look at captured substring number one, whichever alternative + matched. This construct is useful when you want to capture part, but not all, of one of a number of alternatives. Inside a (?| group, paren- - theses are numbered as usual, but the number is reset at the start of - each branch. The numbers of any capturing parentheses that follow the - subpattern start after the highest number used in any branch. The fol- + theses are numbered as usual, but the number is reset at the start of + each branch. The numbers of any capturing parentheses that follow the + subpattern start after the highest number used in any branch. The fol- lowing example is taken from the Perl documentation. The numbers under- neath show in which buffer the captured content will be stored. @@ -4290,58 +4328,58 @@ DUPLICATE SUBPATTERN NUMBERS / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x # 1 2 2 3 2 3 4 - A back reference to a numbered subpattern uses the most recent value - that is set for that number by any subpattern. The following pattern + A back reference to a numbered subpattern uses the most recent value + that is set for that number by any subpattern. The following pattern matches "abcabc" or "defdef": /(?|(abc)|(def))\1/ - In contrast, a recursive or "subroutine" call to a numbered subpattern - always refers to the first one in the pattern with the given number. + In contrast, a recursive or "subroutine" call to a numbered subpattern + always refers to the first one in the pattern with the given number. The following pattern matches "abcabc" or "defabc": /(?|(abc)|(def))(?1)/ - If a condition test for a subpattern's having matched refers to a non- - unique number, the test is true if any of the subpatterns of that num- + If a condition test for a subpattern's having matched refers to a non- + unique number, the test is true if any of the subpatterns of that num- ber have matched. - An alternative approach to using this "branch reset" feature is to use + An alternative approach to using this "branch reset" feature is to use duplicate named subpatterns, as described in the next section. NAMED SUBPATTERNS - Identifying capturing parentheses by number is simple, but it can be - very hard to keep track of the numbers in complicated regular expres- - sions. Furthermore, if an expression is modified, the numbers may - change. To help with this difficulty, PCRE supports the naming of sub- + Identifying capturing parentheses by number is simple, but it can be + very hard to keep track of the numbers in complicated regular expres- + sions. Furthermore, if an expression is modified, the numbers may + change. To help with this difficulty, PCRE supports the naming of sub- patterns. This feature was not added to Perl until release 5.10. Python - had the feature earlier, and PCRE introduced it at release 4.0, using - the Python syntax. PCRE now supports both the Perl and the Python syn- - tax. Perl allows identically numbered subpatterns to have different + had the feature earlier, and PCRE introduced it at release 4.0, using + the Python syntax. PCRE now supports both the Perl and the Python syn- + tax. Perl allows identically numbered subpatterns to have different names, but PCRE does not. - In PCRE, a subpattern can be named in one of three ways: (?<name>...) - or (?'name'...) as in Perl, or (?P<name>...) as in Python. References - to capturing parentheses from other parts of the pattern, such as back - references, recursion, and conditions, can be made by name as well as + In PCRE, a subpattern can be named in one of three ways: (?<name>...) + or (?'name'...) as in Perl, or (?P<name>...) as in Python. References + to capturing parentheses from other parts of the pattern, such as back + references, recursion, and conditions, can be made by name as well as by number. - Names consist of up to 32 alphanumeric characters and underscores. - Named capturing parentheses are still allocated numbers as well as - names, exactly as if the names were not present. The PCRE API provides + Names consist of up to 32 alphanumeric characters and underscores. + Named capturing parentheses are still allocated numbers as well as + names, exactly as if the names were not present. The PCRE API provides function calls for extracting the name-to-number translation table from a compiled pattern. There is also a convenience function for extracting a captured substring by name. - By default, a name must be unique within a pattern, but it is possible + By default, a name must be unique within a pattern, but it is possible to relax this constraint by setting the PCRE_DUPNAMES option at compile - time. (Duplicate names are also always permitted for subpatterns with - the same number, set up as described in the previous section.) Dupli- - cate names can be useful for patterns where only one instance of the - named parentheses can match. Suppose you want to match the name of a - weekday, either as a 3-letter abbreviation or as the full name, and in + time. (Duplicate names are also always permitted for subpatterns with + the same number, set up as described in the previous section.) Dupli- + cate names can be useful for patterns where only one instance of the + named parentheses can match. Suppose you want to match the name of a + weekday, either as a 3-letter abbreviation or as the full name, and in both cases you want to extract the abbreviation. This pattern (ignoring the line breaks) does the job: @@ -4351,38 +4389,38 @@ NAMED SUBPATTERNS (?<DN>Thu)(?:rsday)?| (?<DN>Sat)(?:urday)? - There are five capturing substrings, but only one is ever set after a + There are five capturing substrings, but only one is ever set after a match. (An alternative way of solving this problem is to use a "branch reset" subpattern, as described in the previous section.) - The convenience function for extracting the data by name returns the - substring for the first (and in this example, the only) subpattern of - that name that matched. This saves searching to find which numbered + The convenience function for extracting the data by name returns the + substring for the first (and in this example, the only) subpattern of + that name that matched. This saves searching to find which numbered subpattern it was. - If you make a back reference to a non-unique named subpattern from - elsewhere in the pattern, the one that corresponds to the first occur- + If you make a back reference to a non-unique named subpattern from + elsewhere in the pattern, the one that corresponds to the first occur- rence of the name is used. In the absence of duplicate numbers (see the - previous section) this is the one with the lowest number. If you use a - named reference in a condition test (see the section about conditions - below), either to check whether a subpattern has matched, or to check - for recursion, all subpatterns with the same name are tested. If the - condition is true for any one of them, the overall condition is true. + previous section) this is the one with the lowest number. If you use a + named reference in a condition test (see the section about conditions + below), either to check whether a subpattern has matched, or to check + for recursion, all subpatterns with the same name are tested. If the + condition is true for any one of them, the overall condition is true. This is the same behaviour as testing by number. For further details of the interfaces for handling named subpatterns, see the pcreapi documen- tation. Warning: You cannot use different names to distinguish between two sub- - patterns with the same number because PCRE uses only the numbers when + patterns with the same number because PCRE uses only the numbers when matching. For this reason, an error is given at compile time if differ- - ent names are given to subpatterns with the same number. However, you - can give the same name to subpatterns with the same number, even when + ent names are given to subpatterns with the same number. However, you + can give the same name to subpatterns with the same number, even when PCRE_DUPNAMES is not set. REPETITION - Repetition is specified by quantifiers, which can follow any of the + Repetition is specified by quantifiers, which can follow any of the following items: a literal data character @@ -4396,17 +4434,17 @@ REPETITION a parenthesized subpattern (unless it is an assertion) a recursive or "subroutine" call to a subpattern - The general repetition quantifier specifies a minimum and maximum num- - ber of permitted matches, by giving the two numbers in curly brackets - (braces), separated by a comma. The numbers must be less than 65536, + The general repetition quantifier specifies a minimum and maximum num- + ber of permitted matches, by giving the two numbers in curly brackets + (braces), separated by a comma. The numbers must be less than 65536, and the first must be less than or equal to the second. For example: z{2,4} - matches "zz", "zzz", or "zzzz". A closing brace on its own is not a - special character. If the second number is omitted, but the comma is - present, there is no upper limit; if the second number and the comma - are both omitted, the quantifier specifies an exact number of required + matches "zz", "zzz", or "zzzz". A closing brace on its own is not a + special character. If the second number is omitted, but the comma is + present, there is no upper limit; if the second number and the comma + are both omitted, the quantifier specifies an exact number of required matches. Thus [aeiou]{3,} @@ -4415,50 +4453,50 @@ REPETITION \d{8} - matches exactly 8 digits. An opening curly bracket that appears in a - position where a quantifier is not allowed, or one that does not match - the syntax of a quantifier, is taken as a literal character. For exam- + matches exactly 8 digits. An opening curly bracket that appears in a + position where a quantifier is not allowed, or one that does not match + the syntax of a quantifier, is taken as a literal character. For exam- ple, {,6} is not a quantifier, but a literal string of four characters. - In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to + In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual bytes. Thus, for example, \x{100}{2} matches two UTF-8 char- acters, each of which is represented by a two-byte sequence. Similarly, when Unicode property support is available, \X{3} matches three Unicode - extended sequences, each of which may be several bytes long (and they + extended sequences, each of which may be several bytes long (and they may be of different lengths). The quantifier {0} is permitted, causing the expression to behave as if the previous item and the quantifier were not present. This may be use- - ful for subpatterns that are referenced as subroutines from elsewhere + ful for subpatterns that are referenced as subroutines from elsewhere in the pattern (but see also the section entitled "Defining subpatterns - for use by reference only" below). Items other than subpatterns that + for use by reference only" below). Items other than subpatterns that have a {0} quantifier are omitted from the compiled pattern. - For convenience, the three most common quantifiers have single-charac- + For convenience, the three most common quantifiers have single-charac- ter abbreviations: * is equivalent to {0,} + is equivalent to {1,} ? is equivalent to {0,1} - It is possible to construct infinite loops by following a subpattern + It is possible to construct infinite loops by following a subpattern that can match no characters with a quantifier that has no upper limit, for example: (a?)* Earlier versions of Perl and PCRE used to give an error at compile time - for such patterns. However, because there are cases where this can be - useful, such patterns are now accepted, but if any repetition of the - subpattern does in fact match no characters, the loop is forcibly bro- + for such patterns. However, because there are cases where this can be + useful, such patterns are now accepted, but if any repetition of the + subpattern does in fact match no characters, the loop is forcibly bro- ken. - By default, the quantifiers are "greedy", that is, they match as much - as possible (up to the maximum number of permitted times), without - causing the rest of the pattern to fail. The classic example of where + By default, the quantifiers are "greedy", that is, they match as much + as possible (up to the maximum number of permitted times), without + causing the rest of the pattern to fail. The classic example of where this gives problems is in trying to match comments in C programs. These - appear between /* and */ and within the comment, individual * and / - characters may appear. An attempt to match C comments by applying the + appear between /* and */ and within the comment, individual * and / + characters may appear. An attempt to match C comments by applying the pattern /\*.*\*/ @@ -4467,19 +4505,19 @@ REPETITION /* first comment */ not comment /* second comment */ - fails, because it matches the entire string owing to the greediness of + fails, because it matches the entire string owing to the greediness of the .* item. - However, if a quantifier is followed by a question mark, it ceases to + However, if a quantifier is followed by a question mark, it ceases to be greedy, and instead matches the minimum number of times possible, so the pattern /\*.*?\*/ - does the right thing with the C comments. The meaning of the various - quantifiers is not otherwise changed, just the preferred number of - matches. Do not confuse this use of question mark with its use as a - quantifier in its own right. Because it has two uses, it can sometimes + does the right thing with the C comments. The meaning of the various + quantifiers is not otherwise changed, just the preferred number of + matches. Do not confuse this use of question mark with its use as a + quantifier in its own right. Because it has two uses, it can sometimes appear doubled, as in \d??\d @@ -4487,36 +4525,36 @@ REPETITION which matches one digit by preference, but can match two if that is the only way the rest of the pattern matches. - If the PCRE_UNGREEDY option is set (an option that is not available in - Perl), the quantifiers are not greedy by default, but individual ones - can be made greedy by following them with a question mark. In other + If the PCRE_UNGREEDY option is set (an option that is not available in + Perl), the quantifiers are not greedy by default, but individual ones + can be made greedy by following them with a question mark. In other words, it inverts the default behaviour. - When a parenthesized subpattern is quantified with a minimum repeat - count that is greater than 1 or with a limited maximum, more memory is - required for the compiled pattern, in proportion to the size of the + When a parenthesized subpattern is quantified with a minimum repeat + count that is greater than 1 or with a limited maximum, more memory is + required for the compiled pattern, in proportion to the size of the minimum or maximum. If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equiv- - alent to Perl's /s) is set, thus allowing the dot to match newlines, - the pattern is implicitly anchored, because whatever follows will be - tried against every character position in the subject string, so there - is no point in retrying the overall match at any position after the - first. PCRE normally treats such a pattern as though it were preceded + alent to Perl's /s) is set, thus allowing the dot to match newlines, + the pattern is implicitly anchored, because whatever follows will be + tried against every character position in the subject string, so there + is no point in retrying the overall match at any position after the + first. PCRE normally treats such a pattern as though it were preceded by \A. - In cases where it is known that the subject string contains no new- - lines, it is worth setting PCRE_DOTALL in order to obtain this opti- + In cases where it is known that the subject string contains no new- + lines, it is worth setting PCRE_DOTALL in order to obtain this opti- mization, or alternatively using ^ to indicate anchoring explicitly. - However, there is one situation where the optimization cannot be used. + However, there is one situation where the optimization cannot be used. When .* is inside capturing parentheses that are the subject of a back reference elsewhere in the pattern, a match at the start may fail where a later one succeeds. Consider, for example: (.*)abc\1 - If the subject is "xyz123abc123" the match point is the fourth charac- + If the subject is "xyz123abc123" the match point is the fourth charac- ter. For this reason, such a pattern is not implicitly anchored. When a capturing subpattern is repeated, the value captured is the sub- @@ -4525,8 +4563,8 @@ REPETITION (tweedle[dume]{3}\s*)+ has matched "tweedledum tweedledee" the value of the captured substring - is "tweedledee". However, if there are nested capturing subpatterns, - the corresponding captured values may have been set in previous itera- + is "tweedledee". However, if there are nested capturing subpatterns, + the corresponding captured values may have been set in previous itera- tions. For example, after /(a|(b))+/ @@ -4536,53 +4574,53 @@ REPETITION ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS - With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") - repetition, failure of what follows normally causes the repeated item - to be re-evaluated to see if a different number of repeats allows the - rest of the pattern to match. Sometimes it is useful to prevent this, - either to change the nature of the match, or to cause it fail earlier - than it otherwise might, when the author of the pattern knows there is + With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") + repetition, failure of what follows normally causes the repeated item + to be re-evaluated to see if a different number of repeats allows the + rest of the pattern to match. Sometimes it is useful to prevent this, + either to change the nature of the match, or to cause it fail earlier + than it otherwise might, when the author of the pattern knows there is no point in carrying on. - Consider, for example, the pattern \d+foo when applied to the subject + Consider, for example, the pattern \d+foo when applied to the subject line 123456bar After matching all 6 digits and then failing to match "foo", the normal - action of the matcher is to try again with only 5 digits matching the - \d+ item, and then with 4, and so on, before ultimately failing. - "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides - the means for specifying that once a subpattern has matched, it is not + action of the matcher is to try again with only 5 digits matching the + \d+ item, and then with 4, and so on, before ultimately failing. + "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides + the means for specifying that once a subpattern has matched, it is not to be re-evaluated in this way. - If we use atomic grouping for the previous example, the matcher gives - up immediately on failing to match "foo" the first time. The notation + If we use atomic grouping for the previous example, the matcher gives + up immediately on failing to match "foo" the first time. The notation is a kind of special parenthesis, starting with (?> as in this example: (?>\d+)foo - This kind of parenthesis "locks up" the part of the pattern it con- - tains once it has matched, and a failure further into the pattern is - prevented from backtracking into it. Backtracking past it to previous + This kind of parenthesis "locks up" the part of the pattern it con- + tains once it has matched, and a failure further into the pattern is + prevented from backtracking into it. Backtracking past it to previous items, however, works as normal. - An alternative description is that a subpattern of this type matches - the string of characters that an identical standalone pattern would + An alternative description is that a subpattern of this type matches + the string of characters that an identical standalone pattern would match, if anchored at the current point in the subject string. Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as the above example can be thought of as a maximizing repeat that - must swallow everything it can. So, while both \d+ and \d+? are pre- - pared to adjust the number of digits they match in order to make the + must swallow everything it can. So, while both \d+ and \d+? are pre- + pared to adjust the number of digits they match in order to make the rest of the pattern match, (?>\d+) can only match an entire sequence of digits. - Atomic groups in general can of course contain arbitrarily complicated - subpatterns, and can be nested. However, when the subpattern for an + Atomic groups in general can of course contain arbitrarily complicated + subpatterns, and can be nested. However, when the subpattern for an atomic group is just a single repeated item, as in the example above, a - simpler notation, called a "possessive quantifier" can be used. This - consists of an additional + character following a quantifier. Using + simpler notation, called a "possessive quantifier" can be used. This + consists of an additional + character following a quantifier. Using this notation, the previous example can be rewritten as \d++foo @@ -4592,45 +4630,45 @@ ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS (abc|xyz){2,3}+ - Possessive quantifiers are always greedy; the setting of the + Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY option is ignored. They are a convenient notation for the - simpler forms of atomic group. However, there is no difference in the - meaning of a possessive quantifier and the equivalent atomic group, - though there may be a performance difference; possessive quantifiers + simpler forms of atomic group. However, there is no difference in the + meaning of a possessive quantifier and the equivalent atomic group, + though there may be a performance difference; possessive quantifiers should be slightly faster. - The possessive quantifier syntax is an extension to the Perl 5.8 syn- - tax. Jeffrey Friedl originated the idea (and the name) in the first + The possessive quantifier syntax is an extension to the Perl 5.8 syn- + tax. Jeffrey Friedl originated the idea (and the name) in the first edition of his book. Mike McCloskey liked it, so implemented it when he - built Sun's Java package, and PCRE copied it from there. It ultimately + built Sun's Java package, and PCRE copied it from there. It ultimately found its way into Perl at release 5.10. PCRE has an optimization that automatically "possessifies" certain sim- - ple pattern constructs. For example, the sequence A+B is treated as - A++B because there is no point in backtracking into a sequence of A's + ple pattern constructs. For example, the sequence A+B is treated as + A++B because there is no point in backtracking into a sequence of A's when B must follow. - When a pattern contains an unlimited repeat inside a subpattern that - can itself be repeated an unlimited number of times, the use of an - atomic group is the only way to avoid some failing matches taking a + When a pattern contains an unlimited repeat inside a subpattern that + can itself be repeated an unlimited number of times, the use of an + atomic group is the only way to avoid some failing matches taking a very long time indeed. The pattern (\D+|<\d+>)*[!?] - matches an unlimited number of substrings that either consist of non- - digits, or digits enclosed in <>, followed by either ! or ?. When it + matches an unlimited number of substrings that either consist of non- + digits, or digits enclosed in <>, followed by either ! or ?. When it matches, it runs quickly. However, if it is applied to aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa - it takes a long time before reporting failure. This is because the - string can be divided between the internal \D+ repeat and the external - * repeat in a large number of ways, and all have to be tried. (The - example uses [!?] rather than a single character at the end, because - both PCRE and Perl have an optimization that allows for fast failure - when a single character is used. They remember the last single charac- - ter that is required for a match, and fail early if it is not present - in the string.) If the pattern is changed so that it uses an atomic + it takes a long time before reporting failure. This is because the + string can be divided between the internal \D+ repeat and the external + * repeat in a large number of ways, and all have to be tried. (The + example uses [!?] rather than a single character at the end, because + both PCRE and Perl have an optimization that allows for fast failure + when a single character is used. They remember the last single charac- + ter that is required for a match, and fail early if it is not present + in the string.) If the pattern is changed so that it uses an atomic group, like this: ((?>\D+)|<\d+>)*[!?] @@ -4642,28 +4680,28 @@ BACK REFERENCES Outside a character class, a backslash followed by a digit greater than 0 (and possibly further digits) is a back reference to a capturing sub- - pattern earlier (that is, to its left) in the pattern, provided there + pattern earlier (that is, to its left) in the pattern, provided there have been that many previous capturing left parentheses. However, if the decimal number following the backslash is less than 10, - it is always taken as a back reference, and causes an error only if - there are not that many capturing left parentheses in the entire pat- - tern. In other words, the parentheses that are referenced need not be - to the left of the reference for numbers less than 10. A "forward back - reference" of this type can make sense when a repetition is involved - and the subpattern to the right has participated in an earlier itera- + it is always taken as a back reference, and causes an error only if + there are not that many capturing left parentheses in the entire pat- + tern. In other words, the parentheses that are referenced need not be + to the left of the reference for numbers less than 10. A "forward back + reference" of this type can make sense when a repetition is involved + and the subpattern to the right has participated in an earlier itera- tion. - It is not possible to have a numerical "forward back reference" to a - subpattern whose number is 10 or more using this syntax because a - sequence such as \50 is interpreted as a character defined in octal. + It is not possible to have a numerical "forward back reference" to a + subpattern whose number is 10 or more using this syntax because a + sequence such as \50 is interpreted as a character defined in octal. See the subsection entitled "Non-printing characters" above for further - details of the handling of digits following a backslash. There is no - such problem when named parentheses are used. A back reference to any + details of the handling of digits following a backslash. There is no + such problem when named parentheses are used. A back reference to any subpattern is possible using named parentheses (see below). - Another way of avoiding the ambiguity inherent in the use of digits - following a backslash is to use the \g escape sequence. This escape + Another way of avoiding the ambiguity inherent in the use of digits + following a backslash is to use the \g escape sequence. This escape must be followed by an unsigned number or a negative number, optionally enclosed in braces. These examples are all identical: @@ -4671,7 +4709,7 @@ BACK REFERENCES (ring), \g1 (ring), \g{1} - An unsigned number specifies an absolute reference without the ambigu- + An unsigned number specifies an absolute reference without the ambigu- ity that is present in the older syntax. It is also useful when literal digits follow the reference. A negative number is a relative reference. Consider this example: @@ -4679,10 +4717,11 @@ BACK REFERENCES (abc(def)ghi)\g{-1} The sequence \g{-1} is a reference to the most recently started captur- - ing subpattern before \g, that is, is it equivalent to \2. Similarly, - \g{-2} would be equivalent to \1. The use of relative references can be - helpful in long patterns, and also in patterns that are created by - joining together fragments that contain references within themselves. + ing subpattern before \g, that is, is it equivalent to \2 in this exam- + ple. Similarly, \g{-2} would be equivalent to \1. The use of relative + references can be helpful in long patterns, and also in patterns that + are created by joining together fragments that contain references + within themselves. A back reference matches whatever actually matched the capturing sub- pattern in the current subject string, rather than anything matching @@ -4802,7 +4841,7 @@ ASSERTIONS most convenient way to do it is with (?!) because an empty string always matches, so an assertion that requires there not to be an empty string must always fail. The backtracking control verb (*FAIL) or (*F) - is essentially a synonym for (?!). + is a synonym for (?!). Lookbehind assertions @@ -4945,63 +4984,64 @@ CONDITIONAL SUBPATTERNS the condition is true if a capturing subpattern of that number has pre- viously matched. If there is more than one capturing subpattern with the same number (see the earlier section about duplicate subpattern - numbers), the condition is true if any of them have been set. An alter- + numbers), the condition is true if any of them have matched. An alter- native notation is to precede the digits with a plus or minus sign. In this case, the subpattern number is relative rather than absolute. The most recently opened parentheses can be referenced by (?(-1), the next - most recent by (?(-2), and so on. In looping constructs it can also - make sense to refer to subsequent groups with constructs such as - (?(+2). + most recent by (?(-2), and so on. Inside loops it can also make sense + to refer to subsequent groups. The next parentheses to be opened can be + referenced as (?(+1), and so on. (The value zero in any of these forms + is not used; it provokes a compile-time error.) - Consider the following pattern, which contains non-significant white + Consider the following pattern, which contains non-significant white space to make it more readable (assume the PCRE_EXTENDED option) and to divide it into three parts for ease of discussion: ( \( )? [^()]+ (?(1) \) ) - The first part matches an optional opening parenthesis, and if that + The first part matches an optional opening parenthesis, and if that character is present, sets it as the first captured substring. The sec- - ond part matches one or more characters that are not parentheses. The - third part is a conditional subpattern that tests whether the first set - of parentheses matched or not. If they did, that is, if subject started - with an opening parenthesis, the condition is true, and so the yes-pat- - tern is executed and a closing parenthesis is required. Otherwise, - since no-pattern is not present, the subpattern matches nothing. In - other words, this pattern matches a sequence of non-parentheses, + ond part matches one or more characters that are not parentheses. The + third part is a conditional subpattern that tests whether or not the + first set of parentheses matched. If they did, that is, if subject + started with an opening parenthesis, the condition is true, and so the + yes-pattern is executed and a closing parenthesis is required. Other- + wise, since no-pattern is not present, the subpattern matches nothing. + In other words, this pattern matches a sequence of non-parentheses, optionally enclosed in parentheses. - If you were embedding this pattern in a larger one, you could use a + If you were embedding this pattern in a larger one, you could use a relative reference: ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... - This makes the fragment independent of the parentheses in the larger + This makes the fragment independent of the parentheses in the larger pattern. Checking for a used subpattern by name - Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a - used subpattern by name. For compatibility with earlier versions of - PCRE, which had this facility before Perl, the syntax (?(name)...) is - also recognized. However, there is a possible ambiguity with this syn- - tax, because subpattern names may consist entirely of digits. PCRE - looks first for a named subpattern; if it cannot find one and the name - consists entirely of digits, PCRE looks for a subpattern of that num- - ber, which must be greater than zero. Using subpattern names that con- + Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a + used subpattern by name. For compatibility with earlier versions of + PCRE, which had this facility before Perl, the syntax (?(name)...) is + also recognized. However, there is a possible ambiguity with this syn- + tax, because subpattern names may consist entirely of digits. PCRE + looks first for a named subpattern; if it cannot find one and the name + consists entirely of digits, PCRE looks for a subpattern of that num- + ber, which must be greater than zero. Using subpattern names that con- sist entirely of digits is not recommended. Rewriting the above example to use a named subpattern gives this: (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) - If the name used in a condition of this kind is a duplicate, the test - is applied to all subpatterns of the same name, and is true if any one + If the name used in a condition of this kind is a duplicate, the test + is applied to all subpatterns of the same name, and is true if any one of them has matched. Checking for pattern recursion If the condition is the string (R), and there is no subpattern with the - name R, the condition is true if a recursive call to the whole pattern + name R, the condition is true if a recursive call to the whole pattern or any subpattern has been made. If digits or a name preceded by amper- sand follow the letter R, for example: @@ -5009,23 +5049,24 @@ CONDITIONAL SUBPATTERNS the condition is true if the most recent recursion is into a subpattern whose number or name is given. This condition does not check the entire - recursion stack. If the name used in a condition of this kind is a + recursion stack. If the name used in a condition of this kind is a duplicate, the test is applied to all subpatterns of the same name, and is true if any one of them is the most recent recursion. - At "top level", all these recursion test conditions are false. The + At "top level", all these recursion test conditions are false. The syntax for recursive patterns is described below. Defining subpatterns for use by reference only - If the condition is the string (DEFINE), and there is no subpattern - with the name DEFINE, the condition is always false. In this case, - there may be only one alternative in the subpattern. It is always - skipped if control reaches this point in the pattern; the idea of - DEFINE is that it can be used to define "subroutines" that can be ref- - erenced from elsewhere. (The use of "subroutines" is described below.) - For example, a pattern to match an IPv4 address could be written like - this (ignore whitespace and line breaks): + If the condition is the string (DEFINE), and there is no subpattern + with the name DEFINE, the condition is always false. In this case, + there may be only one alternative in the subpattern. It is always + skipped if control reaches this point in the pattern; the idea of + DEFINE is that it can be used to define "subroutines" that can be ref- + erenced from elsewhere. (The use of "subroutines" is described below.) + For example, a pattern to match an IPv4 address such as + "192.168.23.245" could be written like this (ignore whitespace and line + breaks): (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) \b (?&byte) (\.(?&byte)){3} \b @@ -5242,9 +5283,9 @@ RECURSIVE PATTERNS remaining alternative is at a deeper recursion level, which PCRE cannot use. - To change the pattern so that matches all palindromic strings, not just - those with an odd number of characters, it is tempting to change the - pattern to this: + To change the pattern so that it matches all palindromic strings, not + just those with an odd number of characters, it is tempting to change + the pattern to this: ^((.)(?1)\2|.?)$ @@ -5411,57 +5452,58 @@ BACKTRACKING CONTROL character must be present. When one of these optimizations suppresses the running of a match, any included backtracking verbs will not, of course, be processed. You can suppress the start-of-match optimizations - by setting the PCRE_NO_START_OPTIMIZE option when calling pcre_exec(). + by setting the PCRE_NO_START_OPTIMIZE option when calling pcre_com- + pile() or pcre_exec(), or by starting the pattern with (*NO_START_OPT). Verbs that act immediately - The following verbs act as soon as they are encountered. They may not + The following verbs act as soon as they are encountered. They may not be followed by a name. (*ACCEPT) - This verb causes the match to end successfully, skipping the remainder - of the pattern. When inside a recursion, only the innermost pattern is - ended immediately. If (*ACCEPT) is inside capturing parentheses, the - data so far is captured. (This feature was added to PCRE at release + This verb causes the match to end successfully, skipping the remainder + of the pattern. When inside a recursion, only the innermost pattern is + ended immediately. If (*ACCEPT) is inside capturing parentheses, the + data so far is captured. (This feature was added to PCRE at release 8.00.) For example: A((?:A|B(*ACCEPT)|C)D) - This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- + This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- tured by the outer parentheses. (*FAIL) or (*F) - This verb causes the match to fail, forcing backtracking to occur. It - is equivalent to (?!) but easier to read. The Perl documentation notes - that it is probably useful only when combined with (?{}) or (??{}). - Those are, of course, Perl features that are not present in PCRE. The - nearest equivalent is the callout feature, as for example in this pat- + This verb causes the match to fail, forcing backtracking to occur. It + is equivalent to (?!) but easier to read. The Perl documentation notes + that it is probably useful only when combined with (?{}) or (??{}). + Those are, of course, Perl features that are not present in PCRE. The + nearest equivalent is the callout feature, as for example in this pat- tern: a+(?C)(*FAIL) - A match with the string "aaaa" always fails, but the callout is taken + A match with the string "aaaa" always fails, but the callout is taken before each backtrack happens (in this example, 10 times). Recording which path was taken - There is one verb whose main purpose is to track how a match was - arrived at, though it also has a secondary use in conjunction with + There is one verb whose main purpose is to track how a match was + arrived at, though it also has a secondary use in conjunction with advancing the match starting point (see (*SKIP) below). (*MARK:NAME) or (*:NAME) - A name is always required with this verb. There may be as many - instances of (*MARK) as you like in a pattern, and their names do not + A name is always required with this verb. There may be as many + instances of (*MARK) as you like in a pattern, and their names do not have to be unique. - When a match succeeds, the name of the last-encountered (*MARK) is - passed back to the caller via the pcre_extra data structure, as + When a match succeeds, the name of the last-encountered (*MARK) is + passed back to the caller via the pcre_extra data structure, as described in the section on pcre_extra in the pcreapi documentation. No - data is returned for a partial match. Here is an example of pcretest - output, where the /K modifier requests the retrieval and outputting of + data is returned for a partial match. Here is an example of pcretest + output, where the /K modifier requests the retrieval and outputting of (*MARK) data: /X(*MARK:A)Y|X(*MARK:B)Z/K @@ -5473,13 +5515,13 @@ BACKTRACKING CONTROL MK: B The (*MARK) name is tagged with "MK:" in this output, and in this exam- - ple it indicates which of the two alternatives matched. This is a more - efficient way of obtaining this information than putting each alterna- + ple it indicates which of the two alternatives matched. This is a more + efficient way of obtaining this information than putting each alterna- tive in its own capturing parentheses. - A name may also be returned after a failed match if the final path - through the pattern involves (*MARK). However, unless (*MARK) used in - conjunction with (*COMMIT), this is unlikely to happen for an unan- + A name may also be returned after a failed match if the final path + through the pattern involves (*MARK). However, unless (*MARK) used in + conjunction with (*COMMIT), this is unlikely to happen for an unan- chored pattern because, as the starting point for matching is advanced, the final check is often with an empty string, causing a failure before (*MARK) is reached. For example: @@ -5489,56 +5531,56 @@ BACKTRACKING CONTROL No match There are three potential starting points for this match (starting with - X, starting with P, and with an empty string). If the pattern is + X, starting with P, and with an empty string). If the pattern is anchored, the result is different: /^X(*MARK:A)Y|^X(*MARK:B)Z/K XP No match, mark = B - PCRE's start-of-match optimizations can also interfere with this. For - example, if, as a result of a call to pcre_study(), it knows the mini- - mum subject length for a match, a shorter subject will not be scanned + PCRE's start-of-match optimizations can also interfere with this. For + example, if, as a result of a call to pcre_study(), it knows the mini- + mum subject length for a match, a shorter subject will not be scanned at all. Note that similar anomalies (though different in detail) exist in Perl, - no doubt for the same reasons. The use of (*MARK) data after a failed - match of an unanchored pattern is not recommended, unless (*COMMIT) is + no doubt for the same reasons. The use of (*MARK) data after a failed + match of an unanchored pattern is not recommended, unless (*COMMIT) is involved. Verbs that act after backtracking The following verbs do nothing when they are encountered. Matching con- - tinues with what follows, but if there is no subsequent match, causing - a backtrack to the verb, a failure is forced. That is, backtracking - cannot pass to the left of the verb. However, when one of these verbs - appears inside an atomic group, its effect is confined to that group, - because once the group has been matched, there is never any backtrack- - ing into it. In this situation, backtracking can "jump back" to the - left of the entire atomic group. (Remember also, as stated above, that + tinues with what follows, but if there is no subsequent match, causing + a backtrack to the verb, a failure is forced. That is, backtracking + cannot pass to the left of the verb. However, when one of these verbs + appears inside an atomic group, its effect is confined to that group, + because once the group has been matched, there is never any backtrack- + ing into it. In this situation, backtracking can "jump back" to the + left of the entire atomic group. (Remember also, as stated above, that this localization also applies in subroutine calls and assertions.) - These verbs differ in exactly what kind of failure occurs when back- + These verbs differ in exactly what kind of failure occurs when back- tracking reaches them. (*COMMIT) - This verb, which may not be followed by a name, causes the whole match + This verb, which may not be followed by a name, causes the whole match to fail outright if the rest of the pattern does not match. Even if the pattern is unanchored, no further attempts to find a match by advancing the starting point take place. Once (*COMMIT) has been passed, - pcre_exec() is committed to finding a match at the current starting + pcre_exec() is committed to finding a match at the current starting point, or not at all. For example: a+(*COMMIT)b - This matches "xxaab" but not "aacaab". It can be thought of as a kind + This matches "xxaab" but not "aacaab". It can be thought of as a kind of dynamic anchor, or "I've started, so I must finish." The name of the - most recently passed (*MARK) in the path is passed back when (*COMMIT) + most recently passed (*MARK) in the path is passed back when (*COMMIT) forces a match failure. - Note that (*COMMIT) at the start of a pattern is not the same as an - anchor, unless PCRE's start-of-match optimizations are turned off, as + Note that (*COMMIT) at the start of a pattern is not the same as an + anchor, unless PCRE's start-of-match optimizations are turned off, as shown in this pcretest example: /(*COMMIT)abc/ @@ -5547,90 +5589,90 @@ BACKTRACKING CONTROL xyzabc\Y No match - PCRE knows that any match must start with "a", so the optimization - skips along the subject to "a" before running the first match attempt, - which succeeds. When the optimization is disabled by the \Y escape in + PCRE knows that any match must start with "a", so the optimization + skips along the subject to "a" before running the first match attempt, + which succeeds. When the optimization is disabled by the \Y escape in the second subject, the match starts at "x" and so the (*COMMIT) causes it to fail without trying any other starting points. (*PRUNE) or (*PRUNE:NAME) - This verb causes the match to fail at the current starting position in - the subject if the rest of the pattern does not match. If the pattern - is unanchored, the normal "bumpalong" advance to the next starting - character then happens. Backtracking can occur as usual to the left of - (*PRUNE), before it is reached, or when matching to the right of - (*PRUNE), but if there is no match to the right, backtracking cannot - cross (*PRUNE). In simple cases, the use of (*PRUNE) is just an alter- - native to an atomic group or possessive quantifier, but there are some + This verb causes the match to fail at the current starting position in + the subject if the rest of the pattern does not match. If the pattern + is unanchored, the normal "bumpalong" advance to the next starting + character then happens. Backtracking can occur as usual to the left of + (*PRUNE), before it is reached, or when matching to the right of + (*PRUNE), but if there is no match to the right, backtracking cannot + cross (*PRUNE). In simple cases, the use of (*PRUNE) is just an alter- + native to an atomic group or possessive quantifier, but there are some uses of (*PRUNE) that cannot be expressed in any other way. The behav- - iour of (*PRUNE:NAME) is the same as (*MARK:NAME)(*PRUNE) when the - match fails completely; the name is passed back if this is the final - attempt. (*PRUNE:NAME) does not pass back a name if the match suc- - ceeds. In an anchored pattern (*PRUNE) has the same effect as (*COM- + iour of (*PRUNE:NAME) is the same as (*MARK:NAME)(*PRUNE) when the + match fails completely; the name is passed back if this is the final + attempt. (*PRUNE:NAME) does not pass back a name if the match suc- + ceeds. In an anchored pattern (*PRUNE) has the same effect as (*COM- MIT). (*SKIP) - This verb, when given without a name, is like (*PRUNE), except that if - the pattern is unanchored, the "bumpalong" advance is not to the next + This verb, when given without a name, is like (*PRUNE), except that if + the pattern is unanchored, the "bumpalong" advance is not to the next character, but to the position in the subject where (*SKIP) was encoun- - tered. (*SKIP) signifies that whatever text was matched leading up to + tered. (*SKIP) signifies that whatever text was matched leading up to it cannot be part of a successful match. Consider: a+(*SKIP)b - If the subject is "aaaac...", after the first match attempt fails - (starting at the first character in the string), the starting point + If the subject is "aaaac...", after the first match attempt fails + (starting at the first character in the string), the starting point skips on to start the next attempt at "c". Note that a possessive quan- - tifer does not have the same effect as this example; although it would - suppress backtracking during the first match attempt, the second - attempt would start at the second character instead of skipping on to + tifer does not have the same effect as this example; although it would + suppress backtracking during the first match attempt, the second + attempt would start at the second character instead of skipping on to "c". (*SKIP:NAME) - When (*SKIP) has an associated name, its behaviour is modified. If the + When (*SKIP) has an associated name, its behaviour is modified. If the following pattern fails to match, the previous path through the pattern - is searched for the most recent (*MARK) that has the same name. If one - is found, the "bumpalong" advance is to the subject position that cor- - responds to that (*MARK) instead of to where (*SKIP) was encountered. - If no (*MARK) with a matching name is found, normal "bumpalong" of one + is searched for the most recent (*MARK) that has the same name. If one + is found, the "bumpalong" advance is to the subject position that cor- + responds to that (*MARK) instead of to where (*SKIP) was encountered. + If no (*MARK) with a matching name is found, normal "bumpalong" of one character happens (the (*SKIP) is ignored). (*THEN) or (*THEN:NAME) - This verb causes a skip to the next alternation in the innermost - enclosing group if the rest of the pattern does not match. That is, it - cancels pending backtracking, but only within the current alternation. - Its name comes from the observation that it can be used for a pattern- + This verb causes a skip to the next alternation in the innermost + enclosing group if the rest of the pattern does not match. That is, it + cancels pending backtracking, but only within the current alternation. + Its name comes from the observation that it can be used for a pattern- based if-then-else block: ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... - If the COND1 pattern matches, FOO is tried (and possibly further items - after the end of the group if FOO succeeds); on failure the matcher - skips to the second alternative and tries COND2, without backtracking - into COND1. The behaviour of (*THEN:NAME) is exactly the same as - (*MARK:NAME)(*THEN) if the overall match fails. If (*THEN) is not + If the COND1 pattern matches, FOO is tried (and possibly further items + after the end of the group if FOO succeeds); on failure the matcher + skips to the second alternative and tries COND2, without backtracking + into COND1. The behaviour of (*THEN:NAME) is exactly the same as + (*MARK:NAME)(*THEN) if the overall match fails. If (*THEN) is not directly inside an alternation, it acts like (*PRUNE). The above verbs provide four different "strengths" of control when sub- - sequent matching fails. (*THEN) is the weakest, carrying on the match - at the next alternation. (*PRUNE) comes next, failing the match at the - current starting position, but allowing an advance to the next charac- - ter (for an unanchored pattern). (*SKIP) is similar, except that the - advance may be more than one character. (*COMMIT) is the strongest, + sequent matching fails. (*THEN) is the weakest, carrying on the match + at the next alternation. (*PRUNE) comes next, failing the match at the + current starting position, but allowing an advance to the next charac- + ter (for an unanchored pattern). (*SKIP) is similar, except that the + advance may be more than one character. (*COMMIT) is the strongest, causing the entire match to fail. - If more than one is present in a pattern, the "stongest" one wins. For - example, consider this pattern, where A, B, etc. are complex pattern + If more than one is present in a pattern, the "stongest" one wins. For + example, consider this pattern, where A, B, etc. are complex pattern fragments: (A(*COMMIT)B(*THEN)C|D) - Once A has matched, PCRE is committed to this match, at the current - starting position. If subsequently B matches, but C does not, the nor- + Once A has matched, PCRE is committed to this match, at the current + starting position. If subsequently B matches, but C does not, the nor- mal (*THEN) action of trying the next alternation (that is, D) does not happen because (*COMMIT) overrides. @@ -5649,11 +5691,11 @@ AUTHOR REVISION - Last updated: 17 November 2010 + Last updated: 21 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRESYNTAX(3) PCRESYNTAX(3) @@ -5677,7 +5719,7 @@ QUOTING CHARACTERS \a alarm, that is, the BEL character (hex 07) - \cx "control-x", where x is any character + \cx "control-x", where x is any ASCII character \e escape (hex 1B) \f formfeed (hex 0C) \n newline (hex 0A) @@ -5896,6 +5938,7 @@ OPTION SETTING The following are recognized only at the start of a pattern or after one of the newline-setting options with similar syntax: + (*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE) (*UTF8) set UTF-8 mode (PCRE_UTF8) (*UCP) set PCRE_UCP (use Unicode properties for \d etc) @@ -6018,11 +6061,11 @@ AUTHOR REVISION - Last updated: 12 May 2010 + Last updated: 21 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREPARTIAL(3) PCREPARTIAL(3) @@ -6441,8 +6484,8 @@ REVISION Last updated: 07 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREPRECOMPILE(3) PCREPRECOMPILE(3) @@ -6550,7 +6593,7 @@ COMPATIBILITY WITH DIFFERENT PCRE RELEASES In general, it is safest to recompile all saved patterns when you update to a new PCRE release, though not all updates actually require - this. Recompiling is definitely needed for release 7.2. + this. AUTHOR @@ -6562,11 +6605,11 @@ AUTHOR REVISION - Last updated: 13 June 2007 - Copyright (c) 1997-2007 University of Cambridge. + Last updated: 17 November 2010 + Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREPERFORM(3) PCREPERFORM(3) @@ -6733,8 +6776,8 @@ REVISION Last updated: 16 May 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCREPOSIX(3) PCREPOSIX(3) @@ -6996,8 +7039,8 @@ REVISION Last updated: 16 May 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + PCRECPP(3) PCRECPP(3) @@ -7337,8 +7380,8 @@ REVISION Last updated: 17 March 2009 ------------------------------------------------------------------------------ - - + + PCRESAMPLE(3) PCRESAMPLE(3) @@ -7397,7 +7440,7 @@ PCRE SAMPLE PROGRAM expressions and the PCRE library. The pcredemo program is provided as a simple coding example. - When you try to run pcredemo when PCRE is not installed in the standard + If you try to run pcredemo when PCRE is not installed in the standard library directory, you may get an error like this on some operating systems (e.g. Solaris): @@ -7421,7 +7464,7 @@ AUTHOR REVISION - Last updated: 26 May 2010 + Last updated: 17 November 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ PCRESTACK(3) PCRESTACK(3) @@ -7573,5 +7616,5 @@ REVISION Last updated: 03 January 2010 Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ - - + + diff --git a/doc/pcreapi.3 b/doc/pcreapi.3 index 702005b..61bd101 100644 --- a/doc/pcreapi.3 +++ b/doc/pcreapi.3 @@ -1499,8 +1499,8 @@ a pre-scan of the subject that takes place before the pattern is run. The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, possibly causing performance to suffer, but ensuring that in cases where the result is "no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK) -are considered at every possible starting position in the subject string. If -PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching +are considered at every possible starting position in the subject string. If +PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching time. .P Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation. diff --git a/doc/pcrecompat.3 b/doc/pcrecompat.3 index 3ebaac0..0e502b9 100644 --- a/doc/pcrecompat.3 +++ b/doc/pcrecompat.3 @@ -103,7 +103,7 @@ would not be possible to distinguish which parentheses matched, because both names map to capturing subpattern number 1. To avoid this confusing situation, an error is given at compile time. .P -12. Perl recognizes comments in some places that PCRE doesn't, for example, +12. Perl recognizes comments in some places that PCRE doesn't, for example, between the ( and ? at the start of a subpattern. .P 13. PCRE provides some extensions to the Perl regular expression facilities. diff --git a/doc/pcredemo.3 b/doc/pcredemo.3 index c419086..efc9041 100644 --- a/doc/pcredemo.3 +++ b/doc/pcredemo.3 @@ -277,7 +277,7 @@ if (!find_all) /* Check for -g */ } /* Before running the loop, check for UTF-8 and whether CRLF is a valid newline -sequence. First, find the options with which the regex was compiled; extract +sequence. First, find the options with which the regex was compiled; extract the UTF-8 state, and mask off all but the newline options. */ (void)pcre_fullinfo(re, NULL, PCRE_INFO_OPTIONS, &option_bits); @@ -303,7 +303,7 @@ if (option_bits == 0) /* See if CRLF is a valid newline sequence. */ -crlf_is_newline = +crlf_is_newline = option_bits == PCRE_NEWLINE_ANY || option_bits == PCRE_NEWLINE_CRLF || option_bits == PCRE_NEWLINE_ANYCRLF; @@ -362,11 +362,11 @@ for (;;) else if (utf8) /* Otherwise, ensure we */ { /* advance a whole UTF-8 */ while (ovector[1] < subject_length) /* character. */ - { + { if ((subject[ovector[1]] & 0xc0) != 0x80) break; ovector[1] += 1; } - } + } continue; /* Go round the loop again */ } diff --git a/doc/pcregrep.1 b/doc/pcregrep.1 index ad6e992..430e54a 100644 --- a/doc/pcregrep.1 +++ b/doc/pcregrep.1 @@ -307,12 +307,12 @@ the value in the \fBLC_ALL\fP or \fBLC_CTYPE\fP environment variables. If no locale is specified, the PCRE library's default (usually the "C" locale) is used. There is no short form for this option. .TP -\fB--match-limit\fP=\fInumber\fP +\fB--match-limit\fP=\fInumber\fP Processing some regular expression patterns can require a very large amount of memory, leading in some cases to a program crash if not enough is available. -Other patterns may take a very long time to search for all possible matching +Other patterns may take a very long time to search for all possible matching strings. The \fBpcre_exec()\fP function that is called by \fBpcregrep\fP to do -the matching has two parameters that can limit the resources that it uses. +the matching has two parameters that can limit the resources that it uses. .sp The \fB--match-limit\fP option provides a means of limiting resource usage when processing patterns that are not going to match, but which have a very @@ -330,7 +330,7 @@ that can be used. The recursion depth is a smaller number than the total number of calls, because not all calls to \fBmatch()\fP are recursive. This limit is of use only if it is set smaller than \fB--match-limit\fP. .sp -There are no short forms for these options. The default settings are specified +There are no short forms for these options. The default settings are specified when the PCRE library is compiled, with the default default being 10 million. .TP \fB-M\fP, \fB--multiline\fP @@ -383,13 +383,13 @@ which case they are shown on an otherwise empty line. This option is mutually exclusive with \fB--file-offsets\fP and \fB--line-offsets\fP. .TP \fB-o\fP\fInumber\fP, \fB--only-matching\fP=\fInumber\fP -Show only the part of the line that matched the capturing parentheses of the +Show only the part of the line that matched the capturing parentheses of the given number. Up to 32 capturing parentheses are supported. Because these options can be given without an argument (see above), if an argument is present, it must be given in the same shell item, for example, -o3 or ---only-matching=2. The comments given for the non-argument case above also -apply to this case. If the specified capturing parentheses do not exist in the -pattern, or were not set in the match, nothing is output unless the file name +--only-matching=2. The comments given for the non-argument case above also +apply to this case. If the specified capturing parentheses do not exist in the +pattern, or were not set in the match, nothing is output unless the file name or line number are being printed. .TP \fB-q\fP, \fB--quiet\fP @@ -484,8 +484,8 @@ exception) in the next command line item. For example: -f/some/file -f /some/file .sp -The exception is the \fB-o\fP option, which may appear with or without data. -Because of this, if data is present, it must follow immediately in the same +The exception is the \fB-o\fP option, which may appear with or without data. +Because of this, if data is present, it must follow immediately in the same item, for example -o3. .P If a long form option is used, the data may appear in the same command line @@ -503,7 +503,7 @@ specially unless it is at the start of an item. The exceptions to the above are the \fB--colour\fP (or \fB--color\fP) and \fB--only-matching\fP options, for which the data is optional. If one of these options does have data, it must be given in the first form, using an equals -character. Otherwise \fBpcregrep\P will assume that it has no data. +character. Otherwise \fBpcregrep\fP will assume that it has no data. . . .SH "MATCHING ERRORS" @@ -519,7 +519,7 @@ there are more than 20 such errors, \fBpcregrep\fP gives up. .P The \fB--match-limit\fP option of \fBpcregrep\fP can be used to set the overall resource limit; there is a second option called \fB--recursion-limit\fP that -sets a limit on the amount of memory (usually stack) that is used (see the +sets a limit on the amount of memory (usually stack) that is used (see the discussion of these options above). . . diff --git a/doc/pcregrep.txt b/doc/pcregrep.txt index 6addf82..6a50e60 100644 --- a/doc/pcregrep.txt +++ b/doc/pcregrep.txt @@ -560,7 +560,7 @@ OPTIONS WITH DATA The exceptions to the above are the --colour (or --color) and --only- matching options, for which the data is optional. If one of these options does have data, it must be given in the first form, using an - equals character. Otherwise pcregrepP will assume that it has no data. + equals character. Otherwise pcregrep will assume that it has no data. MATCHING ERRORS diff --git a/doc/pcrematching.3 b/doc/pcrematching.3 index f8ad9c6..4c88322 100644 --- a/doc/pcrematching.3 +++ b/doc/pcrematching.3 @@ -83,7 +83,7 @@ The scan continues until either the end of the subject is reached, or there are no more unterminated paths. At this point, terminated paths represent the different matching possibilities (if there are none, the match has failed). Thus, if there is more than one possible match, this algorithm finds all of -them, and in particular, it finds the longest. The matches are returned in +them, and in particular, it finds the longest. The matches are returned in decreasing order of length. There is an option to stop the algorithm after the first match (which is necessarily the shortest) is found. .P @@ -158,7 +158,7 @@ more complicated. The .\" HREF \fBpcrepartial\fP .\" -documentation gives details of partial matching and discusses multi-segment +documentation gives details of partial matching and discusses multi-segment matching. . . diff --git a/doc/pcrepartial.3 b/doc/pcrepartial.3 index 6264246..d1aac8e 100644 --- a/doc/pcrepartial.3 +++ b/doc/pcrepartial.3 @@ -49,16 +49,16 @@ subject string is reached successfully, but matching cannot continue because more characters are needed. However, at least one character in the subject must have been inspected. This character need not form part of the final matched string; lookbehind assertions and the \eK escape sequence provide ways of -inspecting characters before the start of a matched substring. The requirement -for inspecting at least one character exists because an empty string can always -be matched; without such a restriction there would always be a partial match of +inspecting characters before the start of a matched substring. The requirement +for inspecting at least one character exists because an empty string can always +be matched; without such a restriction there would always be a partial match of an empty string at the end of the subject. .P If there are at least two slots in the offsets vector when \fBpcre_exec()\fP returns with a partial match, the first slot is set to the offset of the earliest character that was inspected when the partial match was found. For convenience, the second offset points to the end of the subject so that a -substring can easily be identified. +substring can easily be identified. .P For the majority of patterns, the first offset identifies the start of the partially matched string. However, for patterns that contain lookbehind @@ -73,7 +73,7 @@ string is "xyzabc12", the offsets after a partial match are for the substring with extra characters added to the subject. .P What happens when a partial match is identified depends on which of the two -partial matching options are set. +partial matching options are set. . . .SS "PCRE_PARTIAL_SOFT with pcre_exec()" @@ -84,10 +84,10 @@ the partial match is remembered, but matching continues as normal, and other alternatives in the pattern are tried. If no complete match can be found, \fBpcre_exec()\fP returns PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. .P -This option is "soft" because it prefers a complete match over a partial match. -All the various matching items in a pattern behave as if the subject string is +This option is "soft" because it prefers a complete match over a partial match. +All the various matching items in a pattern behave as if the subject string is potentially complete. For example, \ez, \eZ, and $ match at the end of the -subject, as normal, and for \eb and \eB the end of the subject is treated as a +subject, as normal, and for \eb and \eB the end of the subject is treated as a non-alphanumeric. .P If there is more than one partial match, the first one that was found provides @@ -108,16 +108,16 @@ matches the second alternative.) .sp If PCRE_PARTIAL_HARD is set for \fBpcre_exec()\fP, it returns PCRE_ERROR_PARTIAL as soon as a partial match is found, without continuing to -search for possible complete matches. This option is "hard" because it prefers +search for possible complete matches. This option is "hard" because it prefers an earlier partial match over a later complete match. For this reason, the assumption is made that the end of the supplied subject string may not be the true end of the available data, and so, if \ez, \eZ, \eb, \eB, or $ are encountered at the end of the subject, the result is PCRE_ERROR_PARTIAL. .P -Setting PCRE_PARTIAL_HARD also affects the way \fBpcre_exec()\fP checks UTF-8 +Setting PCRE_PARTIAL_HARD also affects the way \fBpcre_exec()\fP checks UTF-8 subject strings for validity. Normally, an invalid UTF-8 sequence causes the -error PCRE_ERROR_BADUTF8. However, in the special case of a truncated UTF-8 -character at the end of the subject, PCRE_ERROR_SHORTUTF8 is returned when +error PCRE_ERROR_BADUTF8. However, in the special case of a truncated UTF-8 +character at the end of the subject, PCRE_ERROR_SHORTUTF8 is returned when PCRE_PARTIAL_HARD is set. . . @@ -280,8 +280,8 @@ From release 8.00, \fBpcre_exec()\fP can also be used to do multi-segment matching. Unlike \fBpcre_dfa_exec()\fP, it is not possible to restart the previous match with a new segment of data. Instead, new data must be added to the previous subject string, and the entire match re-run, starting from the -point where the partial match occurred. Earlier data can be discarded. It is -best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the +point where the partial match occurred. Earlier data can be discarded. It is +best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the end of a segment as the end of the subject when matching \ez, \eZ, \eb, \eB, and $. Consider an unanchored pattern that matches dates: .sp @@ -309,8 +309,8 @@ whichever matching function is used. .P 1. If the pattern contains a test for the beginning of a line, you need to pass the PCRE_NOTBOL option when the subject string for any call does start at the -beginning of a line. There is also a PCRE_NOTEOL option, but in practice when -doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which +beginning of a line. There is also a PCRE_NOTEOL option, but in practice when +doing multi-segment matching you should be using PCRE_PARTIAL_HARD, which includes the effect of PCRE_NOTEOL. .P 2. Lookbehind assertions at the start of a pattern are catered for in the diff --git a/doc/pcrepattern.3 b/doc/pcrepattern.3 index de839e1..737a95c 100644 --- a/doc/pcrepattern.3 +++ b/doc/pcrepattern.3 @@ -53,8 +53,8 @@ instead of recognizing only characters with codes less than 128 via a lookup table. .P If a pattern starts with (*NO_START_OPT), it has the same effect as setting the -PCRE_NO_START_OPTIMIZE option either at compile or matching time. There are -also some more of these special sequences that are concerned with the handling +PCRE_NO_START_OPTIMIZE option either at compile or matching time. There are +also some more of these special sequences that are concerned with the handling of newlines; they are described below. .P The remainder of this document discusses the patterns that are supported by @@ -189,7 +189,7 @@ The following sections describe the use of each of the metacharacters. The backslash character has several uses. Firstly, if it is followed by a character that is not a number or a letter, it takes away any special meaning that character may have. This use of backslash as an escape character applies -both inside and outside character classes. +both inside and outside character classes. .P For example, if you want to match a * character, you write \e* in the pattern. This escaping action applies whether or not the following character would @@ -198,7 +198,7 @@ non-alphanumeric with backslash to specify that it stands for itself. In particular, if you want to match a backslash, you write \e\e. .P In UTF-8 mode, only ASCII numbers and letters have any special meaning after a -backslash. All other characters (in particular, those whose codepoints are +backslash. All other characters (in particular, those whose codepoints are greater than 127) are treated as literals. .P If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the @@ -247,10 +247,10 @@ one of the following escape sequences than the binary character it represents: The precise effect of \ecx is as follows: if x is a lower case letter, it is converted to upper case. Then bit 6 of the character (hex 40) is inverted. Thus \ecz becomes hex 1A (z is 7A), but \ec{ becomes hex 3B ({ is 7B), while -\ec; becomes hex 7B (; is 3B). If the byte following \ec has a value greater -than 127, a compile-time error occurs. This locks out non-ASCII characters in -both byte mode and UTF-8 mode. (When PCRE is compiled in EBCDIC mode, all byte -values are valid. A lower case letter is converted to upper case, and then the +\ec; becomes hex 7B (; is 3B). If the byte following \ec has a value greater +than 127, a compile-time error occurs. This locks out non-ASCII characters in +both byte mode and UTF-8 mode. (When PCRE is compiled in EBCDIC mode, all byte +values are valid. A lower case letter is converted to upper case, and then the 0xc0 bits are flipped.) .P After \ex, from zero to two hexadecimal digits are read (letters can be in @@ -437,7 +437,7 @@ any Unicode letter, and underscore. Note also that PCRE_UCP affects \eb, and \eB because they are defined in terms of \ew and \eW. Matching these sequences is noticeably slower when PCRE_UCP is set. .P -The sequences \eh, \eH, \ev, and \eV are features that were added to Perl at +The sequences \eh, \eH, \ev, and \eV are features that were added to Perl at release 5.10. In contrast to the other sequences, which match only ASCII characters by default, these always match certain high-valued codepoints in UTF-8 mode, whether or not PCRE_UCP is set. The horizontal space characters @@ -960,7 +960,7 @@ The handling of dot is entirely independent of the handling of circumflex and dollar, the only relationship being that they both involve newlines. Dot has no special meaning in a character class. .P -The escape sequence \eN behaves like a dot, except that it is not affected by +The escape sequence \eN behaves like a dot, except that it is not affected by the PCRE_DOTALL option. In other words, it matches any character except one that signifies the end of a line. . @@ -1076,8 +1076,8 @@ default, but cause an error if the PCRE_EXTRA option is set. A circumflex can conveniently be used with the upper case character types to specify a more restricted set of characters than the matching lower case type. For example, the class [^\eW_] matches any letter or digit, but not underscore, -whereas [\ew] includes underscore. A positive character class should be read as -"something OR something OR ..." and a negative class as "NOT something AND NOT +whereas [\ew] includes underscore. A positive character class should be read as +"something OR something OR ..." and a negative class as "NOT something AND NOT something AND NOT ...". .P The only metacharacters that are recognized in character classes are backslash, @@ -1997,9 +1997,9 @@ already been matched. The two possible forms of conditional subpattern are: If the condition is satisfied, the yes-pattern is used; otherwise the no-pattern (if present) is used. If there are more than two alternatives in the subpattern, a compile-time error occurs. Each of the two alternatives may -itself contain nested subpatterns of any form, including conditional +itself contain nested subpatterns of any form, including conditional subpatterns; the restriction to two alternatives applies only at the level of -the condition. This pattern fragment is an example where the alternatives are +the condition. This pattern fragment is an example where the alternatives are complex: .sp (?(1) (A|B|C) | (D | (?(2)E|F) | E) ) @@ -2024,7 +2024,7 @@ the condition is true if any of them have matched. An alternative notation is to precede the digits with a plus or minus sign. In this case, the subpattern number is relative rather than absolute. The most recently opened parentheses can be referenced by (?(-1), the next most recent by (?(-2), and so on. Inside -loops it can also make sense to refer to subsequent groups. The next +loops it can also make sense to refer to subsequent groups. The next parentheses to be opened can be referenced as (?(+1), and so on. (The value zero in any of these forms is not used; it provokes a compile-time error.) .P @@ -2144,7 +2144,7 @@ dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. .SH COMMENTS .rs .sp -There are two ways of including comments in patterns that are processed by +There are two ways of including comments in patterns that are processed by PCRE. In both cases, the start of the comment must not be in a character class, nor in the middle of any other sequence of related characters such as (?: or a subpattern name or number. The characters that make up a comment play no part @@ -2168,7 +2168,7 @@ default newline convention is in force: .sp abc #comment \en still comment .sp -On encountering the # character, \fBpcre_compile()\fP skips along, looking for +On encountering the # character, \fBpcre_compile()\fP skips along, looking for a newline in the pattern. The sequence \en is still literal at this stage, so it does not terminate the comment. Only an actual character with the code value 0x0a (the default newline) does so. @@ -2694,7 +2694,7 @@ matching name is found, normal "bumpalong" of one character happens (the .sp (*THEN) or (*THEN:NAME) .sp -This verb causes a skip to the next alternation in the innermost enclosing +This verb causes a skip to the next alternation in the innermost enclosing group if the rest of the pattern does not match. That is, it cancels pending backtracking, but only within the current alternation. Its name comes from the observation that it can be used for a pattern-based if-then-else block: @@ -2709,7 +2709,7 @@ overall match fails. If (*THEN) is not directly inside an alternation, it acts like (*PRUNE). . .P -The above verbs provide four different "strengths" of control when subsequent +The above verbs provide four different "strengths" of control when subsequent matching fails. (*THEN) is the weakest, carrying on the match at the next alternation. (*PRUNE) comes next, failing the match at the current starting position, but allowing an advance to the next character (for an unanchored diff --git a/doc/pcretest.1 b/doc/pcretest.1 index 6c37d3d..c5d4fd6 100644 --- a/doc/pcretest.1 +++ b/doc/pcretest.1 @@ -179,7 +179,7 @@ options that do not correspond to anything in Perl: \fB/U\fP PCRE_UNGREEDY \fB/W\fP PCRE_UCP \fB/X\fP PCRE_EXTRA - \fB/Y\fP PCRE_NO_START_OPTIMIZE + \fB/Y\fP PCRE_NO_START_OPTIMIZE \fB/<JS>\fP PCRE_JAVASCRIPT_COMPAT \fB/<cr>\fP PCRE_NEWLINE_CR \fB/<lf>\fP PCRE_NEWLINE_LF @@ -222,9 +222,9 @@ empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set in order to search for another, non-empty, match at the same point. If this second match fails, the start offset is advanced, and the normal match is retried. This imitates the way Perl handles such cases when -using the \fB/g\fP modifier or the \fBsplit()\fP function. Normally, the start -offset is advanced by one character, but if the newline convention recognizes -CRLF as a newline, and the current character is CR followed by LF, an advance +using the \fB/g\fP modifier or the \fBsplit()\fP function. Normally, the start +offset is advanced by one character, but if the newline convention recognizes +CRLF as a newline, and the current character is CR followed by LF, an advance of two is used. . . @@ -345,7 +345,7 @@ recognized: \et tab (\ex09) \ev vertical tab (\ex0b) \ennn octal character (up to 3 octal digits) - always a byte unless > 255 in UTF-8 mode + always a byte unless > 255 in UTF-8 mode \exhh hexadecimal byte (up to 2 hex digits) .\" JOIN \ex{hh...} hexadecimal character, any number of digits @@ -418,7 +418,7 @@ recognized: \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP .\" JOIN \e>dd start the match at offset dd (optional "-"; then - any number of digits); this sets the \fIstartoffset\fP + any number of digits); this sets the \fIstartoffset\fP argument for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP .\" JOIN \e<cr> pass the PCRE_NEWLINE_CR option to \fBpcre_exec()\fP @@ -436,11 +436,11 @@ recognized: \e<any> pass the PCRE_NEWLINE_ANY option to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP .sp -Note that \exhh always specifies one byte, even in UTF-8 mode; this makes it -possible to construct invalid UTF-8 sequences for testing purposes. On the +Note that \exhh always specifies one byte, even in UTF-8 mode; this makes it +possible to construct invalid UTF-8 sequences for testing purposes. On the other hand, \ex{hh} is interpreted as a UTF-8 character in UTF-8 mode, -generating more than one byte if the value is greater than 127. When not in -UTF-8 mode, it generates one byte for values less than 256, and causes an error +generating more than one byte if the value is greater than 127. When not in +UTF-8 mode, it generates one byte for values less than 256, and causes an error for greater values. .P The escapes that specify line ending sequences are literal strings, exactly as diff --git a/doc/pcretest.txt b/doc/pcretest.txt index 8b8b12e..7f67d6f 100644 --- a/doc/pcretest.txt +++ b/doc/pcretest.txt @@ -168,6 +168,7 @@ PATTERN MODIFIERS /U PCRE_UNGREEDY /W PCRE_UCP /X PCRE_EXTRA + /Y PCRE_NO_START_OPTIMIZE /<JS> PCRE_JAVASCRIPT_COMPAT /<cr> PCRE_NEWLINE_CR /<lf> PCRE_NEWLINE_LF @@ -708,5 +709,5 @@ AUTHOR REVISION - Last updated: 07 November 2010 + Last updated: 21 November 2010 Copyright (c) 1997-2010 University of Cambridge. diff --git a/maint/README b/maint/README index 06b2883..40a0a3a 100644 --- a/maint/README +++ b/maint/README @@ -109,28 +109,14 @@ distribution for a new release. . Compile and test with many different config options, and combinations of options. The maint/ManyConfigTests script now encapsulates this testing. -. Run perltest.pl on the test data for tests 1, 4, 6, and 11. The first two can - be run with Perl 5.8 or >= 5.10; the last two require Perl >= 5.10. The - output should match the PCRE test output, apart from the version - identification at the start of each test. The other tests are not - Perl-compatible (they use various PCRE-specific features or options). +. Run perltest.pl on the test data for tests 1, 4, 6, and 11. The output should + match the PCRE test output, apart from the version identification at the + start of each test. The other tests are not Perl-compatible (they use various + PCRE-specific features or options). . Test with valgrind by running "RunTest valgrind". There is also "RunGrepTest valgrind", though that takes quite a long time. -. It may also useful to test with Electric Fence, though the fact that it - grumbles for missing free() calls can be a nuisance. (A missing free() in - pcretest is hardly a big problem.) To build with EF, use: - - LIBS='/usr/lib/libefence.a -lpthread' with ./configure. - - Then all normal runs use it to check for buffer overflow. Also run everything - with: - - EF_PROTECT_BELOW=1 <whatever> - - because there have been problems with lookbehinds that looked too far. - . Test with the emulated memmove() function by undefining HAVE_MEMMOVE and HAVE_BCOPY in config.h. You may see a number of "pcre_memmove defined but not used" warnings for the modules in which there is no call to memmove(). These @@ -329,4 +315,4 @@ others are relatively new. Philip Hazel Email local part: ph10 Email domain: cam.ac.uk -Last updated: 03 June 2010 +Last updated: 24 November 2010 @@ -96,7 +96,7 @@ extern "C" { #endif /* Options. Some are compile-time only, some are run-time only, and some are -both, so we keep them all distinct. However, almost all the bits in the options +both, so we keep them all distinct. However, almost all the bits in the options word are now used. In the long run, we may have to re-use some of the compile-time only bits for runtime options, or vice versa. */ diff --git a/pcre_compile.c b/pcre_compile.c index 14c252c..694ddce 100644 --- a/pcre_compile.c +++ b/pcre_compile.c @@ -408,7 +408,7 @@ static const char error_texts[] = "different names for subpatterns of the same number are not allowed\0" "(*MARK) must have an argument\0" "this version of PCRE is not compiled with PCRE_UCP support\0" - "\\c must be followed by an ASCII character\0" + "\\c must be followed by an ASCII character\0" ; /* Table to identify digits and hex digits. This is used when compiling @@ -857,8 +857,8 @@ else if (c > 127) /* Excludes all non-ASCII in either mode */ { *errorcodeptr = ERR68; - break; - } + break; + } if (c >= CHAR_a && c <= CHAR_z) c -= 32; c ^= 0x40; #else /* EBCDIC coding */ @@ -1113,12 +1113,12 @@ that if (?< or (?' or (?P< is encountered, the name will be correctly terminated because that is checked in the first pass. There is now one call to this function in the first pass, to check for a recursive back reference by name (so that we can make the whole group atomic). In this case, we need check -only up to the current position in the pattern, and that is still OK because -and previous occurrences will have been checked. To make this work, the test -for "end of pattern" is a check against cd->end_pattern in the main loop, +only up to the current position in the pattern, and that is still OK because +and previous occurrences will have been checked. To make this work, the test +for "end of pattern" is a check against cd->end_pattern in the main loop, instead of looking for a binary zero. This means that the special first-pass -call can adjust cd->end_pattern temporarily. (Checks for binary zero while -processing items within the loop are OK, because afterwards the main loop will +call can adjust cd->end_pattern temporarily. (Checks for binary zero while +processing items within the loop are OK, because afterwards the main loop will terminate.) Arguments: @@ -1127,7 +1127,7 @@ Arguments: name name to seek, or NULL if seeking a numbered subpattern lorn name length, or subpattern number if name is NULL xmode TRUE if we are in /x mode - utf8 TRUE if we are in UTF-8 mode + utf8 TRUE if we are in UTF-8 mode count pointer to the current capturing subpattern number (updated) Returns: the number of the named subpattern, or -1 if not found @@ -1220,8 +1220,8 @@ if (ptr[0] == CHAR_LEFT_PARENTHESIS) } /* Past any initial parenthesis handling, scan for parentheses or vertical -bars. Stop if we get to cd->end_pattern. Note that this is important for the -first-pass call when this value is temporarily adjusted to stop at the current +bars. Stop if we get to cd->end_pattern. Note that this is important for the +first-pass call when this value is temporarily adjusted to stop at the current position. So DO NOT change this to a test for binary zero. */ for (; ptr < cd->end_pattern; ptr++) @@ -1298,12 +1298,12 @@ for (; ptr < cd->end_pattern; ptr++) if (xmode && *ptr == CHAR_NUMBER_SIGN) { - ptr++; + ptr++; while (*ptr != 0) { if (IS_NEWLINE(ptr)) { ptr += cd->nllen - 1; break; } ptr++; -#ifdef SUPPORT_UTF8 +#ifdef SUPPORT_UTF8 if (utf8) while ((*ptr & 0xc0) == 0x80) ptr++; #endif } @@ -1361,7 +1361,7 @@ Arguments: name name to seek, or NULL if seeking a numbered subpattern lorn name length, or subpattern number if name is NULL xmode TRUE if we are in /x mode - utf8 TRUE if we are in UTF-8 mode + utf8 TRUE if we are in UTF-8 mode Returns: the number of the found subpattern, or -1 if not found */ @@ -2545,12 +2545,12 @@ if ((options & PCRE_EXTENDED) != 0) while ((cd->ctypes[*ptr] & ctype_space) != 0) ptr++; if (*ptr == CHAR_NUMBER_SIGN) { - ptr++; + ptr++; while (*ptr != 0) { if (IS_NEWLINE(ptr)) { ptr += cd->nllen; break; } ptr++; -#ifdef SUPPORT_UTF8 +#ifdef SUPPORT_UTF8 if (utf8) while ((*ptr & 0xc0) == 0x80) ptr++; #endif } @@ -2589,12 +2589,12 @@ if ((options & PCRE_EXTENDED) != 0) while ((cd->ctypes[*ptr] & ctype_space) != 0) ptr++; if (*ptr == CHAR_NUMBER_SIGN) { - ptr++; + ptr++; while (*ptr != 0) { if (IS_NEWLINE(ptr)) { ptr += cd->nllen; break; } ptr++; -#ifdef SUPPORT_UTF8 +#ifdef SUPPORT_UTF8 if (utf8) while ((*ptr & 0xc0) == 0x80) ptr++; #endif } @@ -3170,12 +3170,12 @@ for (;; ptr++) if ((cd->ctypes[c] & ctype_space) != 0) continue; if (c == CHAR_NUMBER_SIGN) { - ptr++; + ptr++; while (*ptr != 0) { if (IS_NEWLINE(ptr)) { ptr += cd->nllen - 1; break; } ptr++; -#ifdef SUPPORT_UTF8 +#ifdef SUPPORT_UTF8 if (utf8) while ((*ptr & 0xc0) == 0x80) ptr++; #endif } @@ -3553,12 +3553,12 @@ for (;; ptr++) continue; /* Perl 5.004 onwards omits VT from \s, but we must preserve it - if it was previously set by something earlier in the character - class. */ + if it was previously set by something earlier in the character + class. */ case ESC_s: classbits[0] |= cbits[cbit_space]; - classbits[1] |= cbits[cbit_space+1] & ~0x08; + classbits[1] |= cbits[cbit_space+1] & ~0x08; for (c = 2; c < 32; c++) classbits[c] |= cbits[c+cbit_space]; continue; @@ -4875,8 +4875,8 @@ for (;; ptr++) if (*code++ == OP_THEN) { PUT(code, 0, code - bcptr->current_branch - 1); - code += LINK_SIZE; - } + code += LINK_SIZE; + } } else @@ -4890,8 +4890,8 @@ for (;; ptr++) if (*code++ == OP_THEN_ARG) { PUT(code, 0, code - bcptr->current_branch - 1); - code += LINK_SIZE; - } + code += LINK_SIZE; + } *code++ = arglen; memcpy(code, arg, arglen); code += arglen; @@ -5395,8 +5395,8 @@ for (;; ptr++) if (lengthptr != NULL) { - const uschar *temp; - + const uschar *temp; + if (namelen == 0) { *errorcodeptr = ERR62; @@ -5412,21 +5412,21 @@ for (;; ptr++) *errorcodeptr = ERR48; goto FAILED; } - + /* The name table does not exist in the first pass, so we cannot - do a simple search as in the code below. Instead, we have to scan the + do a simple search as in the code below. Instead, we have to scan the pattern to find the number. It is important that we scan it only as - far as we have got because the syntax of named subpatterns has not - been checked for the rest of the pattern, and find_parens() assumes - correct syntax. In any case, it's a waste of resources to scan - further. We stop the scan at the current point by temporarily + far as we have got because the syntax of named subpatterns has not + been checked for the rest of the pattern, and find_parens() assumes + correct syntax. In any case, it's a waste of resources to scan + further. We stop the scan at the current point by temporarily adjusting the value of cd->endpattern. */ - + temp = cd->end_pattern; cd->end_pattern = ptr; - recno = find_parens(cd, name, namelen, + recno = find_parens(cd, name, namelen, (options & PCRE_EXTENDED) != 0, utf8); - cd->end_pattern = temp; + cd->end_pattern = temp; if (recno < 0) recno = 0; /* Forward ref; set dummy number */ } diff --git a/pcre_dfa_exec.c b/pcre_dfa_exec.c index 5c0bcb3..4258c87 100644 --- a/pcre_dfa_exec.c +++ b/pcre_dfa_exec.c @@ -831,8 +831,8 @@ for (;;) /*-----------------------------------------------------------------*/ case OP_EOD: - if (ptr >= end_subject) - { + if (ptr >= end_subject) + { if ((md->moptions & PCRE_PARTIAL_HARD) != 0) could_continue = TRUE; else { ADD_ACTIVE(state_offset + 1, 0); } @@ -2963,9 +2963,9 @@ back the character offset. */ #ifdef SUPPORT_UTF8 if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0) { - int tb; + int tb; if ((tb = _pcre_valid_utf8((uschar *)subject, length)) >= 0) - return (tb == length && (options & PCRE_PARTIAL_HARD) != 0)? + return (tb == length && (options & PCRE_PARTIAL_HARD) != 0)? PCRE_ERROR_SHORTUTF8 : PCRE_ERROR_BADUTF8; if (start_offset > 0 && start_offset < length) { @@ -3057,7 +3057,7 @@ for (;;) /* There are some optimizations that avoid running the match if a known starting point is not found. However, there is an option that disables - these, for testing and for ensuring that all callouts do actually occur. + these, for testing and for ensuring that all callouts do actually occur. The option can be set in the regex by (*NO_START_OPT) or passed in match-time options. */ diff --git a/pcre_exec.c b/pcre_exec.c index 96cf50b..3abb70b 100644 --- a/pcre_exec.c +++ b/pcre_exec.c @@ -1705,8 +1705,8 @@ for (;;) if (eptr < md->end_subject) { if (!IS_NEWLINE(eptr)) MRRETURN(MATCH_NOMATCH); } else - { - if (md->noteol) MRRETURN(MATCH_NOMATCH); + { + if (md->noteol) MRRETURN(MATCH_NOMATCH); SCHECK_PARTIAL(); } ecode++; @@ -1717,7 +1717,7 @@ for (;;) if (md->noteol) MRRETURN(MATCH_NOMATCH); if (!md->endonly) goto ASSERT_NL_OR_EOS; } - + /* ... else fall through for endonly */ /* End of subject assertion (\z) */ @@ -1735,9 +1735,9 @@ for (;;) if (eptr < md->end_subject && (!IS_NEWLINE(eptr) || eptr != md->end_subject - md->nllen)) MRRETURN(MATCH_NOMATCH); - + /* Either at end of string or \n before end. */ - + SCHECK_PARTIAL(); ecode++; break; @@ -5801,9 +5801,9 @@ back the character offset. */ #ifdef SUPPORT_UTF8 if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0) { - int tb; + int tb; if ((tb = _pcre_valid_utf8((USPTR)subject, length)) >= 0) - return (tb == length && md->partial > 1)? + return (tb == length && md->partial > 1)? PCRE_ERROR_SHORTUTF8 : PCRE_ERROR_BADUTF8; if (start_offset > 0 && start_offset < length) { @@ -5936,7 +5936,7 @@ for(;;) /* There are some optimizations that avoid running the match if a known starting point is not found, or if a known later character is not present. However, there is an option that disables these, for testing and for ensuring - that all callouts do actually occur. The option can be set in the regex by + that all callouts do actually occur. The option can be set in the regex by (*NO_START_OPT) or passed in match-time options. */ if (((options | re->options) & PCRE_NO_START_OPTIMIZE) == 0) diff --git a/pcre_internal.h b/pcre_internal.h index ae67e0f..5f736d1 100644 --- a/pcre_internal.h +++ b/pcre_internal.h @@ -1571,7 +1571,7 @@ enum { ERR0, ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, ERR30, ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, ERR40, ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47, ERR48, ERR49, ERR50, ERR51, ERR52, ERR53, ERR54, ERR55, ERR56, ERR57, ERR58, ERR59, - ERR60, ERR61, ERR62, ERR63, ERR64, ERR65, ERR66, ERR67, ERR68, + ERR60, ERR61, ERR62, ERR63, ERR64, ERR65, ERR66, ERR67, ERR68, ERRCOUNT }; /* The real format of the start of the pcre block; the index of names and the diff --git a/pcre_printint.src b/pcre_printint.src index cbaa3ef..c7d8629 100644 --- a/pcre_printint.src +++ b/pcre_printint.src @@ -540,19 +540,19 @@ for(;;) fprintf(f, " %s %s", OP_names[*code], code + 2); extra += code[1]; break; - + case OP_THEN: - if (print_lengths) - fprintf(f, " %s %d", OP_names[*code], GET(code, 1)); - else - fprintf(f, " %s", OP_names[*code]); - break; + if (print_lengths) + fprintf(f, " %s %d", OP_names[*code], GET(code, 1)); + else + fprintf(f, " %s", OP_names[*code]); + break; case OP_THEN_ARG: - if (print_lengths) + if (print_lengths) fprintf(f, " %s %d %s", OP_names[*code], GET(code, 1), code + 2 + LINK_SIZE); - else + else fprintf(f, " %s %s", OP_names[*code], code + 2 + LINK_SIZE); extra += code[1+LINK_SIZE]; break; diff --git a/pcre_valid_utf8.c b/pcre_valid_utf8.c index 971d92d..78a96e1 100644 --- a/pcre_valid_utf8.c +++ b/pcre_valid_utf8.c @@ -72,19 +72,19 @@ Arguments: Returns: < 0 if the string is a valid UTF-8 string >= 0 otherwise; the value is the offset of the bad byte - + Bad bytes can be: . An isolated byte whose most significant bits are 0x80, because this can only correctly appear within a UTF-8 character; - + . A byte whose most significant bits are 0xc0, but whose other bits indicate - that there are more than 3 additional bytes (i.e. an RFC 2279 starting + that there are more than 3 additional bytes (i.e. an RFC 2279 starting byte, which is no longer valid under RFC 3629); - + . - -The returned offset may also be equal to the length of the string; this means + +The returned offset may also be equal to the length of the string; this means that one or more bytes is missing from the final UTF-8 character. */ @@ -260,7 +260,7 @@ if (!find_all) /* Check for -g */ } /* Before running the loop, check for UTF-8 and whether CRLF is a valid newline -sequence. First, find the options with which the regex was compiled; extract +sequence. First, find the options with which the regex was compiled; extract the UTF-8 state, and mask off all but the newline options. */ (void)pcre_fullinfo(re, NULL, PCRE_INFO_OPTIONS, &option_bits); @@ -286,7 +286,7 @@ if (option_bits == 0) /* See if CRLF is a valid newline sequence. */ -crlf_is_newline = +crlf_is_newline = option_bits == PCRE_NEWLINE_ANY || option_bits == PCRE_NEWLINE_CRLF || option_bits == PCRE_NEWLINE_ANYCRLF; @@ -345,11 +345,11 @@ for (;;) else if (utf8) /* Otherwise, ensure we */ { /* advance a whole UTF-8 */ while (ovector[1] < subject_length) /* character. */ - { + { if ((subject[ovector[1]] & 0xc0) != 0x80) break; ovector[1] += 1; } - } + } continue; /* Go round the loop again */ } @@ -256,7 +256,7 @@ static option_item optionlist[] = { hyphens in the option names. As this was not discovered for several releases, the incorrect versions are left in the table for compatibility. However, the --help function misses out any option that has an underscore in its name. */ - + { OP_STRING, N_EXCLUDE_DIR,&exclude_dir_pattern, "exclude_dir=pattern","exclude matching directories when recursing" }, { OP_STRING, N_INCLUDE_DIR,&include_dir_pattern, "include_dir=pattern","include matching directories when recursing" }, @@ -956,7 +956,7 @@ if (slen > 200) { slen = 200; msg = "text that starts:\n\n"; - } + } for (i = 0; i < pattern_count; i++) { *mrc = pcre_exec(pattern_list[i], hints_list[i], matchptr, (int)length, 0, @@ -1183,7 +1183,7 @@ while (ptr < endptr) else if (quiet) return 0; - /* The --only-matching option prints just the substring that matched, or a + /* The --only-matching option prints just the substring that matched, or a captured portion of it, as long as this string is not empty, and the --file-offsets and --line-offsets options output offsets for the matching substring (they both force --only-matching = 0). None of these options @@ -1202,19 +1202,19 @@ while (ptr < endptr) fprintf(stdout, "%d,%d\n", (int)(matchptr + offsets[0] - ptr), offsets[1] - offsets[0]); else if (file_offsets) - fprintf(stdout, "%d,%d\n", + fprintf(stdout, "%d,%d\n", (int)(filepos + matchptr + offsets[0] - ptr), offsets[1] - offsets[0]); else if (only_matching < mrc) { int plen = offsets[2*only_matching + 1] - offsets[2*only_matching]; if (plen > 0) - { + { if (do_colour) fprintf(stdout, "%c[%sm", 0x1b, colour_string); FWRITE(matchptr + offsets[only_matching*2], 1, plen, stdout); if (do_colour) fprintf(stdout, "%c[00m", 0x1b); fprintf(stdout, "\n"); - } + } } else if (printname != NULL || number) fprintf(stdout, "\n"); matchptr += offsets[1]; @@ -1791,17 +1791,17 @@ for (op = optionlist; op->one_char != 0; op++) { int n; char s[4]; - + /* Two options were accidentally implemented and documented with underscores instead of hyphens in their names, something that was not noticed for quite a - few releases. When fixing this, I left the underscored versions in the list - in case people were using them. However, we don't want to display them in the - help data. There are no other options that contain underscores, and we do not - expect ever to implement such options. Therefore, just omit any option that + few releases. When fixing this, I left the underscored versions in the list + in case people were using them. However, we don't want to display them in the + help data. There are no other options that contain underscores, and we do not + expect ever to implement such options. Therefore, just omit any option that contains an underscore. */ - - if (strchr(op->long_name, '_') != NULL) continue; - + + if (strchr(op->long_name, '_') != NULL) continue; + if (op->one_char > 0) sprintf(s, "-%c,", op->one_char); else strcpy(s, " "); n = 31 - printf(" %s --%s", s, op->long_name); if (n < 1) n = 1; @@ -2181,8 +2181,8 @@ for (i = 1; i < argc; i++) while (*s != 0) { for (op = optionlist; op->one_char != 0; op++) - { - if (*s == op->one_char) break; + { + if (*s == op->one_char) break; } if (op->one_char == 0) { @@ -2190,23 +2190,23 @@ for (i = 1; i < argc; i++) *s, argv[i]); pcregrep_exit(usage(2)); } - + /* Check for a single-character option that has data: OP_OP_NUMBER - is used for one that either has a numerical number or defaults, i.e. the + is used for one that either has a numerical number or defaults, i.e. the data is optional. If a digit follows, there is data; if not, carry on with other single-character options in the same string. */ - + option_data = s+1; if (op->type == OP_OP_NUMBER) - { - if (isdigit((unsigned char)s[1])) break; + { + if (isdigit((unsigned char)s[1])) break; } else /* Check for end or a dataless option */ - { + { if (op->type != OP_NODATA || s[1] == 0) break; - } - - /* Handle a single-character option with no data, then loop for the + } + + /* Handle a single-character option with no data, then loop for the next character in the string. */ pcre_options = handle_option(*s++, pcre_options); @@ -2225,7 +2225,7 @@ for (i = 1; i < argc; i++) /* If the option type is OP_OP_STRING or OP_OP_NUMBER, it's an option that either has a value or defaults to something. It cannot have data in a - separate item. At the moment, the only such options are "colo(u)r", + separate item. At the moment, the only such options are "colo(u)r", "only-matching", and Jeffrey Friedl's special -S debugging option. */ if (*option_data == 0 && @@ -2236,11 +2236,11 @@ for (i = 1; i < argc; i++) case N_COLOUR: colour_option = (char *)"auto"; break; - + case 'o': only_matching = 0; - break; - + break; + #ifdef JFRIEDL_DEBUG case 'S': S_arg = 0; @@ -2540,7 +2540,7 @@ for (j = 0; j < pattern_count; j++) } hint_count++; } - + /* If --match-limit or --recursion-limit was set, put the value(s) into the pcre_extra block for each pattern. */ @@ -2551,24 +2551,24 @@ if (match_limit > 0 || match_limit_recursion > 0) if (hints_list[j] == NULL) { hints_list[j] = malloc(sizeof(pcre_extra)); - if (hints_list[j] == NULL) + if (hints_list[j] == NULL) { fprintf(stderr, "pcregrep: malloc failed\n"); pcregrep_exit(2); } } if (match_limit > 0) - { + { hints_list[j]->flags |= PCRE_EXTRA_MATCH_LIMIT; hints_list[j]->match_limit = match_limit; - } + } if (match_limit_recursion > 0) - { + { hints_list[j]->flags |= PCRE_EXTRA_MATCH_LIMIT_RECURSION; hints_list[j]->match_limit_recursion = match_limit_recursion; - } + } } - } + } /* If there are include or exclude patterns, compile them. */ @@ -2651,10 +2651,10 @@ if (pattern_list != NULL) } if (hints_list != NULL) { - for (i = 0; i < hint_count; i++) + for (i = 0; i < hint_count; i++) { if (hints_list[i] != NULL) free(hints_list[i]); - } + } free(hints_list); } pcregrep_exit(rc); diff --git a/pcreposix.c b/pcreposix.c index 15ae19d..7a57318 100644 --- a/pcreposix.c +++ b/pcreposix.c @@ -151,7 +151,7 @@ static const int eint[] = { REG_BADPAT, /* different names for subpatterns of the same number are not allowed */ REG_BADPAT, /* (*MARK) must have an argument */ REG_INVARG, /* this version of PCRE is not compiled with PCRE_UCP support */ - REG_BADPAT, /* \c must be followed by an ASCII character */ + REG_BADPAT, /* \c must be followed by an ASCII character */ }; /* Table of texts corresponding to POSIX error codes */ @@ -2134,7 +2134,7 @@ while (!done) int getlist = 0; int gmatched = 0; int start_offset = 0; - int start_offset_sign = 1; + int start_offset_sign = 1; int g_notempty = 0; int use_dfa = 0; @@ -2267,13 +2267,13 @@ while (!done) continue; case '>': - if (*p == '-') + if (*p == '-') { start_offset_sign = -1; p++; - } + } while(isdigit(*p)) start_offset = start_offset * 10 + *p++ - '0'; - start_offset *= start_offset_sign; + start_offset *= start_offset_sign; continue; case 'A': /* Option setting */ @@ -2801,11 +2801,11 @@ while (!done) string - that was checked before setting g_notempty. Complication arises in the case when the newline convention is "any", - "crlf", or "anycrlf". If the previous match was at the end of a line - terminated by CRLF, an advance of one character just passes the \r, + "crlf", or "anycrlf". If the previous match was at the end of a line + terminated by CRLF, an advance of one character just passes the \r, whereas we should prefer the longer newline sequence, as does the code in - pcre_exec(). Fudge the offset value to achieve this. We check for a - newline setting in the pattern; if none was set, use pcre_config() to + pcre_exec(). Fudge the offset value to achieve this. We check for a + newline setting in the pattern; if none was set, use pcre_config() to find the default. Otherwise, in the case of UTF-8 matching, the advance must be one @@ -2843,7 +2843,7 @@ while (!done) while (start_offset + onechar < len) { if ((bptr[start_offset+onechar] & 0xc0) != 0x80) break; - onechar++; + onechar++; } } use_offsets[1] = start_offset + onechar; |