diff options
| author | Ilia Alshanetsky <iliaa@php.net> | 2010-01-21 17:49:38 +0000 |
|---|---|---|
| committer | Ilia Alshanetsky <iliaa@php.net> | 2010-01-21 17:49:38 +0000 |
| commit | 50016d9dec9be2a11bbdf195924a6becfb27adbf (patch) | |
| tree | ace84671409618cb4d3d9c40218ded5ae97ba50b /ext/pcre | |
| parent | 6dda7add548cbd61e93784a6916382f7a7e8879d (diff) | |
| download | php-git-50016d9dec9be2a11bbdf195924a6becfb27adbf.tar.gz | |
Upgraded bundled PCRE to version 8.01
Diffstat (limited to 'ext/pcre')
| -rw-r--r-- | ext/pcre/pcrelib/AUTHORS | 4 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/ChangeLog | 128 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/LICENCE | 4 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/NEWS | 7 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/NON-UNIX-USE | 17 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/README | 10 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/config.h | 25 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/doc/pcre.txt | 712 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre.h | 4 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre_compile.c | 104 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre_exec.c | 37 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre_internal.h | 41 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre_printint.src | 5 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcre_study.c | 30 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/pcredemo.c | 20 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testinput1 | 14 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testinput2 | 37 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testinput5 | 3 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testinput7 | 18 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testoutput1 | 21 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testoutput2 | 140 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testoutput5 | 4 | ||||
| -rw-r--r-- | ext/pcre/pcrelib/testdata/testoutput7 | 26 |
23 files changed, 956 insertions, 455 deletions
diff --git a/ext/pcre/pcrelib/AUTHORS b/ext/pcre/pcrelib/AUTHORS index 44ff433d8d..88db849d4d 100644 --- a/ext/pcre/pcrelib/AUTHORS +++ b/ext/pcre/pcrelib/AUTHORS @@ -8,7 +8,7 @@ Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. -Copyright (c) 1997-2009 University of Cambridge +Copyright (c) 1997-2010 University of Cambridge All rights reserved @@ -17,7 +17,7 @@ THE C++ WRAPPER LIBRARY Written by: Google Inc. -Copyright (c) 2007-2008 Google Inc +Copyright (c) 2007-2010 Google Inc All rights reserved #### diff --git a/ext/pcre/pcrelib/ChangeLog b/ext/pcre/pcrelib/ChangeLog index 40600b861d..731974d158 100644 --- a/ext/pcre/pcrelib/ChangeLog +++ b/ext/pcre/pcrelib/ChangeLog @@ -1,6 +1,134 @@ ChangeLog for PCRE ------------------ +Version 8.01 19-Jan-2010 +------------------------ + +1. If a pattern contained a conditional subpattern with only one branch (in + particular, this includes all (*DEFINE) patterns), a call to pcre_study() + computed the wrong minimum data length (which is of course zero for such + subpatterns). This could cause incorrect "no match" results. + +2. For patterns such as (?i)a(?-i)b|c where an option setting at the start of + the pattern is reset in the first branch, pcre_compile() failed with + "internal error: code overflow at offset...". This happened only when + the reset was to the original external option setting. (An optimization + abstracts leading options settings into an external setting, which was the + cause of this.) + +3. A pattern such as ^(?!a(*SKIP)b) where a negative assertion contained one + of the verbs SKIP, PRUNE, or COMMIT, did not work correctly. When the + assertion pattern did not match (meaning that the assertion was true), it + was incorrectly treated as false if the SKIP had been reached during the + matching. This also applied to assertions used as conditions. + +4. If an item that is not supported by pcre_dfa_exec() was encountered in an + assertion subpattern, including such a pattern used as a condition, + unpredictable results occurred, instead of the error return + PCRE_ERROR_DFA_UITEM. + +5. The C++ GlobalReplace function was not working like Perl for the special + situation when an empty string is matched. It now does the fancy magic + stuff that is necessary. + +6. In pcre_internal.h, obsolete includes to setjmp.h and stdarg.h have been + removed. (These were left over from very, very early versions of PCRE.) + +7. Some cosmetic changes to the code to make life easier when compiling it + as part of something else: + + (a) Change DEBUG to PCRE_DEBUG. + + (b) In pcre_compile(), rename the member of the "branch_chain" structure + called "current" as "current_branch", to prevent a collision with the + Linux macro when compiled as a kernel module. + + (c) In pcre_study(), rename the function set_bit() as set_table_bit(), to + prevent a collision with the Linux macro when compiled as a kernel + module. + +8. In pcre_compile() there are some checks for integer overflows that used to + cast potentially large values to (double). This has been changed to that + when building, a check for int64_t is made, and if it is found, it is used + instead, thus avoiding the use of floating point arithmetic. (There is no + other use of FP in PCRE.) If int64_t is not found, the fallback is to + double. + +9. Added two casts to avoid signed/unsigned warnings from VS Studio Express + 2005 (difference between two addresses compared to an unsigned value). + +10. Change the standard AC_CHECK_LIB test for libbz2 in configure.ac to a + custom one, because of the following reported problem in Windows: + + - libbz2 uses the Pascal calling convention (WINAPI) for the functions + under Win32. + - The standard autoconf AC_CHECK_LIB fails to include "bzlib.h", + therefore missing the function definition. + - The compiler thus generates a "C" signature for the test function. + - The linker fails to find the "C" function. + - PCRE fails to configure if asked to do so against libbz2. + +11. When running libtoolize from libtool-2.2.6b as part of autogen.sh, these + messages were output: + + Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and + rerunning libtoolize, to keep the correct libtool macros in-tree. + Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am. + + I have done both of these things. + +12. Although pcre_dfa_exec() does not use nearly as much stack as pcre_exec() + most of the time, it *can* run out if it is given a pattern that contains a + runaway infinite recursion. I updated the discussion in the pcrestack man + page. + +13. Now that we have gone to the x.xx style of version numbers, the minor + version may start with zero. Using 08 or 09 is a bad idea because users + might check the value of PCRE_MINOR in their code, and 08 or 09 may be + interpreted as invalid octal numbers. I've updated the previous comment in + configure.ac, and also added a check that gives an error if 08 or 09 are + used. + +14. Change 8.00/11 was not quite complete: code had been accidentally omitted, + causing partial matching to fail when the end of the subject matched \W + in a UTF-8 pattern where \W was quantified with a minimum of 3. + +15. There were some discrepancies between the declarations in pcre_internal.h + of _pcre_is_newline(), _pcre_was_newline(), and _pcre_valid_utf8() and + their definitions. The declarations used "const uschar *" and the + definitions used USPTR. Even though USPTR is normally defined as "const + unsigned char *" (and uschar is typedeffed as "unsigned char"), it was + reported that: "This difference in casting confuses some C++ compilers, for + example, SunCC recognizes above declarations as different functions and + generates broken code for hbpcre." I have changed the declarations to use + USPTR. + +16. GNU libtool is named differently on some systems. The autogen.sh script now + tries several variants such as glibtoolize (MacOSX) and libtoolize1x + (FreeBSD). + +17. Applied Craig's patch that fixes an HP aCC compile error in pcre 8.00 + (strtoXX undefined when compiling pcrecpp.cc). The patch contains this + comment: "Figure out how to create a longlong from a string: strtoll and + equivalent. It's not enough to call AC_CHECK_FUNCS: hpux has a strtoll, for + instance, but it only takes 2 args instead of 3!" + +18. A subtle bug concerned with back references has been fixed by a change of + specification, with a corresponding code fix. A pattern such as + ^(xa|=?\1a)+$ which contains a back reference inside the group to which it + refers, was giving matches when it shouldn't. For example, xa=xaaa would + match that pattern. Interestingly, Perl (at least up to 5.11.3) has the + same bug. Such groups have to be quantified to be useful, or contained + inside another quantified group. (If there's no repetition, the reference + can never match.) The problem arises because, having left the group and + moved on to the rest of the pattern, a later failure that backtracks into + the group uses the captured value from the final iteration of the group + rather than the correct earlier one. I have fixed this in PCRE by forcing + any group that contains a reference to itself to be an atomic group; that + is, there cannot be any backtracking into it once it has completed. This is + similar to recursive and subroutine calls. + + Version 8.00 19-Oct-09 ---------------------- diff --git a/ext/pcre/pcrelib/LICENCE b/ext/pcre/pcrelib/LICENCE index 73f8cde3d8..0dd22571ad 100644 --- a/ext/pcre/pcrelib/LICENCE +++ b/ext/pcre/pcrelib/LICENCE @@ -22,7 +22,7 @@ Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. -Copyright (c) 1997-2009 University of Cambridge +Copyright (c) 1997-2010 University of Cambridge All rights reserved. @@ -31,7 +31,7 @@ THE C++ WRAPPER FUNCTIONS Contributed by: Google Inc. -Copyright (c) 2007-2008, Google Inc. +Copyright (c) 2007-2010, Google Inc. All rights reserved. diff --git a/ext/pcre/pcrelib/NEWS b/ext/pcre/pcrelib/NEWS index 71cba379eb..6fc8c68078 100644 --- a/ext/pcre/pcrelib/NEWS +++ b/ext/pcre/pcrelib/NEWS @@ -1,6 +1,13 @@ News about PCRE releases ------------------------ +Release 8.01 19-Jan-2010 +------------------------ + +This is a bug-fix release. Several bugs in the code itself and some bugs and +infelicities in the build system have been fixed. + + Release 8.00 19-Oct-09 ---------------------- diff --git a/ext/pcre/pcrelib/NON-UNIX-USE b/ext/pcre/pcrelib/NON-UNIX-USE index aca81bd563..69668cbb4b 100644 --- a/ext/pcre/pcrelib/NON-UNIX-USE +++ b/ext/pcre/pcrelib/NON-UNIX-USE @@ -127,12 +127,16 @@ The following are generic comments about building the PCRE C library "by hand". your system has static and shared libraries, you may have to do this once for each type. - (8) Similarly, compile pcreposix.c (remembering -DHAVE_CONFIG_H if necessary) - and link the result (on its own) as the pcreposix library. + (8) Similarly, if you want to build the POSIX wrapper functions, ensure that + you have the pcreposix.h file and then compile pcreposix.c (remembering + -DHAVE_CONFIG_H if necessary). Link the result (on its own) as the + pcreposix library. (9) Compile the test program pcretest.c (again, don't forget -DHAVE_CONFIG_H). - This needs the functions in the pcre and pcreposix libraries when linking. - It also needs the pcre_printint.src source file, which it #includes. + This needs the functions in the PCRE library when linking. It also needs + the pcreposix wrapper functions unless you compile it with -DNOPOSIX. The + pcretest.c program also needs the pcre_printint.src source file, which it + #includes. (10) Run pcretest on the testinput files in the testdata directory, and check that the output matches the corresponding testoutput files. Note that the @@ -273,8 +277,7 @@ gcc and MinGW's gcc). So, a user can: The test files that are supplied with PCRE are in Unix format, with LF characters as line terminators. It may be necessary to change the line -terminators in order to get some of the tests to work. We hope to improve -things in this area in future. +terminators in order to get some of the tests to work. BUILDING PCRE ON WINDOWS WITH CMAKE @@ -494,5 +497,5 @@ build.log file in the root of the package also. ========================= -Last Updated: 05 October 2009 +Last Updated: 19 January 2010 **** diff --git a/ext/pcre/pcrelib/README b/ext/pcre/pcrelib/README index c6156e25c1..2f3e926946 100644 --- a/ext/pcre/pcrelib/README +++ b/ext/pcre/pcrelib/README @@ -311,10 +311,10 @@ The "configure" script builds the following files for the basic C library: . RunTest is a script for running tests on the basic C library . RunGrepTest is a script for running tests on the pcregrep command -Versions of config.h and pcre.h are distributed in the PCRE tarballs under -the names config.h.generic and pcre.h.generic. These are provided for the -benefit of those who have to built PCRE without the benefit of "configure". If -you use "configure", the .generic versions are not used. +Versions of config.h and pcre.h are distributed in the PCRE tarballs under the +names config.h.generic and pcre.h.generic. These are provided for those who +have to built PCRE without using "configure" or CMake. If you use "configure" +or CMake, the .generic versions are not used. If a C++ compiler is found, the following files are also built: @@ -796,4 +796,4 @@ The distribution should contain the following files: Philip Hazel Email local part: ph10 Email domain: cam.ac.uk -Last updated: 19 October 2009 +Last updated: 19 January 2010 diff --git a/ext/pcre/pcrelib/config.h b/ext/pcre/pcrelib/config.h index 949a0a209a..a72c12ddd7 100644 --- a/ext/pcre/pcrelib/config.h +++ b/ext/pcre/pcrelib/config.h @@ -144,14 +144,17 @@ them both to 0; an emulation function will be used. */ #define HAVE_STRING_H 1 #endif -/* Define to 1 if you have the `strtoll' function. */ -/* #undef HAVE_STRTOLL */ +/* Define to 1 if you have `strtoimax'. */ +/* #undef HAVE_STRTOIMAX */ -/* Define to 1 if you have the `strtoq' function. */ -#ifndef HAVE_STRTOQ -#define HAVE_STRTOQ 1 +/* Define to 1 if you have `strtoll'. */ +#ifndef HAVE_STRTOLL +#define HAVE_STRTOLL 1 #endif +/* Define to 1 if you have `strtoq'. */ +/* #undef HAVE_STRTOQ */ + /* Define to 1 if you have the <sys/stat.h> header file. */ #ifndef HAVE_SYS_STAT_H #define HAVE_SYS_STAT_H 1 @@ -183,7 +186,7 @@ them both to 0; an emulation function will be used. */ #define HAVE_ZLIB_H 1 #endif -/* Define to 1 if you have the `_strtoi64' function. */ +/* Define to 1 if you have `_strtoi64'. */ /* #undef HAVE__STRTOI64 */ /* The value of LINK_SIZE determines the number of bytes used to store links @@ -268,13 +271,13 @@ them both to 0; an emulation function will be used. */ #define PACKAGE_NAME "PCRE" /* Define to the full name and version of this package. */ -#define PACKAGE_STRING "PCRE 8.00" +#define PACKAGE_STRING "PCRE 8.01" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "pcre" /* Define to the version of this package. */ -#define PACKAGE_VERSION "8.00" +#define PACKAGE_VERSION "8.01" /* If you are compiling for a system other than a Unix-like system or @@ -330,11 +333,15 @@ them both to 0; an emulation function will be used. */ /* Version number of package */ #ifndef VERSION -#define VERSION "8.00" +#define VERSION "8.01" #endif /* Define to empty if `const' does not conform to ANSI C. */ /* #undef const */ +/* Define to the type of a signed integer type of width exactly 64 bits if + such a type exists and the standard includes do not define it. */ +/* #undef int64_t */ + /* Define to `unsigned int' if <sys/types.h> does not define. */ /* #undef size_t */ diff --git a/ext/pcre/pcrelib/doc/pcre.txt b/ext/pcre/pcrelib/doc/pcre.txt index 2ccc7bb4b6..0dda0cf579 100644 --- a/ext/pcre/pcrelib/doc/pcre.txt +++ b/ext/pcre/pcrelib/doc/pcre.txt @@ -3246,7 +3246,7 @@ BACKSLASH \n linefeed (hex 0A) \r carriage return (hex 0D) \t tab (hex 09) - \ddd character with octal code ddd, or backreference + \ddd character with octal code ddd, or back reference \xhh character with hex code hh \x{hhh..} character with hex code hhh.. @@ -4051,7 +4051,7 @@ DUPLICATE SUBPATTERN NUMBERS / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x # 1 2 2 3 2 3 4 - A backreference to a numbered subpattern uses the most recent value + A back reference to a numbered subpattern uses the most recent value that is set for that number by any subpattern. The following pattern matches "abcabc" or "defdef": @@ -4085,7 +4085,7 @@ NAMED SUBPATTERNS In PCRE, a subpattern can be named in one of three ways: (?<name>...) or (?'name'...) as in Perl, or (?P<name>...) as in Python. References - to capturing parentheses from other parts of the pattern, such as back- + to capturing parentheses from other parts of the pattern, such as back references, recursion, and conditions, can be made by name as well as by number. @@ -4121,10 +4121,10 @@ NAMED SUBPATTERNS that name that matched. This saves searching to find which numbered subpattern it was. - If you make a backreference to a non-unique named subpattern from else- - where in the pattern, the one that corresponds to the first occurrence - of the name is used. In the absence of duplicate numbers (see the pre- - vious section) this is the one with the lowest number. If you use a + If you make a back reference to a non-unique named subpattern from + elsewhere in the pattern, the one that corresponds to the first occur- + rence of the name is used. In the absence of duplicate numbers (see the + previous section) this is the one with the lowest number. If you use a named reference in a condition test (see the section about conditions below), either to check whether a subpattern has matched, or to check for recursion, all subpatterns with the same name are tested. If the @@ -4270,9 +4270,9 @@ REPETITION mization, or alternatively using ^ to indicate anchoring explicitly. However, there is one situation where the optimization cannot be used. - When .* is inside capturing parentheses that are the subject of a - backreference elsewhere in the pattern, a match at the start may fail - where a later one succeeds. Consider, for example: + When .* is inside capturing parentheses that are the subject of a back + reference elsewhere in the pattern, a match at the start may fail where + a later one succeeds. Consider, for example: (.*)abc\1 @@ -4494,6 +4494,8 @@ BACK REFERENCES PCRE_EXTENDED option is set, this can be whitespace. Otherwise, the \g{ syntax or an empty comment (see "Comments" below) can be used. + Recursive back references + A back reference that occurs inside the parentheses to which it refers fails when the subpattern is first used, so, for example, (a\1) never matches. However, such references can be useful inside repeated sub- @@ -4508,26 +4510,31 @@ BACK REFERENCES to match the back reference. This can be done using alternation, as in the example above, or by a quantifier with a minimum of zero. + Back references of this type cause the group that they reference to be + treated as an atomic group. Once the whole group has been matched, a + subsequent matching failure cannot cause backtracking into the middle + of the group. + ASSERTIONS - An assertion is a test on the characters following or preceding the - current matching point that does not actually consume any characters. - The simple assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are + An assertion is a test on the characters following or preceding the + current matching point that does not actually consume any characters. + The simple assertions coded as \b, \B, \A, \G, \Z, \z, ^ and $ are described above. - More complicated assertions are coded as subpatterns. There are two - kinds: those that look ahead of the current position in the subject - string, and those that look behind it. An assertion subpattern is - matched in the normal way, except that it does not cause the current + More complicated assertions are coded as subpatterns. There are two + kinds: those that look ahead of the current position in the subject + string, and those that look behind it. An assertion subpattern is + matched in the normal way, except that it does not cause the current matching position to be changed. - Assertion subpatterns are not capturing subpatterns, and may not be - repeated, because it makes no sense to assert the same thing several - times. If any kind of assertion contains capturing subpatterns within - it, these are counted for the purposes of numbering the capturing sub- + Assertion subpatterns are not capturing subpatterns, and may not be + repeated, because it makes no sense to assert the same thing several + times. If any kind of assertion contains capturing subpatterns within + it, these are counted for the purposes of numbering the capturing sub- patterns in the whole pattern. However, substring capturing is carried - out only for positive assertions, because it does not make sense for + out only for positive assertions, because it does not make sense for negative assertions. Lookahead assertions @@ -4537,38 +4544,38 @@ ASSERTIONS \w+(?=;) - matches a word followed by a semicolon, but does not include the semi- + matches a word followed by a semicolon, but does not include the semi- colon in the match, and foo(?!bar) - matches any occurrence of "foo" that is not followed by "bar". Note + matches any occurrence of "foo" that is not followed by "bar". Note that the apparently similar pattern (?!foo)bar - does not find an occurrence of "bar" that is preceded by something - other than "foo"; it finds any occurrence of "bar" whatsoever, because + does not find an occurrence of "bar" that is preceded by something + other than "foo"; it finds any occurrence of "bar" whatsoever, because the assertion (?!foo) is always true when the next three characters are "bar". A lookbehind assertion is needed to achieve the other effect. If you want to force a matching failure at some point in a pattern, the - most convenient way to do it is with (?!) because an empty string - always matches, so an assertion that requires there not to be an empty - string must always fail. The Perl 5.10 backtracking control verb + most convenient way to do it is with (?!) because an empty string + always matches, so an assertion that requires there not to be an empty + string must always fail. The Perl 5.10 backtracking control verb (*FAIL) or (*F) is essentially a synonym for (?!). Lookbehind assertions - Lookbehind assertions start with (?<= for positive assertions and (?<! + Lookbehind assertions start with (?<= for positive assertions and (?<! for negative assertions. For example, (?<!foo)bar - does find an occurrence of "bar" that is not preceded by "foo". The - contents of a lookbehind assertion are restricted such that all the + does find an occurrence of "bar" that is not preceded by "foo". The + contents of a lookbehind assertion are restricted such that all the strings it matches must have a fixed length. However, if there are sev- - eral top-level alternatives, they do not all have to have the same + eral top-level alternatives, they do not all have to have the same fixed length. Thus (?<=bullock|donkey) @@ -4577,62 +4584,62 @@ ASSERTIONS (?<!dogs?|cats?) - causes an error at compile time. Branches that match different length - strings are permitted only at the top level of a lookbehind assertion. - This is an extension compared with Perl (5.8 and 5.10), which requires + causes an error at compile time. Branches that match different length + strings are permitted only at the top level of a lookbehind assertion. + This is an extension compared with Perl (5.8 and 5.10), which requires all branches to match the same length of string. An assertion such as (?<=ab(c|de)) - is not permitted, because its single top-level branch can match two + is not permitted, because its single top-level branch can match two different lengths, but it is acceptable to PCRE if rewritten to use two top-level branches: (?<=abc|abde) In some cases, the Perl 5.10 escape sequence \K (see above) can be used - instead of a lookbehind assertion to get round the fixed-length + instead of a lookbehind assertion to get round the fixed-length restriction. - The implementation of lookbehind assertions is, for each alternative, - to temporarily move the current position back by the fixed length and + The implementation of lookbehind assertions is, for each alternative, + to temporarily move the current position back by the fixed length and then try to match. If there are insufficient characters before the cur- rent position, the assertion fails. PCRE does not allow the \C escape (which matches a single byte in UTF-8 - mode) to appear in lookbehind assertions, because it makes it impossi- - ble to calculate the length of the lookbehind. The \X and \R escapes, + mode) to appear in lookbehind assertions, because it makes it impossi- + ble to calculate the length of the lookbehind. The \X and \R escapes, which can match different numbers of bytes, are also not permitted. - "Subroutine" calls (see below) such as (?2) or (?&X) are permitted in - lookbehinds, as long as the subpattern matches a fixed-length string. + "Subroutine" calls (see below) such as (?2) or (?&X) are permitted in + lookbehinds, as long as the subpattern matches a fixed-length string. Recursion, however, is not supported. - Possessive quantifiers can be used in conjunction with lookbehind + Possessive quantifiers can be used in conjunction with lookbehind assertions to specify efficient matching of fixed-length strings at the end of subject strings. Consider a simple pattern such as abcd$ - when applied to a long string that does not match. Because matching + when applied to a long string that does not match. Because matching proceeds from left to right, PCRE will look for each "a" in the subject - and then see if what follows matches the rest of the pattern. If the + and then see if what follows matches the rest of the pattern. If the pattern is specified as ^.*abcd$ - the initial .* matches the entire string at first, but when this fails + the initial .* matches the entire string at first, but when this fails (because there is no following "a"), it backtracks to match all but the - last character, then all but the last two characters, and so on. Once - again the search for "a" covers the entire string, from right to left, + last character, then all but the last two characters, and so on. Once + again the search for "a" covers the entire string, from right to left, so we are no better off. However, if the pattern is written as ^.*+(?<=abcd) - there can be no backtracking for the .*+ item; it can match only the - entire string. The subsequent lookbehind assertion does a single test - on the last four characters. If it fails, the match fails immediately. - For long strings, this approach makes a significant difference to the + there can be no backtracking for the .*+ item; it can match only the + entire string. The subsequent lookbehind assertion does a single test + on the last four characters. If it fails, the match fails immediately. + For long strings, this approach makes a significant difference to the processing time. Using multiple assertions @@ -4641,18 +4648,18 @@ ASSERTIONS (?<=\d{3})(?<!999)foo - matches "foo" preceded by three digits that are not "999". Notice that - each of the assertions is applied independently at the same point in - the subject string. First there is a check that the previous three - characters are all digits, and then there is a check that the same + matches "foo" preceded by three digits that are not "999". Notice that + each of the assertions is applied independently at the same point in + the subject string. First there is a check that the previous three + characters are all digits, and then there is a check that the same three characters are not "999". This pattern does not match "foo" pre- - ceded by six characters, the first of which are digits and the last - three of which are not "999". For example, it doesn't match "123abc- + ceded by six characters, the first of which are digits and the last + three of which are not "999". For example, it doesn't match "123abc- foo". A pattern to do that is (?<=\d{3}...)(?<!999)foo - This time the first assertion looks at the preceding six characters, + This time the first assertion looks at the preceding six characters, checking that the first three are digits, and then the second assertion checks that the preceding three characters are not "999". @@ -4660,96 +4667,96 @@ ASSERTIONS (?<=(?<!foo)bar)baz - matches an occurrence of "baz" that is preceded by "bar" which in turn + matches an occurrence of "baz" that is preceded by "bar" which in turn is not preceded by "foo", while (?<=\d{3}(?!999)...)foo - is another pattern that matches "foo" preceded by three digits and any + is another pattern that matches "foo" preceded by three digits and any three characters that are not "999". CONDITIONAL SUBPATTERNS - It is possible to cause the matching process to obey a subpattern con- - ditionally or to choose between two alternative subpatterns, depending - on the result of an assertion, or whether a specific capturing subpat- - tern has already been matched. The two possible forms of conditional + It is possible to cause the matching process to obey a subpattern con- + ditionally or to choose between two alternative subpatterns, depending + on the result of an assertion, or whether a specific capturing subpat- + tern has already been matched. The two possible forms of conditional subpattern are: (?(condition)yes-pattern) (?(condition)yes-pattern|no-pattern) - If the condition is satisfied, the yes-pattern is used; otherwise the - no-pattern (if present) is used. If there are more than two alterna- + If the condition is satisfied, the yes-pattern is used; otherwise the + no-pattern (if present) is used. If there are more than two alterna- tives in the subpattern, a compile-time error occurs. - There are four kinds of condition: references to subpatterns, refer- + There are four kinds of condition: references to subpatterns, refer- ences to recursion, a pseudo-condition called DEFINE, and assertions. Checking for a used subpattern by number - If the text between the parentheses consists of a sequence of digits, + If the text between the parentheses consists of a sequence of digits, the condition is true if a capturing subpattern of that number has pre- - viously matched. If there is more than one capturing subpattern with - the same number (see the earlier section about duplicate subpattern + viously matched. If there is more than one capturing subpattern with + the same number (see the earlier section about duplicate subpattern numbers), the condition is true if any of them have been set. An alter- - native notation is to precede the digits with a plus or minus sign. In - this case, the subpattern number is relative rather than absolute. The - most recently opened parentheses can be referenced by (?(-1), the next - most recent by (?(-2), and so on. In looping constructs it can also - make sense to refer to subsequent groups with constructs such as + native notation is to precede the digits with a plus or minus sign. In + this case, the subpattern number is relative rather than absolute. The + most recently opened parentheses can be referenced by (?(-1), the next + most recent by (?(-2), and so on. In looping constructs it can also + make sense to refer to subsequent groups with constructs such as (?(+2). - Consider the following pattern, which contains non-significant white + Consider the following pattern, which contains non-significant white space to make it more readable (assume the PCRE_EXTENDED option) and to divide it into three parts for ease of discussion: ( \( )? [^()]+ (?(1) \) ) - The first part matches an optional opening parenthesis, and if that + The first part matches an optional opening parenthesis, and if that character is present, sets it as the first captured substring. The sec- - ond part matches one or more characters that are not parentheses. The + ond part matches one or more characters that are not parentheses. The third part is a conditional subpattern that tests whether the first set of parentheses matched or not. If they did, that is, if subject started with an opening parenthesis, the condition is true, and so the yes-pat- - tern is executed and a closing parenthesis is required. Otherwise, - since no-pattern is not present, the subpattern matches nothing. In - other words, this pattern matches a sequence of non-parentheses, + tern is executed and a closing parenthesis is required. Otherwise, + since no-pattern is not present, the subpattern matches nothing. In + other words, this pattern matches a sequence of non-parentheses, optionally enclosed in parentheses. - If you were embedding this pattern in a larger one, you could use a + If you were embedding this pattern in a larger one, you could use a relative reference: ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... - This makes the fragment independent of the parentheses in the larger + This makes the fragment independent of the parentheses in the larger pattern. Checking for a used subpattern by name - Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a - used subpattern by name. For compatibility with earlier versions of - PCRE, which had this facility before Perl, the syntax (?(name)...) is - also recognized. However, there is a possible ambiguity with this syn- - tax, because subpattern names may consist entirely of digits. PCRE - looks first for a named subpattern; if it cannot find one and the name - consists entirely of digits, PCRE looks for a subpattern of that num- - ber, which must be greater than zero. Using subpattern names that con- + Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a + used subpattern by name. For compatibility with earlier versions of + PCRE, which had this facility before Perl, the syntax (?(name)...) is + also recognized. However, there is a possible ambiguity with this syn- + tax, because subpattern names may consist entirely of digits. PCRE + looks first for a named subpattern; if it cannot find one and the name + consists entirely of digits, PCRE looks for a subpattern of that num- + ber, which must be greater than zero. Using subpattern names that con- sist entirely of digits is not recommended. Rewriting the above example to use a named subpattern gives this: (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) - If the name used in a condition of this kind is a duplicate, the test - is applied to all subpatterns of the same name, and is true if any one + If the name used in a condition of this kind is a duplicate, the test + is applied to all subpatterns of the same name, and is true if any one of them has matched. Checking for pattern recursion If the condition is the string (R), and there is no subpattern with the - name R, the condition is true if a recursive call to the whole pattern + name R, the condition is true if a recursive call to the whole pattern or any subpattern has been made. If digits or a name preceded by amper- sand follow the letter R, for example: @@ -4757,77 +4764,77 @@ CONDITIONAL SUBPATTERNS the condition is true if the most recent recursion is into a subpattern whose number or name is given. This condition does not check the entire - recursion stack. If the name used in a condition of this kind is a + recursion stack. If the name used in a condition of this kind is a duplicate, the test is applied to all subpatterns of the same name, and is true if any one of them is the most recent recursion. - At "top level", all these recursion test conditions are false. The + At "top level", all these recursion test conditions are false. The syntax for recursive patterns is described below. Defining subpatterns for use by reference only - If the condition is the string (DEFINE), and there is no subpattern - with the name DEFINE, the condition is always false. In this case, - there may be only one alternative in the subpattern. It is always - skipped if control reaches this point in the pattern; the idea of - DEFINE is that it can be used to define "subroutines" that can be ref- - erenced from elsewhere. (The use of "subroutines" is described below.) - For example, a pattern to match an IPv4 address could be written like + If the condition is the string (DEFINE), and there is no subpattern + with the name DEFINE, the condition is always false. In this case, + there may be only one alternative in the subpattern. It is always + skipped if control reaches this point in the pattern; the idea of + DEFINE is that it can be used to define "subroutines" that can be ref- + erenced from elsewhere. (The use of "subroutines" is described below.) + For example, a pattern to match an IPv4 address could be written like this (ignore whitespace and line breaks): (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) \b (?&byte) (\.(?&byte)){3} \b - The first part of the pattern is a DEFINE group inside which a another - group named "byte" is defined. This matches an individual component of - an IPv4 address (a number less than 256). When matching takes place, - this part of the pattern is skipped because DEFINE acts like a false - condition. The rest of the pattern uses references to the named group - to match the four dot-separated components of an IPv4 address, insist- + The first part of the pattern is a DEFINE group inside which a another + group named "byte" is defined. This matches an individual component of + an IPv4 address (a number less than 256). When matching takes place, + this part of the pattern is skipped because DEFINE acts like a false + condition. The rest of the pattern uses references to the named group + to match the four dot-separated components of an IPv4 address, insist- ing on a word boundary at each end. Assertion conditions - If the condition is not in any of the above formats, it must be an - assertion. This may be a positive or negative lookahead or lookbehind - assertion. Consider this pattern, again containing non-significant + If the condition is not in any of the above formats, it must be an + assertion. This may be a positive or negative lookahead or lookbehind + assertion. Consider this pattern, again containing non-significant white space, and with the two alternatives on the second line: (?(?=[^a-z]*[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) - The condition is a positive lookahead assertion that matches an - optional sequence of non-letters followed by a letter. In other words, - it tests for the presence of at least one letter in the subject. If a - letter is found, the subject is matched against the first alternative; - otherwise it is matched against the second. This pattern matches - strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are + The condition is a positive lookahead assertion that matches an + optional sequence of non-letters followed by a letter. In other words, + it tests for the presence of at least one letter in the subject. If a + letter is found, the subject is matched against the first alternative; + otherwise it is matched against the second. This pattern matches + strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. COMMENTS - The sequence (?# marks the start of a comment that continues up to the - next closing parenthesis. Nested parentheses are not permitted. The - characters that make up a comment play no part in the pattern matching + The sequence (?# marks the start of a comment that continues up to the + next closing parenthesis. Nested parentheses are not permitted. The + characters that make up a comment play no part in the pattern matching at all. - If the PCRE_EXTENDED option is set, an unescaped # character outside a - character class introduces a comment that continues to immediately + If the PCRE_EXTENDED option is set, an unescaped # character outside a + character class introduces a comment that continues to immediately after the next newline in the pattern. RECURSIVE PATTERNS - Consider the problem of matching a string in parentheses, allowing for - unlimited nested parentheses. Without the use of recursion, the best - that can be done is to use a pattern that matches up to some fixed - depth of nesting. It is not possible to handle an arbitrary nesting + Consider the problem of matching a string in parentheses, allowing for + unlimited nested parentheses. Without the use of recursion, the best + that can be done is to use a pattern that matches up to some fixed + depth of nesting. It is not possible to handle an arbitrary nesting depth. For some time, Perl has provided a facility that allows regular expres- - sions to recurse (amongst other things). It does this by interpolating - Perl code in the expression at run time, and the code can refer to the + sions to recurse (amongst other things). It does this by interpolating + Perl code in the expression at run time, and the code can refer to the expression itself. A Perl pattern using code interpolation to solve the parentheses problem can be created like this: @@ -4837,182 +4844,182 @@ RECURSIVE PATTERNS refers recursively to the pattern in which it appears. Obviously, PCRE cannot support the interpolation of Perl code. Instead, - it supports special syntax for recursion of the entire pattern, and - also for individual subpattern recursion. After its introduction in - PCRE and Python, this kind of recursion was subsequently introduced + it supports special syntax for recursion of the entire pattern, and + also for individual subpattern recursion. After its introduction in + PCRE and Python, this kind of recursion was subsequently introduced into Perl at release 5.10. - A special item that consists of (? followed by a number greater than + A special item that consists of (? followed by a number greater than zero and a closing parenthesis is a recursive call of the subpattern of - the given number, provided that it occurs inside that subpattern. (If - not, it is a "subroutine" call, which is described in the next sec- - tion.) The special item (?R) or (?0) is a recursive call of the entire + the given number, provided that it occurs inside that subpattern. (If + not, it is a "subroutine" call, which is described in the next sec- + tion.) The special item (?R) or (?0) is a recursive call of the entire regular expression. - This PCRE pattern solves the nested parentheses problem (assume the + This PCRE pattern solves the nested parentheses problem (assume the PCRE_EXTENDED option is set so that white space is ignored): \( ( [^()]++ | (?R) )* \) - First it matches an opening parenthesis. Then it matches any number of - substrings which can either be a sequence of non-parentheses, or a - recursive match of the pattern itself (that is, a correctly parenthe- + First it matches an opening parenthesis. Then it matches any number of + substrings which can either be a sequence of non-parentheses, or a + recursive match of the pattern itself (that is, a correctly parenthe- sized substring). Finally there is a closing parenthesis. Note the use of a possessive quantifier to avoid backtracking into sequences of non- parentheses. - If this were part of a larger pattern, you would not want to recurse + If this were part of a larger pattern, you would not want to recurse the entire pattern, so instead you could use this: ( \( ( [^()]++ | (?1) )* \) ) - We have put the pattern into parentheses, and caused the recursion to + We have put the pattern into parentheses, and caused the recursion to refer to them instead of the whole pattern. - In a larger pattern, keeping track of parenthesis numbers can be - tricky. This is made easier by the use of relative references (a Perl - 5.10 feature). Instead of (?1) in the pattern above you can write + In a larger pattern, keeping track of parenthesis numbers can be + tricky. This is made easier by the use of relative references (a Perl + 5.10 feature). Instead of (?1) in the pattern above you can write (?-2) to refer to the second most recently opened parentheses preceding - the recursion. In other words, a negative number counts capturing + the recursion. In other words, a negative number counts capturing parentheses leftwards from the point at which it is encountered. - It is also possible to refer to subsequently opened parentheses, by - writing references such as (?+2). However, these cannot be recursive - because the reference is not inside the parentheses that are refer- - enced. They are always "subroutine" calls, as described in the next + It is also possible to refer to subsequently opened parentheses, by + writing references such as (?+2). However, these cannot be recursive + because the reference is not inside the parentheses that are refer- + enced. They are always "subroutine" calls, as described in the next section. - An alternative approach is to use named parentheses instead. The Perl - syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also + An alternative approach is to use named parentheses instead. The Perl + syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We could rewrite the above example as follows: (?<pn> \( ( [^()]++ | (?&pn) )* \) ) - If there is more than one subpattern with the same name, the earliest + If there is more than one subpattern with the same name, the earliest one is used. - This particular example pattern that we have been looking at contains + This particular example pattern that we have been looking at contains nested unlimited repeats, and so the use of a possessive quantifier for matching strings of non-parentheses is important when applying the pat- - tern to strings that do not match. For example, when this pattern is + tern to strings that do not match. For example, when this pattern is applied to (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() - it yields "no match" quickly. However, if a possessive quantifier is - not used, the match runs for a very long time indeed because there are - so many different ways the + and * repeats can carve up the subject, + it yields "no match" quickly. However, if a possessive quantifier is + not used, the match runs for a very long time indeed because there are + so many different ways the + and * repeats can carve up the subject, and all have to be tested before failure can be reported. - At the end of a match, the values of capturing parentheses are those - from the outermost level. If you want to obtain intermediate values, a - callout function can be used (see below and the pcrecallout documenta- + At the end of a match, the values of capturing parentheses are those + from the outermost level. If you want to obtain intermediate values, a + callout function can be used (see below and the pcrecallout documenta- tion). If the pattern above is matched against (ab(cd)ef) - the value for the inner capturing parentheses (numbered 2) is "ef", - which is the last value taken on at the top level. If a capturing sub- + the value for the inner capturing parentheses (numbered 2) is "ef", + which is the last value taken on at the top level. If a capturing sub- pattern is not matched at the top level, its final value is unset, even if it is (temporarily) set at a deeper level. - If there are more than 15 capturing parentheses in a pattern, PCRE has - to obtain extra memory to store data during a recursion, which it does + If there are more than 15 capturing parentheses in a pattern, PCRE has + to obtain extra memory to store data during a recursion, which it does by using pcre_malloc, freeing it via pcre_free afterwards. If no memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. - Do not confuse the (?R) item with the condition (R), which tests for - recursion. Consider this pattern, which matches text in angle brack- - ets, allowing for arbitrary nesting. Only digits are allowed in nested - brackets (that is, when recursing), whereas any characters are permit- + Do not confuse the (?R) item with the condition (R), which tests for + recursion. Consider this pattern, which matches text in angle brack- + ets, allowing for arbitrary nesting. Only digits are allowed in nested + brackets (that is, when recursing), whereas any characters are permit- ted at the outer level. < (?: (?(R) \d++ | [^<>]*+) | (?R)) * > - In this pattern, (?(R) is the start of a conditional subpattern, with - two different alternatives for the recursive and non-recursive cases. + In this pattern, (?(R) is the start of a conditional subpattern, with + two different alternatives for the recursive and non-recursive cases. The (?R) item is the actual recursive call. Recursion difference from Perl - In PCRE (like Python, but unlike Perl), a recursive subpattern call is + In PCRE (like Python, but unlike Perl), a recursive subpattern call is always treated as an atomic group. That is, once it has matched some of the subject string, it is never re-entered, even if it contains untried - alternatives and there is a subsequent matching failure. This can be - illustrated by the following pattern, which purports to match a palin- - dromic string that contains an odd number of characters (for example, + alternatives and there is a subsequent matching failure. This can be + illustrated by the following pattern, which purports to match a palin- + dromic string that contains an odd number of characters (for example, "a", "aba", "abcba", "abcdcba"): ^(.|(.)(?1)\2)$ The idea is that it either matches a single character, or two identical - characters surrounding a sub-palindrome. In Perl, this pattern works; - in PCRE it does not if the pattern is longer than three characters. + characters surrounding a sub-palindrome. In Perl, this pattern works; + in PCRE it does not if the pattern is longer than three characters. Consider the subject string "abcba": - At the top level, the first character is matched, but as it is not at + At the top level, the first character is matched, but as it is not at the end of the string, the first alternative fails; the second alterna- tive is taken and the recursion kicks in. The recursive call to subpat- - tern 1 successfully matches the next character ("b"). (Note that the + tern 1 successfully matches the next character ("b"). (Note that the beginning and end of line tests are not part of the recursion). - Back at the top level, the next character ("c") is compared with what - subpattern 2 matched, which was "a". This fails. Because the recursion - is treated as an atomic group, there are now no backtracking points, - and so the entire match fails. (Perl is able, at this point, to re- - enter the recursion and try the second alternative.) However, if the + Back at the top level, the next character ("c") is compared with what + subpattern 2 matched, which was "a". This fails. Because the recursion + is treated as an atomic group, there are now no backtracking points, + and so the entire match fails. (Perl is able, at this point, to re- + enter the recursion and try the second alternative.) However, if the pattern is written with the alternatives in the other order, things are different: ^((.)(?1)\2|.)$ - This time, the recursing alternative is tried first, and continues to - recurse until it runs out of characters, at which point the recursion - fails. But this time we do have another alternative to try at the - higher level. That is the big difference: in the previous case the + This time, the recursing alternative is tried first, and continues to + recurse until it runs out of characters, at which point the recursion + fails. But this time we do have another alternative to try at the + higher level. That is the big difference: in the previous case the remaining alternative is at a deeper recursion level, which PCRE cannot use. To change the pattern so that matches all palindromic strings, not just - those with an odd number of characters, it is tempting to change the + those with an odd number of characters, it is tempting to change the pattern to this: ^((.)(?1)\2|.?)$ - Again, this works in Perl, but not in PCRE, and for the same reason. - When a deeper recursion has matched a single character, it cannot be - entered again in order to match an empty string. The solution is to - separate the two cases, and write out the odd and even cases as alter- + Again, this works in Perl, but not in PCRE, and for the same reason. + When a deeper recursion has matched a single character, it cannot be + entered again in order to match an empty string. The solution is to + separate the two cases, and write out the odd and even cases as alter- natives at the higher level: ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) - If you want to match typical palindromic phrases, the pattern has to + If you want to match typical palindromic phrases, the pattern has to ignore all non-word characters, which can be done like this: ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ If run with the PCRE_CASELESS option, this pattern matches phrases such as "A man, a plan, a canal: Panama!" and it works well in both PCRE and - Perl. Note the use of the possessive quantifier *+ to avoid backtrack- - ing into sequences of non-word characters. Without this, PCRE takes a - great deal longer (ten times or more) to match typical phrases, and + Perl. Note the use of the possessive quantifier *+ to avoid backtrack- + ing into sequences of non-word characters. Without this, PCRE takes a + great deal longer (ten times or more) to match typical phrases, and Perl takes so long that you think it has gone into a loop. - WARNING: The palindrome-matching patterns above work only if the sub- - ject string does not start with a palindrome that is shorter than the - entire string. For example, although "abcba" is correctly matched, if - the subject is "ababa", PCRE finds the palindrome "aba" at the start, - then fails at top level because the end of the string does not follow. - Once again, it cannot jump back into the recursion to try other alter- + WARNING: The palindrome-matching patterns above work only if the sub- + ject string does not start with a palindrome that is shorter than the + entire string. For example, although "abcba" is correctly matched, if + the subject is "ababa", PCRE finds the palindrome "aba" at the start, + then fails at top level because the end of the string does not follow. + Once again, it cannot jump back into the recursion to try other alter- natives, so the entire match fails. SUBPATTERNS AS SUBROUTINES If the syntax for a recursive subpattern reference (either by number or - by name) is used outside the parentheses to which it refers, it oper- - ates like a subroutine in a programming language. The "called" subpat- + by name) is used outside the parentheses to which it refers, it oper- + ates like a subroutine in a programming language. The "called" subpat- tern may be defined before or after the reference. A numbered reference can be absolute or relative, as in these examples: @@ -5024,113 +5031,113 @@ SUBPATTERNS AS SUBROUTINES (sens|respons)e and \1ibility - matches "sense and sensibility" and "response and responsibility", but + matches "sense and sensibility" and "response and responsibility", but not "sense and responsibility". If instead the pattern (sens|respons)e and (?1)ibility - is used, it does match "sense and responsibility" as well as the other - two strings. Another example is given in the discussion of DEFINE + is used, it does match "sense and responsibility" as well as the other + two strings. Another example is given in the discussion of DEFINE above. - Like recursive subpatterns, a subroutine call is always treated as an - atomic group. That is, once it has matched some of the subject string, - it is never re-entered, even if it contains untried alternatives and - there is a subsequent matching failure. Any capturing parentheses that - are set during the subroutine call revert to their previous values + Like recursive subpatterns, a subroutine call is always treated as an + atomic group. That is, once it has matched some of the subject string, + it is never re-entered, even if it contains untried alternatives and + there is a subsequent matching failure. Any capturing parentheses that + are set during the subroutine call revert to their previous values afterwards. - When a subpattern is used as a subroutine, processing options such as + When a subpattern is used as a subroutine, processing options such as case-independence are fixed when the subpattern is defined. They cannot be changed for different calls. For example, consider this pattern: (abc)(?i:(?-1)) - It matches "abcabc". It does not match "abcABC" because the change of + It matches "abcabc". It does not match "abcABC" because the change of processing option does not affect the called subpattern. ONIGURUMA SUBROUTINE SYNTAX - For compatibility with Oniguruma, the non-Perl syntax \g followed by a + For compatibility with Oniguruma, the non-Perl syntax \g followed by a name or a number enclosed either in angle brackets or single quotes, is - an alternative syntax for referencing a subpattern as a subroutine, - possibly recursively. Here are two of the examples used above, rewrit- + an alternative syntax for referencing a subpattern as a subroutine, + possibly recursively. Here are two of the examples used above, rewrit- ten using this syntax: (?<pn> \( ( (?>[^()]+) | \g<pn> )* \) ) (sens|respons)e and \g'1'ibility - PCRE supports an extension to Oniguruma: if a number is preceded by a + PCRE supports an extension to Oniguruma: if a number is preceded by a plus or a minus sign it is taken as a relative reference. For example: (abc)(?i:\g<-1>) - Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are not - synonymous. The former is a back reference; the latter is a subroutine + Note that \g{...} (Perl syntax) and \g<...> (Oniguruma syntax) are not + synonymous. The former is a back reference; the latter is a subroutine call. CALLOUTS Perl has a feature whereby using the sequence (?{...}) causes arbitrary - Perl code to be obeyed in the middle of matching a regular expression. + Perl code to be obeyed in the middle of matching a regular expression. This makes it possible, amongst other things, to extract different sub- strings that match the same pair of parentheses when there is a repeti- tion. PCRE provides a similar feature, but of course it cannot obey arbitrary Perl code. The feature is called "callout". The caller of PCRE provides - an external function by putting its entry point in the global variable - pcre_callout. By default, this variable contains NULL, which disables + an external function by putting its entry point in the global variable + pcre_callout. By default, this variable contains NULL, which disables all calling out. - Within a regular expression, (?C) indicates the points at which the - external function is to be called. If you want to identify different - callout points, you can put a number less than 256 after the letter C. - The default value is zero. For example, this pattern has two callout + Within a regular expression, (?C) indicates the points at which the + external function is to be called. If you want to identify different + callout points, you can put a number less than 256 after the letter C. + The default value is zero. For example, this pattern has two callout points: (?C1)abc(?C2)def If the PCRE_AUTO_CALLOUT flag is passed to pcre_compile(), callouts are - automatically installed before each item in the pattern. They are all + automatically installed before each item in the pattern. They are all numbered 255. During matching, when PCRE reaches a callout point (and pcre_callout is - set), the external function is called. It is provided with the number - of the callout, the position in the pattern, and, optionally, one item - of data originally supplied by the caller of pcre_exec(). The callout - function may cause matching to proceed, to backtrack, or to fail alto- + set), the external function is called. It is provided with the number + of the callout, the position in the pattern, and, optionally, one item + of data originally supplied by the caller of pcre_exec(). The callout + function may cause matching to proceed, to backtrack, or to fail alto- gether. A complete description of the interface to the callout function is given in the pcrecallout documentation. BACKTRACKING CONTROL - Perl 5.10 introduced a number of "Special Backtracking Control Verbs", + Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which are described in the Perl documentation as "experimental and sub- - ject to change or removal in a future version of Perl". It goes on to - say: "Their usage in production code should be noted to avoid problems + ject to change or removal in a future version of Perl". It goes on to + say: "Their usage in production code should be noted to avoid problems during upgrades." The same remarks apply to the PCRE features described in this section. - Since these verbs are specifically related to backtracking, most of - them can be used only when the pattern is to be matched using + Since these verbs are specifically related to backtracking, most of + them can be used only when the pattern is to be matched using pcre_exec(), which uses a backtracking algorithm. With the exception of (*FAIL), which behaves like a failing negative assertion, they cause an error if encountered by pcre_dfa_exec(). If any of these verbs are used in an assertion or subroutine subpattern - (including recursive subpatterns), their effect is confined to that - subpattern; it does not extend to the surrounding pattern. Note that - such subpatterns are processed as anchored at the point where they are + (including recursive subpatterns), their effect is confined to that + subpattern; it does not extend to the surrounding pattern. Note that + such subpatterns are processed as anchored at the point where they are tested. - The new verbs make use of what was previously invalid syntax: an open- + The new verbs make use of what was previously invalid syntax: an open- ing parenthesis followed by an asterisk. In Perl, they are generally of the form (*VERB:ARG) but PCRE does not support the use of arguments, so - its general form is just (*VERB). Any number of these verbs may occur + its general form is just (*VERB). Any number of these verbs may occur in a pattern. There are two kinds: Verbs that act immediately @@ -5139,94 +5146,94 @@ BACKTRACKING CONTROL (*ACCEPT) - This verb causes the match to end successfully, skipping the remainder - of the pattern. When inside a recursion, only the innermost pattern is - ended immediately. If (*ACCEPT) is inside capturing parentheses, the - data so far is captured. (This feature was added to PCRE at release + This verb causes the match to end successfully, skipping the remainder + of the pattern. When inside a recursion, only the innermost pattern is + ended immediately. If (*ACCEPT) is inside capturing parentheses, the + data so far is captured. (This feature was added to PCRE at release 8.00.) For example: A((?:A|B(*ACCEPT)|C)D) - This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- + This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- tured by the outer parentheses. (*FAIL) or (*F) - This verb causes the match to fail, forcing backtracking to occur. It - is equivalent to (?!) but easier to read. The Perl documentation notes - that it is probably useful only when combined with (?{}) or (??{}). - Those are, of course, Perl features that are not present in PCRE. The - nearest equivalent is the callout feature, as for example in this pat- + This verb causes the match to fail, forcing backtracking to occur. It + is equivalent to (?!) but easier to read. The Perl documentation notes + that it is probably useful only when combined with (?{}) or (??{}). + Those are, of course, Perl features that are not present in PCRE. The + nearest equivalent is the callout feature, as for example in this pat- tern: a+(?C)(*FAIL) - A match with the string "aaaa" always fails, but the callout is taken + A match with the string "aaaa" always fails, but the callout is taken before each backtrack happens (in this example, 10 times). Verbs that act after backtracking The following verbs do nothing when they are encountered. Matching con- - tinues with what follows, but if there is no subsequent match, a fail- - ure is forced. The verbs differ in exactly what kind of failure + tinues with what follows, but if there is no subsequent match, a fail- + ure is forced. The verbs differ in exactly what kind of failure occurs. (*COMMIT) - This verb causes the whole match to fail outright if the rest of the - pattern does not match. Even if the pattern is unanchored, no further - attempts to find a match by advancing the starting point take place. - Once (*COMMIT) has been passed, pcre_exec() is committed to finding a + This verb causes the whole match to fail outright if the rest of the + pattern does not match. Even if the pattern is unanchored, no further + attempts to find a match by advancing the starting point take place. + Once (*COMMIT) has been passed, pcre_exec() is committed to finding a match at the current starting point, or not at all. For example: a+(*COMMIT)b - This matches "xxaab" but not "aacaab". It can be thought of as a kind + This matches "xxaab" but not "aacaab". It can be thought of as a kind of dynamic anchor, or "I've started, so I must finish." (*PRUNE) - This verb causes the match to fail at the current position if the rest + This verb causes the match to fail at the current position if the rest of the pattern does not match. If the pattern is unanchored, the normal - "bumpalong" advance to the next starting character then happens. Back- - tracking can occur as usual to the left of (*PRUNE), or when matching - to the right of (*PRUNE), but if there is no match to the right, back- - tracking cannot cross (*PRUNE). In simple cases, the use of (*PRUNE) + "bumpalong" advance to the next starting character then happens. Back- + tracking can occur as usual to the left of (*PRUNE), or when matching + to the right of (*PRUNE), but if there is no match to the right, back- + tracking cannot cross (*PRUNE). In simple cases, the use of (*PRUNE) is just an alternative to an atomic group or possessive quantifier, but - there are some uses of (*PRUNE) that cannot be expressed in any other + there are some uses of (*PRUNE) that cannot be expressed in any other way. (*SKIP) - This verb is like (*PRUNE), except that if the pattern is unanchored, - the "bumpalong" advance is not to the next character, but to the posi- - tion in the subject where (*SKIP) was encountered. (*SKIP) signifies - that whatever text was matched leading up to it cannot be part of a + This verb is like (*PRUNE), except that if the pattern is unanchored, + the "bumpalong" advance is not to the next character, but to the posi- + tion in the subject where (*SKIP) was encountered. (*SKIP) signifies + that whatever text was matched leading up to it cannot be part of a successful match. Consider: a+(*SKIP)b - If the subject is "aaaac...", after the first match attempt fails - (starting at the first character in the string), the starting point + If the subject is "aaaac...", after the first match attempt fails + (starting at the first character in the string), the starting point skips on to start the next attempt at "c". Note that a possessive quan- - tifer does not have the same effect as this example; although it would - suppress backtracking during the first match attempt, the second - attempt would start at the second character instead of skipping on to + tifer does not have the same effect as this example; although it would + suppress backtracking during the first match attempt, the second + attempt would start at the second character instead of skipping on to "c". (*THEN) This verb causes a skip to the next alternation if the rest of the pat- tern does not match. That is, it cancels pending backtracking, but only - within the current alternation. Its name comes from the observation + within the current alternation. Its name comes from the observation that it can be used for a pattern-based if-then-else block: ( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... - If the COND1 pattern matches, FOO is tried (and possibly further items - after the end of the group if FOO succeeds); on failure the matcher - skips to the second alternative and tries COND2, without backtracking - into COND1. If (*THEN) is used outside of any alternation, it acts + If the COND1 pattern matches, FOO is tried (and possibly further items + after the end of the group if FOO succeeds); on failure the matcher + skips to the second alternative and tries COND2, without backtracking + into COND1. If (*THEN) is used outside of any alternation, it acts exactly like (*PRUNE). @@ -5244,8 +5251,8 @@ AUTHOR REVISION - Last updated: 18 October 2009 - Copyright (c) 1997-2009 University of Cambridge. + Last updated: 11 January 2010 + Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ @@ -6958,87 +6965,100 @@ PCRE DISCUSSION OF STACK USAGE restarted instead. The pcre_dfa_exec() function operates in an entirely different way, and - hardly uses recursion at all. The limit on its complexity is the amount - of workspace it is given. The comments that follow do NOT apply to - pcre_dfa_exec(); they are relevant only for pcre_exec(). - - You can set limits on the number of times that match() is called, both - in total and recursively. If the limit is exceeded, an error occurs. - For details, see the section on extra data for pcre_exec() in the - pcreapi documentation. - - Each time that match() is actually called recursively, it uses memory - from the process stack. For certain kinds of pattern and data, very - large amounts of stack may be needed, despite the recognition of "tail - recursion". You can often reduce the amount of recursion, and there- - fore the amount of stack used, by modifying the pattern that is being + uses recursion only when there is a regular expression recursion or + subroutine call in the pattern. This includes the processing of asser- + tion and "once-only" subpatterns, which are handled like subroutine + calls. Normally, these are never very deep, and the limit on the com- + plexity of pcre_dfa_exec() is controlled by the amount of workspace it + is given. However, it is possible to write patterns with runaway infi- + nite recursions; such patterns will cause pcre_dfa_exec() to run out of + stack. At present, there is no protection against this. + + The comments that follow do NOT apply to pcre_dfa_exec(); they are rel- + evant only for pcre_exec(). + + Reducing pcre_exec()'s stack usage + + Each time that match() is actually called recursively, it uses memory + from the process stack. For certain kinds of pattern and data, very + large amounts of stack may be needed, despite the recognition of "tail + recursion". You can often reduce the amount of recursion, and there- + fore the amount of stack used, by modifying the pattern that is being matched. Consider, for example, this pattern: ([^<]|<(?!inet))+ - It matches from wherever it starts until it encounters "<inet" or the - end of the data, and is the kind of pattern that might be used when + It matches from wherever it starts until it encounters "<inet" or the + end of the data, and is the kind of pattern that might be used when processing an XML file. Each iteration of the outer parentheses matches - either one character that is not "<" or a "<" that is not followed by - "inet". However, each time a parenthesis is processed, a recursion + either one character that is not "<" or a "<" that is not followed by + "inet". However, each time a parenthesis is processed, a recursion occurs, so this formulation uses a stack frame for each matched charac- - ter. For a long string, a lot of stack is required. Consider now this + ter. For a long string, a lot of stack is required. Consider now this rewritten pattern, which matches exactly the same strings: ([^<]++|<(?!inet))+ - This uses very much less stack, because runs of characters that do not - contain "<" are "swallowed" in one item inside the parentheses. Recur- - sion happens only when a "<" character that is not followed by "inet" - is encountered (and we assume this is relatively rare). A possessive - quantifier is used to stop any backtracking into the runs of non-"<" + This uses very much less stack, because runs of characters that do not + contain "<" are "swallowed" in one item inside the parentheses. Recur- + sion happens only when a "<" character that is not followed by "inet" + is encountered (and we assume this is relatively rare). A possessive + quantifier is used to stop any backtracking into the runs of non-"<" characters, but that is not related to stack usage. - This example shows that one way of avoiding stack problems when match- + This example shows that one way of avoiding stack problems when match- ing long subject strings is to write repeated parenthesized subpatterns to match more than one character whenever possible. - Compiling PCRE to use heap instead of stack - - In environments where stack memory is constrained, you might want to - compile PCRE to use heap memory instead of stack for remembering back- - up points. This makes it run a lot more slowly, however. Details of how - to do this are given in the pcrebuild documentation. When built in this - way, instead of using the stack, PCRE obtains and frees memory by call- - ing the functions that are pointed to by the pcre_stack_malloc and - pcre_stack_free variables. By default, these point to malloc() and - free(), but you can replace the pointers to cause PCRE to use your own - functions. Since the block sizes are always the same, and are always - freed in reverse order, it may be possible to implement customized mem- - ory handlers that are more efficient than the standard functions. - - Limiting PCRE's stack usage - - PCRE has an internal counter that can be used to limit the depth of - recursion, and thus cause pcre_exec() to give an error code before it - runs out of stack. By default, the limit is very large, and unlikely - ever to operate. It can be changed when PCRE is built, and it can also - be set when pcre_exec() is called. For details of these interfaces, see - the pcrebuild and pcreapi documentation. + Compiling PCRE to use heap instead of stack for pcre_exec() + + In environments where stack memory is constrained, you might want to + compile PCRE to use heap memory instead of stack for remembering back- + up points when pcre_exec() is running. This makes it run a lot more + slowly, however. Details of how to do this are given in the pcrebuild + documentation. When built in this way, instead of using the stack, PCRE + obtains and frees memory by calling the functions that are pointed to + by the pcre_stack_malloc and pcre_stack_free variables. By default, + these point to malloc() and free(), but you can replace the pointers to + cause PCRE to use your own functions. Since the block sizes are always + the same, and are always freed in reverse order, it may be possible to + implement customized memory handlers that are more efficient than the + standard functions. + + Limiting pcre_exec()'s stack usage + + You can set limits on the number of times that match() is called, both + in total and recursively. If a limit is exceeded, pcre_exec() returns + an error code. Setting suitable limits should prevent it from running + out of stack. The default values of the limits are very large, and + unlikely ever to operate. They can be changed when PCRE is built, and + they can also be set when pcre_exec() is called. For details of these + interfaces, see the pcrebuild documentation and the section on extra + data for pcre_exec() in the pcreapi documentation. As a very rough rule of thumb, you should reckon on about 500 bytes per recursion. Thus, if you want to limit your stack usage to 8Mb, you should set the limit at 16000 recursions. A 64Mb stack, on the other - hand, can support around 128000 recursions. The pcretest test program - has a command line option (-S) that can be used to increase the size of - its stack. + hand, can support around 128000 recursions. + + In Unix-like environments, the pcretest test program has a command line + option (-S) that can be used to increase the size of its stack. As long + as the stack is large enough, another option (-M) can be used to find + the smallest limits that allow a particular pattern to match a given + subject string. This is done by calling pcre_exec() repeatedly with + different limits. Changing stack size in Unix-like systems - In Unix-like environments, there is not often a problem with the stack - unless very long strings are involved, though the default limit on - stack size varies from system to system. Values from 8Mb to 64Mb are + In Unix-like environments, there is not often a problem with the stack + unless very long strings are involved, though the default limit on + stack size varies from system to system. Values from 8Mb to 64Mb are common. You can find your default limit by running the command: ulimit -s - Unfortunately, the effect of running out of stack is often SIGSEGV, - though sometimes a more explicit error message is given. You can nor- + Unfortunately, the effect of running out of stack is often SIGSEGV, + though sometimes a more explicit error message is given. You can nor- mally increase the limit on stack size by code such as this: struct rlimit rlim; @@ -7046,15 +7066,15 @@ PCRE DISCUSSION OF STACK USAGE rlim.rlim_cur = 100*1024*1024; setrlimit(RLIMIT_STACK, &rlim); - This reads the current limits (soft and hard) using getrlimit(), then - attempts to increase the soft limit to 100Mb using setrlimit(). You + This reads the current limits (soft and hard) using getrlimit(), then + attempts to increase the soft limit to 100Mb using setrlimit(). You must do this before calling pcre_exec(). Changing stack size in Mac OS X Using setrlimit(), as described above, should also work on Mac OS X. It is also possible to set a stack size when linking a program. There is a - discussion about stack sizes in Mac OS X at this web site: + discussion about stack sizes in Mac OS X at this web site: http://developer.apple.com/qa/qa2005/qa1419.html. @@ -7067,8 +7087,8 @@ AUTHOR REVISION - Last updated: 09 July 2008 - Copyright (c) 1997-2008 University of Cambridge. + Last updated: 03 January 2010 + Copyright (c) 1997-2010 University of Cambridge. ------------------------------------------------------------------------------ diff --git a/ext/pcre/pcrelib/pcre.h b/ext/pcre/pcrelib/pcre.h index 93dff102ac..dbb542681b 100644 --- a/ext/pcre/pcrelib/pcre.h +++ b/ext/pcre/pcrelib/pcre.h @@ -42,9 +42,9 @@ POSSIBILITY OF SUCH DAMAGE. /* The current PCRE version information. */ #define PCRE_MAJOR 8 -#define PCRE_MINOR 00 +#define PCRE_MINOR 01 #define PCRE_PRERELEASE -#define PCRE_DATE 2009-10-19 +#define PCRE_DATE 2010-01-19 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE, the appropriate diff --git a/ext/pcre/pcrelib/pcre_compile.c b/ext/pcre/pcrelib/pcre_compile.c index b197c1b548..eaf3d90ce8 100644 --- a/ext/pcre/pcrelib/pcre_compile.c +++ b/ext/pcre/pcrelib/pcre_compile.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2009 University of Cambridge + Copyright (c) 1997-2010 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -51,10 +51,11 @@ supporting internal functions that are not used by other modules. */ #include "pcre_internal.h" -/* When DEBUG is defined, we need the pcre_printint() function, which is also -used by pcretest. DEBUG is not defined when building a production library. */ +/* When PCRE_DEBUG is defined, we need the pcre_printint() function, which is +also used by pcretest. PCRE_DEBUG is not defined when building a production +library. */ -#ifdef DEBUG +#ifdef PCRE_DEBUG #include "pcre_printint.src" #endif @@ -1992,9 +1993,10 @@ static BOOL could_be_empty(const uschar *code, const uschar *endcode, branch_chain *bcptr, BOOL utf8) { -while (bcptr != NULL && bcptr->current >= code) +while (bcptr != NULL && bcptr->current_branch >= code) { - if (!could_be_empty_branch(bcptr->current, endcode, utf8)) return FALSE; + if (!could_be_empty_branch(bcptr->current_branch, endcode, utf8)) + return FALSE; bcptr = bcptr->outer; } return TRUE; @@ -2656,7 +2658,7 @@ BOOL utf8 = FALSE; uschar *utf8_char = NULL; #endif -#ifdef DEBUG +#ifdef PCRE_DEBUG if (lengthptr != NULL) DPRINTF((">> start branch\n")); #endif @@ -2715,7 +2717,7 @@ for (;; ptr++) if (lengthptr != NULL) { -#ifdef DEBUG +#ifdef PCRE_DEBUG if (code > cd->hwm) cd->hwm = code; /* High water info */ #endif if (code > cd->start_workspace + COMPILE_WORK_SIZE) /* Check for overrun */ @@ -4211,13 +4213,15 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ { /* In the pre-compile phase, we don't actually do the replication. We just adjust the length as if we had. Do some paranoid checks for - potential integer overflow. */ + potential integer overflow. The INT64_OR_DOUBLE type is a 64-bit + integer type when available, otherwise double. */ if (lengthptr != NULL) { int delta = (repeat_min - 1)*length_prevgroup; - if ((double)(repeat_min - 1)*(double)length_prevgroup > - (double)INT_MAX || + if ((INT64_OR_DOUBLE)(repeat_min - 1)* + (INT64_OR_DOUBLE)length_prevgroup > + (INT64_OR_DOUBLE)INT_MAX || OFLOW_MAX - *lengthptr < delta) { *errorcodeptr = ERR20; @@ -4263,15 +4267,16 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ just adjust the length as if we had. For each repetition we must add 1 to the length for BRAZERO and for all but the last repetition we must add 2 + 2*LINKSIZE to allow for the nesting that occurs. Do some - paranoid checks to avoid integer overflow. */ + paranoid checks to avoid integer overflow. The INT64_OR_DOUBLE type is + a 64-bit integer type when available, otherwise double. */ if (lengthptr != NULL && repeat_max > 0) { int delta = repeat_max * (length_prevgroup + 1 + 2 + 2*LINK_SIZE) - 2 - 2*LINK_SIZE; /* Last one doesn't nest */ - if ((double)repeat_max * - (double)(length_prevgroup + 1 + 2 + 2*LINK_SIZE) - > (double)INT_MAX || + if ((INT64_OR_DOUBLE)repeat_max * + (INT64_OR_DOUBLE)(length_prevgroup + 1 + 2 + 2*LINK_SIZE) + > (INT64_OR_DOUBLE)INT_MAX || OFLOW_MAX - *lengthptr < delta) { *errorcodeptr = ERR20; @@ -5246,7 +5251,7 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ { cd->external_options = newoptions; } - else + else { if ((options & PCRE_IMS) != (newoptions & PCRE_IMS)) { @@ -5583,6 +5588,7 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ if (-c >= ESC_REF) { + open_capitem *oc; recno = -c - ESC_REF; HANDLE_REFERENCE: /* Come here from named backref handling */ @@ -5592,6 +5598,19 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ PUT2INC(code, 0, recno); cd->backref_map |= (recno < 32)? (1 << recno) : 1; if (recno > cd->top_backref) cd->top_backref = recno; + + /* Check to see if this back reference is recursive, that it, it + is inside the group that it references. A flag is set so that the + group can be made atomic. */ + + for (oc = cd->open_caps; oc != NULL; oc = oc->next) + { + if (oc->number == recno) + { + oc->flag = TRUE; + break; + } + } } /* So are Unicode property matches, if supported. */ @@ -5781,10 +5800,11 @@ int branchfirstbyte, branchreqbyte; int length; int orig_bracount; int max_bracount; +int old_external_options = cd->external_options; branch_chain bc; bc.outer = bcptr; -bc.current = code; +bc.current_branch = code; firstbyte = reqbyte = REQ_UNSET; @@ -5803,13 +5823,15 @@ them global. It tests the value of length for (2 + 2*LINK_SIZE) in the pre-compile phase to find out whether anything has yet been compiled or not. */ /* If this is a capturing subpattern, add to the chain of open capturing items -so that we can detect them if (*ACCEPT) is encountered. */ +so that we can detect them if (*ACCEPT) is encountered. This is also used to +detect groups that contain recursive back references to themselves. */ if (*code == OP_CBRA) { capnumber = GET2(code, 1 + LINK_SIZE); capitem.number = capnumber; capitem.next = cd->open_caps; + capitem.flag = FALSE; cd->open_caps = &capitem; } @@ -5857,6 +5879,15 @@ for (;;) return FALSE; } + /* If the external options have changed during this branch, it means that we + are at the top level, and a leading option setting has been encountered. We + need to re-set the original option values to take account of this so that, + during the pre-compile phase, we know to allow for a re-set at the start of + subsequent branches. */ + + if (old_external_options != cd->external_options) + oldims = cd->external_options & PCRE_IMS; + /* Keep the highest bracket count in case (?| was used and some branch has fewer than the rest. */ @@ -5957,17 +5988,34 @@ for (;;) while (branch_length > 0); } - /* If it was a capturing subpattern, remove it from the chain. */ - - if (capnumber > 0) cd->open_caps = cd->open_caps->next; - /* Fill in the ket */ *code = OP_KET; PUT(code, 1, code - start_bracket); code += 1 + LINK_SIZE; - /* Resetting option if needed */ + /* If it was a capturing subpattern, check to see if it contained any + recursive back references. If so, we must wrap it in atomic brackets. + In any event, remove the block from the chain. */ + + if (capnumber > 0) + { + if (cd->open_caps->flag) + { + memmove(start_bracket + 1 + LINK_SIZE, start_bracket, + code - start_bracket); + *start_bracket = OP_ONCE; + code += 1 + LINK_SIZE; + PUT(start_bracket, 1, code - start_bracket); + *code = OP_KET; + PUT(code, 1, code - start_bracket); + code += 1 + LINK_SIZE; + length += 2 + 2*LINK_SIZE; + } + cd->open_caps = cd->open_caps->next; + } + + /* Reset options if needed. */ if ((options & PCRE_IMS) != oldims && *ptr == CHAR_RIGHT_PARENTHESIS) { @@ -6016,7 +6064,7 @@ for (;;) { *code = OP_ALT; PUT(code, 1, code - last_branch); - bc.current = last_branch = code; + bc.current_branch = last_branch = code; code += 1 + LINK_SIZE; } @@ -6432,7 +6480,7 @@ while (ptr[skipatstart] == CHAR_LEFT_PARENTHESIS && #ifdef SUPPORT_UTF8 if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0 && - (*erroroffset = _pcre_valid_utf8((uschar *)pattern, -1)) >= 0) + (*erroroffset = _pcre_valid_utf8((USPTR)pattern, -1)) >= 0) { errorcode = ERR44; goto PCRE_EARLY_ERROR_RETURN2; @@ -6629,7 +6677,7 @@ if debugging, leave the test till after things are printed out. */ *code++ = OP_END; -#ifndef DEBUG +#ifndef PCRE_DEBUG if (code - codestart > length) errorcode = ERR23; #endif @@ -6753,7 +6801,7 @@ if (reqbyte >= 0 && /* Print out the compiled data if debugging is enabled. This is never the case when building a production library. */ -#ifdef DEBUG +#ifdef PCRE_DEBUG printf("Length = %d top_bracket = %d top_backref = %d\n", length, re->top_bracket, re->top_backref); @@ -6791,7 +6839,7 @@ if (code - codestart > length) if (errorcodeptr != NULL) *errorcodeptr = ERR23; return NULL; } -#endif /* DEBUG */ +#endif /* PCRE_DEBUG */ return (pcre *)re; } diff --git a/ext/pcre/pcrelib/pcre_exec.c b/ext/pcre/pcrelib/pcre_exec.c index ca3079b0a2..46f667e21b 100644 --- a/ext/pcre/pcrelib/pcre_exec.c +++ b/ext/pcre/pcrelib/pcre_exec.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2009 University of Cambridge + Copyright (c) 1997-2010 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -87,7 +87,7 @@ static const char rep_max[] = { 0, 0, 0, 0, 1, 1 }; -#ifdef DEBUG +#ifdef PCRE_DEBUG /************************************************* * Debugging function to print chars * *************************************************/ @@ -139,7 +139,7 @@ match_ref(int offset, register USPTR eptr, int length, match_data *md, { USPTR p = md->start_subject + md->offset_vector[offset]; -#ifdef DEBUG +#ifdef PCRE_DEBUG if (eptr >= md->end_subject) printf("matching subject <null>"); else @@ -252,7 +252,7 @@ actuall used in this definition. */ #ifndef NO_RECURSE #define REGISTER register -#ifdef DEBUG +#ifdef PCRE_DEBUG #define RMATCH(ra,rb,rc,rd,re,rf,rg,rw) \ { \ printf("match() called in line %d\n", __LINE__); \ @@ -620,7 +620,7 @@ TAIL_RECURSE: /* OK, now we can get on with the real code of the function. Recursive calls are specified by the macro RMATCH and RRETURN is used to return. When NO_RECURSE is *not* defined, these just turn into a recursive call to match() -and a "return", respectively (possibly with some debugging if DEBUG is +and a "return", respectively (possibly with some debugging if PCRE_DEBUG is defined). However, RMATCH isn't like a function call because it's quite a complicated macro. It has to be used in one particular way. This shouldn't, however, impact performance when true recursion is being used. */ @@ -711,7 +711,7 @@ for (;;) number = GET2(ecode, 1+LINK_SIZE); offset = number << 1; -#ifdef DEBUG +#ifdef PCRE_DEBUG printf("start bracket %d\n", number); printf("subject="); pchars(eptr, 16, TRUE, md); @@ -1037,7 +1037,7 @@ for (;;) number = GET2(ecode, 1); offset = number << 1; -#ifdef DEBUG +#ifdef PCRE_DEBUG printf("end bracket %d at *ACCEPT", number); printf("\n"); #endif @@ -1131,7 +1131,9 @@ for (;;) offset_top = md->end_offset_top; continue; - /* Negative assertion: all branches must fail to match */ + /* Negative assertion: all branches must fail to match. Encountering SKIP, + PRUNE, or COMMIT means we must assume failure without checking subsequent + branches. */ case OP_ASSERT_NOT: case OP_ASSERTBACK_NOT: @@ -1140,6 +1142,11 @@ for (;;) RMATCH(eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, 0, RM5); if (rrc == MATCH_MATCH) RRETURN(MATCH_NOMATCH); + if (rrc == MATCH_SKIP || rrc == MATCH_PRUNE || rrc == MATCH_COMMIT) + { + do ecode += GET(ecode,1); while (*ecode == OP_ALT); + break; + } if (rrc != MATCH_NOMATCH && rrc != MATCH_THEN) RRETURN(rrc); ecode += GET(ecode,1); } @@ -1459,7 +1466,7 @@ for (;;) number = GET2(prev, 1+LINK_SIZE); offset = number << 1; -#ifdef DEBUG +#ifdef PCRE_DEBUG printf("end bracket %d", number); printf("\n"); #endif @@ -3686,8 +3693,12 @@ for (;;) case OP_NOT_WORDCHAR: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || - (*eptr < 128 && (md->ctypes[*eptr] & ctype_word) != 0)) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (*eptr < 128 && (md->ctypes[*eptr] & ctype_word) != 0) RRETURN(MATCH_NOMATCH); while (++eptr < md->end_subject && (*eptr & 0xc0) == 0x80); } @@ -5563,7 +5574,7 @@ for(;;) bytes to avoid spending too much time in this optimization. */ if (study != NULL && (study->flags & PCRE_STUDY_MINLEN) != 0 && - end_subject - start_match < study->minlength) + (pcre_uint32)(end_subject - start_match) < study->minlength) { rc = MATCH_NOMATCH; break; @@ -5626,7 +5637,7 @@ for(;;) } } -#ifdef DEBUG /* Sigh. Some compilers never learn. */ +#ifdef PCRE_DEBUG /* Sigh. Some compilers never learn. */ printf(">>>> Match against: "); pchars(start_match, end_subject - start_match, TRUE, md); printf("\n"); diff --git a/ext/pcre/pcrelib/pcre_internal.h b/ext/pcre/pcrelib/pcre_internal.h index de0961435b..aa1983e8f9 100644 --- a/ext/pcre/pcrelib/pcre_internal.h +++ b/ext/pcre/pcrelib/pcre_internal.h @@ -7,7 +7,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2009 University of Cambridge + Copyright (c) 1997-2010 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -45,10 +45,10 @@ functions whose names all begin with "_pcre_". */ #ifndef PCRE_INTERNAL_H #define PCRE_INTERNAL_H -/* Define DEBUG to get debugging output on stdout. */ +/* Define PCRE_DEBUG to get debugging output on stdout. */ #if 0 -#define DEBUG +#define PCRE_DEBUG #endif /* We do not support both EBCDIC and UTF-8 at the same time. The "configure" @@ -74,7 +74,7 @@ It turns out that the Mac Debugging.h header also defines the macro DPRINTF, so be absolutely sure we get our version. */ #undef DPRINTF -#ifdef DEBUG +#ifdef PCRE_DEBUG #define DPRINTF(p) printf p #else #define DPRINTF(p) /* Nothing */ @@ -86,8 +86,6 @@ setjmp and stdarg are used is when NO_RECURSE is set. */ #include <ctype.h> #include <limits.h> -#include <setjmp.h> -#include <stdarg.h> #include <stddef.h> #include <stdio.h> #include <stdlib.h> @@ -186,6 +184,23 @@ preprocessor time in standard C environments. */ #error Cannot determine a type for 32-bit unsigned integers #endif +/* When checking for integer overflow in pcre_compile(), we need to handle +large integers. If a 64-bit integer type is available, we can use that. +Otherwise we have to cast to double, which of course requires floating point +arithmetic. Handle this by defining a macro for the appropriate type. If +stdint.h is available, include it; it may define INT64_MAX. The macro int64_t +may be set by "configure". */ + +#if HAVE_STDINT_H +#include <stdint.h> +#endif + +#if defined INT64_MAX || defined int64_t +#define INT64_OR_DOUBLE int64_t +#else +#define INT64_OR_DOUBLE double +#endif + /* All character handling must be done as unsigned characters. Otherwise there are problems with top-bit-set characters and functions such as isspace(). However, we leave the interface to the outside world as char *, because that @@ -1536,11 +1551,13 @@ typedef struct pcre_study_data { /* Structure for building a chain of open capturing subpatterns during compiling, so that instructions to close them can be compiled when (*ACCEPT) is -encountered. */ +encountered. This is also used to identify subpatterns that contain recursive +back references to themselves, so that they can be made atomic. */ typedef struct open_capitem { struct open_capitem *next; /* Chain link */ pcre_uint16 number; /* Capture number */ + pcre_uint16 flag; /* Set TRUE if recursive back ref */ } open_capitem; /* Structure for passing "static" information around between the functions @@ -1579,7 +1596,7 @@ branches, for testing for left recursion. */ typedef struct branch_chain { struct branch_chain *outer; - uschar *current; + uschar *current_branch; } branch_chain; /* Structure for items in a linked list that represents an explicit recursive @@ -1738,14 +1755,12 @@ one of the exported public functions. They have to be "external" in the C sense, but are not part of the PCRE public API. */ extern const uschar *_pcre_find_bracket(const uschar *, BOOL, int); -extern BOOL _pcre_is_newline(const uschar *, int, const uschar *, - int *, BOOL); +extern BOOL _pcre_is_newline(USPTR, int, USPTR, int *, BOOL); extern int _pcre_ord2utf8(int, uschar *); extern real_pcre *_pcre_try_flipped(const real_pcre *, real_pcre *, const pcre_study_data *, pcre_study_data *); -extern int _pcre_valid_utf8(const uschar *, int); -extern BOOL _pcre_was_newline(const uschar *, int, const uschar *, - int *, BOOL); +extern int _pcre_valid_utf8(USPTR, int); +extern BOOL _pcre_was_newline(USPTR, int, USPTR, int *, BOOL); extern BOOL _pcre_xclass(int, const uschar *); diff --git a/ext/pcre/pcrelib/pcre_printint.src b/ext/pcre/pcrelib/pcre_printint.src index acfc4ca688..9b2def1c86 100644 --- a/ext/pcre/pcrelib/pcre_printint.src +++ b/ext/pcre/pcrelib/pcre_printint.src @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2009 University of Cambridge + Copyright (c) 1997-2010 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -43,7 +43,8 @@ internal form of a compiled regular expression, along with some supporting local functions. This source file is used in two places: (1) It is #included by pcre_compile.c when it is compiled in debugging mode -(DEBUG defined in pcre_internal.h). It is not included in production compiles. +(PCRE_DEBUG defined in pcre_internal.h). It is not included in production +compiles. (2) It is always #included by pcretest.c, which can be asked to print out a compiled regex for debugging purposes. */ diff --git a/ext/pcre/pcrelib/pcre_study.c b/ext/pcre/pcrelib/pcre_study.c index a2e9f44faa..2653624e0a 100644 --- a/ext/pcre/pcrelib/pcre_study.c +++ b/ext/pcre/pcrelib/pcre_study.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2009 University of Cambridge + Copyright (c) 1997-2010 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -94,13 +94,28 @@ for (;;) switch (op) { + case OP_COND: + case OP_SCOND: + + /* If there is only one branch in a condition, the implied branch has zero + length, so we don't add anything. This covers the DEFINE "condition" + automatically. */ + + cs = cc + GET(cc, 1); + if (*cs != OP_ALT) + { + cc = cs + 1 + LINK_SIZE; + break; + } + + /* Otherwise we can fall through and treat it the same as any other + subpattern. */ + case OP_CBRA: case OP_SCBRA: case OP_BRA: case OP_SBRA: case OP_ONCE: - case OP_COND: - case OP_SCOND: d = find_minlength(cc, startcode, options); if (d < 0) return d; branchlength += d; @@ -427,7 +442,8 @@ Returns: nothing */ static void -set_bit(uschar *start_bits, unsigned int c, BOOL caseless, compile_data *cd) +set_table_bit(uschar *start_bits, unsigned int c, BOOL caseless, + compile_data *cd) { start_bits[c/8] |= (1 << (c&7)); if (caseless && (cd->ctypes[c] & ctype_letter) != 0) @@ -589,7 +605,7 @@ do case OP_QUERY: case OP_MINQUERY: case OP_POSQUERY: - set_bit(start_bits, tcode[1], caseless, cd); + set_table_bit(start_bits, tcode[1], caseless, cd); tcode += 2; #ifdef SUPPORT_UTF8 if (utf8 && tcode[-1] >= 0xc0) @@ -602,7 +618,7 @@ do case OP_UPTO: case OP_MINUPTO: case OP_POSUPTO: - set_bit(start_bits, tcode[3], caseless, cd); + set_table_bit(start_bits, tcode[3], caseless, cd); tcode += 4; #ifdef SUPPORT_UTF8 if (utf8 && tcode[-1] >= 0xc0) @@ -620,7 +636,7 @@ do case OP_PLUS: case OP_MINPLUS: case OP_POSPLUS: - set_bit(start_bits, tcode[1], caseless, cd); + set_table_bit(start_bits, tcode[1], caseless, cd); try_next = FALSE; break; diff --git a/ext/pcre/pcrelib/pcredemo.c b/ext/pcre/pcrelib/pcredemo.c index c6ba56ed5c..d565aecddd 100644 --- a/ext/pcre/pcrelib/pcredemo.c +++ b/ext/pcre/pcrelib/pcredemo.c @@ -7,14 +7,24 @@ of calling the PCRE regular expression library from a C program. See the pcresample documentation for a short discussion ("man pcresample" if you have the PCRE man pages installed). -In Unix-like environments, compile this program thuswise: +In Unix-like environments, if PCRE is installed in your standard system +libraries, you should be able to compile this program using this command: - gcc -Wall pcredemo.c -I/usr/local/include -L/usr/local/lib \ - -R/usr/local/lib -lpcre +gcc -Wall pcredemo.c -lpcre -o pcredemo + +If PCRE is not installed in a standard place, it is likely to be installed with +support for the pkg-config mechanism. If you have pkg-config, you can compile +this program using this command: + +gcc -Wall pcredemo.c `pkg-config --cflags --libs libpcre` -o pcredemo + +If you do not have pkg-config, you may have to use this: + +gcc -Wall pcredemo.c -I/usr/local/include -L/usr/local/lib \ + -R/usr/local/lib -lpcre -o pcredemo Replace "/usr/local/include" and "/usr/local/lib" with wherever the include and -library files for PCRE are installed on your system. You don't need -I and -L -if PCRE is installed in the standard system libraries. Only some operating +library files for PCRE are installed on your system. Only some operating systems (e.g. Solaris) use the -R option. Building under Windows: diff --git a/ext/pcre/pcrelib/testdata/testinput1 b/ext/pcre/pcrelib/testdata/testinput1 index 6cacbb5124..d999d2948a 100644 --- a/ext/pcre/pcrelib/testdata/testinput1 +++ b/ext/pcre/pcrelib/testdata/testinput1 @@ -2332,15 +2332,14 @@ baz foobarbaz -/The case of aaaaaa is missed out below because I think Perl 5.005_02 gets/ -/it wrong; it sets $1 to aaa rather than aa. Compare the following test,/ -/where it does set $1 to aa when matching aaaaaa./ +/The cases of aaaa and aaaaaa are missed out below because Perl does things/ +/differently. We know that odd, and maybe incorrect, things happen with/ +/recursive references in Perl, as far as 5.11.3 - see some stuff in test #2./ /^(a\1?){4}$/ a aa aaa - aaaa aaaaa aaaaaaa aaaaaaaa @@ -4067,4 +4066,11 @@ /^%((?(?=[a])[^%])|b)*%$/ %ab% +/(?i)a(?-i)b|c/ + XabX + XAbX + CcC + ** Failers + XABX + /-- End of testinput1 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput2 b/ext/pcre/pcrelib/testdata/testinput2 index 3319bb7327..1c1a72dd57 100644 --- a/ext/pcre/pcrelib/testdata/testinput2 +++ b/ext/pcre/pcrelib/testdata/testinput2 @@ -344,10 +344,25 @@ *** Failers a -/This one is here because I think Perl 5.005_02 gets the setting of $1 wrong/I +/This one is here because Perl behaves differently; see also the following/I /^(a\1?){4}$/I + aaaa aaaaaa + +/Perl does not fail these two for the final subjects. Neither did PCRE until/ +/release 8.01. The problem is in backtracking into a subpattern that contains/ +/a recursive reference to itself. PCRE has now made these into atomic patterns./ + +/^(xa|=?\1a){2}$/ + xa=xaa + ** Failers + xa=xaaa + +/^(xa|=?\1a)+$/ + xa=xaa + ** Failers + xa=xaaa /These are syntax tests from Perl 5.005/I @@ -3169,4 +3184,24 @@ a random value. /Ix /(\3)(\1)(a)/SI cat +/i(?(DEFINE)(?<s>a))/SI + i + +/()i(?(1)a)/SI + ia + +/(?i)a(?-i)b|c/BZ + XabX + XAbX + CcC + ** Failers + XABX + +/(?i)a(?s)b|c/BZ + +/(?i)a(?s-i)b|c/BZ + +/^(ab(c\1)d|x){2}$/BZ + xabcxd + /-- End of testinput2 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput5 b/ext/pcre/pcrelib/testdata/testinput5 index 82818d7dc1..68795e6095 100644 --- a/ext/pcre/pcrelib/testdata/testinput5 +++ b/ext/pcre/pcrelib/testdata/testinput5 @@ -742,4 +742,7 @@ can't tell the difference.) --/ xxxxabcde\P xxxxabcde\P\P +/X\W{3}X/8 + \PX + /-- End of testinput5 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput7 b/ext/pcre/pcrelib/testdata/testinput7 index 710d9ee9b6..5d27311927 100644 --- a/ext/pcre/pcrelib/testdata/testinput7 +++ b/ext/pcre/pcrelib/testdata/testinput7 @@ -4542,4 +4542,22 @@ CAD BAD +/^(?!a(*SKIP)b)/ + ac + +/^(?=a(*SKIP)b|ac)/ + ** Failers + ac + +/^(?=a(*THEN)b|ac)/ + ac + +/^(?=a(*PRUNE)b)/ + ab + ** Failers + ac + +/^(?(?!a(*SKIP)b))/ + ac + /-- End of testinput7 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput1 b/ext/pcre/pcrelib/testdata/testoutput1 index a2a6dd4e22..2fd033cc3f 100644 --- a/ext/pcre/pcrelib/testdata/testoutput1 +++ b/ext/pcre/pcrelib/testdata/testoutput1 @@ -3752,10 +3752,10 @@ No match foobarbaz No match -/The case of aaaaaa is missed out below because I think Perl 5.005_02 gets/ -/it wrong; it sets $1 to aaa rather than aa. Compare the following test,/ +/The cases of aaaa and aaaaaa are missed out below because Perl does things/ +/differently. We know that odd, and maybe incorrect, things happen with/ No match -/where it does set $1 to aa when matching aaaaaa./ +/recursive references in Perl, as far as 5.11.3 - see some stuff in test #2./ No match /^(a\1?){4}$/ @@ -3765,9 +3765,6 @@ No match No match aaa No match - aaaa - 0: aaaa - 1: a aaaaa 0: aaaaa 1: a @@ -6649,4 +6646,16 @@ No match 0: %ab% 1: +/(?i)a(?-i)b|c/ + XabX + 0: ab + XAbX + 0: Ab + CcC + 0: c + ** Failers +No match + XABX +No match + /-- End of testinput1 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput2 b/ext/pcre/pcrelib/testdata/testoutput2 index 30ea63b9cc..4b27de0543 100644 --- a/ext/pcre/pcrelib/testdata/testoutput2 +++ b/ext/pcre/pcrelib/testdata/testoutput2 @@ -857,7 +857,7 @@ No match a No match -/This one is here because I think Perl 5.005_02 gets the setting of $1 wrong/I +/This one is here because Perl behaves differently; see also the following/I Capturing subpattern count = 0 No options First char = 'T' @@ -869,9 +869,34 @@ Max back reference = 1 Options: anchored No first char No need char + aaaa +No match aaaaaa - 0: aaaaaa - 1: aa +No match + +/Perl does not fail these two for the final subjects. Neither did PCRE until/ +/release 8.01. The problem is in backtracking into a subpattern that contains/ +No match +/a recursive reference to itself. PCRE has now made these into atomic patterns./ +No match + +/^(xa|=?\1a){2}$/ + xa=xaa + 0: xa=xaa + 1: =xaa + ** Failers +No match + xa=xaaa +No match + +/^(xa|=?\1a)+$/ + xa=xaa + 0: xa=xaa + 1: =xaa + ** Failers +No match + xa=xaaa +No match /These are syntax tests from Perl 5.005/I Capturing subpattern count = 0 @@ -10463,4 +10488,113 @@ No set of starting bytes cat No match +/i(?(DEFINE)(?<s>a))/SI +Capturing subpattern count = 1 +Named capturing subpatterns: + s 1 +No options +First char = 'i' +No need char +Subject length lower bound = 1 +No set of starting bytes + i + 0: i + +/()i(?(1)a)/SI +Capturing subpattern count = 1 +No options +No first char +Need char = 'i' +Subject length lower bound = 1 +Starting byte set: i + ia + 0: ia + 1: + +/(?i)a(?-i)b|c/BZ +------------------------------------------------------------------ + Bra + NC a + 00 Opt + b + Alt + 00 Opt + c + Ket + End +------------------------------------------------------------------ + XabX + 0: ab + XAbX + 0: Ab + CcC + 0: c + ** Failers +No match + XABX +No match + +/(?i)a(?s)b|c/BZ +------------------------------------------------------------------ + Bra + NC a + 05 Opt + NC b + Alt + 05 Opt + NC c + Ket + End +------------------------------------------------------------------ + +/(?i)a(?s-i)b|c/BZ +------------------------------------------------------------------ + Bra + NC a + 04 Opt + b + Alt + 04 Opt + c + Ket + End +------------------------------------------------------------------ + +/^(ab(c\1)d|x){2}$/BZ +------------------------------------------------------------------ + Bra + ^ + Once + CBra 1 + ab + CBra 2 + c + \1 + Ket + d + Alt + x + Ket + Ket + Once + CBra 1 + ab + CBra 2 + c + \1 + Ket + d + Alt + x + Ket + Ket + $ + Ket + End +------------------------------------------------------------------ + xabcxd + 0: xabcxd + 1: abcxd + 2: cx + /-- End of testinput2 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput5 b/ext/pcre/pcrelib/testdata/testoutput5 index f5de747ba0..9d815ffa82 100644 --- a/ext/pcre/pcrelib/testdata/testoutput5 +++ b/ext/pcre/pcrelib/testdata/testoutput5 @@ -2072,4 +2072,8 @@ Partial match: abca xxxxabcde\P\P Partial match: abcde +/X\W{3}X/8 + \PX +Partial match: X + /-- End of testinput5 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput7 b/ext/pcre/pcrelib/testdata/testoutput7 index c6c9df46f5..2aab80d74b 100644 --- a/ext/pcre/pcrelib/testdata/testoutput7 +++ b/ext/pcre/pcrelib/testdata/testoutput7 @@ -7584,4 +7584,30 @@ No match BAD No match +/^(?!a(*SKIP)b)/ + ac +Error -16 + +/^(?=a(*SKIP)b|ac)/ + ** Failers +No match + ac +Error -16 + +/^(?=a(*THEN)b|ac)/ + ac +Error -16 + +/^(?=a(*PRUNE)b)/ + ab +Error -16 + ** Failers +No match + ac +Error -16 + +/^(?(?!a(*SKIP)b))/ + ac +Error -16 + /-- End of testinput7 --/ |