diff options
| author | Scott MacVicar <scottmac@php.net> | 2009-11-03 12:15:03 +0000 |
|---|---|---|
| committer | Scott MacVicar <scottmac@php.net> | 2009-11-03 12:15:03 +0000 |
| commit | f03b175f7cb5b1ffc0b4f59d83836beb3794ebfd (patch) | |
| tree | 30ce6bf2f30beb84bbbc70d257a2666d032e0398 | |
| parent | 26e3082abca953a2add5ae02b6fd3f6f301e00ec (diff) | |
| download | php-git-f03b175f7cb5b1ffc0b4f59d83836beb3794ebfd.tar.gz | |
Update PCRE to 8.00
42 files changed, 7018 insertions, 3395 deletions
diff --git a/ext/pcre/pcrelib/ChangeLog b/ext/pcre/pcrelib/ChangeLog index 93a5415ee7..40600b861d 100644 --- a/ext/pcre/pcrelib/ChangeLog +++ b/ext/pcre/pcrelib/ChangeLog @@ -1,6 +1,170 @@ ChangeLog for PCRE ------------------ +Version 8.00 19-Oct-09 +---------------------- + +1. The table for translating pcre_compile() error codes into POSIX error codes + was out-of-date, and there was no check on the pcre_compile() error code + being within the table. This could lead to an OK return being given in + error. + +2. Changed the call to open a subject file in pcregrep from fopen(pathname, + "r") to fopen(pathname, "rb"), which fixed a problem with some of the tests + in a Windows environment. + +3. The pcregrep --count option prints the count for each file even when it is + zero, as does GNU grep. However, pcregrep was also printing all files when + --files-with-matches was added. Now, when both options are given, it prints + counts only for those files that have at least one match. (GNU grep just + prints the file name in this circumstance, but including the count seems + more useful - otherwise, why use --count?) Also ensured that the + combination -clh just lists non-zero counts, with no names. + +4. The long form of the pcregrep -F option was incorrectly implemented as + --fixed_strings instead of --fixed-strings. This is an incompatible change, + but it seems right to fix it, and I didn't think it was worth preserving + the old behaviour. + +5. The command line items --regex=pattern and --regexp=pattern were not + recognized by pcregrep, which required --regex pattern or --regexp pattern + (with a space rather than an '='). The man page documented the '=' forms, + which are compatible with GNU grep; these now work. + +6. No libpcreposix.pc file was created for pkg-config; there was just + libpcre.pc and libpcrecpp.pc. The omission has been rectified. + +7. Added #ifndef SUPPORT_UCP into the pcre_ucd.c module, to reduce its size + when UCP support is not needed, by modifying the Python script that + generates it from Unicode data files. This should not matter if the module + is correctly used as a library, but I received one complaint about 50K of + unwanted data. My guess is that the person linked everything into his + program rather than using a library. Anyway, it does no harm. + +8. A pattern such as /\x{123}{2,2}+/8 was incorrectly compiled; the trigger + was a minimum greater than 1 for a wide character in a possessive + repetition. The same bug could also affect patterns like /(\x{ff}{0,2})*/8 + which had an unlimited repeat of a nested, fixed maximum repeat of a wide + character. Chaos in the form of incorrect output or a compiling loop could + result. + +9. The restrictions on what a pattern can contain when partial matching is + requested for pcre_exec() have been removed. All patterns can now be + partially matched by this function. In addition, if there are at least two + slots in the offset vector, the offset of the earliest inspected character + for the match and the offset of the end of the subject are set in them when + PCRE_ERROR_PARTIAL is returned. + +10. Partial matching has been split into two forms: PCRE_PARTIAL_SOFT, which is + synonymous with PCRE_PARTIAL, for backwards compatibility, and + PCRE_PARTIAL_HARD, which causes a partial match to supersede a full match, + and may be more useful for multi-segment matching. + +11. Partial matching with pcre_exec() is now more intuitive. A partial match + used to be given if ever the end of the subject was reached; now it is + given only if matching could not proceed because another character was + needed. This makes a difference in some odd cases such as Z(*FAIL) with the + string "Z", which now yields "no match" instead of "partial match". In the + case of pcre_dfa_exec(), "no match" is given if every matching path for the + final character ended with (*FAIL). + +12. Restarting a match using pcre_dfa_exec() after a partial match did not work + if the pattern had a "must contain" character that was already found in the + earlier partial match, unless partial matching was again requested. For + example, with the pattern /dog.(body)?/, the "must contain" character is + "g". If the first part-match was for the string "dog", restarting with + "sbody" failed. This bug has been fixed. + +13. The string returned by pcre_dfa_exec() after a partial match has been + changed so that it starts at the first inspected character rather than the + first character of the match. This makes a difference only if the pattern + starts with a lookbehind assertion or \b or \B (\K is not supported by + pcre_dfa_exec()). It's an incompatible change, but it makes the two + matching functions compatible, and I think it's the right thing to do. + +14. Added a pcredemo man page, created automatically from the pcredemo.c file, + so that the demonstration program is easily available in environments where + PCRE has not been installed from source. + +15. Arranged to add -DPCRE_STATIC to cflags in libpcre.pc, libpcreposix.cp, + libpcrecpp.pc and pcre-config when PCRE is not compiled as a shared + library. + +16. Added REG_UNGREEDY to the pcreposix interface, at the request of a user. + It maps to PCRE_UNGREEDY. It is not, of course, POSIX-compatible, but it + is not the first non-POSIX option to be added. Clearly some people find + these options useful. + +17. If a caller to the POSIX matching function regexec() passes a non-zero + value for nmatch with a NULL value for pmatch, the value of + nmatch is forced to zero. + +18. RunGrepTest did not have a test for the availability of the -u option of + the diff command, as RunTest does. It now checks in the same way as + RunTest, and also checks for the -b option. + +19. If an odd number of negated classes containing just a single character + interposed, within parentheses, between a forward reference to a named + subpattern and the definition of the subpattern, compilation crashed with + an internal error, complaining that it could not find the referenced + subpattern. An example of a crashing pattern is /(?&A)(([^m])(?<A>))/. + [The bug was that it was starting one character too far in when skipping + over the character class, thus treating the ] as data rather than + terminating the class. This meant it could skip too much.] + +20. Added PCRE_NOTEMPTY_ATSTART in order to be able to correctly implement the + /g option in pcretest when the pattern contains \K, which makes it possible + to have an empty string match not at the start, even when the pattern is + anchored. Updated pcretest and pcredemo to use this option. + +21. If the maximum number of capturing subpatterns in a recursion was greater + than the maximum at the outer level, the higher number was returned, but + with unset values at the outer level. The correct (outer level) value is + now given. + +22. If (*ACCEPT) appeared inside capturing parentheses, previous releases of + PCRE did not set those parentheses (unlike Perl). I have now found a way to + make it do so. The string so far is captured, making this feature + compatible with Perl. + +23. The tests have been re-organized, adding tests 11 and 12, to make it + possible to check the Perl 5.10 features against Perl 5.10. + +24. Perl 5.10 allows subroutine calls in lookbehinds, as long as the subroutine + pattern matches a fixed length string. PCRE did not allow this; now it + does. Neither allows recursion. + +25. I finally figured out how to implement a request to provide the minimum + length of subject string that was needed in order to match a given pattern. + (It was back references and recursion that I had previously got hung up + on.) This code has now been added to pcre_study(); it finds a lower bound + to the length of subject needed. It is not necessarily the greatest lower + bound, but using it to avoid searching strings that are too short does give + some useful speed-ups. The value is available to calling programs via + pcre_fullinfo(). + +26. While implementing 25, I discovered to my embarrassment that pcretest had + not been passing the result of pcre_study() to pcre_dfa_exec(), so the + study optimizations had never been tested with that matching function. + Oops. What is worse, even when it was passed study data, there was a bug in + pcre_dfa_exec() that meant it never actually used it. Double oops. There + were also very few tests of studied patterns with pcre_dfa_exec(). + +27. If (?| is used to create subpatterns with duplicate numbers, they are now + allowed to have the same name, even if PCRE_DUPNAMES is not set. However, + on the other side of the coin, they are no longer allowed to have different + names, because these cannot be distinguished in PCRE, and this has caused + confusion. (This is a difference from Perl.) + +28. When duplicate subpattern names are present (necessarily with different + numbers, as required by 27 above), and a test is made by name in a + conditional pattern, either for a subpattern having been matched, or for + recursion in such a pattern, all the associated numbered subpatterns are + tested, and the overall condition is true if the condition is true for any + one of them. This is the way Perl works, and is also more like the way + testing by number works. + + Version 7.9 11-Apr-09 --------------------- diff --git a/ext/pcre/pcrelib/HACKING b/ext/pcre/pcrelib/HACKING index 1f30d4c487..8c6a42de39 100644 --- a/ext/pcre/pcrelib/HACKING +++ b/ext/pcre/pcrelib/HACKING @@ -67,22 +67,22 @@ many tests of the mode that might slow it down. So I re-factored the compiling functions to work this way. This got rid of about 600 lines of source. It should make future maintenance and development easier. As this was such a major change, I never released 6.8, instead upping the number to 7.0 (other quite -major changes are also present in the 7.0 release). +major changes were also present in the 7.0 release). -A side effect of this work is that the previous limit of 200 on the nesting +A side effect of this work was that the previous limit of 200 on the nesting depth of parentheses was removed. However, there is a downside: pcre_compile() runs more slowly than before (30% or more, depending on the pattern) because it -is doing a full analysis of the pattern. My hope is that this is not a big -issue. +is doing a full analysis of the pattern. My hope was that this would not be a +big issue, and in the event, nobody has commented on it. Traditional matching function ----------------------------- The "traditional", and original, matching function is called pcre_exec(), and it implements an NFA algorithm, similar to the original Henry Spencer algorithm -and the way that Perl works. Not surprising, since it is intended to be as -compatible with Perl as possible. This is the function most users of PCRE will -use most of the time. +and the way that Perl works. This is not surprising, since it is intended to be +as compatible with Perl as possible. This is the function most users of PCRE +will use most of the time. Supplementary matching function ------------------------------- @@ -119,6 +119,7 @@ quantifiers) are always just two bytes long. A list of the opcodes follows: + Opcodes with no following data ------------------------------ @@ -150,12 +151,12 @@ These items are all just one byte long OP_EXTUNI match an extended Unicode character OP_ANYNL match any Unicode newline sequence - OP_ACCEPT ) - OP_COMMIT ) - OP_FAIL ) These are Perl 5.10's "backtracking - OP_PRUNE ) control verbs". - OP_SKIP ) - OP_THEN ) + OP_ACCEPT ) These are Perl 5.10's "backtracking + OP_COMMIT ) control verbs". If OP_ACCEPT is inside + OP_FAIL ) capturing parentheses, it may be preceded + OP_PRUNE ) by one or more OP_CLOSE, followed by a 2-byte + OP_SKIP ) number, indicating which parentheses must be + OP_THEN ) closed. Repeating single characters @@ -372,12 +373,15 @@ These are like other subpatterns, but they start with the opcode OP_COND, or OP_SCOND for one that might match an empty string in an unbounded repeat. If the condition is a back reference, this is stored at the start of the subpattern using the opcode OP_CREF followed by two bytes containing the -reference number. If the condition is "in recursion" (coded as "(?(R)"), or "in -recursion of group x" (coded as "(?(Rx)"), the group number is stored at the -start of the subpattern using the opcode OP_RREF, and a value of zero for "the -whole pattern". For a DEFINE condition, just the single byte OP_DEF is used (it -has no associated data). Otherwise, a conditional subpattern always starts with -one of the assertions. +reference number. OP_NCREF is used instead if the reference was generated by +name (so that the runtime code knows to check for duplicate names). + +If the condition is "in recursion" (coded as "(?(R)"), or "in recursion of +group x" (coded as "(?(Rx)"), the group number is stored at the start of the +subpattern using the opcode OP_RREF or OP_NRREF (cf OP_NCREF), and a value of +zero for "the whole pattern". For a DEFINE condition, just the single byte +OP_DEF is used (it has no associated data). Otherwise, a conditional subpattern +always starts with one of the assertions. Recursion @@ -415,4 +419,4 @@ at compile time, and so does not cause anything to be put into the compiled data. Philip Hazel -April 2008 +October 2009 diff --git a/ext/pcre/pcrelib/LICENCE b/ext/pcre/pcrelib/LICENCE index ff443a929c..73f8cde3d8 100644 --- a/ext/pcre/pcrelib/LICENCE +++ b/ext/pcre/pcrelib/LICENCE @@ -4,7 +4,7 @@ PCRE LICENCE PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Release 7 of PCRE is distributed under the terms of the "BSD" licence, as +Release 8 of PCRE is distributed under the terms of the "BSD" licence, as specified below. The documentation for PCRE, supplied in the "doc" directory, is distributed under the same terms as the software itself. diff --git a/ext/pcre/pcrelib/NEWS b/ext/pcre/pcrelib/NEWS index 2b26fccf51..71cba379eb 100644 --- a/ext/pcre/pcrelib/NEWS +++ b/ext/pcre/pcrelib/NEWS @@ -1,6 +1,21 @@ News about PCRE releases ------------------------ +Release 8.00 19-Oct-09 +---------------------- + +Bugs have been fixed in the library and in pcregrep. There are also some +enhancements. Restrictions on patterns used for partial matching have been +removed, extra information is given for partial matches, the partial matching +process has been improved, and an option to make a partial match override a +full match is available. The "study" process has been enhanced by finding a +lower bound matching length. Groups with duplicate numbers may now have +duplicated names without the use of PCRE_DUPNAMES. However, they may not have +different names. The documentation has been revised to reflect these changes. +The version number has been expanded to 3 digits as it is clear that the rate +of change is not slowing down. + + Release 7.9 11-Apr-09 --------------------- diff --git a/ext/pcre/pcrelib/NON-UNIX-USE b/ext/pcre/pcrelib/NON-UNIX-USE index 803e73e98b..aca81bd563 100644 --- a/ext/pcre/pcrelib/NON-UNIX-USE +++ b/ext/pcre/pcrelib/NON-UNIX-USE @@ -12,9 +12,10 @@ This document contains the following sections: Comments about Win32 builds Building PCRE on Windows with CMake Use of relative paths with CMake on Windows - Testing with runtest.bat + Testing with RunTest.bat Building under Windows with BCC5.5 Building PCRE on OpenVMS + Building PCRE on Stratus OpenVOS GENERAL @@ -36,10 +37,10 @@ wrapper functions are a separate issue (see below). The PCRE distribution includes a "configure" file for use by the Configure/Make build system, as found in many Unix-like environments. There is also support -support for CMake, which some users prefer, in particular in Windows -environments. There are some instructions for CMake under Windows in the -section entitled "Building PCRE with CMake" below. CMake can also be used to -build PCRE in Unix-like systems. +support for CMake, which some users prefer, especially in Windows environments. +There are some instructions for CMake under Windows in the section entitled +"Building PCRE with CMake" below. CMake can also be used to build PCRE in +Unix-like systems. GENERIC INSTRUCTIONS FOR THE PCRE C LIBRARY @@ -278,40 +279,42 @@ things in this area in future. BUILDING PCRE ON WINDOWS WITH CMAKE -CMake is an alternative build facility that can be used instead of the -traditional Unix "configure". CMake version 2.4.7 supports Borland makefiles, -MinGW makefiles, MSYS makefiles, NMake makefiles, UNIX makefiles, Visual Studio -6, Visual Studio 7, Visual Studio 8, and Watcom W8. The following instructions +CMake is an alternative configuration facility that can be used instead of the +traditional Unix "configure". CMake creates project files (make files, solution +files, etc.) tailored to numerous development environments, including Visual +Studio, Borland, Msys, MinGW, NMake, and Unix. The following instructions were contributed by a PCRE user. -1. Download CMake 2.4.7 or above from http://www.cmake.org/, install and ensure - that cmake\bin is on your path. +1. Install the latest CMake version available from http://www.cmake.org/, and + ensure that cmake\bin is on your path. 2. Unzip (retaining folder structure) the PCRE source tree into a source directory such as C:\pcre. -3. Create a new, empty build directory: C:\pcre\build\ +3. Create a new, empty build directory, for example C:\pcre\build\ -4. Run CMakeSetup from the Shell envirornment of your build tool, e.g., Msys - for Msys/MinGW or Visual Studio Command Prompt for VC/VC++ +4. Run cmake-gui from the Shell envirornment of your build tool, for example, + Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++. 5. Enter C:\pcre\pcre-xx and C:\pcre\build for the source and build - directories, respectively + directories, respectively. 6. Hit the "Configure" button. -7. Select the particular IDE / build tool that you are using (Visual Studio, - MSYS makefiles, MinGW makefiles, etc.) +7. Select the particular IDE / build tool that you are using (Visual + Studio, MSYS makefiles, MinGW makefiles, etc.) -8. The GUI will then list several configuration options. This is where you can - enable UTF-8 support, etc. +8. The GUI will then list several configuration options. This is where + you can enable UTF-8 support or other PCRE optional features. -9. Hit "Configure" again. The adjacent "OK" button should now be active. +9. Hit "Configure" again. The adjacent "Generate" button should now be + active. -10. Hit "OK". +10. Hit "Generate". 11. The build directory should now contain a usable build system, be it a - solution file for Visual Studio, makefiles for MinGW, etc. + solution file for Visual Studio, makefiles for MinGW, etc. Exit from + cmake-gui and use the generated build system with your compiler or IDE. USE OF RELATIVE PATHS WITH CMAKE ON WINDOWS @@ -444,5 +447,52 @@ $! Locale could not be set to fr $! ========================= -Last Updated: 17 March 2009 + +BUILDING PCRE ON STRATUS OPENVOS + +These notes on the port of PCRE to VOS (lightly edited) were supplied by +Ashutosh Warikoo, whose email address has the local part awarikoo and the +domain nse.co.in. The port was for version 7.9 in August 2009. + +1. Building PCRE + +I built pcre on OpenVOS Release 17.0.1at using GNU Tools 3.4a without any +problems. I used the following packages to build PCRE: + + ftp://ftp.stratus.com/pub/vos/posix/ga/posix.save.evf.gz + +Please read and follow the instructions that come with these packages. To start +the build of pcre, from the root of the package type: + + ./build.sh + +2. Installing PCRE + +Once you have successfully built PCRE, login to the SysAdmin group, switch to +the root user, and type + + [ !create_dir (master_disk)>usr --if needed ] + [ !create_dir (master_disk)>usr>local --if needed ] + !gmake install + +This installs PCRE and its man pages into /usr/local. You can add +(master_disk)>usr>local>bin to your command search paths, or if you are in +BASH, add /usr/local/bin to the PATH environment variable. + +4. Restrictions + +This port requires readline library optionally. However during the build I +faced some yet unexplored errors while linking with readline. As it was an +optional component I chose to disable it. + +5. Known Problems + +I ran a the test suite, but you will have to be your own judge of whether this +command, and this port, suits your purposes. If you find any problems that +appear to be related to the port itself, please let me know. Please see the +build.log file in the root of the package also. + + +========================= +Last Updated: 05 October 2009 **** diff --git a/ext/pcre/pcrelib/README b/ext/pcre/pcrelib/README index 6b7c83fed3..c6156e25c1 100644 --- a/ext/pcre/pcrelib/README +++ b/ext/pcre/pcrelib/README @@ -24,6 +24,7 @@ The contents of this README file are: Shared libraries on Unix-like systems Cross-compiling on Unix-like systems Using HP's ANSI C++ compiler (aCC) + Using PCRE from MySQL Making new tarballs Testing PCRE Character tables @@ -111,8 +112,8 @@ Building PCRE on non-Unix systems For a non-Unix system, please read the comments in the file NON-UNIX-USE, though if your system supports the use of "configure" and "make" you may be able to build PCRE in the same way as for Unix-like systems. PCRE can also be -configured in many platform environments using the GUI facility of CMake's -CMakeSetup. It creates Makefiles, solution files, etc. +configured in many platform environments using the GUI facility provided by +CMake's cmake-gui command. This creates Makefiles, solution files, etc. PCRE has been compiled on many different operating systems. It should be straightforward to build PCRE on any system that has a Standard C compiler and @@ -478,6 +479,26 @@ running the "configure" script: CXXLDFLAGS="-lstd_v2 -lCsup_v2" +Using Sun's compilers for Solaris +--------------------------------- + +A user reports that the following configurations work on Solaris 9 sparcv9 and +Solaris 9 x86 (32-bit): + + Solaris 9 sparcv9: ./configure --disable-cpp CC=/bin/cc CFLAGS="-m64 -g" + Solaris 9 x86: ./configure --disable-cpp CC=/bin/cc CFLAGS="-g" + + +Using PCRE from MySQL +--------------------- + +On systems where both PCRE and MySQL are installed, it is possible to make use +of PCRE from within MySQL, as an alternative to the built-in pattern matching. +There is a web page that tells you how to do this: + + http://www.mysqludf.org/lib_mysqludf_preg/index.php + + Making new tarballs ------------------- @@ -553,22 +574,32 @@ document entitled NON-UNIX-USE.] The fourth test checks the UTF-8 support. It is not run automatically unless PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when -running "configure". This file can be also fed directly to the perltest script, -provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch, -commented in the script, can be be used.) +running "configure". This file can be also fed directly to the perltest.pl +script, provided you are running Perl 5.8 or higher. The fifth test checks error handling with UTF-8 encoding, and internal UTF-8 features of PCRE that are not relevant to Perl. -The sixth test checks the support for Unicode character properties. It it not -run automatically unless PCRE is built with Unicode property support. To to -this you must set --enable-unicode-properties when running "configure". +The sixth test (which is Perl-5.10 compatible) checks the support for Unicode +character properties. It it not run automatically unless PCRE is built with +Unicode property support. To to this you must set --enable-unicode-properties +when running "configure". The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode property support, respectively. The eighth and ninth tests are not run automatically unless PCRE is build with the relevant support. +The tenth test checks some internal offsets and code size features; it is run +only when the default "link size" of 2 is set (in other cases the sizes +change). + +The eleventh test checks out features that are new in Perl 5.10, and the +twelfth test checks a number internals and non-Perl features concerned with +Unicode property support. It it not run automatically unless PCRE is built with +Unicode property support. To to this you must set --enable-unicode-properties +when running "configure". + Character tables ---------------- @@ -712,7 +743,7 @@ The distribution should contain the following files: ) "configure" and config.h depcomp ) script to find program dependencies, generated by ) automake - doc/*.3 man page sources for the PCRE functions + doc/*.3 man page sources for PCRE doc/*.1 man page sources for pcregrep and pcretest doc/index.html.src the base HTML page doc/html/* HTML documentation @@ -721,6 +752,7 @@ The distribution should contain the following files: doc/perltest.txt plain text documentation of Perl test program install-sh a shell script for installing files libpcre.pc.in template for libpcre.pc for pkg-config + libpcreposix.pc.in template for libpcreposix.pc for pkg-config libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config ltmain.sh file used to build a libtool script missing ) common stub for a few missing GNU programs while @@ -764,4 +796,4 @@ The distribution should contain the following files: Philip Hazel Email local part: ph10 Email domain: cam.ac.uk -Last updated: 21 March 2009 +Last updated: 19 October 2009 diff --git a/ext/pcre/pcrelib/config.h b/ext/pcre/pcrelib/config.h index 8133d978ba..949a0a209a 100644 --- a/ext/pcre/pcrelib/config.h +++ b/ext/pcre/pcrelib/config.h @@ -196,6 +196,12 @@ them both to 0; an emulation function will be used. */ #define LINK_SIZE 2 #endif +/* Define to the sub-directory in which libtool stores uninstalled libraries. + */ +#ifndef LT_OBJDIR +#define LT_OBJDIR ".libs/" +#endif + /* The value of MATCH_LIMIT determines the default number of times the internal match() function can be called during a single execution of pcre_exec(). There is a runtime interface for setting a different limit. @@ -262,13 +268,13 @@ them both to 0; an emulation function will be used. */ #define PACKAGE_NAME "PCRE" /* Define to the full name and version of this package. */ -#define PACKAGE_STRING "PCRE 7.9" +#define PACKAGE_STRING "PCRE 8.00" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "pcre" /* Define to the version of this package. */ -#define PACKAGE_VERSION "7.9" +#define PACKAGE_VERSION "8.00" /* If you are compiling for a system other than a Unix-like system or @@ -324,7 +330,7 @@ them both to 0; an emulation function will be used. */ /* Version number of package */ #ifndef VERSION -#define VERSION "7.9" +#define VERSION "8.00" #endif /* Define to empty if `const' does not conform to ANSI C. */ diff --git a/ext/pcre/pcrelib/doc/pcre.txt b/ext/pcre/pcrelib/doc/pcre.txt index 9a2ce31598..2ccc7bb4b6 100644 --- a/ext/pcre/pcrelib/doc/pcre.txt +++ b/ext/pcre/pcrelib/doc/pcre.txt @@ -2,8 +2,9 @@ This file contains a concatenation of the PCRE man pages, converted to plain text format for ease of searching with a text editor, or for use on systems that do not have a man page processor. The small individual files that give -synopses of each function in the library have not been included. There are -separate text files for the pcregrep and pcretest commands. +synopses of each function in the library have not been included. Neither has +the pcredemo program. There are separate text files for the pcregrep and +pcretest commands. ----------------------------------------------------------------------------- @@ -18,23 +19,23 @@ INTRODUCTION The PCRE library is a set of functions that implement regular expres- sion pattern matching using the same syntax and semantics as Perl, with - just a few differences. Certain features that appeared in Python and - PCRE before they appeared in Perl are also available using the Python - syntax. There is also some support for certain .NET and Oniguruma syn- - tax items, and there is an option for requesting some minor changes - that give better JavaScript compatibility. - - The current implementation of PCRE (release 7.x) corresponds approxi- - mately with Perl 5.10, including support for UTF-8 encoded strings and - Unicode general category properties. However, UTF-8 and Unicode support - has to be explicitly enabled; it is not the default. The Unicode tables - correspond to Unicode release 5.1. + just a few differences. Some features that appeared in Python and PCRE + before they appeared in Perl are also available using the Python syn- + tax, there is some support for one or two .NET and Oniguruma syntax + items, and there is an option for requesting some minor changes that + give better JavaScript compatibility. + + The current implementation of PCRE corresponds approximately with Perl + 5.10, including support for UTF-8 encoded strings and Unicode general + category properties. However, UTF-8 and Unicode support has to be + explicitly enabled; it is not the default. The Unicode tables corre- + spond to Unicode release 5.1. In addition to the Perl-compatible matching function, PCRE contains an - alternative matching function that matches the same compiled patterns - in a different way. In certain circumstances, the alternative function - has some advantages. For a discussion of the two matching algorithms, - see the pcrematching page. + alternative function that matches the same compiled patterns in a dif- + ferent way. In certain circumstances, the alternative function has some + advantages. For a discussion of the two matching algorithms, see the + pcrematching page. PCRE is written in C and released as a C library. A number of people have written wrappers and interfaces of various kinds. In particular, @@ -54,8 +55,8 @@ INTRODUCTION library is built. The pcre_config() function makes it possible for a client to discover which features are available. The features them- selves are described in the pcrebuild page. Documentation about build- - ing PCRE for various operating systems can be found in the README file - in the source distribution. + ing PCRE for various operating systems can be found in the README and + NON-UNIX-USE files in the source distribution. The library contains a number of undocumented internal functions and data tables that are used by more than one of the exported external @@ -71,8 +72,9 @@ USER DOCUMENTATION The user documentation for PCRE comprises a number of different sec- tions. In the "man" format, each of these is a separate "man page". In the HTML format, each is a separate page, linked from the index page. - In the plain text format, all the sections are concatenated, for ease - of searching. The sections are as follows: + In the plain text format, all the sections, except the pcredemo sec- + tion, are concatenated, for ease of searching. The sections are as fol- + lows: pcre this document pcre-config show PCRE installation configuration information @@ -81,34 +83,35 @@ USER DOCUMENTATION pcrecallout details of the callout feature pcrecompat discussion of Perl compatibility pcrecpp details of the C++ wrapper + pcredemo a demonstration C program that uses PCRE pcregrep description of the pcregrep command pcrematching discussion of the two matching algorithms pcrepartial details of the partial matching facility pcrepattern syntax and semantics of supported regular expressions - pcresyntax quick syntax reference pcreperform discussion of performance issues pcreposix the POSIX-compatible C API pcreprecompile details of saving and re-using precompiled patterns - pcresample discussion of the sample program + pcresample discussion of the pcredemo program pcrestack discussion of stack usage + pcresyntax quick syntax reference pcretest description of the pcretest testing command - In addition, in the "man" and HTML formats, there is a short page for + In addition, in the "man" and HTML formats, there is a short page for each C library function, listing its arguments and results. LIMITATIONS - There are some size limitations in PCRE but it is hoped that they will + There are some size limitations in PCRE but it is hoped that they will never in practice be relevant. - The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE + The maximum length of a compiled pattern is 65539 (sic) bytes if PCRE is compiled with the default internal linkage size of 2. If you want to - process regular expressions that are truly enormous, you can compile - PCRE with an internal linkage size of 3 or 4 (see the README file in - the source distribution and the pcrebuild documentation for details). - In these cases the limit is substantially larger. However, the speed + process regular expressions that are truly enormous, you can compile + PCRE with an internal linkage size of 3 or 4 (see the README file in + the source distribution and the pcrebuild documentation for details). + In these cases the limit is substantially larger. However, the speed of execution is slower. All values in repeating quantifiers must be less than 65536. @@ -119,131 +122,131 @@ LIMITATIONS The maximum length of name for a named subpattern is 32 characters, and the maximum number of named subpatterns is 10000. - The maximum length of a subject string is the largest positive number - that an integer variable can hold. However, when using the traditional + The maximum length of a subject string is the largest positive number + that an integer variable can hold. However, when using the traditional matching function, PCRE uses recursion to handle subpatterns and indef- - inite repetition. This means that the available stack space may limit + inite repetition. This means that the available stack space may limit the size of a subject string that can be processed by certain patterns. For a discussion of stack issues, see the pcrestack documentation. UTF-8 AND UNICODE PROPERTY SUPPORT - From release 3.3, PCRE has had some support for character strings - encoded in the UTF-8 format. For release 4.0 this was greatly extended - to cover most common requirements, and in release 5.0 additional sup- + From release 3.3, PCRE has had some support for character strings + encoded in the UTF-8 format. For release 4.0 this was greatly extended + to cover most common requirements, and in release 5.0 additional sup- port for Unicode general category properties was added. - In order process UTF-8 strings, you must build PCRE to include UTF-8 - support in the code, and, in addition, you must call pcre_compile() - with the PCRE_UTF8 option flag, or the pattern must start with the - sequence (*UTF8). When either of these is the case, both the pattern - and any subject strings that are matched against it are treated as - UTF-8 strings instead of just strings of bytes. + In order process UTF-8 strings, you must build PCRE to include UTF-8 + support in the code, and, in addition, you must call pcre_compile() + with the PCRE_UTF8 option flag, or the pattern must start with the + sequence (*UTF8). When either of these is the case, both the pattern + and any subject strings that are matched against it are treated as + UTF-8 strings instead of strings of 1-byte characters. - If you compile PCRE with UTF-8 support, but do not use it at run time, - the library will be a bit bigger, but the additional run time overhead + If you compile PCRE with UTF-8 support, but do not use it at run time, + the library will be a bit bigger, but the additional run time overhead is limited to testing the PCRE_UTF8 flag occasionally, so should not be very big. If PCRE is built with Unicode character property support (which implies - UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup- + UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup- ported. The available properties that can be tested are limited to the - general category properties such as Lu for an upper case letter or Nd - for a decimal number, the Unicode script names such as Arabic or Han, - and the derived properties Any and L&. A full list is given in the + general category properties such as Lu for an upper case letter or Nd + for a decimal number, the Unicode script names such as Arabic or Han, + and the derived properties Any and L&. A full list is given in the pcrepattern documentation. Only the short names for properties are sup- - ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let- - ter}, is not supported. Furthermore, in Perl, many properties may - optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE + ported. For example, \p{L} matches a letter. Its Perl synonym, \p{Let- + ter}, is not supported. Furthermore, in Perl, many properties may + optionally be prefixed by "Is", for compatibility with Perl 5.6. PCRE does not support this. Validity of UTF-8 strings - When you set the PCRE_UTF8 flag, the strings passed as patterns and + When you set the PCRE_UTF8 flag, the strings passed as patterns and subjects are (by default) checked for validity on entry to the relevant - functions. From release 7.3 of PCRE, the check is according the rules - of RFC 3629, which are themselves derived from the Unicode specifica- - tion. Earlier releases of PCRE followed the rules of RFC 2279, which - allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current + functions. From release 7.3 of PCRE, the check is according the rules + of RFC 3629, which are themselves derived from the Unicode specifica- + tion. Earlier releases of PCRE followed the rules of RFC 2279, which + allows the full range of 31-bit values (0 to 0x7FFFFFFF). The current check allows only values in the range U+0 to U+10FFFF, excluding U+D800 to U+DFFF. - The excluded code points are the "Low Surrogate Area" of Unicode, of - which the Unicode Standard says this: "The Low Surrogate Area does not - contain any character assignments, consequently no character code + The excluded code points are the "Low Surrogate Area" of Unicode, of + which the Unicode Standard says this: "The Low Surrogate Area does not + contain any character assignments, consequently no character code charts or namelists are provided for this area. Surrogates are reserved - for use with UTF-16 and then must be used in pairs." The code points - that are encoded by UTF-16 pairs are available as independent code - points in the UTF-8 encoding. (In other words, the whole surrogate + for use with UTF-16 and then must be used in pairs." The code points + that are encoded by UTF-16 pairs are available as independent code + points in the UTF-8 encoding. (In other words, the whole surrogate thing is a fudge for UTF-16 which unfortunately messes up UTF-8.) - If an invalid UTF-8 string is passed to PCRE, an error return + If an invalid UTF-8 string is passed to PCRE, an error return (PCRE_ERROR_BADUTF8) is given. In some situations, you may already know that your strings are valid, and therefore want to skip these checks in order to improve performance. If you set the PCRE_NO_UTF8_CHECK flag at - compile time or at run time, PCRE assumes that the pattern or subject - it is given (respectively) contains only valid UTF-8 codes. In this + compile time or at run time, PCRE assumes that the pattern or subject + it is given (respectively) contains only valid UTF-8 codes. In this case, it does not diagnose an invalid UTF-8 string. - If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, - what happens depends on why the string is invalid. If the string con- + If you pass an invalid UTF-8 string when PCRE_NO_UTF8_CHECK is set, + what happens depends on why the string is invalid. If the string con- forms to the "old" definition of UTF-8 (RFC 2279), it is processed as a - string of characters in the range 0 to 0x7FFFFFFF. In other words, + string of characters in the range 0 to 0x7FFFFFFF. In other words, apart from the initial validity test, PCRE (when in UTF-8 mode) handles - strings according to the more liberal rules of RFC 2279. However, if - the string does not even conform to RFC 2279, the result is undefined. + strings according to the more liberal rules of RFC 2279. However, if + the string does not even conform to RFC 2279, the result is undefined. Your program may crash. - If you want to process strings of values in the full range 0 to - 0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can + If you want to process strings of values in the full range 0 to + 0x7FFFFFFF, encoded in a UTF-8-like manner as per the old RFC, you can set PCRE_NO_UTF8_CHECK to bypass the more restrictive test. However, in this situation, you will have to apply your own validity check. General comments about UTF-8 mode - 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a + 1. An unbraced hexadecimal escape sequence (such as \xb3) matches a two-byte UTF-8 character if the value is greater than 127. - 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8 + 2. Octal numbers up to \777 are recognized, and match two-byte UTF-8 characters for values greater than \177. - 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi- + 3. Repeat quantifiers apply to complete UTF-8 characters, not to indi- vidual bytes, for example: \x{100}{3}. - 4. The dot metacharacter matches one UTF-8 character instead of a sin- + 4. The dot metacharacter matches one UTF-8 character instead of a sin- gle byte. - 5. The escape sequence \C can be used to match a single byte in UTF-8 - mode, but its use can lead to some strange effects. This facility is + 5. The escape sequence \C can be used to match a single byte in UTF-8 + mode, but its use can lead to some strange effects. This facility is not available in the alternative matching function, pcre_dfa_exec(). - 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly - test characters of any code value, but the characters that PCRE recog- - nizes as digits, spaces, or word characters remain the same set as + 6. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly + test characters of any code value, but the characters that PCRE recog- + nizes as digits, spaces, or word characters remain the same set as before, all with values less than 256. This remains true even when PCRE - includes Unicode property support, because to do otherwise would slow - down PCRE in many common cases. If you really want to test for a wider - sense of, say, "digit", you must use Unicode property tests such as - \p{Nd}. Note that this also applies to \b, because it is defined in + includes Unicode property support, because to do otherwise would slow + down PCRE in many common cases. If you really want to test for a wider + sense of, say, "digit", you must use Unicode property tests such as + \p{Nd}. Note that this also applies to \b, because it is defined in terms of \w and \W. - 7. Similarly, characters that match the POSIX named character classes + 7. Similarly, characters that match the POSIX named character classes are all low-valued characters. - 8. However, the Perl 5.10 horizontal and vertical whitespace matching + 8. However, the Perl 5.10 horizontal and vertical whitespace matching escapes (\h, \H, \v, and \V) do match all the appropriate Unicode char- acters. - 9. Case-insensitive matching applies only to characters whose values - are less than 128, unless PCRE is built with Unicode property support. - Even when Unicode property support is available, PCRE still uses its - own character tables when checking the case of low-valued characters, - so as not to degrade performance. The Unicode property information is + 9. Case-insensitive matching applies only to characters whose values + are less than 128, unless PCRE is built with Unicode property support. + Even when Unicode property support is available, PCRE still uses its + own character tables when checking the case of low-valued characters, + so as not to degrade performance. The Unicode property information is used only for characters with higher values. Even when Unicode property support is available, PCRE supports case-insensitive matching only when - there is a one-to-one mapping between a letter's cases. There are a - small number of many-to-one mappings in Unicode; these are not sup- + there is a one-to-one mapping between a letter's cases. There are a + small number of many-to-one mappings in Unicode; these are not sup- ported by PCRE. @@ -253,14 +256,14 @@ AUTHOR University Computing Service Cambridge CB2 3QH, England. - Putting an actual email address here seems to have been a spam magnet, - so I've taken it away. If you want to email me, use my two initials, + Putting an actual email address here seems to have been a spam magnet, + so I've taken it away. If you want to email me, use my two initials, followed by the two digits 10, at the domain cam.ac.uk. REVISION - Last updated: 11 April 2009 + Last updated: 28 September 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -279,20 +282,25 @@ PCRE BUILD-TIME OPTIONS script, where the optional features are selected or deselected by pro- viding options to configure before running the make command. However, the same options can be selected in both Unix-like and non-Unix-like - environments using the GUI facility of CMakeSetup if you are using - CMake instead of configure to build PCRE. + environments using the GUI facility of cmake-gui if you are using CMake + instead of configure to build PCRE. + + There is a lot more information about building PCRE in non-Unix-like + environments in the file called NON_UNIX_USE, which is part of the PCRE + distribution. You should consult this file as well as the README file + if you are building in a non-Unix-like environment. The complete list of options for configure (which includes the standard - ones such as the selection of the installation directory) can be + ones such as the selection of the installation directory) can be obtained by running ./configure --help - The following sections include descriptions of options whose names + The following sections include descriptions of options whose names begin with --enable or --disable. These settings specify changes to the - defaults for the configure command. Because of the way that configure - works, --enable and --disable always come in pairs, so the complemen- - tary option always exists as well, but as it specifies the default, it + defaults for the configure command. Because of the way that configure + works, --enable and --disable always come in pairs, so the complemen- + tary option always exists as well, but as it specifies the default, it is not described. @@ -313,46 +321,46 @@ UTF-8 SUPPORT --enable-utf8 - to the configure command. Of itself, this does not make PCRE treat - strings as UTF-8. As well as compiling PCRE with this option, you also - have have to set the PCRE_UTF8 option when you call the pcre_compile() - function. + to the configure command. Of itself, this does not make PCRE treat + strings as UTF-8. As well as compiling PCRE with this option, you also + have have to set the PCRE_UTF8 option when you call the pcre_compile() + or pcre_compile2() functions. - If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE + If you set --enable-utf8 when compiling in an EBCDIC environment, PCRE expects its input to be either ASCII or UTF-8 (depending on the runtime - option). It is not possible to support both EBCDIC and UTF-8 codes in - the same version of the library. Consequently, --enable-utf8 and + option). It is not possible to support both EBCDIC and UTF-8 codes in + the same version of the library. Consequently, --enable-utf8 and --enable-ebcdic are mutually exclusive. UNICODE CHARACTER PROPERTY SUPPORT - UTF-8 support allows PCRE to process character values greater than 255 - in the strings that it handles. On its own, however, it does not pro- + UTF-8 support allows PCRE to process character values greater than 255 + in the strings that it handles. On its own, however, it does not pro- vide any facilities for accessing the properties of such characters. If - you want to be able to use the pattern escapes \P, \p, and \X, which + you want to be able to use the pattern escapes \P, \p, and \X, which refer to Unicode character properties, you must add --enable-unicode-properties - to the configure command. This implies UTF-8 support, even if you have + to the configure command. This implies UTF-8 support, even if you have not explicitly requested it. - Including Unicode property support adds around 30K of tables to the - PCRE library. Only the general category properties such as Lu and Nd + Including Unicode property support adds around 30K of tables to the + PCRE library. Only the general category properties such as Lu and Nd are supported. Details are given in the pcrepattern documentation. CODE VALUE OF NEWLINE - By default, PCRE interprets the linefeed (LF) character as indicating - the end of a line. This is the normal newline character on Unix-like - systems. You can compile PCRE to use carriage return (CR) instead, by + By default, PCRE interprets the linefeed (LF) character as indicating + the end of a line. This is the normal newline character on Unix-like + systems. You can compile PCRE to use carriage return (CR) instead, by adding --enable-newline-is-cr - to the configure command. There is also a --enable-newline-is-lf + to the configure command. There is also a --enable-newline-is-lf option, which explicitly specifies linefeed as the newline character. Alternatively, you can specify that line endings are to be indicated by @@ -364,35 +372,35 @@ CODE VALUE OF NEWLINE --enable-newline-is-anycrlf - which causes PCRE to recognize any of the three sequences CR, LF, or + which causes PCRE to recognize any of the three sequences CR, LF, or CRLF as indicating a line ending. Finally, a fifth option, specified by --enable-newline-is-any causes PCRE to recognize any Unicode newline sequence. - Whatever line ending convention is selected when PCRE is built can be - overridden when the library functions are called. At build time it is + Whatever line ending convention is selected when PCRE is built can be + overridden when the library functions are called. At build time it is conventional to use the standard for your operating system. WHAT \R MATCHES - By default, the sequence \R in a pattern matches any Unicode newline - sequence, whatever has been selected as the line ending sequence. If + By default, the sequence \R in a pattern matches any Unicode newline + sequence, whatever has been selected as the line ending sequence. If you specify --enable-bsr-anycrlf - the default is changed so that \R matches only CR, LF, or CRLF. What- - ever is selected when PCRE is built can be overridden when the library + the default is changed so that \R matches only CR, LF, or CRLF. What- + ever is selected when PCRE is built can be overridden when the library functions are called. BUILDING SHARED AND STATIC LIBRARIES - The PCRE building process uses libtool to build both shared and static - Unix libraries by default. You can suppress one of these by adding one + The PCRE building process uses libtool to build both shared and static + Unix libraries by default. You can suppress one of these by adding one of --disable-shared @@ -404,9 +412,9 @@ BUILDING SHARED AND STATIC LIBRARIES POSIX MALLOC USAGE When PCRE is called through the POSIX interface (see the pcreposix doc- - umentation), additional working storage is required for holding the - pointers to capturing substrings, because PCRE requires three integers - per substring, whereas the POSIX interface provides only two. If the + umentation), additional working storage is required for holding the + pointers to capturing substrings, because PCRE requires three integers + per substring, whereas the POSIX interface provides only two. If the number of expected substrings is small, the wrapper function uses space on the stack, because this is faster than using malloc() for each call. The default threshold above which the stack is no longer used is 10; it @@ -419,50 +427,49 @@ POSIX MALLOC USAGE HANDLING VERY LARGE PATTERNS - Within a compiled pattern, offset values are used to point from one - part to another (for example, from an opening parenthesis to an alter- - nation metacharacter). By default, two-byte values are used for these - offsets, leading to a maximum size for a compiled pattern of around - 64K. This is sufficient to handle all but the most gigantic patterns. - Nevertheless, some people do want to process enormous patterns, so it - is possible to compile PCRE to use three-byte or four-byte offsets by - adding a setting such as + Within a compiled pattern, offset values are used to point from one + part to another (for example, from an opening parenthesis to an alter- + nation metacharacter). By default, two-byte values are used for these + offsets, leading to a maximum size for a compiled pattern of around + 64K. This is sufficient to handle all but the most gigantic patterns. + Nevertheless, some people do want to process truyl enormous patterns, + so it is possible to compile PCRE to use three-byte or four-byte off- + sets by adding a setting such as --with-link-size=3 - to the configure command. The value given must be 2, 3, or 4. Using - longer offsets slows down the operation of PCRE because it has to load + to the configure command. The value given must be 2, 3, or 4. Using + longer offsets slows down the operation of PCRE because it has to load additional bytes when handling them. AVOIDING EXCESSIVE STACK USAGE When matching with the pcre_exec() function, PCRE implements backtrack- - ing by making recursive calls to an internal function called match(). - In environments where the size of the stack is limited, this can se- - verely limit PCRE's operation. (The Unix environment does not usually + ing by making recursive calls to an internal function called match(). + In environments where the size of the stack is limited, this can se- + verely limit PCRE's operation. (The Unix environment does not usually suffer from this problem, but it may sometimes be necessary to increase - the maximum stack size. There is a discussion in the pcrestack docu- - mentation.) An alternative approach to recursion that uses memory from - the heap to remember data, instead of using recursive function calls, - has been implemented to work round the problem of limited stack size. + the maximum stack size. There is a discussion in the pcrestack docu- + mentation.) An alternative approach to recursion that uses memory from + the heap to remember data, instead of using recursive function calls, + has been implemented to work round the problem of limited stack size. If you want to build a version of PCRE that works this way, add --disable-stack-for-recursion - to the configure command. With this configuration, PCRE will use the - pcre_stack_malloc and pcre_stack_free variables to call memory manage- - ment functions. By default these point to malloc() and free(), but you - can replace the pointers so that your own functions are used. + to the configure command. With this configuration, PCRE will use the + pcre_stack_malloc and pcre_stack_free variables to call memory manage- + ment functions. By default these point to malloc() and free(), but you + can replace the pointers so that your own functions are used instead. - Separate functions are provided rather than using pcre_malloc and - pcre_free because the usage is very predictable: the block sizes - requested are always the same, and the blocks are always freed in - reverse order. A calling program might be able to implement optimized - functions that perform better than malloc() and free(). PCRE runs + Separate functions are provided rather than using pcre_malloc and + pcre_free because the usage is very predictable: the block sizes + requested are always the same, and the blocks are always freed in + reverse order. A calling program might be able to implement optimized + functions that perform better than malloc() and free(). PCRE runs noticeably more slowly when built in this way. This option affects only - the pcre_exec() function; it is not relevant for the the - pcre_dfa_exec() function. + the pcre_exec() function; it is not relevant for pcre_dfa_exec(). LIMITING PCRE RESOURCE USAGE @@ -551,7 +558,7 @@ PCRETEST OPTION FOR LIBREADLINE SUPPORT to the configure command, pcretest is linked with the libreadline library, and when its input is from a terminal, it reads it using the readline() function. This provides line-editing and history facilities. - Note that libreadline is GPL-licenced, so if you distribute a binary of + Note that libreadline is GPL-licensed, so if you distribute a binary of pcretest linked in this way, there may be licensing issues. Setting this option causes the -lreadline option to be added to the @@ -587,7 +594,7 @@ AUTHOR REVISION - Last updated: 17 March 2009 + Last updated: 29 September 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -675,13 +682,19 @@ THE ALTERNATIVE MATCHING ALGORITHM though it is not implemented as a traditional finite state machine (it keeps multiple states active simultaneously). + Although the general principle of this matching algorithm is that it + scans the subject string only once, without backtracking, there is one + exception: when a lookaround assertion is encountered, the characters + following or preceding the current point have to be independently + inspected. + The scan continues until either the end of the subject is reached, or there are no more unterminated paths. At this point, terminated paths represent the different matching possibilities (if there are none, the match has failed). Thus, if there is more than one possible match, this algorithm finds all of them, and in particular, it finds the long- - est. In PCRE, there is an option to stop the algorithm after the first - match (which is necessarily the shortest) has been found. + est. There is an option to stop the algorithm after the first match + (which is necessarily the shortest) is found. Note that all the matches that are found start at the same point in the subject. If the pattern @@ -751,24 +764,19 @@ ADVANTAGES OF THE ALTERNATIVE ALGORITHM more than one match using the standard algorithm, you have to do kludgy things with callouts. - 2. There is much better support for partial matching. The restrictions - on the content of the pattern that apply when using the standard algo- - rithm for partial matching do not apply to the alternative algorithm. - For non-anchored patterns, the starting position of a partial match is - available. - - 3. Because the alternative algorithm scans the subject string just + 2. Because the alternative algorithm scans the subject string just once, and never needs to backtrack, it is possible to pass very long subject strings to the matching function in several pieces, checking - for partial matching each time. + for partial matching each time. The pcrepartial documentation gives + details of partial matching. DISADVANTAGES OF THE ALTERNATIVE ALGORITHM The alternative algorithm suffers from a number of disadvantages: - 1. It is substantially slower than the standard algorithm. This is - partly because it has to search for all possible matches, but is also + 1. It is substantially slower than the standard algorithm. This is + partly because it has to search for all possible matches, but is also because it is less susceptible to optimization. 2. Capturing parentheses and back references are not supported. @@ -786,8 +794,8 @@ AUTHOR REVISION - Last updated: 19 April 2008 - Copyright (c) 1997-2008 University of Cambridge. + Last updated: 29 September 2009 + Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -898,16 +906,18 @@ PCRE API OVERVIEW pcre_exec() are used for compiling and matching regular expressions in a Perl-compatible manner. A sample program that demonstrates the sim- plest way of using them is provided in the file called pcredemo.c in - the source distribution. The pcresample documentation describes how to - compile and run it. + the PCRE source distribution. A listing of this program is given in the + pcredemo documentation, and the pcresample documentation describes how + to compile and run it. A second matching function, pcre_dfa_exec(), which is not Perl-compati- - ble, is also provided. This uses a different algorithm for the match- - ing. The alternative algorithm finds all possible matches (at a given - point in the subject), and scans the subject just once. However, this - algorithm does not return captured substrings. A description of the two - matching algorithms and their advantages and disadvantages is given in - the pcrematching documentation. + ble, is also provided. This uses a different algorithm for the match- + ing. The alternative algorithm finds all possible matches (at a given + point in the subject), and scans the subject just once (unless there + are lookbehind assertions). However, this algorithm does not return + captured substrings. A description of the two matching algorithms and + their advantages and disadvantages is given in the pcrematching docu- + mentation. In addition to the main compiling and matching functions, there are convenience functions for extracting captured substrings from a subject @@ -1117,7 +1127,9 @@ COMPILING A PATTERN Either of the functions pcre_compile() or pcre_compile2() can be called to compile a pattern into an internal form. The only difference between the two interfaces is that pcre_compile2() has an additional argument, - errorcodeptr, via which a numerical error code can be returned. + errorcodeptr, via which a numerical error code can be returned. To + avoid too much repetition, we refer just to pcre_compile() below, but + the information applies equally to pcre_compile2(). The pattern is a C string terminated by a binary zero, and is passed in the pattern argument. A pointer to a single block of memory that is @@ -1135,22 +1147,24 @@ COMPILING A PATTERN The options argument contains various bit settings that affect the com- pilation. It should be zero if no options are required. The available options are described below. Some of them (in particular, those that - are compatible with Perl, but also some others) can also be set and + are compatible with Perl, but some others as well) can also be set and unset from within the pattern (see the detailed description in the pcrepattern documentation). For those options that can be different in different parts of the pattern, the contents of the options argument - specifies their initial settings at the start of compilation and execu- - tion. The PCRE_ANCHORED and PCRE_NEWLINE_xxx options can be set at the - time of matching as well as at compile time. + specifies their settings at the start of compilation and execution. The + PCRE_ANCHORED, PCRE_BSR_xxx, and PCRE_NEWLINE_xxx options can be set at + the time of matching as well as at compile time. If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, if compilation of a pattern fails, pcre_compile() returns NULL, and sets the variable pointed to by errptr to point to a textual error mes- sage. This is a static string that is part of the library. You must not - try to free it. The offset from the start of the pattern to the charac- - ter where the error was discovered is placed in the variable pointed to - by erroffset, which must not be NULL. If it is, an immediate error is - given. + try to free it. The byte offset from the start of the pattern to the + character that was being processed when the error was discovered is + placed in the variable pointed to by erroffset, which must not be NULL. + If it is, an immediate error is given. Some errors are not detected + until checks are carried out when the whole pattern has been scanned; + in this case the offset is set to the end of the pattern. If pcre_compile2() is used instead of pcre_compile(), and the error- codeptr argument is not NULL, a non-zero error code number is returned @@ -1480,14 +1494,14 @@ STUDYING A PATTERN the results of the study. The returned value from pcre_study() can be passed directly to - pcre_exec(). However, a pcre_extra block also contains other fields - that can be set by the caller before the block is passed; these are - described below in the section on matching a pattern. + pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- + tains other fields that can be set by the caller before the block is + passed; these are described below in the section on matching a pattern. - If studying the pattern does not produce any additional information + If studying the pattern does not produce any useful information, pcre_study() returns NULL. In that circumstance, if the calling program - wants to pass any of the other fields to pcre_exec(), it must set up - its own pcre_extra block. + wants to pass any of the other fields to pcre_exec() or + pcre_dfa_exec(), it must set up its own pcre_extra block. The second argument of pcre_study() contains option bits. At present, no options are defined, and this argument should always be zero. @@ -1507,63 +1521,72 @@ STUDYING A PATTERN 0, /* no options exist */ &error); /* set to NULL or points to a message */ - At present, studying a pattern is useful only for non-anchored patterns - that do not have a single fixed starting character. A bitmap of possi- - ble starting bytes is created. + Studying a pattern does two things: first, a lower bound for the length + of subject string that is needed to match the pattern is computed. This + does not mean that there are any strings of that length that match, but + it does guarantee that no shorter strings match. The value is used by + pcre_exec() and pcre_dfa_exec() to avoid wasting time by trying to + match strings that are shorter than the lower bound. You can find out + the value in a calling program via the pcre_fullinfo() function. + + Studying a pattern is also useful for non-anchored patterns that do not + have a single fixed starting character. A bitmap of possible starting + bytes is created. This speeds up finding a position in the subject at + which to start matching. LOCALE SUPPORT - PCRE handles caseless matching, and determines whether characters are - letters, digits, or whatever, by reference to a set of tables, indexed - by character value. When running in UTF-8 mode, this applies only to - characters with codes less than 128. Higher-valued codes never match - escapes such as \w or \d, but can be tested with \p if PCRE is built - with Unicode character property support. The use of locales with Uni- - code is discouraged. If you are handling characters with codes greater - than 128, you should either use UTF-8 and Unicode, or use locales, but + PCRE handles caseless matching, and determines whether characters are + letters, digits, or whatever, by reference to a set of tables, indexed + by character value. When running in UTF-8 mode, this applies only to + characters with codes less than 128. Higher-valued codes never match + escapes such as \w or \d, but can be tested with \p if PCRE is built + with Unicode character property support. The use of locales with Uni- + code is discouraged. If you are handling characters with codes greater + than 128, you should either use UTF-8 and Unicode, or use locales, but not try to mix the two. - PCRE contains an internal set of tables that are used when the final - argument of pcre_compile() is NULL. These are sufficient for many + PCRE contains an internal set of tables that are used when the final + argument of pcre_compile() is NULL. These are sufficient for many applications. Normally, the internal tables recognize only ASCII char- acters. However, when PCRE is built, it is possible to cause the inter- nal tables to be rebuilt in the default "C" locale of the local system, which may cause them to be different. - The internal tables can always be overridden by tables supplied by the + The internal tables can always be overridden by tables supplied by the application that calls PCRE. These may be created in a different locale - from the default. As more and more applications change to using Uni- + from the default. As more and more applications change to using Uni- code, the need for this locale support is expected to die away. - External tables are built by calling the pcre_maketables() function, - which has no arguments, in the relevant locale. The result can then be - passed to pcre_compile() or pcre_exec() as often as necessary. For - example, to build and use tables that are appropriate for the French - locale (where accented characters with values greater than 128 are + External tables are built by calling the pcre_maketables() function, + which has no arguments, in the relevant locale. The result can then be + passed to pcre_compile() or pcre_exec() as often as necessary. For + example, to build and use tables that are appropriate for the French + locale (where accented characters with values greater than 128 are treated as letters), the following code could be used: setlocale(LC_CTYPE, "fr_FR"); tables = pcre_maketables(); re = pcre_compile(..., tables); - The locale name "fr_FR" is used on Linux and other Unix-like systems; + The locale name "fr_FR" is used on Linux and other Unix-like systems; if you are using Windows, the name for the French locale is "french". - When pcre_maketables() runs, the tables are built in memory that is - obtained via pcre_malloc. It is the caller's responsibility to ensure - that the memory containing the tables remains available for as long as + When pcre_maketables() runs, the tables are built in memory that is + obtained via pcre_malloc. It is the caller's responsibility to ensure + that the memory containing the tables remains available for as long as it is needed. The pointer that is passed to pcre_compile() is saved with the compiled - pattern, and the same tables are used via this pointer by pcre_study() + pattern, and the same tables are used via this pointer by pcre_study() and normally also by pcre_exec(). Thus, by default, for any single pat- tern, compilation, studying and matching all happen in the same locale, but different patterns can be compiled in different locales. - It is possible to pass a table pointer or NULL (indicating the use of - the internal tables) to pcre_exec(). Although not intended for this - purpose, this facility could be used to match a pattern in a different + It is possible to pass a table pointer or NULL (indicating the use of + the internal tables) to pcre_exec(). Although not intended for this + purpose, this facility could be used to match a pattern in a different locale from the one in which it was compiled. Passing table pointers at run time is discussed below in the section on matching a pattern. @@ -1573,15 +1596,15 @@ INFORMATION ABOUT A PATTERN int pcre_fullinfo(const pcre *code, const pcre_extra *extra, int what, void *where); - The pcre_fullinfo() function returns information about a compiled pat- + The pcre_fullinfo() function returns information about a compiled pat- tern. It replaces the obsolete pcre_info() function, which is neverthe- less retained for backwards compability (and is documented below). - The first argument for pcre_fullinfo() is a pointer to the compiled - pattern. The second argument is the result of pcre_study(), or NULL if - the pattern was not studied. The third argument specifies which piece - of information is required, and the fourth argument is a pointer to a - variable to receive the data. The yield of the function is zero for + The first argument for pcre_fullinfo() is a pointer to the compiled + pattern. The second argument is the result of pcre_study(), or NULL if + the pattern was not studied. The third argument specifies which piece + of information is required, and the fourth argument is a pointer to a + variable to receive the data. The yield of the function is zero for success, or one of the following negative numbers: PCRE_ERROR_NULL the argument code was NULL @@ -1589,9 +1612,9 @@ INFORMATION ABOUT A PATTERN PCRE_ERROR_BADMAGIC the "magic number" was not found PCRE_ERROR_BADOPTION the value of what was invalid - The "magic number" is placed at the start of each compiled pattern as - an simple check against passing an arbitrary memory pointer. Here is a - typical call of pcre_fullinfo(), to obtain the length of the compiled + The "magic number" is placed at the start of each compiled pattern as + an simple check against passing an arbitrary memory pointer. Here is a + typical call of pcre_fullinfo(), to obtain the length of the compiled pattern: int rc; @@ -1602,111 +1625,131 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE, /* what is required */ &length); /* where to put the data */ - The possible values for the third argument are defined in pcre.h, and + The possible values for the third argument are defined in pcre.h, and are as follows: PCRE_INFO_BACKREFMAX - Return the number of the highest back reference in the pattern. The - fourth argument should point to an int variable. Zero is returned if + Return the number of the highest back reference in the pattern. The + fourth argument should point to an int variable. Zero is returned if there are no back references. PCRE_INFO_CAPTURECOUNT - Return the number of capturing subpatterns in the pattern. The fourth + Return the number of capturing subpatterns in the pattern. The fourth argument should point to an int variable. PCRE_INFO_DEFAULT_TABLES - Return a pointer to the internal default character tables within PCRE. - The fourth argument should point to an unsigned char * variable. This + Return a pointer to the internal default character tables within PCRE. + The fourth argument should point to an unsigned char * variable. This information call is provided for internal use by the pcre_study() func- - tion. External callers can cause PCRE to use its internal tables by + tion. External callers can cause PCRE to use its internal tables by passing a NULL table pointer. PCRE_INFO_FIRSTBYTE - Return information about the first byte of any matched string, for a - non-anchored pattern. The fourth argument should point to an int vari- - able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name + Return information about the first byte of any matched string, for a + non-anchored pattern. The fourth argument should point to an int vari- + able. (This option used to be called PCRE_INFO_FIRSTCHAR; the old name is still recognized for backwards compatibility.) - If there is a fixed first byte, for example, from a pattern such as + If there is a fixed first byte, for example, from a pattern such as (cat|cow|coyote), its value is returned. Otherwise, if either - (a) the pattern was compiled with the PCRE_MULTILINE option, and every + (a) the pattern was compiled with the PCRE_MULTILINE option, and every branch starts with "^", or (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set (if it were set, the pattern would be anchored), - -1 is returned, indicating that the pattern matches only at the start - of a subject string or after any newline within the string. Otherwise + -1 is returned, indicating that the pattern matches only at the start + of a subject string or after any newline within the string. Otherwise -2 is returned. For anchored patterns, -2 is returned. PCRE_INFO_FIRSTTABLE - If the pattern was studied, and this resulted in the construction of a + If the pattern was studied, and this resulted in the construction of a 256-bit table indicating a fixed set of bytes for the first byte in any - matching string, a pointer to the table is returned. Otherwise NULL is - returned. The fourth argument should point to an unsigned char * vari- + matching string, a pointer to the table is returned. Otherwise NULL is + returned. The fourth argument should point to an unsigned char * vari- able. PCRE_INFO_HASCRORLF - Return 1 if the pattern contains any explicit matches for CR or LF - characters, otherwise 0. The fourth argument should point to an int - variable. An explicit match is either a literal CR or LF character, or + Return 1 if the pattern contains any explicit matches for CR or LF + characters, otherwise 0. The fourth argument should point to an int + variable. An explicit match is either a literal CR or LF character, or \r or \n. PCRE_INFO_JCHANGED - Return 1 if the (?J) or (?-J) option setting is used in the pattern, - otherwise 0. The fourth argument should point to an int variable. (?J) + Return 1 if the (?J) or (?-J) option setting is used in the pattern, + otherwise 0. The fourth argument should point to an int variable. (?J) and (?-J) set and unset the local PCRE_DUPNAMES option, respectively. PCRE_INFO_LASTLITERAL - Return the value of the rightmost literal byte that must exist in any - matched string, other than at its start, if such a byte has been + Return the value of the rightmost literal byte that must exist in any + matched string, other than at its start, if such a byte has been recorded. The fourth argument should point to an int variable. If there - is no such byte, -1 is returned. For anchored patterns, a last literal - byte is recorded only if it follows something of variable length. For + is no such byte, -1 is returned. For anchored patterns, a last literal + byte is recorded only if it follows something of variable length. For example, for the pattern /^a\d+z\d+/ the returned value is "z", but for /^a\dz\d/ the returned value is -1. + PCRE_INFO_MINLENGTH + + If the pattern was studied and a minimum length for matching subject + strings was computed, its value is returned. Otherwise the returned + value is -1. The value is a number of characters, not bytes (this may + be relevant in UTF-8 mode). The fourth argument should point to an int + variable. A non-negative value is a lower bound to the length of any + matching string. There may not be any strings of that length that do + actually match, but every string that does match is at least that long. + PCRE_INFO_NAMECOUNT PCRE_INFO_NAMEENTRYSIZE PCRE_INFO_NAMETABLE - PCRE supports the use of named as well as numbered capturing parenthe- - ses. The names are just an additional way of identifying the parenthe- + PCRE supports the use of named as well as numbered capturing parenthe- + ses. The names are just an additional way of identifying the parenthe- ses, which still acquire numbers. Several convenience functions such as - pcre_get_named_substring() are provided for extracting captured sub- - strings by name. It is also possible to extract the data directly, by - first converting the name to a number in order to access the correct + pcre_get_named_substring() are provided for extracting captured sub- + strings by name. It is also possible to extract the data directly, by + first converting the name to a number in order to access the correct pointers in the output vector (described with pcre_exec() below). To do - the conversion, you need to use the name-to-number map, which is + the conversion, you need to use the name-to-number map, which is described by these three values. The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size - of each entry; both of these return an int value. The entry size - depends on the length of the longest name. PCRE_INFO_NAMETABLE returns - a pointer to the first entry of the table (a pointer to char). The + of each entry; both of these return an int value. The entry size + depends on the length of the longest name. PCRE_INFO_NAMETABLE returns + a pointer to the first entry of the table (a pointer to char). The first two bytes of each entry are the number of the capturing parenthe- - sis, most significant byte first. The rest of the entry is the corre- - sponding name, zero terminated. The names are in alphabetical order. - When PCRE_DUPNAMES is set, duplicate names are in order of their paren- - theses numbers. For example, consider the following pattern (assume - PCRE_EXTENDED is set, so white space - including newlines - is - ignored): + sis, most significant byte first. The rest of the entry is the corre- + sponding name, zero terminated. + + The names are in alphabetical order. Duplicate names may appear if (?| + is used to create multiple groups with the same number, as described in + the section on duplicate subpattern numbers in the pcrepattern page. + Duplicate names for subpatterns with different numbers are permitted + only if PCRE_DUPNAMES is set. In all cases of duplicate names, they + appear in the table in the order in which they were found in the pat- + tern. In the absence of (?| this is the order of increasing number; + when (?| is used this is not necessarily the case because later subpat- + terns may have lower numbers. + + As a simple example of the name/number table, consider the following + pattern (assume PCRE_EXTENDED is set, so white space - including new- + lines - is ignored): (?<date> (?<year>(\d\d)?\d\d) - (?<month>\d\d) - (?<day>\d\d) ) - There are four named subpatterns, so the table has four entries, and - each entry in the table is eight bytes long. The table is as follows, + There are four named subpatterns, so the table has four entries, and + each entry in the table is eight bytes long. The table is as follows, with non-printing bytes shows in hexadecimal, and undefined bytes shown as ??: @@ -1715,29 +1758,31 @@ INFORMATION ABOUT A PATTERN 00 04 m o n t h 00 00 02 y e a r 00 ?? - When writing code to extract data from named subpatterns using the - name-to-number map, remember that the length of the entries is likely + When writing code to extract data from named subpatterns using the + name-to-number map, remember that the length of the entries is likely to be different for each compiled pattern. PCRE_INFO_OKPARTIAL - Return 1 if the pattern can be used for partial matching, otherwise 0. - The fourth argument should point to an int variable. The pcrepartial - documentation lists the restrictions that apply to patterns when par- - tial matching is used. + Return 1 if the pattern can be used for partial matching with + pcre_exec(), otherwise 0. The fourth argument should point to an int + variable. From release 8.00, this always returns 1, because the + restrictions that previously applied to partial matching have been + lifted. The pcrepartial documentation gives details of partial match- + ing. PCRE_INFO_OPTIONS - Return a copy of the options with which the pattern was compiled. The - fourth argument should point to an unsigned long int variable. These + Return a copy of the options with which the pattern was compiled. The + fourth argument should point to an unsigned long int variable. These option bits are those specified in the call to pcre_compile(), modified by any top-level option settings at the start of the pattern itself. In - other words, they are the options that will be in force when matching - starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with - the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, + other words, they are the options that will be in force when matching + starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with + the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, and PCRE_EXTENDED. - A pattern is automatically anchored by PCRE if all of its top-level + A pattern is automatically anchored by PCRE if all of its top-level alternatives begin with one of the following: ^ unless PCRE_MULTILINE is set @@ -1751,7 +1796,7 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE - Return the size of the compiled pattern, that is, the value that was + Return the size of the compiled pattern, that is, the value that was passed as the argument to pcre_malloc() when PCRE was getting memory in which to place the compiled data. The fourth argument should point to a size_t variable. @@ -1759,9 +1804,10 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_STUDYSIZE Return the size of the data block pointed to by the study_data field in - a pcre_extra block. That is, it is the value that was passed to + a pcre_extra block. That is, it is the value that was passed to pcre_malloc() when PCRE was getting memory into which to place the data - created by pcre_study(). The fourth argument should point to a size_t + created by pcre_study(). If pcre_extra is NULL, or there is no study + data, zero is returned. The fourth argument should point to a size_t variable. @@ -1817,7 +1863,7 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION The function pcre_exec() is called to match a subject string against a compiled pattern, which is passed in the code argument. If the pattern - has been studied, the result of the study should be passed in the extra + was studied, the result of the study should be passed in the extra argument. This function is the main matching facility of the library, and it operates in a Perl-like manner. For specialist use there is also an alternative matching function, which is described below in the sec- @@ -1876,8 +1922,8 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION The match_limit field provides a means of preventing PCRE from using up a vast amount of resources when running patterns that are not going to match, but which have a very large number of possibilities in their - search trees. The classic example is the use of nested unlimited - repeats. + search trees. The classic example is a pattern that uses nested unlim- + ited repeats. Internally, PCRE uses a function called match() which it calls repeat- edly (sometimes recursively). The limit set by match_limit is imposed @@ -1910,8 +1956,8 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_EXTRA_MATCH_LIMIT_RECURSION is set in the flags field. If the limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. - The pcre_callout field is used in conjunction with the "callout" fea- - ture, which is described in the pcrecallout documentation. + The callout_data field is used in conjunction with the "callout" fea- + ture, and is described in the pcrecallout documentation. The tables field is used to pass a character tables pointer to pcre_exec(); this overrides the value that is stored with the compiled @@ -1928,22 +1974,23 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION The unused bits of the options argument for pcre_exec() must be zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, - PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_START_OPTIMIZE, - PCRE_NO_UTF8_CHECK and PCRE_PARTIAL. + PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, + PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and + PCRE_PARTIAL_HARD. PCRE_ANCHORED - The PCRE_ANCHORED option limits pcre_exec() to matching at the first - matching position. If a pattern was compiled with PCRE_ANCHORED, or - turned out to be anchored by virtue of its contents, it cannot be made + The PCRE_ANCHORED option limits pcre_exec() to matching at the first + matching position. If a pattern was compiled with PCRE_ANCHORED, or + turned out to be anchored by virtue of its contents, it cannot be made unachored at matching time. PCRE_BSR_ANYCRLF PCRE_BSR_UNICODE These options (which are mutually exclusive) control what the \R escape - sequence matches. The choice is either to match only CR, LF, or CRLF, - or to match any Unicode newline sequence. These options override the + sequence matches. The choice is either to match only CR, LF, or CRLF, + or to match any Unicode newline sequence. These options override the choice that was made or defaulted when the pattern was compiled. PCRE_NEWLINE_CR @@ -1952,76 +1999,83 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_NEWLINE_ANYCRLF PCRE_NEWLINE_ANY - These options override the newline definition that was chosen or - defaulted when the pattern was compiled. For details, see the descrip- - tion of pcre_compile() above. During matching, the newline choice - affects the behaviour of the dot, circumflex, and dollar metacharac- - ters. It may also alter the way the match position is advanced after a + These options override the newline definition that was chosen or + defaulted when the pattern was compiled. For details, see the descrip- + tion of pcre_compile() above. During matching, the newline choice + affects the behaviour of the dot, circumflex, and dollar metacharac- + ters. It may also alter the way the match position is advanced after a match failure for an unanchored pattern. - When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is - set, and a match attempt for an unanchored pattern fails when the cur- - rent position is at a CRLF sequence, and the pattern contains no - explicit matches for CR or LF characters, the match position is + When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is + set, and a match attempt for an unanchored pattern fails when the cur- + rent position is at a CRLF sequence, and the pattern contains no + explicit matches for CR or LF characters, the match position is advanced by two characters instead of one, in other words, to after the CRLF. The above rule is a compromise that makes the most common cases work as - expected. For example, if the pattern is .+A (and the PCRE_DOTALL + expected. For example, if the pattern is .+A (and the PCRE_DOTALL option is not set), it does not match the string "\r\nA" because, after - failing at the start, it skips both the CR and the LF before retrying. - However, the pattern [\r\n]A does match that string, because it con- + failing at the start, it skips both the CR and the LF before retrying. + However, the pattern [\r\n]A does match that string, because it con- tains an explicit CR or LF reference, and so advances only by one char- acter after the first failure. An explicit match for CR of LF is either a literal appearance of one of - those characters, or one of the \r or \n escape sequences. Implicit - matches such as [^X] do not count, nor does \s (which includes CR and + those characters, or one of the \r or \n escape sequences. Implicit + matches such as [^X] do not count, nor does \s (which includes CR and LF in the characters that it matches). - Notwithstanding the above, anomalous effects may still occur when CRLF + Notwithstanding the above, anomalous effects may still occur when CRLF is a valid newline sequence and explicit \r or \n escapes appear in the pattern. PCRE_NOTBOL This option specifies that first character of the subject string is not - the beginning of a line, so the circumflex metacharacter should not - match before it. Setting this without PCRE_MULTILINE (at compile time) - causes circumflex never to match. This option affects only the behav- + the beginning of a line, so the circumflex metacharacter should not + match before it. Setting this without PCRE_MULTILINE (at compile time) + causes circumflex never to match. This option affects only the behav- iour of the circumflex metacharacter. It does not affect \A. PCRE_NOTEOL This option specifies that the end of the subject string is not the end - of a line, so the dollar metacharacter should not match it nor (except - in multiline mode) a newline immediately before it. Setting this with- + of a line, so the dollar metacharacter should not match it nor (except + in multiline mode) a newline immediately before it. Setting this with- out PCRE_MULTILINE (at compile time) causes dollar never to match. This - option affects only the behaviour of the dollar metacharacter. It does + option affects only the behaviour of the dollar metacharacter. It does not affect \Z or \z. PCRE_NOTEMPTY An empty string is not considered to be a valid match if this option is - set. If there are alternatives in the pattern, they are tried. If all - the alternatives match the empty string, the entire match fails. For + set. If there are alternatives in the pattern, they are tried. If all + the alternatives match the empty string, the entire match fails. For example, if the pattern a?b? - is applied to a string not beginning with "a" or "b", it matches the - empty string at the start of the subject. With PCRE_NOTEMPTY set, this + is applied to a string not beginning with "a" or "b", it matches an + empty string at the start of the subject. With PCRE_NOTEMPTY set, this match is not valid, so PCRE searches further into the string for occur- rences of "a" or "b". - Perl has no direct equivalent of PCRE_NOTEMPTY, but it does make a spe- - cial case of a pattern match of the empty string within its split() - function, and when using the /g modifier. It is possible to emulate - Perl's behaviour after matching a null string by first trying the match - again at the same offset with PCRE_NOTEMPTY and PCRE_ANCHORED, and then - if that fails by advancing the starting offset (see below) and trying - an ordinary match again. There is some code that demonstrates how to do - this in the pcredemo.c sample program. + PCRE_NOTEMPTY_ATSTART + + This is like PCRE_NOTEMPTY, except that an empty string match that is + not at the start of the subject is permitted. If the pattern is + anchored, such a match can occur only if the pattern contains \K. + + Perl has no direct equivalent of PCRE_NOTEMPTY or + PCRE_NOTEMPTY_ATSTART, but it does make a special case of a pattern + match of the empty string within its split() function, and when using + the /g modifier. It is possible to emulate Perl's behaviour after + matching a null string by first trying the match again at the same off- + set with PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED, and then if that + fails, by advancing the starting offset (see below) and trying an ordi- + nary match again. There is some code that demonstrates how to do this + in the pcredemo sample program. PCRE_NO_START_OPTIMIZE @@ -2056,128 +2110,132 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION value of startoffset that does not point to the start of a UTF-8 char- acter, is undefined. Your program may crash. - PCRE_PARTIAL - - This option turns on the partial matching feature. If the subject - string fails to match the pattern, but at some point during the match- - ing process the end of the subject was reached (that is, the subject - partially matches the pattern and the failure to match occurred only - because there were not enough subject characters), pcre_exec() returns - PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. When PCRE_PARTIAL is - used, there are restrictions on what may appear in the pattern. These - are discussed in the pcrepartial documentation. + PCRE_PARTIAL_HARD + PCRE_PARTIAL_SOFT + + These options turn on the partial matching feature. For backwards com- + patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial + match occurs if the end of the subject string is reached successfully, + but there are not enough subject characters to complete the match. If + this happens when PCRE_PARTIAL_HARD is set, pcre_exec() immediately + returns PCRE_ERROR_PARTIAL. Otherwise, if PCRE_PARTIAL_SOFT is set, + matching continues by testing any other alternatives. Only if they all + fail is PCRE_ERROR_PARTIAL returned (instead of PCRE_ERROR_NOMATCH). + The portion of the string that was inspected when the partial match was + found is set as the first matching string. There is a more detailed + discussion in the pcrepartial documentation. The string to be matched by pcre_exec() - The subject string is passed to pcre_exec() as a pointer in subject, a + The subject string is passed to pcre_exec() as a pointer in subject, a length (in bytes) in length, and a starting byte offset in startoffset. In UTF-8 mode, the byte offset must point to the start of a UTF-8 char- - acter. Unlike the pattern string, the subject may contain binary zero - bytes. When the starting offset is zero, the search for a match starts - at the beginning of the subject, and this is by far the most common + acter. Unlike the pattern string, the subject may contain binary zero + bytes. When the starting offset is zero, the search for a match starts + at the beginning of the subject, and this is by far the most common case. - A non-zero starting offset is useful when searching for another match - in the same subject by calling pcre_exec() again after a previous suc- - cess. Setting startoffset differs from just passing over a shortened - string and setting PCRE_NOTBOL in the case of a pattern that begins + A non-zero starting offset is useful when searching for another match + in the same subject by calling pcre_exec() again after a previous suc- + cess. Setting startoffset differs from just passing over a shortened + string and setting PCRE_NOTBOL in the case of a pattern that begins with any kind of lookbehind. For example, consider the pattern \Biss\B - which finds occurrences of "iss" in the middle of words. (\B matches - only if the current position in the subject is not a word boundary.) - When applied to the string "Mississipi" the first call to pcre_exec() - finds the first occurrence. If pcre_exec() is called again with just - the remainder of the subject, namely "issipi", it does not match, + which finds occurrences of "iss" in the middle of words. (\B matches + only if the current position in the subject is not a word boundary.) + When applied to the string "Mississipi" the first call to pcre_exec() + finds the first occurrence. If pcre_exec() is called again with just + the remainder of the subject, namely "issipi", it does not match, because \B is always false at the start of the subject, which is deemed - to be a word boundary. However, if pcre_exec() is passed the entire + to be a word boundary. However, if pcre_exec() is passed the entire string again, but with startoffset set to 4, it finds the second occur- - rence of "iss" because it is able to look behind the starting point to + rence of "iss" because it is able to look behind the starting point to discover that it is preceded by a letter. - If a non-zero starting offset is passed when the pattern is anchored, + If a non-zero starting offset is passed when the pattern is anchored, one attempt to match at the given offset is made. This can only succeed - if the pattern does not require the match to be at the start of the + if the pattern does not require the match to be at the start of the subject. How pcre_exec() returns captured substrings - In general, a pattern matches a certain portion of the subject, and in - addition, further substrings from the subject may be picked out by - parts of the pattern. Following the usage in Jeffrey Friedl's book, - this is called "capturing" in what follows, and the phrase "capturing - subpattern" is used for a fragment of a pattern that picks out a sub- - string. PCRE supports several other kinds of parenthesized subpattern + In general, a pattern matches a certain portion of the subject, and in + addition, further substrings from the subject may be picked out by + parts of the pattern. Following the usage in Jeffrey Friedl's book, + this is called "capturing" in what follows, and the phrase "capturing + subpattern" is used for a fragment of a pattern that picks out a sub- + string. PCRE supports several other kinds of parenthesized subpattern that do not cause substrings to be captured. Captured substrings are returned to the caller via a vector of integers - whose address is passed in ovector. The number of elements in the vec- - tor is passed in ovecsize, which must be a non-negative number. Note: + whose address is passed in ovector. The number of elements in the vec- + tor is passed in ovecsize, which must be a non-negative number. Note: this argument is NOT the size of ovector in bytes. - The first two-thirds of the vector is used to pass back captured sub- - strings, each substring using a pair of integers. The remaining third - of the vector is used as workspace by pcre_exec() while matching cap- - turing subpatterns, and is not available for passing back information. - The number passed in ovecsize should always be a multiple of three. If + The first two-thirds of the vector is used to pass back captured sub- + strings, each substring using a pair of integers. The remaining third + of the vector is used as workspace by pcre_exec() while matching cap- + turing subpatterns, and is not available for passing back information. + The number passed in ovecsize should always be a multiple of three. If it is not, it is rounded down. - When a match is successful, information about captured substrings is - returned in pairs of integers, starting at the beginning of ovector, - and continuing up to two-thirds of its length at the most. The first - element of each pair is set to the byte offset of the first character - in a substring, and the second is set to the byte offset of the first - character after the end of a substring. Note: these values are always + When a match is successful, information about captured substrings is + returned in pairs of integers, starting at the beginning of ovector, + and continuing up to two-thirds of its length at the most. The first + element of each pair is set to the byte offset of the first character + in a substring, and the second is set to the byte offset of the first + character after the end of a substring. Note: these values are always byte offsets, even in UTF-8 mode. They are not character counts. - The first pair of integers, ovector[0] and ovector[1], identify the - portion of the subject string matched by the entire pattern. The next - pair is used for the first capturing subpattern, and so on. The value + The first pair of integers, ovector[0] and ovector[1], identify the + portion of the subject string matched by the entire pattern. The next + pair is used for the first capturing subpattern, and so on. The value returned by pcre_exec() is one more than the highest numbered pair that - has been set. For example, if two substrings have been captured, the - returned value is 3. If there are no capturing subpatterns, the return + has been set. For example, if two substrings have been captured, the + returned value is 3. If there are no capturing subpatterns, the return value from a successful match is 1, indicating that just the first pair of offsets has been set. If a capturing subpattern is matched repeatedly, it is the last portion of the string that it matched that is returned. - If the vector is too small to hold all the captured substring offsets, + If the vector is too small to hold all the captured substring offsets, it is used as far as possible (up to two-thirds of its length), and the - function returns a value of zero. If the substring offsets are not of - interest, pcre_exec() may be called with ovector passed as NULL and - ovecsize as zero. However, if the pattern contains back references and - the ovector is not big enough to remember the related substrings, PCRE - has to get additional memory for use during matching. Thus it is usu- + function returns a value of zero. If the substring offsets are not of + interest, pcre_exec() may be called with ovector passed as NULL and + ovecsize as zero. However, if the pattern contains back references and + the ovector is not big enough to remember the related substrings, PCRE + has to get additional memory for use during matching. Thus it is usu- ally advisable to supply an ovector. - The pcre_info() function can be used to find out how many capturing - subpatterns there are in a compiled pattern. The smallest size for - ovector that will allow for n captured substrings, in addition to the + The pcre_fullinfo() function can be used to find out how many capturing + subpatterns there are in a compiled pattern. The smallest size for + ovector that will allow for n captured substrings, in addition to the offsets of the substring matched by the whole pattern, is (n+1)*3. - It is possible for capturing subpattern number n+1 to match some part + It is possible for capturing subpattern number n+1 to match some part of the subject when subpattern n has not been used at all. For example, - if the string "abc" is matched against the pattern (a|(z))(bc) the + if the string "abc" is matched against the pattern (a|(z))(bc) the return from the function is 4, and subpatterns 1 and 3 are matched, but - 2 is not. When this happens, both values in the offset pairs corre- + 2 is not. When this happens, both values in the offset pairs corre- sponding to unused subpatterns are set to -1. - Offset values that correspond to unused subpatterns at the end of the - expression are also set to -1. For example, if the string "abc" is - matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not - matched. The return from the function is 2, because the highest used + Offset values that correspond to unused subpatterns at the end of the + expression are also set to -1. For example, if the string "abc" is + matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not + matched. The return from the function is 2, because the highest used capturing subpattern number is 1. However, you can refer to the offsets - for the second and third capturing subpatterns if you wish (assuming + for the second and third capturing subpatterns if you wish (assuming the vector is large enough, of course). - Some convenience functions are provided for extracting the captured + Some convenience functions are provided for extracting the captured substrings as separate strings. These are described below. Error return values from pcre_exec() - If pcre_exec() fails, it returns a negative number. The following are + If pcre_exec() fails, it returns a negative number. The following are defined in the header file: PCRE_ERROR_NOMATCH (-1) @@ -2186,7 +2244,7 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_ERROR_NULL (-2) - Either code or subject was passed as NULL, or ovector was NULL and + Either code or subject was passed as NULL, or ovector was NULL and ovecsize was not zero. PCRE_ERROR_BADOPTION (-3) @@ -2195,65 +2253,66 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION PCRE_ERROR_BADMAGIC (-4) - PCRE stores a 4-byte "magic number" at the start of the compiled code, + PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch the case when it is passed a junk pointer and to detect when a pattern that was compiled in an environment of one endianness is run in - an environment with the other endianness. This is the error that PCRE + an environment with the other endianness. This is the error that PCRE gives when the magic number is not present. PCRE_ERROR_UNKNOWN_OPCODE (-5) While running the pattern match, an unknown item was encountered in the - compiled pattern. This error could be caused by a bug in PCRE or by + compiled pattern. This error could be caused by a bug in PCRE or by overwriting of the compiled pattern. PCRE_ERROR_NOMEMORY (-6) - If a pattern contains back references, but the ovector that is passed + If a pattern contains back references, but the ovector that is passed to pcre_exec() is not big enough to remember the referenced substrings, - PCRE gets a block of memory at the start of matching to use for this - purpose. If the call via pcre_malloc() fails, this error is given. The + PCRE gets a block of memory at the start of matching to use for this + purpose. If the call via pcre_malloc() fails, this error is given. The memory is automatically freed at the end of matching. PCRE_ERROR_NOSUBSTRING (-7) - This error is used by the pcre_copy_substring(), pcre_get_substring(), + This error is used by the pcre_copy_substring(), pcre_get_substring(), and pcre_get_substring_list() functions (see below). It is never returned by pcre_exec(). PCRE_ERROR_MATCHLIMIT (-8) - The backtracking limit, as specified by the match_limit field in a - pcre_extra structure (or defaulted) was reached. See the description + The backtracking limit, as specified by the match_limit field in a + pcre_extra structure (or defaulted) was reached. See the description above. PCRE_ERROR_CALLOUT (-9) This error is never generated by pcre_exec() itself. It is provided for - use by callout functions that want to yield a distinctive error code. + use by callout functions that want to yield a distinctive error code. See the pcrecallout documentation for details. PCRE_ERROR_BADUTF8 (-10) - A string that contains an invalid UTF-8 byte sequence was passed as a + A string that contains an invalid UTF-8 byte sequence was passed as a subject. PCRE_ERROR_BADUTF8_OFFSET (-11) The UTF-8 byte sequence that was passed as a subject was valid, but the - value of startoffset did not point to the beginning of a UTF-8 charac- + value of startoffset did not point to the beginning of a UTF-8 charac- ter. PCRE_ERROR_PARTIAL (-12) - The subject string did not match, but it did match partially. See the + The subject string did not match, but it did match partially. See the pcrepartial documentation for details of partial matching. PCRE_ERROR_BADPARTIAL (-13) - The PCRE_PARTIAL option was used with a compiled pattern containing - items that are not supported for partial matching. See the pcrepartial - documentation for details of partial matching. + This code is no longer in use. It was formerly returned when the + PCRE_PARTIAL option was used with a compiled pattern containing items + that were not supported for partial matching. From release 8.00 + onwards, there are no restrictions on partial matching. PCRE_ERROR_INTERNAL (-14) @@ -2412,10 +2471,13 @@ EXTRACTING CAPTURED SUBSTRINGS BY NAME ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the behaviour may not be what you want (see the next section). - Warning: If the pattern uses the "(?|" feature to set up multiple sub- - patterns with the same number, you cannot use names to distinguish - them, because names are not included in the compiled code. The matching - process uses only numbers. + Warning: If the pattern uses the (?| feature to set up multiple subpat- + terns with the same number, as described in the section on duplicate + subpattern numbers in the pcrepattern page, you cannot use names to + distinguish the different subpatterns, because names are not included + in the compiled code. The matching process uses only numbers. For this + reason, the use of different names for subpatterns of the same number + causes an error at compile time. DUPLICATE SUBPATTERN NAMES @@ -2423,47 +2485,51 @@ DUPLICATE SUBPATTERN NAMES int pcre_get_stringtable_entries(const pcre *code, const char *name, char **first, char **last); - When a pattern is compiled with the PCRE_DUPNAMES option, names for - subpatterns are not required to be unique. Normally, patterns with - duplicate names are such that in any one match, only one of the named - subpatterns participates. An example is shown in the pcrepattern docu- - mentation. + When a pattern is compiled with the PCRE_DUPNAMES option, names for + subpatterns are not required to be unique. (Duplicate names are always + allowed for subpatterns with the same number, created by using the (?| + feature. Indeed, if such subpatterns are named, they are required to + use the same names.) + + Normally, patterns with duplicate names are such that in any one match, + only one of the named subpatterns participates. An example is shown in + the pcrepattern documentation. - When duplicates are present, pcre_copy_named_substring() and - pcre_get_named_substring() return the first substring corresponding to - the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING - (-7) is returned; no data is returned. The pcre_get_stringnumber() - function returns one of the numbers that are associated with the name, + When duplicates are present, pcre_copy_named_substring() and + pcre_get_named_substring() return the first substring corresponding to + the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING + (-7) is returned; no data is returned. The pcre_get_stringnumber() + function returns one of the numbers that are associated with the name, but it is not defined which it is. - If you want to get full details of all captured substrings for a given - name, you must use the pcre_get_stringtable_entries() function. The + If you want to get full details of all captured substrings for a given + name, you must use the pcre_get_stringtable_entries() function. The first argument is the compiled pattern, and the second is the name. The - third and fourth are pointers to variables which are updated by the + third and fourth are pointers to variables which are updated by the function. After it has run, they point to the first and last entries in - the name-to-number table for the given name. The function itself - returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if - there are none. The format of the table is described above in the sec- - tion entitled Information about a pattern. Given all the relevant - entries for the name, you can extract each of their numbers, and hence + the name-to-number table for the given name. The function itself + returns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if + there are none. The format of the table is described above in the sec- + tion entitled Information about a pattern. Given all the relevant + entries for the name, you can extract each of their numbers, and hence the captured data, if any. FINDING ALL POSSIBLE MATCHES - The traditional matching function uses a similar algorithm to Perl, + The traditional matching function uses a similar algorithm to Perl, which stops when it finds the first match, starting at a given point in - the subject. If you want to find all possible matches, or the longest - possible match, consider using the alternative matching function (see - below) instead. If you cannot use the alternative function, but still - need to find all possible matches, you can kludge it up by making use + the subject. If you want to find all possible matches, or the longest + possible match, consider using the alternative matching function (see + below) instead. If you cannot use the alternative function, but still + need to find all possible matches, you can kludge it up by making use of the callout facility, which is described in the pcrecallout documen- tation. What you have to do is to insert a callout right at the end of the pat- - tern. When your callout function is called, extract and save the cur- - rent matched substring. Then return 1, which forces pcre_exec() to - backtrack and try other alternatives. Ultimately, when it runs out of + tern. When your callout function is called, extract and save the cur- + rent matched substring. Then return 1, which forces pcre_exec() to + backtrack and try other alternatives. Ultimately, when it runs out of matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. @@ -2474,14 +2540,15 @@ MATCHING A PATTERN: THE ALTERNATIVE FUNCTION int options, int *ovector, int ovecsize, int *workspace, int wscount); - The function pcre_dfa_exec() is called to match a subject string - against a compiled pattern, using a matching algorithm that scans the - subject string just once, and does not backtrack. This has different - characteristics to the normal algorithm, and is not compatible with - Perl. Some of the features of PCRE patterns are not supported. Never- - theless, there are times when this kind of matching can be useful. For - a discussion of the two matching algorithms, see the pcrematching docu- - mentation. + The function pcre_dfa_exec() is called to match a subject string + against a compiled pattern, using a matching algorithm that scans the + subject string just once, and does not backtrack. This has different + characteristics to the normal algorithm, and is not compatible with + Perl. Some of the features of PCRE patterns are not supported. Never- + theless, there are times when this kind of matching can be useful. For + a discussion of the two matching algorithms, and a list of features + that pcre_dfa_exec() does not support, see the pcrematching documenta- + tion. The arguments for the pcre_dfa_exec() function are the same as for pcre_exec(), plus two extras. The ovector argument is used in a differ- @@ -2516,38 +2583,43 @@ MATCHING A PATTERN: THE ALTERNATIVE FUNCTION The unused bits of the options argument for pcre_dfa_exec() must be zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW- - LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, - PCRE_PARTIAL, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last - three of these are the same as for pcre_exec(), so their description is - not repeated here. - - PCRE_PARTIAL - - This has the same general effect as it does for pcre_exec(), but the - details are slightly different. When PCRE_PARTIAL is set for - pcre_dfa_exec(), the return code PCRE_ERROR_NOMATCH is converted into - PCRE_ERROR_PARTIAL if the end of the subject is reached, there have - been no complete matches, but there is still at least one matching pos- - sibility. The portion of the string that provided the partial match is - set as the first matching string. + LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, + PCRE_NOTEMPTY_ATSTART, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, PCRE_PAR- + TIAL_SOFT, PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last + four of these are exactly the same as for pcre_exec(), so their + description is not repeated here. + + PCRE_PARTIAL_HARD + PCRE_PARTIAL_SOFT + + These have the same general effect as they do for pcre_exec(), but the + details are slightly different. When PCRE_PARTIAL_HARD is set for + pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub- + ject is reached and there is still at least one matching possibility + that requires additional characters. This happens even if some complete + matches have also been found. When PCRE_PARTIAL_SOFT is set, the return + code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end + of the subject is reached, there have been no complete matches, but + there is still at least one matching possibility. The portion of the + string that was inspected when the longest partial match was found is + set as the first matching string in both cases. PCRE_DFA_SHORTEST - Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to + Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to stop as soon as it has found one match. Because of the way the alterna- - tive algorithm works, this is necessarily the shortest possible match + tive algorithm works, this is necessarily the shortest possible match at the first possible matching point in the subject string. PCRE_DFA_RESTART - When pcre_dfa_exec() is called with the PCRE_PARTIAL option, and - returns a partial match, it is possible to call it again, with addi- - tional subject characters, and have it continue with the same match. - The PCRE_DFA_RESTART option requests this action; when it is set, the - workspace and wscount options must reference the same vector as before - because data about the match so far is left in them after a partial - match. There is more discussion of this facility in the pcrepartial - documentation. + When pcre_dfa_exec() returns a partial match, it is possible to call it + again, with additional subject characters, and have it continue with + the same match. The PCRE_DFA_RESTART option requests this action; when + it is set, the workspace and wscount options must reference the same + vector as before because data about the match so far is left in them + after a partial match. There is more discussion of this facility in the + pcrepartial documentation. Successful returns from pcre_dfa_exec() @@ -2636,7 +2708,7 @@ AUTHOR REVISION - Last updated: 11 April 2009 + Last updated: 03 October 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -2666,10 +2738,10 @@ PCRE CALLOUTS (?C1)abc(?C2)def - If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() is - called, PCRE automatically inserts callouts, all with number 255, - before each item in the pattern. For example, if PCRE_AUTO_CALLOUT is - used with the pattern + If the PCRE_AUTO_CALLOUT option bit is set when pcre_compile() or + pcre_compile2() is called, PCRE automatically inserts callouts, all + with number 255, before each item in the pattern. For example, if + PCRE_AUTO_CALLOUT is used with the pattern A(\d{2}|--) @@ -2698,18 +2770,23 @@ MISSING CALLOUTS ever start, and the callout is never reached. However, with "abyd", though the result is still no match, the callout is obeyed. - You can disable these optimizations by passing the PCRE_NO_START_OPTI- - MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the - matching process, but does ensure that callouts such as the example + If the pattern is studied, PCRE knows the minimum length of a matching + string, and will immediately give a "no match" return without actually + running a match if the subject is not long enough, or, for unanchored + patterns, if it has been scanned far enough. + + You can disable these optimizations by passing the PCRE_NO_START_OPTI- + MIZE option to pcre_exec() or pcre_dfa_exec(). This slows down the + matching process, but does ensure that callouts such as the example above are obeyed. THE CALLOUT INTERFACE - During matching, when PCRE reaches a callout point, the external func- - tion defined by pcre_callout is called (if it is set). This applies to - both the pcre_exec() and the pcre_dfa_exec() matching functions. The - only argument to the callout function is a pointer to a pcre_callout + During matching, when PCRE reaches a callout point, the external func- + tion defined by pcre_callout is called (if it is set). This applies to + both the pcre_exec() and the pcre_dfa_exec() matching functions. The + only argument to the callout function is a pointer to a pcre_callout block. This structure contains the following fields: int version; @@ -2725,81 +2802,81 @@ THE CALLOUT INTERFACE int pattern_position; int next_item_length; - The version field is an integer containing the version number of the - block format. The initial version was 0; the current version is 1. The - version number will change again in future if additional fields are + The version field is an integer containing the version number of the + block format. The initial version was 0; the current version is 1. The + version number will change again in future if additional fields are added, but the intention is never to remove any of the existing fields. - The callout_number field contains the number of the callout, as com- - piled into the pattern (that is, the number after ?C for manual call- + The callout_number field contains the number of the callout, as com- + piled into the pattern (that is, the number after ?C for manual call- outs, and 255 for automatically generated callouts). - The offset_vector field is a pointer to the vector of offsets that was - passed by the caller to pcre_exec() or pcre_dfa_exec(). When - pcre_exec() is used, the contents can be inspected in order to extract - substrings that have been matched so far, in the same way as for - extracting substrings after a match has completed. For pcre_dfa_exec() + The offset_vector field is a pointer to the vector of offsets that was + passed by the caller to pcre_exec() or pcre_dfa_exec(). When + pcre_exec() is used, the contents can be inspected in order to extract + substrings that have been matched so far, in the same way as for + extracting substrings after a match has completed. For pcre_dfa_exec() this field is not useful. The subject and subject_length fields contain copies of the values that were passed to pcre_exec(). - The start_match field normally contains the offset within the subject - at which the current match attempt started. However, if the escape - sequence \K has been encountered, this value is changed to reflect the - modified starting point. If the pattern is not anchored, the callout + The start_match field normally contains the offset within the subject + at which the current match attempt started. However, if the escape + sequence \K has been encountered, this value is changed to reflect the + modified starting point. If the pattern is not anchored, the callout function may be called several times from the same point in the pattern for different starting points in the subject. - The current_position field contains the offset within the subject of + The current_position field contains the offset within the subject of the current match pointer. - When the pcre_exec() function is used, the capture_top field contains - one more than the number of the highest numbered captured substring so - far. If no substrings have been captured, the value of capture_top is - one. This is always the case when pcre_dfa_exec() is used, because it + When the pcre_exec() function is used, the capture_top field contains + one more than the number of the highest numbered captured substring so + far. If no substrings have been captured, the value of capture_top is + one. This is always the case when pcre_dfa_exec() is used, because it does not support captured substrings. - The capture_last field contains the number of the most recently cap- - tured substring. If no substrings have been captured, its value is -1. + The capture_last field contains the number of the most recently cap- + tured substring. If no substrings have been captured, its value is -1. This is always the case when pcre_dfa_exec() is used. - The callout_data field contains a value that is passed to pcre_exec() - or pcre_dfa_exec() specifically so that it can be passed back in call- - outs. It is passed in the pcre_callout field of the pcre_extra data - structure. If no such data was passed, the value of callout_data in a - pcre_callout block is NULL. There is a description of the pcre_extra + The callout_data field contains a value that is passed to pcre_exec() + or pcre_dfa_exec() specifically so that it can be passed back in call- + outs. It is passed in the pcre_callout field of the pcre_extra data + structure. If no such data was passed, the value of callout_data in a + pcre_callout block is NULL. There is a description of the pcre_extra structure in the pcreapi documentation. - The pattern_position field is present from version 1 of the pcre_call- + The pattern_position field is present from version 1 of the pcre_call- out structure. It contains the offset to the next item to be matched in the pattern string. - The next_item_length field is present from version 1 of the pcre_call- + The next_item_length field is present from version 1 of the pcre_call- out structure. It contains the length of the next item to be matched in - the pattern string. When the callout immediately precedes an alterna- - tion bar, a closing parenthesis, or the end of the pattern, the length - is zero. When the callout precedes an opening parenthesis, the length + the pattern string. When the callout immediately precedes an alterna- + tion bar, a closing parenthesis, or the end of the pattern, the length + is zero. When the callout precedes an opening parenthesis, the length is that of the entire subpattern. - The pattern_position and next_item_length fields are intended to help - in distinguishing between different automatic callouts, which all have + The pattern_position and next_item_length fields are intended to help + in distinguishing between different automatic callouts, which all have the same callout number. However, they are set for all callouts. RETURN VALUES - The external callout function returns an integer to PCRE. If the value - is zero, matching proceeds as normal. If the value is greater than - zero, matching fails at the current point, but the testing of other + The external callout function returns an integer to PCRE. If the value + is zero, matching proceeds as normal. If the value is greater than + zero, matching fails at the current point, but the testing of other matching possibilities goes ahead, just as if a lookahead assertion had - failed. If the value is less than zero, the match is abandoned, and - pcre_exec() (or pcre_dfa_exec()) returns the negative value. + failed. If the value is less than zero, the match is abandoned, and + pcre_exec() or pcre_dfa_exec() returns the negative value. - Negative values should normally be chosen from the set of + Negative values should normally be chosen from the set of PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- - dard "no match" failure. The error number PCRE_ERROR_CALLOUT is - reserved for use by callout functions; it will never be used by PCRE + dard "no match" failure. The error number PCRE_ERROR_CALLOUT is + reserved for use by callout functions; it will never be used by PCRE itself. @@ -2812,7 +2889,7 @@ AUTHOR REVISION - Last updated: 15 March 2009 + Last updated: 29 September 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -2827,46 +2904,49 @@ NAME DIFFERENCES BETWEEN PCRE AND PERL This document describes the differences in the ways that PCRE and Perl - handle regular expressions. The differences described here are mainly - with respect to Perl 5.8, though PCRE versions 7.0 and later contain - some features that are expected to be in the forthcoming Perl 5.10. + handle regular expressions. The differences described here are with + respect to Perl 5.10. - 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details - of what it does have are given in the section on UTF-8 support in the + 1. PCRE has only a subset of Perl's UTF-8 and Unicode support. Details + of what it does have are given in the section on UTF-8 support in the main pcre page. 2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl - permits them, but they do not mean what you might think. For example, + permits them, but they do not mean what you might think. For example, (?!a){3} does not assert that the next three characters are not "a". It just asserts that the next character is not "a" three times. - 3. Capturing subpatterns that occur inside negative lookahead asser- - tions are counted, but their entries in the offsets vector are never - set. Perl sets its numerical variables from any such patterns that are + 3. Capturing subpatterns that occur inside negative lookahead asser- + tions are counted, but their entries in the offsets vector are never + set. Perl sets its numerical variables from any such patterns that are matched before the assertion fails to match something (thereby succeed- - ing), but only if the negative lookahead assertion contains just one + ing), but only if the negative lookahead assertion contains just one branch. - 4. Though binary zero characters are supported in the subject string, + 4. Though binary zero characters are supported in the subject string, they are not allowed in a pattern string because it is passed as a nor- mal C string, terminated by zero. The escape sequence \0 can be used in the pattern to represent a binary zero. - 5. The following Perl escape sequences are not supported: \l, \u, \L, + 5. The following Perl escape sequences are not supported: \l, \u, \L, \U, and \N. In fact these are implemented by Perl's general string-han- - dling and are not part of its pattern matching engine. If any of these + dling and are not part of its pattern matching engine. If any of these are encountered by PCRE, an error is generated. - 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE - is built with Unicode character property support. The properties that - can be tested with \p and \P are limited to the general category prop- - erties such as Lu and Nd, script names such as Greek or Han, and the - derived properties Any and L&. + 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE + is built with Unicode character property support. The properties that + can be tested with \p and \P are limited to the general category prop- + erties such as Lu and Nd, script names such as Greek or Han, and the + derived properties Any and L&. PCRE does support the Cs (surrogate) + property, which Perl does not; the Perl documentation says "Because + Perl hides the need for the user to understand the internal representa- + tion of Unicode characters, there is no need to implement the somewhat + messy concept of surrogates." 7. PCRE does support the \Q...\E escape for quoting substrings. Charac- - ters in between are treated as literals. This is slightly different - from Perl in that $ and @ are also handled as literals inside the - quotes. In Perl, they cause variable interpolation (but of course PCRE + ters in between are treated as literals. This is slightly different + from Perl in that $ and @ are also handled as literals inside the + quotes. In Perl, they cause variable interpolation (but of course PCRE does not have variables). Note the following examples: Pattern PCRE matches Perl matches @@ -2876,55 +2956,68 @@ DIFFERENCES BETWEEN PCRE AND PERL \Qabc\$xyz\E abc\$xyz abc\$xyz \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - The \Q...\E sequence is recognized both inside and outside character + The \Q...\E sequence is recognized both inside and outside character classes. 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) - constructions. However, there is support for recursive patterns. This - is not available in Perl 5.8, but will be in Perl 5.10. Also, the PCRE - "callout" feature allows an external function to be called during pat- + constructions. However, there is support for recursive patterns. This + is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE + "callout" feature allows an external function to be called during pat- tern matching. See the pcrecallout documentation for details. - 9. Subpatterns that are called recursively or as "subroutines" are - always treated as atomic groups in PCRE. This is like Python, but - unlike Perl. + 9. Subpatterns that are called recursively or as "subroutines" are + always treated as atomic groups in PCRE. This is like Python, but + unlike Perl. There is a discussion of an example that explains this in + more detail in the section on recursion differences from Perl in the + pcrepattern page. - 10. There are some differences that are concerned with the settings of - captured strings when part of a pattern is repeated. For example, - matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 + 10. There are some differences that are concerned with the settings of + captured strings when part of a pattern is repeated. For example, + matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 unset, but in PCRE it is set to "b". 11. PCRE does support Perl 5.10's backtracking verbs (*ACCEPT), - (*FAIL), (*F), (*COMMIT), (*PRUNE), (*SKIP), and (*THEN), but only in - the forms without an argument. PCRE does not support (*MARK). If - (*ACCEPT) is within capturing parentheses, PCRE does not set that cap- - ture group; this is different to Perl. - - 12. PCRE provides some extensions to the Perl regular expression facil- - ities. Perl 5.10 will include new features that are not in earlier - versions, some of which (such as named parentheses) have been in PCRE - for some time. This list is with respect to Perl 5.10: - - (a) Although lookbehind assertions must match fixed length strings, - each alternative branch of a lookbehind assertion can match a different - length of string. Perl requires them all to have the same length. + (*FAIL), (*F), (*COMMIT), (*PRUNE), (*SKIP), and (*THEN), but only in + the forms without an argument. PCRE does not support (*MARK). + + 12. PCRE's handling of duplicate subpattern numbers and duplicate sub- + pattern names is not as general as Perl's. This is a consequence of the + fact the PCRE works internally just with numbers, using an external ta- + ble to translate between numbers and names. In particular, a pattern + such as (?|(?<a>A)|(?<b)B), where the two capturing parentheses have + the same number but different names, is not supported, and causes an + error at compile time. If it were allowed, it would not be possible to + distinguish which parentheses matched, because both names map to cap- + turing subpattern number 1. To avoid this confusing situation, an error + is given at compile time. + + 13. PCRE provides some extensions to the Perl regular expression facil- + ities. Perl 5.10 includes new features that are not in earlier ver- + sions of Perl, some of which (such as named parentheses) have been in + PCRE for some time. This list is with respect to Perl 5.10: + + (a) Although lookbehind assertions in PCRE must match fixed length + strings, each alternative branch of a lookbehind assertion can match a + different length of string. Perl requires them all to have the same + length. - (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ + (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ meta-character matches only at the very end of the string. (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe- cial meaning is faulted. Otherwise, like Perl, the backslash is quietly ignored. (Perl can be made to issue a warning.) - (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- + (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- fiers is inverted, that is, by default they are not greedy, but if fol- lowed by a question mark they are. (e) PCRE_ANCHORED can be used at matching time to force a pattern to be tried only at the first matching position in the subject string. - (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and PCRE_NO_AUTO_CAP- - TURE options for pcre_exec() have no Perl equivalents. + (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, + and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva- + lents. (g) The \R escape sequence can be restricted to match only CR, LF, or CRLF by the PCRE_BSR_ANYCRLF option. @@ -2953,8 +3046,8 @@ AUTHOR REVISION - Last updated: 11 September 2007 - Copyright (c) 1997-2007 University of Cambridge. + Last updated: 04 October 2009 + Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -2984,9 +3077,9 @@ PCRE REGULAR EXPRESSION DETAILS The original operation of PCRE was on strings of one-byte characters. However, there is now also support for UTF-8 character strings. To use - this, you must build PCRE to include UTF-8 support, and then call - pcre_compile() with the PCRE_UTF8 option. There is also a special - sequence that can be given at the start of a pattern: + this, PCRE must be built to include UTF-8 support, and you must call + pcre_compile() or pcre_compile2() with the PCRE_UTF8 option. There is + also a special sequence that can be given at the start of a pattern: (*UTF8) @@ -3024,9 +3117,9 @@ NEWLINE CONVENTIONS (*ANYCRLF) any of the three above (*ANY) all Unicode newline sequences - These override the default and the options given to pcre_compile(). For - example, on a Unix system where LF is the default newline sequence, the - pattern + These override the default and the options given to pcre_compile() or + pcre_compile2(). For example, on a Unix system where LF is the default + newline sequence, the pattern (*CR)a.b @@ -3143,7 +3236,7 @@ BACKSLASH acters in patterns in a visible manner. There is no restriction on the appearance of non-printing characters, apart from the binary zero that terminates a pattern, but when a pattern is being prepared by text - editing, it is usually easier to use one of the following escape + editing, it is often easier to use one of the following escape sequences than the binary character it represents: \a alarm, that is, the BEL character (hex 07) @@ -3355,13 +3448,13 @@ BACKSLASH (*BSR_ANYCRLF) CR, LF, or CRLF only (*BSR_UNICODE) any Unicode newline sequence - These override the default and the options given to pcre_compile(), but - they can be overridden by options given to pcre_exec(). Note that these - special settings, which are not Perl-compatible, are recognized only at - the very start of a pattern, and that they must be in upper case. If - more than one of them is present, the last one is used. They can be - combined with a change of newline convention, for example, a pattern - can start with: + These override the default and the options given to pcre_compile() or + pcre_compile2(), but they can be overridden by options given to + pcre_exec() or pcre_dfa_exec(). Note that these special settings, which + are not Perl-compatible, are recognized only at the very start of a + pattern, and that they must be in upper case. If more than one of them + is present, the last one is used. They can be combined with a change of + newline convention, for example, a pattern can start with: (*ANY)(*BSR_ANYCRLF) @@ -3472,9 +3565,9 @@ BACKSLASH U+D800 to U+DFFF. Such characters are not valid in UTF-8 strings (see RFC 3629) and so cannot be tested by PCRE, unless UTF-8 validity check- ing has been turned off (see the discussion of PCRE_NO_UTF8_CHECK in - the pcreapi page). + the pcreapi page). Perl does not support the Cs property. - The long synonyms for these properties that Perl supports (such as + The long synonyms for property names that Perl supports (such as \p{Letter}) are not supported by PCRE, nor is it permitted to prefix any of these properties with "Is". @@ -3544,34 +3637,37 @@ BACKSLASH A word boundary is a position in the subject string where the current character and the previous character do not both match \w or \W (i.e. one matches \w and the other matches \W), or the start or end of the - string if the first or last character matches \w, respectively. + string if the first or last character matches \w, respectively. Neither + PCRE nor Perl has a separte "start of word" or "end of word" metase- + quence. However, whatever follows \b normally determines which it is. + For example, the fragment \ba matches "a" at the start of a word. - The \A, \Z, and \z assertions differ from the traditional circumflex + The \A, \Z, and \z assertions differ from the traditional circumflex and dollar (described in the next section) in that they only ever match - at the very start and end of the subject string, whatever options are - set. Thus, they are independent of multiline mode. These three asser- + at the very start and end of the subject string, whatever options are + set. Thus, they are independent of multiline mode. These three asser- tions are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options, which - affect only the behaviour of the circumflex and dollar metacharacters. - However, if the startoffset argument of pcre_exec() is non-zero, indi- + affect only the behaviour of the circumflex and dollar metacharacters. + However, if the startoffset argument of pcre_exec() is non-zero, indi- cating that matching is to start at a point other than the beginning of - the subject, \A can never match. The difference between \Z and \z is + the subject, \A can never match. The difference between \Z and \z is that \Z matches before a newline at the end of the string as well as at the very end, whereas \z matches only at the end. - The \G assertion is true only when the current matching position is at - the start point of the match, as specified by the startoffset argument - of pcre_exec(). It differs from \A when the value of startoffset is - non-zero. By calling pcre_exec() multiple times with appropriate argu- + The \G assertion is true only when the current matching position is at + the start point of the match, as specified by the startoffset argument + of pcre_exec(). It differs from \A when the value of startoffset is + non-zero. By calling pcre_exec() multiple times with appropriate argu- ments, you can mimic Perl's /g option, and it is in this kind of imple- mentation where \G can be useful. - Note, however, that PCRE's interpretation of \G, as the start of the + Note, however, that PCRE's interpretation of \G, as the start of the current match, is subtly different from Perl's, which defines it as the - end of the previous match. In Perl, these can be different when the - previously matched string was empty. Because PCRE does just one match + end of the previous match. In Perl, these can be different when the + previously matched string was empty. Because PCRE does just one match at a time, it cannot reproduce this behaviour. - If all the alternatives of a pattern begin with \G, the expression is + If all the alternatives of a pattern begin with \G, the expression is anchored to the starting match position, and the "anchored" flag is set in the compiled regular expression. @@ -3579,90 +3675,90 @@ BACKSLASH CIRCUMFLEX AND DOLLAR Outside a character class, in the default matching mode, the circumflex - character is an assertion that is true only if the current matching - point is at the start of the subject string. If the startoffset argu- - ment of pcre_exec() is non-zero, circumflex can never match if the - PCRE_MULTILINE option is unset. Inside a character class, circumflex + character is an assertion that is true only if the current matching + point is at the start of the subject string. If the startoffset argu- + ment of pcre_exec() is non-zero, circumflex can never match if the + PCRE_MULTILINE option is unset. Inside a character class, circumflex has an entirely different meaning (see below). - Circumflex need not be the first character of the pattern if a number - of alternatives are involved, but it should be the first thing in each - alternative in which it appears if the pattern is ever to match that - branch. If all possible alternatives start with a circumflex, that is, - if the pattern is constrained to match only at the start of the sub- - ject, it is said to be an "anchored" pattern. (There are also other + Circumflex need not be the first character of the pattern if a number + of alternatives are involved, but it should be the first thing in each + alternative in which it appears if the pattern is ever to match that + branch. If all possible alternatives start with a circumflex, that is, + if the pattern is constrained to match only at the start of the sub- + ject, it is said to be an "anchored" pattern. (There are also other constructs that can cause a pattern to be anchored.) - A dollar character is an assertion that is true only if the current - matching point is at the end of the subject string, or immediately + A dollar character is an assertion that is true only if the current + matching point is at the end of the subject string, or immediately before a newline at the end of the string (by default). Dollar need not - be the last character of the pattern if a number of alternatives are - involved, but it should be the last item in any branch in which it + be the last character of the pattern if a number of alternatives are + involved, but it should be the last item in any branch in which it appears. Dollar has no special meaning in a character class. - The meaning of dollar can be changed so that it matches only at the - very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at + The meaning of dollar can be changed so that it matches only at the + very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This does not affect the \Z assertion. The meanings of the circumflex and dollar characters are changed if the - PCRE_MULTILINE option is set. When this is the case, a circumflex - matches immediately after internal newlines as well as at the start of - the subject string. It does not match after a newline that ends the - string. A dollar matches before any newlines in the string, as well as - at the very end, when PCRE_MULTILINE is set. When newline is specified - as the two-character sequence CRLF, isolated CR and LF characters do + PCRE_MULTILINE option is set. When this is the case, a circumflex + matches immediately after internal newlines as well as at the start of + the subject string. It does not match after a newline that ends the + string. A dollar matches before any newlines in the string, as well as + at the very end, when PCRE_MULTILINE is set. When newline is specified + as the two-character sequence CRLF, isolated CR and LF characters do not indicate newlines. - For example, the pattern /^abc$/ matches the subject string "def\nabc" - (where \n represents a newline) in multiline mode, but not otherwise. - Consequently, patterns that are anchored in single line mode because - all branches start with ^ are not anchored in multiline mode, and a - match for circumflex is possible when the startoffset argument of - pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if + For example, the pattern /^abc$/ matches the subject string "def\nabc" + (where \n represents a newline) in multiline mode, but not otherwise. + Consequently, patterns that are anchored in single line mode because + all branches start with ^ are not anchored in multiline mode, and a + match for circumflex is possible when the startoffset argument of + pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. - Note that the sequences \A, \Z, and \z can be used to match the start - and end of the subject in both modes, and if all branches of a pattern - start with \A it is always anchored, whether or not PCRE_MULTILINE is + Note that the sequences \A, \Z, and \z can be used to match the start + and end of the subject in both modes, and if all branches of a pattern + start with \A it is always anchored, whether or not PCRE_MULTILINE is set. FULL STOP (PERIOD, DOT) Outside a character class, a dot in the pattern matches any one charac- - ter in the subject string except (by default) a character that signi- - fies the end of a line. In UTF-8 mode, the matched character may be + ter in the subject string except (by default) a character that signi- + fies the end of a line. In UTF-8 mode, the matched character may be more than one byte long. - When a line ending is defined as a single character, dot never matches - that character; when the two-character sequence CRLF is used, dot does - not match CR if it is immediately followed by LF, but otherwise it - matches all characters (including isolated CRs and LFs). When any Uni- - code line endings are being recognized, dot does not match CR or LF or + When a line ending is defined as a single character, dot never matches + that character; when the two-character sequence CRLF is used, dot does + not match CR if it is immediately followed by LF, but otherwise it + matches all characters (including isolated CRs and LFs). When any Uni- + code line endings are being recognized, dot does not match CR or LF or any of the other line ending characters. - The behaviour of dot with regard to newlines can be changed. If the - PCRE_DOTALL option is set, a dot matches any one character, without + The behaviour of dot with regard to newlines can be changed. If the + PCRE_DOTALL option is set, a dot matches any one character, without exception. If the two-character sequence CRLF is present in the subject string, it takes two dots to match it. - The handling of dot is entirely independent of the handling of circum- - flex and dollar, the only relationship being that they both involve + The handling of dot is entirely independent of the handling of circum- + flex and dollar, the only relationship being that they both involve newlines. Dot has no special meaning in a character class. MATCHING A SINGLE BYTE Outside a character class, the escape sequence \C matches any one byte, - both in and out of UTF-8 mode. Unlike a dot, it always matches any - line-ending characters. The feature is provided in Perl in order to - match individual bytes in UTF-8 mode. Because it breaks up UTF-8 char- - acters into individual bytes, what remains in the string may be a mal- - formed UTF-8 string. For this reason, the \C escape sequence is best + both in and out of UTF-8 mode. Unlike a dot, it always matches any + line-ending characters. The feature is provided in Perl in order to + match individual bytes in UTF-8 mode. Because it breaks up UTF-8 char- + acters into individual bytes, what remains in the string may be a mal- + formed UTF-8 string. For this reason, the \C escape sequence is best avoided. - PCRE does not allow \C to appear in lookbehind assertions (described - below), because in UTF-8 mode this would make it impossible to calcu- + PCRE does not allow \C to appear in lookbehind assertions (described + below), because in UTF-8 mode this would make it impossible to calcu- late the length of the lookbehind. @@ -3670,97 +3766,99 @@ SQUARE BRACKETS AND CHARACTER CLASSES An opening square bracket introduces a character class, terminated by a closing square bracket. A closing square bracket on its own is not spe- - cial. If a closing square bracket is required as a member of the class, - it should be the first data character in the class (after an initial - circumflex, if present) or escaped with a backslash. - - A character class matches a single character in the subject. In UTF-8 - mode, the character may occupy more than one byte. A matched character + cial by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, + a lone closing square bracket causes a compile-time error. If a closing + square bracket is required as a member of the class, it should be the + first data character in the class (after an initial circumflex, if + present) or escaped with a backslash. + + A character class matches a single character in the subject. In UTF-8 + mode, the character may be more than one byte long. A matched character must be in the set of characters defined by the class, unless the first - character in the class definition is a circumflex, in which case the - subject character must not be in the set defined by the class. If a - circumflex is actually required as a member of the class, ensure it is + character in the class definition is a circumflex, in which case the + subject character must not be in the set defined by the class. If a + circumflex is actually required as a member of the class, ensure it is not the first character, or escape it with a backslash. - For example, the character class [aeiou] matches any lower case vowel, - while [^aeiou] matches any character that is not a lower case vowel. + For example, the character class [aeiou] matches any lower case vowel, + while [^aeiou] matches any character that is not a lower case vowel. Note that a circumflex is just a convenient notation for specifying the - characters that are in the class by enumerating those that are not. A - class that starts with a circumflex is not an assertion: it still con- - sumes a character from the subject string, and therefore it fails if + characters that are in the class by enumerating those that are not. A + class that starts with a circumflex is not an assertion; it still con- + sumes a character from the subject string, and therefore it fails if the current pointer is at the end of the string. - In UTF-8 mode, characters with values greater than 255 can be included - in a class as a literal string of bytes, or by using the \x{ escaping + In UTF-8 mode, characters with values greater than 255 can be included + in a class as a literal string of bytes, or by using the \x{ escaping mechanism. - When caseless matching is set, any letters in a class represent both - their upper case and lower case versions, so for example, a caseless - [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not - match "A", whereas a caseful version would. In UTF-8 mode, PCRE always - understands the concept of case for characters whose values are less - than 128, so caseless matching is always possible. For characters with - higher values, the concept of case is supported if PCRE is compiled - with Unicode property support, but not otherwise. If you want to use - caseless matching for characters 128 and above, you must ensure that - PCRE is compiled with Unicode property support as well as with UTF-8 - support. - - Characters that might indicate line breaks are never treated in any - special way when matching character classes, whatever line-ending - sequence is in use, and whatever setting of the PCRE_DOTALL and + When caseless matching is set, any letters in a class represent both + their upper case and lower case versions, so for example, a caseless + [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not + match "A", whereas a caseful version would. In UTF-8 mode, PCRE always + understands the concept of case for characters whose values are less + than 128, so caseless matching is always possible. For characters with + higher values, the concept of case is supported if PCRE is compiled + with Unicode property support, but not otherwise. If you want to use + caseless matching in UTF8-mode for characters 128 and above, you must + ensure that PCRE is compiled with Unicode property support as well as + with UTF-8 support. + + Characters that might indicate line breaks are never treated in any + special way when matching character classes, whatever line-ending + sequence is in use, and whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class such as [^a] always matches one of these characters. - The minus (hyphen) character can be used to specify a range of charac- - ters in a character class. For example, [d-m] matches any letter - between d and m, inclusive. If a minus character is required in a - class, it must be escaped with a backslash or appear in a position - where it cannot be interpreted as indicating a range, typically as the + The minus (hyphen) character can be used to specify a range of charac- + ters in a character class. For example, [d-m] matches any letter + between d and m, inclusive. If a minus character is required in a + class, it must be escaped with a backslash or appear in a position + where it cannot be interpreted as indicating a range, typically as the first or last character in the class. It is not possible to have the literal character "]" as the end charac- - ter of a range. A pattern such as [W-]46] is interpreted as a class of - two characters ("W" and "-") followed by a literal string "46]", so it - would match "W46]" or "-46]". However, if the "]" is escaped with a - backslash it is interpreted as the end of range, so [W-\]46] is inter- - preted as a class containing a range followed by two other characters. - The octal or hexadecimal representation of "]" can also be used to end + ter of a range. A pattern such as [W-]46] is interpreted as a class of + two characters ("W" and "-") followed by a literal string "46]", so it + would match "W46]" or "-46]". However, if the "]" is escaped with a + backslash it is interpreted as the end of range, so [W-\]46] is inter- + preted as a class containing a range followed by two other characters. + The octal or hexadecimal representation of "]" can also be used to end a range. - Ranges operate in the collating sequence of character values. They can - also be used for characters specified numerically, for example - [\000-\037]. In UTF-8 mode, ranges can include characters whose values + Ranges operate in the collating sequence of character values. They can + also be used for characters specified numerically, for example + [\000-\037]. In UTF-8 mode, ranges can include characters whose values are greater than 255, for example [\x{100}-\x{2ff}]. If a range that includes letters is used when caseless matching is set, it matches the letters in either case. For example, [W-c] is equivalent - to [][\\^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if - character tables for a French locale are in use, [\xc8-\xcb] matches - accented E characters in both cases. In UTF-8 mode, PCRE supports the - concept of case for characters with values greater than 128 only when + to [][\\^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if + character tables for a French locale are in use, [\xc8-\xcb] matches + accented E characters in both cases. In UTF-8 mode, PCRE supports the + concept of case for characters with values greater than 128 only when it is compiled with Unicode property support. - The character types \d, \D, \p, \P, \s, \S, \w, and \W may also appear - in a character class, and add the characters that they match to the + The character types \d, \D, \p, \P, \s, \S, \w, and \W may also appear + in a character class, and add the characters that they match to the class. For example, [\dABCDEF] matches any hexadecimal digit. A circum- - flex can conveniently be used with the upper case character types to - specify a more restricted set of characters than the matching lower - case type. For example, the class [^\W_] matches any letter or digit, + flex can conveniently be used with the upper case character types to + specify a more restricted set of characters than the matching lower + case type. For example, the class [^\W_] matches any letter or digit, but not underscore. - The only metacharacters that are recognized in character classes are - backslash, hyphen (only where it can be interpreted as specifying a - range), circumflex (only at the start), opening square bracket (only - when it can be interpreted as introducing a POSIX class name - see the - next section), and the terminating closing square bracket. However, + The only metacharacters that are recognized in character classes are + backslash, hyphen (only where it can be interpreted as specifying a + range), circumflex (only at the start), opening square bracket (only + when it can be interpreted as introducing a POSIX class name - see the + next section), and the terminating closing square bracket. However, escaping other non-alphanumeric characters does no harm. POSIX CHARACTER CLASSES Perl supports the POSIX notation for character classes. This uses names - enclosed by [: and :] within the enclosing square brackets. PCRE also + enclosed by [: and :] within the enclosing square brackets. PCRE also supports this notation. For example, [01[:alpha:]%] @@ -3783,18 +3881,18 @@ POSIX CHARACTER CLASSES word "word" characters (same as \w) xdigit hexadecimal digits - The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), - and space (32). Notice that this list includes the VT character (code + The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), + and space (32). Notice that this list includes the VT character (code 11). This makes "space" different to \s, which does not include VT (for Perl compatibility). - The name "word" is a Perl extension, and "blank" is a GNU extension - from Perl 5.8. Another Perl extension is negation, which is indicated + The name "word" is a Perl extension, and "blank" is a GNU extension + from Perl 5.8. Another Perl extension is negation, which is indicated by a ^ character after the colon. For example, [12[:^digit:]] - matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the + matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not supported, and an error is given if they are encountered. @@ -3804,24 +3902,24 @@ POSIX CHARACTER CLASSES VERTICAL BAR - Vertical bar characters are used to separate alternative patterns. For + Vertical bar characters are used to separate alternative patterns. For example, the pattern gilbert|sullivan - matches either "gilbert" or "sullivan". Any number of alternatives may - appear, and an empty alternative is permitted (matching the empty + matches either "gilbert" or "sullivan". Any number of alternatives may + appear, and an empty alternative is permitted (matching the empty string). The matching process tries each alternative in turn, from left - to right, and the first one that succeeds is used. If the alternatives - are within a subpattern (defined below), "succeeds" means matching the + to right, and the first one that succeeds is used. If the alternatives + are within a subpattern (defined below), "succeeds" means matching the rest of the main pattern as well as the alternative in the subpattern. INTERNAL OPTION SETTING - The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and - PCRE_EXTENDED options (which are Perl-compatible) can be changed from - within the pattern by a sequence of Perl option letters enclosed + The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and + PCRE_EXTENDED options (which are Perl-compatible) can be changed from + within the pattern by a sequence of Perl option letters enclosed between "(?" and ")". The option letters are i for PCRE_CASELESS @@ -3831,46 +3929,46 @@ INTERNAL OPTION SETTING For example, (?im) sets caseless, multiline matching. It is also possi- ble to unset these options by preceding the letter with a hyphen, and a - combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- - LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, - is also permitted. If a letter appears both before and after the + combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- + LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, + is also permitted. If a letter appears both before and after the hyphen, the option is unset. - The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA - can be changed in the same way as the Perl-compatible options by using + The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA + can be changed in the same way as the Perl-compatible options by using the characters J, U and X respectively. - When one of these option changes occurs at top level (that is, not - inside subpattern parentheses), the change applies to the remainder of + When one of these option changes occurs at top level (that is, not + inside subpattern parentheses), the change applies to the remainder of the pattern that follows. If the change is placed right at the start of a pattern, PCRE extracts it into the global options (and it will there- fore show up in data extracted by the pcre_fullinfo() function). - An option change within a subpattern (see below for a description of + An option change within a subpattern (see below for a description of subpatterns) affects only that part of the current pattern that follows it, so (a(?i)b)c matches abc and aBc and no other strings (assuming PCRE_CASELESS is not - used). By this means, options can be made to have different settings - in different parts of the pattern. Any changes made in one alternative - do carry on into subsequent branches within the same subpattern. For + used). By this means, options can be made to have different settings + in different parts of the pattern. Any changes made in one alternative + do carry on into subsequent branches within the same subpattern. For example, (a(?i)b|c) - matches "ab", "aB", "c", and "C", even though when matching "C" the - first branch is abandoned before the option setting. This is because - the effects of option settings happen at compile time. There would be + matches "ab", "aB", "c", and "C", even though when matching "C" the + first branch is abandoned before the option setting. This is because + the effects of option settings happen at compile time. There would be some very weird behaviour otherwise. - Note: There are other PCRE-specific options that can be set by the - application when the compile or match functions are called. In some + Note: There are other PCRE-specific options that can be set by the + application when the compile or match functions are called. In some cases the pattern can contain special leading sequences such as (*CRLF) - to override what the application has set or what has been defaulted. - Details are given in the section entitled "Newline sequences" above. - There is also the (*UTF8) leading sequence that can be used to set + to override what the application has set or what has been defaulted. + Details are given in the section entitled "Newline sequences" above. + There is also the (*UTF8) leading sequence that can be used to set UTF-8 mode; this is equivalent to setting the PCRE_UTF8 option. @@ -3883,18 +3981,18 @@ SUBPATTERNS cat(aract|erpillar|) - matches one of the words "cat", "cataract", or "caterpillar". Without - the parentheses, it would match "cataract", "erpillar" or an empty + matches one of the words "cat", "cataract", or "caterpillar". Without + the parentheses, it would match "cataract", "erpillar" or an empty string. - 2. It sets up the subpattern as a capturing subpattern. This means - that, when the whole pattern matches, that portion of the subject + 2. It sets up the subpattern as a capturing subpattern. This means + that, when the whole pattern matches, that portion of the subject string that matched the subpattern is passed back to the caller via the - ovector argument of pcre_exec(). Opening parentheses are counted from - left to right (starting from 1) to obtain numbers for the capturing + ovector argument of pcre_exec(). Opening parentheses are counted from + left to right (starting from 1) to obtain numbers for the capturing subpatterns. - For example, if the string "the red king" is matched against the pat- + For example, if the string "the red king" is matched against the pat- tern the ((red|white) (king|queen)) @@ -3902,12 +4000,12 @@ SUBPATTERNS the captured substrings are "red king", "red", and "king", and are num- bered 1, 2, and 3, respectively. - The fact that plain parentheses fulfil two functions is not always - helpful. There are often times when a grouping subpattern is required - without a capturing requirement. If an opening parenthesis is followed - by a question mark and a colon, the subpattern does not do any captur- - ing, and is not counted when computing the number of any subsequent - capturing subpatterns. For example, if the string "the white queen" is + The fact that plain parentheses fulfil two functions is not always + helpful. There are often times when a grouping subpattern is required + without a capturing requirement. If an opening parenthesis is followed + by a question mark and a colon, the subpattern does not do any captur- + ing, and is not counted when computing the number of any subsequent + capturing subpatterns. For example, if the string "the white queen" is matched against the pattern the ((?:red|white) (king|queen)) @@ -3915,46 +4013,59 @@ SUBPATTERNS the captured substrings are "white queen" and "queen", and are numbered 1 and 2. The maximum number of capturing subpatterns is 65535. - As a convenient shorthand, if any option settings are required at the - start of a non-capturing subpattern, the option letters may appear + As a convenient shorthand, if any option settings are required at the + start of a non-capturing subpattern, the option letters may appear between the "?" and the ":". Thus the two patterns (?i:saturday|sunday) (?:(?i)saturday|sunday) match exactly the same set of strings. Because alternative branches are - tried from left to right, and options are not reset until the end of - the subpattern is reached, an option setting in one branch does affect - subsequent branches, so the above patterns match "SUNDAY" as well as + tried from left to right, and options are not reset until the end of + the subpattern is reached, an option setting in one branch does affect + subsequent branches, so the above patterns match "SUNDAY" as well as "Saturday". DUPLICATE SUBPATTERN NUMBERS Perl 5.10 introduced a feature whereby each alternative in a subpattern - uses the same numbers for its capturing parentheses. Such a subpattern - starts with (?| and is itself a non-capturing subpattern. For example, + uses the same numbers for its capturing parentheses. Such a subpattern + starts with (?| and is itself a non-capturing subpattern. For example, consider this pattern: (?|(Sat)ur|(Sun))day - Because the two alternatives are inside a (?| group, both sets of cap- - turing parentheses are numbered one. Thus, when the pattern matches, - you can look at captured substring number one, whichever alternative - matched. This construct is useful when you want to capture part, but + Because the two alternatives are inside a (?| group, both sets of cap- + turing parentheses are numbered one. Thus, when the pattern matches, + you can look at captured substring number one, whichever alternative + matched. This construct is useful when you want to capture part, but not all, of one of a number of alternatives. Inside a (?| group, paren- - theses are numbered as usual, but the number is reset at the start of - each branch. The numbers of any capturing buffers that follow the sub- - pattern start after the highest number used in any branch. The follow- - ing example is taken from the Perl documentation. The numbers under- + theses are numbered as usual, but the number is reset at the start of + each branch. The numbers of any capturing buffers that follow the sub- + pattern start after the highest number used in any branch. The follow- + ing example is taken from the Perl documentation. The numbers under- neath show in which buffer the captured content will be stored. # before ---------------branch-reset----------- after / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x # 1 2 2 3 2 3 4 - A backreference or a recursive call to a numbered subpattern always - refers to the first one in the pattern with the given number. + A backreference to a numbered subpattern uses the most recent value + that is set for that number by any subpattern. The following pattern + matches "abcabc" or "defdef": + + /(?|(abc)|(def))\1/ + + In contrast, a recursive or "subroutine" call to a numbered subpattern + always refers to the first one in the pattern with the given number. + The following pattern matches "abcabc" or "defabc": + + /(?|(abc)|(def))(?1)/ + + If a condition test for a subpattern's having matched refers to a non- + unique number, the test is true if any of the subpatterns of that num- + ber have matched. An alternative approach to using this "branch reset" feature is to use duplicate named subpatterns, as described in the next section. @@ -3969,26 +4080,29 @@ NAMED SUBPATTERNS patterns. This feature was not added to Perl until release 5.10. Python had the feature earlier, and PCRE introduced it at release 4.0, using the Python syntax. PCRE now supports both the Perl and the Python syn- - tax. + tax. Perl allows identically numbered subpatterns to have different + names, but PCRE does not. - In PCRE, a subpattern can be named in one of three ways: (?<name>...) - or (?'name'...) as in Perl, or (?P<name>...) as in Python. References + In PCRE, a subpattern can be named in one of three ways: (?<name>...) + or (?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing parentheses from other parts of the pattern, such as back- - references, recursion, and conditions, can be made by name as well as + references, recursion, and conditions, can be made by name as well as by number. - Names consist of up to 32 alphanumeric characters and underscores. - Named capturing parentheses are still allocated numbers as well as - names, exactly as if the names were not present. The PCRE API provides + Names consist of up to 32 alphanumeric characters and underscores. + Named capturing parentheses are still allocated numbers as well as + names, exactly as if the names were not present. The PCRE API provides function calls for extracting the name-to-number translation table from a compiled pattern. There is also a convenience function for extracting a captured substring by name. - By default, a name must be unique within a pattern, but it is possible + By default, a name must be unique within a pattern, but it is possible to relax this constraint by setting the PCRE_DUPNAMES option at compile - time. This can be useful for patterns where only one instance of the - named parentheses can match. Suppose you want to match the name of a - weekday, either as a 3-letter abbreviation or as the full name, and in + time. (Duplicate names are also always permitted for subpatterns with + the same number, set up as described in the previous section.) Dupli- + cate names can be useful for patterns where only one instance of the + named parentheses can match. Suppose you want to match the name of a + weekday, either as a 3-letter abbreviation or as the full name, and in both cases you want to extract the abbreviation. This pattern (ignoring the line breaks) does the job: @@ -3998,26 +4112,38 @@ NAMED SUBPATTERNS (?<DN>Thu)(?:rsday)?| (?<DN>Sat)(?:urday)? - There are five capturing substrings, but only one is ever set after a + There are five capturing substrings, but only one is ever set after a match. (An alternative way of solving this problem is to use a "branch reset" subpattern, as described in the previous section.) - The convenience function for extracting the data by name returns the - substring for the first (and in this example, the only) subpattern of - that name that matched. This saves searching to find which numbered - subpattern it was. If you make a reference to a non-unique named sub- - pattern from elsewhere in the pattern, the one that corresponds to the - lowest number is used. For further details of the interfaces for han- - dling named subpatterns, see the pcreapi documentation. + The convenience function for extracting the data by name returns the + substring for the first (and in this example, the only) subpattern of + that name that matched. This saves searching to find which numbered + subpattern it was. + + If you make a backreference to a non-unique named subpattern from else- + where in the pattern, the one that corresponds to the first occurrence + of the name is used. In the absence of duplicate numbers (see the pre- + vious section) this is the one with the lowest number. If you use a + named reference in a condition test (see the section about conditions + below), either to check whether a subpattern has matched, or to check + for recursion, all subpatterns with the same name are tested. If the + condition is true for any one of them, the overall condition is true. + This is the same behaviour as testing by number. For further details of + the interfaces for handling named subpatterns, see the pcreapi documen- + tation. Warning: You cannot use different names to distinguish between two sub- - patterns with the same number (see the previous section) because PCRE - uses only the numbers when matching. + patterns with the same number because PCRE uses only the numbers when + matching. For this reason, an error is given at compile time if differ- + ent names are given to subpatterns with the same number. However, you + can give the same name to subpatterns with the same number, even when + PCRE_DUPNAMES is not set. REPETITION - Repetition is specified by quantifiers, which can follow any of the + Repetition is specified by quantifiers, which can follow any of the following items: a literal data character @@ -4029,18 +4155,19 @@ REPETITION a character class a back reference (see next section) a parenthesized subpattern (unless it is an assertion) + a recursive or "subroutine" call to a subpattern - The general repetition quantifier specifies a minimum and maximum num- - ber of permitted matches, by giving the two numbers in curly brackets - (braces), separated by a comma. The numbers must be less than 65536, + The general repetition quantifier specifies a minimum and maximum num- + ber of permitted matches, by giving the two numbers in curly brackets + (braces), separated by a comma. The numbers must be less than 65536, and the first must be less than or equal to the second. For example: z{2,4} - matches "zz", "zzz", or "zzzz". A closing brace on its own is not a - special character. If the second number is omitted, but the comma is - present, there is no upper limit; if the second number and the comma - are both omitted, the quantifier specifies an exact number of required + matches "zz", "zzz", or "zzzz". A closing brace on its own is not a + special character. If the second number is omitted, but the comma is + present, there is no upper limit; if the second number and the comma + are both omitted, the quantifier specifies an exact number of required matches. Thus [aeiou]{3,} @@ -4049,49 +4176,49 @@ REPETITION \d{8} - matches exactly 8 digits. An opening curly bracket that appears in a - position where a quantifier is not allowed, or one that does not match - the syntax of a quantifier, is taken as a literal character. For exam- + matches exactly 8 digits. An opening curly bracket that appears in a + position where a quantifier is not allowed, or one that does not match + the syntax of a quantifier, is taken as a literal character. For exam- ple, {,6} is not a quantifier, but a literal string of four characters. - In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to + In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual bytes. Thus, for example, \x{100}{2} matches two UTF-8 char- acters, each of which is represented by a two-byte sequence. Similarly, when Unicode property support is available, \X{3} matches three Unicode - extended sequences, each of which may be several bytes long (and they + extended sequences, each of which may be several bytes long (and they may be of different lengths). The quantifier {0} is permitted, causing the expression to behave as if the previous item and the quantifier were not present. This may be use- - ful for subpatterns that are referenced as subroutines from elsewhere + ful for subpatterns that are referenced as subroutines from elsewhere in the pattern. Items other than subpatterns that have a {0} quantifier are omitted from the compiled pattern. - For convenience, the three most common quantifiers have single-charac- + For convenience, the three most common quantifiers have single-charac- ter abbreviations: * is equivalent to {0,} + is equivalent to {1,} ? is equivalent to {0,1} - It is possible to construct infinite loops by following a subpattern + It is possible to construct infinite loops by following a subpattern that can match no characters with a quantifier that has no upper limit, for example: (a?)* Earlier versions of Perl and PCRE used to give an error at compile time - for such patterns. However, because there are cases where this can be - useful, such patterns are now accepted, but if any repetition of the - subpattern does in fact match no characters, the loop is forcibly bro- + for such patterns. However, because there are cases where this can be + useful, such patterns are now accepted, but if any repetition of the + subpattern does in fact match no characters, the loop is forcibly bro- ken. - By default, the quantifiers are "greedy", that is, they match as much - as possible (up to the maximum number of permitted times), without - causing the rest of the pattern to fail. The classic example of where + By default, the quantifiers are "greedy", that is, they match as much + as possible (up to the maximum number of permitted times), without + causing the rest of the pattern to fail. The classic example of where this gives problems is in trying to match comments in C programs. These - appear between /* and */ and within the comment, individual * and / - characters may appear. An attempt to match C comments by applying the + appear between /* and */ and within the comment, individual * and / + characters may appear. An attempt to match C comments by applying the pattern /\*.*\*/ @@ -4100,19 +4227,19 @@ REPETITION /* first comment */ not comment /* second comment */ - fails, because it matches the entire string owing to the greediness of + fails, because it matches the entire string owing to the greediness of the .* item. - However, if a quantifier is followed by a question mark, it ceases to + However, if a quantifier is followed by a question mark, it ceases to be greedy, and instead matches the minimum number of times possible, so the pattern /\*.*?\*/ - does the right thing with the C comments. The meaning of the various - quantifiers is not otherwise changed, just the preferred number of - matches. Do not confuse this use of question mark with its use as a - quantifier in its own right. Because it has two uses, it can sometimes + does the right thing with the C comments. The meaning of the various + quantifiers is not otherwise changed, just the preferred number of + matches. Do not confuse this use of question mark with its use as a + quantifier in its own right. Because it has two uses, it can sometimes appear doubled, as in \d??\d @@ -4120,36 +4247,36 @@ REPETITION which matches one digit by preference, but can match two if that is the only way the rest of the pattern matches. - If the PCRE_UNGREEDY option is set (an option that is not available in - Perl), the quantifiers are not greedy by default, but individual ones - can be made greedy by following them with a question mark. In other + If the PCRE_UNGREEDY option is set (an option that is not available in + Perl), the quantifiers are not greedy by default, but individual ones + can be made greedy by following them with a question mark. In other words, it inverts the default behaviour. - When a parenthesized subpattern is quantified with a minimum repeat - count that is greater than 1 or with a limited maximum, more memory is - required for the compiled pattern, in proportion to the size of the + When a parenthesized subpattern is quantified with a minimum repeat + count that is greater than 1 or with a limited maximum, more memory is + required for the compiled pattern, in proportion to the size of the minimum or maximum. If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equiv- - alent to Perl's /s) is set, thus allowing the dot to match newlines, - the pattern is implicitly anchored, because whatever follows will be - tried against every character position in the subject string, so there - is no point in retrying the overall match at any position after the - first. PCRE normally treats such a pattern as though it were preceded + alent to Perl's /s) is set, thus allowing the dot to match newlines, + the pattern is implicitly anchored, because whatever follows will be + tried against every character position in the subject string, so there + is no point in retrying the overall match at any position after the + first. PCRE normally treats such a pattern as though it were preceded by \A. - In cases where it is known that the subject string contains no new- - lines, it is worth setting PCRE_DOTALL in order to obtain this opti- + In cases where it is known that the subject string contains no new- + lines, it is worth setting PCRE_DOTALL in order to obtain this opti- mization, or alternatively using ^ to indicate anchoring explicitly. - However, there is one situation where the optimization cannot be used. - When .* is inside capturing parentheses that are the subject of a - backreference elsewhere in the pattern, a match at the start may fail + However, there is one situation where the optimization cannot be used. + When .* is inside capturing parentheses that are the subject of a + backreference elsewhere in the pattern, a match at the start may fail where a later one succeeds. Consider, for example: (.*)abc\1 - If the subject is "xyz123abc123" the match point is the fourth charac- + If the subject is "xyz123abc123" the match point is the fourth charac- ter. For this reason, such a pattern is not implicitly anchored. When a capturing subpattern is repeated, the value captured is the sub- @@ -4158,8 +4285,8 @@ REPETITION (tweedle[dume]{3}\s*)+ has matched "tweedledum tweedledee" the value of the captured substring - is "tweedledee". However, if there are nested capturing subpatterns, - the corresponding captured values may have been set in previous itera- + is "tweedledee". However, if there are nested capturing subpatterns, + the corresponding captured values may have been set in previous itera- tions. For example, after /(a|(b))+/ @@ -4169,53 +4296,53 @@ REPETITION ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS - With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") - repetition, failure of what follows normally causes the repeated item - to be re-evaluated to see if a different number of repeats allows the - rest of the pattern to match. Sometimes it is useful to prevent this, - either to change the nature of the match, or to cause it fail earlier - than it otherwise might, when the author of the pattern knows there is + With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") + repetition, failure of what follows normally causes the repeated item + to be re-evaluated to see if a different number of repeats allows the + rest of the pattern to match. Sometimes it is useful to prevent this, + either to change the nature of the match, or to cause it fail earlier + than it otherwise might, when the author of the pattern knows there is no point in carrying on. - Consider, for example, the pattern \d+foo when applied to the subject + Consider, for example, the pattern \d+foo when applied to the subject line 123456bar After matching all 6 digits and then failing to match "foo", the normal - action of the matcher is to try again with only 5 digits matching the - \d+ item, and then with 4, and so on, before ultimately failing. - "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides - the means for specifying that once a subpattern has matched, it is not + action of the matcher is to try again with only 5 digits matching the + \d+ item, and then with 4, and so on, before ultimately failing. + "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides + the means for specifying that once a subpattern has matched, it is not to be re-evaluated in this way. - If we use atomic grouping for the previous example, the matcher gives - up immediately on failing to match "foo" the first time. The notation + If we use atomic grouping for the previous example, the matcher gives + up immediately on failing to match "foo" the first time. The notation is a kind of special parenthesis, starting with (?> as in this example: (?>\d+)foo - This kind of parenthesis "locks up" the part of the pattern it con- - tains once it has matched, and a failure further into the pattern is - prevented from backtracking into it. Backtracking past it to previous + This kind of parenthesis "locks up" the part of the pattern it con- + tains once it has matched, and a failure further into the pattern is + prevented from backtracking into it. Backtracking past it to previous items, however, works as normal. - An alternative description is that a subpattern of this type matches - the string of characters that an identical standalone pattern would + An alternative description is that a subpattern of this type matches + the string of characters that an identical standalone pattern would match, if anchored at the current point in the subject string. Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as the above example can be thought of as a maximizing repeat that - must swallow everything it can. So, while both \d+ and \d+? are pre- - pared to adjust the number of digits they match in order to make the + must swallow everything it can. So, while both \d+ and \d+? are pre- + pared to adjust the number of digits they match in order to make the rest of the pattern match, (?>\d+) can only match an entire sequence of digits. - Atomic groups in general can of course contain arbitrarily complicated - subpatterns, and can be nested. However, when the subpattern for an + Atomic groups in general can of course contain arbitrarily complicated + subpatterns, and can be nested. However, when the subpattern for an atomic group is just a single repeated item, as in the example above, a - simpler notation, called a "possessive quantifier" can be used. This - consists of an additional + character following a quantifier. Using + simpler notation, called a "possessive quantifier" can be used. This + consists of an additional + character following a quantifier. Using this notation, the previous example can be rewritten as \d++foo @@ -4225,45 +4352,45 @@ ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS (abc|xyz){2,3}+ - Possessive quantifiers are always greedy; the setting of the + Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY option is ignored. They are a convenient notation for the - simpler forms of atomic group. However, there is no difference in the - meaning of a possessive quantifier and the equivalent atomic group, - though there may be a performance difference; possessive quantifiers + simpler forms of atomic group. However, there is no difference in the + meaning of a possessive quantifier and the equivalent atomic group, + though there may be a performance difference; possessive quantifiers should be slightly faster. - The possessive quantifier syntax is an extension to the Perl 5.8 syn- - tax. Jeffrey Friedl originated the idea (and the name) in the first + The possessive quantifier syntax is an extension to the Perl 5.8 syn- + tax. Jeffrey Friedl originated the idea (and the name) in the first edition of his book. Mike McCloskey liked it, so implemented it when he - built Sun's Java package, and PCRE copied it from there. It ultimately + built Sun's Java package, and PCRE copied it from there. It ultimately found its way into Perl at release 5.10. PCRE has an optimization that automatically "possessifies" certain sim- - ple pattern constructs. For example, the sequence A+B is treated as - A++B because there is no point in backtracking into a sequence of A's + ple pattern constructs. For example, the sequence A+B is treated as + A++B because there is no point in backtracking into a sequence of A's when B must follow. - When a pattern contains an unlimited repeat inside a subpattern that - can itself be repeated an unlimited number of times, the use of an - atomic group is the only way to avoid some failing matches taking a + When a pattern contains an unlimited repeat inside a subpattern that + can itself be repeated an unlimited number of times, the use of an + atomic group is the only way to avoid some failing matches taking a very long time indeed. The pattern (\D+|<\d+>)*[!?] - matches an unlimited number of substrings that either consist of non- - digits, or digits enclosed in <>, followed by either ! or ?. When it + matches an unlimited number of substrings that either consist of non- + digits, or digits enclosed in <>, followed by either ! or ?. When it matches, it runs quickly. However, if it is applied to aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa - it takes a long time before reporting failure. This is because the - string can be divided between the internal \D+ repeat and the external - * repeat in a large number of ways, and all have to be tried. (The - example uses [!?] rather than a single character at the end, because - both PCRE and Perl have an optimization that allows for fast failure - when a single character is used. They remember the last single charac- - ter that is required for a match, and fail early if it is not present - in the string.) If the pattern is changed so that it uses an atomic + it takes a long time before reporting failure. This is because the + string can be divided between the internal \D+ repeat and the external + * repeat in a large number of ways, and all have to be tried. (The + example uses [!?] rather than a single character at the end, because + both PCRE and Perl have an optimization that allows for fast failure + when a single character is used. They remember the last single charac- + ter that is required for a match, and fail early if it is not present + in the string.) If the pattern is changed so that it uses an atomic group, like this: ((?>\D+)|<\d+>)*[!?] @@ -4275,37 +4402,37 @@ BACK REFERENCES Outside a character class, a backslash followed by a digit greater than 0 (and possibly further digits) is a back reference to a capturing sub- - pattern earlier (that is, to its left) in the pattern, provided there + pattern earlier (that is, to its left) in the pattern, provided there have been that many previous capturing left parentheses. However, if the decimal number following the backslash is less than 10, - it is always taken as a back reference, and causes an error only if - there are not that many capturing left parentheses in the entire pat- - tern. In other words, the parentheses that are referenced need not be - to the left of the reference for numbers less than 10. A "forward back - reference" of this type can make sense when a repetition is involved - and the subpattern to the right has participated in an earlier itera- + it is always taken as a back reference, and causes an error only if + there are not that many capturing left parentheses in the entire pat- + tern. In other words, the parentheses that are referenced need not be + to the left of the reference for numbers less than 10. A "forward back + reference" of this type can make sense when a repetition is involved + and the subpattern to the right has participated in an earlier itera- tion. - It is not possible to have a numerical "forward back reference" to a - subpattern whose number is 10 or more using this syntax because a - sequence such as \50 is interpreted as a character defined in octal. + It is not possible to have a numerical "forward back reference" to a + subpattern whose number is 10 or more using this syntax because a + sequence such as \50 is interpreted as a character defined in octal. See the subsection entitled "Non-printing characters" above for further - details of the handling of digits following a backslash. There is no - such problem when named parentheses are used. A back reference to any + details of the handling of digits following a backslash. There is no + such problem when named parentheses are used. A back reference to any subpattern is possible using named parentheses (see below). - Another way of avoiding the ambiguity inherent in the use of digits + Another way of avoiding the ambiguity inherent in the use of digits following a backslash is to use the \g escape sequence, which is a fea- - ture introduced in Perl 5.10. This escape must be followed by an - unsigned number or a negative number, optionally enclosed in braces. + ture introduced in Perl 5.10. This escape must be followed by an + unsigned number or a negative number, optionally enclosed in braces. These examples are all identical: (ring), \1 (ring), \g1 (ring), \g{1} - An unsigned number specifies an absolute reference without the ambigu- + An unsigned number specifies an absolute reference without the ambigu- ity that is present in the older syntax. It is also useful when literal digits follow the reference. A negative number is a relative reference. Consider this example: @@ -4313,33 +4440,33 @@ BACK REFERENCES (abc(def)ghi)\g{-1} The sequence \g{-1} is a reference to the most recently started captur- - ing subpattern before \g, that is, is it equivalent to \2. Similarly, + ing subpattern before \g, that is, is it equivalent to \2. Similarly, \g{-2} would be equivalent to \1. The use of relative references can be - helpful in long patterns, and also in patterns that are created by + helpful in long patterns, and also in patterns that are created by joining together fragments that contain references within themselves. - A back reference matches whatever actually matched the capturing sub- - pattern in the current subject string, rather than anything matching + A back reference matches whatever actually matched the capturing sub- + pattern in the current subject string, rather than anything matching the subpattern itself (see "Subpatterns as subroutines" below for a way of doing that). So the pattern (sens|respons)e and \1ibility - matches "sense and sensibility" and "response and responsibility", but - not "sense and responsibility". If caseful matching is in force at the - time of the back reference, the case of letters is relevant. For exam- + matches "sense and sensibility" and "response and responsibility", but + not "sense and responsibility". If caseful matching is in force at the + time of the back reference, the case of letters is relevant. For exam- ple, ((?i)rah)\s+\1 - matches "rah rah" and "RAH RAH", but not "RAH rah", even though the + matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original capturing subpattern is matched caselessly. - There are several different ways of writing back references to named - subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or - \k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's + There are several different ways of writing back references to named + subpatterns. The .NET syntax \k{name} and the Perl syntax \k<name> or + \k'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified back reference syntax, in which \g can be used for both numeric - and named references, is also supported. We could rewrite the above + and named references, is also supported. We could rewrite the above example in any of the following ways: (?<p1>(?i)rah)\s+\k<p1> @@ -4347,22 +4474,25 @@ BACK REFERENCES (?P<p1>(?i)rah)\s+(?P=p1) (?<p1>(?i)rah)\s+\g{p1} - A subpattern that is referenced by name may appear in the pattern + A subpattern that is referenced by name may appear in the pattern before or after the reference. - There may be more than one back reference to the same subpattern. If a - subpattern has not actually been used in a particular match, any back - references to it always fail. For example, the pattern + There may be more than one back reference to the same subpattern. If a + subpattern has not actually been used in a particular match, any back + references to it always fail by default. For example, the pattern (a|(bc))\2 - always fails if it starts to match "a" rather than "bc". Because there - may be many capturing parentheses in a pattern, all digits following - the backslash are taken as part of a potential back reference number. - If the pattern continues with a digit character, some delimiter must be - used to terminate the back reference. If the PCRE_EXTENDED option is - set, this can be whitespace. Otherwise an empty comment (see "Com- - ments" below) can be used. + always fails if it starts to match "a" rather than "bc". However, if + the PCRE_JAVASCRIPT_COMPAT option is set at compile time, a back refer- + ence to an unset value matches an empty string. + + Because there may be many capturing parentheses in a pattern, all dig- + its following a backslash are taken as part of a potential back refer- + ence number. If the pattern continues with a digit character, some + delimiter must be used to terminate the back reference. If the + PCRE_EXTENDED option is set, this can be whitespace. Otherwise, the \g{ + syntax or an empty comment (see "Comments" below) can be used. A back reference that occurs inside the parentheses to which it refers fails when the subpattern is first used, so, for example, (a\1) never @@ -4425,19 +4555,20 @@ ASSERTIONS If you want to force a matching failure at some point in a pattern, the most convenient way to do it is with (?!) because an empty string always matches, so an assertion that requires there not to be an empty - string must always fail. + string must always fail. The Perl 5.10 backtracking control verb + (*FAIL) or (*F) is essentially a synonym for (?!). Lookbehind assertions - Lookbehind assertions start with (?<= for positive assertions and (?<! + Lookbehind assertions start with (?<= for positive assertions and (?<! for negative assertions. For example, (?<!foo)bar - does find an occurrence of "bar" that is not preceded by "foo". The - contents of a lookbehind assertion are restricted such that all the + does find an occurrence of "bar" that is not preceded by "foo". The + contents of a lookbehind assertion are restricted such that all the strings it matches must have a fixed length. However, if there are sev- - eral top-level alternatives, they do not all have to have the same + eral top-level alternatives, they do not all have to have the same fixed length. Thus (?<=bullock|donkey) @@ -4446,23 +4577,22 @@ ASSERTIONS (?<!dogs?|cats?) - causes an error at compile time. Branches that match different length - strings are permitted only at the top level of a lookbehind assertion. - This is an extension compared with Perl (at least for 5.8), which - requires all branches to match the same length of string. An assertion - such as + causes an error at compile time. Branches that match different length + strings are permitted only at the top level of a lookbehind assertion. + This is an extension compared with Perl (5.8 and 5.10), which requires + all branches to match the same length of string. An assertion such as (?<=ab(c|de)) is not permitted, because its single top-level branch can match two - different lengths, but it is acceptable if rewritten to use two top- - level branches: + different lengths, but it is acceptable to PCRE if rewritten to use two + top-level branches: (?<=abc|abde) In some cases, the Perl 5.10 escape sequence \K (see above) can be used - instead of a lookbehind assertion; this is not restricted to a fixed- - length. + instead of a lookbehind assertion to get round the fixed-length + restriction. The implementation of lookbehind assertions is, for each alternative, to temporarily move the current position back by the fixed length and @@ -4474,9 +4604,13 @@ ASSERTIONS ble to calculate the length of the lookbehind. The \X and \R escapes, which can match different numbers of bytes, are also not permitted. + "Subroutine" calls (see below) such as (?2) or (?&X) are permitted in + lookbehinds, as long as the subpattern matches a fixed-length string. + Recursion, however, is not supported. + Possessive quantifiers can be used in conjunction with lookbehind - assertions to specify efficient matching at the end of the subject - string. Consider a simple pattern such as + assertions to specify efficient matching of fixed-length strings at the + end of subject strings. Consider a simple pattern such as abcd$ @@ -4539,9 +4673,9 @@ CONDITIONAL SUBPATTERNS It is possible to cause the matching process to obey a subpattern con- ditionally or to choose between two alternative subpatterns, depending - on the result of an assertion, or whether a previous capturing subpat- - tern matched or not. The two possible forms of conditional subpattern - are + on the result of an assertion, or whether a specific capturing subpat- + tern has already been matched. The two possible forms of conditional + subpattern are: (?(condition)yes-pattern) (?(condition)yes-pattern|no-pattern) @@ -4556,138 +4690,144 @@ CONDITIONAL SUBPATTERNS Checking for a used subpattern by number If the text between the parentheses consists of a sequence of digits, - the condition is true if the capturing subpattern of that number has - previously matched. An alternative notation is to precede the digits - with a plus or minus sign. In this case, the subpattern number is rela- - tive rather than absolute. The most recently opened parentheses can be - referenced by (?(-1), the next most recent by (?(-2), and so on. In - looping constructs it can also make sense to refer to subsequent groups - with constructs such as (?(+2). - - Consider the following pattern, which contains non-significant white + the condition is true if a capturing subpattern of that number has pre- + viously matched. If there is more than one capturing subpattern with + the same number (see the earlier section about duplicate subpattern + numbers), the condition is true if any of them have been set. An alter- + native notation is to precede the digits with a plus or minus sign. In + this case, the subpattern number is relative rather than absolute. The + most recently opened parentheses can be referenced by (?(-1), the next + most recent by (?(-2), and so on. In looping constructs it can also + make sense to refer to subsequent groups with constructs such as + (?(+2). + + Consider the following pattern, which contains non-significant white space to make it more readable (assume the PCRE_EXTENDED option) and to divide it into three parts for ease of discussion: ( \( )? [^()]+ (?(1) \) ) - The first part matches an optional opening parenthesis, and if that + The first part matches an optional opening parenthesis, and if that character is present, sets it as the first captured substring. The sec- - ond part matches one or more characters that are not parentheses. The + ond part matches one or more characters that are not parentheses. The third part is a conditional subpattern that tests whether the first set of parentheses matched or not. If they did, that is, if subject started with an opening parenthesis, the condition is true, and so the yes-pat- - tern is executed and a closing parenthesis is required. Otherwise, - since no-pattern is not present, the subpattern matches nothing. In - other words, this pattern matches a sequence of non-parentheses, + tern is executed and a closing parenthesis is required. Otherwise, + since no-pattern is not present, the subpattern matches nothing. In + other words, this pattern matches a sequence of non-parentheses, optionally enclosed in parentheses. - If you were embedding this pattern in a larger one, you could use a + If you were embedding this pattern in a larger one, you could use a relative reference: ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... - This makes the fragment independent of the parentheses in the larger + This makes the fragment independent of the parentheses in the larger pattern. Checking for a used subpattern by name - Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a - used subpattern by name. For compatibility with earlier versions of - PCRE, which had this facility before Perl, the syntax (?(name)...) is - also recognized. However, there is a possible ambiguity with this syn- - tax, because subpattern names may consist entirely of digits. PCRE - looks first for a named subpattern; if it cannot find one and the name - consists entirely of digits, PCRE looks for a subpattern of that num- - ber, which must be greater than zero. Using subpattern names that con- + Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a + used subpattern by name. For compatibility with earlier versions of + PCRE, which had this facility before Perl, the syntax (?(name)...) is + also recognized. However, there is a possible ambiguity with this syn- + tax, because subpattern names may consist entirely of digits. PCRE + looks first for a named subpattern; if it cannot find one and the name + consists entirely of digits, PCRE looks for a subpattern of that num- + ber, which must be greater than zero. Using subpattern names that con- sist entirely of digits is not recommended. Rewriting the above example to use a named subpattern gives this: (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) + If the name used in a condition of this kind is a duplicate, the test + is applied to all subpatterns of the same name, and is true if any one + of them has matched. Checking for pattern recursion If the condition is the string (R), and there is no subpattern with the - name R, the condition is true if a recursive call to the whole pattern + name R, the condition is true if a recursive call to the whole pattern or any subpattern has been made. If digits or a name preceded by amper- sand follow the letter R, for example: (?(R3)...) or (?(R&name)...) - the condition is true if the most recent recursion is into the subpat- - tern whose number or name is given. This condition does not check the - entire recursion stack. + the condition is true if the most recent recursion is into a subpattern + whose number or name is given. This condition does not check the entire + recursion stack. If the name used in a condition of this kind is a + duplicate, the test is applied to all subpatterns of the same name, and + is true if any one of them is the most recent recursion. - At "top level", all these recursion test conditions are false. Recur- - sive patterns are described below. + At "top level", all these recursion test conditions are false. The + syntax for recursive patterns is described below. Defining subpatterns for use by reference only - If the condition is the string (DEFINE), and there is no subpattern - with the name DEFINE, the condition is always false. In this case, - there may be only one alternative in the subpattern. It is always - skipped if control reaches this point in the pattern; the idea of - DEFINE is that it can be used to define "subroutines" that can be ref- - erenced from elsewhere. (The use of "subroutines" is described below.) - For example, a pattern to match an IPv4 address could be written like + If the condition is the string (DEFINE), and there is no subpattern + with the name DEFINE, the condition is always false. In this case, + there may be only one alternative in the subpattern. It is always + skipped if control reaches this point in the pattern; the idea of + DEFINE is that it can be used to define "subroutines" that can be ref- + erenced from elsewhere. (The use of "subroutines" is described below.) + For example, a pattern to match an IPv4 address could be written like this (ignore whitespace and line breaks): (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) ) \b (?&byte) (\.(?&byte)){3} \b - The first part of the pattern is a DEFINE group inside which a another - group named "byte" is defined. This matches an individual component of - an IPv4 address (a number less than 256). When matching takes place, - this part of the pattern is skipped because DEFINE acts like a false - condition. - - The rest of the pattern uses references to the named group to match the - four dot-separated components of an IPv4 address, insisting on a word - boundary at each end. + The first part of the pattern is a DEFINE group inside which a another + group named "byte" is defined. This matches an individual component of + an IPv4 address (a number less than 256). When matching takes place, + this part of the pattern is skipped because DEFINE acts like a false + condition. The rest of the pattern uses references to the named group + to match the four dot-separated components of an IPv4 address, insist- + ing on a word boundary at each end. Assertion conditions - If the condition is not in any of the above formats, it must be an - assertion. This may be a positive or negative lookahead or lookbehind - assertion. Consider this pattern, again containing non-significant + If the condition is not in any of the above formats, it must be an + assertion. This may be a positive or negative lookahead or lookbehind + assertion. Consider this pattern, again containing non-significant white space, and with the two alternatives on the second line: (?(?=[^a-z]*[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) - The condition is a positive lookahead assertion that matches an - optional sequence of non-letters followed by a letter. In other words, - it tests for the presence of at least one letter in the subject. If a - letter is found, the subject is matched against the first alternative; - otherwise it is matched against the second. This pattern matches - strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are + The condition is a positive lookahead assertion that matches an + optional sequence of non-letters followed by a letter. In other words, + it tests for the presence of at least one letter in the subject. If a + letter is found, the subject is matched against the first alternative; + otherwise it is matched against the second. This pattern matches + strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. COMMENTS - The sequence (?# marks the start of a comment that continues up to the - next closing parenthesis. Nested parentheses are not permitted. The - characters that make up a comment play no part in the pattern matching + The sequence (?# marks the start of a comment that continues up to the + next closing parenthesis. Nested parentheses are not permitted. The + characters that make up a comment play no part in the pattern matching at all. - If the PCRE_EXTENDED option is set, an unescaped # character outside a - character class introduces a comment that continues to immediately + If the PCRE_EXTENDED option is set, an unescaped # character outside a + character class introduces a comment that continues to immediately after the next newline in the pattern. RECURSIVE PATTERNS - Consider the problem of matching a string in parentheses, allowing for - unlimited nested parentheses. Without the use of recursion, the best - that can be done is to use a pattern that matches up to some fixed - depth of nesting. It is not possible to handle an arbitrary nesting + Consider the problem of matching a string in parentheses, allowing for + unlimited nested parentheses. Without the use of recursion, the best + that can be done is to use a pattern that matches up to some fixed + depth of nesting. It is not possible to handle an arbitrary nesting depth. For some time, Perl has provided a facility that allows regular expres- - sions to recurse (amongst other things). It does this by interpolating - Perl code in the expression at run time, and the code can refer to the + sions to recurse (amongst other things). It does this by interpolating + Perl code in the expression at run time, and the code can refer to the expression itself. A Perl pattern using code interpolation to solve the parentheses problem can be created like this: @@ -4697,44 +4837,41 @@ RECURSIVE PATTERNS refers recursively to the pattern in which it appears. Obviously, PCRE cannot support the interpolation of Perl code. Instead, - it supports special syntax for recursion of the entire pattern, and - also for individual subpattern recursion. After its introduction in - PCRE and Python, this kind of recursion was introduced into Perl at - release 5.10. + it supports special syntax for recursion of the entire pattern, and + also for individual subpattern recursion. After its introduction in + PCRE and Python, this kind of recursion was subsequently introduced + into Perl at release 5.10. - A special item that consists of (? followed by a number greater than + A special item that consists of (? followed by a number greater than zero and a closing parenthesis is a recursive call of the subpattern of - the given number, provided that it occurs inside that subpattern. (If - not, it is a "subroutine" call, which is described in the next sec- - tion.) The special item (?R) or (?0) is a recursive call of the entire + the given number, provided that it occurs inside that subpattern. (If + not, it is a "subroutine" call, which is described in the next sec- + tion.) The special item (?R) or (?0) is a recursive call of the entire regular expression. - In PCRE (like Python, but unlike Perl), a recursive subpattern call is - always treated as an atomic group. That is, once it has matched some of - the subject string, it is never re-entered, even if it contains untried - alternatives and there is a subsequent matching failure. - This PCRE pattern solves the nested parentheses problem (assume the PCRE_EXTENDED option is set so that white space is ignored): - \( ( (?>[^()]+) | (?R) )* \) + \( ( [^()]++ | (?R) )* \) First it matches an opening parenthesis. Then it matches any number of substrings which can either be a sequence of non-parentheses, or a recursive match of the pattern itself (that is, a correctly parenthe- - sized substring). Finally there is a closing parenthesis. + sized substring). Finally there is a closing parenthesis. Note the use + of a possessive quantifier to avoid backtracking into sequences of non- + parentheses. If this were part of a larger pattern, you would not want to recurse the entire pattern, so instead you could use this: - ( \( ( (?>[^()]+) | (?1) )* \) ) + ( \( ( [^()]++ | (?1) )* \) ) We have put the pattern into parentheses, and caused the recursion to refer to them instead of the whole pattern. In a larger pattern, keeping track of parenthesis numbers can be - tricky. This is made easier by the use of relative references. (A Perl - 5.10 feature.) Instead of (?1) in the pattern above you can write + tricky. This is made easier by the use of relative references (a Perl + 5.10 feature). Instead of (?1) in the pattern above you can write (?-2) to refer to the second most recently opened parentheses preceding the recursion. In other words, a negative number counts capturing parentheses leftwards from the point at which it is encountered. @@ -4749,46 +4886,40 @@ RECURSIVE PATTERNS syntax for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We could rewrite the above example as follows: - (?<pn> \( ( (?>[^()]+) | (?&pn) )* \) ) + (?<pn> \( ( [^()]++ | (?&pn) )* \) ) If there is more than one subpattern with the same name, the earliest one is used. This particular example pattern that we have been looking at contains - nested unlimited repeats, and so the use of atomic grouping for match- - ing strings of non-parentheses is important when applying the pattern - to strings that do not match. For example, when this pattern is applied - to + nested unlimited repeats, and so the use of a possessive quantifier for + matching strings of non-parentheses is important when applying the pat- + tern to strings that do not match. For example, when this pattern is + applied to (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() - it yields "no match" quickly. However, if atomic grouping is not used, - the match runs for a very long time indeed because there are so many - different ways the + and * repeats can carve up the subject, and all - have to be tested before failure can be reported. + it yields "no match" quickly. However, if a possessive quantifier is + not used, the match runs for a very long time indeed because there are + so many different ways the + and * repeats can carve up the subject, + and all have to be tested before failure can be reported. - At the end of a match, the values set for any capturing subpatterns are - those from the outermost level of the recursion at which the subpattern - value is set. If you want to obtain intermediate values, a callout - function can be used (see below and the pcrecallout documentation). If - the pattern above is matched against + At the end of a match, the values of capturing parentheses are those + from the outermost level. If you want to obtain intermediate values, a + callout function can be used (see below and the pcrecallout documenta- + tion). If the pattern above is matched against (ab(cd)ef) - the value for the capturing parentheses is "ef", which is the last - value taken on at the top level. If additional parentheses are added, - giving - - \( ( ( (?>[^()]+) | (?R) )* ) \) - ^ ^ - ^ ^ + the value for the inner capturing parentheses (numbered 2) is "ef", + which is the last value taken on at the top level. If a capturing sub- + pattern is not matched at the top level, its final value is unset, even + if it is (temporarily) set at a deeper level. - the string they capture is "ab(cd)ef", the contents of the top level - parentheses. If there are more than 15 capturing parentheses in a pat- - tern, PCRE has to obtain extra memory to store data during a recursion, - which it does by using pcre_malloc, freeing it via pcre_free after- - wards. If no memory can be obtained, the match fails with the - PCRE_ERROR_NOMEMORY error. + If there are more than 15 capturing parentheses in a pattern, PCRE has + to obtain extra memory to store data during a recursion, which it does + by using pcre_malloc, freeing it via pcre_free afterwards. If no memory + can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. Do not confuse the (?R) item with the condition (R), which tests for recursion. Consider this pattern, which matches text in angle brack- @@ -4802,6 +4933,80 @@ RECURSIVE PATTERNS two different alternatives for the recursive and non-recursive cases. The (?R) item is the actual recursive call. + Recursion difference from Perl + + In PCRE (like Python, but unlike Perl), a recursive subpattern call is + always treated as an atomic group. That is, once it has matched some of + the subject string, it is never re-entered, even if it contains untried + alternatives and there is a subsequent matching failure. This can be + illustrated by the following pattern, which purports to match a palin- + dromic string that contains an odd number of characters (for example, + "a", "aba", "abcba", "abcdcba"): + + ^(.|(.)(?1)\2)$ + + The idea is that it either matches a single character, or two identical + characters surrounding a sub-palindrome. In Perl, this pattern works; + in PCRE it does not if the pattern is longer than three characters. + Consider the subject string "abcba": + + At the top level, the first character is matched, but as it is not at + the end of the string, the first alternative fails; the second alterna- + tive is taken and the recursion kicks in. The recursive call to subpat- + tern 1 successfully matches the next character ("b"). (Note that the + beginning and end of line tests are not part of the recursion). + + Back at the top level, the next character ("c") is compared with what + subpattern 2 matched, which was "a". This fails. Because the recursion + is treated as an atomic group, there are now no backtracking points, + and so the entire match fails. (Perl is able, at this point, to re- + enter the recursion and try the second alternative.) However, if the + pattern is written with the alternatives in the other order, things are + different: + + ^((.)(?1)\2|.)$ + + This time, the recursing alternative is tried first, and continues to + recurse until it runs out of characters, at which point the recursion + fails. But this time we do have another alternative to try at the + higher level. That is the big difference: in the previous case the + remaining alternative is at a deeper recursion level, which PCRE cannot + use. + + To change the pattern so that matches all palindromic strings, not just + those with an odd number of characters, it is tempting to change the + pattern to this: + + ^((.)(?1)\2|.?)$ + + Again, this works in Perl, but not in PCRE, and for the same reason. + When a deeper recursion has matched a single character, it cannot be + entered again in order to match an empty string. The solution is to + separate the two cases, and write out the odd and even cases as alter- + natives at the higher level: + + ^(?:((.)(?1)\2|)|((.)(?3)\4|.)) + + If you want to match typical palindromic phrases, the pattern has to + ignore all non-word characters, which can be done like this: + + ^\W*+(?:((.)\W*+(?1)\W*+\2|)|((.)\W*+(?3)\W*+\4|\W*+.\W*+))\W*+$ + + If run with the PCRE_CASELESS option, this pattern matches phrases such + as "A man, a plan, a canal: Panama!" and it works well in both PCRE and + Perl. Note the use of the possessive quantifier *+ to avoid backtrack- + ing into sequences of non-word characters. Without this, PCRE takes a + great deal longer (ten times or more) to match typical phrases, and + Perl takes so long that you think it has gone into a loop. + + WARNING: The palindrome-matching patterns above work only if the sub- + ject string does not start with a palindrome that is shorter than the + entire string. For example, although "abcba" is correctly matched, if + the subject is "ababa", PCRE finds the palindrome "aba" at the start, + then fails at top level because the end of the string does not follow. + Once again, it cannot jump back into the recursion to try other alter- + natives, so the entire match fails. + SUBPATTERNS AS SUBROUTINES @@ -4828,10 +5033,12 @@ SUBPATTERNS AS SUBROUTINES two strings. Another example is given in the discussion of DEFINE above. - Like recursive subpatterns, a "subroutine" call is always treated as an + Like recursive subpatterns, a subroutine call is always treated as an atomic group. That is, once it has matched some of the subject string, it is never re-entered, even if it contains untried alternatives and - there is a subsequent matching failure. + there is a subsequent matching failure. Any capturing parentheses that + are set during the subroutine call revert to their previous values + afterwards. When a subpattern is used as a subroutine, processing options such as case-independence are fixed when the subpattern is defined. They cannot @@ -4914,6 +5121,12 @@ BACKTRACKING CONTROL (*FAIL), which behaves like a failing negative assertion, they cause an error if encountered by pcre_dfa_exec(). + If any of these verbs are used in an assertion or subroutine subpattern + (including recursive subpatterns), their effect is confined to that + subpattern; it does not extend to the surrounding pattern. Note that + such subpatterns are processed as anchored at the point where they are + tested. + The new verbs make use of what was previously invalid syntax: an open- ing parenthesis followed by an asterisk. In Perl, they are generally of the form (*VERB:ARG) but PCRE does not support the use of arguments, so @@ -4928,14 +5141,14 @@ BACKTRACKING CONTROL This verb causes the match to end successfully, skipping the remainder of the pattern. When inside a recursion, only the innermost pattern is - ended immediately. PCRE differs from Perl in what happens if the - (*ACCEPT) is inside capturing parentheses. In Perl, the data so far is - captured: in PCRE no data is captured. For example: + ended immediately. If (*ACCEPT) is inside capturing parentheses, the + data so far is captured. (This feature was added to PCRE at release + 8.00.) For example: - A(A|B(*ACCEPT)|C)D + A((?:A|B(*ACCEPT)|C)D) - This matches "AB", "AAD", or "ACD", but when it matches "AB", no data - is captured. + This matches "AB", "AAD", or "ACD"; when it matches "AB", "B" is cap- + tured by the outer parentheses. (*FAIL) or (*F) @@ -4962,9 +5175,9 @@ BACKTRACKING CONTROL This verb causes the whole match to fail outright if the rest of the pattern does not match. Even if the pattern is unanchored, no further - attempts to find a match by advancing the start point take place. Once - (*COMMIT) has been passed, pcre_exec() is committed to finding a match - at the current starting point, or not at all. For example: + attempts to find a match by advancing the starting point take place. + Once (*COMMIT) has been passed, pcre_exec() is committed to finding a + match at the current starting point, or not at all. For example: a+(*COMMIT)b @@ -4996,7 +5209,7 @@ BACKTRACKING CONTROL If the subject is "aaaac...", after the first match attempt fails (starting at the first character in the string), the starting point skips on to start the next attempt at "c". Note that a possessive quan- - tifer does not have the same effect in this example; although it would + tifer does not have the same effect as this example; although it would suppress backtracking during the first match attempt, the second attempt would start at the second character instead of skipping on to "c". @@ -5019,7 +5232,7 @@ BACKTRACKING CONTROL SEE ALSO - pcreapi(3), pcrecallout(3), pcrematching(3), pcre(3). + pcreapi(3), pcrecallout(3), pcrematching(3), pcresyntax(3), pcre(3). AUTHOR @@ -5031,7 +5244,7 @@ AUTHOR REVISION - Last updated: 11 April 2009 + Last updated: 18 October 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -5412,77 +5625,182 @@ PARTIAL MATCHING IN PCRE If the application sees the user's keystrokes one by one, and can check that what has been typed so far is potentially valid, it is able to - raise an error as soon as a mistake is made, possibly beeping and not - reflecting the character that has been typed. This immediate feedback - is likely to be a better user interface than a check that is delayed - until the entire string has been entered. - - PCRE supports the concept of partial matching by means of the PCRE_PAR- - TIAL option, which can be set when calling pcre_exec() or - pcre_dfa_exec(). When this flag is set for pcre_exec(), the return code - PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any time - during the matching process the last part of the subject string matched - part of the pattern. Unfortunately, for non-anchored matching, it is - not possible to obtain the position of the start of the partial match. - No captured data is set when PCRE_ERROR_PARTIAL is returned. + raise an error as soon as a mistake is made, by beeping and not + reflecting the character that has been typed, for example. This immedi- + ate feedback is likely to be a better user interface than a check that + is delayed until the entire string has been entered. Partial matching + can also sometimes be useful when the subject string is very long and + is not all available at once. + + PCRE supports partial matching by means of the PCRE_PARTIAL_SOFT and + PCRE_PARTIAL_HARD options, which can be set when calling pcre_exec() or + pcre_dfa_exec(). For backwards compatibility, PCRE_PARTIAL is a synonym + for PCRE_PARTIAL_SOFT. The essential difference between the two options + is whether or not a partial match is preferred to an alternative com- + plete match, though the details differ between the two matching func- + tions. If both options are set, PCRE_PARTIAL_HARD takes precedence. + + Setting a partial matching option disables two of PCRE's optimizations. + PCRE remembers the last literal byte in a pattern, and abandons match- + ing immediately if such a byte is not present in the subject string. + This optimization cannot be used for a subject string that might match + only partially. If the pattern was studied, PCRE knows the minimum + length of a matching string, and does not bother to run the matching + function on shorter strings. This optimization is also disabled for + partial matching. + + +PARTIAL MATCHING USING pcre_exec() + + A partial match occurs during a call to pcre_exec() whenever the end of + the subject string is reached successfully, but matching cannot con- + tinue because more characters are needed. However, at least one charac- + ter must have been matched. (In other words, a partial match can never + be an empty string.) + + If PCRE_PARTIAL_SOFT is set, the partial match is remembered, but + matching continues as normal, and other alternatives in the pattern are + tried. If no complete match can be found, pcre_exec() returns + PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. If there are at least + two slots in the offsets vector, the first of them is set to the offset + of the earliest character that was inspected when the partial match was + found. For convenience, the second offset points to the end of the + string so that a substring can easily be identified. + + For the majority of patterns, the first offset identifies the start of + the partially matched string. However, for patterns that contain look- + behind assertions, or \K, or begin with \b or \B, earlier characters + have been inspected while carrying out the match. For example: + + /(?<=abc)123/ + + This pattern matches "123", but only if it is preceded by "abc". If the + subject string is "xyzabc12", the offsets after a partial match are for + the substring "abc12", because all these characters are needed if + another match is tried with extra characters added. + + If there is more than one partial match, the first one that was found + provides the data that is returned. Consider this pattern: + + /123\w+X|dogY/ + + If this is matched against the subject string "abc123dog", both alter- + natives fail to match, but the end of the subject is reached during + matching, so PCRE_ERROR_PARTIAL is returned instead of + PCRE_ERROR_NOMATCH. The offsets are set to 3 and 9, identifying + "123dog" as the first partial match that was found. (In this example, + there are two partial matches, because "dog" on its own partially + matches the second alternative.) + + If PCRE_PARTIAL_HARD is set for pcre_exec(), it returns PCRE_ERROR_PAR- + TIAL as soon as a partial match is found, without continuing to search + for possible complete matches. The difference between the two options + can be illustrated by a pattern such as: + + /dog(sbody)?/ + + This matches either "dog" or "dogsbody", greedily (that is, it prefers + the longer string if possible). If it is matched against the string + "dog" with PCRE_PARTIAL_SOFT, it yields a complete match for "dog". + However, if PCRE_PARTIAL_HARD is set, the result is PCRE_ERROR_PARTIAL. + On the other hand, if the pattern is made ungreedy the result is dif- + ferent: + + /dog(sbody)??/ + + In this case the result is always a complete match because pcre_exec() + finds that first, and it never continues after finding a match. It + might be easier to follow this explanation by thinking of the two pat- + terns like this: + + /dog(sbody)?/ is the same as /dogsbody|dog/ + /dog(sbody)??/ is the same as /dog|dogsbody/ + + The second pattern will never match "dogsbody" when pcre_exec() is + used, because it will always find the shorter match first. + + +PARTIAL MATCHING USING pcre_dfa_exec() + + The pcre_dfa_exec() function moves along the subject string character + by character, without backtracking, searching for all possible matches + simultaneously. If the end of the subject is reached before the end of + the pattern, there is the possibility of a partial match, again pro- + vided that at least one character has matched. + + When PCRE_PARTIAL_SOFT is set, PCRE_ERROR_PARTIAL is returned only if + there have been no complete matches. Otherwise, the complete matches + are returned. However, if PCRE_PARTIAL_HARD is set, a partial match + takes precedence over any complete matches. The portion of the string + that was inspected when the longest partial match was found is set as + the first matching string, provided there are at least two slots in the + offsets vector. + + Because pcre_dfa_exec() always searches for all possible matches, and + there is no difference between greedy and ungreedy repetition, its be- + haviour is different from pcre_exec when PCRE_PARTIAL_HARD is set. Con- + sider the string "dog" matched against the ungreedy pattern shown + above: - When PCRE_PARTIAL is set for pcre_dfa_exec(), the return code - PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end of - the subject is reached, there have been no complete matches, but there - is still at least one matching possibility. The portion of the string - that provided the partial match is set as the first matching string. + /dog(sbody)??/ - Using PCRE_PARTIAL disables one of PCRE's optimizations. PCRE remembers - the last literal byte in a pattern, and abandons matching immediately - if such a byte is not present in the subject string. This optimization - cannot be used for a subject string that might match only partially. + Whereas pcre_exec() stops as soon as it finds the complete match for + "dog", pcre_dfa_exec() also finds the partial match for "dogsbody", and + so returns that when PCRE_PARTIAL_HARD is set. -RESTRICTED PATTERNS FOR PCRE_PARTIAL +PARTIAL MATCHING AND WORD BOUNDARIES - Because of the way certain internal optimizations are implemented in - the pcre_exec() function, the PCRE_PARTIAL option cannot be used with - all patterns. These restrictions do not apply when pcre_dfa_exec() is - used. For pcre_exec(), repeated single characters such as + If a pattern ends with one of sequences \b or \B, which test for word + boundaries, partial matching with PCRE_PARTIAL_SOFT can give counter- + intuitive results. Consider this pattern: - a{2,4} + /\bcat\b/ - and repeated single metasequences such as + This matches "cat", provided there is a word boundary at either end. If + the subject string is "the cat", the comparison of the final "t" with a + following character cannot take place, so a partial match is found. + However, pcre_exec() carries on with normal matching, which matches \b + at the end of the subject when the last character is a letter, thus + finding a complete match. The result, therefore, is not PCRE_ERROR_PAR- + TIAL. The same thing happens with pcre_dfa_exec(), because it also + finds the complete match. - \d+ + Using PCRE_PARTIAL_HARD in this case does yield PCRE_ERROR_PARTIAL, + because then the partial match takes precedence. - are not permitted if the maximum number of occurrences is greater than - one. Optional items such as \d? (where the maximum is one) are permit- - ted. Quantifiers with any values are permitted after parentheses, so - the invalid examples above can be coded thus: - (a){2,4} - (\d)+ +FORMERLY RESTRICTED PATTERNS - These constructions run more slowly, but for the kinds of application - that are envisaged for this facility, this is not felt to be a major - restriction. + For releases of PCRE prior to 8.00, because of the way certain internal + optimizations were implemented in the pcre_exec() function, the + PCRE_PARTIAL option (predecessor of PCRE_PARTIAL_SOFT) could not be + used with all patterns. From release 8.00 onwards, the restrictions no + longer apply, and partial matching with pcre_exec() can be requested + for any pattern. - If PCRE_PARTIAL is set for a pattern that does not conform to the - restrictions, pcre_exec() returns the error code PCRE_ERROR_BADPARTIAL - (-13). You can use the PCRE_INFO_OKPARTIAL call to pcre_fullinfo() to - find out if a compiled pattern can be used for partial matching. + Items that were formerly restricted were repeated single characters and + repeated metasequences. If PCRE_PARTIAL was set for a pattern that did + not conform to the restrictions, pcre_exec() returned the error code + PCRE_ERROR_BADPARTIAL (-13). This error code is no longer in use. The + PCRE_INFO_OKPARTIAL call to pcre_fullinfo() to find out if a compiled + pattern can be used for partial matching now always returns 1. EXAMPLE OF PARTIAL MATCHING USING PCRETEST If the escape sequence \P is present in a pcretest data line, the - PCRE_PARTIAL flag is used for the match. Here is a run of pcretest that - uses the date example quoted above: + PCRE_PARTIAL_SOFT option is used for the match. Here is a run of + pcretest that uses the date example quoted above: re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ data> 25jun04\P 0: 25jun04 1: jun data> 25dec3\P - Partial match + Partial match: 23dec3 data> 3ju\P - Partial match + Partial match: 3ju data> 3juj\P No match data> j\P @@ -5490,36 +5808,23 @@ EXAMPLE OF PARTIAL MATCHING USING PCRETEST The first data string is matched completely, so pcretest shows the matched substrings. The remaining four strings do not match the com- - plete pattern, but the first two are partial matches. The same test, - using pcre_dfa_exec() matching (by means of the \D escape sequence), - produces the following output: + plete pattern, but the first two are partial matches. Similar output is + obtained when pcre_dfa_exec() is used. - re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ - data> 25jun04\P\D - 0: 25jun04 - data> 23dec3\P\D - Partial match: 23dec3 - data> 3ju\P\D - Partial match: 3ju - data> 3juj\P\D - No match - data> j\P\D - No match - - Notice that in this case the portion of the string that was matched is - made available. + If the escape sequence \P is present more than once in a pcretest data + line, the PCRE_PARTIAL_HARD option is set for the match. MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() When a partial match has been found using pcre_dfa_exec(), it is possi- - ble to continue the match by providing additional subject data and - calling pcre_dfa_exec() again with the same compiled regular expres- - sion, this time setting the PCRE_DFA_RESTART option. You must also pass - the same working space as before, because this is where details of the - previous partial match are stored. Here is an example using pcretest, - using the \R escape sequence to set the PCRE_DFA_RESTART option (\P and - \D are as above): + ble to continue the match by providing additional subject data and + calling pcre_dfa_exec() again with the same compiled regular expres- + sion, this time setting the PCRE_DFA_RESTART option. You must pass the + same working space as before, because this is where details of the pre- + vious partial match are stored. Here is an example using pcretest, + using the \R escape sequence to set the PCRE_DFA_RESTART option (\D + specifies the use of pcre_dfa_exec()): re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ data> 23ja\P\D @@ -5527,38 +5832,77 @@ MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() data> n05\R\D 0: n05 - The first call has "23ja" as the subject, and requests partial match- - ing; the second call has "n05" as the subject for the continued - (restarted) match. Notice that when the match is complete, only the - last part is shown; PCRE does not retain the previously partially- - matched string. It is up to the calling program to do that if it needs + The first call has "23ja" as the subject, and requests partial match- + ing; the second call has "n05" as the subject for the continued + (restarted) match. Notice that when the match is complete, only the + last part is shown; PCRE does not retain the previously partially- + matched string. It is up to the calling program to do that if it needs to. - You can set PCRE_PARTIAL with PCRE_DFA_RESTART to continue partial - matching over multiple segments. This facility can be used to pass very - long subject strings to pcre_dfa_exec(). However, some care is needed - for certain types of pattern. + You can set the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with + PCRE_DFA_RESTART to continue partial matching over multiple segments. + This facility can be used to pass very long subject strings to + pcre_dfa_exec(). - 1. If the pattern contains tests for the beginning or end of a line, - you need to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropri- - ate, when the subject string for any call does not contain the begin- - ning or end of a line. - 2. If the pattern contains backward assertions (including \b or \B), - you need to arrange for some overlap in the subject strings to allow - for this. For example, you could pass the subject in chunks that are - 500 bytes long, but in a buffer of 700 bytes, with the starting offset - set to 200 and the previous 200 bytes at the start of the buffer. +MULTI-SEGMENT MATCHING WITH pcre_exec() + + From release 8.00, pcre_exec() can also be used to do multi-segment + matching. Unlike pcre_dfa_exec(), it is not possible to restart the + previous match with a new segment of data. Instead, new data must be + added to the previous subject string, and the entire match re-run, + starting from the point where the partial match occurred. Earlier data + can be discarded. Consider an unanchored pattern that matches dates: + + re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/ + data> The date is 23ja\P + Partial match: 23ja + + At this stage, an application could discard the text preceding "23ja", + add on text from the next segment, and call pcre_exec() again. Unlike + pcre_dfa_exec(), the entire matching string must always be available, + and the complete matching process occurs for each call, so more memory + and more processing time is needed. - 3. Matching a subject string that is split into multiple segments does - not always produce exactly the same result as matching over one single - long string. The difference arises when there are multiple matching - possibilities, because a partial match result is given only when there - are no completed matches in a call to pcre_dfa_exec(). This means that - as soon as the shortest match has been found, continuation to a new - subject segment is no longer possible. Consider this pcretest example: + Note: If the pattern contains lookbehind assertions, or \K, or starts + with \b or \B, the string that is returned for a partial match will + include characters that precede the partially matched string itself, + because these must be retained when adding on more characters for a + subsequent matching attempt. + + +ISSUES WITH MULTI-SEGMENT MATCHING + + Certain types of pattern may give problems with multi-segment matching, + whichever matching function is used. + + 1. If the pattern contains tests for the beginning or end of a line, + you need to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropri- + ate, when the subject string for any call does not contain the begin- + ning or end of a line. + + 2. Lookbehind assertions at the start of a pattern are catered for in + the offsets that are returned for a partial match. However, in theory, + a lookbehind assertion later in the pattern could require even earlier + characters to be inspected, and it might not have been reached when a + partial match occurs. This is probably an extremely unlikely case; you + could guard against it to a certain extent by always including extra + characters at the start. + + 3. Matching a subject string that is split into multiple segments may + not always produce exactly the same result as matching over one single + long string, especially when PCRE_PARTIAL_SOFT is used. The section + "Partial Matching and Word Boundaries" above describes an issue that + arises if the pattern ends with \b or \B. Another kind of difference + may occur when there are multiple matching possibilities, because a + partial match result is given only when there are no completed matches. + This means that as soon as the shortest match has been found, continua- + tion to a new subject segment is no longer possible. Consider again + this pcretest example: re> /dog(sbody)?/ + data> dogsb\P + 0: dog data> do\P\D Partial match: do data> gsb\R\P\D @@ -5567,33 +5911,63 @@ MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() 0: dogsbody 1: dog - The pattern matches the words "dog" or "dogsbody". When the subject is - presented in several parts ("do" and "gsb" being the first two) the - match stops when "dog" has been found, and it is not possible to con- - tinue. On the other hand, if "dogsbody" is presented as a single - string, both matches are found. + The first data line passes the string "dogsb" to pcre_exec(), setting + the PCRE_PARTIAL_SOFT option. Although the string is a partial match + for "dogsbody", the result is not PCRE_ERROR_PARTIAL, because the + shorter string "dog" is a complete match. Similarly, when the subject + is presented to pcre_dfa_exec() in several parts ("do" and "gsb" being + the first two) the match stops when "dog" has been found, and it is not + possible to continue. On the other hand, if "dogsbody" is presented as + a single string, pcre_dfa_exec() finds both matches. + + Because of these problems, it is probably best to use PCRE_PARTIAL_HARD + when matching multi-segment data. The example above then behaves dif- + ferently: + + re> /dog(sbody)?/ + data> dogsb\P\P + Partial match: dogsb + data> do\P\D + Partial match: do + data> gsb\R\P\P\D + Partial match: gsb - Because of this phenomenon, it does not usually make sense to end a - pattern that is going to be matched in this way with a variable repeat. 4. Patterns that contain alternatives at the top level which do not all - start with the same pattern item may not work as expected. For example, - consider this pattern: + start with the same pattern item may not work as expected when + PCRE_DFA_RESTART is used with pcre_dfa_exec(). For example, consider + this pattern: 1234|3789 - If the first part of the subject is "ABC123", a partial match of the - first alternative is found at offset 3. There is no partial match for + If the first part of the subject is "ABC123", a partial match of the + first alternative is found at offset 3. There is no partial match for the second alternative, because such a match does not start at the same - point in the subject string. Attempting to continue with the string - "789" does not yield a match because only those alternatives that match - at one point in the subject are remembered. The problem arises because - the start of the second alternative matches within the first alterna- - tive. There is no problem with anchored patterns or patterns such as: + point in the subject string. Attempting to continue with the string + "7890" does not yield a match because only those alternatives that + match at one point in the subject are remembered. The problem arises + because the start of the second alternative matches within the first + alternative. There is no problem with anchored patterns or patterns + such as: 1234|ABCD - where no string can be a partial match for both alternatives. + where no string can be a partial match for both alternatives. This is + not a problem if pcre_exec() is used, because the entire match has to + be rerun each time: + + re> /1234|3789/ + data> ABC123\P + Partial match: 123 + data> 1237890 + 0: 3789 + + Of course, instead of using PCRE_DFA_PARTIAL, the same technique of re- + running the entire match can also be used with pcre_dfa_exec(). Another + possibility is to work with two buffers. If a partial match at offset n + in the first buffer is followed by "no match" when PCRE_DFA_RESTART is + used on the second buffer, you can then try a new match starting at + offset n+1 in the first buffer. AUTHOR @@ -5605,8 +5979,8 @@ AUTHOR REVISION - Last updated: 04 June 2007 - Copyright (c) 1997-2007 University of Cambridge. + Last updated: 19 October 2009 + Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -5927,6 +6301,10 @@ DESCRIPTION easier to slot in PCRE as a replacement library. Other POSIX options are not even defined. + There are also some other options that are not defined by POSIX. These + have been added at the request of users who want to make use of certain + PCRE-specific features via the POSIX calling interface. + When PCRE is called via these functions, it is only the API that is POSIX-like in style. The syntax and semantics of the regular expres- sions themselves are still those of Perl, subject to the setting of @@ -5981,6 +6359,12 @@ COMPILING A PATTERN ing, the nmatch and pmatch arguments are ignored, and no captured strings are returned. + REG_UNGREEDY + + The PCRE_UNGREEDY option is set when the regular expression is passed + for compilation to the native function. Note that REG_UNGREEDY is not + part of the POSIX standard. + REG_UTF8 The PCRE_UTF8 option is set when the regular expression is passed for @@ -5993,7 +6377,7 @@ COMPILING A PATTERN semantics. In particular, the way it handles newline characters in the subject string is the Perl way, not the POSIX way. Note that setting PCRE_MULTILINE has only some of the effects specified for REG_NEWLINE. - It does not affect the way newlines are matched by . (they aren't) or + It does not affect the way newlines are matched by . (they are not) or by a negative class such as [^a] (they are). The yield of regcomp() is zero on success, and non-zero otherwise. The @@ -6001,6 +6385,10 @@ COMPILING A PATTERN is public: re_nsub contains the number of capturing subpatterns in the regular expression. Various error codes are defined in the header file. + NOTE: If the yield of regcomp() is non-zero, you must not attempt to + use the contents of the preg structure. If, for example, you pass it to + regexec(), the result is undefined and your program is likely to crash. + MATCHING NEWLINE CHARACTERS @@ -6076,36 +6464,39 @@ MATCHING A PATTERN matched strings is returned. The nmatch and pmatch arguments of regexec() are ignored. + If the value of nmatch is zero, or if the value pmatch is NULL, no data + about any matched strings is returned. + Otherwise,the portion of the string that was matched, and also any cap- tured substrings, are returned via the pmatch argument, which points to - an array of nmatch structures of type regmatch_t, containing the mem- - bers rm_so and rm_eo. These contain the offset to the first character - of each substring and the offset to the first character after the end - of each substring, respectively. The 0th element of the vector relates - to the entire portion of string that was matched; subsequent elements - relate to the capturing subpatterns of the regular expression. Unused + an array of nmatch structures of type regmatch_t, containing the mem- + bers rm_so and rm_eo. These contain the offset to the first character + of each substring and the offset to the first character after the end + of each substring, respectively. The 0th element of the vector relates + to the entire portion of string that was matched; subsequent elements + relate to the capturing subpatterns of the regular expression. Unused entries in the array have both structure members set to -1. - A successful match yields a zero return; various error codes are - defined in the header file, of which REG_NOMATCH is the "expected" + A successful match yields a zero return; various error codes are + defined in the header file, of which REG_NOMATCH is the "expected" failure code. ERROR MESSAGES The regerror() function maps a non-zero errorcode from either regcomp() - or regexec() to a printable message. If preg is not NULL, the error + or regexec() to a printable message. If preg is not NULL, the error should have arisen from the use of that structure. A message terminated - by a binary zero is placed in errbuf. The length of the message, - including the zero, is limited to errbuf_size. The yield of the func- + by a binary zero is placed in errbuf. The length of the message, + including the zero, is limited to errbuf_size. The yield of the func- tion is the size of buffer needed to hold the whole message. MEMORY USAGE - Compiling a regular expression causes memory to be allocated and asso- - ciated with the preg structure. The function regfree() frees all such - memory, after which preg may no longer be used as a compiled expres- + Compiling a regular expression causes memory to be allocated and asso- + ciated with the preg structure. The function regfree() frees all such + memory, after which preg may no longer be used as a compiled expres- sion. @@ -6118,7 +6509,7 @@ AUTHOR REVISION - Last updated: 11 March 2009 + Last updated: 02 September 2009 Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ @@ -6474,53 +6865,56 @@ NAME PCRE SAMPLE PROGRAM A simple, complete demonstration program, to get you started with using - PCRE, is supplied in the file pcredemo.c in the PCRE distribution. + PCRE, is supplied in the file pcredemo.c in the PCRE distribution. A + listing of this program is given in the pcredemo documentation. If you + do not have a copy of the PCRE distribution, you can save this listing + to re-create pcredemo.c. The program compiles the regular expression that is its first argument, - and matches it against the subject string in its second argument. No - PCRE options are set, and default character tables are used. If match- - ing succeeds, the program outputs the portion of the subject that + and matches it against the subject string in its second argument. No + PCRE options are set, and default character tables are used. If match- + ing succeeds, the program outputs the portion of the subject that matched, together with the contents of any captured substrings. If the -g option is given on the command line, the program then goes on to check for further matches of the same regular expression in the same - subject string. The logic is a little bit tricky because of the possi- - bility of matching an empty string. Comments in the code explain what + subject string. The logic is a little bit tricky because of the possi- + bility of matching an empty string. Comments in the code explain what is going on. - If PCRE is installed in the standard include and library directories - for your system, you should be able to compile the demonstration pro- - gram using this command: + If PCRE is installed in the standard include and library directories + for your operating system, you should be able to compile the demonstra- + tion program using this command: gcc -o pcredemo pcredemo.c -lpcre - If PCRE is installed elsewhere, you may need to add additional options - to the command line. For example, on a Unix-like system that has PCRE - installed in /usr/local, you can compile the demonstration program + If PCRE is installed elsewhere, you may need to add additional options + to the command line. For example, on a Unix-like system that has PCRE + installed in /usr/local, you can compile the demonstration program using a command like this: gcc -o pcredemo -I/usr/local/include pcredemo.c \ -L/usr/local/lib -lpcre - Once you have compiled the demonstration program, you can run simple + Once you have compiled the demonstration program, you can run simple tests like this: ./pcredemo 'cat|dog' 'the cat sat on the mat' ./pcredemo -g 'cat|dog' 'the dog sat on the cat' - Note that there is a much more comprehensive test program, called - pcretest, which supports many more facilities for testing regular + Note that there is a much more comprehensive test program, called + pcretest, which supports many more facilities for testing regular expressions and the PCRE library. The pcredemo program is provided as a simple coding example. - On some operating systems (e.g. Solaris), when PCRE is not installed in - the standard library directory, you may get an error like this when you - try to run pcredemo: + When you try to run pcredemo when PCRE is not installed in the standard + library directory, you may get an error like this on some operating + systems (e.g. Solaris): - ld.so.1: a.out: fatal: libpcre.so.0: open failed: No such file or + ld.so.1: a.out: fatal: libpcre.so.0: open failed: No such file or directory - This is caused by the way shared library support works on those sys- + This is caused by the way shared library support works on those sys- tems. You need to add -R/usr/local/lib @@ -6537,8 +6931,8 @@ AUTHOR REVISION - Last updated: 23 January 2008 - Copyright (c) 1997-2008 University of Cambridge. + Last updated: 30 September 2009 + Copyright (c) 1997-2009 University of Cambridge. ------------------------------------------------------------------------------ PCRESTACK(3) PCRESTACK(3) diff --git a/ext/pcre/pcrelib/pcre.h b/ext/pcre/pcrelib/pcre.h index c5fc4c13e4..93dff102ac 100644 --- a/ext/pcre/pcrelib/pcre.h +++ b/ext/pcre/pcrelib/pcre.h @@ -41,10 +41,10 @@ POSSIBILITY OF SUCH DAMAGE. /* The current PCRE version information. */ -#define PCRE_MAJOR 7 -#define PCRE_MINOR 9 +#define PCRE_MAJOR 8 +#define PCRE_MINOR 00 #define PCRE_PRERELEASE -#define PCRE_DATE 2009-04-11 +#define PCRE_DATE 2009-10-19 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE, the appropriate @@ -113,7 +113,8 @@ both, so we keep them all distinct. */ #define PCRE_NO_AUTO_CAPTURE 0x00001000 #define PCRE_NO_UTF8_CHECK 0x00002000 #define PCRE_AUTO_CALLOUT 0x00004000 -#define PCRE_PARTIAL 0x00008000 +#define PCRE_PARTIAL_SOFT 0x00008000 +#define PCRE_PARTIAL 0x00008000 /* Backwards compatible synonym */ #define PCRE_DFA_SHORTEST 0x00010000 #define PCRE_DFA_RESTART 0x00020000 #define PCRE_FIRSTLINE 0x00040000 @@ -128,6 +129,8 @@ both, so we keep them all distinct. */ #define PCRE_JAVASCRIPT_COMPAT 0x02000000 #define PCRE_NO_START_OPTIMIZE 0x04000000 #define PCRE_NO_START_OPTIMISE 0x04000000 +#define PCRE_PARTIAL_HARD 0x08000000 +#define PCRE_NOTEMPTY_ATSTART 0x10000000 /* Exec-time and get/set-time error codes */ @@ -174,6 +177,7 @@ both, so we keep them all distinct. */ #define PCRE_INFO_OKPARTIAL 12 #define PCRE_INFO_JCHANGED 13 #define PCRE_INFO_HASCRORLF 14 +#define PCRE_INFO_MINLENGTH 15 /* Request types for pcre_config(). Do not re-arrange, in order to remain compatible. */ diff --git a/ext/pcre/pcrelib/pcre_compile.c b/ext/pcre/pcrelib/pcre_compile.c index 1e0672c5cd..b197c1b548 100644 --- a/ext/pcre/pcrelib/pcre_compile.c +++ b/ext/pcre/pcrelib/pcre_compile.c @@ -339,7 +339,9 @@ static const char error_texts[] = "number is too big\0" "subpattern name expected\0" "digit expected after (?+\0" - "] is an invalid data character in JavaScript compatibility mode"; + "] is an invalid data character in JavaScript compatibility mode\0" + /* 65 */ + "different names for subpatterns of the same number are not allowed"; /* Table to identify digits and hex digits. This is used when compiling @@ -1098,6 +1100,7 @@ if (ptr[0] == CHAR_LEFT_PARENTHESIS) if (name != NULL && lorn == ptr - thisname && strncmp((const char *)name, (const char *)thisname, lorn) == 0) return *count; + term++; } } } @@ -1132,19 +1135,21 @@ for (; *ptr != 0; ptr++) BOOL negate_class = FALSE; for (;;) { - int c = *(++ptr); - if (c == CHAR_BACKSLASH) + if (ptr[1] == CHAR_BACKSLASH) { - if (ptr[1] == CHAR_E) - ptr++; - else if (strncmp((const char *)ptr+1, + if (ptr[2] == CHAR_E) + ptr+= 2; + else if (strncmp((const char *)ptr+2, STR_Q STR_BACKSLASH STR_E, 3) == 0) - ptr += 3; + ptr += 4; else break; } - else if (!negate_class && c == CHAR_CIRCUMFLEX_ACCENT) + else if (!negate_class && ptr[1] == CHAR_CIRCUMFLEX_ACCENT) + { negate_class = TRUE; + ptr++; + } else break; } @@ -1310,7 +1315,9 @@ for (;;) case OP_CALLOUT: case OP_CREF: + case OP_NCREF: case OP_RREF: + case OP_NRREF: case OP_DEF: code += _pcre_OP_lengths[*code]; break; @@ -1326,23 +1333,34 @@ for (;;) /************************************************* -* Find the fixed length of a pattern * +* Find the fixed length of a branch * *************************************************/ -/* Scan a pattern and compute the fixed length of subject that will match it, +/* Scan a branch and compute the fixed length of subject that will match it, if the length is fixed. This is needed for dealing with backward assertions. -In UTF8 mode, the result is in characters rather than bytes. +In UTF8 mode, the result is in characters rather than bytes. The branch is +temporarily terminated with OP_END when this function is called. + +This function is called when a backward assertion is encountered, so that if it +fails, the error message can point to the correct place in the pattern. +However, we cannot do this when the assertion contains subroutine calls, +because they can be forward references. We solve this by remembering this case +and doing the check at the end; a flag specifies which mode we are running in. Arguments: code points to the start of the pattern (the bracket) options the compiling options + atend TRUE if called when the pattern is complete + cd the "compile data" structure -Returns: the fixed length, or -1 if there is no fixed length, +Returns: the fixed length, + or -1 if there is no fixed length, or -2 if \C was encountered + or -3 if an OP_RECURSE item was encountered and atend is FALSE */ static int -find_fixedlength(uschar *code, int options) +find_fixedlength(uschar *code, int options, BOOL atend, compile_data *cd) { int length = -1; @@ -1355,6 +1373,7 @@ branch, check the length against that of the other branches. */ for (;;) { int d; + uschar *ce, *cs; register int op = *cc; switch (op) { @@ -1362,7 +1381,7 @@ for (;;) case OP_BRA: case OP_ONCE: case OP_COND: - d = find_fixedlength(cc + ((op == OP_CBRA)? 2:0), options); + d = find_fixedlength(cc + ((op == OP_CBRA)? 2:0), options, atend, cd); if (d < 0) return d; branchlength += d; do cc += GET(cc, 1); while (*cc == OP_ALT); @@ -1385,6 +1404,21 @@ for (;;) branchlength = 0; break; + /* A true recursion implies not fixed length, but a subroutine call may + be OK. If the subroutine is a forward reference, we can't deal with + it until the end of the pattern, so return -3. */ + + case OP_RECURSE: + if (!atend) return -3; + cs = ce = (uschar *)cd->start_code + GET(cc, 1); /* Start subpattern */ + do ce += GET(ce, 1); while (*ce == OP_ALT); /* End subpattern */ + if (cc > cs && cc < ce) return -1; /* Recursion */ + d = find_fixedlength(cs + 2, options, atend, cd); + if (d < 0) return d; + branchlength += d; + cc += 1 + LINK_SIZE; + break; + /* Skip over assertive subpatterns */ case OP_ASSERT: @@ -1398,7 +1432,9 @@ for (;;) case OP_REVERSE: case OP_CREF: + case OP_NCREF: case OP_RREF: + case OP_NRREF: case OP_DEF: case OP_OPT: case OP_CALLOUT: @@ -1421,10 +1457,8 @@ for (;;) branchlength++; cc += 2; #ifdef SUPPORT_UTF8 - if ((options & PCRE_UTF8) != 0) - { - while ((*cc & 0xc0) == 0x80) cc++; - } + if ((options & PCRE_UTF8) != 0 && cc[-1] >= 0xc0) + cc += _pcre_utf8_table4[cc[-1] & 0x3f]; #endif break; @@ -1435,10 +1469,8 @@ for (;;) branchlength += GET2(cc,1); cc += 4; #ifdef SUPPORT_UTF8 - if ((options & PCRE_UTF8) != 0) - { - while((*cc & 0x80) == 0x80) cc++; - } + if ((options & PCRE_UTF8) != 0 && cc[-1] >= 0xc0) + cc += _pcre_utf8_table4[cc[-1] & 0x3f]; #endif break; @@ -1517,22 +1549,25 @@ for (;;) /************************************************* -* Scan compiled regex for numbered bracket * +* Scan compiled regex for specific bracket * *************************************************/ /* This little function scans through a compiled pattern until it finds a -capturing bracket with the given number. +capturing bracket with the given number, or, if the number is negative, an +instance of OP_REVERSE for a lookbehind. The function is global in the C sense +so that it can be called from pcre_study() when finding the minimum matching +length. Arguments: code points to start of expression utf8 TRUE in UTF-8 mode - number the required bracket number + number the required bracket number or negative to find a lookbehind Returns: pointer to the opcode for the bracket, or NULL if not found */ -static const uschar * -find_bracket(const uschar *code, BOOL utf8, int number) +const uschar * +_pcre_find_bracket(const uschar *code, BOOL utf8, int number) { for (;;) { @@ -1545,6 +1580,14 @@ for (;;) if (c == OP_XCLASS) code += GET(code, 1); + /* Handle recursion */ + + else if (c == OP_REVERSE) + { + if (number < 0) return (uschar *)code; + code += _pcre_OP_lengths[c]; + } + /* Handle capturing bracket */ else if (c == OP_CBRA) @@ -1910,10 +1953,13 @@ for (code = first_significant_code(code + _pcre_OP_lengths[*code], NULL, 0, TRUE case OP_QUERY: case OP_MINQUERY: case OP_POSQUERY: + if (utf8 && code[1] >= 0xc0) code += _pcre_utf8_table4[code[1] & 0x3f]; + break; + case OP_UPTO: case OP_MINUPTO: case OP_POSUPTO: - if (utf8) while ((code[2] & 0xc0) == 0x80) code++; + if (utf8 && code[3] >= 0xc0) code += _pcre_utf8_table4[code[3] & 0x3f]; break; #endif } @@ -3867,10 +3913,15 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ if (repeat_max == 0) goto END_REPEAT; + /*--------------------------------------------------------------------*/ + /* This code is obsolete from release 8.00; the restriction was finally + removed: */ + /* All real repeats make it impossible to handle partial matching (maybe one day we will be able to remove this restriction). */ - if (repeat_max != 1) cd->external_flags |= PCRE_NOPARTIAL; + /* if (repeat_max != 1) cd->external_flags |= PCRE_NOPARTIAL; */ + /*--------------------------------------------------------------------*/ /* Combine the op_type with the repeat_type */ @@ -4017,10 +4068,15 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ goto END_REPEAT; } + /*--------------------------------------------------------------------*/ + /* This code is obsolete from release 8.00; the restriction was finally + removed: */ + /* All real repeats make it impossible to handle partial matching (maybe one day we will be able to remove this restriction). */ - if (repeat_max != 1) cd->external_flags |= PCRE_NOPARTIAL; + /* if (repeat_max != 1) cd->external_flags |= PCRE_NOPARTIAL; */ + /*--------------------------------------------------------------------*/ if (repeat_min == 0 && repeat_max == -1) *code++ = OP_CRSTAR + repeat_type; @@ -4335,11 +4391,20 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ if (possessive_quantifier) { int len; - if (*tempcode == OP_EXACT || *tempcode == OP_TYPEEXACT || - *tempcode == OP_NOTEXACT) + + if (*tempcode == OP_TYPEEXACT) tempcode += _pcre_OP_lengths[*tempcode] + - ((*tempcode == OP_TYPEEXACT && - (tempcode[3] == OP_PROP || tempcode[3] == OP_NOTPROP))? 2:0); + ((tempcode[3] == OP_PROP || tempcode[3] == OP_NOTPROP)? 2 : 0); + + else if (*tempcode == OP_EXACT || *tempcode == OP_NOTEXACT) + { + tempcode += _pcre_OP_lengths[*tempcode]; +#ifdef SUPPORT_UTF8 + if (utf8 && tempcode[-1] >= 0xc0) + tempcode += _pcre_utf8_table4[tempcode[-1] & 0x3f]; +#endif + } + len = code - tempcode; if (len > 0) switch (*tempcode) { @@ -4417,8 +4482,19 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ if (namelen == verbs[i].len && strncmp((char *)name, vn, namelen) == 0) { - *code = verbs[i].op; - if (*code++ == OP_ACCEPT) cd->had_accept = TRUE; + /* Check for open captures before ACCEPT */ + + if (verbs[i].op == OP_ACCEPT) + { + open_capitem *oc; + cd->had_accept = TRUE; + for (oc = cd->open_caps; oc != NULL; oc = oc->next) + { + *code++ = OP_CLOSE; + PUT2INC(code, 0, oc->number); + } + } + *code++ = verbs[i].op; break; } vn += verbs[i].len + 1; @@ -4580,7 +4656,10 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ } /* Otherwise (did not start with "+" or "-"), start by looking for the - name. */ + name. If we find a name, add one to the opcode to change OP_CREF or + OP_RREF into OP_NCREF or OP_NRREF. These behave exactly the same, + except they record that the reference was originally to a name. The + information is used to check duplicate names. */ slot = cd->name_table; for (i = 0; i < cd->names_found; i++) @@ -4595,6 +4674,7 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ { recno = GET2(slot, 0); PUT2(code, 2+LINK_SIZE, recno); + code[1+LINK_SIZE]++; } /* Search the pattern for a forward reference */ @@ -4603,6 +4683,7 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ (options & PCRE_EXTENDED) != 0)) > 0) { PUT2(code, 2+LINK_SIZE, i); + code[1+LINK_SIZE]++; } /* If terminator == 0 it means that the name followed directly after @@ -4795,11 +4876,24 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ } } - /* In the real compile, create the entry in the table */ + /* In the real compile, create the entry in the table, maintaining + alphabetical order. Duplicate names for different numbers are + permitted only if PCRE_DUPNAMES is set. Duplicate names for the same + number are always OK. (An existing number can be re-used if (?| + appears in the pattern.) In either event, a duplicate name results in + a duplicate entry in the table, even if the number is the same. This + is because the number of names, and hence the table size, is computed + in the pre-compile, and it affects various numbers and pointers which + would all have to be modified, and the compiled code moved down, if + duplicates with the same number were omitted from the table. This + doesn't seem worth the hassle. However, *different* names for the + same number are not permitted. */ else { + BOOL dupname = FALSE; slot = cd->name_table; + for (i = 0; i < cd->names_found; i++) { int crc = memcmp(name, slot+2, namelen); @@ -4807,33 +4901,66 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ { if (slot[2+namelen] == 0) { - if ((options & PCRE_DUPNAMES) == 0) + if (GET2(slot, 0) != cd->bracount + 1 && + (options & PCRE_DUPNAMES) == 0) { *errorcodeptr = ERR43; goto FAILED; } + else dupname = TRUE; } - else crc = -1; /* Current name is substring */ + else crc = -1; /* Current name is a substring */ } + + /* Make space in the table and break the loop for an earlier + name. For a duplicate or later name, carry on. We do this for + duplicates so that in the simple case (when ?(| is not used) they + are in order of their numbers. */ + if (crc < 0) { memmove(slot + cd->name_entry_size, slot, (cd->names_found - i) * cd->name_entry_size); break; } + + /* Continue the loop for a later or duplicate name */ + slot += cd->name_entry_size; } + /* For non-duplicate names, check for a duplicate number before + adding the new name. */ + + if (!dupname) + { + uschar *cslot = cd->name_table; + for (i = 0; i < cd->names_found; i++) + { + if (cslot != slot) + { + if (GET2(cslot, 0) == cd->bracount + 1) + { + *errorcodeptr = ERR65; + goto FAILED; + } + } + else i--; + cslot += cd->name_entry_size; + } + } + PUT2(slot, 0, cd->bracount + 1); memcpy(slot + 2, name, namelen); slot[2+namelen] = 0; } } - /* In both cases, count the number of names we've encountered. */ + /* In both pre-compile and compile, count the number of names we've + encountered. */ - ptr++; /* Move past > or ' */ cd->names_found++; + ptr++; /* Move past > or ' */ goto NUMBERED_GROUP; @@ -5002,7 +5129,8 @@ we set the flag only if there is a literal "\r" or "\n" in the class. */ if (lengthptr == NULL) { *code = OP_END; - if (recno != 0) called = find_bracket(cd->start_code, utf8, recno); + if (recno != 0) + called = _pcre_find_bracket(cd->start_code, utf8, recno); /* Forward reference */ @@ -5646,6 +5774,8 @@ uschar *code = *codeptr; uschar *last_branch = code; uschar *start_bracket = code; uschar *reverse_count = NULL; +open_capitem capitem; +int capnumber = 0; int firstbyte, reqbyte; int branchfirstbyte, branchreqbyte; int length; @@ -5672,6 +5802,17 @@ the code that abstracts option settings at the start of the pattern and makes them global. It tests the value of length for (2 + 2*LINK_SIZE) in the pre-compile phase to find out whether anything has yet been compiled or not. */ +/* If this is a capturing subpattern, add to the chain of open capturing items +so that we can detect them if (*ACCEPT) is encountered. */ + +if (*code == OP_CBRA) + { + capnumber = GET2(code, 1 + LINK_SIZE); + capitem.number = capnumber; + capitem.next = cd->open_caps; + cd->open_caps = &capitem; + } + /* Offset is set zero to mark that this bracket is still open */ PUT(code, 1, 0); @@ -5766,21 +5907,29 @@ for (;;) /* If lookbehind, check that this branch matches a fixed-length string, and put the length into the OP_REVERSE item. Temporarily mark the end of the - branch with OP_END. */ + branch with OP_END. If the branch contains OP_RECURSE, the result is -3 + because there may be forward references that we can't check here. Set a + flag to cause another lookbehind check at the end. Why not do it all at the + end? Because common, erroneous checks are picked up here and the offset of + the problem can be shown. */ if (lookbehind) { int fixed_length; *code = OP_END; - fixed_length = find_fixedlength(last_branch, options); + fixed_length = find_fixedlength(last_branch, options, FALSE, cd); DPRINTF(("fixed length = %d\n", fixed_length)); - if (fixed_length < 0) + if (fixed_length == -3) + { + cd->check_lookbehind = TRUE; + } + else if (fixed_length < 0) { *errorcodeptr = (fixed_length == -2)? ERR36 : ERR25; *ptrptr = ptr; return FALSE; } - PUT(reverse_count, 0, fixed_length); + else { PUT(reverse_count, 0, fixed_length); } } } @@ -5808,6 +5957,10 @@ for (;;) while (branch_length > 0); } + /* If it was a capturing subpattern, remove it from the chain. */ + + if (capnumber > 0) cd->open_caps = cd->open_caps->next; + /* Fill in the ket */ *code = OP_KET; @@ -6010,7 +6163,9 @@ do { switch (*scode) { case OP_CREF: + case OP_NCREF: case OP_RREF: + case OP_NRREF: case OP_DEF: return FALSE; @@ -6179,9 +6334,7 @@ int length = 1; /* For final END opcode */ int firstbyte, reqbyte, newline; int errorcode = 0; int skipatstart = 0; -#ifdef SUPPORT_UTF8 -BOOL utf8; -#endif +BOOL utf8 = (options & PCRE_UTF8) != 0; size_t size; uschar *code; const uschar *codestart; @@ -6278,7 +6431,6 @@ while (ptr[skipatstart] == CHAR_LEFT_PARENTHESIS && /* Can't support UTF8 unless PCRE has been compiled to include the code. */ #ifdef SUPPORT_UTF8 -utf8 = (options & PCRE_UTF8) != 0; if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0 && (*erroroffset = _pcre_valid_utf8((uschar *)pattern, -1)) >= 0) { @@ -6286,7 +6438,7 @@ if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0 && goto PCRE_EARLY_ERROR_RETURN2; } #else -if ((options & PCRE_UTF8) != 0) +if (utf8) { errorcode = ERR32; goto PCRE_EARLY_ERROR_RETURN; @@ -6375,6 +6527,7 @@ cd->end_pattern = (const uschar *)(pattern + strlen(pattern)); cd->req_varyopt = 0; cd->external_options = options; cd->external_flags = 0; +cd->open_caps = NULL; /* Now do the pre-compile. On error, errorcode will be set non-zero, so we don't need to look at the result of the function here. The initial options have @@ -6449,6 +6602,8 @@ cd->start_code = codestart; cd->hwm = cworkspace; cd->req_varyopt = 0; cd->had_accept = FALSE; +cd->check_lookbehind = FALSE; +cd->open_caps = NULL; /* Set up a starting, non-extracting bracket, then compile the expression. On error, errorcode will be set non-zero, so we don't need to look at the result @@ -6487,7 +6642,7 @@ while (errorcode == 0 && cd->hwm > cworkspace) cd->hwm -= LINK_SIZE; offset = GET(cd->hwm, 0); recno = GET(codestart, offset); - groupptr = find_bracket(codestart, (re->options & PCRE_UTF8) != 0, recno); + groupptr = _pcre_find_bracket(codestart, utf8, recno); if (groupptr == NULL) errorcode = ERR53; else PUT(((uschar *)codestart), offset, groupptr - codestart); } @@ -6497,6 +6652,47 @@ subpattern. */ if (errorcode == 0 && re->top_backref > re->top_bracket) errorcode = ERR15; +/* If there were any lookbehind assertions that contained OP_RECURSE +(recursions or subroutine calls), a flag is set for them to be checked here, +because they may contain forward references. Actual recursions can't be fixed +length, but subroutine calls can. It is done like this so that those without +OP_RECURSE that are not fixed length get a diagnosic with a useful offset. The +exceptional ones forgo this. We scan the pattern to check that they are fixed +length, and set their lengths. */ + +if (cd->check_lookbehind) + { + uschar *cc = (uschar *)codestart; + + /* Loop, searching for OP_REVERSE items, and process those that do not have + their length set. (Actually, it will also re-process any that have a length + of zero, but that is a pathological case, and it does no harm.) When we find + one, we temporarily terminate the branch it is in while we scan it. */ + + for (cc = (uschar *)_pcre_find_bracket(codestart, utf8, -1); + cc != NULL; + cc = (uschar *)_pcre_find_bracket(cc, utf8, -1)) + { + if (GET(cc, 1) == 0) + { + int fixed_length; + uschar *be = cc - 1 - LINK_SIZE + GET(cc, -LINK_SIZE); + int end_op = *be; + *be = OP_END; + fixed_length = find_fixedlength(cc, re->options, TRUE, cd); + *be = end_op; + DPRINTF(("fixed length = %d\n", fixed_length)); + if (fixed_length < 0) + { + errorcode = (fixed_length == -2)? ERR36 : ERR25; + break; + } + PUT(cc, 1, fixed_length); + } + cc += 1 + LINK_SIZE; + } + } + /* Failed to compile, or error while post-processing */ if (errorcode != 0) diff --git a/ext/pcre/pcrelib/pcre_exec.c b/ext/pcre/pcrelib/pcre_exec.c index 073cf2410a..ca3079b0a2 100644 --- a/ext/pcre/pcrelib/pcre_exec.c +++ b/ext/pcre/pcrelib/pcre_exec.c @@ -396,10 +396,32 @@ typedef struct heapframe { /* This function is called recursively in many circumstances. Whenever it returns a negative (error) response, the outer incarnation must also return the -same response. +same response. */ -Performance note: It might be tempting to extract commonly used fields from the -md structure (e.g. utf8, end_subject) into individual variables to improve +/* These macros pack up tests that are used for partial matching, and which +appears several times in the code. We set the "hit end" flag if the pointer is +at the end of the subject and also past the start of the subject (i.e. +something has been matched). For hard partial matching, we then return +immediately. The second one is used when we already know we are past the end of +the subject. */ + +#define CHECK_PARTIAL()\ + if (md->partial != 0 && eptr >= md->end_subject && eptr > mstart)\ + {\ + md->hitend = TRUE;\ + if (md->partial > 1) RRETURN(PCRE_ERROR_PARTIAL);\ + } + +#define SCHECK_PARTIAL()\ + if (md->partial != 0 && eptr > mstart)\ + {\ + md->hitend = TRUE;\ + if (md->partial > 1) RRETURN(PCRE_ERROR_PARTIAL);\ + } + + +/* Performance note: It might be tempting to extract commonly used fields from +the md structure (e.g. utf8, end_subject) into individual variables to improve performance. Tests using gcc on a SPARC disproved this; in the first case, it made performance worse. @@ -640,14 +662,6 @@ for (;;) minimize = possessive = FALSE; op = *ecode; - /* For partial matching, remember if we ever hit the end of the subject after - matching at least one subject character. */ - - if (md->partial && - eptr >= md->end_subject && - eptr > mstart) - md->hitend = TRUE; - switch(op) { case OP_FAIL: @@ -823,18 +837,139 @@ for (;;) /* Now see what the actual condition is */ - if (condcode == OP_RREF) /* Recursion test */ + if (condcode == OP_RREF || condcode == OP_NRREF) /* Recursion test */ { - offset = GET2(ecode, LINK_SIZE + 2); /* Recursion group number*/ - condition = md->recursive != NULL && - (offset == RREF_ANY || offset == md->recursive->group_num); - ecode += condition? 3 : GET(ecode, 1); + if (md->recursive == NULL) /* Not recursing => FALSE */ + { + condition = FALSE; + ecode += GET(ecode, 1); + } + else + { + int recno = GET2(ecode, LINK_SIZE + 2); /* Recursion group number*/ + condition = (recno == RREF_ANY || recno == md->recursive->group_num); + + /* If the test is for recursion into a specific subpattern, and it is + false, but the test was set up by name, scan the table to see if the + name refers to any other numbers, and test them. The condition is true + if any one is set. */ + + if (!condition && condcode == OP_NRREF && recno != RREF_ANY) + { + uschar *slotA = md->name_table; + for (i = 0; i < md->name_count; i++) + { + if (GET2(slotA, 0) == recno) break; + slotA += md->name_entry_size; + } + + /* Found a name for the number - there can be only one; duplicate + names for different numbers are allowed, but not vice versa. First + scan down for duplicates. */ + + if (i < md->name_count) + { + uschar *slotB = slotA; + while (slotB > md->name_table) + { + slotB -= md->name_entry_size; + if (strcmp((char *)slotA + 2, (char *)slotB + 2) == 0) + { + condition = GET2(slotB, 0) == md->recursive->group_num; + if (condition) break; + } + else break; + } + + /* Scan up for duplicates */ + + if (!condition) + { + slotB = slotA; + for (i++; i < md->name_count; i++) + { + slotB += md->name_entry_size; + if (strcmp((char *)slotA + 2, (char *)slotB + 2) == 0) + { + condition = GET2(slotB, 0) == md->recursive->group_num; + if (condition) break; + } + else break; + } + } + } + } + + /* Chose branch according to the condition */ + + ecode += condition? 3 : GET(ecode, 1); + } } - else if (condcode == OP_CREF) /* Group used test */ + else if (condcode == OP_CREF || condcode == OP_NCREF) /* Group used test */ { offset = GET2(ecode, LINK_SIZE+2) << 1; /* Doubled ref number */ condition = offset < offset_top && md->offset_vector[offset] >= 0; + + /* If the numbered capture is unset, but the reference was by name, + scan the table to see if the name refers to any other numbers, and test + them. The condition is true if any one is set. This is tediously similar + to the code above, but not close enough to try to amalgamate. */ + + if (!condition && condcode == OP_NCREF) + { + int refno = offset >> 1; + uschar *slotA = md->name_table; + + for (i = 0; i < md->name_count; i++) + { + if (GET2(slotA, 0) == refno) break; + slotA += md->name_entry_size; + } + + /* Found a name for the number - there can be only one; duplicate names + for different numbers are allowed, but not vice versa. First scan down + for duplicates. */ + + if (i < md->name_count) + { + uschar *slotB = slotA; + while (slotB > md->name_table) + { + slotB -= md->name_entry_size; + if (strcmp((char *)slotA + 2, (char *)slotB + 2) == 0) + { + offset = GET2(slotB, 0) << 1; + condition = offset < offset_top && + md->offset_vector[offset] >= 0; + if (condition) break; + } + else break; + } + + /* Scan up for duplicates */ + + if (!condition) + { + slotB = slotA; + for (i++; i < md->name_count; i++) + { + slotB += md->name_entry_size; + if (strcmp((char *)slotA + 2, (char *)slotB + 2) == 0) + { + offset = GET2(slotB, 0) << 1; + condition = offset < offset_top && + md->offset_vector[offset] >= 0; + if (condition) break; + } + else break; + } + } + } + } + + /* Chose branch according to the condition */ + ecode += condition? 3 : GET(ecode, 1); } @@ -895,6 +1030,30 @@ for (;;) break; + /* Before OP_ACCEPT there may be any number of OP_CLOSE opcodes, + to close any currently open capturing brackets. */ + + case OP_CLOSE: + number = GET2(ecode, 1); + offset = number << 1; + +#ifdef DEBUG + printf("end bracket %d at *ACCEPT", number); + printf("\n"); +#endif + + md->capture_last = number; + if (offset >= md->offset_max) md->offset_overflow = TRUE; else + { + md->offset_vector[offset] = + md->offset_vector[md->offset_end - number]; + md->offset_vector[offset+1] = eptr - md->start_subject; + if (offset_top <= offset) offset_top = offset + 2; + } + ecode += 3; + break; + + /* End of the pattern, either real or forced. If we are in a top-level recursion, we should restore the offsets appropriately and continue from after the call. */ @@ -908,16 +1067,26 @@ for (;;) md->recursive = rec->prevrec; memmove(md->offset_vector, rec->offset_save, rec->saved_max * sizeof(int)); + offset_top = rec->save_offset_top; mstart = rec->save_start; ims = original_ims; ecode = rec->after_call; break; } - /* Otherwise, if PCRE_NOTEMPTY is set, fail if we have matched an empty - string - backtracking will then try other alternatives, if any. */ + /* Otherwise, if we have matched an empty string, fail if PCRE_NOTEMPTY is + set, or if PCRE_NOTEMPTY_ATSTART is set and we have matched at the start of + the subject. In both cases, backtracking will then try other alternatives, + if any. */ + + if (eptr == mstart && + (md->notempty || + (md->notempty_atstart && + mstart == md->start_subject + md->start_offset))) + RRETURN(MATCH_NOMATCH); + + /* Otherwise, we have a match. */ - if (md->notempty && eptr == mstart) RRETURN(MATCH_NOMATCH); md->end_match_ptr = eptr; /* Record where we ended */ md->end_offset_top = offset_top; /* and how many extracts were taken */ md->start_match_ptr = mstart; /* and the start (\K can modify) */ @@ -1008,8 +1177,9 @@ for (;;) if (eptr < md->start_subject) RRETURN(MATCH_NOMATCH); } - /* Skip to next op code */ + /* Save the earliest consulted character, then skip to next op code */ + if (eptr < md->start_used_ptr) md->start_used_ptr = eptr; ecode += 1 + LINK_SIZE; break; @@ -1089,6 +1259,7 @@ for (;;) memcpy(new_recursive.offset_save, md->offset_vector, new_recursive.saved_max * sizeof(int)); new_recursive.save_start = mstart; + new_recursive.save_offset_top = offset_top; mstart = eptr; /* OK, now we can do the recursion. For each top-level alternative we @@ -1313,6 +1484,7 @@ for (;;) mstart = rec->save_start; memcpy(md->offset_vector, rec->offset_save, rec->saved_max * sizeof(int)); + offset_top = rec->save_offset_top; ecode = rec->after_call; ims = original_ims; break; @@ -1452,7 +1624,8 @@ for (;;) /* Find out if the previous and current characters are "word" characters. It takes a bit more work in UTF-8 mode. Characters > 255 are assumed to - be "non-word" characters. */ + be "non-word" characters. Remember the earliest consulted character for + partial matching. */ #ifdef SUPPORT_UTF8 if (utf8) @@ -1461,10 +1634,16 @@ for (;;) { USPTR lastptr = eptr - 1; while((*lastptr & 0xc0) == 0x80) lastptr--; + if (lastptr < md->start_used_ptr) md->start_used_ptr = lastptr; GETCHAR(c, lastptr); prev_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; } - if (eptr >= md->end_subject) cur_is_word = FALSE; else + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + cur_is_word = FALSE; + } + else { GETCHAR(c, eptr); cur_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; @@ -1473,13 +1652,20 @@ for (;;) else #endif - /* More streamlined when not in UTF-8 mode */ + /* Not in UTF-8 mode */ { - prev_is_word = (eptr != md->start_subject) && - ((md->ctypes[eptr[-1]] & ctype_word) != 0); - cur_is_word = (eptr < md->end_subject) && - ((md->ctypes[*eptr] & ctype_word) != 0); + if (eptr == md->start_subject) prev_is_word = FALSE; else + { + if (eptr <= md->start_used_ptr) md->start_used_ptr = eptr - 1; + prev_is_word = ((md->ctypes[eptr[-1]] & ctype_word) != 0); + } + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + cur_is_word = FALSE; + } + else cur_is_word = ((md->ctypes[*eptr] & ctype_word) != 0); } /* Now see if the situation is what we want */ @@ -1497,7 +1683,11 @@ for (;;) /* Fall through */ case OP_ALLANY: - if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr++ >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if (utf8) while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; ecode++; break; @@ -1506,12 +1696,20 @@ for (;;) any byte, even newline, independent of the setting of PCRE_DOTALL. */ case OP_ANYBYTE: - if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr++ >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } ecode++; break; case OP_NOT_DIGIT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1524,7 +1722,11 @@ for (;;) break; case OP_DIGIT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1537,7 +1739,11 @@ for (;;) break; case OP_NOT_WHITESPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1550,7 +1756,11 @@ for (;;) break; case OP_WHITESPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1563,7 +1773,11 @@ for (;;) break; case OP_NOT_WORDCHAR: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1576,7 +1790,11 @@ for (;;) break; case OP_WORDCHAR: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if ( #ifdef SUPPORT_UTF8 @@ -1589,7 +1807,11 @@ for (;;) break; case OP_ANYNL: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); switch(c) { @@ -1613,7 +1835,11 @@ for (;;) break; case OP_NOT_HSPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); switch(c) { @@ -1643,7 +1869,11 @@ for (;;) break; case OP_HSPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); switch(c) { @@ -1673,7 +1903,11 @@ for (;;) break; case OP_NOT_VSPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); switch(c) { @@ -1691,7 +1925,11 @@ for (;;) break; case OP_VSPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); switch(c) { @@ -1714,7 +1952,11 @@ for (;;) case OP_PROP: case OP_NOTPROP: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); { const ucd_record *prop = GET_UCD(c); @@ -1759,7 +2001,11 @@ for (;;) is in the binary; otherwise a compile-time error occurs. */ case OP_EXTUNI: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); { int category = UCD_CATEGORY(c); @@ -1839,7 +2085,11 @@ for (;;) break; default: /* No repeat follows */ - if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); + if (!match_ref(offset, eptr, length, md, ims)) + { + CHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } eptr += length; continue; /* With the main loop */ } @@ -1855,7 +2105,11 @@ for (;;) for (i = 1; i <= min; i++) { - if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); + if (!match_ref(offset, eptr, length, md, ims)) + { + CHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } eptr += length; } @@ -1872,8 +2126,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM14); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || !match_ref(offset, eptr, length, md, ims)) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (!match_ref(offset, eptr, length, md, ims)) + { + CHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } eptr += length; } /* Control never gets here */ @@ -1886,7 +2144,11 @@ for (;;) pp = eptr; for (i = min; i < max; i++) { - if (!match_ref(offset, eptr, length, md, ims)) break; + if (!match_ref(offset, eptr, length, md, ims)) + { + CHECK_PARTIAL(); + break; + } eptr += length; } while (eptr >= pp) @@ -1900,8 +2162,6 @@ for (;;) } /* Control never gets here */ - - /* Match a bit-mapped character class, possibly repeatedly. This op code is used when all the characters in the class have values in the range 0-255, and either the matching is caseful, or the characters are in the range @@ -1956,7 +2216,11 @@ for (;;) { for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); if (c > 255) { @@ -1974,7 +2238,11 @@ for (;;) { for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } c = *eptr++; if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); } @@ -1998,7 +2266,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM16); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); if (c > 255) { @@ -2018,7 +2291,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM17); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } c = *eptr++; if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); } @@ -2039,7 +2317,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c > 255) { @@ -2065,7 +2347,11 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if ((data[c/8] & (1 << (c&7))) == 0) break; eptr++; @@ -2127,7 +2413,11 @@ for (;;) for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if (!_pcre_xclass(c, data)) RRETURN(MATCH_NOMATCH); } @@ -2146,7 +2436,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM20); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); if (!_pcre_xclass(c, data)) RRETURN(MATCH_NOMATCH); } @@ -2161,7 +2456,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLENTEST(c, eptr, len); if (!_pcre_xclass(c, data)) break; eptr += len; @@ -2189,7 +2488,11 @@ for (;;) length = 1; ecode++; GETCHARLEN(fc, ecode, length); - if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + if (length > md->end_subject - eptr) + { + CHECK_PARTIAL(); /* Not SCHECK_PARTIAL() */ + RRETURN(MATCH_NOMATCH); + } while (length-- > 0) if (*ecode++ != *eptr++) RRETURN(MATCH_NOMATCH); } else @@ -2197,7 +2500,11 @@ for (;;) /* Non-UTF-8 mode */ { - if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); + if (md->end_subject - eptr < 1) + { + SCHECK_PARTIAL(); /* This one can use SCHECK_PARTIAL() */ + RRETURN(MATCH_NOMATCH); + } if (ecode[1] != *eptr++) RRETURN(MATCH_NOMATCH); ecode += 2; } @@ -2213,7 +2520,11 @@ for (;;) ecode++; GETCHARLEN(fc, ecode, length); - if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + if (length > md->end_subject - eptr) + { + CHECK_PARTIAL(); /* Not SCHECK_PARTIAL() */ + RRETURN(MATCH_NOMATCH); + } /* If the pattern character's value is < 128, we have only one byte, and can use the fast lookup table. */ @@ -2248,7 +2559,11 @@ for (;;) /* Non-UTF-8 mode */ { - if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); + if (md->end_subject - eptr < 1) + { + SCHECK_PARTIAL(); /* This one can use SCHECK_PARTIAL() */ + RRETURN(MATCH_NOMATCH); + } if (md->lcc[ecode[1]] != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); ecode += 2; } @@ -2302,13 +2617,12 @@ for (;;) case OP_MINQUERY: c = *ecode++ - OP_STAR; minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ max = rep_max[c]; /* zero for max => infinity */ if (max == 0) max = INT_MAX; - /* Common code for all repeated single-character matches. We can give - up quickly if there are fewer than the minimum number of characters left in - the subject. */ + /* Common code for all repeated single-character matches. */ REPEATCHAR: #ifdef SUPPORT_UTF8 @@ -2317,7 +2631,6 @@ for (;;) length = 1; charptr = ecode; GETCHARLEN(fc, ecode, length); - if (min * length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); ecode += length; /* Handle multibyte character matching specially here. There is @@ -2335,18 +2648,18 @@ for (;;) for (i = 1; i <= min; i++) { - if (memcmp(eptr, charptr, length) == 0) eptr += length; + if (eptr <= md->end_subject - length && + memcmp(eptr, charptr, length) == 0) eptr += length; #ifdef SUPPORT_UCP - /* Need braces because of following else */ - else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } + else if (oclength > 0 && + eptr <= md->end_subject - oclength && + memcmp(eptr, occhars, oclength) == 0) eptr += oclength; +#endif /* SUPPORT_UCP */ else { - if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); - eptr += oclength; + CHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); } -#else /* without SUPPORT_UCP */ - else { RRETURN(MATCH_NOMATCH); } -#endif /* SUPPORT_UCP */ } if (min == max) continue; @@ -2357,19 +2670,19 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM22); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - if (memcmp(eptr, charptr, length) == 0) eptr += length; + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr <= md->end_subject - length && + memcmp(eptr, charptr, length) == 0) eptr += length; #ifdef SUPPORT_UCP - /* Need braces because of following else */ - else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } + else if (oclength > 0 && + eptr <= md->end_subject - oclength && + memcmp(eptr, occhars, oclength) == 0) eptr += oclength; +#endif /* SUPPORT_UCP */ else { - if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); - eptr += oclength; + CHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); } -#else /* without SUPPORT_UCP */ - else { RRETURN (MATCH_NOMATCH); } -#endif /* SUPPORT_UCP */ } /* Control never gets here */ } @@ -2379,33 +2692,34 @@ for (;;) pp = eptr; for (i = min; i < max; i++) { - if (eptr > md->end_subject - length) break; - if (memcmp(eptr, charptr, length) == 0) eptr += length; + if (eptr <= md->end_subject - length && + memcmp(eptr, charptr, length) == 0) eptr += length; #ifdef SUPPORT_UCP - else if (oclength == 0) break; + else if (oclength > 0 && + eptr <= md->end_subject - oclength && + memcmp(eptr, occhars, oclength) == 0) eptr += oclength; +#endif /* SUPPORT_UCP */ else { - if (memcmp(eptr, occhars, oclength) != 0) break; - eptr += oclength; + CHECK_PARTIAL(); + break; } -#else /* without SUPPORT_UCP */ - else break; -#endif /* SUPPORT_UCP */ } if (possessive) continue; + for(;;) - { - RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM23); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr == pp) RRETURN(MATCH_NOMATCH); + { + RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM23); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr == pp) { RRETURN(MATCH_NOMATCH); } #ifdef SUPPORT_UCP - eptr--; - BACKCHAR(eptr); + eptr--; + BACKCHAR(eptr); #else /* without SUPPORT_UCP */ - eptr -= length; + eptr -= length; #endif /* SUPPORT_UCP */ - } + } } /* Control never gets here */ } @@ -2418,10 +2732,8 @@ for (;;) #endif /* SUPPORT_UTF8 */ /* When not in UTF-8 mode, load a single-byte character. */ - { - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - fc = *ecode++; - } + + fc = *ecode++; /* The value of fc at this point is always less than 256, though we may or may not be in UTF-8 mode. The code is duplicated for the caseless and @@ -2439,7 +2751,14 @@ for (;;) { fc = md->lcc[fc]; for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if (fc != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + } if (min == max) continue; if (minimize) { @@ -2447,9 +2766,13 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM24); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || - fc != md->lcc[*eptr++]) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } + if (fc != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); } /* Control never gets here */ } @@ -2458,10 +2781,17 @@ for (;;) pp = eptr; for (i = min; i < max; i++) { - if (eptr >= md->end_subject || fc != md->lcc[*eptr]) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (fc != md->lcc[*eptr]) break; eptr++; } + if (possessive) continue; + while (eptr >= pp) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM25); @@ -2477,16 +2807,31 @@ for (;;) else { - for (i = 1; i <= min; i++) if (fc != *eptr++) RRETURN(MATCH_NOMATCH); + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (fc != *eptr++) RRETURN(MATCH_NOMATCH); + } + if (min == max) continue; + if (minimize) { for (fi = min;; fi++) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM26); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc != *eptr++) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } + if (fc != *eptr++) RRETURN(MATCH_NOMATCH); } /* Control never gets here */ } @@ -2495,10 +2840,16 @@ for (;;) pp = eptr; for (i = min; i < max; i++) { - if (eptr >= md->end_subject || fc != *eptr) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (fc != *eptr) break; eptr++; } if (possessive) continue; + while (eptr >= pp) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM27); @@ -2514,7 +2865,11 @@ for (;;) checking can be multibyte. */ case OP_NOT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } ecode++; GETCHARINCTEST(c, eptr); if ((ims & PCRE_CASELESS) != 0) @@ -2591,12 +2946,9 @@ for (;;) max = rep_max[c]; /* zero for max => infinity */ if (max == 0) max = INT_MAX; - /* Common code for all repeated single-byte matches. We can give up quickly - if there are fewer than the minimum number of bytes left in the - subject. */ + /* Common code for all repeated single-byte matches. */ REPEATNOTCHAR: - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); fc = *ecode++; /* The code is duplicated for the caseless and caseful cases, for speed, @@ -2621,6 +2973,11 @@ for (;;) register unsigned int d; for (i = 1; i <= min; i++) { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(d, eptr); if (d < 256) d = md->lcc[d]; if (fc == d) RRETURN(MATCH_NOMATCH); @@ -2632,7 +2989,14 @@ for (;;) /* Not UTF-8 mode */ { for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if (fc == md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + } } if (min == max) continue; @@ -2648,11 +3012,15 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM28); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(d, eptr); if (d < 256) d = md->lcc[d]; if (fc == d) RRETURN(MATCH_NOMATCH); - } } else @@ -2663,8 +3031,13 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM29); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc == md->lcc[*eptr++]) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } + if (fc == md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); } } /* Control never gets here */ @@ -2684,7 +3057,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(d, eptr, len); if (d < 256) d = md->lcc[d]; if (fc == d) break; @@ -2705,7 +3082,12 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject || fc == md->lcc[*eptr]) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (fc == md->lcc[*eptr]) break; eptr++; } if (possessive) continue; @@ -2733,6 +3115,11 @@ for (;;) register unsigned int d; for (i = 1; i <= min; i++) { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(d, eptr); if (fc == d) RRETURN(MATCH_NOMATCH); } @@ -2742,7 +3129,14 @@ for (;;) /* Not UTF-8 mode */ { for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if (fc == *eptr++) RRETURN(MATCH_NOMATCH); + } } if (min == max) continue; @@ -2758,7 +3152,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM32); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(d, eptr); if (fc == d) RRETURN(MATCH_NOMATCH); } @@ -2771,8 +3170,13 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM33); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc == *eptr++) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } + if (fc == *eptr++) RRETURN(MATCH_NOMATCH); } } /* Control never gets here */ @@ -2792,7 +3196,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(d, eptr, len); if (fc == d) break; eptr += len; @@ -2812,7 +3220,12 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject || fc == *eptr) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (fc == *eptr) break; eptr++; } if (possessive) continue; @@ -2906,13 +3319,10 @@ for (;;) /* First, ensure the minimum number of matches are present. Use inline code for maximizing the speed, and do the type test once at the start - (i.e. keep it out of the loop). Also we can test that there are at least - the minimum number of bytes before we start. This isn't as effective in - UTF-8 mode, but it does no harm. Separate the UTF-8 code completely as that + (i.e. keep it out of the loop). Separate the UTF-8 code completely as that is tidier. Also separate the UCP code, which can be the same for both UTF-8 and single-bytes. */ - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); if (min > 0) { #ifdef SUPPORT_UCP @@ -2924,7 +3334,11 @@ for (;;) if (prop_fail_result) RRETURN(MATCH_NOMATCH); for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); } break; @@ -2932,7 +3346,11 @@ for (;;) case PT_LAMP: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == ucp_Lu || @@ -2945,7 +3363,11 @@ for (;;) case PT_GC: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_category = UCD_CATEGORY(c); if ((prop_category == prop_value) == prop_fail_result) @@ -2956,7 +3378,11 @@ for (;;) case PT_PC: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == prop_value) == prop_fail_result) @@ -2967,7 +3393,11 @@ for (;;) case PT_SC: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_script = UCD_SCRIPT(c); if ((prop_script == prop_value) == prop_fail_result) @@ -2987,16 +3417,19 @@ for (;;) { for (i = 1; i <= min; i++) { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_category = UCD_CATEGORY(c); if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); while (eptr < md->end_subject) { int len = 1; - if (!utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } + if (!utf8) c = *eptr; + else { GETCHARLEN(c, eptr, len); } prop_category = UCD_CATEGORY(c); if (prop_category != ucp_M) break; eptr += len; @@ -3015,8 +3448,12 @@ for (;;) case OP_ANY: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || IS_NEWLINE(eptr)) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); RRETURN(MATCH_NOMATCH); + } + if (IS_NEWLINE(eptr)) RRETURN(MATCH_NOMATCH); eptr++; while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; } @@ -3025,20 +3462,29 @@ for (;;) case OP_ALLANY: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } eptr++; while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; } break; case OP_ANYBYTE: + if (eptr > md->end_subject - min) RRETURN(MATCH_NOMATCH); eptr += min; break; case OP_ANYNL: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); switch(c) { @@ -3064,7 +3510,11 @@ for (;;) case OP_NOT_HSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); switch(c) { @@ -3096,7 +3546,11 @@ for (;;) case OP_HSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); switch(c) { @@ -3128,7 +3582,11 @@ for (;;) case OP_NOT_VSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); switch(c) { @@ -3148,7 +3606,11 @@ for (;;) case OP_VSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); switch(c) { @@ -3168,7 +3630,11 @@ for (;;) case OP_NOT_DIGIT: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); if (c < 128 && (md->ctypes[c] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); @@ -3178,8 +3644,12 @@ for (;;) case OP_DIGIT: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_digit) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (*eptr >= 128 || (md->ctypes[*eptr++] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); /* No need to skip more bytes - we know it's a 1-byte character */ } @@ -3188,8 +3658,12 @@ for (;;) case OP_NOT_WHITESPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || - (*eptr < 128 && (md->ctypes[*eptr] & ctype_space) != 0)) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (*eptr < 128 && (md->ctypes[*eptr] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); while (++eptr < md->end_subject && (*eptr & 0xc0) == 0x80); } @@ -3198,8 +3672,12 @@ for (;;) case OP_WHITESPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_space) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (*eptr >= 128 || (md->ctypes[*eptr++] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); /* No need to skip more bytes - we know it's a 1-byte character */ } @@ -3218,8 +3696,12 @@ for (;;) case OP_WORDCHAR: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_word) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (*eptr >= 128 || (md->ctypes[*eptr++] & ctype_word) == 0) RRETURN(MATCH_NOMATCH); /* No need to skip more bytes - we know it's a 1-byte character */ } @@ -3233,34 +3715,49 @@ for (;;) #endif /* SUPPORT_UTF8 */ /* Code for the non-UTF-8 case for minimum matching of operators other - than OP_PROP and OP_NOTPROP. We can assume that there are the minimum - number of bytes present, as this was tested above. */ + than OP_PROP and OP_NOTPROP. */ switch(ctype) { case OP_ANY: for (i = 1; i <= min; i++) { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if (IS_NEWLINE(eptr)) RRETURN(MATCH_NOMATCH); eptr++; } break; case OP_ALLANY: + if (eptr > md->end_subject - min) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } eptr += min; break; case OP_ANYBYTE: + if (eptr > md->end_subject - min) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } eptr += min; break; - /* Because of the CRLF case, we can't assume the minimum number of - bytes are present in this case. */ - case OP_ANYNL: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } switch(*eptr++) { default: RRETURN(MATCH_NOMATCH); @@ -3282,7 +3779,11 @@ for (;;) case OP_NOT_HSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } switch(*eptr++) { default: break; @@ -3297,7 +3798,11 @@ for (;;) case OP_HSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } switch(*eptr++) { default: RRETURN(MATCH_NOMATCH); @@ -3312,7 +3817,11 @@ for (;;) case OP_NOT_VSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } switch(*eptr++) { default: break; @@ -3329,7 +3838,11 @@ for (;;) case OP_VSPACE: for (i = 1; i <= min; i++) { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } switch(*eptr++) { default: RRETURN(MATCH_NOMATCH); @@ -3345,34 +3858,76 @@ for (;;) case OP_NOT_DIGIT: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); + } break; case OP_DIGIT: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); + } break; case OP_NOT_WHITESPACE: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); + } break; case OP_WHITESPACE: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); + } break; case OP_NOT_WORDCHAR: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_word) != 0) RRETURN(MATCH_NOMATCH); + } break; case OP_WORDCHAR: for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } if ((md->ctypes[*eptr++] & ctype_word) == 0) RRETURN(MATCH_NOMATCH); + } break; default: @@ -3400,7 +3955,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM36); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); if (prop_fail_result) RRETURN(MATCH_NOMATCH); } @@ -3411,7 +3971,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM37); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == ucp_Lu || @@ -3426,7 +3991,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM38); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); prop_category = UCD_CATEGORY(c); if ((prop_category == prop_value) == prop_fail_result) @@ -3439,7 +4009,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM39); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == prop_value) == prop_fail_result) @@ -3452,7 +4027,12 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM40); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINC(c, eptr); prop_script = UCD_SCRIPT(c); if ((prop_script == prop_value) == prop_fail_result) @@ -3474,17 +4054,20 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM41); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } GETCHARINCTEST(c, eptr); prop_category = UCD_CATEGORY(c); if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); while (eptr < md->end_subject) { int len = 1; - if (!utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } + if (!utf8) c = *eptr; + else { GETCHARLEN(c, eptr, len); } prop_category = UCD_CATEGORY(c); if (prop_category != ucp_M) break; eptr += len; @@ -3503,10 +4086,14 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM42); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || - (ctype == OP_ANY && IS_NEWLINE(eptr))) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (ctype == OP_ANY && IS_NEWLINE(eptr)) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); switch(ctype) { @@ -3662,10 +4249,14 @@ for (;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM43); if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || - (ctype == OP_ANY && IS_NEWLINE(eptr))) + if (fi >= max) RRETURN(MATCH_NOMATCH); + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + RRETURN(MATCH_NOMATCH); + } + if (ctype == OP_ANY && IS_NEWLINE(eptr)) RRETURN(MATCH_NOMATCH); - c = *eptr++; switch(ctype) { @@ -3790,7 +4381,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (prop_fail_result) break; eptr+= len; @@ -3801,7 +4396,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == ucp_Lu || @@ -3816,7 +4415,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); prop_category = UCD_CATEGORY(c); if ((prop_category == prop_value) == prop_fail_result) @@ -3829,7 +4432,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); prop_chartype = UCD_CHARTYPE(c); if ((prop_chartype == prop_value) == prop_fail_result) @@ -3842,7 +4449,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); prop_script = UCD_SCRIPT(c); if ((prop_script == prop_value) == prop_fail_result) @@ -3871,7 +4482,11 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARINCTEST(c, eptr); prop_category = UCD_CATEGORY(c); if (prop_category == ucp_M) break; @@ -3891,6 +4506,7 @@ for (;;) /* eptr is now past the end of the maximum run */ if (possessive) continue; + for(;;) { RMATCH(eptr, ecode, offset_top, md, ims, eptrb, 0, RM45); @@ -3926,7 +4542,12 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject || IS_NEWLINE(eptr)) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (IS_NEWLINE(eptr)) break; eptr++; while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; } @@ -3938,7 +4559,12 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject || IS_NEWLINE(eptr)) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (IS_NEWLINE(eptr)) break; eptr++; while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; } @@ -3950,7 +4576,11 @@ for (;;) { for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } eptr++; while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; } @@ -3963,15 +4593,22 @@ for (;;) case OP_ANYBYTE: c = max - min; if (c > (unsigned int)(md->end_subject - eptr)) - c = md->end_subject - eptr; - eptr += c; + { + eptr = md->end_subject; + SCHECK_PARTIAL(); + } + else eptr += c; break; case OP_ANYNL: for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c == 0x000d) { @@ -3996,7 +4633,11 @@ for (;;) { BOOL gotspace; int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); switch(c) { @@ -4034,7 +4675,11 @@ for (;;) { BOOL gotspace; int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); switch(c) { @@ -4058,7 +4703,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c < 256 && (md->ctypes[c] & ctype_digit) != 0) break; eptr+= len; @@ -4069,7 +4718,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c >= 256 ||(md->ctypes[c] & ctype_digit) == 0) break; eptr+= len; @@ -4080,7 +4733,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c < 256 && (md->ctypes[c] & ctype_space) != 0) break; eptr+= len; @@ -4091,7 +4748,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c >= 256 ||(md->ctypes[c] & ctype_space) == 0) break; eptr+= len; @@ -4102,7 +4763,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c < 256 && (md->ctypes[c] & ctype_word) != 0) break; eptr+= len; @@ -4113,7 +4778,11 @@ for (;;) for (i = min; i < max; i++) { int len = 1; - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } GETCHARLEN(c, eptr, len); if (c >= 256 || (md->ctypes[c] & ctype_word) == 0) break; eptr+= len; @@ -4145,7 +4814,12 @@ for (;;) case OP_ANY: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || IS_NEWLINE(eptr)) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } + if (IS_NEWLINE(eptr)) break; eptr++; } break; @@ -4154,14 +4828,21 @@ for (;;) case OP_ANYBYTE: c = max - min; if (c > (unsigned int)(md->end_subject - eptr)) - c = md->end_subject - eptr; - eptr += c; + { + eptr = md->end_subject; + SCHECK_PARTIAL(); + } + else eptr += c; break; case OP_ANYNL: for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if (c == 0x000d) { @@ -4182,7 +4863,11 @@ for (;;) case OP_NOT_HSPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if (c == 0x09 || c == 0x20 || c == 0xa0) break; eptr++; @@ -4192,7 +4877,11 @@ for (;;) case OP_HSPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if (c != 0x09 && c != 0x20 && c != 0xa0) break; eptr++; @@ -4202,7 +4891,11 @@ for (;;) case OP_NOT_VSPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if (c == 0x0a || c == 0x0b || c == 0x0c || c == 0x0d || c == 0x85) break; @@ -4213,7 +4906,11 @@ for (;;) case OP_VSPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject) break; + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); + break; + } c = *eptr; if (c != 0x0a && c != 0x0b && c != 0x0c && c != 0x0d && c != 0x85) break; @@ -4224,8 +4921,12 @@ for (;;) case OP_NOT_DIGIT: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) != 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_digit) != 0) break; eptr++; } break; @@ -4233,8 +4934,12 @@ for (;;) case OP_DIGIT: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_digit) == 0) break; eptr++; } break; @@ -4242,8 +4947,12 @@ for (;;) case OP_NOT_WHITESPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) != 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_space) != 0) break; eptr++; } break; @@ -4251,8 +4960,12 @@ for (;;) case OP_WHITESPACE: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_space) == 0) break; eptr++; } break; @@ -4260,8 +4973,12 @@ for (;;) case OP_NOT_WORDCHAR: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) != 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_word) != 0) break; eptr++; } break; @@ -4269,8 +4986,12 @@ for (;;) case OP_WORDCHAR: for (i = min; i < max; i++) { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) == 0) + if (eptr >= md->end_subject) + { + SCHECK_PARTIAL(); break; + } + if ((md->ctypes[*eptr] & ctype_word) == 0) break; eptr++; } break; @@ -4448,6 +5169,7 @@ const uschar *tables; const uschar *start_bits = NULL; USPTR start_match = (USPTR)subject + start_offset; USPTR end_subject; +USPTR start_partial = NULL; USPTR req_byte_ptr = start_match - 1; pcre_study_data internal_study; @@ -4464,6 +5186,13 @@ if (re == NULL || subject == NULL || (offsets == NULL && offsetcount > 0)) return PCRE_ERROR_NULL; if (offsetcount < 0) return PCRE_ERROR_BADCOUNT; +/* This information is for finding all the numbers associated with a given +name, for condition testing. */ + +md->name_table = (uschar *)re + re->name_table_offset; +md->name_count = re->name_count; +md->name_entry_size = re->name_entry_size; + /* Fish out the optional data from the extra_data structure, first setting the default values. */ @@ -4531,7 +5260,9 @@ md->jscript_compat = (re->options & PCRE_JAVASCRIPT_COMPAT) != 0; md->notbol = (options & PCRE_NOTBOL) != 0; md->noteol = (options & PCRE_NOTEOL) != 0; md->notempty = (options & PCRE_NOTEMPTY) != 0; -md->partial = (options & PCRE_PARTIAL) != 0; +md->notempty_atstart = (options & PCRE_NOTEMPTY_ATSTART) != 0; +md->partial = ((options & PCRE_PARTIAL_HARD) != 0)? 2 : + ((options & PCRE_PARTIAL_SOFT) != 0)? 1 : 0; md->hitend = FALSE; md->recursive = NULL; /* No recursion at top level */ @@ -4605,8 +5336,9 @@ else } } -/* Partial matching is supported only for a restricted set of regexes at the -moment. */ +/* Partial matching was originally supported only for a restricted set of +regexes; from release 8.00 there are no restrictions, but the bits are still +defined (though never set). So there's no harm in leaving this code. */ if (md->partial && (re->flags & PCRE_NOPARTIAL) != 0) return PCRE_ERROR_BADPARTIAL; @@ -4693,7 +5425,7 @@ if (!anchored) } else if (!startline && study != NULL && - (study->options & PCRE_STUDY_MAPPED) != 0) + (study->flags & PCRE_STUDY_MAPPED) != 0) start_bits = study->start_bits; } @@ -4820,79 +5552,94 @@ for(;;) end_subject = save_end_subject; -#ifdef DEBUG /* Sigh. Some compilers never learn. */ - printf(">>>> Match against: "); - pchars(start_match, end_subject - start_match, TRUE, md); - printf("\n"); -#endif - - /* If req_byte is set, we know that that character must appear in the - subject for the match to succeed. If the first character is set, req_byte - must be later in the subject; otherwise the test starts at the match point. - This optimization can save a huge amount of backtracking in patterns with - nested unlimited repeats that aren't going to match. Writing separate code - for cased/caseless versions makes it go faster, as does using an - autoincrement and backing off on a match. - - HOWEVER: when the subject string is very, very long, searching to its end - can take a long time, and give bad performance on quite ordinary patterns. - This showed up when somebody was matching something like /^\d+C/ on a - 32-megabyte string... so we don't do this when the string is sufficiently - long. - - ALSO: this processing is disabled when partial matching is requested, or if + /* The following two optimizations are disabled for partial matching or if disabling is explicitly requested. */ - if ((options & PCRE_NO_START_OPTIMIZE) == 0 && - req_byte >= 0 && - end_subject - start_match < REQ_BYTE_MAX && - !md->partial) + if ((options & PCRE_NO_START_OPTIMIZE) == 0 && !md->partial) { - register USPTR p = start_match + ((first_byte >= 0)? 1 : 0); + /* If the pattern was studied, a minimum subject length may be set. This is + a lower bound; no actual string of that length may actually match the + pattern. Although the value is, strictly, in characters, we treat it as + bytes to avoid spending too much time in this optimization. */ - /* We don't need to repeat the search if we haven't yet reached the - place we found it at last time. */ + if (study != NULL && (study->flags & PCRE_STUDY_MINLEN) != 0 && + end_subject - start_match < study->minlength) + { + rc = MATCH_NOMATCH; + break; + } - if (p > req_byte_ptr) + /* If req_byte is set, we know that that character must appear in the + subject for the match to succeed. If the first character is set, req_byte + must be later in the subject; otherwise the test starts at the match point. + This optimization can save a huge amount of backtracking in patterns with + nested unlimited repeats that aren't going to match. Writing separate code + for cased/caseless versions makes it go faster, as does using an + autoincrement and backing off on a match. + + HOWEVER: when the subject string is very, very long, searching to its end + can take a long time, and give bad performance on quite ordinary patterns. + This showed up when somebody was matching something like /^\d+C/ on a + 32-megabyte string... so we don't do this when the string is sufficiently + long. */ + + if (req_byte >= 0 && end_subject - start_match < REQ_BYTE_MAX) { - if (req_byte_caseless) + register USPTR p = start_match + ((first_byte >= 0)? 1 : 0); + + /* We don't need to repeat the search if we haven't yet reached the + place we found it at last time. */ + + if (p > req_byte_ptr) { - while (p < end_subject) + if (req_byte_caseless) { - register int pp = *p++; - if (pp == req_byte || pp == req_byte2) { p--; break; } + while (p < end_subject) + { + register int pp = *p++; + if (pp == req_byte || pp == req_byte2) { p--; break; } + } } - } - else - { - while (p < end_subject) + else { - if (*p++ == req_byte) { p--; break; } + while (p < end_subject) + { + if (*p++ == req_byte) { p--; break; } + } } - } - /* If we can't find the required character, break the matching loop, - forcing a match failure. */ + /* If we can't find the required character, break the matching loop, + forcing a match failure. */ - if (p >= end_subject) - { - rc = MATCH_NOMATCH; - break; - } + if (p >= end_subject) + { + rc = MATCH_NOMATCH; + break; + } - /* If we have found the required character, save the point where we - found it, so that we don't search again next time round the loop if - the start hasn't passed this character yet. */ + /* If we have found the required character, save the point where we + found it, so that we don't search again next time round the loop if + the start hasn't passed this character yet. */ - req_byte_ptr = p; + req_byte_ptr = p; + } } } - /* OK, we can now run the match. */ +#ifdef DEBUG /* Sigh. Some compilers never learn. */ + printf(">>>> Match against: "); + pchars(start_match, end_subject - start_match, TRUE, md); + printf("\n"); +#endif + + /* OK, we can now run the match. If "hitend" is set afterwards, remember the + first starting point for which a partial match was found. */ md->start_match_ptr = start_match; + md->start_used_ptr = start_match; md->match_call_count = 0; rc = match(start_match, md->start_code, start_match, 2, md, ims, NULL, 0, 0); + if (md->hitend && start_partial == NULL) start_partial = md->start_used_ptr; switch(rc) { @@ -4922,7 +5669,7 @@ for(;;) rc = MATCH_NOMATCH; goto ENDLOOP; - /* Any other return is some kind of error. */ + /* Any other return is either a match, or some kind of error. */ default: goto ENDLOOP; @@ -5028,14 +5775,19 @@ if (using_temporary_offsets) (pcre_free)(md->offset_vector); } -if (rc != MATCH_NOMATCH) +if (rc != MATCH_NOMATCH && rc != PCRE_ERROR_PARTIAL) { DPRINTF((">>>> error: returning %d\n", rc)); return rc; } -else if (md->partial && md->hitend) +else if (start_partial != NULL) { DPRINTF((">>>> returning PCRE_ERROR_PARTIAL\n")); + if (offsetcount > 1) + { + offsets[0] = start_partial - (USPTR)subject; + offsets[1] = end_subject - (USPTR)subject; + } return PCRE_ERROR_PARTIAL; } else diff --git a/ext/pcre/pcrelib/pcre_fullinfo.c b/ext/pcre/pcrelib/pcre_fullinfo.c index 44fa91bbc0..6725ad7ba5 100644 --- a/ext/pcre/pcrelib/pcre_fullinfo.c +++ b/ext/pcre/pcrelib/pcre_fullinfo.c @@ -117,10 +117,16 @@ switch (what) case PCRE_INFO_FIRSTTABLE: *((const uschar **)where) = - (study != NULL && (study->options & PCRE_STUDY_MAPPED) != 0)? + (study != NULL && (study->flags & PCRE_STUDY_MAPPED) != 0)? ((const pcre_study_data *)extra_data->study_data)->start_bits : NULL; break; + case PCRE_INFO_MINLENGTH: + *((int *)where) = + (study != NULL && (study->flags & PCRE_STUDY_MINLEN) != 0)? + study->minlength : -1; + break; + case PCRE_INFO_LASTLITERAL: *((int *)where) = ((re->flags & PCRE_REQCHSET) != 0)? re->req_byte : -1; @@ -142,6 +148,9 @@ switch (what) *((const uschar **)where) = (const uschar *)(_pcre_default_tables); break; + /* From release 8.00 this will always return TRUE because NOPARTIAL is + no longer ever set (the restrictions have been removed). */ + case PCRE_INFO_OKPARTIAL: *((int *)where) = (re->flags & PCRE_NOPARTIAL) == 0; break; diff --git a/ext/pcre/pcrelib/pcre_internal.h b/ext/pcre/pcrelib/pcre_internal.h index e168f3909b..de0961435b 100644 --- a/ext/pcre/pcrelib/pcre_internal.h +++ b/ext/pcre/pcrelib/pcre_internal.h @@ -535,7 +535,9 @@ Standard C system should have one. */ /* Private flags containing information about the compiled regex. They used to live at the top end of the options word, but that got almost full, so now they -are in a 16-bit flags word. */ +are in a 16-bit flags word. From release 8.00, PCRE_NOPARTIAL is unused, as +the restrictions on partial matching have been lifted. It remains for backwards +compatibility. */ #define PCRE_NOPARTIAL 0x0001 /* can't use partial with this regex */ #define PCRE_FIRSTSET 0x0002 /* first_byte is set */ @@ -547,6 +549,7 @@ are in a 16-bit flags word. */ /* Options for the "extra" block produced by pcre_study(). */ #define PCRE_STUDY_MAPPED 0x01 /* a map of starting chars exists */ +#define PCRE_STUDY_MINLEN 0x02 /* a minimum length field exists */ /* Masks for identifying the public options that are permitted at compile time, run time, or study time, respectively. */ @@ -562,14 +565,15 @@ time, run time, or study time, respectively. */ PCRE_JAVASCRIPT_COMPAT) #define PUBLIC_EXEC_OPTIONS \ - (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NO_UTF8_CHECK| \ - PCRE_PARTIAL|PCRE_NEWLINE_BITS|PCRE_BSR_ANYCRLF|PCRE_BSR_UNICODE| \ - PCRE_NO_START_OPTIMIZE) + (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NOTEMPTY_ATSTART| \ + PCRE_NO_UTF8_CHECK|PCRE_PARTIAL_HARD|PCRE_PARTIAL_SOFT|PCRE_NEWLINE_BITS| \ + PCRE_BSR_ANYCRLF|PCRE_BSR_UNICODE|PCRE_NO_START_OPTIMIZE) #define PUBLIC_DFA_EXEC_OPTIONS \ - (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NO_UTF8_CHECK| \ - PCRE_PARTIAL|PCRE_DFA_SHORTEST|PCRE_DFA_RESTART|PCRE_NEWLINE_BITS| \ - PCRE_BSR_ANYCRLF|PCRE_BSR_UNICODE|PCRE_NO_START_OPTIMIZE) + (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NOTEMPTY_ATSTART| \ + PCRE_NO_UTF8_CHECK|PCRE_PARTIAL_HARD|PCRE_PARTIAL_SOFT|PCRE_DFA_SHORTEST| \ + PCRE_DFA_RESTART|PCRE_NEWLINE_BITS|PCRE_BSR_ANYCRLF|PCRE_BSR_UNICODE| \ + PCRE_NO_START_OPTIMIZE) #define PUBLIC_STUDY_OPTIONS 0 /* None defined */ @@ -1206,8 +1210,8 @@ enum { ESC_A = 1, ESC_G, ESC_K, ESC_B, ESC_b, ESC_D, ESC_d, ESC_S, ESC_s, OP_EOD must correspond in order to the list of escapes immediately above. *** NOTE NOTE NOTE *** Whenever this list is updated, the two macro definitions -that follow must also be updated to match. There is also a table called -"coptable" in pcre_dfa_exec.c that must be updated. */ +that follow must also be updated to match. There are also tables called +"coptable" and "poptable" in pcre_dfa_exec.c that must be updated. */ enum { OP_END, /* 0 End of pattern */ @@ -1343,30 +1347,39 @@ enum { OP_SCBRA, /* 98 Start of capturing bracket, check empty */ OP_SCOND, /* 99 Conditional group, check empty */ + /* The next two pairs must (respectively) be kept together. */ + OP_CREF, /* 100 Used to hold a capture number as condition */ - OP_RREF, /* 101 Used to hold a recursion number as condition */ - OP_DEF, /* 102 The DEFINE condition */ + OP_NCREF, /* 101 Same, but generaged by a name reference*/ + OP_RREF, /* 102 Used to hold a recursion number as condition */ + OP_NRREF, /* 103 Same, but generaged by a name reference*/ + OP_DEF, /* 104 The DEFINE condition */ - OP_BRAZERO, /* 103 These two must remain together and in this */ - OP_BRAMINZERO, /* 104 order. */ + OP_BRAZERO, /* 105 These two must remain together and in this */ + OP_BRAMINZERO, /* 106 order. */ /* These are backtracking control verbs */ - OP_PRUNE, /* 105 */ - OP_SKIP, /* 106 */ - OP_THEN, /* 107 */ - OP_COMMIT, /* 108 */ + OP_PRUNE, /* 107 */ + OP_SKIP, /* 108 */ + OP_THEN, /* 109 */ + OP_COMMIT, /* 110 */ /* These are forced failure and success verbs */ - OP_FAIL, /* 109 */ - OP_ACCEPT, /* 110 */ + OP_FAIL, /* 111 */ + OP_ACCEPT, /* 112 */ + OP_CLOSE, /* 113 Used before OP_ACCEPT to close open captures */ /* This is used to skip a subpattern with a {0} quantifier */ - OP_SKIPZERO /* 111 */ + OP_SKIPZERO /* 114 */ }; +/* *** NOTE NOTE NOTE *** Whenever the list above is updated, the two macro +definitions that follow must also be updated to match. There are also tables +called "coptable" cna "poptable" in pcre_dfa_exec.c that must be updated. */ + /* This macro defines textual names for all the opcodes. These are used only for debugging. The macro is referenced only in pcre_printint.c. */ @@ -1388,9 +1401,10 @@ for debugging. The macro is referenced only in pcre_printint.c. */ "Alt", "Ket", "KetRmax", "KetRmin", "Assert", "Assert not", \ "AssertB", "AssertB not", "Reverse", \ "Once", "Bra", "CBra", "Cond", "SBra", "SCBra", "SCond", \ - "Cond ref", "Cond rec", "Cond def", "Brazero", "Braminzero", \ + "Cond ref", "Cond nref", "Cond rec", "Cond nrec", "Cond def", \ + "Brazero", "Braminzero", \ "*PRUNE", "*SKIP", "*THEN", "*COMMIT", "*FAIL", "*ACCEPT", \ - "Skip zero" + "Close", "Skip zero" /* This macro defines the length of fixed length operations in the compiled @@ -1450,15 +1464,16 @@ in UTF-8 mode. The code that uses this table must know about such things. */ 1+LINK_SIZE, /* SBRA */ \ 3+LINK_SIZE, /* SCBRA */ \ 1+LINK_SIZE, /* SCOND */ \ - 3, /* CREF */ \ - 3, /* RREF */ \ + 3, 3, /* CREF, NCREF */ \ + 3, 3, /* RREF, NRREF */ \ 1, /* DEF */ \ 1, 1, /* BRAZERO, BRAMINZERO */ \ 1, 1, 1, 1, /* PRUNE, SKIP, THEN, COMMIT, */ \ - 1, 1, 1 /* FAIL, ACCEPT, SKIPZERO */ + 1, 1, 3, 1 /* FAIL, ACCEPT, CLOSE, SKIPZERO */ -/* A magic value for OP_RREF to indicate the "any recursion" condition. */ +/* A magic value for OP_RREF and OP_NRREF to indicate the "any recursion" +condition. */ #define RREF_ANY 0xffff @@ -1471,7 +1486,7 @@ enum { ERR0, ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, ERR30, ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, ERR40, ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47, ERR48, ERR49, ERR50, ERR51, ERR52, ERR53, ERR54, ERR55, ERR56, ERR57, ERR58, ERR59, - ERR60, ERR61, ERR62, ERR63, ERR64 }; + ERR60, ERR61, ERR62, ERR63, ERR64, ERR65 }; /* The real format of the start of the pcre block; the index of names and the code vector run on as long as necessary after the end. We store an explicit @@ -1487,7 +1502,7 @@ Because people can now save and re-use compiled patterns, any additions to this structure should be made at the end, and something earlier (e.g. a new flag in the options or one of the dummy fields) should indicate that the new fields are present. Currently PCRE always sets the dummy fields to zero. -NOTE NOTE NOTE: +NOTE NOTE NOTE */ typedef struct real_pcre { @@ -1514,10 +1529,20 @@ remark (see NOTE above) about extending this structure applies. */ typedef struct pcre_study_data { pcre_uint32 size; /* Total that was malloced */ - pcre_uint32 options; - uschar start_bits[32]; + pcre_uint32 flags; /* Private flags */ + uschar start_bits[32]; /* Starting char bits */ + pcre_uint32 minlength; /* Minimum subject length */ } pcre_study_data; +/* Structure for building a chain of open capturing subpatterns during +compiling, so that instructions to close them can be compiled when (*ACCEPT) is +encountered. */ + +typedef struct open_capitem { + struct open_capitem *next; /* Chain link */ + pcre_uint16 number; /* Capture number */ +} open_capitem; + /* Structure for passing "static" information around between the functions doing the compiling, so that they are thread-safe. */ @@ -1530,6 +1555,7 @@ typedef struct compile_data { const uschar *start_code; /* The start of the compiled code */ const uschar *start_pattern; /* The start of the pattern */ const uschar *end_pattern; /* The end of the pattern */ + open_capitem *open_caps; /* Chain of open capture items */ uschar *hwm; /* High watermark of workspace */ uschar *name_table; /* The name/number table */ int names_found; /* Number of entries so far */ @@ -1542,6 +1568,7 @@ typedef struct compile_data { int external_flags; /* External flag bits to be set */ int req_varyopt; /* "After variable item" flag for reqbyte */ BOOL had_accept; /* (*ACCEPT) encountered */ + BOOL check_lookbehind; /* Lookbehinds need later checking */ int nltype; /* Newline type */ int nllen; /* Newline string length */ uschar nl[4]; /* Newline string when fixed length */ @@ -1565,6 +1592,7 @@ typedef struct recursion_info { USPTR save_start; /* Old value of mstart */ int *offset_save; /* Pointer to start of saved offsets */ int saved_max; /* Number of saved offsets */ + int save_offset_top; /* Current value of offset_top */ } recursion_info; /* Structure for building a chain of data for holding the values of the subject @@ -1589,6 +1617,9 @@ typedef struct match_data { int offset_max; /* The maximum usable for return data */ int nltype; /* Newline type */ int nllen; /* Newline string length */ + int name_count; /* Number of names in name table */ + int name_entry_size; /* Size of entry in names table */ + uschar *name_table; /* Table of names */ uschar nl[4]; /* Newline string when fixed */ const uschar *lcc; /* Points to lower casing table */ const uschar *ctypes; /* Points to table of type maps */ @@ -1599,7 +1630,7 @@ typedef struct match_data { BOOL jscript_compat; /* JAVASCRIPT_COMPAT flag */ BOOL endonly; /* Dollar not before final \n */ BOOL notempty; /* Empty string match not wanted */ - BOOL partial; /* PARTIAL flag */ + BOOL notempty_atstart; /* Empty string match at start not wanted */ BOOL hitend; /* Hit the end of the subject at some point */ BOOL bsr_anycrlf; /* \R is just any CRLF, not full Unicode */ const uschar *start_code; /* For use when recursing */ @@ -1607,6 +1638,8 @@ typedef struct match_data { USPTR end_subject; /* End of the subject string */ USPTR start_match_ptr; /* Start of matched string */ USPTR end_match_ptr; /* Subject position at end match */ + USPTR start_used_ptr; /* Earliest consulted character */ + int partial; /* PARTIAL options */ int end_offset_top; /* Highwater mark at end of match */ int capture_last; /* Most recent capture number */ int start_offset; /* The start offset value */ @@ -1623,7 +1656,9 @@ typedef struct dfa_match_data { const uschar *start_code; /* Start of the compiled pattern */ const uschar *start_subject; /* Start of the subject string */ const uschar *end_subject; /* End of subject string */ + const uschar *start_used_ptr; /* Earliest consulted character */ const uschar *tables; /* Character tables */ + int start_offset; /* The start offset value */ int moptions; /* Match options */ int poptions; /* Pattern options */ int nltype; /* Newline type */ @@ -1702,15 +1737,16 @@ extern const uschar _pcre_OP_lengths[]; one of the exported public functions. They have to be "external" in the C sense, but are not part of the PCRE public API. */ -extern BOOL _pcre_is_newline(const uschar *, int, const uschar *, - int *, BOOL); -extern int _pcre_ord2utf8(int, uschar *); -extern real_pcre *_pcre_try_flipped(const real_pcre *, real_pcre *, - const pcre_study_data *, pcre_study_data *); -extern int _pcre_valid_utf8(const uschar *, int); -extern BOOL _pcre_was_newline(const uschar *, int, const uschar *, - int *, BOOL); -extern BOOL _pcre_xclass(int, const uschar *); +extern const uschar *_pcre_find_bracket(const uschar *, BOOL, int); +extern BOOL _pcre_is_newline(const uschar *, int, const uschar *, + int *, BOOL); +extern int _pcre_ord2utf8(int, uschar *); +extern real_pcre *_pcre_try_flipped(const real_pcre *, real_pcre *, + const pcre_study_data *, pcre_study_data *); +extern int _pcre_valid_utf8(const uschar *, int); +extern BOOL _pcre_was_newline(const uschar *, int, const uschar *, + int *, BOOL); +extern BOOL _pcre_xclass(int, const uschar *); /* Unicode character database (UCD) */ diff --git a/ext/pcre/pcrelib/pcre_printint.src b/ext/pcre/pcrelib/pcre_printint.src index 5f45fc1985..acfc4ca688 100644 --- a/ext/pcre/pcrelib/pcre_printint.src +++ b/ext/pcre/pcrelib/pcre_printint.src @@ -246,7 +246,12 @@ for(;;) fprintf(f, "%s", OP_names[*code]); break; + case OP_CLOSE: + fprintf(f, " %s %d", OP_names[*code], GET2(code, 1)); + break; + case OP_CREF: + case OP_NCREF: fprintf(f, "%3d %s", GET2(code,1), OP_names[*code]); break; @@ -258,6 +263,14 @@ for(;;) fprintf(f, " Cond recurse %d", c); break; + case OP_NRREF: + c = GET2(code, 1); + if (c == RREF_ANY) + fprintf(f, " Cond nrecurse any"); + else + fprintf(f, " Cond nrecurse %d", c); + break; + case OP_DEF: fprintf(f, " Cond def"); break; diff --git a/ext/pcre/pcrelib/pcre_study.c b/ext/pcre/pcrelib/pcre_study.c index 226cc65941..a2e9f44faa 100644 --- a/ext/pcre/pcrelib/pcre_study.c +++ b/ext/pcre/pcrelib/pcre_study.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2008 University of Cambridge + Copyright (c) 1997-2009 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -52,6 +52,364 @@ supporting functions. */ enum { SSB_FAIL, SSB_DONE, SSB_CONTINUE }; + +/************************************************* +* Find the minimum subject length for a group * +*************************************************/ + +/* Scan a parenthesized group and compute the minimum length of subject that +is needed to match it. This is a lower bound; it does not mean there is a +string of that length that matches. In UTF8 mode, the result is in characters +rather than bytes. + +Arguments: + code pointer to start of group (the bracket) + startcode pointer to start of the whole pattern + options the compiling options + +Returns: the minimum length + -1 if \C was encountered + -2 internal error (missing capturing bracket) +*/ + +static int +find_minlength(const uschar *code, const uschar *startcode, int options) +{ +int length = -1; +BOOL utf8 = (options & PCRE_UTF8) != 0; +BOOL had_recurse = FALSE; +register int branchlength = 0; +register uschar *cc = (uschar *)code + 1 + LINK_SIZE; + +if (*code == OP_CBRA || *code == OP_SCBRA) cc += 2; + +/* Scan along the opcodes for this branch. If we get to the end of the +branch, check the length against that of the other branches. */ + +for (;;) + { + int d, min; + uschar *cs, *ce; + register int op = *cc; + + switch (op) + { + case OP_CBRA: + case OP_SCBRA: + case OP_BRA: + case OP_SBRA: + case OP_ONCE: + case OP_COND: + case OP_SCOND: + d = find_minlength(cc, startcode, options); + if (d < 0) return d; + branchlength += d; + do cc += GET(cc, 1); while (*cc == OP_ALT); + cc += 1 + LINK_SIZE; + break; + + /* Reached end of a branch; if it's a ket it is the end of a nested + call. If it's ALT it is an alternation in a nested call. If it is + END it's the end of the outer call. All can be handled by the same code. */ + + case OP_ALT: + case OP_KET: + case OP_KETRMAX: + case OP_KETRMIN: + case OP_END: + if (length < 0 || (!had_recurse && branchlength < length)) + length = branchlength; + if (*cc != OP_ALT) return length; + cc += 1 + LINK_SIZE; + branchlength = 0; + had_recurse = FALSE; + break; + + /* Skip over assertive subpatterns */ + + case OP_ASSERT: + case OP_ASSERT_NOT: + case OP_ASSERTBACK: + case OP_ASSERTBACK_NOT: + do cc += GET(cc, 1); while (*cc == OP_ALT); + /* Fall through */ + + /* Skip over things that don't match chars */ + + case OP_REVERSE: + case OP_CREF: + case OP_NCREF: + case OP_RREF: + case OP_NRREF: + case OP_DEF: + case OP_OPT: + case OP_CALLOUT: + case OP_SOD: + case OP_SOM: + case OP_EOD: + case OP_EODN: + case OP_CIRC: + case OP_DOLL: + case OP_NOT_WORD_BOUNDARY: + case OP_WORD_BOUNDARY: + cc += _pcre_OP_lengths[*cc]; + break; + + /* Skip over a subpattern that has a {0} or {0,x} quantifier */ + + case OP_BRAZERO: + case OP_BRAMINZERO: + case OP_SKIPZERO: + cc += _pcre_OP_lengths[*cc]; + do cc += GET(cc, 1); while (*cc == OP_ALT); + cc += 1 + LINK_SIZE; + break; + + /* Handle literal characters and + repetitions */ + + case OP_CHAR: + case OP_CHARNC: + case OP_NOT: + case OP_PLUS: + case OP_MINPLUS: + case OP_POSPLUS: + case OP_NOTPLUS: + case OP_NOTMINPLUS: + case OP_NOTPOSPLUS: + branchlength++; + cc += 2; +#ifdef SUPPORT_UTF8 + if (utf8 && cc[-1] >= 0xc0) cc += _pcre_utf8_table4[cc[-1] & 0x3f]; +#endif + break; + + case OP_TYPEPLUS: + case OP_TYPEMINPLUS: + case OP_TYPEPOSPLUS: + branchlength++; + cc += (cc[1] == OP_PROP || cc[1] == OP_NOTPROP)? 4 : 2; + break; + + /* Handle exact repetitions. The count is already in characters, but we + need to skip over a multibyte character in UTF8 mode. */ + + case OP_EXACT: + case OP_NOTEXACT: + branchlength += GET2(cc,1); + cc += 4; +#ifdef SUPPORT_UTF8 + if (utf8 && cc[-1] >= 0xc0) cc += _pcre_utf8_table4[cc[-1] & 0x3f]; +#endif + break; + + case OP_TYPEEXACT: + branchlength += GET2(cc,1); + cc += (cc[3] == OP_PROP || cc[3] == OP_NOTPROP)? 6 : 4; + break; + + /* Handle single-char non-literal matchers */ + + case OP_PROP: + case OP_NOTPROP: + cc += 2; + /* Fall through */ + + case OP_NOT_DIGIT: + case OP_DIGIT: + case OP_NOT_WHITESPACE: + case OP_WHITESPACE: + case OP_NOT_WORDCHAR: + case OP_WORDCHAR: + case OP_ANY: + case OP_ALLANY: + case OP_EXTUNI: + case OP_HSPACE: + case OP_NOT_HSPACE: + case OP_VSPACE: + case OP_NOT_VSPACE: + branchlength++; + cc++; + break; + + /* "Any newline" might match two characters */ + + case OP_ANYNL: + branchlength += 2; + cc++; + break; + + /* The single-byte matcher means we can't proceed in UTF-8 mode */ + + case OP_ANYBYTE: +#ifdef SUPPORT_UTF8 + if (utf8) return -1; +#endif + branchlength++; + cc++; + break; + + /* For repeated character types, we have to test for \p and \P, which have + an extra two bytes of parameters. */ + + case OP_TYPESTAR: + case OP_TYPEMINSTAR: + case OP_TYPEQUERY: + case OP_TYPEMINQUERY: + case OP_TYPEPOSSTAR: + case OP_TYPEPOSQUERY: + if (cc[1] == OP_PROP || cc[1] == OP_NOTPROP) cc += 2; + cc += _pcre_OP_lengths[op]; + break; + + case OP_TYPEUPTO: + case OP_TYPEMINUPTO: + case OP_TYPEPOSUPTO: + if (cc[3] == OP_PROP || cc[3] == OP_NOTPROP) cc += 2; + cc += _pcre_OP_lengths[op]; + break; + + /* Check a class for variable quantification */ + +#ifdef SUPPORT_UTF8 + case OP_XCLASS: + cc += GET(cc, 1) - 33; + /* Fall through */ +#endif + + case OP_CLASS: + case OP_NCLASS: + cc += 33; + + switch (*cc) + { + case OP_CRPLUS: + case OP_CRMINPLUS: + branchlength++; + /* Fall through */ + + case OP_CRSTAR: + case OP_CRMINSTAR: + case OP_CRQUERY: + case OP_CRMINQUERY: + cc++; + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + branchlength += GET2(cc,1); + cc += 5; + break; + + default: + branchlength++; + break; + } + break; + + /* Backreferences and subroutine calls are treated in the same way: we find + the minimum length for the subpattern. A recursion, however, causes an + a flag to be set that causes the length of this branch to be ignored. The + logic is that a recursion can only make sense if there is another + alternation that stops the recursing. That will provide the minimum length + (when no recursion happens). A backreference within the group that it is + referencing behaves in the same way. + + If PCRE_JAVASCRIPT_COMPAT is set, a backreference to an unset bracket + matches an empty string (by default it causes a matching failure), so in + that case we must set the minimum length to zero. */ + + case OP_REF: + if ((options & PCRE_JAVASCRIPT_COMPAT) == 0) + { + ce = cs = (uschar *)_pcre_find_bracket(startcode, utf8, GET2(cc, 1)); + if (cs == NULL) return -2; + do ce += GET(ce, 1); while (*ce == OP_ALT); + if (cc > cs && cc < ce) + { + d = 0; + had_recurse = TRUE; + } + else d = find_minlength(cs, startcode, options); + } + else d = 0; + cc += 3; + + /* Handle repeated back references */ + + switch (*cc) + { + case OP_CRSTAR: + case OP_CRMINSTAR: + case OP_CRQUERY: + case OP_CRMINQUERY: + min = 0; + cc++; + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + min = GET2(cc, 1); + cc += 5; + break; + + default: + min = 1; + break; + } + + branchlength += min * d; + break; + + case OP_RECURSE: + cs = ce = (uschar *)startcode + GET(cc, 1); + if (cs == NULL) return -2; + do ce += GET(ce, 1); while (*ce == OP_ALT); + if (cc > cs && cc < ce) + had_recurse = TRUE; + else + branchlength += find_minlength(cs, startcode, options); + cc += 1 + LINK_SIZE; + break; + + /* Anything else does not or need not match a character. We can get the + item's length from the table, but for those that can match zero occurrences + of a character, we must take special action for UTF-8 characters. */ + + case OP_UPTO: + case OP_NOTUPTO: + case OP_MINUPTO: + case OP_NOTMINUPTO: + case OP_POSUPTO: + case OP_STAR: + case OP_MINSTAR: + case OP_NOTMINSTAR: + case OP_POSSTAR: + case OP_NOTPOSSTAR: + case OP_QUERY: + case OP_MINQUERY: + case OP_NOTMINQUERY: + case OP_POSQUERY: + case OP_NOTPOSQUERY: + cc += _pcre_OP_lengths[op]; +#ifdef SUPPORT_UTF8 + if (utf8 && cc[-1] >= 0xc0) cc += _pcre_utf8_table4[cc[-1] & 0x3f]; +#endif + break; + + /* For the record, these are the opcodes that are matched by "default": + OP_ACCEPT, OP_CLOSE, OP_COMMIT, OP_FAIL, OP_PRUNE, OP_SET_SOM, OP_SKIP, + OP_THEN. */ + + default: + cc += _pcre_OP_lengths[op]; + break; + } + } +/* Control never gets here */ +} + + + /************************************************* * Set a bit and maybe its alternate case * *************************************************/ @@ -498,13 +856,15 @@ Arguments: set NULL unless error Returns: pointer to a pcre_extra block, with study_data filled in and the - appropriate flag set; + appropriate flags set; NULL on error or if no optimization possible */ PCRE_EXP_DEFN pcre_extra * PCRE_CALL_CONVENTION pcre_study(const pcre *external_re, int options, const char **errorptr) { +int min; +BOOL bits_set = FALSE; uschar start_bits[32]; pcre_extra *extra; pcre_study_data *study; @@ -531,30 +891,39 @@ code = (uschar *)re + re->name_table_offset + (re->name_count * re->name_entry_size); /* For an anchored pattern, or an unanchored pattern that has a first char, or -a multiline pattern that matches only at "line starts", no further processing -at present. */ +a multiline pattern that matches only at "line starts", there is no point in +seeking a list of starting bytes. */ -if ((re->options & PCRE_ANCHORED) != 0 || - (re->flags & (PCRE_FIRSTSET|PCRE_STARTLINE)) != 0) - return NULL; +if ((re->options & PCRE_ANCHORED) == 0 && + (re->flags & (PCRE_FIRSTSET|PCRE_STARTLINE)) == 0) + { + /* Set the character tables in the block that is passed around */ -/* Set the character tables in the block that is passed around */ + tables = re->tables; + if (tables == NULL) + (void)pcre_fullinfo(external_re, NULL, PCRE_INFO_DEFAULT_TABLES, + (void *)(&tables)); -tables = re->tables; -if (tables == NULL) - (void)pcre_fullinfo(external_re, NULL, PCRE_INFO_DEFAULT_TABLES, - (void *)(&tables)); + compile_block.lcc = tables + lcc_offset; + compile_block.fcc = tables + fcc_offset; + compile_block.cbits = tables + cbits_offset; + compile_block.ctypes = tables + ctypes_offset; -compile_block.lcc = tables + lcc_offset; -compile_block.fcc = tables + fcc_offset; -compile_block.cbits = tables + cbits_offset; -compile_block.ctypes = tables + ctypes_offset; + /* See if we can find a fixed set of initial characters for the pattern. */ + + memset(start_bits, 0, 32 * sizeof(uschar)); + bits_set = set_start_bits(code, start_bits, + (re->options & PCRE_CASELESS) != 0, (re->options & PCRE_UTF8) != 0, + &compile_block) == SSB_DONE; + } -/* See if we can find a fixed set of initial characters for the pattern. */ +/* Find the minimum length of subject string. */ -memset(start_bits, 0, 32 * sizeof(uschar)); -if (set_start_bits(code, start_bits, (re->options & PCRE_CASELESS) != 0, - (re->options & PCRE_UTF8) != 0, &compile_block) != SSB_DONE) return NULL; +min = find_minlength(code, code, re->options); + +/* Return NULL if no optimization is possible. */ + +if (!bits_set && min < 0) return NULL; /* Get a pcre_extra block and a pcre_study_data block. The study data is put in the latter, which is pointed to by the former, which may also get additional @@ -577,8 +946,19 @@ extra->flags = PCRE_EXTRA_STUDY_DATA; extra->study_data = study; study->size = sizeof(pcre_study_data); -study->options = PCRE_STUDY_MAPPED; -memcpy(study->start_bits, start_bits, sizeof(start_bits)); +study->flags = 0; + +if (bits_set) + { + study->flags |= PCRE_STUDY_MAPPED; + memcpy(study->start_bits, start_bits, sizeof(start_bits)); + } + +if (min >= 0) + { + study->flags |= PCRE_STUDY_MINLEN; + study->minlength = min; + } return extra; } diff --git a/ext/pcre/pcrelib/pcre_try_flipped.c b/ext/pcre/pcrelib/pcre_try_flipped.c index 5e67943cc0..91ec76aae0 100644 --- a/ext/pcre/pcrelib/pcre_try_flipped.c +++ b/ext/pcre/pcrelib/pcre_try_flipped.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2008 University of Cambridge + Copyright (c) 1997-2009 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -126,7 +126,9 @@ if (study != NULL) { *internal_study = *study; /* To copy other fields */ internal_study->size = byteflip(study->size, sizeof(study->size)); - internal_study->options = byteflip(study->options, sizeof(study->options)); + internal_study->flags = byteflip(study->flags, sizeof(study->flags)); + internal_study->minlength = byteflip(study->minlength, + sizeof(study->minlength)); } return internal_re; diff --git a/ext/pcre/pcrelib/pcre_ucd.c b/ext/pcre/pcrelib/pcre_ucd.c index 33394a0395..0e4a42237d 100644 --- a/ext/pcre/pcrelib/pcre_ucd.c +++ b/ext/pcre/pcrelib/pcre_ucd.c @@ -1,9 +1,26 @@ #include "config.h" + #include "pcre_internal.h" /* Unicode character database. */ /* This file was autogenerated by the MultiStage2.py script. */ /* Total size: 52808 bytes, block size: 128. */ + +/* The tables herein are needed only when UCP support is built */ +/* into PCRE. This module should not be referenced otherwise, so */ +/* it should not matter whether it is compiled or not. However */ +/* a comment was received about space saving - maybe the guy linked */ +/* all the modules rather than using a library - so we include a */ +/* condition to cut out the tables when not needed. But don't leave */ +/* a totally empty module because some compilers barf at that. */ +/* Instead, just supply small dummy tables. */ + +#ifndef SUPPORT_UCP +const ucd_record _pcre_ucd_records[] = {{0,0,0 }}; +const uschar _pcre_ucd_stage1[] = {0}; +const pcre_uint16 _pcre_ucd_stage2[] = {0}; +#else + /* When recompiling tables with a new Unicode version, please check types in the structure definition from pcre_internal.h: typedef struct { @@ -2606,3 +2623,4 @@ const pcre_uint16 _pcre_ucd_stage2[] = { /* 40448 bytes, block = 128 */ #if UCD_BLOCK_SIZE != 128 #error Please correct UCD_BLOCK_SIZE in pcre_internal.h #endif +#endif /* SUPPORT_UCP */ diff --git a/ext/pcre/pcrelib/pcredemo.c b/ext/pcre/pcrelib/pcredemo.c index 3647568766..c6ba56ed5c 100644 --- a/ext/pcre/pcrelib/pcredemo.c +++ b/ext/pcre/pcrelib/pcredemo.c @@ -223,12 +223,12 @@ if (namecount <= 0) printf("No named substrings\n"); else * * * If the previous match WAS for an empty string, we can't do that, as it * * would lead to an infinite loop. Instead, a special call of pcre_exec() * -* is made with the PCRE_NOTEMPTY and PCRE_ANCHORED flags set. The first * -* of these tells PCRE that an empty string is not a valid match; other * -* possibilities must be tried. The second flag restricts PCRE to one * -* match attempt at the initial string position. If this match succeeds, * -* an alternative to the empty string match has been found, and we can * -* proceed round the loop. * +* is made with the PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED flags set. * +* The first of these tells PCRE that an empty string at the start of the * +* subject is not a valid match; other possibilities must be tried. The * +* second flag restricts PCRE to one match attempt at the initial string * +* position. If this match succeeds, an alternative to the empty string * +* match has been found, and we can proceed round the loop. * *************************************************************************/ if (!find_all) @@ -251,7 +251,7 @@ for (;;) if (ovector[0] == ovector[1]) { if (ovector[0] == subject_length) break; - options = PCRE_NOTEMPTY | PCRE_ANCHORED; + options = PCRE_NOTEMPTY_ATSTART | PCRE_ANCHORED; } /* Run the next matching operation */ diff --git a/ext/pcre/pcrelib/pcreposix.c b/ext/pcre/pcrelib/pcreposix.c index 645e060acc..1cc063d3e0 100644 --- a/ext/pcre/pcrelib/pcreposix.c +++ b/ext/pcre/pcrelib/pcreposix.c @@ -68,64 +68,80 @@ static const int eint[] = { REG_EESCAPE, /* \c at end of pattern */ REG_EESCAPE, /* unrecognized character follows \ */ REG_BADBR, /* numbers out of order in {} quantifier */ + /* 5 */ REG_BADBR, /* number too big in {} quantifier */ REG_EBRACK, /* missing terminating ] for character class */ REG_ECTYPE, /* invalid escape sequence in character class */ REG_ERANGE, /* range out of order in character class */ REG_BADRPT, /* nothing to repeat */ + /* 10 */ REG_BADRPT, /* operand of unlimited repeat could match the empty string */ REG_ASSERT, /* internal error: unexpected repeat */ REG_BADPAT, /* unrecognized character after (? */ REG_BADPAT, /* POSIX named classes are supported only within a class */ REG_EPAREN, /* missing ) */ + /* 15 */ REG_ESUBREG, /* reference to non-existent subpattern */ REG_INVARG, /* erroffset passed as NULL */ REG_INVARG, /* unknown option bit(s) set */ REG_EPAREN, /* missing ) after comment */ REG_ESIZE, /* parentheses nested too deeply */ + /* 20 */ REG_ESIZE, /* regular expression too large */ REG_ESPACE, /* failed to get memory */ - REG_EPAREN, /* unmatched brackets */ + REG_EPAREN, /* unmatched parentheses */ REG_ASSERT, /* internal error: code overflow */ REG_BADPAT, /* unrecognized character after (?< */ + /* 25 */ REG_BADPAT, /* lookbehind assertion is not fixed length */ REG_BADPAT, /* malformed number or name after (?( */ REG_BADPAT, /* conditional group contains more than two branches */ REG_BADPAT, /* assertion expected after (?( */ REG_BADPAT, /* (?R or (?[+-]digits must be followed by ) */ + /* 30 */ REG_ECTYPE, /* unknown POSIX class name */ REG_BADPAT, /* POSIX collating elements are not supported */ REG_INVARG, /* this version of PCRE is not compiled with PCRE_UTF8 support */ REG_BADPAT, /* spare error */ REG_BADPAT, /* character value in \x{...} sequence is too large */ + /* 35 */ REG_BADPAT, /* invalid condition (?(0) */ REG_BADPAT, /* \C not allowed in lookbehind assertion */ REG_EESCAPE, /* PCRE does not support \L, \l, \N, \U, or \u */ REG_BADPAT, /* number after (?C is > 255 */ REG_BADPAT, /* closing ) for (?C expected */ + /* 40 */ REG_BADPAT, /* recursive call could loop indefinitely */ REG_BADPAT, /* unrecognized character after (?P */ REG_BADPAT, /* syntax error in subpattern name (missing terminator) */ REG_BADPAT, /* two named subpatterns have the same name */ REG_BADPAT, /* invalid UTF-8 string */ + /* 45 */ REG_BADPAT, /* support for \P, \p, and \X has not been compiled */ REG_BADPAT, /* malformed \P or \p sequence */ REG_BADPAT, /* unknown property name after \P or \p */ REG_BADPAT, /* subpattern name is too long (maximum 32 characters) */ REG_BADPAT, /* too many named subpatterns (maximum 10,000) */ + /* 50 */ REG_BADPAT, /* repeated subpattern is too long */ REG_BADPAT, /* octal value is greater than \377 (not in UTF-8 mode) */ REG_BADPAT, /* internal error: overran compiling workspace */ REG_BADPAT, /* internal error: previously-checked referenced subpattern not found */ REG_BADPAT, /* DEFINE group contains more than one branch */ + /* 55 */ REG_BADPAT, /* repeating a DEFINE group is not allowed */ REG_INVARG, /* inconsistent NEWLINE options */ REG_BADPAT, /* \g is not followed followed by an (optionally braced) non-zero number */ - REG_BADPAT, /* (?+ or (?- must be followed by a non-zero number */ + REG_BADPAT, /* a numbered reference must not be zero */ + REG_BADPAT, /* (*VERB) with an argument is not supported */ + /* 60 */ + REG_BADPAT, /* (*VERB) not recognized */ REG_BADPAT, /* number is too big */ REG_BADPAT, /* subpattern name expected */ REG_BADPAT, /* digit expected after (?+ */ - REG_BADPAT /* ] is an invalid data character in JavaScript compatibility mode */ + REG_BADPAT, /* ] is an invalid data character in JavaScript compatibility mode */ + /* 65 */ + REG_BADPAT /* different names for subpatterns of the same number are not allowed */ }; /* Table of texts corresponding to POSIX error codes */ @@ -224,17 +240,25 @@ int erroffset; int errorcode; int options = 0; -if ((cflags & REG_ICASE) != 0) options |= PCRE_CASELESS; -if ((cflags & REG_NEWLINE) != 0) options |= PCRE_MULTILINE; -if ((cflags & REG_DOTALL) != 0) options |= PCRE_DOTALL; -if ((cflags & REG_NOSUB) != 0) options |= PCRE_NO_AUTO_CAPTURE; -if ((cflags & REG_UTF8) != 0) options |= PCRE_UTF8; +if ((cflags & REG_ICASE) != 0) options |= PCRE_CASELESS; +if ((cflags & REG_NEWLINE) != 0) options |= PCRE_MULTILINE; +if ((cflags & REG_DOTALL) != 0) options |= PCRE_DOTALL; +if ((cflags & REG_NOSUB) != 0) options |= PCRE_NO_AUTO_CAPTURE; +if ((cflags & REG_UTF8) != 0) options |= PCRE_UTF8; +if ((cflags & REG_UNGREEDY) != 0) options |= PCRE_UNGREEDY; preg->re_pcre = pcre_compile2(pattern, options, &errorcode, &errorptr, &erroffset, NULL); preg->re_erroffset = erroffset; -if (preg->re_pcre == NULL) return eint[errorcode]; +/* Safety: if the error code is too big for the translation vector (which +should not happen, but we all make mistakes), return REG_BADPAT. */ + +if (preg->re_pcre == NULL) + { + return (errorcode < sizeof(eint)/sizeof(const int))? + eint[errorcode] : REG_BADPAT; + } preg->re_nsub = pcre_info((const pcre *)preg->re_pcre, NULL, NULL); return 0; @@ -276,10 +300,11 @@ if ((eflags & REG_NOTEMPTY) != 0) options |= PCRE_NOTEMPTY; ((regex_t *)preg)->re_erroffset = (size_t)(-1); /* Only has meaning after compile */ -/* When no string data is being returned, ensure that nmatch is zero. -Otherwise, ensure the vector for holding the return data is large enough. */ +/* When no string data is being returned, or no vector has been passed in which +to put it, ensure that nmatch is zero. Otherwise, ensure the vector for holding +the return data is large enough. */ -if (nosub) nmatch = 0; +if (nosub || pmatch == NULL) nmatch = 0; else if (nmatch > 0) { diff --git a/ext/pcre/pcrelib/pcreposix.h b/ext/pcre/pcrelib/pcreposix.h index 7c5af72437..8ad58f5935 100644 --- a/ext/pcre/pcrelib/pcreposix.h +++ b/ext/pcre/pcrelib/pcreposix.h @@ -50,17 +50,18 @@ POSSIBILITY OF SUCH DAMAGE. extern "C" { #endif -/* Options, mostly defined by POSIX, but with a couple of extras. */ - -#define REG_ICASE 0x0001 -#define REG_NEWLINE 0x0002 -#define REG_NOTBOL 0x0004 -#define REG_NOTEOL 0x0008 -#define REG_DOTALL 0x0010 /* NOT defined by POSIX. */ -#define REG_NOSUB 0x0020 -#define REG_UTF8 0x0040 /* NOT defined by POSIX. */ +/* Options, mostly defined by POSIX, but with some extras. */ + +#define REG_ICASE 0x0001 /* Maps to PCRE_CASELESS */ +#define REG_NEWLINE 0x0002 /* Maps to PCRE_MULTILINE */ +#define REG_NOTBOL 0x0004 /* Maps to PCRE_NOTBOL */ +#define REG_NOTEOL 0x0008 /* Maps to PCRE_NOTEOL */ +#define REG_DOTALL 0x0010 /* NOT defined by POSIX; maps to PCRE_DOTALL */ +#define REG_NOSUB 0x0020 /* Maps to PCRE_NO_AUTO_CAPTURE */ +#define REG_UTF8 0x0040 /* NOT defined by POSIX; maps to PCRE_UTF8 */ #define REG_STARTEND 0x0080 /* BSD feature: pass subject string by so,eo */ -#define REG_NOTEMPTY 0x0100 /* NOT defined by POSIX. */ +#define REG_NOTEMPTY 0x0100 /* NOT defined by POSIX; maps to PCRE_NOTEMPTY */ +#define REG_UNGREEDY 0x0200 /* NOT defined by POSIX; maps to PCRE_UNGREEDY */ /* This is not used by PCRE, but by defining it we make it easier to slot PCRE into existing programs that make POSIX calls. */ diff --git a/ext/pcre/pcrelib/testdata/grepoutput b/ext/pcre/pcrelib/testdata/grepoutput index 882344e11f..da7a370270 100644 --- a/ext/pcre/pcrelib/testdata/grepoutput +++ b/ext/pcre/pcrelib/testdata/grepoutput @@ -423,3 +423,27 @@ This time it [1;31mjumps[00m and [1;31mjumps[00m and [1;31mjumps[00m. Here is the [1;31mpattern[00m again. That time it was on a [1;31mline by itself[00m. This line contains [1;31mpattern[00m not on a [1;31mline by itself[00m. +---------------------------- Test 55 ----------------------------- +./testdata/grepinput:456 +./testdata/grepinput8:0 +./testdata/grepinputv:1 +./testdata/grepinputx:0 +---------------------------- Test 56 ----------------------------- +./testdata/grepinput:456 +./testdata/grepinputv:1 +---------------------------- Test 57 ----------------------------- +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. +Check up on PATTERN near the end. +---------------------------- Test 58 ----------------------------- +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. +Check up on PATTERN near the end. +---------------------------- Test 59 ----------------------------- +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. +Check up on PATTERN near the end. +---------------------------- Test 60 ----------------------------- +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. +Check up on PATTERN near the end. diff --git a/ext/pcre/pcrelib/testdata/testinput1 b/ext/pcre/pcrelib/testdata/testinput1 index 8b0caa4cc9..6cacbb5124 100644 --- a/ext/pcre/pcrelib/testdata/testinput1 +++ b/ext/pcre/pcrelib/testdata/testinput1 @@ -1,3 +1,6 @@ +/-- This set of tests is for features that are compatible with all versions of + Perl 5, in non-UTF-8 mode. --/ + /the quick brown fox/ the quick brown fox The quick brown FOX @@ -4064,4 +4067,4 @@ /^%((?(?=[a])[^%])|b)*%$/ %ab% -/ End of testinput1 / +/-- End of testinput1 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput10 b/ext/pcre/pcrelib/testdata/testinput10 index 726a3890a2..420c274325 100644 --- a/ext/pcre/pcrelib/testdata/testinput10 +++ b/ext/pcre/pcrelib/testdata/testinput10 @@ -121,4 +121,4 @@ are all themselves checked in other tests. --/ /[^\xaa]/8BM -/ End of testinput10 / +/-- End of testinput10 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput2 b/ext/pcre/pcrelib/testdata/testinput2 index beca157c6a..3319bb7327 100644 --- a/ext/pcre/pcrelib/testdata/testinput2 +++ b/ext/pcre/pcrelib/testdata/testinput2 @@ -1,3 +1,14 @@ +/-- This set of tests is not Perl-compatible. It checks on special features + of PCRE's API, error diagnostics, and the compiled code of some patterns. + It also checks the non-Perl syntax the PCRE supports (Python, .NET, + Oniguruma). Finally, there are some tests where PCRE and Perl differ, + either because PCRE can't be compatible, or there is potential Perl + bug. --/ + +/-- Originally, the Perl 5.10 things were in here too, but now I have separated + many (most?) of them out into test 11. However, there may still be some + that were overlooked. --/ + /(a)b|/I /abc/I @@ -123,38 +134,38 @@ defabc \Zdefabc -/abc/IP +/abc/P abc *** Failers -/^abc|def/IP +/^abc|def/P abcdef abcdef\B -/.*((abc)$|(def))/IP +/.*((abc)$|(def))/P defabc \Zdefabc -/the quick brown fox/IP +/the quick brown fox/P the quick brown fox *** Failers The Quick Brown Fox -/the quick brown fox/IPi +/the quick brown fox/Pi the quick brown fox The Quick Brown Fox -/abc.def/IP +/abc.def/P *** Failers abc\ndef -/abc$/IP +/abc$/P abc abc\n -/(abc)\2/IP +/(abc)\2/P -/(abc\1)/IP +/(abc\1)/P abc /)/ @@ -593,7 +604,7 @@ *** Failers \Nabc -/a*(b+)(z)(z)/IP +/a*(b+)(z)(z)/P aaaabbbbzzzz aaaabbbbzzzz\O0 aaaabbbbzzzz\O1 @@ -1122,14 +1133,6 @@ /(a(?1)+b)/DZ -/^\W*(?:((.)\W*(?1)\W*\2|)|((.)\W*(?3)\W*\4|\W*.\W*))\W*$/Ii - 1221 - Satan, oscillate my metallic sonatas! - A man, a plan, a canal: Panama! - Able was I ere I saw Elba. - *** Failers - The quick brown fox - /^(\d+|\((?1)([+*-])(?1)\)|-(?1))$/I 12 (((2+2)*-3)-7) @@ -1419,13 +1422,13 @@ ** Failers line one\nthis is a line\nbreak in the second line -/ab.cd/IP +/ab.cd/P ab-cd ab=cd ** Failers ab\ncd -/ab.cd/IPs +/ab.cd/Ps ab-cd ab=cd ab\ncd @@ -1480,10 +1483,10 @@ (this) ((this)) -/a(b)c/IPN +/a(b)c/PN abc -/a(?P<name>b)c/IPN +/a(?P<name>b)c/PN abc /\x{100}/I @@ -1915,13 +1918,6 @@ a random value. /Ix /(?=(?'abc'\w+))\k<abc>:/I abcd: -/(?'abc'\w+):\k<abc>{2}/ - a:aaxyz - ab:ababxyz - ** Failers - a:axyz - ab:abxyz - /(?'abc'a|b)(?<abc>d|e)\k<abc>{2}/J adaa ** Failers @@ -1934,10 +1930,6 @@ a random value. /Ix ** Failers bddd -/^(?<ab>a)? (?(<ab>)b|c) (?('ab')d|e)/x - abd - ce - /(?(<bc))/ /(?(''))/ @@ -1955,16 +1947,6 @@ a random value. /Ix /(?<1> (?'B' abc (?(R) (?(R&1)1) (?(R&B)2) X | (?1) (?2) (?R) ))) /x abcabc1Xabc2XabcXabcabc -/^(?(DEFINE) (?<A> a) (?<B> b) ) (?&A) (?&B) /x - abcd - -/(?<NAME>(?&NAME_PAT))\s+(?<ADDR>(?&ADDRESS_PAT)) - (?(DEFINE) - (?<NAME_PAT>[a-z]+) - (?<ADDRESS_PAT>\d+) - )/x - metcalfe 33 - /^(?(DEFINE) abc | xyz ) /x /(?(DEFINE) abc) xyz/xI @@ -2053,22 +2035,6 @@ a random value. /Ix /(?1)X(?<abc>P)/I abcPXP123 -/(?(DEFINE)(?<byte>2[0-4]\d|25[0-5]|1\d\d|[1-9]?\d))\b(?&byte)(\.(?&byte)){3}/ - 1.2.3.4 - 131.111.10.206 - 10.0.0.0 - ** Failers - 10.6 - 455.3.4.5 - -/\b(?&byte)(\.(?&byte)){3}(?(DEFINE)(?<byte>2[0-4]\d|25[0-5]|1\d\d|[1-9]?\d))/ - 1.2.3.4 - 131.111.10.206 - 10.0.0.0 - ** Failers - 10.6 - 455.3.4.5 - /(?:a(?&abc)b)*(?<abc>x)/ 123axbaxbaxbx456 123axbaxbaxb456 @@ -2090,9 +2056,6 @@ a random value. /Ix defabcabcxyz DEFabcABCXYZ -/^(a(b))\1\g1\g{1}\g-1\g{-1}\g{-02}Z/ - ababababbbabZXXXX - /^(a)\g-2/ /^(a)\g/ @@ -2191,26 +2154,12 @@ a random value. /Ix /^(?(+1)X|Y)(.)/BZ Y! -/(foo)\Kbar/ - foobar - -/(foo)(\Kbar|baz)/ - foobar - foobaz - -/(foo\Kbar)baz/ - foobarbaz - /(?<A>tom|bon)-\k{A}/ tom-tom bon-bon ** Failers tom-bon -/(?<A>tom|bon)-\g{A}/ - tom-tom - bon-bon - /\g{A/ /(?|(abc)|(xyz))/BZ @@ -2225,50 +2174,6 @@ a random value. /Ix xabcpqrx xxyzx -/(?|(abc)|(xyz))\1/ - abcabc - xyzxyz - ** Failers - abcxyz - xyzabc - -/(?|(abc)|(xyz))(?1)/ - abcabc - xyzabc - ** Failers - xyzxyz - -/\H\h\V\v/ - X X\x0a - X\x09X\x0b - ** Failers - \xa0 X\x0a - -/\H*\h+\V?\v{3,4}/ - \x09\x20\xa0X\x0a\x0b\x0c\x0d\x0a - \x09\x20\xa0\x0a\x0b\x0c\x0d\x0a - \x09\x20\xa0\x0a\x0b\x0c - ** Failers - \x09\x20\xa0\x0a\x0b - -/\H{3,4}/ - XY ABCDE - XY PQR ST - -/.\h{3,4}./ - XY AB PQRS - -/\h*X\h?\H+Y\H?Z/ - >XNNNYZ - > X NYQZ - ** Failers - >XYZ - > X NY Z - -/\v*X\v?Y\v+Z\V*\x0a\V+\x0b\V{2,3}\x0c/ - >XY\x0aZ\x0aA\x0bNN\x0c - >\x0a\x0dX\x0aY\x0a\x0bZZZ\x0aAAA\x0bNNN\x0c - /[\h]/BZ >\x09< @@ -2341,49 +2246,6 @@ a random value. /Ix /A(*PRUNE)B(*SKIP)C(*THEN)D(*COMMIT)E(*F)F(*FAIL)G(?!)H(*ACCEPT)I/BZ -/^a+(*FAIL)/ - aaaaaa - -/a+b?c+(*FAIL)/ - aaabccc - -/a+b?(*PRUNE)c+(*FAIL)/ - aaabccc - -/a+b?(*COMMIT)c+(*FAIL)/ - aaabccc - -/a+b?(*SKIP)c+(*FAIL)/ - aaabcccaaabccc - -/^(?:aaa(*THEN)\w{6}|bbb(*THEN)\w{5}|ccc(*THEN)\w{4}|\w{3})/ - aaaxxxxxx - aaa++++++ - bbbxxxxx - bbb+++++ - cccxxxx - ccc++++ - dddddddd - -/^(aaa(*THEN)\w{6}|bbb(*THEN)\w{5}|ccc(*THEN)\w{4}|\w{3})/ - aaaxxxxxx - aaa++++++ - bbbxxxxx - bbb+++++ - cccxxxx - ccc++++ - dddddddd - -/a+b?(*THEN)c+(*FAIL)/ - aaabccc - -/(A (A|B(*ACCEPT)|C) D)(E)/x - ABX - AADE - ACDE - ** Failers - AD - /^a+(*FAIL)/C aaaaaa @@ -2589,66 +2451,8 @@ a random value. /Ix /[[:a\dz:]]/ -/^(?<name>a|b\g<name>c)/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - -/^(?<name>a|b\g'name'c)/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - -/^(a|b\g<1>c)/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - -/^(a|b\g'1'c)/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - -/^(a|b\g'-1'c)/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - -/(^(a|b\g<-1>c))/ - aaaa - bacxxx - bbaccxxx - bbbacccxx - /(^(a|b\g<-1'c))/ -/(^(a|b\g{-1}))/ - bacxxx - -/(?-i:\g<name>)(?i:(?<name>a))/ - XaaX - XAAX - -/(?i:\g<name>)(?-i:(?<name>a))/ - XaaX - ** Failers - XAAX - -/(?-i:\g<+1>)(?i:(a))/ - XaaX - XAAX - -/(?=(?<regex>(?#simplesyntax)\$(?<name>[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*)(?:\[(?<index>[a-zA-Z0-9_\x{7f}-\x{ff}]+|\$\g<name>)\]|->\g<name>(\(.*?\))?)?|(?#simple syntax withbraces)\$\{(?:\g<name>(?<indices>\[(?:\g<index>|'(?:\\.|[^'\\])*'|"(?:\g<regex>|\\.|[^"\\])*")\])?|\g<complex>|\$\{\g<complex>\})\}|(?#complexsyntax)\{(?<complex>\$(?<segment>\g<name>(\g<indices>*|\(.*?\))?)(?:->\g<segment>)*|\$\g<complex>|\$\{\g<complex>\})\}))\{/ - -/(?<n>a|b|c)\g<n>*/ - abc - accccbbb - /^(?+1)(?<a>x|y){0}z/ xzxx yzyy @@ -2755,22 +2559,614 @@ a random value. /Ix /^"((?(?=[a])[^"])|b)*"$/ "ab" -/^X(?5)(a)(?|(b)|(q))(c)(d)(Y)/ - XYabcdY - /^X(?5)(a)(?|(b)|(q))(c)(d)Y/ XYabcdY /^X(?&N)(a)(?|(b)|(q))(c)(d)(?<N>Y)/ XYabcdY +/Xa{2,4}b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/Xa{2,4}?b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/Xa{2,4}+b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\d{2,4}b/ + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\d{2,4}?b/ + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\d{2,4}+b/ + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\D{2,4}b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\D{2,4}?b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\D{2,4}+b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc]{2,4}b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc]{2,4}?b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc]{2,4}+b/ + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[^a]{2,4}b/ + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/X[^a]{2,4}?b/ + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/X[^a]{2,4}+b/ + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/(Y)X\1{2,4}b/ + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/(Y)X\1{2,4}?b/ + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/(Y)X\1{2,4}+b/ + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/\++\KZ|\d+X|9+Y/ + ++++123999\P + ++++123999Y\P + ++++Z1234\P + +/Z(*F)/ + Z\P + ZA\P + +/Z(?!)/ + Z\P + ZA\P + +/dog(sbody)?/ + dogs\P + dogs\P\P + +/dog(sbody)??/ + dogs\P + dogs\P\P + +/dog|dogsbody/ + dogs\P + dogs\P\P + +/dogsbody|dog/ + dogs\P + dogs\P\P + +/\bthe cat\b/ + the cat\P + the cat\P\P + +/abc/ + abc\P + abc\P\P + +/\w+A/P + CDAAAAB + +/\w+A/PU + CDAAAAB + +/abc\K123/ + xyzabc123pqr + xyzabc12\P + xyzabc12\P\P + +/(?<=abc)123/ + xyzabc123pqr + xyzabc12\P + xyzabc12\P\P + +/\babc\b/ + +++abc+++ + +++ab\P + +++ab\P\P + +/(?&word)(?&element)(?(DEFINE)(?<element><[^m][^>]>[^<])(?<word>\w*+))/BZ + +/(?&word)(?&element)(?(DEFINE)(?<element><[^\d][^>]>[^<])(?<word>\w*+))/BZ + +/(ab)(x(y)z(cd(*ACCEPT)))pq/BZ + +/abc\K/+ + abcdef + abcdef\N\N + xyzabcdef\N\N + ** Failers + abcdef\N + xyzabcdef\N + +/^(?:(?=abc)|abc\K)/+ + abcdef + abcdef\N\N + ** Failers + abcdef\N + +/a?b?/+ + xyz + xyzabc + xyzabc\N + xyzabc\N\N + xyz\N\N + ** Failers + xyz\N + +/^a?b?/+ + xyz + xyzabc + ** Failers + xyzabc\N + xyzabc\N\N + xyz\N\N + xyz\N + +/^(?<name>a|b\g<name>c)/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/^(?<name>a|b\g'name'c)/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/^(a|b\g<1>c)/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/^(a|b\g'1'c)/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/^(a|b\g'-1'c)/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/(^(a|b\g<-1>c))/ + aaaa + bacxxx + bbaccxxx + bbbacccxx + +/(?-i:\g<name>)(?i:(?<name>a))/ + XaaX + XAAX + +/(?i:\g<name>)(?-i:(?<name>a))/ + XaaX + ** Failers + XAAX + +/(?-i:\g<+1>)(?i:(a))/ + XaaX + XAAX + +/(?=(?<regex>(?#simplesyntax)\$(?<name>[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*)(?:\[(?<index>[a-zA-Z0-9_\x{7f}-\x{ff}]+|\$\g<name>)\]|->\g<name>(\(.*?\))?)?|(?#simple syntax withbraces)\$\{(?:\g<name>(?<indices>\[(?:\g<index>|'(?:\\.|[^'\\])*'|"(?:\g<regex>|\\.|[^"\\])*")\])?|\g<complex>|\$\{\g<complex>\})\}|(?#complexsyntax)\{(?<complex>\$(?<segment>\g<name>(\g<indices>*|\(.*?\))?)(?:->\g<segment>)*|\$\g<complex>|\$\{\g<complex>\})\}))\{/ + +/(?<n>a|b|c)\g<n>*/ + abc + accccbbb + /^X(?7)(a)(?|(b)|(q)(r)(s))(c)(d)(Y)/ XYabcdY -/^X(?7)(a)(?|(b|(r)(s))|(q))(c)(d)(Y)/ - XYabcdY +/(?<=b(?1)|zzz)(a)/ + xbaax + xzzzax + +/(a)(?<=b\1)/ + +/(a)(?<=b+(?1))/ + +/(a+)(?<=b(?1))/ + +/(a(?<=b(?1)))/ + +/(?<=b(?1))xyz/ + +/(?<=b(?1))xyz(b+)pqrstuvew/ + +/(a|bc)\1/SI + +/(a|bc)\1{2,3}/SI + +/(a|bc)(?1)/SI + +/(a|b\1)(a|b\1)/SI + +/(a|b\1){2}/SI + +/(a|bbbb\1)(a|bbbb\1)/SI + +/(a|bbbb\1){2}/SI + +/^From +([^ ]+) +[a-zA-Z][a-zA-Z][a-zA-Z] +[a-zA-Z][a-zA-Z][a-zA-Z] +[0-9]?[0-9] +[0-9][0-9]:[0-9][0-9]/SI + +/ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional leading comment +(?: (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # one word, optionally followed by.... +(?: +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] | # atom and space parts, or... +\( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) | # comments, or... + +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +# quoted strings +)* +< (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # leading < +(?: @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* + +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* , (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +)* # further okay, if led by comma +: # closing colon +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* )? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address spec +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* > # trailing > +# name and address +) (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional trailing comment +/xSI + +/<tr([\w\W\s\d][^<>]{0,})><TD([\w\W\s\d][^<>]{0,})>([\d]{0,}\.)(.*)((<BR>([\w\W\s\d][^<>]{0,})|[\s]{0,}))<\/a><\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><\/TR>/isIS + +"(?>.*/)foo"SI + +/(?(?=[^a-z]+[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) /xSI + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a|b|c))))))))))/iSI + +/(?:c|d)(?:)(?:aaaaaaaa(?:)(?:bbbbbbbb)(?:bbbbbbbb(?:))(?:bbbbbbbb(?:)(?:bbbbbbbb)))/SI + +/<a[\s]+href[\s]*=[\s]* # find <a href= + ([\"\'])? # find single or double quote + (?(1) (.*?)\1 | ([^\s]+)) # if quote found, match up to next matching + # quote, otherwise match up to next space +/isxSI + +/^(?!:) # colon disallowed at start + (?: # start of item + (?: [0-9a-f]{1,4} | # 1-4 hex digits or + (?(1)0 | () ) ) # if null previously matched, fail; else null + : # followed by colon + ){1,7} # end item; 1-7 of them required + [0-9a-f]{1,4} $ # final hex number at end of string + (?(1)|.) # check that there was an empty component + /xiIS + +/(?|(?<a>A)|(?<a>B))/I + AB\Ca + BA\Ca + +/(?|(?<a>A)|(?<b>B))/ + +/(?:a(?<quote> (?<apostrophe>')|(?<realquote>")) | + b(?<quote> (?<apostrophe>')|(?<realquote>")) ) + (?('quote')[a-z]+|[0-9]+)/JIx + a"aaaaa + b"aaaaa + ** Failers + b"11111 + a"11111 + +/^(?|(a)(b)(c)(?<D>d)|(?<D>e)) (?('D')X|Y)/JDZx + abcdX + eX + ** Failers + abcdY + ey + +/(?<A>a) (b)(c) (?<A>d (?(R&A)$ | (?4)) )/JDZx + abcdd + ** Failers + abcdde + +/abcd*/ + xxxxabcd\P + xxxxabcd\P\P + +/abcd*/i + xxxxabcd\P + xxxxabcd\P\P + XXXXABCD\P + XXXXABCD\P\P + +/abc\d*/ + xxxxabc1\P + xxxxabc1\P\P + +/(a)bc\1*/ + xxxxabca\P + xxxxabca\P\P + +/abc[de]*/ + xxxxabcde\P + xxxxabcde\P\P + +/-- This is not in the Perl 5.10 test because Perl seems currently to be broken + and not behaving as specified in that it *does* bumpalong after hitting + (*COMMIT). --/ + +/(?1)(A(*COMMIT)|B)D/ + ABD + XABD + BAD + ABXABD + ** Failers + ABX + BAXBAD -/^X(?7)(a)(?|(b|(?|(r)|(t))(s))|(q))(c)(d)(Y)/ - XYabcdY +/(\3)(\1)(a)/<JS> + cat + +/(\3)(\1)(a)/SI<JS> + cat + +/(\3)(\1)(a)/SI + cat -/ End of testinput2 / +/-- End of testinput2 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput3 b/ext/pcre/pcrelib/testdata/testinput3 index 1376c213b4..6fea2f5b3b 100644 --- a/ext/pcre/pcrelib/testdata/testinput3 +++ b/ext/pcre/pcrelib/testdata/testinput3 @@ -1,3 +1,7 @@ +/-- This set of tests checks local-specific features, using the fr_FR locale. + It is not Perl-compatible. There is different version called wintestinput3 + f or use on Windows, where the locale is called "french". --/ + /^[\w]+/ *** Failers École @@ -88,4 +92,4 @@ /[[:alpha:]][[:lower:]][[:upper:]]/DZLfr_FR -/ End of testinput3 / +/-- End of testinput3 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput4 b/ext/pcre/pcrelib/testdata/testinput4 index e5f61946e2..12f4c7e3b6 100644 --- a/ext/pcre/pcrelib/testdata/testinput4 +++ b/ext/pcre/pcrelib/testdata/testinput4 @@ -1,7 +1,6 @@ -/-- Do not use the \x{} construct except with patterns that have the --/ -/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ -/-- that option is set. However, the latest Perls recognize them always. --/ - +/-- This set of tests if for UTF-8 support, excluding Unicode properties. It is + compatible with all versions of Perl 5. --/ + /a.b/8 acb a\x7fb @@ -623,4 +622,22 @@ /(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8 -/ End of testinput4 / +/^[a\x{c0}]b/8 + \x{c0}b + +/^([a\x{c0}]*?)aa/8 + a\x{c0}aaaa/ + +/^([a\x{c0}]*?)aa/8 + a\x{c0}aaaa/ + a\x{c0}a\x{c0}aaa/ + +/^([a\x{c0}]*)aa/8 + a\x{c0}aaaa/ + a\x{c0}a\x{c0}aaa/ + +/^([a\x{c0}]*)a\x{c0}/8 + a\x{c0}aaaa/ + a\x{c0}a\x{c0}aaa/ + +/-- End of testinput4 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput5 b/ext/pcre/pcrelib/testdata/testinput5 index 38e22b8c01..82818d7dc1 100644 --- a/ext/pcre/pcrelib/testdata/testinput5 +++ b/ext/pcre/pcrelib/testdata/testinput5 @@ -1,3 +1,6 @@ +/-- This set of tests checks the API, internals, and non-Perl stuff for UTF-8 + support, excluding Unicode properties. --/ + /\x{100}/8DZ /\x{1000}/8DZ @@ -53,30 +56,6 @@ /.{3,5}?/DZ8 \x{212ab}\x{212ab}\x{212ab}\x{861} -/-- These tests are here rather than in testinput4 because Perl 5.6 has some -problems with UTF-8 support, in the area of \x{..} where the value is < 255. -It grumbles about invalid UTF-8 strings. --/ - -/^[a\x{c0}]b/8 - \x{c0}b - -/^([a\x{c0}]*?)aa/8 - a\x{c0}aaaa/ - -/^([a\x{c0}]*?)aa/8 - a\x{c0}aaaa/ - a\x{c0}a\x{c0}aaa/ - -/^([a\x{c0}]*)aa/8 - a\x{c0}aaaa/ - a\x{c0}a\x{c0}aaa/ - -/^([a\x{c0}]*)a\x{c0}/8 - a\x{c0}aaaa/ - a\x{c0}a\x{c0}aaa/ - -/-- --/ - /(?<=\C)X/8 Should produce an error diagnostic @@ -485,4 +464,282 @@ can't tell the difference.) --/ /(*CRLF)(*UTF8)(*BSR_UNICODE)a\Rb/I -/ End of testinput5 / +/Xa{2,4}b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/Xa{2,4}?b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/Xa{2,4}+b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\x{123}{2,4}b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X\x{123}{2,4}?b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X\x{123}{2,4}+b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X\x{123}{2,4}b/8 + Xx\P + X\x{123}x\P + X\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}\x{123}x\P + +/X\x{123}{2,4}?b/8 + Xx\P + X\x{123}x\P + X\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}\x{123}x\P + +/X\x{123}{2,4}+b/8 + Xx\P + X\x{123}x\P + X\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}x\P + X\x{123}\x{123}\x{123}\x{123}x\P + +/X\d{2,4}b/8 + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\d{2,4}?b/8 + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\d{2,4}+b/8 + X\P + X3\P + X33\P + X333\P + X3333\P + +/X\D{2,4}b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\D{2,4}?b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\D{2,4}+b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X\D{2,4}b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X\D{2,4}?b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X\D{2,4}+b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[abc]{2,4}b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc]{2,4}?b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc]{2,4}+b/8 + X\P + Xa\P + Xaa\P + Xaaa\P + Xaaaa\P + +/X[abc\x{123}]{2,4}b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[abc\x{123}]{2,4}?b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[abc\x{123}]{2,4}+b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[^a]{2,4}b/8 + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/X[^a]{2,4}?b/8 + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/X[^a]{2,4}+b/8 + X\P + Xz\P + Xzz\P + Xzzz\P + Xzzzz\P + +/X[^a]{2,4}b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[^a]{2,4}?b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/X[^a]{2,4}+b/8 + X\P + X\x{123}\P + X\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\P + X\x{123}\x{123}\x{123}\x{123}\P + +/(Y)X\1{2,4}b/8 + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/(Y)X\1{2,4}?b/8 + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/(Y)X\1{2,4}+b/8 + YX\P + YXY\P + YXYY\P + YXYYY\P + YXYYYY\P + +/(\x{123})X\1{2,4}b/8 + \x{123}X\P + \x{123}X\x{123}\P + \x{123}X\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\x{123}\P + +/(\x{123})X\1{2,4}?b/8 + \x{123}X\P + \x{123}X\x{123}\P + \x{123}X\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\x{123}\P + +/(\x{123})X\1{2,4}+b/8 + \x{123}X\P + \x{123}X\x{123}\P + \x{123}X\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\P + \x{123}X\x{123}\x{123}\x{123}\x{123}\P + +/\bthe cat\b/8 + the cat\P + the cat\P\P + +/abcd*/8 + xxxxabcd\P + xxxxabcd\P\P + +/abcd*/i8 + xxxxabcd\P + xxxxabcd\P\P + XXXXABCD\P + XXXXABCD\P\P + +/abc\d*/8 + xxxxabc1\P + xxxxabc1\P\P + +/(a)bc\1*/8 + xxxxabca\P + xxxxabca\P\P + +/abc[de]*/8 + xxxxabcde\P + xxxxabcde\P\P + +/-- End of testinput5 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput6 b/ext/pcre/pcrelib/testdata/testinput6 index 628646d7d3..f4249da814 100644 --- a/ext/pcre/pcrelib/testdata/testinput6 +++ b/ext/pcre/pcrelib/testdata/testinput6 @@ -1,3 +1,7 @@ +/-- This set of tests is for Unicode property support. It is compatible with + Perl 5.10, but not 5.8 because it tests some extra properties that are + not in the earlier release. --/ + /^\pC\pL\pM\pN\pP\pS\pZ</8 \x7f\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< \np\x{300}9!\$ < @@ -60,11 +64,6 @@ ** Failers \x{09f} -/^\p{Cs}/8 - \?\x{dfff} - ** Failers - \x{09f} - /^\p{Ll}/8 a ** Failers @@ -199,13 +198,6 @@ } \x{f3b} -/^\p{Sc}+/8 - $\x{a2}\x{a3}\x{a4}\x{a5}\x{a6} - \x{9f2} - ** Failers - X - \x{2c2} - /^\p{Sk}/8 \x{2c2} ** Failers @@ -237,17 +229,6 @@ X \x{2028} -/^\p{Zs}/8 - \ \ - \x{a0} - \x{1680} - \x{180e} - \x{2000} - \x{2001} - ** Failers - \x{2028} - \x{200d} - /\p{Nd}+(..)/8 \x{660}\x{661}\x{662}ABC @@ -291,23 +272,6 @@ ** Failers \x{660}\x{661}\x{662}ABC -/\p{Lu}/8i - A - a\x{10a0}B - ** Failers - a - \x{1d00} - -/\p{^Lu}/8i - 1234 - ** Failers - ABC - -/\P{Lu}/8i - 1234 - ** Failers - ABC - /(?<=A\p{Nd})XYZ/8 A2XYZ 123A5XYZPQR @@ -323,26 +287,6 @@ ** Failers WXYZ -/[\p{L}]/DZ - -/[\p{^L}]/DZ - -/[\P{L}]/DZ - -/[\P{^L}]/DZ - -/[abc\p{L}\x{0660}]/8DZ - -/[\p{Nd}]/8DZ - 1234 - -/[\p{Nd}+-]+/8DZ - 1234 - 12-34 - 12+\x{661}-34 - ** Failers - abcd - /[\P{Nd}]+/8 abcd ** Failers @@ -394,20 +338,6 @@ ** Failers ABC -/\p{Ll}/8i - a - Az - ** Failers - ABC - -/^\x{c0}$/8i - \x{c0} - \x{e0} - -/^\x{e0}$/8i - \x{c0} - \x{e0} - /A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8 A\x{391}\x{10427}\x{ff3a}\x{1fb0} ** Failers @@ -425,14 +355,6 @@ A\x{391}\x{10427}\x{ff5a}\x{1fb0} A\x{391}\x{10427}\x{ff3a}\x{1fb8} -/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8iDZ - -/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8DZ - -/AB\x{1fb0}/8DZ - -/AB\x{1fb0}/8DZi - /\x{391}+/8i \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} @@ -448,35 +370,6 @@ \x{3b1} \x{ff5a} -/[\x{c0}\x{391}]/8i - \x{c0} - \x{e0} - -/[\x{105}-\x{109}]/8iDZ - \x{104} - \x{105} - \x{109} - ** Failers - \x{100} - \x{10a} - -/[z-\x{100}]/8iDZ - Z - z - \x{39c} - \x{178} - | - \x{80} - \x{ff} - \x{100} - \x{101} - ** Failers - \x{102} - Y - y - -/[z-\x{100}]/8DZi - /^\X/8 A A\x{300}BC @@ -747,31 +640,9 @@ /([\pL]=(abc))*X/ L=abcX -/The next two should be Perl-compatible, but it fails to match \x{e0}. PCRE -will match it only with UCP support, because without that it has no notion -of case for anything other than the ASCII letters. / - -/((?i)[\x{c0}])/8 - \x{c0} - \x{e0} - -/(?i:[\x{c0}])/8 - \x{c0} - \x{e0} - /^\p{Balinese}\p{Cuneiform}\p{Nko}\p{Phags_Pa}\p{Phoenician}/8 \x{1b00}\x{12000}\x{7c0}\x{a840}\x{10900} -/The next two are special cases where the lengths of the different cases of the -same character differ. The first went wrong with heap frame storage; the 2nd -was broken in all cases./ - -/^\x{023a}+?(\x{0130}+)/8i - \x{023a}\x{2c65}\x{0130} - -/^\x{023a}+([^X])/8i - \x{023a}\x{2c65}X - /Check property support in non-UTF-8 mode/ /\p{L}{4}/ @@ -790,48 +661,6 @@ was broken in all cases./ /[\PPP\x8a]{1,}\x80/ A\x80 -/(?:[\PPa*]*){8,}/ - -/[\P{Any}]/BZ - -/[\P{Any}\E]/BZ - -/(\P{Yi}+\277)/ - -/(\P{Yi}+\277)?/ - -/(?<=\P{Yi}{3}A)X/ - -/\p{Yi}+(\P{Yi}+)(?1)/ - -/(\P{Yi}{2}\277)?/ - -/[\P{Yi}A]/ - -/[\P{Yi}\P{Yi}\P{Yi}A]/ - -/[^\P{Yi}A]/ - -/[^\P{Yi}\P{Yi}\P{Yi}A]/ - -/(\P{Yi}*\277)*/ - -/(\P{Yi}*?\277)*/ - -/(\p{Yi}*+\277)*/ - -/(\P{Yi}?\277)*/ - -/(\P{Yi}??\277)*/ - -/(\p{Yi}?+\277)*/ - -/(\P{Yi}{0,3}\277)*/ - -/(\P{Yi}{0,3}?\277)*/ - -/(\p{Yi}{0,3}+\277)*/ - /^[\p{Arabic}]/8 \x{60e} \x{656} @@ -895,24 +724,6 @@ was broken in all cases./ \x{1049f} \x{104aa} -/\p{Zl}{2,3}+/8BZ - \xe2\x80\xa8\xe2\x80\xa8 - \x{2028}\x{2028}\x{2028} - -/\p{Zl}/8BZ - -/\p{Lu}{3}+/8BZ - -/\pL{2}+/8BZ - -/\p{Cc}{2}+/8BZ - -/\x{c0}+\x{116}+/8i - \x{c0}\x{e0}\x{116}\x{117} - -/[\x{c0}\x{116}]+/8i - \x{c0}\x{e0}\x{116}\x{117} - /\p{Carian}\p{Cham}\p{Kayah_Li}\p{Lepcha}\p{Lycian}\p{Lydian}\p{Ol_Chiki}\p{Rejang}\p{Saurashtra}\p{Sundanese}\p{Vai}/8 \x{102A4}\x{AA52}\x{A91D}\x{1C46}\x{10283}\x{1092E}\x{1C6B}\x{A93B}\x{A8BF}\x{1BA0}\x{A50A}==== @@ -931,12 +742,6 @@ was broken in all cases./ aa aA -/(\x{de})\1/8i - \x{de}\x{de} - \x{de}\x{fe} - \x{fe}\x{fe} - \x{fe}\x{de} - /(\x{10a})\1/8i \x{10a}\x{10a} \x{10a}\x{10b} @@ -951,4 +756,4 @@ was broken in all cases./ /[\p{Lu}\x20]+/ \x41\x20\x50\xC2\x54\xC9\x20\x54\x4F\x44\x41\x59 -/ End of testinput6 / +/-- End of testinput6 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput7 b/ext/pcre/pcrelib/testdata/testinput7 index 047a975790..710d9ee9b6 100644 --- a/ext/pcre/pcrelib/testdata/testinput7 +++ b/ext/pcre/pcrelib/testdata/testinput7 @@ -1,3 +1,6 @@ +/-- This set of tests check the DFA matching functionality of pcre_dfa_exec(). + The -dfa flag must be used with pcretest when running it. --/ + /abc/ abc @@ -4421,4 +4424,122 @@ "ab" \C-"ab" -/ End of testinput7 / +/\d+X|9+Y/ + ++++123999\P + ++++123999Y\P + +/Z(*F)/ + Z\P + ZA\P + +/Z(?!)/ + Z\P + ZA\P + +/dog(sbody)?/ + dogs\P + dogs\P\P + +/dog(sbody)??/ + dogs\P + dogs\P\P + +/dog|dogsbody/ + dogs\P + dogs\P\P + +/dogsbody|dog/ + dogs\P + dogs\P\P + +/Z(*F)Q|ZXY/ + Z\P + ZA\P + X\P + +/\bthe cat\b/ + the cat\P + the cat\P\P + +/dog(sbody)?/ + dogs\D\P + body\D\R + +/dog(sbody)?/ + dogs\D\P\P + body\D\R + +/abc/ + abc\P + abc\P\P + +/abc\K123/ + xyzabc123pqr + +/(?<=abc)123/ + xyzabc123pqr + xyzabc12\P + xyzabc12\P\P + +/\babc\b/ + +++abc+++ + +++ab\P + +++ab\P\P + +/(?=C)/g+ + ABCDECBA + +/(abc|def|xyz)/I + terhjk;abcdaadsfe + the quick xyz brown fox + \Yterhjk;abcdaadsfe + \Ythe quick xyz brown fox + ** Failers + thejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd + \Ythejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd + +/(abc|def|xyz)/SI + terhjk;abcdaadsfe + the quick xyz brown fox + \Yterhjk;abcdaadsfe + \Ythe quick xyz brown fox + ** Failers + thejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd + \Ythejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd + +/abcd*/+ + xxxxabcd\P + xxxxabcd\P\P + dddxxx\R + xxxxabcd\P\P + xxx\R + +/abcd*/i + xxxxabcd\P + xxxxabcd\P\P + XXXXABCD\P + XXXXABCD\P\P + +/abc\d*/ + xxxxabc1\P + xxxxabc1\P\P + +/abc[de]*/ + xxxxabcde\P + xxxxabcde\P\P + +/(?:(?1)|B)(A(*F)|C)/ + ABCD + CCD + ** Failers + CAD + +/^(?:(?1)|B)(A(*F)|C)/ + CCD + BCD + ** Failers + ABCD + CAD + BAD + +/-- End of testinput7 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput8 b/ext/pcre/pcrelib/testdata/testinput8 index 11884adf65..1c6f684b43 100644 --- a/ext/pcre/pcrelib/testdata/testinput8 +++ b/ext/pcre/pcrelib/testdata/testinput8 @@ -1,6 +1,6 @@ -/-- Do not use the \x{} construct except with patterns that have the --/ -/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ -/-- that option is set. However, the latest Perls recognize them always. --/ +/-- This set of tests checks UTF-8 support with the DFA matching functionality + of pcre_dfa_exec(). The -dfa flag must be used with pcretest when running + it. --/ /\x{100}ab/8 \x{100}ab @@ -667,4 +667,22 @@ /X/8f<any> A\x{1ec5}ABCXYZ -/ End of testinput 8 / +/abcd*/8 + xxxxabcd\P + xxxxabcd\P\P + +/abcd*/i8 + xxxxabcd\P + xxxxabcd\P\P + XXXXABCD\P + XXXXABCD\P\P + +/abc\d*/8 + xxxxabc1\P + xxxxabc1\P\P + +/abc[de]*/8 + xxxxabcde\P + xxxxabcde\P\P + +/-- End of testinput8 --/ diff --git a/ext/pcre/pcrelib/testdata/testinput9 b/ext/pcre/pcrelib/testdata/testinput9 index ab044558d1..590420c8ea 100644 --- a/ext/pcre/pcrelib/testdata/testinput9 +++ b/ext/pcre/pcrelib/testdata/testinput9 @@ -1,3 +1,7 @@ +/-- This set of tests check Unicode property support with the DFA matching + functionality of pcre_dfa_exec(). The -dfa flag must be used with pcretest + when running it. --/ + /\pL\P{Nd}/8 AB *** Failers @@ -843,4 +847,4 @@ ** Failers \x{1d79}\x{a77d} -/ End / +/-- End of testinput9 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput1 b/ext/pcre/pcrelib/testdata/testoutput1 index 81b0cb868a..a2a6dd4e22 100644 --- a/ext/pcre/pcrelib/testdata/testoutput1 +++ b/ext/pcre/pcrelib/testdata/testoutput1 @@ -1,3 +1,6 @@ +/-- This set of tests is for features that are compatible with all versions of + Perl 5, in non-UTF-8 mode. --/ + /the quick brown fox/ the quick brown fox 0: the quick brown fox @@ -6646,4 +6649,4 @@ No match 0: %ab% 1: -/ End of testinput1 / +/-- End of testinput1 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput10 b/ext/pcre/pcrelib/testdata/testoutput10 index 4eaaa39183..a0db64035a 100644 --- a/ext/pcre/pcrelib/testdata/testoutput10 +++ b/ext/pcre/pcrelib/testdata/testoutput10 @@ -666,4 +666,4 @@ Memory allocation (code space): 40 39 End ------------------------------------------------------------------ -/ End of testinput10 / +/-- End of testinput10 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput2 b/ext/pcre/pcrelib/testdata/testoutput2 index 420e75dc40..30ea63b9cc 100644 --- a/ext/pcre/pcrelib/testdata/testoutput2 +++ b/ext/pcre/pcrelib/testdata/testoutput2 @@ -1,3 +1,14 @@ +/-- This set of tests is not Perl-compatible. It checks on special features + of PCRE's API, error diagnostics, and the compiled code of some patterns. + It also checks the non-Perl syntax the PCRE supports (Python, .NET, + Oniguruma). Finally, there are some tests where PCRE and Perl differ, + either because PCRE can't be compatible, or there is potential Perl + bug. --/ + +/-- Originally, the Perl 5.10 things were in here too, but now I have separated + many (most?) of them out into test 11. However, there may still be some + that were overlooked. --/ + /(a)b|/I Capturing subpattern count = 1 No options @@ -40,28 +51,24 @@ No match /a+bc/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'c' /a*bc/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'c' /a{3}bc/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'c' /(abc|a+z)/I Capturing subpattern count = 1 -Partial matching not supported No options First char = 'a' No need char @@ -113,14 +120,12 @@ Failed: unrecognized character after (? or (?- at offset 2 /.*b/I Capturing subpattern count = 0 -Partial matching not supported No options First char at start or follows newline Need char = 'b' /.*?b/I Capturing subpattern count = 0 -Partial matching not supported No options First char at start or follows newline Need char = 'b' @@ -140,6 +145,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 3 Starting byte set: c d e this sentence eventually mentions a cat 0: cat @@ -151,6 +157,7 @@ Capturing subpattern count = 0 Options: caseless No first char No need char +Subject length lower bound = 3 Starting byte set: C D E c d e this sentence eventually mentions a CAT cat 0: CAT @@ -162,6 +169,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b c d /(a|[^\dZ])/IS @@ -169,6 +177,7 @@ Capturing subpattern count = 1 No options No first char No need char +Subject length lower bound = 1 Starting byte set: \x00 \x01 \x02 \x03 \x04 \x05 \x06 \x07 \x08 \x09 \x0a \x0b \x0c \x0d \x0e \x0f \x10 \x11 \x12 \x13 \x14 \x15 \x16 \x17 \x18 \x19 \x1a \x1b \x1c \x1d \x1e \x1f \x20 ! " # $ % & ' ( ) * + , - . / : ; < = > @@ -189,6 +198,7 @@ Capturing subpattern count = 1 No options No first char No need char +Subject length lower bound = 1 Starting byte set: \x09 \x0a \x0c \x0d \x20 a b /(ab\2)/ @@ -324,7 +334,6 @@ No need char /.*((abc)$|(def))/I Capturing subpattern count = 3 -Partial matching not supported No options First char at start or follows newline No need char @@ -338,19 +347,19 @@ No need char 2: <unset> 3: def -/abc/IP +/abc/P abc 0: abc *** Failers No match: POSIX code 17: match failed -/^abc|def/IP +/^abc|def/P abcdef 0: abc abcdef\B 0: def -/.*((abc)$|(def))/IP +/.*((abc)$|(def))/P defabc 0: defabc 1: abc @@ -360,7 +369,7 @@ No match: POSIX code 17: match failed 1: def 3: def -/the quick brown fox/IP +/the quick brown fox/P the quick brown fox 0: the quick brown fox *** Failers @@ -368,28 +377,28 @@ No match: POSIX code 17: match failed The Quick Brown Fox No match: POSIX code 17: match failed -/the quick brown fox/IPi +/the quick brown fox/Pi the quick brown fox 0: the quick brown fox The Quick Brown Fox 0: The Quick Brown Fox -/abc.def/IP +/abc.def/P *** Failers No match: POSIX code 17: match failed abc\ndef No match: POSIX code 17: match failed -/abc$/IP +/abc$/P abc 0: abc abc\n 0: abc -/(abc)\2/IP +/(abc)\2/P Failed: POSIX code 15: bad back reference at offset 7 -/(abc\1)/IP +/(abc\1)/P abc No match: POSIX code 17: match failed @@ -401,7 +410,6 @@ Failed: missing terminating ] for character class at offset 4 /[^aeiou ]{3,}/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -410,7 +418,6 @@ No need char /<.*>/I Capturing subpattern count = 0 -Partial matching not supported No options First char = '<' Need char = '>' @@ -419,7 +426,6 @@ Need char = '>' /<.*?>/I Capturing subpattern count = 0 -Partial matching not supported No options First char = '<' Need char = '>' @@ -428,7 +434,6 @@ Need char = '>' /<.*>/IU Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = '<' Need char = '>' @@ -437,7 +442,6 @@ Need char = '>' /(?U)<.*>/I Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = '<' Need char = '>' @@ -446,7 +450,6 @@ Need char = '>' /<.*?>/IU Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = '<' Need char = '>' @@ -455,7 +458,6 @@ Need char = '>' /={3,}/IU Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = '=' Need char = '=' @@ -464,7 +466,6 @@ Need char = '=' /(?U)={3,}?/I Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = '=' Need char = '=' @@ -522,7 +523,6 @@ No need char /(?s).*/I Capturing subpattern count = 0 -Partial matching not supported Options: anchored dotall No first char No need char @@ -532,6 +532,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b c d /(?i)[abcd]/IS @@ -539,6 +540,7 @@ Capturing subpattern count = 0 Options: caseless No first char No need char +Subject length lower bound = 1 Starting byte set: A B C D a b c d /(?m)[xy]|(b|c)/IS @@ -546,6 +548,7 @@ Capturing subpattern count = 1 Options: multiline No first char No need char +Subject length lower bound = 1 Starting byte set: b c x y /(^a|^b)/Im @@ -584,7 +587,6 @@ Failed: syntax error in subpattern name (missing terminator) at offset 7 /((?s)blah)\s+\1/I Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported No options First char = 'b' Need char = 'h' @@ -592,7 +594,6 @@ Need char = 'h' /((?i)blah)\s+\1/I Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported No options First char = 'b' (caseless) Need char = 'h' (caseless) @@ -612,14 +613,15 @@ Capturing subpattern count = 1 No options First char = 'b' (caseless) No need char -Study returned NULL +Subject length lower bound = 1 +No set of starting bytes /(a*b|(?i:c*(?-i)d))/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 1 Starting byte set: C a b c d /a$/I @@ -668,7 +670,6 @@ Need char = 'c' /^((a+)(?U)([ab]+)(?-U)([bc]+)(\w*))/I Capturing subpattern count = 5 -Partial matching not supported Options: anchored No first char No need char @@ -685,6 +686,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b /(?<!foo)(alpha|omega)/IS @@ -692,6 +694,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'a' +Subject length lower bound = 5 Starting byte set: a o /(?!alphabet)[ab]/IS @@ -699,6 +702,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b /(?<=foo\n)^bar/Im @@ -784,7 +788,6 @@ No need char /(?>.*)(?<=(abcd)|(xyz))/I Capturing subpattern count = 2 -Partial matching not supported No options First char at start or follows newline No need char @@ -1003,7 +1006,6 @@ copy substring 5 failed -7 /(.{20})/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -1021,7 +1023,6 @@ No need char /(.{15})/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -1036,7 +1037,6 @@ No need char /(.{16})/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -1103,7 +1103,6 @@ No need char )((?:[a-zA-Z0-9]+ )((?:[a-zA-Z0-9]+ )((?:[a-zA-Z0-9]+ )((?:[a-zA-Z0-9]+ )?)?)?)?)?)?)?)?)?otherword/I Capturing subpattern count = 8 -Partial matching not supported Contains explicit CR or LF match No options First char = 'w' @@ -1118,7 +1117,6 @@ Need char = 'd' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options First char at start or follows newline Need char = 'X' @@ -1132,7 +1130,6 @@ Need char = 'X' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored dotall No first char Need char = 'X' @@ -1151,7 +1148,6 @@ Need char = 'X' End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported No options First char at start or follows newline No need char @@ -1170,7 +1166,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: anchored dotall No first char No need char @@ -1189,7 +1184,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: anchored dotall No first char No need char @@ -1211,7 +1205,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char No need char @@ -1296,7 +1289,6 @@ No need char /.*iss/Ig+ Capturing subpattern count = 0 -Partial matching not supported No options First char at start or follows newline Need char = 's' @@ -1399,42 +1391,36 @@ Need char = 'c' /a*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char /a+/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char /(baa|a+)/I Capturing subpattern count = 1 -Partial matching not supported No options No first char Need char = 'a' /a{0,3}/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char /baa{3,}/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'b' Need char = 'a' /"([^\\"]+|\\.)*"/I Capturing subpattern count = 1 -Partial matching not supported No options First char = '"' Need char = '"' @@ -1489,70 +1475,60 @@ Need char = 'b' /abc*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'b' /ab.c*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'b' /a.c*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char /.c*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char /ac*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char /(a.c*|b.c*)/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char /a.c*|aba/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char /.+a/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'a' /(?=abcda)a.*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'a' /(?=a)a.*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char @@ -1565,14 +1541,12 @@ No need char /a\d*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char /ab\d*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -1591,7 +1565,6 @@ Need char = 'd' /ab\d+/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -1666,7 +1639,7 @@ No need char \Nabc No match -/a*(b+)(z)(z)/IP +/a*(b+)(z)(z)/P aaaabbbbzzzz 0: aaaabbbbzz 1: bbbb @@ -1698,7 +1671,8 @@ Capturing subpattern count = 0 Options: anchored No first char Need char = 'd' -Study returned NULL +Subject length lower bound = 4 +No set of starting bytes /\( # ( at start (?: # Non-capturing bracket @@ -1709,7 +1683,6 @@ Study returned NULL \) # Closing ) /Ix Capturing subpattern count = 0 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1738,7 +1711,6 @@ No match /\( ( (?>[^()]+) | (?R) )* \) /Ixg Capturing subpattern count = 1 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1753,7 +1725,6 @@ Need char = ')' /\( (?: (?>[^()]+) | (?R) ) \) /Ix Capturing subpattern count = 0 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1772,7 +1743,6 @@ No match /\( (?: (?>[^()]+) | (?R) )? \) /Ix Capturing subpattern count = 0 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1783,7 +1753,6 @@ Need char = ')' /\( ( (?>[^()]+) | (?R) )* \) /Ix Capturing subpattern count = 1 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1793,7 +1762,6 @@ Need char = ')' /\( ( ( (?>[^()]+) | (?R) )* ) \) /Ix Capturing subpattern count = 2 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1804,7 +1772,6 @@ Need char = ')' /\( (123)? ( ( (?>[^()]+) | (?R) )* ) \) /Ix Capturing subpattern count = 3 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1821,7 +1788,6 @@ Need char = ')' /\( ( (123)? ( (?>[^()]+) | (?R) )* ) \) /Ix Capturing subpattern count = 3 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1838,7 +1804,6 @@ Need char = ')' /\( (((((((((( ( (?>[^()]+) | (?R) )* )))))))))) \) /Ix Capturing subpattern count = 11 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1858,7 +1823,6 @@ Need char = ')' /\( ( ( (?>[^()<>]+) | ((?>[^()]+)) | (?R) )* ) \) /Ix Capturing subpattern count = 3 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1866,11 +1830,9 @@ Need char = ')' 0: (abcd(xyz<p>qrs)123) 1: abcd(xyz<p>qrs)123 2: 123 - 3: <unset> /\( ( ( (?>[^()]+) | ((?R)) )* ) \) /Ix Capturing subpattern count = 3 -Partial matching not supported Options: extended First char = '(' Need char = ')' @@ -1942,6 +1904,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z @@ -2003,6 +1966,7 @@ Contains explicit CR or LF match No options No first char No need char +Subject length lower bound = 1 Starting byte set: \x09 \x0a \x0b \x0c \x0d \x20 /^[[:cntrl:]]/DZ @@ -2249,7 +2213,6 @@ Need char = 'd' /(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\d+(?:\s|$))(\w+)\s+(\270)/I Capturing subpattern count = 271 Max back reference = 270 -Partial matching not supported No options No first char No need char @@ -2930,7 +2893,6 @@ Need char = 'c' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -2959,7 +2921,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options First char = 'x' No need char @@ -2975,7 +2936,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options First char = 'x' No need char @@ -2999,7 +2959,6 @@ No need char /^(\w++|\s++)*$/I Capturing subpattern count = 1 -Partial matching not supported Options: anchored No first char No need char @@ -3013,7 +2972,6 @@ No match /(\d++)(\w)/I Capturing subpattern count = 2 -Partial matching not supported No options No first char No need char @@ -3028,7 +2986,6 @@ No match /a++b/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -3037,7 +2994,6 @@ Need char = 'b' /(a++b)/I Capturing subpattern count = 1 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -3047,7 +3003,6 @@ Need char = 'b' /(a++)b/I Capturing subpattern count = 1 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -3057,7 +3012,6 @@ Need char = 'b' /([^()]++|\([^()]*\))+/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -3067,7 +3021,6 @@ No need char /\(([^()]++|\([^()]+\))+\)/I Capturing subpattern count = 1 -Partial matching not supported No options First char = '(' Need char = ')' @@ -3130,7 +3083,6 @@ Failed: nothing to repeat at offset 7 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options First char = 'x' Need char = 'b' @@ -3147,7 +3099,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: ungreedy First char = 'x' Need char = 'b' @@ -3176,7 +3127,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 5 -Partial matching not supported Options: anchored No first char No need char @@ -3192,7 +3142,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char Need char = 'b' @@ -3210,7 +3159,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: anchored No first char Need char = 'b' @@ -3286,7 +3234,6 @@ No need char /< (?: (?(R) \d++ | [^<>]*+) | (?R)) * >/Ix Capturing subpattern count = 0 -Partial matching not supported Options: extended First char = '<' Need char = '>' @@ -3336,14 +3283,12 @@ Need char = 'X' /(.*)\d+\1/I Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported No options No first char No need char /(.*)\d+/I Capturing subpattern count = 1 -Partial matching not supported No options First char at start or follows newline No need char @@ -3351,14 +3296,12 @@ No need char /(.*)\d+\1/Is Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported Options: dotall No first char No need char /(.*)\d+/Is Capturing subpattern count = 1 -Partial matching not supported Options: anchored dotall No first char No need char @@ -3366,7 +3309,6 @@ No need char /(.*(xyz))\d+\2/I Capturing subpattern count = 2 Max back reference = 2 -Partial matching not supported No options First char at start or follows newline Need char = 'z' @@ -3374,7 +3316,6 @@ Need char = 'z' /((.*))\d+\1/I Capturing subpattern count = 2 Max back reference = 1 -Partial matching not supported No options No first char No need char @@ -3391,7 +3332,6 @@ Need char = 'b' /(?=a).*/I Capturing subpattern count = 0 -Partial matching not supported No options First char = 'a' No need char @@ -3497,6 +3437,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b /[^a]/I @@ -3516,6 +3457,7 @@ Capturing subpattern count = 0 No options No first char Need char = '6' +Subject length lower bound = 4 Starting byte set: 0 1 2 3 4 5 6 7 8 9 /a^b/I @@ -3549,6 +3491,7 @@ Capturing subpattern count = 0 Options: caseless No first char No need char +Subject length lower bound = 1 Starting byte set: A B a b /[ab](?i)cd/IS @@ -3556,6 +3499,7 @@ Capturing subpattern count = 0 No options No first char Need char = 'd' (caseless) +Subject length lower bound = 3 Starting byte set: a b /abc(?C)def/I @@ -3713,7 +3657,6 @@ No need char /(\d{3}(?C))*/I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -3842,7 +3785,6 @@ Callout 0: last capture = 1 /a(b+)(c*)(?C1)/I Capturing subpattern count = 2 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -3868,7 +3810,6 @@ No match /a(b+?)(c*?)(?C1)/I Capturing subpattern count = 2 -Partial matching not supported No options First char = 'a' Need char = 'b' @@ -3909,6 +3850,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b /(?R)/I @@ -4028,43 +3970,8 @@ No options First char = 'a' Need char = 'b' -/^\W*(?:((.)\W*(?1)\W*\2|)|((.)\W*(?3)\W*\4|\W*.\W*))\W*$/Ii -Capturing subpattern count = 4 -Max back reference = 4 -Partial matching not supported -Options: anchored caseless -No first char -No need char - 1221 - 0: 1221 - 1: 1221 - 2: 1 - Satan, oscillate my metallic sonatas! - 0: Satan, oscillate my metallic sonatas! - 1: <unset> - 2: <unset> - 3: Satan, oscillate my metallic sonatas - 4: S - A man, a plan, a canal: Panama! - 0: A man, a plan, a canal: Panama! - 1: <unset> - 2: <unset> - 3: A man, a plan, a canal: Panama - 4: A - Able was I ere I saw Elba. - 0: Able was I ere I saw Elba. - 1: <unset> - 2: <unset> - 3: Able was I ere I saw Elba - 4: A - *** Failers -No match - The quick brown fox -No match - /^(\d+|\((?1)([+*-])(?1)\)|-(?1))$/I Capturing subpattern count = 2 -Partial matching not supported Options: anchored No first char No need char @@ -4105,7 +4012,6 @@ No match /((< (?: (?(R) \d++ | [^<>]*+) | (?2)) * >))/Ix Capturing subpattern count = 2 -Partial matching not supported Options: extended First char = '<' Need char = '>' @@ -4279,7 +4185,6 @@ Named capturing subpatterns: one 1 three 3 two 2 -Partial matching not supported Options: anchored caseless No first char No need char @@ -4325,7 +4230,6 @@ No need char /(.*)a/Is Capturing subpattern count = 1 -Partial matching not supported Options: anchored dotall No first char Need char = 'a' @@ -4333,7 +4237,6 @@ Need char = 'a' /(.*)a\1/Is Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported Options: dotall No first char Need char = 'a' @@ -4341,14 +4244,12 @@ Need char = 'a' /(.*)a(b)\2/Is Capturing subpattern count = 2 Max back reference = 2 -Partial matching not supported Options: anchored dotall No first char Need char = 'b' /((.*)a|(.*)b)z/Is Capturing subpattern count = 3 -Partial matching not supported Options: anchored dotall No first char Need char = 'z' @@ -4356,7 +4257,6 @@ Need char = 'z' /((.*)a|(.*)b)z\1/Is Capturing subpattern count = 3 Max back reference = 1 -Partial matching not supported Options: dotall No first char Need char = 'z' @@ -4364,7 +4264,6 @@ Need char = 'z' /((.*)a|(.*)b)z\2/Is Capturing subpattern count = 3 Max back reference = 2 -Partial matching not supported Options: dotall No first char Need char = 'z' @@ -4372,7 +4271,6 @@ Need char = 'z' /((.*)a|(.*)b)z\3/Is Capturing subpattern count = 3 Max back reference = 3 -Partial matching not supported Options: dotall No first char Need char = 'z' @@ -4380,14 +4278,12 @@ Need char = 'z' /((.*)a|^(.*)b)z\3/Is Capturing subpattern count = 3 Max back reference = 3 -Partial matching not supported Options: anchored dotall No first char Need char = 'z' /(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)a/Is Capturing subpattern count = 31 -Partial matching not supported Options: anchored dotall No first char No need char @@ -4395,7 +4291,6 @@ No need char /(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)a\31/Is Capturing subpattern count = 31 Max back reference = 31 -Partial matching not supported Options: dotall No first char No need char @@ -4403,7 +4298,6 @@ No need char /(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)|(.*)a\32/Is Capturing subpattern count = 32 Max back reference = 32 -Partial matching not supported Options: dotall No first char No need char @@ -4470,7 +4364,6 @@ Need char = 'c' /(a+)*zz/I Capturing subpattern count = 1 -Partial matching not supported No options No first char Need char = 'z' @@ -4607,7 +4500,6 @@ Failed: two named subpatterns have the same name at offset 30 Capturing subpattern count = 3 Named capturing subpatterns: elem 2 -Partial matching not supported No options First char = '[' Need char = ']' @@ -4625,7 +4517,6 @@ No match Capturing subpattern count = 3 Named capturing subpatterns: elem 2 -Partial matching not supported No options First char = '[' Need char = ']' @@ -4732,7 +4623,6 @@ No need char /((w\/|-|with)*(free|immediate)*.*?shipping\s*[!.-]*)/Ii Capturing subpattern count = 3 -Partial matching not supported Options: caseless No first char Need char = 'g' (caseless) @@ -4742,11 +4632,11 @@ Need char = 'g' (caseless) /((w\/|-|with)*(free|immediate)*.*?shipping\s*[!.-]*)/IiS Capturing subpattern count = 3 -Partial matching not supported Options: caseless No first char Need char = 'g' (caseless) -Study returned NULL +Subject length lower bound = 8 +No set of starting bytes Baby Bjorn Active Carrier - With free SHIPPING!! 0: Baby Bjorn Active Carrier - With free SHIPPING!! 1: Baby Bjorn Active Carrier - With free SHIPPING!! @@ -4761,11 +4651,11 @@ Study returned NULL End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'b' -Study returned NULL +Subject length lower bound = 1 +No set of starting bytes /(a|b)*.?c/ISDZ ------------------------------------------------------------------ @@ -4785,7 +4675,8 @@ Capturing subpattern count = 1 No options No first char Need char = 'c' -Study returned NULL +Subject length lower bound = 1 +No set of starting bytes /abc(?C255)de(?C)f/DZ ------------------------------------------------------------------ @@ -4854,7 +4745,6 @@ No match End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: No first char Need char = 'b' @@ -4899,7 +4789,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: First char = 'a' Need char = 'b' @@ -5337,7 +5226,6 @@ No match End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: No first char Need char = '3' @@ -5369,7 +5257,6 @@ Need char = '3' /\b.*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5378,7 +5265,6 @@ No need char /\b.*/Is Capturing subpattern count = 0 -Partial matching not supported Options: dotall No first char No need char @@ -5387,7 +5273,6 @@ No need char /(?!.bcd).*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5400,15 +5285,15 @@ No options First char = 'a' Need char = 'e' ab\P -Partial match +Partial match: ab abc\P -Partial match +Partial match: abc abcd\P -Partial match +Partial match: abcd abcde\P 0: abcde the quick brown abc\P -Partial match +Partial match: abc ** Failers\P No match the quick brown abxyz fox\P @@ -5433,23 +5318,23 @@ Need char = '/' 1: 02 2: 05 1\P -Partial match +Partial match: 1 1/2\P -Partial match +Partial match: 1/2 1/2/0\P -Partial match +Partial match: 1/2/0 1/2/04\P 0: 1/2/04 1: 1 2: 2 0\P -Partial match +Partial match: 0 02/\P -Partial match +Partial match: 02/ 02/0\P -Partial match +Partial match: 02/0 02/1\P -Partial match +Partial match: 02/1 ** Failers\P No match \P @@ -5471,28 +5356,24 @@ No match /0{0,2}ABC/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'C' /\d{3,}ABC/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'C' /\d*ABC/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'C' /[abc]+DE/I Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = 'E' @@ -5505,13 +5386,13 @@ Need char = '3' 123\P 0: 123 a\P -Partial match +Partial match: a b\P -Partial match +Partial match: b c\P -Partial match +Partial match: c c12\P -Partial match +Partial match: c12 c123\P 0: c123 @@ -5521,17 +5402,17 @@ Options: anchored No first char Need char = 'X' 1\P -Partial match +Partial match: 1 123\P -Partial match +Partial match: 123 123X 0: 123X 1234\P -Partial match +Partial match: 1234 1234X 0: 1234X 12345\P -Partial match +Partial match: 12345 12345X 0: 12345X *** Failers @@ -5578,6 +5459,7 @@ Capturing subpattern count = 1 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b Compiled regex written to testsavedregex Study data written to testsavedregex @@ -5598,6 +5480,7 @@ Capturing subpattern count = 1 No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b Compiled regex written to testsavedregex Study data written to testsavedregex @@ -5616,7 +5499,6 @@ No match ~<(\w+)/?>(.)*</(\1)>~smgI Capturing subpattern count = 3 Max back reference = 1 -Partial matching not supported Options: multiline dotall First char = '<' Need char = '>' @@ -5669,7 +5551,7 @@ No match line one\nthis is a line\nbreak in the second line No match -/ab.cd/IP +/ab.cd/P ab-cd 0: ab-cd ab=cd @@ -5679,7 +5561,7 @@ No match: POSIX code 17: match failed ab\ncd No match: POSIX code 17: match failed -/ab.cd/IPs +/ab.cd/Ps ab-cd 0: ab-cd ab=cd @@ -5923,7 +5805,6 @@ Matched, but too many substrings /[^()]*(?:\((?R)\)[^()]*)*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5936,7 +5817,6 @@ No need char /[^()]*(?:\((?>(?R))\)[^()]*)*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5947,7 +5827,6 @@ No need char /[^()]*(?:\((?R)\))*[^()]*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5958,7 +5837,6 @@ No need char /(?:\((?R)\))*[^()]*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5971,7 +5849,6 @@ No need char /(?:\((?R)\))|[^()]*/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char @@ -5984,11 +5861,11 @@ No need char ((this)) 0: ((this)) -/a(b)c/IPN +/a(b)c/PN abc Matched with REG_NOSUB -/a(?P<name>b)c/IPN +/a(?P<name>b)c/PN abc Matched with REG_NOSUB @@ -6313,10 +6190,10 @@ Failed: octal value is greater than \377 (not in UTF-8 mode) at offset 3 /\s*,\s*/IS Capturing subpattern count = 0 -Partial matching not supported No options No first char Need char = ',' +Subject length lower bound = 1 Starting byte set: \x09 \x0a \x0c \x0d \x20 , \x0b,\x0b 0: , @@ -6443,7 +6320,6 @@ Unknown newline type at: <bad> /.*/I<lf> Capturing subpattern count = 0 -Partial matching not supported Options: Forced newline sequence: LF First char at start or follows newline @@ -6469,7 +6345,6 @@ No need char /\w+(.)(.)?def/Is Capturing subpattern count = 2 -Partial matching not supported Options: dotall No first char Need char = 'f' @@ -6486,7 +6361,6 @@ Need char = 'f' +((?:\s|//.*\\n|/[*](?:\\n|.)*?[*]/)*)+I Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char @@ -6621,10 +6495,10 @@ No need char /(a*b|(?i:c*(?-i)d))/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 1 Starting byte set: C a b c d /()[ab]xyz/IS @@ -6632,6 +6506,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b /(|)[ab]xyz/IS @@ -6639,6 +6514,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b /(|c)[ab]xyz/IS @@ -6646,6 +6522,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b c /(|c?)[ab]xyz/IS @@ -6653,6 +6530,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b c /(d?|c?)[ab]xyz/IS @@ -6660,6 +6538,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b c d /(d?|c)[ab]xyz/IS @@ -6667,6 +6546,7 @@ Capturing subpattern count = 1 No options No first char Need char = 'z' +Subject length lower bound = 4 Starting byte set: a b c d /^a*b\d/DZ @@ -6680,7 +6560,6 @@ Starting byte set: a b c d End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char Need char = 'b' @@ -6696,7 +6575,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char Need char = 'b' @@ -6712,7 +6590,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char Need char = 'b' @@ -6728,7 +6605,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored No first char Need char = 'A' @@ -6750,7 +6626,6 @@ No match End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: anchored caseless No first char Need char = 'A' (caseless) @@ -6761,34 +6636,34 @@ Need char = 'A' (caseless) /(a*|b*)[cd]/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b c d /(a+|b*)[cd]/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b c d /(a*|b+)[cd]/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 1 Starting byte set: a b c d /(a+|b+)[cd]/IS Capturing subpattern count = 1 -Partial matching not supported No options No first char No need char +Subject length lower bound = 2 Starting byte set: a b /(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((( @@ -7594,7 +7469,7 @@ No match ^ CBra 1 Cond - 2 Cond ref + 2 Cond nref y Ket [()] @@ -7660,7 +7535,6 @@ Named capturing subpatterns: one 1 three 3 two 2 -Partial matching not supported Options: anchored caseless No first char No need char @@ -7694,7 +7568,6 @@ No match /(?=(\w+))\1:/I Capturing subpattern count = 1 Max back reference = 1 -Partial matching not supported No options No first char Need char = ':' @@ -7707,7 +7580,6 @@ Capturing subpattern count = 1 Max back reference = 1 Named capturing subpatterns: abc 1 -Partial matching not supported No options No first char Need char = ':' @@ -7715,20 +7587,6 @@ Need char = ':' 0: abcd: 1: abcd -/(?'abc'\w+):\k<abc>{2}/ - a:aaxyz - 0: a:aa - 1: a - ab:ababxyz - 0: ab:abab - 1: ab - ** Failers -No match - a:axyz -No match - ab:abxyz -No match - /(?'abc'a|b)(?<abc>d|e)\k<abc>{2}/J adaa 0: adaa @@ -7755,13 +7613,6 @@ No match bddd No match -/^(?<ab>a)? (?(<ab>)b|c) (?('ab')d|e)/x - abd - 0: abd - 1: a - ce - 0: ce - /(?(<bc))/ Failed: malformed number or name after (?( at offset 6 @@ -7792,24 +7643,6 @@ Failed: reference to non-existent subpattern at offset 29 1: abcabc1Xabc2XabcX 2: abcabc1Xabc2XabcX -/^(?(DEFINE) (?<A> a) (?<B> b) ) (?&A) (?&B) /x - abcd - 0: ab - 1: <unset> - 2: <unset> - -/(?<NAME>(?&NAME_PAT))\s+(?<ADDR>(?&ADDRESS_PAT)) - (?(DEFINE) - (?<NAME_PAT>[a-z]+) - (?<ADDRESS_PAT>\d+) - )/x - metcalfe 33 - 0: metcalfe 33 - 1: metcalfe - 2: 33 - 3: <unset> - 4: <unset> - /^(?(DEFINE) abc | xyz ) /x Failed: DEFINE group contains more than one branch at offset 22 @@ -7987,46 +7820,6 @@ Need char = 'P' 0: PXP 1: P -/(?(DEFINE)(?<byte>2[0-4]\d|25[0-5]|1\d\d|[1-9]?\d))\b(?&byte)(\.(?&byte)){3}/ - 1.2.3.4 - 0: 1.2.3.4 - 1: <unset> - 2: .4 - 131.111.10.206 - 0: 131.111.10.206 - 1: <unset> - 2: .206 - 10.0.0.0 - 0: 10.0.0.0 - 1: <unset> - 2: .0 - ** Failers -No match - 10.6 -No match - 455.3.4.5 -No match - -/\b(?&byte)(\.(?&byte)){3}(?(DEFINE)(?<byte>2[0-4]\d|25[0-5]|1\d\d|[1-9]?\d))/ - 1.2.3.4 - 0: 1.2.3.4 - 1: .4 - 2: <unset> - 131.111.10.206 - 0: 131.111.10.206 - 1: .206 - 2: <unset> - 10.0.0.0 - 0: 10.0.0.0 - 1: .0 - 2: <unset> - ** Failers -No match - 10.6 -No match - 455.3.4.5 -No match - /(?:a(?&abc)b)*(?<abc>x)/ 123axbaxbaxbx456 0: axbaxbaxbx @@ -8064,12 +7857,6 @@ No match DEFabcABCXYZ No match -/^(a(b))\1\g1\g{1}\g-1\g{-1}\g{-02}Z/ - ababababbbabZXXXX - 0: ababababbbabZ - 1: ab - 2: b - /^(a)\g-2/ Failed: reference to non-existent subpattern at offset 7 @@ -8325,26 +8112,6 @@ Failed: reference to non-existent subpattern at offset 7 0: Y! 1: ! -/(foo)\Kbar/ - foobar - 0: bar - 1: foo - -/(foo)(\Kbar|baz)/ - foobar - 0: bar - 1: foo - 2: bar - foobaz - 0: foobaz - 1: foo - 2: baz - -/(foo\Kbar)baz/ - foobarbaz - 0: barbaz - 1: foobar - /(?<A>tom|bon)-\k{A}/ tom-tom 0: tom-tom @@ -8357,14 +8124,6 @@ No match tom-bon No match -/(?<A>tom|bon)-\g{A}/ - tom-tom - 0: tom-tom - 1: tom - bon-bon - 0: bon-bon - 1: bon - /\g{A/ Failed: syntax error in subpattern name (missing terminator) at offset 4 @@ -8459,82 +8218,6 @@ Failed: syntax error in subpattern name (missing terminator) at offset 4 3: <unset> 4: x -/(?|(abc)|(xyz))\1/ - abcabc - 0: abcabc - 1: abc - xyzxyz - 0: xyzxyz - 1: xyz - ** Failers -No match - abcxyz -No match - xyzabc -No match - -/(?|(abc)|(xyz))(?1)/ - abcabc - 0: abcabc - 1: abc - xyzabc - 0: xyzabc - 1: xyz - ** Failers -No match - xyzxyz -No match - -/\H\h\V\v/ - X X\x0a - 0: X X\x0a - X\x09X\x0b - 0: X\x09X\x0b - ** Failers -No match - \xa0 X\x0a -No match - -/\H*\h+\V?\v{3,4}/ - \x09\x20\xa0X\x0a\x0b\x0c\x0d\x0a - 0: \x09 \xa0X\x0a\x0b\x0c\x0d - \x09\x20\xa0\x0a\x0b\x0c\x0d\x0a - 0: \x09 \xa0\x0a\x0b\x0c\x0d - \x09\x20\xa0\x0a\x0b\x0c - 0: \x09 \xa0\x0a\x0b\x0c - ** Failers -No match - \x09\x20\xa0\x0a\x0b -No match - -/\H{3,4}/ - XY ABCDE - 0: ABCD - XY PQR ST - 0: PQR - -/.\h{3,4}./ - XY AB PQRS - 0: B P - -/\h*X\h?\H+Y\H?Z/ - >XNNNYZ - 0: XNNNYZ - > X NYQZ - 0: X NYQZ - ** Failers -No match - >XYZ -No match - > X NY Z -No match - -/\v*X\v?Y\v+Z\V*\x0a\V+\x0b\V{2,3}\x0c/ - >XY\x0aZ\x0aA\x0bNN\x0c - 0: XY\x0aZ\x0aA\x0bNN\x0c - >\x0a\x0dX\x0aY\x0a\x0bZZZ\x0aAAA\x0bNNN\x0c - 0: \x0a\x0dX\x0aY\x0a\x0bZZZ\x0aAAA\x0bNNN\x0c - /[\h]/BZ ------------------------------------------------------------------ Bra @@ -8831,87 +8514,6 @@ Failed: missing terminating ] for character class at offset 10 End ------------------------------------------------------------------ -/^a+(*FAIL)/ - aaaaaa -No match - -/a+b?c+(*FAIL)/ - aaabccc -No match - -/a+b?(*PRUNE)c+(*FAIL)/ - aaabccc -No match - -/a+b?(*COMMIT)c+(*FAIL)/ - aaabccc -No match - -/a+b?(*SKIP)c+(*FAIL)/ - aaabcccaaabccc -No match - -/^(?:aaa(*THEN)\w{6}|bbb(*THEN)\w{5}|ccc(*THEN)\w{4}|\w{3})/ - aaaxxxxxx - 0: aaaxxxxxx - aaa++++++ - 0: aaa - bbbxxxxx - 0: bbbxxxxx - bbb+++++ - 0: bbb - cccxxxx - 0: cccxxxx - ccc++++ - 0: ccc - dddddddd - 0: ddd - -/^(aaa(*THEN)\w{6}|bbb(*THEN)\w{5}|ccc(*THEN)\w{4}|\w{3})/ - aaaxxxxxx - 0: aaaxxxxxx - 1: aaaxxxxxx - aaa++++++ - 0: aaa - 1: aaa - bbbxxxxx - 0: bbbxxxxx - 1: bbbxxxxx - bbb+++++ - 0: bbb - 1: bbb - cccxxxx - 0: cccxxxx - 1: cccxxxx - ccc++++ - 0: ccc - 1: ccc - dddddddd - 0: ddd - 1: ddd - -/a+b?(*THEN)c+(*FAIL)/ - aaabccc -No match - -/(A (A|B(*ACCEPT)|C) D)(E)/x - ABX - 0: AB - AADE - 0: AADE - 1: AAD - 2: A - 3: E - ACDE - 0: ACDE - 1: ACD - 2: C - 3: E - ** Failers -No match - AD -No match - /^a+(*FAIL)/C aaaaaa --->aaaaaa @@ -9216,7 +8818,6 @@ No match /a\R{2,4}b/I<bsr_anycrlf> Capturing subpattern count = 0 -Partial matching not supported Options: bsr_anycrlf First char = 'a' Need char = 'b' @@ -9235,7 +8836,6 @@ No match /a\R{2,4}b/I<bsr_unicode> Capturing subpattern count = 0 -Partial matching not supported Options: bsr_unicode First char = 'a' Need char = 'b' @@ -9385,143 +8985,14 @@ Failed: unknown POSIX class name at offset 6 /[[:a\dz:]]/ Failed: unknown POSIX class name at offset 3 -/^(?<name>a|b\g<name>c)/ - aaaa - 0: a - 1: a - bacxxx - 0: bac - 1: bac - bbaccxxx - 0: bbacc - 1: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - -/^(?<name>a|b\g'name'c)/ - aaaa - 0: a - 1: a - bacxxx - 0: bac - 1: bac - bbaccxxx - 0: bbacc - 1: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - -/^(a|b\g<1>c)/ - aaaa - 0: a - 1: a - bacxxx - 0: bac - 1: bac - bbaccxxx - 0: bbacc - 1: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - -/^(a|b\g'1'c)/ - aaaa - 0: a - 1: a - bacxxx - 0: bac - 1: bac - bbaccxxx - 0: bbacc - 1: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - -/^(a|b\g'-1'c)/ - aaaa - 0: a - 1: a - bacxxx - 0: bac - 1: bac - bbaccxxx - 0: bbacc - 1: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - -/(^(a|b\g<-1>c))/ - aaaa - 0: a - 1: a - 2: a - bacxxx - 0: bac - 1: bac - 2: bac - bbaccxxx - 0: bbacc - 1: bbacc - 2: bbacc - bbbacccxx - 0: bbbaccc - 1: bbbaccc - 2: bbbaccc - /(^(a|b\g<-1'c))/ Failed: \g is not followed by a braced, angle-bracketed, or quoted name/number or by a plain number at offset 15 -/(^(a|b\g{-1}))/ - bacxxx -No match - -/(?-i:\g<name>)(?i:(?<name>a))/ - XaaX - 0: aa - 1: a - XAAX - 0: AA - 1: A - -/(?i:\g<name>)(?-i:(?<name>a))/ - XaaX - 0: aa - 1: a - ** Failers -No match - XAAX -No match - -/(?-i:\g<+1>)(?i:(a))/ - XaaX - 0: aa - 1: a - XAAX - 0: AA - 1: A - -/(?=(?<regex>(?#simplesyntax)\$(?<name>[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*)(?:\[(?<index>[a-zA-Z0-9_\x{7f}-\x{ff}]+|\$\g<name>)\]|->\g<name>(\(.*?\))?)?|(?#simple syntax withbraces)\$\{(?:\g<name>(?<indices>\[(?:\g<index>|'(?:\\.|[^'\\])*'|"(?:\g<regex>|\\.|[^"\\])*")\])?|\g<complex>|\$\{\g<complex>\})\}|(?#complexsyntax)\{(?<complex>\$(?<segment>\g<name>(\g<indices>*|\(.*?\))?)(?:->\g<segment>)*|\$\g<complex>|\$\{\g<complex>\})\}))\{/ - -/(?<n>a|b|c)\g<n>*/ - abc - 0: abc - 1: a - accccbbb - 0: accccbbb - 1: a - /^(?+1)(?<a>x|y){0}z/ xzxx 0: xz - 1: <unset> yzyy 0: yz - 1: <unset> ** Failers No match xxz @@ -9636,11 +9107,11 @@ Capturing subpattern count = 1 No options No first char No need char +Subject length lower bound = 1 Starting byte set: x y z /(?(?=.*b)b|^)/CI Capturing subpattern count = 0 -Partial matching not supported Options: No first char No need char @@ -9680,14 +9151,12 @@ No need char /(?(?=b).*b|^d)/I Capturing subpattern count = 0 -Partial matching not supported No options No first char No need char /(?(?=.*b).*b|^d)/I Capturing subpattern count = 0 -Partial matching not supported No options First char at start or follows newline No need char @@ -9786,15 +9255,6 @@ No match 0: "ab" 1: -/^X(?5)(a)(?|(b)|(q))(c)(d)(Y)/ - XYabcdY - 0: XYabcdY - 1: a - 2: b - 3: c - 4: d - 5: Y - /^X(?5)(a)(?|(b)|(q))(c)(d)Y/ Failed: reference to non-existent subpattern at offset 5 @@ -9807,6 +9267,579 @@ Failed: reference to non-existent subpattern at offset 5 4: d 5: Y +/Xa{2,4}b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/Xa{2,4}?b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/Xa{2,4}+b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\d{2,4}b/ + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\d{2,4}?b/ + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\d{2,4}+b/ + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\D{2,4}b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\D{2,4}?b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\D{2,4}+b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc]{2,4}b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc]{2,4}?b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc]{2,4}+b/ + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[^a]{2,4}b/ + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/X[^a]{2,4}?b/ + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/X[^a]{2,4}+b/ + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/(Y)X\1{2,4}b/ + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/(Y)X\1{2,4}?b/ + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/(Y)X\1{2,4}+b/ + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/\++\KZ|\d+X|9+Y/ + ++++123999\P +Partial match: 123999 + ++++123999Y\P + 0: 999Y + ++++Z1234\P + 0: Z + +/Z(*F)/ + Z\P +No match + ZA\P +No match + +/Z(?!)/ + Z\P +No match + ZA\P +No match + +/dog(sbody)?/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/dog(sbody)??/ + dogs\P + 0: dog + dogs\P\P + 0: dog + +/dog|dogsbody/ + dogs\P + 0: dog + dogs\P\P + 0: dog + +/dogsbody|dog/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/\bthe cat\b/ + the cat\P + 0: the cat + the cat\P\P +Partial match: the cat + +/abc/ + abc\P + 0: abc + abc\P\P + 0: abc + +/\w+A/P + CDAAAAB + 0: CDAAAA + +/\w+A/PU + CDAAAAB + 0: CDA + +/abc\K123/ + xyzabc123pqr + 0: 123 + xyzabc12\P +Partial match: abc12 + xyzabc12\P\P +Partial match: abc12 + +/(?<=abc)123/ + xyzabc123pqr + 0: 123 + xyzabc12\P +Partial match: abc12 + xyzabc12\P\P +Partial match: abc12 + +/\babc\b/ + +++abc+++ + 0: abc + +++ab\P +Partial match: +ab + +++ab\P\P +Partial match: +ab + +/(?&word)(?&element)(?(DEFINE)(?<element><[^m][^>]>[^<])(?<word>\w*+))/BZ +------------------------------------------------------------------ + Bra + Once + Recurse + Ket + Once + Recurse + Ket + Cond + Cond def + CBra 1 + < + [^m] + [^>] + > + [^<] + Ket + CBra 2 + \w*+ + Ket + Ket + Ket + End +------------------------------------------------------------------ + +/(?&word)(?&element)(?(DEFINE)(?<element><[^\d][^>]>[^<])(?<word>\w*+))/BZ +------------------------------------------------------------------ + Bra + Once + Recurse + Ket + Once + Recurse + Ket + Cond + Cond def + CBra 1 + < + [\x00-/:-\xff] (neg) + [^>] + > + [^<] + Ket + CBra 2 + \w*+ + Ket + Ket + Ket + End +------------------------------------------------------------------ + +/(ab)(x(y)z(cd(*ACCEPT)))pq/BZ +------------------------------------------------------------------ + Bra + CBra 1 + ab + Ket + CBra 2 + x + CBra 3 + y + Ket + z + CBra 4 + cd + Close 4 + Close 2 + *ACCEPT + Ket + Ket + pq + Ket + End +------------------------------------------------------------------ + +/abc\K/+ + abcdef + 0: + 0+ def + abcdef\N\N + 0: + 0+ def + xyzabcdef\N\N + 0: + 0+ def + ** Failers +No match + abcdef\N +No match + xyzabcdef\N +No match + +/^(?:(?=abc)|abc\K)/+ + abcdef + 0: + 0+ abcdef + abcdef\N\N + 0: + 0+ def + ** Failers +No match + abcdef\N +No match + +/a?b?/+ + xyz + 0: + 0+ xyz + xyzabc + 0: + 0+ xyzabc + xyzabc\N + 0: ab + 0+ c + xyzabc\N\N + 0: + 0+ yzabc + xyz\N\N + 0: + 0+ yz + ** Failers + 0: + 0+ ** Failers + xyz\N +No match + +/^a?b?/+ + xyz + 0: + 0+ xyz + xyzabc + 0: + 0+ xyzabc + ** Failers + 0: + 0+ ** Failers + xyzabc\N +No match + xyzabc\N\N +No match + xyz\N\N +No match + xyz\N +No match + +/^(?<name>a|b\g<name>c)/ + aaaa + 0: a + 1: a + bacxxx + 0: bac + 1: bac + bbaccxxx + 0: bbacc + 1: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + +/^(?<name>a|b\g'name'c)/ + aaaa + 0: a + 1: a + bacxxx + 0: bac + 1: bac + bbaccxxx + 0: bbacc + 1: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + +/^(a|b\g<1>c)/ + aaaa + 0: a + 1: a + bacxxx + 0: bac + 1: bac + bbaccxxx + 0: bbacc + 1: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + +/^(a|b\g'1'c)/ + aaaa + 0: a + 1: a + bacxxx + 0: bac + 1: bac + bbaccxxx + 0: bbacc + 1: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + +/^(a|b\g'-1'c)/ + aaaa + 0: a + 1: a + bacxxx + 0: bac + 1: bac + bbaccxxx + 0: bbacc + 1: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + +/(^(a|b\g<-1>c))/ + aaaa + 0: a + 1: a + 2: a + bacxxx + 0: bac + 1: bac + 2: bac + bbaccxxx + 0: bbacc + 1: bbacc + 2: bbacc + bbbacccxx + 0: bbbaccc + 1: bbbaccc + 2: bbbaccc + +/(?-i:\g<name>)(?i:(?<name>a))/ + XaaX + 0: aa + 1: a + XAAX + 0: AA + 1: A + +/(?i:\g<name>)(?-i:(?<name>a))/ + XaaX + 0: aa + 1: a + ** Failers +No match + XAAX +No match + +/(?-i:\g<+1>)(?i:(a))/ + XaaX + 0: aa + 1: a + XAAX + 0: AA + 1: A + +/(?=(?<regex>(?#simplesyntax)\$(?<name>[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*)(?:\[(?<index>[a-zA-Z0-9_\x{7f}-\x{ff}]+|\$\g<name>)\]|->\g<name>(\(.*?\))?)?|(?#simple syntax withbraces)\$\{(?:\g<name>(?<indices>\[(?:\g<index>|'(?:\\.|[^'\\])*'|"(?:\g<regex>|\\.|[^"\\])*")\])?|\g<complex>|\$\{\g<complex>\})\}|(?#complexsyntax)\{(?<complex>\$(?<segment>\g<name>(\g<indices>*|\(.*?\))?)(?:->\g<segment>)*|\$\g<complex>|\$\{\g<complex>\})\}))\{/ + +/(?<n>a|b|c)\g<n>*/ + abc + 0: abc + 1: a + accccbbb + 0: accccbbb + 1: a + /^X(?7)(a)(?|(b)|(q)(r)(s))(c)(d)(Y)/ XYabcdY 0: XYabcdY @@ -9818,26 +9851,616 @@ Failed: reference to non-existent subpattern at offset 5 6: d 7: Y -/^X(?7)(a)(?|(b|(r)(s))|(q))(c)(d)(Y)/ - XYabcdY - 0: XYabcdY +/(?<=b(?1)|zzz)(a)/ + xbaax + 0: a + 1: a + xzzzax + 0: a 1: a - 2: b - 3: <unset> - 4: <unset> - 5: c - 6: d - 7: Y -/^X(?7)(a)(?|(b|(?|(r)|(t))(s))|(q))(c)(d)(Y)/ - XYabcdY - 0: XYabcdY +/(a)(?<=b\1)/ +Failed: lookbehind assertion is not fixed length at offset 10 + +/(a)(?<=b+(?1))/ +Failed: lookbehind assertion is not fixed length at offset 13 + +/(a+)(?<=b(?1))/ +Failed: lookbehind assertion is not fixed length at offset 14 + +/(a(?<=b(?1)))/ +Failed: lookbehind assertion is not fixed length at offset 13 + +/(?<=b(?1))xyz/ +Failed: reference to non-existent subpattern at offset 8 + +/(?<=b(?1))xyz(b+)pqrstuvew/ +Failed: lookbehind assertion is not fixed length at offset 26 + +/(a|bc)\1/SI +Capturing subpattern count = 1 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/(a|bc)\1{2,3}/SI +Capturing subpattern count = 1 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 3 +Starting byte set: a b + +/(a|bc)(?1)/SI +Capturing subpattern count = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/(a|b\1)(a|b\1)/SI +Capturing subpattern count = 2 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/(a|b\1){2}/SI +Capturing subpattern count = 1 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/(a|bbbb\1)(a|bbbb\1)/SI +Capturing subpattern count = 2 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/(a|bbbb\1){2}/SI +Capturing subpattern count = 1 +Max back reference = 1 +No options +No first char +No need char +Subject length lower bound = 2 +Starting byte set: a b + +/^From +([^ ]+) +[a-zA-Z][a-zA-Z][a-zA-Z] +[a-zA-Z][a-zA-Z][a-zA-Z] +[0-9]?[0-9] +[0-9][0-9]:[0-9][0-9]/SI +Capturing subpattern count = 1 +Options: anchored +No first char +Need char = ':' +Subject length lower bound = 22 +No set of starting bytes + +/ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional leading comment +(?: (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # one word, optionally followed by.... +(?: +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] | # atom and space parts, or... +\( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) | # comments, or... + +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +# quoted strings +)* +< (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # leading < +(?: @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* + +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* , (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +)* # further okay, if led by comma +: # closing colon +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* )? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address spec +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* > # trailing > +# name and address +) (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional trailing comment +/xSI +Capturing subpattern count = 0 +Contains explicit CR or LF match +Options: extended +No first char +No need char +Subject length lower bound = 3 +Starting byte set: \x09 \x20 ! " # $ % & ' ( * + - / 0 1 2 3 4 5 6 7 8 + 9 = ? A B C D E F G H I J K L M N O P Q R S T U V W X Y Z ^ _ ` a b c d e + f g h i j k l m n o p q r s t u v w x y z { | } ~ \x7f + +/<tr([\w\W\s\d][^<>]{0,})><TD([\w\W\s\d][^<>]{0,})>([\d]{0,}\.)(.*)((<BR>([\w\W\s\d][^<>]{0,})|[\s]{0,}))<\/a><\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><\/TR>/isIS +Capturing subpattern count = 11 +Options: caseless dotall +First char = '<' +Need char = '>' +Subject length lower bound = 47 +No set of starting bytes + +"(?>.*/)foo"SI +Capturing subpattern count = 0 +No options +First char at start or follows newline +Need char = 'o' +Subject length lower bound = 4 +No set of starting bytes + +/(?(?=[^a-z]+[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) /xSI +Capturing subpattern count = 0 +Options: extended +No first char +Need char = '-' +Subject length lower bound = 8 +No set of starting bytes + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a|b|c))))))))))/iSI +Capturing subpattern count = 1 +Options: caseless +No first char +No need char +Subject length lower bound = 1 +Starting byte set: A B C a b c + +/(?:c|d)(?:)(?:aaaaaaaa(?:)(?:bbbbbbbb)(?:bbbbbbbb(?:))(?:bbbbbbbb(?:)(?:bbbbbbbb)))/SI +Capturing subpattern count = 0 +No options +No first char +Need char = 'b' +Subject length lower bound = 41 +Starting byte set: c d + +/<a[\s]+href[\s]*=[\s]* # find <a href= + ([\"\'])? # find single or double quote + (?(1) (.*?)\1 | ([^\s]+)) # if quote found, match up to next matching + # quote, otherwise match up to next space +/isxSI +Capturing subpattern count = 3 +Max back reference = 1 +Options: caseless extended dotall +First char = '<' +Need char = '=' +Subject length lower bound = 9 +No set of starting bytes + +/^(?!:) # colon disallowed at start + (?: # start of item + (?: [0-9a-f]{1,4} | # 1-4 hex digits or + (?(1)0 | () ) ) # if null previously matched, fail; else null + : # followed by colon + ){1,7} # end item; 1-7 of them required + [0-9a-f]{1,4} $ # final hex number at end of string + (?(1)|.) # check that there was an empty component + /xiIS +Capturing subpattern count = 1 +Options: anchored caseless extended +No first char +Need char = ':' +Subject length lower bound = 2 +No set of starting bytes + +/(?|(?<a>A)|(?<a>B))/I +Capturing subpattern count = 1 +Named capturing subpatterns: + a 1 + a 1 +No options +No first char +No need char + AB\Ca + 0: A + 1: A + C A (1) a + BA\Ca + 0: B + 1: B + C B (1) a + +/(?|(?<a>A)|(?<b>B))/ +Failed: different names for subpatterns of the same number are not allowed at offset 15 + +/(?:a(?<quote> (?<apostrophe>')|(?<realquote>")) | + b(?<quote> (?<apostrophe>')|(?<realquote>")) ) + (?('quote')[a-z]+|[0-9]+)/JIx +Capturing subpattern count = 6 +Named capturing subpatterns: + apostrophe 2 + apostrophe 5 + quote 1 + quote 4 + realquote 3 + realquote 6 +Options: extended dupnames +No first char +No need char + a"aaaaa + 0: a"aaaaa + 1: " + 2: <unset> + 3: " + b"aaaaa + 0: b"aaaaa + 1: <unset> + 2: <unset> + 3: <unset> + 4: " + 5: <unset> + 6: " + ** Failers +No match + b"11111 +No match + a"11111 +No match + +/^(?|(a)(b)(c)(?<D>d)|(?<D>e)) (?('D')X|Y)/JDZx +------------------------------------------------------------------ + Bra + ^ + Bra + CBra 1 + a + Ket + CBra 2 + b + Ket + CBra 3 + c + Ket + CBra 4 + d + Ket + Alt + CBra 1 + e + Ket + Ket + Cond + 4 Cond nref + X + Alt + Y + Ket + Ket + End +------------------------------------------------------------------ +Capturing subpattern count = 4 +Named capturing subpatterns: + D 4 + D 1 +Options: anchored extended dupnames +No first char +No need char + abcdX + 0: abcdX 1: a 2: b - 3: <unset> - 4: <unset> - 5: c - 6: d - 7: Y + 3: c + 4: d + eX + 0: eX + 1: e + ** Failers +No match + abcdY +No match + ey +No match + +/(?<A>a) (b)(c) (?<A>d (?(R&A)$ | (?4)) )/JDZx +------------------------------------------------------------------ + Bra + CBra 1 + a + Ket + CBra 2 + b + Ket + CBra 3 + c + Ket + CBra 4 + d + Cond + Cond nrecurse 1 + $ + Alt + Once + Recurse + Ket + Ket + Ket + Ket + End +------------------------------------------------------------------ +Capturing subpattern count = 4 +Named capturing subpatterns: + A 1 + A 4 +Options: extended dupnames +First char = 'a' +Need char = 'd' + abcdd + 0: abcdd + 1: a + 2: b + 3: c + 4: dd + ** Failers +No match + abcdde +No match + +/abcd*/ + xxxxabcd\P + 0: abcd + xxxxabcd\P\P +Partial match: abcd + +/abcd*/i + xxxxabcd\P + 0: abcd + xxxxabcd\P\P +Partial match: abcd + XXXXABCD\P + 0: ABCD + XXXXABCD\P\P +Partial match: ABCD + +/abc\d*/ + xxxxabc1\P + 0: abc1 + xxxxabc1\P\P +Partial match: abc1 + +/(a)bc\1*/ + xxxxabca\P + 0: abca + 1: a + xxxxabca\P\P +Partial match: abca + +/abc[de]*/ + xxxxabcde\P + 0: abcde + xxxxabcde\P\P +Partial match: abcde + +/-- This is not in the Perl 5.10 test because Perl seems currently to be broken + and not behaving as specified in that it *does* bumpalong after hitting + (*COMMIT). --/ + +/(?1)(A(*COMMIT)|B)D/ + ABD + 0: ABD + 1: B + XABD + 0: ABD + 1: B + BAD + 0: BAD + 1: A + ABXABD + 0: ABD + 1: B + ** Failers +No match + ABX +No match + BAXBAD +No match + +/(\3)(\1)(a)/<JS> + cat + 0: a + 1: + 2: + 3: a + +/(\3)(\1)(a)/SI<JS> +Capturing subpattern count = 3 +Max back reference = 3 +Options: +No first char +Need char = 'a' +Subject length lower bound = 1 +No set of starting bytes + cat + 0: a + 1: + 2: + 3: a + +/(\3)(\1)(a)/SI +Capturing subpattern count = 3 +Max back reference = 3 +No options +No first char +Need char = 'a' +Subject length lower bound = 3 +No set of starting bytes + cat +No match -/ End of testinput2 / +/-- End of testinput2 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput3 b/ext/pcre/pcrelib/testdata/testoutput3 index 28b1c3aaaf..7b0a3e926e 100644 --- a/ext/pcre/pcrelib/testdata/testoutput3 +++ b/ext/pcre/pcrelib/testdata/testoutput3 @@ -1,3 +1,7 @@ +/-- This set of tests checks local-specific features, using the fr_FR locale. + It is not Perl-compatible. There is different version called wintestinput3 + f or use on Windows, where the locale is called "french". --/ + /^[\w]+/ *** Failers No match @@ -83,6 +87,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z @@ -91,6 +96,7 @@ Capturing subpattern count = 0 No options No first char No need char +Subject length lower bound = 1 Starting byte set: 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z ª µ º À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß à á â @@ -160,4 +166,4 @@ No options No first char No need char -/ End of testinput3 / +/-- End of testinput3 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput4 b/ext/pcre/pcrelib/testdata/testoutput4 index e67a95895f..128afe4386 100644 --- a/ext/pcre/pcrelib/testdata/testoutput4 +++ b/ext/pcre/pcrelib/testdata/testoutput4 @@ -1,9 +1,6 @@ -/-- Do not use the \x{} construct except with patterns that have the --/ -/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ -No match -/-- that option is set. However, the latest Perls recognize them always. --/ -No match - +/-- This set of tests if for UTF-8 support, excluding Unicode properties. It is + compatible with all versions of Perl 5. --/ + /a.b/8 acb 0: acb @@ -1089,4 +1086,37 @@ No match /(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8 -/ End of testinput4 / +/^[a\x{c0}]b/8 + \x{c0}b + 0: \x{c0}b + +/^([a\x{c0}]*?)aa/8 + a\x{c0}aaaa/ + 0: a\x{c0}aa + 1: a\x{c0} + +/^([a\x{c0}]*?)aa/8 + a\x{c0}aaaa/ + 0: a\x{c0}aa + 1: a\x{c0} + a\x{c0}a\x{c0}aaa/ + 0: a\x{c0}a\x{c0}aa + 1: a\x{c0}a\x{c0} + +/^([a\x{c0}]*)aa/8 + a\x{c0}aaaa/ + 0: a\x{c0}aaaa + 1: a\x{c0}aa + a\x{c0}a\x{c0}aaa/ + 0: a\x{c0}a\x{c0}aaa + 1: a\x{c0}a\x{c0}a + +/^([a\x{c0}]*)a\x{c0}/8 + a\x{c0}aaaa/ + 0: a\x{c0} + 1: + a\x{c0}a\x{c0}aaa/ + 0: a\x{c0}a\x{c0} + 1: a\x{c0} + +/-- End of testinput4 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput5 b/ext/pcre/pcrelib/testdata/testoutput5 index 75630b4500..f5de747ba0 100644 --- a/ext/pcre/pcrelib/testdata/testoutput5 +++ b/ext/pcre/pcrelib/testdata/testoutput5 @@ -1,3 +1,6 @@ +/-- This set of tests checks the API, internals, and non-Perl stuff for UTF-8 + support, excluding Unicode properties. --/ + /\x{100}/8DZ ------------------------------------------------------------------ Bra @@ -252,7 +255,6 @@ Need char = 171 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char Need char = 'X' @@ -269,52 +271,12 @@ Need char = 'X' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char \x{212ab}\x{212ab}\x{212ab}\x{861} 0: \x{212ab}\x{212ab}\x{212ab} -/-- These tests are here rather than in testinput4 because Perl 5.6 has some -problems with UTF-8 support, in the area of \x{..} where the value is < 255. -It grumbles about invalid UTF-8 strings. --/ - -/^[a\x{c0}]b/8 - \x{c0}b - 0: \x{c0}b - -/^([a\x{c0}]*?)aa/8 - a\x{c0}aaaa/ - 0: a\x{c0}aa - 1: a\x{c0} - -/^([a\x{c0}]*?)aa/8 - a\x{c0}aaaa/ - 0: a\x{c0}aa - 1: a\x{c0} - a\x{c0}a\x{c0}aaa/ - 0: a\x{c0}a\x{c0}aa - 1: a\x{c0}a\x{c0} - -/^([a\x{c0}]*)aa/8 - a\x{c0}aaaa/ - 0: a\x{c0}aaaa - 1: a\x{c0}aa - a\x{c0}a\x{c0}aaa/ - 0: a\x{c0}a\x{c0}aaa - 1: a\x{c0}a\x{c0}a - -/^([a\x{c0}]*)a\x{c0}/8 - a\x{c0}aaaa/ - 0: a\x{c0} - 1: - a\x{c0}a\x{c0}aaa/ - 0: a\x{c0}a\x{c0} - 1: a\x{c0} - -/-- --/ - /(?<=\C)X/8 Failed: \C not allowed in lookbehind assertion at offset 6 @@ -389,6 +351,7 @@ Capturing subpattern count = 0 Options: utf8 No first char No need char +Subject length lower bound = 1 Starting byte set: \x00 \x01 \x02 \x03 \x04 \x05 \x06 \x07 \x08 \x09 \x0a \x0b \x0c \x0d \x0e \x0f \x10 \x11 \x12 \x13 \x14 \x15 \x16 \x17 \x18 \x19 \x1a \x1b \x1c \x1d \x1e \x1f \x20 ! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 @@ -423,11 +386,11 @@ No match End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 196 Need char = 128 -Study returned NULL +Subject length lower bound = 3 +No set of starting bytes \x{100}\x{100}\x{100}\x{100\x{100} 0: \x{100}\x{100}\x{100} @@ -443,10 +406,10 @@ Study returned NULL End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: utf8 No first char No need char +Subject length lower bound = 1 Starting byte set: x \xc4 /(\x{100}*a|x)/8SDZ @@ -462,10 +425,10 @@ Starting byte set: x \xc4 End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: utf8 No first char No need char +Subject length lower bound = 1 Starting byte set: a x \xc4 /(\x{100}{0,2}a|x)/8SDZ @@ -481,10 +444,10 @@ Starting byte set: a x \xc4 End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: utf8 No first char No need char +Subject length lower bound = 1 Starting byte set: a x \xc4 /(\x{100}{1,2}a|x)/8SDZ @@ -501,10 +464,10 @@ Starting byte set: a x \xc4 End ------------------------------------------------------------------ Capturing subpattern count = 1 -Partial matching not supported Options: utf8 No first char No need char +Subject length lower bound = 1 Starting byte set: x \xc4 /\x{100}*(\d+|"(?1)")/8 @@ -551,7 +514,6 @@ Need char = 128 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -565,7 +527,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 'a' No need char @@ -579,7 +540,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 'a' Need char = 'b' @@ -593,7 +553,6 @@ Need char = 'b' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 'a' Need char = 128 @@ -607,7 +566,6 @@ Need char = 128 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 'a' Need char = 129 @@ -621,7 +579,6 @@ Need char = 129 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char Need char = 'A' @@ -640,7 +597,6 @@ Need char = 'A' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1122,7 +1078,6 @@ Need char = 191 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1136,7 +1091,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1150,7 +1104,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1164,7 +1117,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1178,7 +1130,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1192,7 +1143,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 No first char No need char @@ -1206,7 +1156,6 @@ No need char End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 196 Need char = 128 @@ -1220,7 +1169,6 @@ Need char = 128 End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 196 Need char = 'X' @@ -1234,7 +1182,6 @@ Need char = 'X' End ------------------------------------------------------------------ Capturing subpattern count = 0 -Partial matching not supported Options: utf8 First char = 'X' Need char = 128 @@ -1652,4 +1599,477 @@ Forced newline sequence: CRLF First char = 'a' Need char = 'b' -/ End of testinput5 / +/Xa{2,4}b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/Xa{2,4}?b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/Xa{2,4}+b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\x{123}{2,4}b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X\x{123}{2,4}?b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X\x{123}{2,4}+b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X\x{123}{2,4}b/8 + Xx\P +No match + X\x{123}x\P +No match + X\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}\x{123}x\P +No match + +/X\x{123}{2,4}?b/8 + Xx\P +No match + X\x{123}x\P +No match + X\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}\x{123}x\P +No match + +/X\x{123}{2,4}+b/8 + Xx\P +No match + X\x{123}x\P +No match + X\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}x\P +No match + X\x{123}\x{123}\x{123}\x{123}x\P +No match + +/X\d{2,4}b/8 + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\d{2,4}?b/8 + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\d{2,4}+b/8 + X\P +Partial match: X + X3\P +Partial match: X3 + X33\P +Partial match: X33 + X333\P +Partial match: X333 + X3333\P +Partial match: X3333 + +/X\D{2,4}b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\D{2,4}?b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\D{2,4}+b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X\D{2,4}b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X\D{2,4}?b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X\D{2,4}+b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[abc]{2,4}b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc]{2,4}?b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc]{2,4}+b/8 + X\P +Partial match: X + Xa\P +Partial match: Xa + Xaa\P +Partial match: Xaa + Xaaa\P +Partial match: Xaaa + Xaaaa\P +Partial match: Xaaaa + +/X[abc\x{123}]{2,4}b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[abc\x{123}]{2,4}?b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[abc\x{123}]{2,4}+b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[^a]{2,4}b/8 + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/X[^a]{2,4}?b/8 + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/X[^a]{2,4}+b/8 + X\P +Partial match: X + Xz\P +Partial match: Xz + Xzz\P +Partial match: Xzz + Xzzz\P +Partial match: Xzzz + Xzzzz\P +Partial match: Xzzzz + +/X[^a]{2,4}b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[^a]{2,4}?b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/X[^a]{2,4}+b/8 + X\P +Partial match: X + X\x{123}\P +Partial match: X\x{123} + X\x{123}\x{123}\P +Partial match: X\x{123}\x{123} + X\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123} + X\x{123}\x{123}\x{123}\x{123}\P +Partial match: X\x{123}\x{123}\x{123}\x{123} + +/(Y)X\1{2,4}b/8 + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/(Y)X\1{2,4}?b/8 + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/(Y)X\1{2,4}+b/8 + YX\P +Partial match: YX + YXY\P +Partial match: YXY + YXYY\P +Partial match: YXYY + YXYYY\P +Partial match: YXYYY + YXYYYY\P +Partial match: YXYYYY + +/(\x{123})X\1{2,4}b/8 + \x{123}X\P +Partial match: \x{123}X + \x{123}X\x{123}\P +Partial match: \x{123}X\x{123} + \x{123}X\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123}\x{123} + +/(\x{123})X\1{2,4}?b/8 + \x{123}X\P +Partial match: \x{123}X + \x{123}X\x{123}\P +Partial match: \x{123}X\x{123} + \x{123}X\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123}\x{123} + +/(\x{123})X\1{2,4}+b/8 + \x{123}X\P +Partial match: \x{123}X + \x{123}X\x{123}\P +Partial match: \x{123}X\x{123} + \x{123}X\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123} + \x{123}X\x{123}\x{123}\x{123}\x{123}\P +Partial match: \x{123}X\x{123}\x{123}\x{123}\x{123} + +/\bthe cat\b/8 + the cat\P + 0: the cat + the cat\P\P +Partial match: the cat + +/abcd*/8 + xxxxabcd\P + 0: abcd + xxxxabcd\P\P +Partial match: abcd + +/abcd*/i8 + xxxxabcd\P + 0: abcd + xxxxabcd\P\P +Partial match: abcd + XXXXABCD\P + 0: ABCD + XXXXABCD\P\P +Partial match: ABCD + +/abc\d*/8 + xxxxabc1\P + 0: abc1 + xxxxabc1\P\P +Partial match: abc1 + +/(a)bc\1*/8 + xxxxabca\P + 0: abca + 1: a + xxxxabca\P\P +Partial match: abca + +/abc[de]*/8 + xxxxabcde\P + 0: abcde + xxxxabcde\P\P +Partial match: abcde + +/-- End of testinput5 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput6 b/ext/pcre/pcrelib/testdata/testoutput6 index cb6f7d55ca..4245d6a373 100644 --- a/ext/pcre/pcrelib/testdata/testoutput6 +++ b/ext/pcre/pcrelib/testdata/testoutput6 @@ -1,3 +1,7 @@ +/-- This set of tests is for Unicode property support. It is compatible with + Perl 5.10, but not 5.8 because it tests some extra properties that are + not in the earlier release. --/ + /^\pC\pL\pM\pN\pP\pS\pZ</8 \x7f\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< 0: \x{7f}\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< @@ -98,14 +102,6 @@ No match \x{09f} No match -/^\p{Cs}/8 - \?\x{dfff} - 0: \x{dfff} - ** Failers -No match - \x{09f} -No match - /^\p{Ll}/8 a 0: a @@ -338,18 +334,6 @@ No match \x{f3b} No match -/^\p{Sc}+/8 - $\x{a2}\x{a3}\x{a4}\x{a5}\x{a6} - 0: $\x{a2}\x{a3}\x{a4}\x{a5} - \x{9f2} - 0: \x{9f2} - ** Failers -No match - X -No match - \x{2c2} -No match - /^\p{Sk}/8 \x{2c2} 0: \x{2c2} @@ -402,26 +386,6 @@ No match \x{2028} No match -/^\p{Zs}/8 - \ \ - 0: - \x{a0} - 0: \x{a0} - \x{1680} - 0: \x{1680} - \x{180e} - 0: \x{180e} - \x{2000} - 0: \x{2000} - \x{2001} - 0: \x{2001} - ** Failers -No match - \x{2028} -No match - \x{200d} -No match - /\p{Nd}+(..)/8 \x{660}\x{661}\x{662}ABC 0: \x{660}\x{661}\x{662}AB @@ -494,34 +458,6 @@ No match \x{660}\x{661}\x{662}ABC No match -/\p{Lu}/8i - A - 0: A - a\x{10a0}B - 0: \x{10a0} - ** Failers - 0: F - a -No match - \x{1d00} -No match - -/\p{^Lu}/8i - 1234 - 0: 1 - ** Failers - 0: * - ABC -No match - -/\P{Lu}/8i - 1234 - 0: 1 - ** Failers - 0: * - ABC -No match - /(?<=A\p{Nd})XYZ/8 A2XYZ 0: XYZ @@ -548,103 +484,6 @@ No match WXYZ No match -/[\p{L}]/DZ ------------------------------------------------------------------- - Bra - [\p{L}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -No options -No first char -No need char - -/[\p{^L}]/DZ ------------------------------------------------------------------- - Bra - [\P{L}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -No options -No first char -No need char - -/[\P{L}]/DZ ------------------------------------------------------------------- - Bra - [\P{L}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -No options -No first char -No need char - -/[\P{^L}]/DZ ------------------------------------------------------------------- - Bra - [\p{L}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -No options -No first char -No need char - -/[abc\p{L}\x{0660}]/8DZ ------------------------------------------------------------------- - Bra - [a-c\p{L}\x{660}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: utf8 -No first char -No need char - -/[\p{Nd}]/8DZ ------------------------------------------------------------------- - Bra - [\p{Nd}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: utf8 -No first char -No need char - 1234 - 0: 1 - -/[\p{Nd}+-]+/8DZ ------------------------------------------------------------------- - Bra - [+\-\p{Nd}]+ - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Partial matching not supported -Options: utf8 -No first char -No need char - 1234 - 0: 1234 - 12-34 - 0: 12-34 - 12+\x{661}-34 - 0: 12+\x{661}-34 - ** Failers -No match - abcd -No match - /[\P{Nd}]+/8 abcd 0: abcd @@ -725,28 +564,6 @@ No match ABC No match -/\p{Ll}/8i - a - 0: a - Az - 0: z - ** Failers - 0: a - ABC -No match - -/^\x{c0}$/8i - \x{c0} - 0: \x{c0} - \x{e0} - 0: \x{e0} - -/^\x{e0}$/8i - \x{c0} - 0: \x{c0} - \x{e0} - 0: \x{e0} - /A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8 A\x{391}\x{10427}\x{ff3a}\x{1fb0} 0: A\x{391}\x{10427}\x{ff3a}\x{1fb0} @@ -777,54 +594,6 @@ No match A\x{391}\x{10427}\x{ff3a}\x{1fb8} 0: A\x{391}\x{10427}\x{ff3a}\x{1fb8} -/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8iDZ ------------------------------------------------------------------- - Bra - NC A\x{391}\x{10427}\x{ff3a}\x{1fb0} - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: caseless utf8 -First char = 'A' (caseless) -No need char - -/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8DZ ------------------------------------------------------------------- - Bra - A\x{391}\x{10427}\x{ff3a}\x{1fb0} - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: utf8 -First char = 'A' -Need char = 176 - -/AB\x{1fb0}/8DZ ------------------------------------------------------------------- - Bra - AB\x{1fb0} - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: utf8 -First char = 'A' -Need char = 176 - -/AB\x{1fb0}/8DZi ------------------------------------------------------------------- - Bra - NC AB\x{1fb0} - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: caseless utf8 -First char = 'A' (caseless) -Need char = 'B' (caseless) - /\x{391}+/8i \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} 0: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} @@ -849,86 +618,6 @@ Need char = 'B' (caseless) \x{ff5a} 0: \x{ff5a} -/[\x{c0}\x{391}]/8i - \x{c0} - 0: \x{c0} - \x{e0} - 0: \x{e0} - -/[\x{105}-\x{109}]/8iDZ ------------------------------------------------------------------- - Bra - [\x{104}-\x{109}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: caseless utf8 -No first char -No need char - \x{104} - 0: \x{104} - \x{105} - 0: \x{105} - \x{109} - 0: \x{109} - ** Failers -No match - \x{100} -No match - \x{10a} -No match - -/[z-\x{100}]/8iDZ ------------------------------------------------------------------- - Bra - [Z\x{39c}\x{178}z-\x{101}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: caseless utf8 -No first char -No need char - Z - 0: Z - z - 0: z - \x{39c} - 0: \x{39c} - \x{178} - 0: \x{178} - | - 0: | - \x{80} - 0: \x{80} - \x{ff} - 0: \x{ff} - \x{100} - 0: \x{100} - \x{101} - 0: \x{101} - ** Failers -No match - \x{102} -No match - Y -No match - y -No match - -/[z-\x{100}]/8DZi ------------------------------------------------------------------- - Bra - [Z\x{39c}\x{178}z-\x{101}] - Ket - End ------------------------------------------------------------------- -Capturing subpattern count = 0 -Options: caseless utf8 -No first char -No need char - /^\X/8 A 0: A @@ -1408,42 +1097,10 @@ No match 1: L=abc 2: abc -/The next two should be Perl-compatible, but it fails to match \x{e0}. PCRE -will match it only with UCP support, because without that it has no notion -of case for anything other than the ASCII letters. / - -/((?i)[\x{c0}])/8 - \x{c0} - 0: \x{c0} - 1: \x{c0} - \x{e0} - 0: \x{e0} - 1: \x{e0} - -/(?i:[\x{c0}])/8 - \x{c0} - 0: \x{c0} - \x{e0} - 0: \x{e0} - /^\p{Balinese}\p{Cuneiform}\p{Nko}\p{Phags_Pa}\p{Phoenician}/8 \x{1b00}\x{12000}\x{7c0}\x{a840}\x{10900} 0: \x{1b00}\x{12000}\x{7c0}\x{a840}\x{10900} -/The next two are special cases where the lengths of the different cases of the -same character differ. The first went wrong with heap frame storage; the 2nd -was broken in all cases./ - -/^\x{023a}+?(\x{0130}+)/8i - \x{023a}\x{2c65}\x{0130} - 0: \x{23a}\x{2c65}\x{130} - 1: \x{130} - -/^\x{023a}+([^X])/8i - \x{023a}\x{2c65}X - 0: \x{23a}\x{2c65} - 1: \x{2c65} - /Check property support in non-UTF-8 mode/ /\p{L}{4}/ @@ -1468,60 +1125,6 @@ No match A\x80 0: A\x80 -/(?:[\PPa*]*){8,}/ - -/[\P{Any}]/BZ ------------------------------------------------------------------- - Bra - [\P{Any}] - Ket - End ------------------------------------------------------------------- - -/[\P{Any}\E]/BZ ------------------------------------------------------------------- - Bra - [\P{Any}] - Ket - End ------------------------------------------------------------------- - -/(\P{Yi}+\277)/ - -/(\P{Yi}+\277)?/ - -/(?<=\P{Yi}{3}A)X/ - -/\p{Yi}+(\P{Yi}+)(?1)/ - -/(\P{Yi}{2}\277)?/ - -/[\P{Yi}A]/ - -/[\P{Yi}\P{Yi}\P{Yi}A]/ - -/[^\P{Yi}A]/ - -/[^\P{Yi}\P{Yi}\P{Yi}A]/ - -/(\P{Yi}*\277)*/ - -/(\P{Yi}*?\277)*/ - -/(\p{Yi}*+\277)*/ - -/(\P{Yi}?\277)*/ - -/(\P{Yi}??\277)*/ - -/(\p{Yi}?+\277)*/ - -/(\P{Yi}{0,3}\277)*/ - -/(\P{Yi}{0,3}?\277)*/ - -/(\p{Yi}{0,3}+\277)*/ - /^[\p{Arabic}]/8 \x{60e} 0: \x{60e} @@ -1634,59 +1237,6 @@ No match \x{104aa} No match -/\p{Zl}{2,3}+/8BZ ------------------------------------------------------------------- - Bra - prop Zl {2} - prop Zl ?+ - Ket - End ------------------------------------------------------------------- - \xe2\x80\xa8\xe2\x80\xa8 - 0: \x{2028}\x{2028} - \x{2028}\x{2028}\x{2028} - 0: \x{2028}\x{2028}\x{2028} - -/\p{Zl}/8BZ ------------------------------------------------------------------- - Bra - prop Zl - Ket - End ------------------------------------------------------------------- - -/\p{Lu}{3}+/8BZ ------------------------------------------------------------------- - Bra - prop Lu {3} - Ket - End ------------------------------------------------------------------- - -/\pL{2}+/8BZ ------------------------------------------------------------------- - Bra - prop L {2} - Ket - End ------------------------------------------------------------------- - -/\p{Cc}{2}+/8BZ ------------------------------------------------------------------- - Bra - prop Cc {2} - Ket - End ------------------------------------------------------------------- - -/\x{c0}+\x{116}+/8i - \x{c0}\x{e0}\x{116}\x{117} - 0: \x{c0}\x{e0}\x{116}\x{117} - -/[\x{c0}\x{116}]+/8i - \x{c0}\x{e0}\x{116}\x{117} - 0: \x{c0}\x{e0}\x{116}\x{117} - /\p{Carian}\p{Cham}\p{Kayah_Li}\p{Lepcha}\p{Lycian}\p{Lydian}\p{Ol_Chiki}\p{Rejang}\p{Saurashtra}\p{Sundanese}\p{Vai}/8 \x{102A4}\x{AA52}\x{A91D}\x{1C46}\x{10283}\x{1092E}\x{1C6B}\x{A93B}\x{A8BF}\x{1BA0}\x{A50A}==== 0: \x{102a4}\x{aa52}\x{a91d}\x{1c46}\x{10283}\x{1092e}\x{1c6b}\x{a93b}\x{a8bf}\x{1ba0}\x{a50a} @@ -1719,20 +1269,6 @@ No match 0: aA 1: a -/(\x{de})\1/8i - \x{de}\x{de} - 0: \x{de}\x{de} - 1: \x{de} - \x{de}\x{fe} - 0: \x{de}\x{fe} - 1: \x{de} - \x{fe}\x{fe} - 0: \x{fe}\x{fe} - 1: \x{fe} - \x{fe}\x{de} - 0: \x{fe}\x{de} - 1: \x{fe} - /(\x{10a})\1/8i \x{10a}\x{10a} 0: \x{10a}\x{10a} @@ -1757,4 +1293,4 @@ No match \x41\x20\x50\xC2\x54\xC9\x20\x54\x4F\x44\x41\x59 0: A P\xc2T\xc9 TODAY -/ End of testinput6 / +/-- End of testinput6 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput7 b/ext/pcre/pcrelib/testdata/testoutput7 index 78c892313c..c6c9df46f5 100644 --- a/ext/pcre/pcrelib/testdata/testoutput7 +++ b/ext/pcre/pcrelib/testdata/testoutput7 @@ -1,3 +1,6 @@ +/-- This set of tests check the DFA matching functionality of pcre_dfa_exec(). + The -dfa flag must be used with pcretest when running it. --/ + /abc/ abc 0: abc @@ -981,7 +984,7 @@ Partial match: abc xyzfo\P No match foob\P\>2 -Partial match: b +Partial match: foob foobar...\R\P\>4 0: ar xyzfo\P @@ -7168,7 +7171,6 @@ No match /a\R{2,4}b/I<bsr_anycrlf> Capturing subpattern count = 0 -Partial matching not supported Options: bsr_anycrlf First char = 'a' Need char = 'b' @@ -7187,7 +7189,6 @@ No match /a\R{2,4}b/I<bsr_unicode> Capturing subpattern count = 0 -Partial matching not supported Options: bsr_unicode First char = 'a' Need char = 'b' @@ -7370,4 +7371,217 @@ No match \C-"ab" 0: "ab" -/ End of testinput7 / +/\d+X|9+Y/ + ++++123999\P +Partial match: 123999 + ++++123999Y\P + 0: 999Y + +/Z(*F)/ + Z\P +No match + ZA\P +No match + +/Z(?!)/ + Z\P +No match + ZA\P +No match + +/dog(sbody)?/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/dog(sbody)??/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/dog|dogsbody/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/dogsbody|dog/ + dogs\P + 0: dog + dogs\P\P +Partial match: dogs + +/Z(*F)Q|ZXY/ + Z\P +Partial match: Z + ZA\P +No match + X\P +No match + +/\bthe cat\b/ + the cat\P + 0: the cat + the cat\P\P +Partial match: the cat + +/dog(sbody)?/ + dogs\D\P + 0: dog + body\D\R + 0: body + +/dog(sbody)?/ + dogs\D\P\P +Partial match: dogs + body\D\R + 0: body + +/abc/ + abc\P + 0: abc + abc\P\P + 0: abc + +/abc\K123/ + xyzabc123pqr +Error -16 + +/(?<=abc)123/ + xyzabc123pqr + 0: 123 + xyzabc12\P +Partial match: abc12 + xyzabc12\P\P +Partial match: abc12 + +/\babc\b/ + +++abc+++ + 0: abc + +++ab\P +Partial match: +ab + +++ab\P\P +Partial match: +ab + +/(?=C)/g+ + ABCDECBA + 0: + 0+ CDECBA + 0: + 0+ CBA + +/(abc|def|xyz)/I +Capturing subpattern count = 1 +No options +No first char +No need char + terhjk;abcdaadsfe + 0: abc + the quick xyz brown fox + 0: xyz + \Yterhjk;abcdaadsfe + 0: abc + \Ythe quick xyz brown fox + 0: xyz + ** Failers +No match + thejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd +No match + \Ythejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd +No match + +/(abc|def|xyz)/SI +Capturing subpattern count = 1 +No options +No first char +No need char +Subject length lower bound = 3 +Starting byte set: a d x + terhjk;abcdaadsfe + 0: abc + the quick xyz brown fox + 0: xyz + \Yterhjk;abcdaadsfe + 0: abc + \Ythe quick xyz brown fox + 0: xyz + ** Failers +No match + thejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd +No match + \Ythejk;adlfj aenjl;fda asdfasd ehj;kjxyasiupd +No match + +/abcd*/+ + xxxxabcd\P + 0: abcd + 0+ + 1: abc + xxxxabcd\P\P +Partial match: abcd + dddxxx\R + 0: ddd + 0+ xxx + 1: dd + 2: d + 3: + xxxxabcd\P\P +Partial match: abcd + xxx\R + 0: + 0+ xxx + +/abcd*/i + xxxxabcd\P + 0: abcd + 1: abc + xxxxabcd\P\P +Partial match: abcd + XXXXABCD\P + 0: ABCD + 1: ABC + XXXXABCD\P\P +Partial match: ABCD + +/abc\d*/ + xxxxabc1\P + 0: abc1 + 1: abc + xxxxabc1\P\P +Partial match: abc1 + +/abc[de]*/ + xxxxabcde\P + 0: abcde + 1: abcd + 2: abc + xxxxabcde\P\P +Partial match: abcde + +/(?:(?1)|B)(A(*F)|C)/ + ABCD + 0: BC + CCD + 0: CC + ** Failers +No match + CAD +No match + +/^(?:(?1)|B)(A(*F)|C)/ + CCD + 0: CC + BCD + 0: BC + ** Failers +No match + ABCD +No match + CAD +No match + BAD +No match + +/-- End of testinput7 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput8 b/ext/pcre/pcrelib/testdata/testoutput8 index af10c4ae7d..0cc87d79d0 100644 --- a/ext/pcre/pcrelib/testdata/testoutput8 +++ b/ext/pcre/pcrelib/testdata/testoutput8 @@ -1,8 +1,6 @@ -/-- Do not use the \x{} construct except with patterns that have the --/ -/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ -No match -/-- that option is set. However, the latest Perls recognize them always. --/ -No match +/-- This set of tests checks UTF-8 support with the DFA matching functionality + of pcre_dfa_exec(). The -dfa flag must be used with pcretest when running + it. --/ /\x{100}ab/8 \x{100}ab @@ -1288,4 +1286,38 @@ No match A\x{1ec5}ABCXYZ 0: X -/ End of testinput 8 / +/abcd*/8 + xxxxabcd\P + 0: abcd + 1: abc + xxxxabcd\P\P +Partial match: abcd + +/abcd*/i8 + xxxxabcd\P + 0: abcd + 1: abc + xxxxabcd\P\P +Partial match: abcd + XXXXABCD\P + 0: ABCD + 1: ABC + XXXXABCD\P\P +Partial match: ABCD + +/abc\d*/8 + xxxxabc1\P + 0: abc1 + 1: abc + xxxxabc1\P\P +Partial match: abc1 + +/abc[de]*/8 + xxxxabcde\P + 0: abcde + 1: abcd + 2: abc + xxxxabcde\P\P +Partial match: abcde + +/-- End of testinput8 --/ diff --git a/ext/pcre/pcrelib/testdata/testoutput9 b/ext/pcre/pcrelib/testdata/testoutput9 index 04e3873bb8..851e3f020f 100644 --- a/ext/pcre/pcrelib/testdata/testoutput9 +++ b/ext/pcre/pcrelib/testdata/testoutput9 @@ -1,3 +1,7 @@ +/-- This set of tests check Unicode property support with the DFA matching + functionality of pcre_dfa_exec(). The -dfa flag must be used with pcretest + when running it. --/ + /\pL\P{Nd}/8 AB 0: AB @@ -1670,4 +1674,4 @@ No match \x{1d79}\x{a77d} No match -/ End / +/-- End of testinput9 --/ diff --git a/ext/pcre/upgrade-pcre.php b/ext/pcre/upgrade-pcre.php index 8c2061d44c..338b2079f7 100644 --- a/ext/pcre/upgrade-pcre.php +++ b/ext/pcre/upgrade-pcre.php @@ -84,7 +84,12 @@ recurse('pcrelib'); $dirorig = scandir('pcrelib/testdata'); $k = array_search('CVS', $dirorig); -unset($dirorig[$k]); +if ($k !== false) + unset($dirorig[$k]); + +$k = array_search('.svn', $dirorig); +if ($k !== false) + unset($dirorig[$k]); $dirnew = scandir("$newpcre/testdata"); $diff = array_diff($dirorig, $dirnew); |